2023-11-26T06:58:12.497 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-26T06:58:12.497 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --verbose --archive /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348 --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348/orig.config.yaml 2023-11-26T06:58:12.542 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-26T06:58:12.628 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} email: null first_in_suite: false job_id: '7467348' kernel: client: branch: testing flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 44c2237fe77713214148c1fd622c872f346f1b99 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 44c2237fe77713214148c1fd622c872f346f1b99 kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-mgr-vol-misc sha1: 44c2237fe77713214148c1fd622c872f346f1b99 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1863 sha1: 44c2237fe77713214148c1fd622c872f346f1b99 sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-mgr-vol-misc suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 44c2237fe77713214148c1fd622c872f346f1b99 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFdUem5b4uJ1RP0qDFIJWqlOYtyO8Uys9f1w0JQufAK1nRJT1UvZ7YTajtMvazOYiXkxfaefbDWQOCeDXdf4mw= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRZs30f+Ls5Q5Etnx4X6xuZ6NMQ3Xg3Xl2Q7odrw+rGhijjK+x3/VA3x/9HKNd3t3oMpcyH0ELdOzKCFkCQ6XI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-26_06:51:12 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-26T06:58:12.628 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa; will attempt to use it 2023-11-26T06:58:12.629 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks 2023-11-26T06:58:12.629 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-26T06:58:12.630 INFO:teuthology.task.internal:Checking packages... 2023-11-26T06:58:12.655 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '44c2237fe77713214148c1fd622c872f346f1b99' 2023-11-26T06:58:12.655 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-26T06:58:12.655 INFO:teuthology.packaging:ref: None 2023-11-26T06:58:12.655 INFO:teuthology.packaging:tag: None 2023-11-26T06:58:12.655 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-26T06:58:12.655 INFO:teuthology.packaging:sha1: 44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T06:58:12.655 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-26T06:58:12.800 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484-g44c2237f-1jammy 2023-11-26T06:58:12.801 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-26T06:58:12.808 INFO:teuthology.task.internal:no buildpackages task found 2023-11-26T06:58:12.808 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-26T06:58:12.825 INFO:teuthology.task.internal:Saving configuration 2023-11-26T06:58:12.840 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-26T06:58:12.846 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-26T06:58:12.875 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:36.447848', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKC+1ItWNZJ5YwuAK00cwAJrxBaoi85V7hMAMDXANClR'} 2023-11-26T06:58:12.894 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:36.446314', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDCQB8FobE8FThGejvz2//d4u6r+DrgCO5V/G/D3dITk'} 2023-11-26T06:58:12.895 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-26T06:58:12.901 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-26T06:58:12.901 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-26T06:58:12.901 INFO:teuthology.run_tasks:Running task console_log... 2023-11-26T06:58:12.960 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdd389fa0d0>, signals=[15]) 2023-11-26T06:58:12.961 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-26T06:58:12.967 INFO:teuthology.task.internal:Opening connections... 2023-11-26T06:58:12.967 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T06:58:12.969 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T06:58:13.045 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T06:58:13.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T06:58:13.120 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-26T06:58:13.127 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2023-11-26T06:58:13.132 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2023-11-26T06:58:13.132 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-11-26T06:58:13.179 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-26T06:58:13.179 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Ubuntu" 2023-11-26T06:58:13.179 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="22.04" 2023-11-26T06:58:13.179 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-26T06:58:13.179 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_CODENAME=jammy 2023-11-26T06:58:13.179 INFO:teuthology.orchestra.run.smithi031.stdout:ID=ubuntu 2023-11-26T06:58:13.179 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE=debian 2023-11-26T06:58:13.179 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-26T06:58:13.179 INFO:teuthology.orchestra.run.smithi031.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-26T06:58:13.180 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-26T06:58:13.180 INFO:teuthology.orchestra.run.smithi031.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-26T06:58:13.180 INFO:teuthology.orchestra.run.smithi031.stdout:UBUNTU_CODENAME=jammy 2023-11-26T06:58:13.180 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-11-26T06:58:13.202 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2023-11-26T06:58:13.207 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2023-11-26T06:58:13.208 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2023-11-26T06:58:13.254 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-11-26T06:58:13.254 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Ubuntu" 2023-11-26T06:58:13.254 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="22.04" 2023-11-26T06:58:13.254 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-11-26T06:58:13.254 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_CODENAME=jammy 2023-11-26T06:58:13.255 INFO:teuthology.orchestra.run.smithi097.stdout:ID=ubuntu 2023-11-26T06:58:13.255 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE=debian 2023-11-26T06:58:13.255 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.ubuntu.com/" 2023-11-26T06:58:13.255 INFO:teuthology.orchestra.run.smithi097.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-11-26T06:58:13.255 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-11-26T06:58:13.255 INFO:teuthology.orchestra.run.smithi097.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-11-26T06:58:13.255 INFO:teuthology.orchestra.run.smithi097.stdout:UBUNTU_CODENAME=jammy 2023-11-26T06:58:13.255 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2023-11-26T06:58:13.277 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-26T06:58:13.286 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-26T06:58:13.291 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-26T06:58:13.291 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2023-11-26T06:58:13.294 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2023-11-26T06:58:13.302 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-26T06:58:13.308 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-26T06:58:13.308 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2023-11-26T06:58:13.340 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2023-11-26T06:58:13.362 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-26T06:58:13.419 INFO:teuthology.run_tasks:Running task kernel... 2023-11-26T06:58:13.433 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-26T06:58:13.434 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-11-26T06:58:13.434 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-11-26T06:58:13.434 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-26T06:58:13.435 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-26T06:58:13.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T06:58:13.439 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-11-26T06:58:13.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T06:58:13.443 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-11-26T06:58:13.644 DEBUG:teuthology.task.kernel:sha1 for client.1 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T06:58:13.644 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-26T06:58:13.645 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2023-11-26T06:58:13.650 INFO:teuthology.orchestra.run.smithi097.stdout:5.15.0-56-generic 2023-11-26T06:58:13.650 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T06:58:13.650 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-26T06:58:13.651 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2023-11-26T06:58:13.651 DEBUG:teuthology.orchestra.run.smithi097:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-26T06:58:13.654 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T06:58:13.655 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-26T06:58:13.655 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2023-11-26T06:58:13.660 INFO:teuthology.orchestra.run.smithi031.stdout:5.15.0-56-generic 2023-11-26T06:58:13.660 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T06:58:13.660 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-26T06:58:13.661 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-11-26T06:58:13.661 DEBUG:teuthology.orchestra.run.smithi031:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-11-26T06:58:13.716 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-11-26T06:58:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-11-26T06:58:13.842 INFO:teuthology.orchestra.run.smithi097.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-11-26T06:58:13.842 INFO:teuthology.orchestra.run.smithi097.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-11-26T06:58:13.842 INFO:teuthology.orchestra.run.smithi097.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-11-26T06:58:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:hint: 2023-11-26T06:58:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:hint: git config --global init.defaultBranch 2023-11-26T06:58:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:hint: 2023-11-26T06:58:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-11-26T06:58:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-11-26T06:58:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:hint: 2023-11-26T06:58:13.843 INFO:teuthology.orchestra.run.smithi097.stderr:hint: git branch -m 2023-11-26T06:58:13.843 INFO:teuthology.orchestra.run.smithi097.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-26T06:58:13.845 DEBUG:teuthology.orchestra.run.smithi097:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-26T06:58:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-11-26T06:58:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-11-26T06:58:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-11-26T06:58:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:hint: 2023-11-26T06:58:13.866 INFO:teuthology.orchestra.run.smithi031.stderr:hint: git config --global init.defaultBranch 2023-11-26T06:58:13.867 INFO:teuthology.orchestra.run.smithi031.stderr:hint: 2023-11-26T06:58:13.867 INFO:teuthology.orchestra.run.smithi031.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-11-26T06:58:13.867 INFO:teuthology.orchestra.run.smithi031.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-11-26T06:58:13.867 INFO:teuthology.orchestra.run.smithi031.stderr:hint: 2023-11-26T06:58:13.867 INFO:teuthology.orchestra.run.smithi031.stderr:hint: git branch -m 2023-11-26T06:58:13.867 INFO:teuthology.orchestra.run.smithi031.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-11-26T06:58:13.869 DEBUG:teuthology.orchestra.run.smithi031:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-11-26T06:58:13.910 DEBUG:teuthology.orchestra.run.smithi097:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-26T06:58:13.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-11-26T06:58:13.967 DEBUG:teuthology.orchestra.run.smithi097:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-26T06:58:13.997 DEBUG:teuthology.orchestra.run.smithi031:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-11-26T07:01:53.290 INFO:teuthology.orchestra.run.smithi031.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-26T07:01:53.291 INFO:teuthology.orchestra.run.smithi031.stderr: * [new branch] main -> origin/main 2023-11-26T07:01:53.323 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190312 -> 20190312 2023-11-26T07:01:53.323 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190416 -> 20190416 2023-11-26T07:01:53.323 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190514 -> 20190514 2023-11-26T07:01:53.323 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190618 -> 20190618 2023-11-26T07:01:53.324 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190717 -> 20190717 2023-11-26T07:01:53.324 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190815 -> 20190815 2023-11-26T07:01:53.324 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190923 -> 20190923 2023-11-26T07:01:53.324 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20191022 -> 20191022 2023-11-26T07:01:53.324 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20191215 -> 20191215 2023-11-26T07:01:53.324 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200122 -> 20200122 2023-11-26T07:01:53.327 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200316 -> 20200316 2023-11-26T07:01:53.327 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200421 -> 20200421 2023-11-26T07:01:53.327 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200519 -> 20200519 2023-11-26T07:01:53.327 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200619 -> 20200619 2023-11-26T07:01:53.327 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200721 -> 20200721 2023-11-26T07:01:53.327 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200817 -> 20200817 2023-11-26T07:01:53.327 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200918 -> 20200918 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20201022 -> 20201022 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20201118 -> 20201118 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20201218 -> 20201218 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210208 -> 20210208 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210315 -> 20210315 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210511 -> 20210511 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210716 -> 20210716 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210818 -> 20210818 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210919 -> 20210919 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20211027 -> 20211027 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20211216 -> 20211216 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220209 -> 20220209 2023-11-26T07:01:53.328 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220310 -> 20220310 2023-11-26T07:01:53.329 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220411 -> 20220411 2023-11-26T07:01:53.329 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220509 -> 20220509 2023-11-26T07:01:53.329 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220610 -> 20220610 2023-11-26T07:01:53.329 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220708 -> 20220708 2023-11-26T07:01:53.329 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220815 -> 20220815 2023-11-26T07:01:53.329 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220913 -> 20220913 2023-11-26T07:01:53.330 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20221012 -> 20221012 2023-11-26T07:01:53.330 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20221109 -> 20221109 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20221214 -> 20221214 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230117 -> 20230117 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230210 -> 20230210 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230310 -> 20230310 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230404 -> 20230404 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230515 -> 20230515 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230625 -> 20230625 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230804 -> 20230804 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230919 -> 20230919 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20231030 -> 20231030 2023-11-26T07:01:53.331 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20231111 -> 20231111 2023-11-26T07:01:57.532 INFO:teuthology.orchestra.run.smithi031.stderr:Updating files: 48% (1591/3248) Updating files: 49% (1592/3248) Updating files: 50% (1624/3248) Updating files: 51% (1657/3248) Updating files: 52% (1689/3248) Updating files: 53% (1722/3248) Updating files: 54% (1754/3248) Updating files: 55% (1787/3248) Updating files: 56% (1819/3248) Updating files: 57% (1852/3248) Updating files: 58% (1884/3248) Updating files: 59% (1917/3248) Updating files: 60% (1949/3248) Updating files: 61% (1982/3248) Updating files: 62% (2014/3248) Updating files: 63% (2047/3248) Updating files: 63% (2073/3248) Updating files: 64% (2079/3248) Updating files: 65% (2112/3248) Updating files: 66% (2144/3248) Updating files: 67% (2177/3248) Updating files: 68% (2209/3248) Updating files: 69% (2242/3248) Updating files: 70% (2274/3248) Updating files: 71% (2307/3248) Updating files: 72% (2339/3248) Updating files: 73% (2372/3248) Updating files: 74% (2404/3248) Updating files: 75% (2436/3248) Updating files: 76% (2469/3248) Updating files: 77% (2501/3248) Updating files: 78% (2534/3248) Updating files: 79% (2566/3248) Updating files: 80% (2599/3248) Updating files: 80% (2617/3248) Updating files: 81% (2631/3248) Updating files: 82% (2664/3248) Updating files: 83% (2696/3248) Updating files: 84% (2729/3248) Updating files: 85% (2761/3248) Updating files: 86% (2794/3248) Updating files: 86% (2820/3248) Updating files: 87% (2826/3248) Updating files: 88% (2859/3248) Updating files: 89% (2891/3248) Updating files: 90% (2924/3248) Updating files: 91% (2956/3248) Updating files: 92% (2989/3248) Updating files: 93% (3021/3248) Updating files: 94% (3054/3248) Updating files: 95% (3086/3248) Updating files: 96% (3119/3248) Updating files: 97% (3151/3248) Updating files: 98% (3184/3248) Updating files: 99% (3216/3248) Updating files: 100% (3248/3248) Updating files: 100% (3248/3248), done. 2023-11-26T07:01:57.537 INFO:teuthology.orchestra.run.smithi031.stdout:HEAD is now at 9552083a Merge branch 'robot/pr-0-1700470117' into 'main' 2023-11-26T07:01:57.559 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-26T07:01:57.559 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T07:01:57.817 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ 2023-11-26T07:01:57.818 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ --input-file=- 2023-11-26T07:01:57.820 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi031.front.sepia.ceph.com to complete... 2023-11-26T07:02:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26 07:02:00 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb [38938726/38938726] -> "/tmp/linux-image.deb" [1] 2023-11-26T07:02:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:FINISHED --2023-11-26 07:02:00-- 2023-11-26T07:02:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:Total wall clock time: 2.2s 2023-11-26T07:02:00.073 INFO:teuthology.orchestra.run.smithi031.stderr:Downloaded: 1 files, 37M in 1.9s (19.3 MB/s) 2023-11-26T07:02:00.075 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-26T07:02:00.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo dpkg -i /tmp/linux-image.deb 2023-11-26T07:02:00.323 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package linux-image-6.6.0-g8247bbb5b470. 2023-11-26T07:02:02.307 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-11-26T07:02:02.312 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-26T07:02:02.402 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-26T07:02:04.741 INFO:teuthology.orchestra.run.smithi097.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-11-26T07:02:04.742 INFO:teuthology.orchestra.run.smithi097.stderr: * [new branch] main -> origin/main 2023-11-26T07:02:04.773 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190312 -> 20190312 2023-11-26T07:02:04.773 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190416 -> 20190416 2023-11-26T07:02:04.773 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190514 -> 20190514 2023-11-26T07:02:04.773 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190618 -> 20190618 2023-11-26T07:02:04.774 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190717 -> 20190717 2023-11-26T07:02:04.774 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190815 -> 20190815 2023-11-26T07:02:04.774 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20190923 -> 20190923 2023-11-26T07:02:04.776 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20191022 -> 20191022 2023-11-26T07:02:04.777 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20191215 -> 20191215 2023-11-26T07:02:04.777 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200122 -> 20200122 2023-11-26T07:02:04.777 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200316 -> 20200316 2023-11-26T07:02:04.777 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200421 -> 20200421 2023-11-26T07:02:04.777 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200519 -> 20200519 2023-11-26T07:02:04.777 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200619 -> 20200619 2023-11-26T07:02:04.777 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200721 -> 20200721 2023-11-26T07:02:04.777 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200817 -> 20200817 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20200918 -> 20200918 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20201022 -> 20201022 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20201118 -> 20201118 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20201218 -> 20201218 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210208 -> 20210208 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210315 -> 20210315 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210511 -> 20210511 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210716 -> 20210716 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210818 -> 20210818 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20210919 -> 20210919 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20211027 -> 20211027 2023-11-26T07:02:04.778 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20211216 -> 20211216 2023-11-26T07:02:04.779 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220209 -> 20220209 2023-11-26T07:02:04.779 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220310 -> 20220310 2023-11-26T07:02:04.781 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220411 -> 20220411 2023-11-26T07:02:04.781 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220509 -> 20220509 2023-11-26T07:02:04.781 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220610 -> 20220610 2023-11-26T07:02:04.781 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220708 -> 20220708 2023-11-26T07:02:04.781 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220815 -> 20220815 2023-11-26T07:02:04.781 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20220913 -> 20220913 2023-11-26T07:02:04.781 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20221012 -> 20221012 2023-11-26T07:02:04.781 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20221109 -> 20221109 2023-11-26T07:02:04.781 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20221214 -> 20221214 2023-11-26T07:02:04.781 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230117 -> 20230117 2023-11-26T07:02:04.782 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230210 -> 20230210 2023-11-26T07:02:04.782 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230310 -> 20230310 2023-11-26T07:02:04.782 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230404 -> 20230404 2023-11-26T07:02:04.782 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230515 -> 20230515 2023-11-26T07:02:04.782 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230625 -> 20230625 2023-11-26T07:02:04.782 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230804 -> 20230804 2023-11-26T07:02:04.782 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20230919 -> 20230919 2023-11-26T07:02:04.782 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20231030 -> 20231030 2023-11-26T07:02:04.782 INFO:teuthology.orchestra.run.smithi097.stderr: * [new tag] 20231111 -> 20231111 2023-11-26T07:02:05.797 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-26T07:02:05.901 INFO:teuthology.orchestra.run.smithi031.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:08.756 INFO:teuthology.orchestra.run.smithi097.stderr:Updating files: 51% (1665/3248) Updating files: 52% (1689/3248) Updating files: 53% (1722/3248) Updating files: 54% (1754/3248) Updating files: 55% (1787/3248) Updating files: 56% (1819/3248) Updating files: 57% (1852/3248) Updating files: 58% (1884/3248) Updating files: 59% (1917/3248) Updating files: 60% (1949/3248) Updating files: 61% (1982/3248) Updating files: 62% (2014/3248) Updating files: 63% (2047/3248) Updating files: 64% (2079/3248) Updating files: 65% (2112/3248) Updating files: 65% (2113/3248) Updating files: 66% (2144/3248) Updating files: 67% (2177/3248) Updating files: 68% (2209/3248) Updating files: 69% (2242/3248) Updating files: 70% (2274/3248) Updating files: 71% (2307/3248) Updating files: 72% (2339/3248) Updating files: 73% (2372/3248) Updating files: 74% (2404/3248) Updating files: 75% (2436/3248) Updating files: 76% (2469/3248) Updating files: 77% (2501/3248) Updating files: 78% (2534/3248) Updating files: 79% (2566/3248) Updating files: 80% (2599/3248) Updating files: 81% (2631/3248) Updating files: 81% (2648/3248) Updating files: 82% (2664/3248) Updating files: 83% (2696/3248) Updating files: 84% (2729/3248) Updating files: 85% (2761/3248) Updating files: 86% (2794/3248) Updating files: 87% (2826/3248) Updating files: 88% (2859/3248) Updating files: 89% (2891/3248) Updating files: 90% (2924/3248) Updating files: 91% (2956/3248) Updating files: 92% (2989/3248) Updating files: 93% (3021/3248) Updating files: 94% (3054/3248) Updating files: 95% (3086/3248) Updating files: 96% (3119/3248) Updating files: 97% (3151/3248) Updating files: 98% (3184/3248) Updating files: 99% (3216/3248) Updating files: 100% (3248/3248) Updating files: 100% (3248/3248), done. 2023-11-26T07:02:08.760 INFO:teuthology.orchestra.run.smithi097.stdout:HEAD is now at 9552083a Merge branch 'robot/pr-0-1700470117' into 'main' 2023-11-26T07:02:08.781 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-11-26T07:02:08.781 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-26T07:02:08.928 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ 2023-11-26T07:02:08.928 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/ --input-file=- 2023-11-26T07:02:08.931 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi097.front.sepia.ceph.com to complete... 2023-11-26T07:02:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26 07:02:12 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g8247bbb5b470/linux-image-6.6.0-g8247bbb5b470_6.6.0-g8247bbb5b470-1_amd64.deb [38938726/38938726] -> "/tmp/linux-image.deb" [1] 2023-11-26T07:02:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:FINISHED --2023-11-26 07:02:12-- 2023-11-26T07:02:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:Total wall clock time: 3.3s 2023-11-26T07:02:12.242 INFO:teuthology.orchestra.run.smithi097.stderr:Downloaded: 1 files, 37M in 3.0s (12.3 MB/s) 2023-11-26T07:02:12.245 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-11-26T07:02:12.245 DEBUG:teuthology.orchestra.run.smithi097:> sudo dpkg -i /tmp/linux-image.deb 2023-11-26T07:02:12.514 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package linux-image-6.6.0-g8247bbb5b470. 2023-11-26T07:02:14.536 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-11-26T07:02:14.540 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-11-26T07:02:14.726 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-26T07:02:18.200 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up linux-image-6.6.0-g8247bbb5b470 (6.6.0-g8247bbb5b470-1) ... 2023-11-26T07:02:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:Sourcing file `/etc/default/grub' 2023-11-26T07:02:26.143 INFO:teuthology.orchestra.run.smithi031.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-26T07:02:26.167 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2023-11-26T07:02:26.488 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:26.504 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:26.807 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-26T07:02:26.823 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-26T07:02:26.885 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-26T07:02:26.891 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-26T07:02:27.032 INFO:teuthology.orchestra.run.smithi031.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-26T07:02:27.032 INFO:teuthology.orchestra.run.smithi031.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-26T07:02:27.032 INFO:teuthology.orchestra.run.smithi031.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-26T07:02:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:done 2023-11-26T07:02:27.252 DEBUG:teuthology.orchestra.run.smithi031:> dpkg-deb -c /tmp/linux-image.deb 2023-11-26T07:02:27.300 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./ 2023-11-26T07:02:27.301 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./boot/ 2023-11-26T07:02:27.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 4415617 2023-11-23 03:47 ./boot/System.map-6.6.0-g8247bbb5b470 2023-11-26T07:02:27.313 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 169214 2023-11-23 03:47 ./boot/config-6.6.0-g8247bbb5b470 2023-11-26T07:02:27.314 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11352832 2023-11-23 03:47 ./boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:27.335 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/ 2023-11-26T07:02:27.335 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/ 2023-11-26T07:02:27.336 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postinst.d/ 2023-11-26T07:02:27.336 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postrm.d/ 2023-11-26T07:02:27.336 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/preinst.d/ 2023-11-26T07:02:27.337 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/prerm.d/ 2023-11-26T07:02:27.337 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/ 2023-11-26T07:02:27.337 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/ 2023-11-26T07:02:27.337 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/ 2023-11-26T07:02:27.337 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/ 2023-11-26T07:02:27.338 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/ 2023-11-26T07:02:27.338 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/ 2023-11-26T07:02:27.338 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ 2023-11-26T07:02:27.339 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 490233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-26T07:02:27.341 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-26T07:02:27.342 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-26T07:02:27.342 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-26T07:02:27.342 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 93113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-26T07:02:27.342 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 106729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-26T07:02:27.343 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 152241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-26T07:02:27.343 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-26T07:02:27.343 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-26T07:02:27.343 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 83889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-26T07:02:27.344 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-26T07:02:27.344 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-26T07:02:27.344 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-26T07:02:27.344 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-26T07:02:27.345 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-26T07:02:27.345 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-26T07:02:27.345 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-26T07:02:27.345 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-26T07:02:27.346 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-26T07:02:27.346 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-26T07:02:27.346 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/ 2023-11-26T07:02:27.346 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/ 2023-11-26T07:02:27.347 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/ 2023-11-26T07:02:27.347 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-26T07:02:27.347 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpuid.ko 2023-11-26T07:02:27.347 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/msr.ko 2023-11-26T07:02:27.348 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/ 2023-11-26T07:02:27.348 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 354433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-26T07:02:27.348 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 881625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-26T07:02:27.348 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 2275865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-26T07:02:27.395 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ 2023-11-26T07:02:27.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/af_alg.ko 2023-11-26T07:02:27.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_hash.ko 2023-11-26T07:02:27.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_skcipher.ko 2023-11-26T07:02:27.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-26T07:02:27.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/ 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/ 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-26T07:02:27.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-26T07:02:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-26T07:02:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-26T07:02:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-26T07:02:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-26T07:02:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-26T07:02:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-26T07:02:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-26T07:02:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-26T07:02:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-26T07:02:27.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cryptd.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_simd.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-26T07:02:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-26T07:02:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-26T07:02:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lrw.ko 2023-11-26T07:02:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-26T07:02:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-26T07:02:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-26T07:02:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-26T07:02:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-26T07:02:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-26T07:02:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-26T07:02:27.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-26T07:02:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-26T07:02:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 149193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-26T07:02:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-26T07:02:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-26T07:02:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-26T07:02:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-26T07:02:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-26T07:02:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-26T07:02:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-26T07:02:27.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xts.ko 2023-11-26T07:02:27.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-26T07:02:27.401 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ 2023-11-26T07:02:27.401 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ 2023-11-26T07:02:27.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_extlog.ko 2023-11-26T07:02:27.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-26T07:02:27.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-26T07:02:27.401 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/ 2023-11-26T07:02:27.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/einj.ko 2023-11-26T07:02:27.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-26T07:02:27.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-26T07:02:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-26T07:02:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-26T07:02:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 103025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-26T07:02:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ 2023-11-26T07:02:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-26T07:02:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 88865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci.ko 2023-11-26T07:02:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-26T07:02:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 88289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci.ko 2023-11-26T07:02:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-26T07:02:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-26T07:02:27.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_legacy.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-26T07:02:27.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_radisys.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rz1000.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-26T07:02:27.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 83345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-26T07:02:27.406 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-26T07:02:27.407 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-26T07:02:27.407 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-26T07:02:27.407 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-26T07:02:27.407 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ 2023-11-26T07:02:27.407 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-26T07:02:27.407 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/ 2023-11-26T07:02:27.407 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 124473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-26T07:02:27.407 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/ 2023-11-26T07:02:27.407 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/ 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 113665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/ 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 965585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 173225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/ 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 94297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 115793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/ 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 110385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 98193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-26T07:02:27.408 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 263769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/ 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 98145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/ 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/applicom.ko 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/ 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-26T07:02:27.409 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-26T07:02:27.410 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-26T07:02:27.410 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-26T07:02:27.410 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ 2023-11-26T07:02:27.410 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-26T07:02:27.410 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 148601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-26T07:02:27.410 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-26T07:02:27.410 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 139313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-26T07:02:27.410 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-26T07:02:27.410 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-26T07:02:27.410 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/ 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 84097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/nvram.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/ 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-26T07:02:27.411 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/ 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/ 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-palmas.ko 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-si5351.ko 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-twl6040.ko 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-wm831x.ko 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/ 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-26T07:02:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 243025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/ 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/ 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 438761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/ 2023-11-26T07:02:27.413 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/ 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/ 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 122569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 79953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-26T07:02:27.414 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/ 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-gpio.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max14577.ko 2023-11-26T07:02:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max77693.ko 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max8997.ko 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-palmas.ko 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/ 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/ 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/ 2023-11-26T07:02:27.416 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-26T07:02:27.417 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-26T07:02:27.417 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-arizona.ko 2023-11-26T07:02:27.417 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9052.ko 2023-11-26T07:02:27.417 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9055.ko 2023-11-26T07:02:27.417 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-dln2.ko 2023-11-26T07:02:27.417 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 77849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-26T07:02:27.417 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-generic.ko 2023-11-26T07:02:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-26T07:02:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-26T07:02:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-kempld.ko 2023-11-26T07:02:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-26T07:02:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max7300.ko 2023-11-26T07:02:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max730x.ko 2023-11-26T07:02:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max732x.ko 2023-11-26T07:02:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-26T07:02:27.418 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-vx855.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-26T07:02:27.419 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/ 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-a4tech.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-apple.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-belkin.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cherry.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-chicony.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cp2112.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cypress.ko 2023-11-26T07:02:27.420 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-26T07:02:27.421 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-26T07:02:27.421 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-26T07:02:27.421 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-26T07:02:27.421 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ezkey.ko 2023-11-26T07:02:27.421 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-26T07:02:27.422 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-generic.ko 2023-11-26T07:02:27.422 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-26T07:02:27.422 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-26T07:02:27.422 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-26T07:02:27.422 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-26T07:02:27.422 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-26T07:02:27.422 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-26T07:02:27.422 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-26T07:02:27.422 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kensington.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lg-g15.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 101481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 80993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-magicmouse.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-microsoft.ko 2023-11-26T07:02:27.423 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-monterey.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ntrig.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-26T07:02:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-26T07:02:27.425 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 71073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-26T07:02:27.426 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-26T07:02:27.427 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 143473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-26T07:02:27.427 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-26T07:02:27.427 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-26T07:02:27.427 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-26T07:02:27.427 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 267825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid.ko 2023-11-26T07:02:27.427 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-26T07:02:27.427 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/ 2023-11-26T07:02:27.427 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 125721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-26T07:02:27.427 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 217873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/ 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 64201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 305465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-26T07:02:27.428 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 97977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-26T07:02:27.429 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-26T07:02:27.430 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 99601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc2103.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 94457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-26T07:02:27.431 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hih6130.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 130449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/jc42.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-26T07:02:27.432 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-26T07:02:27.433 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 108841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-26T07:02:27.434 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 119817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-26T07:02:27.435 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 82337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 82577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-26T07:02:27.436 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-26T07:02:27.437 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-26T07:02:27.438 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 71345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 71321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 101473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 119505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/ 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/ 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-26T07:02:27.439 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/ 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 6929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-26T07:02:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-26T07:02:27.441 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-26T07:02:27.442 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/ 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 273609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-26T07:02:27.443 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 879969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 324217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 97985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 282273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 493073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/ 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 419737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/ 2023-11-26T07:02:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 739025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-26T07:02:27.489 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ 2023-11-26T07:02:27.489 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 296625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-26T07:02:27.490 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-26T07:02:27.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 153697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-26T07:02:27.490 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/ 2023-11-26T07:02:27.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 168969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-26T07:02:27.490 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ 2023-11-26T07:02:27.490 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-26T07:02:27.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 263081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-26T07:02:27.490 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ 2023-11-26T07:02:27.490 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 100241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-26T07:02:27.490 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 111425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 128753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 129529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/evbug.ko 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ff-memless.ko 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/ 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-26T07:02:27.491 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/ 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8323.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8333.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-26T07:02:27.492 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt1070.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt2160.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/stowaway.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-26T07:02:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/matrix-keymap.ko 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/ 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 123681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 335929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/psmouse.ko 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-26T07:02:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/ 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 109873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ct82c710.ko 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/parkbd.ko 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/pcips2.ko 2023-11-26T07:02:27.495 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ps2mult.ko 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serport.ko 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/ 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/ 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/ 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-88pm860x.ko 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-adp5520.ko 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-bd2802.ko 2023-11-26T07:02:27.496 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da903x.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da9052.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-gpio.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3533.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm355x.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3642.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp8788.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-max8997.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-mc13783.ko 2023-11-26T07:02:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca9532.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca955x.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca963x.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-tca6507.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm8350.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-26T07:02:27.498 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/ 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/ 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 654969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bufio.ko 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 156009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-26T07:02:27.499 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 106897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-era.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-mirror.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-region-hash.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-26T07:02:27.500 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 109529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-snapshot.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 181865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-zero.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/ 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 199937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-26T07:02:27.501 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 113073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 158681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 395721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/ 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/ 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 178137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 123321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 79041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-26T07:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm800.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm805.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm80x.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona-i2c.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 123721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/bcm590xx.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/dln2.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/janz-cmodio.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/kempld-core.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-core.ko 2023-11-26T07:02:27.503 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lp3943.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/max8907.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/menf21bmc.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-26T07:02:27.504 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/retu-mfd.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 80425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/si476x-core.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps65010.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps6507x.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wm8994.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot.ko 2023-11-26T07:02:27.505 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/ 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/ 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/core.ko 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/ 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ds1682.ko 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/dummy-irq.ko 2023-11-26T07:02:27.506 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/ 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/echo.ko 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/ 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/ 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 152201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hmc6352.ko 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-26T07:02:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ics932s401.ko 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/ 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/ 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 125209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 310113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/phantom.ko 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/ 2023-11-26T07:02:27.508 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/ 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 183609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/ 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 490129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/ 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/ 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 106033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-26T07:02:27.509 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 453737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/ 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 130553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 68849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-26T07:02:27.510 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 99049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/ 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/ 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/ 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/ 2023-11-26T07:02:27.511 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/ 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 87297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/ 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 97337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 104673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-26T07:02:27.512 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 81569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/ 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 176761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1421497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 128569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 132089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 312081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-26T07:02:27.513 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/ 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 323513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/ 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 193497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 347417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1044497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 166193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-26T07:02:27.514 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/ 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 204345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/ 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 141225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-26T07:02:27.515 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/ 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/ 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 281137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/ 2023-11-26T07:02:27.516 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 295577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 584497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 246209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 925825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-26T07:02:27.517 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 302937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/ 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 514641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 93593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 734305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 151769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 86921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/ 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-26T07:02:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 122649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/ 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 727601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 278937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1113681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 95665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-26T07:02:27.519 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/ 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 108649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 73441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/ 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 150905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/ 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 136721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/ 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/ 2023-11-26T07:02:27.520 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 227897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 87081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 672049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/ 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/ 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 83985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 189801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-26T07:02:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/ 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 104241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/ 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1052497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/ 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/ 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/ 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 73409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-26T07:02:27.522 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 68177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/ 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 487025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/ 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 103337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 166097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-26T07:02:27.523 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/ 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/ 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 86529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/ 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 80273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 83633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/ 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/ 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/ 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/defxx.ko 2023-11-26T07:02:27.524 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/ 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 216985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 77345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/ 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 235521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/ 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 83769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/ 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-26T07:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/ 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-26T07:02:27.526 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 82049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 76113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 97857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-26T07:02:27.527 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/ 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/plip.ko 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-26T07:02:27.528 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sb1000.ko 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/ 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 64449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/ 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 106169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-26T07:02:27.529 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 87017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/ 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 144297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/ 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 297329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/ 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 144585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-26T07:02:27.530 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/ 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/ 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 265121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/ 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 133841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 84465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/ 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/ 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-26T07:02:27.531 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pci-stub.ko 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/ 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 145809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia.ko 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 112889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/ 2023-11-26T07:02:27.532 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/ 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/ 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/ 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/ 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/ 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/ 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-26T07:02:27.533 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 127441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-26T07:02:27.534 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/ 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-26T07:02:27.535 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/ 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-26T07:02:27.536 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9030_battery.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9052-battery.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/gpio-charger.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-26T07:02:27.537 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max14577_charger.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17040_battery.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17042_battery.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8903_charger.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8925_power.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/sbs-battery.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/test_power.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_power.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm8350_power.ko 2023-11-26T07:02:27.538 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/ 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/ 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/ 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-26T07:02:27.539 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/ 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9052.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9055.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9063.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 68577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-26T07:02:27.540 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-26T07:02:27.541 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8907.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8925.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8997.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8998.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-palmas.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-26T07:02:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-26T07:02:27.543 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 77617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 135185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/ 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 272305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 100617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/ 2023-11-26T07:02:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 356201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 295801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/ 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 189185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/ 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 106881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/ 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 246985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/ 2023-11-26T07:02:27.545 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1171945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-26T07:02:27.585 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/ 2023-11-26T07:02:27.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 237273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-26T07:02:27.585 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/ 2023-11-26T07:02:27.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 115369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-26T07:02:27.585 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-26T07:02:27.585 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/ 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 308921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/ 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 76145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 101449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 146689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 75553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/ 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 212425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-26T07:02:27.586 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/ 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 158505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fdomain.ko 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/ 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 263489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 201817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/imm.ko 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-26T07:02:27.587 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 266537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 101881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/ 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 317153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/ 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 314865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 145585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/ 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 216665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/ 2023-11-26T07:02:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 2197249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/ 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 73569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 310617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 84721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/ 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 659825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/ 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 119561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/ 2023-11-26T07:02:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 81625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/ 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 364105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 123617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ppa.ko 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/ 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1876689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-26T07:02:27.590 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/ 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 559033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qlogicfas408.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 238761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 173937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 279313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 97729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 73881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 115889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-26T07:02:27.591 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 162121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/xen-scsifront.ko 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 129441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/ 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/ 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 736601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/ 2023-11-26T07:02:27.592 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 930257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 122521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/ 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 75593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/ 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/ 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-26T07:02:27.593 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-26T07:02:27.594 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/moxa.ko 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/mxser.ko 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 108601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/ 2023-11-26T07:02:27.595 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/ 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_uart.ko 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/ 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 103073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/rp2.ko 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-26T07:02:27.596 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/ 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 531265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/ 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_mf624.ko 2023-11-26T07:02:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_netx.ko 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/ 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/ 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 73297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 144377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-26T07:02:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/ 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 87089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ulpi.ko 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/ 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/ 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-26T07:02:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 448505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/ 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/ 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 92489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 122081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-26T07:02:27.600 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/ 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-26T07:02:27.601 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-26T07:02:27.602 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 161225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/ 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 157681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 76057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 75073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 73169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 65809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-26T07:02:27.603 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 81505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 156073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 80113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 87353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811_cs.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-26T07:02:27.604 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/ 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cytherm.ko 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ehset.ko 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-26T07:02:27.605 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/lvstest.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/ 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 64841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbtest.ko 2023-11-26T07:02:27.606 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/ 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/usbmon.ko 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/ 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 301697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/ 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ 2023-11-26T07:02:27.607 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/f81232.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 175137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-26T07:02:27.608 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 137081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 124833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 107521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/metro-usb.ko 2023-11-26T07:02:27.609 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mxuport.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 191137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-26T07:02:27.610 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 125609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usbserial.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-26T07:02:27.611 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-26T07:02:27.612 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 179969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/ 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 107001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/ 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/ 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 154297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-26T07:02:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 108737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 77609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/ 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 113785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/ 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/ 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-26T07:02:27.614 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/bd6107.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-26T07:02:27.615 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/ 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-26T07:02:27.616 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/ 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-26T07:02:27.617 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/acquirewdt.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/advantechwdt.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/dw_wdt.ko 2023-11-26T07:02:27.618 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-26T07:02:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-26T07:02:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-26T07:02:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-26T07:02:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-26T07:02:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-26T07:02:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-26T07:02:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-26T07:02:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-26T07:02:27.619 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-26T07:02:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-26T07:02:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-26T07:02:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-26T07:02:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-26T07:02:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-26T07:02:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-26T07:02:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-26T07:02:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-26T07:02:27.620 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/retu_wdt.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-26T07:02:27.621 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/ 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-26T07:02:27.622 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/ 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 176377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 71249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-scsiback.ko 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/ 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/ 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 166425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/ 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/adfs.ko 2023-11-26T07:02:27.623 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/ 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 163593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/ 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 932817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/kafs.ko 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/ 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 101057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/autofs4.ko 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/ 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 64545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/ 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/bfs.ko 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-26T07:02:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/ 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 3402729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/ 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 306009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1333017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/ 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 100801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/ 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 108393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/configfs.ko 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/ 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-26T07:02:27.625 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/ 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 642513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/ 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/efs.ko 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/ 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1505025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/ 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/ 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/freevxfs.ko 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/ 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 226481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-26T07:02:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/ 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/ 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1231529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/ 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 141209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/ 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 222817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/ 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 192329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/hpfs.ko 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/ 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-26T07:02:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/ 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 419497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/ 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 280313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/ 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/ 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 124649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/ 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/ 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 81449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/ 2023-11-26T07:02:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/ 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 117465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1055721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv2.ko 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 120177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 2042137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/ 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/ 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1516873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/ 2023-11-26T07:02:27.629 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 592833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/ 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-26T07:02:27.630 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ascii.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-26T07:02:27.631 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 94137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 142673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 165745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 118121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-26T07:02:27.632 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-26T07:02:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 219417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ntfs.ko 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 244409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 440737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-26T07:02:27.634 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ 2023-11-26T07:02:27.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-26T07:02:27.635 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 2592433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-26T07:02:27.668 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/ 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/omfs.ko 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/ 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 352161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/ 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/qnx4.ko 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/ 2023-11-26T07:02:27.669 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/qnx6.ko 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/ 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_tree.ko 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v1.ko 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v2.ko 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/ 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 532673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/ 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/ 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/ 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 2668401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-26T07:02:27.670 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/ 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/ 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 134641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/ 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/ 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 247737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 162497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/ 2023-11-26T07:02:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 4011017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/ 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/ 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 75489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/locktorture.ko 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/ 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/test_udelay.ko 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 6833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 6529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc7.ko 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-26T07:02:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/ 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/interval_tree_test.ko 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/ 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/ 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 6617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/memory-notifier-error-inject.ko 2023-11-26T07:02:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/notifier-error-inject.ko 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/percpu_test.ko 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/pm-notifier-error-inject.ko 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/ 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 143433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/rbtree_test.ko 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/ 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-kstrtox.ko 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-string_helpers.ko 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 4760425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_bpf.ko 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_firmware.ko 2023-11-26T07:02:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 6369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_module.ko 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_user_copy.ko 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/ 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/xz_dec_test.ko 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/ 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/ 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-26T07:02:27.675 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 6985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/ 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-26T07:02:27.676 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/ 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 81737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/ 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 198337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/ 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 649297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/ 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 777689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-26T07:02:27.677 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-26T07:02:27.678 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/ 2023-11-26T07:02:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1046833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/ 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 125993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/ 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 194209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/ 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 390473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-26T07:02:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/ 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 99729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/hsr.ko 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 254625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-26T07:02:27.681 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-26T07:02:27.682 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-26T07:02:27.683 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-26T07:02:27.684 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-26T07:02:27.685 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 80897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_gre.ko 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 93577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-26T07:02:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/mip6.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-26T07:02:27.687 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-26T07:02:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/ 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 82833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/ 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 125281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-26T07:02:27.689 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/ 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/lapb.ko 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/ 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 267913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc2.ko 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/ 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 290417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/ 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-26T07:02:27.690 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 119585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 100297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 101441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 103417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 112849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-26T07:02:27.691 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 109593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 113129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 112585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 111585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 114465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 411729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-26T07:02:27.692 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 408257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-26T07:02:27.693 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 118145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 107233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-26T07:02:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 95353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 615649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-26T07:02:27.695 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-26T07:02:27.696 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_osf.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_synproxy.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tproxy.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/x_tables.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-26T07:02:27.697 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-26T07:02:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-26T07:02:27.699 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-26T07:02:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-26T07:02:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-26T07:02:27.702 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpudp.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/ 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/ 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/ 2023-11-26T07:02:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 380033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/ 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/ 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 110201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/phonet.ko 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/pn_pep.ko 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/ 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 251481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 164937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-26T07:02:27.704 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/ 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 819657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/rxrpc.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/ 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-26T07:02:27.705 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_cgroup.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-26T07:02:27.706 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq_codel.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-26T07:02:27.707 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-26T07:02:27.708 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/ 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 929745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/ 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/ 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 336529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1520097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/ 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 707657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/ 2023-11-26T07:02:27.709 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 862665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/ 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/ 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 68049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 96193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/ 2023-11-26T07:02:27.710 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 125177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/x25.ko 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/ 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_algo.ko 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_user.ko 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/ 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/ 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 640081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 624107 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7663 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23428 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9821 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-26T07:02:27.711 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70004 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 146313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 232548 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 210 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72606 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 879 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 254438 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 309363 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/ 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/ 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/ 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/ 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 172 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/changelog.Debian.gz 2023-11-26T07:02:27.712 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 691 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/copyright 2023-11-26T07:02:27.713 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-26T07:02:27.713 INFO:teuthology.task.kernel:searching for kernel 6.6.0-g8247bbb5b470 2023-11-26T07:02:27.714 DEBUG:teuthology.orchestra.run.smithi031:> egrep '(submenu|menuentry.*6.6.0-g8247bbb5b470).*{' /boot/grub/grub.cfg 2023-11-26T07:02:27.720 INFO:teuthology.orchestra.run.smithi031.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:27.720 INFO:teuthology.orchestra.run.smithi031.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:27.720 INFO:teuthology.orchestra.run.smithi031.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:27.721 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-26T07:02:27.721 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-g8247bbb5b470 2023-11-26T07:02:27.721 DEBUG:teuthology.orchestra.run.smithi031:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-26T07:02:27.762 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi031.front.sepia.ceph.com to complete... 2023-11-26T07:02:28.170 INFO:teuthology.orchestra.run.smithi031.stderr:Sourcing file `/etc/default/grub' 2023-11-26T07:02:28.172 INFO:teuthology.orchestra.run.smithi031.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-26T07:02:28.192 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2023-11-26T07:02:28.417 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:28.423 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:28.663 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-26T07:02:28.669 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-26T07:02:28.711 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-26T07:02:28.717 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-26T07:02:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-26T07:02:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-26T07:02:28.885 INFO:teuthology.orchestra.run.smithi031.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-26T07:02:28.905 INFO:teuthology.orchestra.run.smithi031.stderr:done 2023-11-26T07:02:39.547 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub' 2023-11-26T07:02:39.570 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-26T07:02:39.598 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2023-11-26T07:02:39.962 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:39.982 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:40.299 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-26T07:02:40.312 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-26T07:02:40.372 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-26T07:02:40.377 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-26T07:02:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-26T07:02:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-26T07:02:40.557 INFO:teuthology.orchestra.run.smithi097.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-26T07:02:40.592 INFO:teuthology.orchestra.run.smithi097.stderr:done 2023-11-26T07:02:40.816 DEBUG:teuthology.orchestra.run.smithi097:> dpkg-deb -c /tmp/linux-image.deb 2023-11-26T07:02:40.863 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./ 2023-11-26T07:02:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./boot/ 2023-11-26T07:02:40.864 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 4415617 2023-11-23 03:47 ./boot/System.map-6.6.0-g8247bbb5b470 2023-11-26T07:02:40.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 169214 2023-11-23 03:47 ./boot/config-6.6.0-g8247bbb5b470 2023-11-26T07:02:40.876 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11352832 2023-11-23 03:47 ./boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:40.896 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/ 2023-11-26T07:02:40.896 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/ 2023-11-26T07:02:40.897 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postinst.d/ 2023-11-26T07:02:40.897 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/postrm.d/ 2023-11-26T07:02:40.897 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/preinst.d/ 2023-11-26T07:02:40.897 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./etc/kernel/prerm.d/ 2023-11-26T07:02:40.897 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/ 2023-11-26T07:02:40.898 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/ 2023-11-26T07:02:40.898 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/ 2023-11-26T07:02:40.899 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/ 2023-11-26T07:02:40.899 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/ 2023-11-26T07:02:40.899 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/ 2023-11-26T07:02:40.899 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ 2023-11-26T07:02:40.899 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 490233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/aesni-intel.ko 2023-11-26T07:02:40.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-26T07:02:40.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-26T07:02:40.905 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-26T07:02:40.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 93113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-26T07:02:40.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 106729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-26T07:02:40.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 152241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-26T07:02:40.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-26T07:02:40.906 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-26T07:02:40.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 83889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-26T07:02:40.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-26T07:02:40.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-26T07:02:40.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-26T07:02:40.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-26T07:02:40.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-26T07:02:40.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-26T07:02:40.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-26T07:02:40.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-26T07:02:40.907 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-26T07:02:40.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-26T07:02:40.908 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/ 2023-11-26T07:02:40.908 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/ 2023-11-26T07:02:40.908 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/ 2023-11-26T07:02:40.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-26T07:02:40.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpuid.ko 2023-11-26T07:02:40.908 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/msr.ko 2023-11-26T07:02:40.909 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/ 2023-11-26T07:02:40.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 354433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-26T07:02:40.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 881625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-26T07:02:40.909 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 2275865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-26T07:02:40.983 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ 2023-11-26T07:02:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/af_alg.ko 2023-11-26T07:02:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_hash.ko 2023-11-26T07:02:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/algif_skcipher.ko 2023-11-26T07:02:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-26T07:02:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-26T07:02:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-26T07:02:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/ 2023-11-26T07:02:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-11-26T07:02:40.984 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/ 2023-11-26T07:02:40.985 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-26T07:02:40.985 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-26T07:02:40.985 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-26T07:02:40.985 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-26T07:02:40.985 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-26T07:02:40.985 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-26T07:02:40.985 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-26T07:02:40.986 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-26T07:02:40.986 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-26T07:02:40.986 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-26T07:02:40.986 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-26T07:02:40.986 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-26T07:02:40.986 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-26T07:02:40.986 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-26T07:02:40.986 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-26T07:02:40.987 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-26T07:02:40.987 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-26T07:02:40.987 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-26T07:02:40.987 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cryptd.ko 2023-11-26T07:02:40.987 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-26T07:02:40.987 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_simd.ko 2023-11-26T07:02:40.987 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-26T07:02:40.987 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-26T07:02:40.987 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-26T07:02:40.987 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-26T07:02:40.988 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-26T07:02:40.988 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-26T07:02:40.988 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-26T07:02:40.988 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-26T07:02:40.988 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lrw.ko 2023-11-26T07:02:40.988 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-26T07:02:40.988 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-26T07:02:40.988 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-26T07:02:40.988 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-26T07:02:40.988 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-26T07:02:40.989 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-26T07:02:40.989 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-26T07:02:40.989 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-26T07:02:40.989 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-26T07:02:40.989 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 149193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-26T07:02:40.989 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-26T07:02:40.989 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-26T07:02:40.989 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-26T07:02:40.990 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-26T07:02:40.990 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-26T07:02:40.990 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-26T07:02:40.990 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-26T07:02:40.990 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xts.ko 2023-11-26T07:02:40.990 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-26T07:02:40.990 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ 2023-11-26T07:02:40.990 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ 2023-11-26T07:02:40.990 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_extlog.ko 2023-11-26T07:02:40.990 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-26T07:02:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-26T07:02:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/ 2023-11-26T07:02:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/apei/einj.ko 2023-11-26T07:02:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-26T07:02:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-26T07:02:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-26T07:02:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-26T07:02:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 103025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-26T07:02:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ 2023-11-26T07:02:40.991 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-26T07:02:40.992 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 88865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci.ko 2023-11-26T07:02:40.992 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-26T07:02:40.992 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 88289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci.ko 2023-11-26T07:02:40.992 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-26T07:02:40.992 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-26T07:02:40.992 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-26T07:02:40.992 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-26T07:02:40.992 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-26T07:02:40.992 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-26T07:02:40.992 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-26T07:02:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-26T07:02:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-26T07:02:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-26T07:02:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-26T07:02:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-26T07:02:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-26T07:02:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-26T07:02:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-26T07:02:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-26T07:02:40.993 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-26T07:02:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-26T07:02:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_legacy.ko 2023-11-26T07:02:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-26T07:02:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-26T07:02:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-26T07:02:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-26T07:02:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-26T07:02:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-26T07:02:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-26T07:02:40.994 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_radisys.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rz1000.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-26T07:02:40.995 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-26T07:02:40.996 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-26T07:02:40.996 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-26T07:02:40.996 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-26T07:02:40.996 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-26T07:02:40.996 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-26T07:02:40.997 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 83345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-26T07:02:40.997 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-26T07:02:40.997 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-26T07:02:40.997 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-26T07:02:40.997 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-26T07:02:40.997 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-26T07:02:40.998 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-26T07:02:40.998 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-26T07:02:40.998 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-26T07:02:40.998 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-26T07:02:40.998 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-26T07:02:40.998 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-26T07:02:40.999 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ 2023-11-26T07:02:40.999 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-26T07:02:40.999 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/ 2023-11-26T07:02:40.999 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 124473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-26T07:02:40.999 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/ 2023-11-26T07:02:40.999 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/ 2023-11-26T07:02:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 113665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-26T07:02:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/ 2023-11-26T07:02:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 965585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-26T07:02:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 173225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-26T07:02:41.000 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/ 2023-11-26T07:02:41.001 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 94297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-26T07:02:41.001 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 115793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-26T07:02:41.001 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/ 2023-11-26T07:02:41.001 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 110385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-26T07:02:41.001 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 98193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-26T07:02:41.001 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 263769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-26T07:02:41.001 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/ 2023-11-26T07:02:41.002 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 98145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-26T07:02:41.002 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/ 2023-11-26T07:02:41.002 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-26T07:02:41.002 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ 2023-11-26T07:02:41.002 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/applicom.ko 2023-11-26T07:02:41.002 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-26T07:02:41.002 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/ 2023-11-26T07:02:41.002 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-26T07:02:41.003 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-26T07:02:41.003 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-26T07:02:41.003 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-26T07:02:41.003 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-26T07:02:41.003 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 148601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 139313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/ 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 84097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/nvram.ko 2023-11-26T07:02:41.004 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/ 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/ 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_class.ko 2023-11-26T07:02:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_core.ko 2023-11-26T07:02:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-11-26T07:02:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/ 2023-11-26T07:02:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-palmas.ko 2023-11-26T07:02:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-si5351.ko 2023-11-26T07:02:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-twl6040.ko 2023-11-26T07:02:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/clk/clk-wm831x.ko 2023-11-26T07:02:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/ 2023-11-26T07:02:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 243025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/ 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/ 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/ 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 438761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-11-26T07:02:41.007 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/ 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/ 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/ 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ 2023-11-26T07:02:41.008 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 122569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-26T07:02:41.009 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ 2023-11-26T07:02:41.009 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 79953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-26T07:02:41.009 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-26T07:02:41.009 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-26T07:02:41.009 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-26T07:02:41.009 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-26T07:02:41.009 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-26T07:02:41.009 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-26T07:02:41.009 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-26T07:02:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-26T07:02:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-26T07:02:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-26T07:02:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-26T07:02:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-26T07:02:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/ 2023-11-26T07:02:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-gpio.ko 2023-11-26T07:02:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max14577.ko 2023-11-26T07:02:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max77693.ko 2023-11-26T07:02:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-max8997.ko 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-palmas.ko 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-rt8973a.ko 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/extcon/extcon-sm5502.ko 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/ 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/dmi-sysfs.ko 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/ 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/efi/efi-pstore.ko 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/ 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-adp5520.ko 2023-11-26T07:02:41.011 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-amd8111.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-arizona.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9052.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-da9055.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-dln2.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 77849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-f7188x.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-generic.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-kempld.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-lp3943.ko 2023-11-26T07:02:41.012 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max7300.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max730x.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-max732x.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pca953x.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-pcf857x.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-rdc321x.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-sch311x.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-tps65912.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl4030.ko 2023-11-26T07:02:41.013 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-twl6040.ko 2023-11-26T07:02:41.014 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-26T07:02:41.014 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-vx855.ko 2023-11-26T07:02:41.014 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm831x.ko 2023-11-26T07:02:41.014 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8350.ko 2023-11-26T07:02:41.014 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-wm8994.ko 2023-11-26T07:02:41.014 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/ 2023-11-26T07:02:41.014 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-a4tech.ko 2023-11-26T07:02:41.015 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-apple.ko 2023-11-26T07:02:41.015 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-26T07:02:41.015 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-26T07:02:41.015 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-26T07:02:41.015 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-belkin.ko 2023-11-26T07:02:41.015 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cherry.ko 2023-11-26T07:02:41.015 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-chicony.ko 2023-11-26T07:02:41.015 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cp2112.ko 2023-11-26T07:02:41.015 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-cypress.ko 2023-11-26T07:02:41.015 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-26T07:02:41.016 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-26T07:02:41.016 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-26T07:02:41.016 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-26T07:02:41.016 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ezkey.ko 2023-11-26T07:02:41.016 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-26T07:02:41.016 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-generic.ko 2023-11-26T07:02:41.016 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-26T07:02:41.016 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-26T07:02:41.016 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kensington.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-26T07:02:41.017 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lg-g15.ko 2023-11-26T07:02:41.018 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-26T07:02:41.018 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 101481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-26T07:02:41.018 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 80993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech.ko 2023-11-26T07:02:41.018 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-magicmouse.ko 2023-11-26T07:02:41.018 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-microsoft.ko 2023-11-26T07:02:41.018 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-monterey.ko 2023-11-26T07:02:41.018 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-26T07:02:41.018 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ntrig.ko 2023-11-26T07:02:41.018 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-26T07:02:41.018 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-26T07:02:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-26T07:02:41.020 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 71073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-26T07:02:41.021 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 143473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 267825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid.ko 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/ 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 125721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbhid.ko 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbkbd.ko 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/usbhid/usbmouse.ko 2023-11-26T07:02:41.022 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 217873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/ 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 64201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 305465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-26T07:02:41.023 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 97977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-26T07:02:41.024 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9052-hwmon.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/da9055-hwmon.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-26T07:02:41.025 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 99601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc2103.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 94457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-26T07:02:41.026 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hih6130.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-26T07:02:41.027 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 130449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/jc42.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-26T07:02:41.028 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 108841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-26T07:02:41.029 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-26T07:02:41.030 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mc13783-adc.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 119817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 82337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-26T07:02:41.031 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 82577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-26T07:02:41.032 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-26T07:02:41.033 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-26T07:02:41.034 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-26T07:02:41.034 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-26T07:02:41.034 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-26T07:02:41.034 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-26T07:02:41.034 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-26T07:02:41.034 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-26T07:02:41.034 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-26T07:02:41.034 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-26T07:02:41.034 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 71345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 71321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 101473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 119505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/ 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/ 2023-11-26T07:02:41.035 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/ 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 6929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-26T07:02:41.036 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-26T07:02:41.037 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-26T07:02:41.038 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/ 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-11-26T07:02:41.039 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 273609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 879969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 324217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 97985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 282273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/ 2023-11-26T07:02:41.040 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 493073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/ 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 419737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/ 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 739025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 296625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 153697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/ 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 168969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ 2023-11-26T07:02:41.041 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 263081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 100241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 111425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 128753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 129529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/evbug.ko 2023-11-26T07:02:41.042 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/ff-memless.ko 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/ 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/ 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-26T07:02:41.043 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lkkbd.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8323.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/lm8333.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/newtonkbd.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt1070.ko 2023-11-26T07:02:41.044 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/qt2160.ko 2023-11-26T07:02:41.045 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-11-26T07:02:41.045 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/stowaway.ko 2023-11-26T07:02:41.045 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/sunkbd.ko 2023-11-26T07:02:41.045 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-11-26T07:02:41.045 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-11-26T07:02:41.045 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-11-26T07:02:41.045 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/xtkbd.ko 2023-11-26T07:02:41.045 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/matrix-keymap.ko 2023-11-26T07:02:41.045 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/ 2023-11-26T07:02:41.045 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 123681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/gpio_mouse.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 335929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/psmouse.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/ 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 109873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-26T07:02:41.046 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ct82c710.ko 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/parkbd.ko 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/pcips2.ko 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/ps2mult.ko 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serport.ko 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-26T07:02:41.047 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/ 2023-11-26T07:02:41.048 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/ 2023-11-26T07:02:41.048 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-26T07:02:41.048 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/ 2023-11-26T07:02:41.048 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-88pm860x.ko 2023-11-26T07:02:41.048 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-adp5520.ko 2023-11-26T07:02:41.048 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-bd2802.ko 2023-11-26T07:02:41.048 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-26T07:02:41.048 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da903x.ko 2023-11-26T07:02:41.048 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-da9052.ko 2023-11-26T07:02:41.048 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-gpio.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3533.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm355x.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3642.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp8788.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-max8997.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-mc13783.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-menf21bmc.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca9532.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca955x.ko 2023-11-26T07:02:41.049 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-pca963x.ko 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-tca6507.ko 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm831x-status.ko 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-wm8350.ko 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-26T07:02:41.050 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/ 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/ 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 654969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bufio.ko 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 156009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 106897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-26T07:02:41.051 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-era.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-mirror.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-region-hash.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-26T07:02:41.052 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 109529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-snapshot.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 181865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-zero.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/ 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 199937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 113073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-26T07:02:41.053 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 158681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 395721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/ 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/ 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 178137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 123321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 79041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-26T07:02:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm800.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm805.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/88pm80x.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona-i2c.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 123721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/arizona.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/bcm590xx.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/cros_ec_dev.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/dln2.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/janz-cmodio.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/kempld-core.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-core.ko 2023-11-26T07:02:41.055 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lp3943.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/max8907.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-core.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/menf21bmc.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-adc.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633-gpio.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/pcf50633.ko 2023-11-26T07:02:41.056 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/retu-mfd.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 80425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/si476x-core.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps65010.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/tps6507x.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wm8994.ko 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ 2023-11-26T07:02:41.057 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ad525x_dpot.ko 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/ 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/ 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/c2port/core.ko 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/ 2023-11-26T07:02:41.058 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ds1682.ko 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/dummy-irq.ko 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/ 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/echo/echo.ko 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/ 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/ 2023-11-26T07:02:41.059 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 152201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hmc6352.ko 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ics932s401.ko 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/ 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-26T07:02:41.060 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/ 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 125209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 310113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/phantom.ko 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/ 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/ti-st/st_drv.ko 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-26T07:02:41.061 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/ 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 183609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/ 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 490129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/ 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/ 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 106033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/ 2023-11-26T07:02:41.062 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 453737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/ 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 130553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 68849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-26T07:02:41.063 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-26T07:02:41.064 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-26T07:02:41.064 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 99049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-26T07:02:41.064 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/ 2023-11-26T07:02:41.064 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-26T07:02:41.064 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/ 2023-11-26T07:02:41.064 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-26T07:02:41.064 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/ 2023-11-26T07:02:41.064 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-26T07:02:41.064 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/ 2023-11-26T07:02:41.064 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/ 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 87297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/ 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/ 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 97337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 104673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-11-26T07:02:41.065 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/ 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 81569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/ 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 176761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1421497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 128569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/ 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 132089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-26T07:02:41.066 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 312081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-26T07:02:41.067 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/ 2023-11-26T07:02:41.067 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/ 2023-11-26T07:02:41.067 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 323513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-26T07:02:41.067 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/ 2023-11-26T07:02:41.067 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-11-26T07:02:41.067 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 193497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-26T07:02:41.067 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-11-26T07:02:41.067 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 347417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-26T07:02:41.199 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-11-26T07:02:41.199 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1044497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-26T07:02:41.200 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-11-26T07:02:41.200 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 166193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-26T07:02:41.200 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-11-26T07:02:41.200 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-26T07:02:41.200 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/ 2023-11-26T07:02:41.200 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/ 2023-11-26T07:02:41.200 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 204345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-26T07:02:41.201 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/ 2023-11-26T07:02:41.201 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/ 2023-11-26T07:02:41.201 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-26T07:02:41.201 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-26T07:02:41.201 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 141225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-26T07:02:41.201 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-26T07:02:41.202 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-26T07:02:41.202 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-26T07:02:41.202 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/ 2023-11-26T07:02:41.202 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-26T07:02:41.202 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-26T07:02:41.202 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-26T07:02:41.202 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ec_bhf.ko 2023-11-26T07:02:41.203 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/ 2023-11-26T07:02:41.203 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/ 2023-11-26T07:02:41.203 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 281137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-26T07:02:41.203 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-26T07:02:41.203 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-26T07:02:41.203 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/ 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/ 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 295577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/ 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 584497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/ 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 246209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/ 2023-11-26T07:02:41.204 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 925825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/ 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 302937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/ 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 514641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/ 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 93593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 734305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 151769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-26T07:02:41.205 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 86921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-26T07:02:41.206 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/ 2023-11-26T07:02:41.206 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-26T07:02:41.206 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-26T07:02:41.206 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 122649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-26T07:02:41.206 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/ 2023-11-26T07:02:41.206 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-11-26T07:02:41.206 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 727601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-26T07:02:41.206 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 278937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-26T07:02:41.206 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-11-26T07:02:41.206 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1113681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 95665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/ 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 108649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ 2023-11-26T07:02:41.207 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 73441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/ 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 150905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/ 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 136721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/ 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/ 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-11-26T07:02:41.208 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 227897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 87081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 672049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/ 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/ 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 83985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 189801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-26T07:02:41.209 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/ 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 104241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/ 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1052497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/ 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/ 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/ 2023-11-26T07:02:41.210 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-26T07:02:41.211 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-26T07:02:41.211 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 73409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-26T07:02:41.211 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 68177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-26T07:02:41.211 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/ 2023-11-26T07:02:41.211 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-11-26T07:02:41.211 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-11-26T07:02:41.211 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-26T07:02:41.211 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-11-26T07:02:41.211 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 487025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-26T07:02:41.211 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/ 2023-11-26T07:02:41.212 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 103337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-26T07:02:41.212 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 166097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-26T07:02:41.212 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-26T07:02:41.212 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-26T07:02:41.212 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/ 2023-11-26T07:02:41.212 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-26T07:02:41.212 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/ 2023-11-26T07:02:41.212 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 86529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-26T07:02:41.212 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/ 2023-11-26T07:02:41.212 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 80273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-26T07:02:41.213 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 83633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-26T07:02:41.213 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/ 2023-11-26T07:02:41.213 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-26T07:02:41.213 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-26T07:02:41.213 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/ 2023-11-26T07:02:41.213 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-26T07:02:41.213 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/ 2023-11-26T07:02:41.213 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/defxx.ko 2023-11-26T07:02:41.213 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/ 2023-11-26T07:02:41.213 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 216985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/fddi/skfp/skfp.ko 2023-11-26T07:02:41.214 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 77345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-26T07:02:41.214 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/ 2023-11-26T07:02:41.214 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 235521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-26T07:02:41.214 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/ 2023-11-26T07:02:41.214 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-26T07:02:41.214 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-26T07:02:41.214 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ 2023-11-26T07:02:41.214 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 83769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-26T07:02:41.214 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-26T07:02:41.215 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-26T07:02:41.215 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/ 2023-11-26T07:02:41.215 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-26T07:02:41.215 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-26T07:02:41.215 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-gpio.ko 2023-11-26T07:02:41.215 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-26T07:02:41.215 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-26T07:02:41.215 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-26T07:02:41.215 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/ 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-26T07:02:41.216 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-26T07:02:41.217 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-26T07:02:41.217 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-26T07:02:41.217 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-26T07:02:41.217 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-26T07:02:41.217 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 82049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-26T07:02:41.217 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 76113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-26T07:02:41.217 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-26T07:02:41.217 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 97857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-26T07:02:41.217 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-26T07:02:41.217 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-26T07:02:41.218 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-26T07:02:41.218 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-26T07:02:41.218 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-26T07:02:41.218 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/ 2023-11-26T07:02:41.218 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/plip/plip.ko 2023-11-26T07:02:41.218 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ 2023-11-26T07:02:41.218 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-26T07:02:41.218 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sb1000.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/ 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 64449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-26T07:02:41.219 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/ 2023-11-26T07:02:41.220 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 106169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-26T07:02:41.220 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-26T07:02:41.220 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-26T07:02:41.220 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-26T07:02:41.220 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-26T07:02:41.220 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-26T07:02:41.220 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 87017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-26T07:02:41.220 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/ 2023-11-26T07:02:41.220 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 144297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/ 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 297329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/ 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 144585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/ 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/ 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 265121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/ 2023-11-26T07:02:41.221 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 133841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 84465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/ 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/ 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pci-stub.ko 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-26T07:02:41.222 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/ 2023-11-26T07:02:41.223 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-26T07:02:41.223 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 145809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia.ko 2023-11-26T07:02:41.223 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_core.ko 2023-11-26T07:02:41.223 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-11-26T07:02:41.223 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-26T07:02:41.223 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 112889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-26T07:02:41.223 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/ 2023-11-26T07:02:41.223 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/ 2023-11-26T07:02:41.223 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-11-26T07:02:41.223 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/ 2023-11-26T07:02:41.224 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-11-26T07:02:41.224 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/ 2023-11-26T07:02:41.224 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/ 2023-11-26T07:02:41.224 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-26T07:02:41.224 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/ 2023-11-26T07:02:41.224 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/ 2023-11-26T07:02:41.224 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-11-26T07:02:41.224 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec.ko 2023-11-26T07:02:41.224 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-11-26T07:02:41.224 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 127441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-26T07:02:41.225 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/ibm_rtl.ko 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/ 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-26T07:02:41.226 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/ 2023-11-26T07:02:41.227 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_battery.ko 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/88pm860x_charger.ko 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq2415x_charger.ko 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24190_charger.ko 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/bq24735-charger.ko 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9030_battery.ko 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/da9052-battery.ko 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2760_battery.ko 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2780_battery.ko 2023-11-26T07:02:41.228 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2781_battery.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/ds2782_battery.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/gpio-charger.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/isp1704_charger.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/lp8727_charger.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max14577_charger.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17040_battery.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max17042_battery.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8903_charger.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/max8925_power.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/pcf50633-charger.ko 2023-11-26T07:02:41.229 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/sbs-battery.ko 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/test_power.ko 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/tps65090-charger.ko 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_backup.ko 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm831x_power.ko 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/power/supply/wm8350_power.ko 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/ 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/ 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/ 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-26T07:02:41.230 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/ 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm80x.ko 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-88pm860x.ko 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9052.ko 2023-11-26T07:02:41.231 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9055.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-da9063.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 68577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-26T07:02:41.232 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-lp8788.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t86.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8907.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8925.ko 2023-11-26T07:02:41.233 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8997.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max8998.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-palmas.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf50633.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-26T07:02:41.234 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rc5t583.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-s35390a.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps6586x.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-tps65910.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm831x.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-wm8350.ko 2023-11-26T07:02:41.235 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 77617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 135185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/ 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 272305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 100617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/ 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 356201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-26T07:02:41.236 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 295801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/ 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 189185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/ 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 106881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/ 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 246985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/ 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1171945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-26T07:02:41.237 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/ 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 237273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/ 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 115369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/ 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 308921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/ 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 76145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-11-26T07:02:41.238 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 101449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 146689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 75553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/ 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 212425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/ 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 158505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fdomain.ko 2023-11-26T07:02:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/ 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 263489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 201817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/imm.ko 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 266537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 101881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/ 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 317153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-26T07:02:41.240 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/ 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 314865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 145585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/ 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 216665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/ 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 2197249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/ 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 73569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-26T07:02:41.241 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 310617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 84721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/ 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 659825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/ 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 119561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/ 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 81625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-11-26T07:02:41.242 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/ 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 364105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 123617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ppa.ko 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/ 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1876689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/ 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 559033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-26T07:02:41.243 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qlogicfas408.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 238761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 173937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 279313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 97729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 73881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 115889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/ 2023-11-26T07:02:41.244 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 162121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/xen-scsifront.ko 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 129441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/ 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/ 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 736601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/ 2023-11-26T07:02:41.245 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 930257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 122521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/ 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 75593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/ 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/ 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/ 2023-11-26T07:02:41.246 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-26T07:02:41.247 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/moxa.ko 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 50953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/mxser.ko 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 108601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-26T07:02:41.248 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/ 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/ 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/altera_uart.ko 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/ 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 103073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/rp2.ko 2023-11-26T07:02:41.249 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/sc16is7xx.ko 2023-11-26T07:02:41.250 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/ 2023-11-26T07:02:41.250 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ 2023-11-26T07:02:41.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 531265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-26T07:02:41.250 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ 2023-11-26T07:02:41.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-26T07:02:41.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-11-26T07:02:41.250 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/ 2023-11-26T07:02:41.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-26T07:02:41.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-26T07:02:41.250 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_dmem_genirq.ko 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_mf624.ko 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_netx.ko 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/ 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/ 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 73297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/c67x00/c67x00.ko 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 144377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-11-26T07:02:41.251 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/ 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 87089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/common/ulpi.ko 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/ 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-11-26T07:02:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/ 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 448505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/dwc3/dwc3.ko 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/ 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/ 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_ether.ko 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/u_serial.ko 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-11-26T07:02:41.253 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 92489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 122081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-11-26T07:02:41.254 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/ 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-11-26T07:02:41.255 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 62185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 161225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/libcomposite.ko 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/ 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/ 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 157681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 69705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-11-26T07:02:41.256 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 76057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2272.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 75073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/net2280.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 73169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 65809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 81505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 156073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/bcma-hcd.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 80113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/isp116x-hcd.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 87353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-11-26T07:02:41.257 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811_cs.ko 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/ssb-hcd.ko 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/ 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-26T07:02:41.258 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/cytherm.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ehset.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/lvstest.ko 2023-11-26T07:02:41.259 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/ 2023-11-26T07:02:41.260 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 64841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-26T07:02:41.260 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-26T07:02:41.260 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-26T07:02:41.260 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-26T07:02:41.260 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-26T07:02:41.260 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbtest.ko 2023-11-26T07:02:41.260 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-26T07:02:41.261 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-26T07:02:41.261 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/ 2023-11-26T07:02:41.261 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/mon/usbmon.ko 2023-11-26T07:02:41.261 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/ 2023-11-26T07:02:41.261 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 301697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/musb/musb_hdrc.ko 2023-11-26T07:02:41.261 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/ 2023-11-26T07:02:41.261 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-26T07:02:41.261 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-isp1301.ko 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-tahvo.ko 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-26T07:02:41.262 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/f81232.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 175137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 137081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 124833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-26T07:02:41.263 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 107521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/metro-usb.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 78217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mxuport.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-26T07:02:41.264 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-26T07:02:41.265 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 191137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-26T07:02:41.265 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-26T07:02:41.265 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-26T07:02:41.265 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-26T07:02:41.265 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-26T07:02:41.265 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-26T07:02:41.265 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-26T07:02:41.265 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 51625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-26T07:02:41.265 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-26T07:02:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-26T07:02:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-26T07:02:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-26T07:02:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-26T07:02:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-26T07:02:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-26T07:02:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 125609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usbserial.ko 2023-11-26T07:02:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-26T07:02:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-26T07:02:41.266 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/wishbone-serial.ko 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 52913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-26T07:02:41.267 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-26T07:02:41.268 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-26T07:02:41.268 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-26T07:02:41.268 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-26T07:02:41.268 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-26T07:02:41.268 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-26T07:02:41.268 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 179969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-26T07:02:41.268 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/ 2023-11-26T07:02:41.268 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-26T07:02:41.268 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 107001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/ 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/ 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 154297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 108737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 77609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/ 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 113785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-26T07:02:41.269 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 58089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/ 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/ 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/aat2870_bl.ko 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp5520_bl.ko 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8860_bl.ko 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/adp8870_bl.ko 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/as3711_bl.ko 2023-11-26T07:02:41.270 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/bd6107.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da903x_bl.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/da9052_bl.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/gpio_backlight.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/kb3886_bl.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3533_bl.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lm3639_bl.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lv5207lp.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/max8925_bl.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pandora_bl.ko 2023-11-26T07:02:41.271 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-11-26T07:02:41.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-26T07:02:41.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/wm831x_bl.ko 2023-11-26T07:02:41.272 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/ 2023-11-26T07:02:41.272 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ 2023-11-26T07:02:41.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-26T07:02:41.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-26T07:02:41.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/matrox_w1.ko 2023-11-26T07:02:41.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/w1-gpio.ko 2023-11-26T07:02:41.272 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/ 2023-11-26T07:02:41.272 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14153 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 56425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-26T07:02:41.273 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/acquirewdt.ko 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/advantechwdt.ko 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/cpu5wdt.ko 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9052_wdt.ko 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9055_wdt.ko 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/da9063_wdt.ko 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/dw_wdt.ko 2023-11-26T07:02:41.274 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/eurotechwdt.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-26T07:02:41.275 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/kempld_wdt.ko 2023-11-26T07:02:41.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-26T07:02:41.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/mena21_wdt.ko 2023-11-26T07:02:41.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-11-26T07:02:41.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-26T07:02:41.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-11-26T07:02:41.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22265 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pc87413_wdt.ko 2023-11-26T07:02:41.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-26T07:02:41.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-26T07:02:41.276 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/retu_wdt.ko 2023-11-26T07:02:41.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-11-26T07:02:41.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-11-26T07:02:41.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-26T07:02:41.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sc1200wdt.ko 2023-11-26T07:02:41.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-26T07:02:41.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-11-26T07:02:41.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-26T07:02:41.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-26T07:02:41.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/twl4030_wdt.ko 2023-11-26T07:02:41.277 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wafer5823wdt.ko 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm831x_wdt.ko 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wm8350_wdt.ko 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/ 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-26T07:02:41.278 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/ 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 176377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 71249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-scsiback.ko 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/ 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/ 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 166425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-26T07:02:41.279 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/ 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/adfs/adfs.ko 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/ 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 163593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/ 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 932817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/afs/kafs.ko 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/ 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 101057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/autofs/autofs4.ko 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/ 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 64545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/ 2023-11-26T07:02:41.280 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/bfs/bfs.ko 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/ 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 3402729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/ 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 306009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1333017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/ 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 100801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/ 2023-11-26T07:02:41.281 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 108393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/configfs/configfs.ko 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/ 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 29625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/ 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 642513 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/ 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/efs/efs.ko 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/ 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1505025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/ 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-26T07:02:41.282 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/ 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/freevxfs/freevxfs.ko 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/ 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 226481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/ 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/ 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1231529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/ 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 141209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/ 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 222817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-26T07:02:41.283 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/ 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 192329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hpfs/hpfs.ko 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/ 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/ 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 419497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/ 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 280313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/ 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/ 2023-11-26T07:02:41.284 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 124649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-26T07:02:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/ 2023-11-26T07:02:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/ 2023-11-26T07:02:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 81449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-26T07:02:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/ 2023-11-26T07:02:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 59449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-26T07:02:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/ 2023-11-26T07:02:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 117465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-26T07:02:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1055721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-26T07:02:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv2.ko 2023-11-26T07:02:41.285 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 120177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 2042137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/ 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/ 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1516873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/ 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 592833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/ 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-26T07:02:41.286 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ascii.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-26T07:02:41.287 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-26T07:02:41.288 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 94137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 142673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 165745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 118121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-26T07:02:41.289 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-26T07:02:41.290 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-26T07:02:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-26T07:02:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-26T07:02:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-26T07:02:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-26T07:02:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ 2023-11-26T07:02:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 219417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ntfs/ntfs.ko 2023-11-26T07:02:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ 2023-11-26T07:02:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ 2023-11-26T07:02:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 244409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-26T07:02:41.291 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ 2023-11-26T07:02:41.292 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 440737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-26T07:02:41.292 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ 2023-11-26T07:02:41.292 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-26T07:02:41.292 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 2592433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-26T07:02:41.292 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-26T07:02:41.292 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-26T07:02:41.292 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-26T07:02:41.292 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/ 2023-11-26T07:02:41.292 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/omfs/omfs.ko 2023-11-26T07:02:41.292 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/ 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 352161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/ 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx4/qnx4.ko 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/ 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 46689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/qnx6/qnx6.ko 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/ 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_tree.ko 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v1.ko 2023-11-26T07:02:41.293 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/quota/quota_v2.ko 2023-11-26T07:02:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/ 2023-11-26T07:02:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 532673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-26T07:02:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/ 2023-11-26T07:02:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-26T07:02:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/ 2023-11-26T07:02:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/ 2023-11-26T07:02:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 2668401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-26T07:02:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/ 2023-11-26T07:02:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-26T07:02:41.294 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/ 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 134641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/ 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 90721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/ 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 247737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 162497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/ 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 4011017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-26T07:02:41.295 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/ 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/ 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 75489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/locking/locktorture.ko 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/ 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/time/test_udelay.ko 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 6833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 6529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc7.ko 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/ 2023-11-26T07:02:41.296 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-26T07:02:41.297 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-26T07:02:41.297 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/interval_tree_test.ko 2023-11-26T07:02:41.297 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-26T07:02:41.297 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-26T07:02:41.297 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/ 2023-11-26T07:02:41.297 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-26T07:02:41.297 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-26T07:02:41.297 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/ 2023-11-26T07:02:41.297 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 6617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-26T07:02:41.297 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/memory-notifier-error-inject.ko 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/notifier-error-inject.ko 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/percpu_test.ko 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/pm-notifier-error-inject.ko 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/ 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 143433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/rbtree_test.ko 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/ 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-kstrtox.ko 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test-string_helpers.ko 2023-11-26T07:02:41.298 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 4760425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_bpf.ko 2023-11-26T07:02:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_firmware.ko 2023-11-26T07:02:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 6369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_module.ko 2023-11-26T07:02:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/test_user_copy.ko 2023-11-26T07:02:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-26T07:02:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-26T07:02:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-26T07:02:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/ 2023-11-26T07:02:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/lib/xz/xz_dec_test.ko 2023-11-26T07:02:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/ 2023-11-26T07:02:41.299 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/ 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 6985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/ 2023-11-26T07:02:41.300 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/ 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 81737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/ 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 198337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26345 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-26T07:02:41.301 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-26T07:02:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/ 2023-11-26T07:02:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 649297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-26T07:02:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/ 2023-11-26T07:02:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 63809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-26T07:02:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 777689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-26T07:02:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ 2023-11-26T07:02:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-26T07:02:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-26T07:02:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-26T07:02:41.302 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13217 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8385 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-26T07:02:41.303 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 55929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/ 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1046833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-26T07:02:41.304 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/ 2023-11-26T07:02:41.305 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 125993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-26T07:02:41.305 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/ 2023-11-26T07:02:41.305 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 194209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-26T07:02:41.305 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-26T07:02:41.305 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 57881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-26T07:02:41.305 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-26T07:02:41.305 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/ 2023-11-26T07:02:41.305 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 390473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-26T07:02:41.306 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-26T07:02:41.306 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-26T07:02:41.306 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-26T07:02:41.306 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-26T07:02:41.306 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/ 2023-11-26T07:02:41.306 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 99729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/hsr/hsr.ko 2023-11-26T07:02:41.306 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ 2023-11-26T07:02:41.306 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ 2023-11-26T07:02:41.306 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 45233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-26T07:02:41.306 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 254625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61225 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-26T07:02:41.307 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ip_tables.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11305 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-11-26T07:02:41.308 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18161 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-26T07:02:41.309 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-26T07:02:41.310 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-26T07:02:41.311 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-26T07:02:41.311 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-26T07:02:41.311 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-26T07:02:41.311 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-26T07:02:41.311 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-26T07:02:41.311 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 41665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-26T07:02:41.311 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-26T07:02:41.311 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 80897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_gre.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 93577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19017 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/mip6.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-11-26T07:02:41.312 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11121 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-26T07:02:41.313 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25945 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9009 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-26T07:02:41.314 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 74073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/ 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 82833 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/ 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 125281 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 25953 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36713 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-26T07:02:41.315 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 53241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/ 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 47977 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/lapb/lapb.ko 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/ 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 267913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc2.ko 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/ 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 290417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/ 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 119585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-26T07:02:41.316 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26913 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 100297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 101441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 103417 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 112849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 60329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 109593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 113129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 112585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-26T07:02:41.317 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 111585 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 114465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 411729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-26T07:02:41.318 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7689 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10681 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7697 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10993 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 408257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33761 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-26T07:02:41.319 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 118145 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 107233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66641 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10113 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9593 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-26T07:02:41.320 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 37129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 95353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11857 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28929 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 44065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 615649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 42561 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30233 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30569 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 39785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-26T07:02:41.321 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10457 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 40721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9601 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14185 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-26T07:02:41.322 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13673 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11841 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_osf.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_synproxy.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tproxy.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 89313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/x_tables.ko 2023-11-26T07:02:41.323 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 18169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13369 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13889 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11489 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11753 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-26T07:02:41.324 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 14337 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17361 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8321 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-26T07:02:41.325 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-26T07:02:41.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-26T07:02:41.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-26T07:02:41.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-26T07:02:41.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-26T07:02:41.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10465 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-26T07:02:41.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-26T07:02:41.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-26T07:02:41.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13129 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-26T07:02:41.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-26T07:02:41.326 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8801 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7865 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11921 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11785 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-26T07:02:41.327 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16041 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8001 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10209 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7545 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11825 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11985 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-26T07:02:41.328 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13329 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 16089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9505 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9473 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8809 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 8553 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-26T07:02:41.329 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15777 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpudp.ko 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/ 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13969 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/ 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 12425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/ 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 380033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 11793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-26T07:02:41.330 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/ 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13057 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/ 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 110201 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/phonet.ko 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54721 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/phonet/pn_pep.ko 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/ 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 251481 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 164937 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 67377 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/ 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 819657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/rxrpc/rxrpc.ko 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/ 2023-11-26T07:02:41.331 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32241 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23577 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26873 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 43449 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35297 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33961 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-26T07:02:41.332 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19169 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_cgroup.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31401 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31769 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 38425 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 61409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7441 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 13409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33289 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7249 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 21649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-26T07:02:41.333 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20849 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 34633 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 33609 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31617 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq_codel.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 35433 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 49817 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22089 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 66393 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26073 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 30273 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10793 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-26T07:02:41.334 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26065 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 36137 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 27497 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10625 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 24729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 48729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 32905 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 26881 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 31353 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28897 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 28105 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/ 2023-11-26T07:02:41.335 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 929745 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 19033 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/ 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/ 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 336529 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 54737 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 1520097 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/ 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 707657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/ 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10025 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 862665 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-26T07:02:41.336 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/ 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15729 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/ 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 85409 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 68049 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 96193 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 10705 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 17521 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/ 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 125177 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/x25/x25.ko 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/ 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 22657 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_algo.ko 2023-11-26T07:02:41.337 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 20649 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 102257 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_user.ko 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/ 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/ 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 15537 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 640081 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 624107 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 7663 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 23428 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 9821 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 70004 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-26T07:02:41.338 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 146313 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 232548 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 210 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 72606 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 879 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 254438 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 309363 2023-11-23 03:47 ./lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/ 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/ 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/ 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:drwxr-xr-x root/root 0 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/ 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 172 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/changelog.Debian.gz 2023-11-26T07:02:41.339 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r-- root/root 691 2023-11-23 03:47 ./usr/share/doc/linux-image-6.6.0-g8247bbb5b470/copyright 2023-11-26T07:02:41.340 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-26T07:02:41.340 INFO:teuthology.task.kernel:searching for kernel 6.6.0-g8247bbb5b470 2023-11-26T07:02:41.340 DEBUG:teuthology.orchestra.run.smithi097:> egrep '(submenu|menuentry.*6.6.0-g8247bbb5b470).*{' /boot/grub/grub.cfg 2023-11-26T07:02:41.347 INFO:teuthology.orchestra.run.smithi097.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:41.347 INFO:teuthology.orchestra.run.smithi097.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g8247bbb5b470 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g8247bbb5b470-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-11-26T07:02:41.348 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-11-26T07:02:41.348 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-g8247bbb5b470 2023-11-26T07:02:41.348 DEBUG:teuthology.orchestra.run.smithi097:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-11-26T07:02:41.390 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi097.front.sepia.ceph.com to complete... 2023-11-26T07:02:41.802 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub' 2023-11-26T07:02:41.803 INFO:teuthology.orchestra.run.smithi097.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-11-26T07:02:41.838 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2023-11-26T07:02:42.066 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-26T07:02:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-6.6.0-g8247bbb5b470 2023-11-26T07:02:42.315 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-11-26T07:02:42.320 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-11-26T07:02:42.362 INFO:teuthology.orchestra.run.smithi097.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-11-26T07:02:42.368 INFO:teuthology.orchestra.run.smithi097.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-11-26T07:02:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-11-26T07:02:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-11-26T07:02:42.509 INFO:teuthology.orchestra.run.smithi097.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-11-26T07:02:42.530 INFO:teuthology.orchestra.run.smithi097.stderr:done 2023-11-26T07:03:00.023 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-26T07:03:00.024 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-11-26T07:03:00.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:03:13.690 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-11-26T07:03:13.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-11-26T07:03:13.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:03:18.370 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2023-11-26T07:03:27.378 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-11-26T07:03:27.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:04:13.733 DEBUG:teuthology.orchestra.remote:timed out 2023-11-26T07:04:22.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-11-26T07:04:22.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:04:22.746 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2023-11-26T07:04:27.386 DEBUG:teuthology.orchestra.remote:timed out 2023-11-26T07:04:34.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-11-26T07:04:34.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:04:34.966 DEBUG:teuthology.orchestra.run.smithi097:> true 2023-11-26T07:04:35.569 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-11-26T07:04:35.569 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-g8247bbb5b470"... 2023-11-26T07:04:35.569 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2023-11-26T07:04:35.621 INFO:teuthology.orchestra.run.smithi097.stdout:6.6.0-g8247bbb5b470 2023-11-26T07:04:35.621 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-26T07:04:35.621 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-26T07:04:35.621 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-26T07:04:36.622 DEBUG:teuthology.parallel:result is None 2023-11-26T07:04:39.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-11-26T07:04:39.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:04:39.603 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-11-26T07:04:40.307 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-11-26T07:04:40.308 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g8247bbb5b470"... 2023-11-26T07:04:40.308 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2023-11-26T07:04:40.361 INFO:teuthology.orchestra.run.smithi031.stdout:6.6.0-g8247bbb5b470 2023-11-26T07:04:40.361 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-26T07:04:40.361 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-26T07:04:40.361 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-11-26T07:04:41.362 DEBUG:teuthology.parallel:result is None 2023-11-26T07:04:41.363 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-26T07:04:41.371 INFO:teuthology.task.internal:Creating test directory... 2023-11-26T07:04:41.372 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-26T07:04:41.376 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-26T07:04:41.385 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-26T07:04:41.412 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-26T07:04:41.431 INFO:teuthology.task.internal:Creating archive directory... 2023-11-26T07:04:41.431 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-26T07:04:41.435 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-26T07:04:41.463 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-26T07:04:41.473 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-26T07:04:41.474 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-26T07:04:41.504 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-26T07:04:41.585 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T07:04:41.589 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T07:04:41.606 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T07:04:41.611 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-26T07:04:41.614 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-26T07:04:41.623 INFO:teuthology.task.internal:Configuring sudo... 2023-11-26T07:04:41.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-26T07:04:41.627 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-26T07:04:41.648 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-26T07:04:41.757 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-26T07:04:41.758 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-26T07:04:41.762 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-26T07:04:41.770 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-26T07:04:41.818 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-26T07:04:41.869 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:04:41.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-26T07:04:41.941 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-26T07:04:41.950 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-26T07:04:42.006 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:04:42.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-26T07:04:42.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2023-11-26T07:04:42.089 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2023-11-26T07:04:42.231 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-26T07:04:42.239 INFO:teuthology.task.internal:Starting timer... 2023-11-26T07:04:42.239 INFO:teuthology.run_tasks:Running task pcp... 2023-11-26T07:04:42.260 INFO:teuthology.run_tasks:Running task selinux... 2023-11-26T07:04:42.273 DEBUG:teuthology.task.selinux:Excluding smithi031: OS 'ubuntu' does not support SELinux 2023-11-26T07:04:42.273 DEBUG:teuthology.task.selinux:Excluding smithi097: OS 'ubuntu' does not support SELinux 2023-11-26T07:04:42.273 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-26T07:04:42.273 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-11-26T07:04:42.274 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-26T07:04:42.274 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-26T07:04:42.286 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-26T07:04:42.286 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-26T07:04:42.302 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-26T07:04:42.303 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi097.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-26T07:13:25.757 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com')] 2023-11-26T07:13:25.758 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-11-26T07:13:25.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:13:25.838 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-11-26T07:13:25.969 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-11-26T07:13:25.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-11-26T07:13:25.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-26T07:13:26.047 DEBUG:teuthology.orchestra.run.smithi097:> true 2023-11-26T07:13:26.181 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-11-26T07:13:26.182 INFO:teuthology.run_tasks:Running task clock... 2023-11-26T07:13:26.192 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-26T07:13:26.192 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-26T07:13:26.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:13:26.196 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-26T07:13:26.196 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:13:26.233 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-26T07:13:26.234 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: Command line: ntpd -gq 2023-11-26T07:13:26.234 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: ---------------------------------------------------- 2023-11-26T07:13:26.234 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: ntp-4 is maintained by Network Time Foundation, 2023-11-26T07:13:26.234 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-26T07:13:26.234 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: corporation. Support and training for ntp-4 are 2023-11-26T07:13:26.234 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: available at https://www.nwtime.org/support 2023-11-26T07:13:26.234 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: ---------------------------------------------------- 2023-11-26T07:13:26.235 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: proto: precision = 0.046 usec (-24) 2023-11-26T07:13:26.235 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: basedate set to 2022-02-04 2023-11-26T07:13:26.235 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: gps base set to 2022-02-06 (week 2196) 2023-11-26T07:13:26.235 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-26T07:13:26.235 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: restrict ::: KOD does nothing without LIMITED. 2023-11-26T07:13:26.235 INFO:teuthology.orchestra.run.smithi031.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-26T07:13:26.236 INFO:teuthology.orchestra.run.smithi031.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-26T07:13:26.237 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: Listen and drop on 0 v6wildcard [::]:123 2023-11-26T07:13:26.237 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-26T07:13:26.237 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: Listen normally on 2 lo 127.0.0.1:123 2023-11-26T07:13:26.237 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: Listen normally on 3 ens1f0 172.21.15.31:123 2023-11-26T07:13:26.237 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: Listen normally on 4 lo [::1]:123 2023-11-26T07:13:26.237 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fea5:caba%4]:123 2023-11-26T07:13:26.237 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:26 ntpd[27868]: Listening on routing socket on fd #22 for interface updates 2023-11-26T07:13:26.263 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-11-26T07:13:26.263 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: Command line: ntpd -gq 2023-11-26T07:13:26.264 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: ---------------------------------------------------- 2023-11-26T07:13:26.264 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: ntp-4 is maintained by Network Time Foundation, 2023-11-26T07:13:26.264 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-11-26T07:13:26.264 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: corporation. Support and training for ntp-4 are 2023-11-26T07:13:26.264 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: available at https://www.nwtime.org/support 2023-11-26T07:13:26.264 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: ---------------------------------------------------- 2023-11-26T07:13:26.265 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: proto: precision = 0.041 usec (-24) 2023-11-26T07:13:26.265 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: basedate set to 2022-02-04 2023-11-26T07:13:26.265 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: gps base set to 2022-02-06 (week 2196) 2023-11-26T07:13:26.265 INFO:teuthology.orchestra.run.smithi097.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-26T07:13:26.266 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-11-26T07:13:26.266 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: restrict ::: KOD does nothing without LIMITED. 2023-11-26T07:13:26.266 INFO:teuthology.orchestra.run.smithi097.stderr:restrict ::: KOD does nothing without LIMITED. 2023-11-26T07:13:26.267 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: Listen and drop on 0 v6wildcard [::]:123 2023-11-26T07:13:26.267 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-11-26T07:13:26.267 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: Listen normally on 2 lo 127.0.0.1:123 2023-11-26T07:13:26.267 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: Listen normally on 3 enp3s0f1 172.21.15.97:123 2023-11-26T07:13:26.267 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: Listen normally on 4 lo [::1]:123 2023-11-26T07:13:26.267 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d805%5]:123 2023-11-26T07:13:26.267 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:26 ntpd[27972]: Listening on routing socket on fd #22 for interface updates 2023-11-26T07:13:27.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:27.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:27.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:27.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:27.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:27.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:27 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:28 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:28.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:28 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:29.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:29.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:29.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:29.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:29.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:29.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:29.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:29.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:29.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:29.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:29.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:29.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:29.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:29.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:29 ntpd[27868]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:29.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:29.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:29.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:29.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:29.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:29.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:29.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:29.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:29.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:29.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:29.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:29.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:29 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:30 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:30.234 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:30 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:30 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:30 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:30 ntpd[27868]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:30.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:30 ntpd[27868]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:30.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:30 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:30.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:30 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:30.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:30 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:30.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:30 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:30.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:30 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:30.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:30 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:31.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:31.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:31.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:31 ntpd[27868]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:31.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:31.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:31.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:31.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:31.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:31.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:31.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:31.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:31.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:31.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:31.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:31.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:31 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:32.234 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:32 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:32 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:32 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:32 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:32 ntpd[27868]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:32.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:32 ntpd[27868]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:32 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:32 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:32.264 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:32 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:32 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:32 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:32.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:32 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:33.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:33.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:33 ntpd[27868]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:33 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:33 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:33 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:33 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:33 ntpd[27972]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:33 ntpd[27972]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:33 ntpd[27972]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:33.265 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:33 ntpd[27972]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:33.266 INFO:teuthology.orchestra.run.smithi097.stdout:26 Nov 07:13:33 ntpd[27972]: ntpd: time slew -0.001509 s 2023-11-26T07:13:33.266 INFO:teuthology.orchestra.run.smithi097.stdout:ntpd: time slew -0.001509s 2023-11-26T07:13:33.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:33 ntpd[27972]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-26T07:13:33.266 INFO:teuthology.orchestra.run.smithi097.stderr:26 Nov 07:13:33 ntpd[27972]: can't open /var/log/ntpstats/loopstats.20231126: Permission denied 2023-11-26T07:13:33.365 INFO:teuthology.orchestra.run.smithi097.stdout: remote refid st t when poll reach delay offset jitter 2023-11-26T07:13:33.365 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================== 2023-11-26T07:13:33.365 INFO:teuthology.orchestra.run.smithi097.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:33.365 INFO:teuthology.orchestra.run.smithi097.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:33.365 INFO:teuthology.orchestra.run.smithi097.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:33.365 INFO:teuthology.orchestra.run.smithi097.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:34.234 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:34 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:34.234 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:34 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:34.234 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:34 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:34.234 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:34 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:34.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:34 ntpd[27868]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:34.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:34 ntpd[27868]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:35.234 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:35 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:35.234 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:35 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:35.234 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:35 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:35.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:35 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:35.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:35 ntpd[27868]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-11-26T07:13:35.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:35 ntpd[27868]: can't open /var/log/ntpstats/rawstats.20231126: Permission denied 2023-11-26T07:13:35.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:35 ntpd[27868]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-11-26T07:13:35.235 INFO:teuthology.orchestra.run.smithi031.stdout:26 Nov 07:13:35 ntpd[27868]: ntpd: time slew +0.000119 s 2023-11-26T07:13:35.235 INFO:teuthology.orchestra.run.smithi031.stdout:ntpd: time slew +0.000119s 2023-11-26T07:13:35.235 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:35 ntpd[27868]: can't open /var/log/ntpstats/peerstats.20231126: Permission denied 2023-11-26T07:13:35.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:35 ntpd[27868]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-11-26T07:13:35.236 INFO:teuthology.orchestra.run.smithi031.stderr:26 Nov 07:13:35 ntpd[27868]: can't open /var/log/ntpstats/loopstats.20231126: Permission denied 2023-11-26T07:13:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: remote refid st t when poll reach delay offset jitter 2023-11-26T07:13:35.343 INFO:teuthology.orchestra.run.smithi031.stdout:============================================================================== 2023-11-26T07:13:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-11-26T07:13:35.344 INFO:teuthology.run_tasks:Running task install... 2023-11-26T07:13:35.355 DEBUG:teuthology.task.install:project ceph 2023-11-26T07:13:35.355 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}} 2023-11-26T07:13:35.355 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'} 2023-11-26T07:13:35.355 INFO:teuthology.task.install:Using flavor: default 2023-11-26T07:13:35.361 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-26T07:13:35.361 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-26T07:13:35.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo apt-key list | grep Ceph 2023-11-26T07:13:35.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-key list | grep Ceph 2023-11-26T07:13:35.469 INFO:teuthology.orchestra.run.smithi097.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-26T07:13:35.498 INFO:teuthology.orchestra.run.smithi031.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-11-26T07:13:35.525 INFO:teuthology.orchestra.run.smithi097.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-26T07:13:35.526 INFO:teuthology.orchestra.run.smithi097.stdout:uid [ unknown] Ceph.com (release key) 2023-11-26T07:13:35.526 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-26T07:13:35.526 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-26T07:13:35.526 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:13:35.557 INFO:teuthology.orchestra.run.smithi031.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-11-26T07:13:35.557 INFO:teuthology.orchestra.run.smithi031.stdout:uid [ unknown] Ceph.com (release key) 2023-11-26T07:13:35.557 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-11-26T07:13:35.558 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-11-26T07:13:35.558 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:13:35.669 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default/ 2023-11-26T07:13:35.670 INFO:teuthology.task.install.deb:Package version is 18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:13:35.706 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default/ 2023-11-26T07:13:35.706 INFO:teuthology.task.install.deb:Package version is 18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:13:36.009 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:13:36.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-26T07:13:36.032 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:13:36.032 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-11-26T07:13:36.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo apt-get update 2023-11-26T07:13:36.054 DEBUG:teuthology.orchestra.run.smithi097:> sudo apt-get update 2023-11-26T07:13:36.221 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-26T07:13:36.221 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-26T07:13:36.227 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-26T07:13:36.243 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-11-26T07:13:36.244 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-26T07:13:36.252 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-11-26T07:13:36.272 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-11-26T07:13:36.360 INFO:teuthology.orchestra.run.smithi097.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-11-26T07:13:36.476 INFO:teuthology.orchestra.run.smithi031.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy InRelease 2023-11-26T07:13:36.496 INFO:teuthology.orchestra.run.smithi097.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy InRelease 2023-11-26T07:13:36.549 INFO:teuthology.orchestra.run.smithi031.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-11-26T07:13:36.569 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-11-26T07:13:36.623 INFO:teuthology.orchestra.run.smithi031.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-26T07:13:36.644 INFO:teuthology.orchestra.run.smithi097.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy Release.gpg 2023-11-26T07:13:36.697 INFO:teuthology.orchestra.run.smithi031.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-11-26T07:13:36.718 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-11-26T07:13:36.857 INFO:teuthology.orchestra.run.smithi031.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2023-11-26T07:13:36.875 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2023-11-26T07:13:36.967 INFO:teuthology.orchestra.run.smithi031.stdout:Fetched 28.5 kB in 1s (40.4 kB/s) 2023-11-26T07:13:36.982 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 28.5 kB in 1s (40.5 kB/s) 2023-11-26T07:13:37.908 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2023-11-26T07:13:37.922 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7484-g44c2237f-1jammy cephadm=18.0.0-7484-g44c2237f-1jammy ceph-mds=18.0.0-7484-g44c2237f-1jammy ceph-mgr=18.0.0-7484-g44c2237f-1jammy ceph-common=18.0.0-7484-g44c2237f-1jammy ceph-fuse=18.0.0-7484-g44c2237f-1jammy ceph-test=18.0.0-7484-g44c2237f-1jammy ceph-volume=18.0.0-7484-g44c2237f-1jammy radosgw=18.0.0-7484-g44c2237f-1jammy python3-rados=18.0.0-7484-g44c2237f-1jammy python3-rgw=18.0.0-7484-g44c2237f-1jammy python3-cephfs=18.0.0-7484-g44c2237f-1jammy python3-rbd=18.0.0-7484-g44c2237f-1jammy libcephfs2=18.0.0-7484-g44c2237f-1jammy libcephfs-dev=18.0.0-7484-g44c2237f-1jammy librados2=18.0.0-7484-g44c2237f-1jammy librbd1=18.0.0-7484-g44c2237f-1jammy rbd-fuse=18.0.0-7484-g44c2237f-1jammy python3-cephfs=18.0.0-7484-g44c2237f-1jammy cephfs-shell=18.0.0-7484-g44c2237f-1jammy cephfs-top=18.0.0-7484-g44c2237f-1jammy cephfs-mirror=18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:13:37.923 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2023-11-26T07:13:37.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7484-g44c2237f-1jammy cephadm=18.0.0-7484-g44c2237f-1jammy ceph-mds=18.0.0-7484-g44c2237f-1jammy ceph-mgr=18.0.0-7484-g44c2237f-1jammy ceph-common=18.0.0-7484-g44c2237f-1jammy ceph-fuse=18.0.0-7484-g44c2237f-1jammy ceph-test=18.0.0-7484-g44c2237f-1jammy ceph-volume=18.0.0-7484-g44c2237f-1jammy radosgw=18.0.0-7484-g44c2237f-1jammy python3-rados=18.0.0-7484-g44c2237f-1jammy python3-rgw=18.0.0-7484-g44c2237f-1jammy python3-cephfs=18.0.0-7484-g44c2237f-1jammy python3-rbd=18.0.0-7484-g44c2237f-1jammy libcephfs2=18.0.0-7484-g44c2237f-1jammy libcephfs-dev=18.0.0-7484-g44c2237f-1jammy librados2=18.0.0-7484-g44c2237f-1jammy librbd1=18.0.0-7484-g44c2237f-1jammy rbd-fuse=18.0.0-7484-g44c2237f-1jammy python3-cephfs=18.0.0-7484-g44c2237f-1jammy cephfs-shell=18.0.0-7484-g44c2237f-1jammy cephfs-top=18.0.0-7484-g44c2237f-1jammy cephfs-mirror=18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:13:37.997 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2023-11-26T07:13:38.014 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2023-11-26T07:13:38.162 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2023-11-26T07:13:38.163 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2023-11-26T07:13:38.183 INFO:teuthology.orchestra.run.smithi031.stdout:Building dependency tree... 2023-11-26T07:13:38.184 INFO:teuthology.orchestra.run.smithi031.stdout:Reading state information... 2023-11-26T07:13:38.316 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2023-11-26T07:13:38.316 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-26T07:13:38.316 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-26T07:13:38.316 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-26T07:13:38.316 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-26T07:13:38.317 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-26T07:13:38.317 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liboath0 libonig5 libpcre2-16-0 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout: libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 librdkafka1 librgw2 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout: libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli python-asyncssh-doc 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common python3-certifi python3-cheroot python3-cherrypy3 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 python3-colorama python3-google-auth python3-idna 2023-11-26T07:13:38.319 INFO:teuthology.orchestra.run.smithi097.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pyperclip 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytest python3-repoze.lru python3-requests python3-requests-oauthlib 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2023-11-26T07:13:38.320 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2023-11-26T07:13:38.321 INFO:teuthology.orchestra.run.smithi097.stdout: socat xmlstarlet 2023-11-26T07:13:38.321 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2023-11-26T07:13:38.321 INFO:teuthology.orchestra.run.smithi097.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-11-26T07:13:38.321 INFO:teuthology.orchestra.run.smithi097.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-11-26T07:13:38.321 INFO:teuthology.orchestra.run.smithi097.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-11-26T07:13:38.321 INFO:teuthology.orchestra.run.smithi097.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-11-26T07:13:38.321 INFO:teuthology.orchestra.run.smithi097.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-11-26T07:13:38.321 INFO:teuthology.orchestra.run.smithi097.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-11-26T07:13:38.321 INFO:teuthology.orchestra.run.smithi097.stdout: python3-watchdog 2023-11-26T07:13:38.322 INFO:teuthology.orchestra.run.smithi097.stdout:Recommended packages: 2023-11-26T07:13:38.322 INFO:teuthology.orchestra.run.smithi097.stdout: btrfs-tools 2023-11-26T07:13:38.341 INFO:teuthology.orchestra.run.smithi031.stdout:The following packages were automatically installed and are no longer required: 2023-11-26T07:13:38.341 INFO:teuthology.orchestra.run.smithi031.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-26T07:13:38.341 INFO:teuthology.orchestra.run.smithi031.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-26T07:13:38.341 INFO:teuthology.orchestra.run.smithi031.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-26T07:13:38.342 INFO:teuthology.orchestra.run.smithi031.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-26T07:13:38.342 INFO:teuthology.orchestra.run.smithi031.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-26T07:13:38.342 INFO:teuthology.orchestra.run.smithi031.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-26T07:13:38.342 INFO:teuthology.orchestra.run.smithi031.stdout:The following additional packages will be installed: 2023-11-26T07:13:38.343 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-11-26T07:13:38.343 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-11-26T07:13:38.343 INFO:teuthology.orchestra.run.smithi031.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liboath0 libonig5 libpcre2-16-0 2023-11-26T07:13:38.343 INFO:teuthology.orchestra.run.smithi031.stdout: libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 librdkafka1 librgw2 2023-11-26T07:13:38.344 INFO:teuthology.orchestra.run.smithi031.stdout: libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli python-asyncssh-doc 2023-11-26T07:13:38.344 INFO:teuthology.orchestra.run.smithi031.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2023-11-26T07:13:38.344 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2023-11-26T07:13:38.344 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common python3-certifi python3-cheroot python3-cherrypy3 2023-11-26T07:13:38.344 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cmd2 python3-colorama python3-google-auth python3-idna 2023-11-26T07:13:38.344 INFO:teuthology.orchestra.run.smithi031.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2023-11-26T07:13:38.344 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2023-11-26T07:13:38.344 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2023-11-26T07:13:38.344 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2023-11-26T07:13:38.345 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2023-11-26T07:13:38.345 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2023-11-26T07:13:38.345 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pyperclip 2023-11-26T07:13:38.345 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytest python3-repoze.lru python3-requests python3-requests-oauthlib 2023-11-26T07:13:38.345 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2023-11-26T07:13:38.345 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2023-11-26T07:13:38.345 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2023-11-26T07:13:38.345 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2023-11-26T07:13:38.345 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2023-11-26T07:13:38.345 INFO:teuthology.orchestra.run.smithi031.stdout: socat xmlstarlet 2023-11-26T07:13:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:Suggested packages: 2023-11-26T07:13:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-11-26T07:13:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-11-26T07:13:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-11-26T07:13:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-11-26T07:13:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-11-26T07:13:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-11-26T07:13:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: python3-watchdog 2023-11-26T07:13:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:Recommended packages: 2023-11-26T07:13:38.346 INFO:teuthology.orchestra.run.smithi031.stdout: btrfs-tools 2023-11-26T07:13:38.463 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2023-11-26T07:13:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-26T07:13:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-26T07:13:38.463 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-26T07:13:38.464 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-26T07:13:38.464 INFO:teuthology.orchestra.run.smithi097.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liboath0 libonig5 libpcre2-16-0 2023-11-26T07:13:38.464 INFO:teuthology.orchestra.run.smithi097.stdout: libqt5core5a libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 2023-11-26T07:13:38.465 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-11-26T07:13:38.465 INFO:teuthology.orchestra.run.smithi097.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-26T07:13:38.465 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-26T07:13:38.465 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-certifi 2023-11-26T07:13:38.465 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2023-11-26T07:13:38.465 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2023-11-26T07:13:38.465 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-26T07:13:38.465 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip python3-pytest python3-rados python3-rbd 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2023-11-26T07:13:38.466 INFO:teuthology.orchestra.run.smithi097.stdout: radosgw rbd-fuse socat xmlstarlet 2023-11-26T07:13:38.493 INFO:teuthology.orchestra.run.smithi031.stdout:The following NEW packages will be installed: 2023-11-26T07:13:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-11-26T07:13:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-11-26T07:13:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-11-26T07:13:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-11-26T07:13:38.493 INFO:teuthology.orchestra.run.smithi031.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liboath0 libonig5 libpcre2-16-0 2023-11-26T07:13:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: libqt5core5a libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 2023-11-26T07:13:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-11-26T07:13:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-11-26T07:13:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-11-26T07:13:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-certifi 2023-11-26T07:13:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2023-11-26T07:13:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2023-11-26T07:13:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-11-26T07:13:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-11-26T07:13:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-11-26T07:13:38.495 INFO:teuthology.orchestra.run.smithi031.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-11-26T07:13:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-11-26T07:13:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-11-26T07:13:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip python3-pytest python3-rados python3-rbd 2023-11-26T07:13:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2023-11-26T07:13:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2023-11-26T07:13:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2023-11-26T07:13:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2023-11-26T07:13:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2023-11-26T07:13:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2023-11-26T07:13:38.496 INFO:teuthology.orchestra.run.smithi031.stdout: radosgw rbd-fuse socat xmlstarlet 2023-11-26T07:13:38.567 INFO:teuthology.orchestra.run.smithi031.stdout:0 upgraded, 114 newly installed, 0 to remove and 264 not upgraded. 2023-11-26T07:13:38.567 INFO:teuthology.orchestra.run.smithi031.stdout:Need to get 186 MB of archives. 2023-11-26T07:13:38.568 INFO:teuthology.orchestra.run.smithi031.stdout:After this operation, 829 MB of additional disk space will be used. 2023-11-26T07:13:38.568 INFO:teuthology.orchestra.run.smithi031.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-26T07:13:38.588 INFO:teuthology.orchestra.run.smithi031.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-26T07:13:38.598 INFO:teuthology.orchestra.run.smithi031.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-26T07:13:38.649 INFO:teuthology.orchestra.run.smithi031.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-26T07:13:38.660 INFO:teuthology.orchestra.run.smithi097.stdout:0 upgraded, 114 newly installed, 0 to remove and 264 not upgraded. 2023-11-26T07:13:38.660 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 186 MB of archives. 2023-11-26T07:13:38.660 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 829 MB of additional disk space will be used. 2023-11-26T07:13:38.660 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-11-26T07:13:38.672 INFO:teuthology.orchestra.run.smithi031.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-26T07:13:38.676 INFO:teuthology.orchestra.run.smithi031.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-26T07:13:38.689 INFO:teuthology.orchestra.run.smithi031.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-26T07:13:38.733 INFO:teuthology.orchestra.run.smithi031.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-26T07:13:38.735 INFO:teuthology.orchestra.run.smithi031.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-26T07:13:38.742 INFO:teuthology.orchestra.run.smithi031.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-26T07:13:38.742 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-11-26T07:13:38.745 INFO:teuthology.orchestra.run.smithi031.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-26T07:13:38.745 INFO:teuthology.orchestra.run.smithi031.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-26T07:13:38.751 INFO:teuthology.orchestra.run.smithi031.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-26T07:13:38.752 INFO:teuthology.orchestra.run.smithi031.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-26T07:13:38.752 INFO:teuthology.orchestra.run.smithi031.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-26T07:13:38.753 INFO:teuthology.orchestra.run.smithi031.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-26T07:13:38.754 INFO:teuthology.orchestra.run.smithi031.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-26T07:13:38.774 INFO:teuthology.orchestra.run.smithi031.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-26T07:13:38.774 INFO:teuthology.orchestra.run.smithi031.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-26T07:13:38.775 INFO:teuthology.orchestra.run.smithi031.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-26T07:13:38.775 INFO:teuthology.orchestra.run.smithi031.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-26T07:13:38.775 INFO:teuthology.orchestra.run.smithi031.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-26T07:13:38.778 INFO:teuthology.orchestra.run.smithi031.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-26T07:13:38.778 INFO:teuthology.orchestra.run.smithi031.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-26T07:13:38.779 INFO:teuthology.orchestra.run.smithi031.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-26T07:13:38.779 INFO:teuthology.orchestra.run.smithi031.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-26T07:13:38.788 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-11-26T07:13:38.793 INFO:teuthology.orchestra.run.smithi031.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-26T07:13:38.800 INFO:teuthology.orchestra.run.smithi031.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-26T07:13:38.800 INFO:teuthology.orchestra.run.smithi031.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-26T07:13:38.801 INFO:teuthology.orchestra.run.smithi031.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-26T07:13:38.802 INFO:teuthology.orchestra.run.smithi031.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-26T07:13:38.803 INFO:teuthology.orchestra.run.smithi031.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-26T07:13:38.807 INFO:teuthology.orchestra.run.smithi031.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-26T07:13:38.807 INFO:teuthology.orchestra.run.smithi031.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-26T07:13:38.808 INFO:teuthology.orchestra.run.smithi031.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-26T07:13:38.813 INFO:teuthology.orchestra.run.smithi031.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-26T07:13:38.828 INFO:teuthology.orchestra.run.smithi031.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-26T07:13:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-26T07:13:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-11-26T07:13:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-26T07:13:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-26T07:13:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-26T07:13:38.835 INFO:teuthology.orchestra.run.smithi031.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-26T07:13:38.837 INFO:teuthology.orchestra.run.smithi031.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-26T07:13:38.886 INFO:teuthology.orchestra.run.smithi031.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-26T07:13:38.887 INFO:teuthology.orchestra.run.smithi031.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-26T07:13:38.888 INFO:teuthology.orchestra.run.smithi031.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-26T07:13:38.888 INFO:teuthology.orchestra.run.smithi031.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-26T07:13:38.889 INFO:teuthology.orchestra.run.smithi031.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-26T07:13:38.890 INFO:teuthology.orchestra.run.smithi031.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-26T07:13:38.909 INFO:teuthology.orchestra.run.smithi031.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-26T07:13:38.911 INFO:teuthology.orchestra.run.smithi031.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-26T07:13:38.912 INFO:teuthology.orchestra.run.smithi031.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-26T07:13:38.929 INFO:teuthology.orchestra.run.smithi031.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-26T07:13:38.929 INFO:teuthology.orchestra.run.smithi031.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-26T07:13:38.930 INFO:teuthology.orchestra.run.smithi031.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-26T07:13:38.930 INFO:teuthology.orchestra.run.smithi031.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-26T07:13:38.931 INFO:teuthology.orchestra.run.smithi031.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-26T07:13:38.932 INFO:teuthology.orchestra.run.smithi031.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-26T07:13:38.944 INFO:teuthology.orchestra.run.smithi031.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-26T07:13:38.944 INFO:teuthology.orchestra.run.smithi031.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-26T07:13:38.948 INFO:teuthology.orchestra.run.smithi031.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-26T07:13:38.963 INFO:teuthology.orchestra.run.smithi031.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-26T07:13:38.965 INFO:teuthology.orchestra.run.smithi031.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-26T07:13:38.966 INFO:teuthology.orchestra.run.smithi031.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-26T07:13:38.969 INFO:teuthology.orchestra.run.smithi031.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-26T07:13:38.972 INFO:teuthology.orchestra.run.smithi031.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-26T07:13:38.972 INFO:teuthology.orchestra.run.smithi031.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-26T07:13:38.977 INFO:teuthology.orchestra.run.smithi031.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-26T07:13:38.979 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7484-g44c2237f-1jammy [3,468 kB] 2023-11-26T07:13:38.983 INFO:teuthology.orchestra.run.smithi031.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-26T07:13:38.983 INFO:teuthology.orchestra.run.smithi031.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-26T07:13:38.984 INFO:teuthology.orchestra.run.smithi031.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-26T07:13:38.990 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-11-26T07:13:39.002 INFO:teuthology.orchestra.run.smithi031.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-26T07:13:39.004 INFO:teuthology.orchestra.run.smithi031.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-26T07:13:39.005 INFO:teuthology.orchestra.run.smithi031.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-26T07:13:39.009 INFO:teuthology.orchestra.run.smithi031.stdout:Get:76 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7484-g44c2237f-1jammy [3,468 kB] 2023-11-26T07:13:39.012 INFO:teuthology.orchestra.run.smithi031.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-26T07:13:39.012 INFO:teuthology.orchestra.run.smithi031.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-26T07:13:39.013 INFO:teuthology.orchestra.run.smithi031.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-26T07:13:39.015 INFO:teuthology.orchestra.run.smithi031.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-26T07:13:39.022 INFO:teuthology.orchestra.run.smithi031.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-26T07:13:39.084 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-11-26T07:13:39.097 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-11-26T07:13:39.148 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-11-26T07:13:39.323 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-11-26T07:13:39.332 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-11-26T07:13:39.359 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-11-26T07:13:39.367 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-11-26T07:13:39.532 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-11-26T07:13:40.029 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7484-g44c2237f-1jammy [3,112 kB] 2023-11-26T07:13:40.056 INFO:teuthology.orchestra.run.smithi031.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7484-g44c2237f-1jammy [3,112 kB] 2023-11-26T07:13:40.117 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-11-26T07:13:40.121 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-11-26T07:13:40.122 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-11-26T07:13:40.131 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-11-26T07:13:40.131 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-11-26T07:13:40.132 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-11-26T07:13:40.132 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-11-26T07:13:40.133 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-11-26T07:13:40.133 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-11-26T07:13:40.198 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-11-26T07:13:40.381 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-11-26T07:13:40.589 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-11-26T07:13:40.614 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7484-g44c2237f-1jammy [691 kB] 2023-11-26T07:13:40.623 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-11-26T07:13:40.654 INFO:teuthology.orchestra.run.smithi031.stdout:Get:83 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7484-g44c2237f-1jammy [691 kB] 2023-11-26T07:13:40.670 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-11-26T07:13:40.715 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7484-g44c2237f-1jammy [356 kB] 2023-11-26T07:13:40.767 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-11-26T07:13:40.778 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-11-26T07:13:40.785 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-11-26T07:13:40.796 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7484-g44c2237f-1jammy [32.8 kB] 2023-11-26T07:13:40.798 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7484-g44c2237f-1jammy [180 kB] 2023-11-26T07:13:40.818 INFO:teuthology.orchestra.run.smithi031.stdout:Get:84 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7484-g44c2237f-1jammy [356 kB] 2023-11-26T07:13:40.853 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-11-26T07:13:40.869 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7484-g44c2237f-1jammy [63.5 kB] 2023-11-26T07:13:40.873 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7484-g44c2237f-1jammy [330 kB] 2023-11-26T07:13:40.876 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-11-26T07:13:40.883 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-11-26T07:13:40.891 INFO:teuthology.orchestra.run.smithi097.stdout:Get:41 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7484-g44c2237f-1jammy [6,230 kB] 2023-11-26T07:13:40.902 INFO:teuthology.orchestra.run.smithi031.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7484-g44c2237f-1jammy [32.8 kB] 2023-11-26T07:13:40.903 INFO:teuthology.orchestra.run.smithi031.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7484-g44c2237f-1jammy [180 kB] 2023-11-26T07:13:40.914 INFO:teuthology.orchestra.run.smithi031.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7484-g44c2237f-1jammy [63.5 kB] 2023-11-26T07:13:40.917 INFO:teuthology.orchestra.run.smithi031.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7484-g44c2237f-1jammy [330 kB] 2023-11-26T07:13:41.000 INFO:teuthology.orchestra.run.smithi031.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7484-g44c2237f-1jammy [6,230 kB] 2023-11-26T07:13:41.001 INFO:teuthology.orchestra.run.smithi097.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-11-26T07:13:41.165 INFO:teuthology.orchestra.run.smithi097.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-11-26T07:13:41.352 INFO:teuthology.orchestra.run.smithi097.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-11-26T07:13:41.398 INFO:teuthology.orchestra.run.smithi097.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-11-26T07:13:41.562 INFO:teuthology.orchestra.run.smithi097.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-11-26T07:13:41.666 INFO:teuthology.orchestra.run.smithi097.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-11-26T07:13:41.685 INFO:teuthology.orchestra.run.smithi097.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-11-26T07:13:41.716 INFO:teuthology.orchestra.run.smithi097.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-11-26T07:13:41.730 INFO:teuthology.orchestra.run.smithi097.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-11-26T07:13:41.771 INFO:teuthology.orchestra.run.smithi097.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-11-26T07:13:41.779 INFO:teuthology.orchestra.run.smithi097.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-11-26T07:13:41.805 INFO:teuthology.orchestra.run.smithi097.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-11-26T07:13:41.994 INFO:teuthology.orchestra.run.smithi097.stdout:Get:54 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7484-g44c2237f-1jammy [112 kB] 2023-11-26T07:13:41.996 INFO:teuthology.orchestra.run.smithi097.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7484-g44c2237f-1jammy [439 kB] 2023-11-26T07:13:42.063 INFO:teuthology.orchestra.run.smithi097.stdout:Get:56 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7484-g44c2237f-1jammy [22.6 MB] 2023-11-26T07:13:42.111 INFO:teuthology.orchestra.run.smithi031.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7484-g44c2237f-1jammy [112 kB] 2023-11-26T07:13:42.164 INFO:teuthology.orchestra.run.smithi031.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7484-g44c2237f-1jammy [439 kB] 2023-11-26T07:13:42.192 INFO:teuthology.orchestra.run.smithi097.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-11-26T07:13:42.211 INFO:teuthology.orchestra.run.smithi031.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7484-g44c2237f-1jammy [22.6 MB] 2023-11-26T07:13:42.530 INFO:teuthology.orchestra.run.smithi097.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-11-26T07:13:42.617 INFO:teuthology.orchestra.run.smithi097.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-11-26T07:13:42.644 INFO:teuthology.orchestra.run.smithi097.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-11-26T07:13:42.658 INFO:teuthology.orchestra.run.smithi097.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-11-26T07:13:42.705 INFO:teuthology.orchestra.run.smithi097.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-11-26T07:13:42.956 INFO:teuthology.orchestra.run.smithi097.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-11-26T07:13:42.965 INFO:teuthology.orchestra.run.smithi097.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-11-26T07:13:42.966 INFO:teuthology.orchestra.run.smithi097.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-11-26T07:13:43.032 INFO:teuthology.orchestra.run.smithi097.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-11-26T07:13:43.032 INFO:teuthology.orchestra.run.smithi097.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-11-26T07:13:43.197 INFO:teuthology.orchestra.run.smithi097.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-11-26T07:13:43.399 INFO:teuthology.orchestra.run.smithi097.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-11-26T07:13:43.454 INFO:teuthology.orchestra.run.smithi097.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-11-26T07:13:43.542 INFO:teuthology.orchestra.run.smithi097.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-11-26T07:13:43.570 INFO:teuthology.orchestra.run.smithi097.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-11-26T07:13:43.603 INFO:teuthology.orchestra.run.smithi097.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-11-26T07:13:43.764 INFO:teuthology.orchestra.run.smithi097.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-11-26T07:13:43.800 INFO:teuthology.orchestra.run.smithi097.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-11-26T07:13:43.822 INFO:teuthology.orchestra.run.smithi097.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-11-26T07:13:43.829 INFO:teuthology.orchestra.run.smithi097.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-11-26T07:13:43.870 INFO:teuthology.orchestra.run.smithi097.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-11-26T07:13:44.058 INFO:teuthology.orchestra.run.smithi097.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-11-26T07:13:44.248 INFO:teuthology.orchestra.run.smithi097.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-11-26T07:13:44.631 INFO:teuthology.orchestra.run.smithi097.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-11-26T07:13:44.691 INFO:teuthology.orchestra.run.smithi097.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-11-26T07:13:44.692 INFO:teuthology.orchestra.run.smithi097.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-11-26T07:13:44.700 INFO:teuthology.orchestra.run.smithi097.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-11-26T07:13:44.702 INFO:teuthology.orchestra.run.smithi097.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-11-26T07:13:44.721 INFO:teuthology.orchestra.run.smithi097.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-11-26T07:13:44.729 INFO:teuthology.orchestra.run.smithi097.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-11-26T07:13:44.791 INFO:teuthology.orchestra.run.smithi097.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-11-26T07:13:44.792 INFO:teuthology.orchestra.run.smithi097.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-11-26T07:13:44.871 INFO:teuthology.orchestra.run.smithi097.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7484-g44c2237f-1jammy [6,900 kB] 2023-11-26T07:13:44.956 INFO:teuthology.orchestra.run.smithi097.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-11-26T07:13:45.408 INFO:teuthology.orchestra.run.smithi097.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-11-26T07:13:45.424 INFO:teuthology.orchestra.run.smithi097.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-11-26T07:13:45.670 INFO:teuthology.orchestra.run.smithi031.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7484-g44c2237f-1jammy [6,900 kB] 2023-11-26T07:13:45.773 INFO:teuthology.orchestra.run.smithi097.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7484-g44c2237f-1jammy [238 kB] 2023-11-26T07:13:45.829 INFO:teuthology.orchestra.run.smithi097.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7484-g44c2237f-1jammy [124 kB] 2023-11-26T07:13:45.838 INFO:teuthology.orchestra.run.smithi097.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7484-g44c2237f-1jammy [1,496 kB] 2023-11-26T07:13:46.070 INFO:teuthology.orchestra.run.smithi097.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7484-g44c2237f-1jammy [9,955 kB] 2023-11-26T07:13:47.013 INFO:teuthology.orchestra.run.smithi031.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7484-g44c2237f-1jammy [238 kB] 2023-11-26T07:13:47.056 INFO:teuthology.orchestra.run.smithi031.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7484-g44c2237f-1jammy [124 kB] 2023-11-26T07:13:47.108 INFO:teuthology.orchestra.run.smithi031.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7484-g44c2237f-1jammy [1,496 kB] 2023-11-26T07:13:47.360 INFO:teuthology.orchestra.run.smithi031.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7484-g44c2237f-1jammy [9,955 kB] 2023-11-26T07:13:47.748 INFO:teuthology.orchestra.run.smithi097.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7484-g44c2237f-1jammy [31.8 MB] 2023-11-26T07:13:49.243 INFO:teuthology.orchestra.run.smithi031.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7484-g44c2237f-1jammy [31.8 MB] 2023-11-26T07:13:52.883 INFO:teuthology.orchestra.run.smithi097.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7484-g44c2237f-1jammy [14.0 kB] 2023-11-26T07:13:52.883 INFO:teuthology.orchestra.run.smithi097.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7484-g44c2237f-1jammy [839 kB] 2023-11-26T07:13:53.007 INFO:teuthology.orchestra.run.smithi097.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7484-g44c2237f-1jammy [2,144 kB] 2023-11-26T07:13:53.351 INFO:teuthology.orchestra.run.smithi097.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7484-g44c2237f-1jammy [805 kB] 2023-11-26T07:13:53.495 INFO:teuthology.orchestra.run.smithi097.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7484-g44c2237f-1jammy [139 kB] 2023-11-26T07:13:53.502 INFO:teuthology.orchestra.run.smithi097.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7484-g44c2237f-1jammy [1,900 kB] 2023-11-26T07:13:53.787 INFO:teuthology.orchestra.run.smithi097.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7484-g44c2237f-1jammy [8,625 kB] 2023-11-26T07:13:54.887 INFO:teuthology.orchestra.run.smithi031.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7484-g44c2237f-1jammy [14.0 kB] 2023-11-26T07:13:54.902 INFO:teuthology.orchestra.run.smithi031.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7484-g44c2237f-1jammy [839 kB] 2023-11-26T07:13:55.028 INFO:teuthology.orchestra.run.smithi097.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7484-g44c2237f-1jammy [14.2 kB] 2023-11-26T07:13:55.029 INFO:teuthology.orchestra.run.smithi097.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7484-g44c2237f-1jammy [48.7 MB] 2023-11-26T07:13:55.047 INFO:teuthology.orchestra.run.smithi031.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7484-g44c2237f-1jammy [2,144 kB] 2023-11-26T07:13:55.483 INFO:teuthology.orchestra.run.smithi031.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7484-g44c2237f-1jammy [805 kB] 2023-11-26T07:13:55.652 INFO:teuthology.orchestra.run.smithi031.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7484-g44c2237f-1jammy [139 kB] 2023-11-26T07:13:55.663 INFO:teuthology.orchestra.run.smithi031.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7484-g44c2237f-1jammy [1,900 kB] 2023-11-26T07:13:56.027 INFO:teuthology.orchestra.run.smithi031.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7484-g44c2237f-1jammy [8,625 kB] 2023-11-26T07:13:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7484-g44c2237f-1jammy [14.2 kB] 2023-11-26T07:13:57.681 INFO:teuthology.orchestra.run.smithi031.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7484-g44c2237f-1jammy [48.7 MB] 2023-11-26T07:14:03.641 INFO:teuthology.orchestra.run.smithi097.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7484-g44c2237f-1jammy [121 kB] 2023-11-26T07:14:03.659 INFO:teuthology.orchestra.run.smithi097.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-7484-g44c2237f-1jammy [225 kB] 2023-11-26T07:14:03.715 INFO:teuthology.orchestra.run.smithi097.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-7484-g44c2237f-1jammy [34.0 kB] 2023-11-26T07:14:03.729 INFO:teuthology.orchestra.run.smithi097.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-7484-g44c2237f-1jammy [23.4 kB] 2023-11-26T07:14:03.731 INFO:teuthology.orchestra.run.smithi097.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7484-g44c2237f-1jammy [38.3 kB] 2023-11-26T07:14:03.732 INFO:teuthology.orchestra.run.smithi097.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7484-g44c2237f-1jammy [12.3 MB] 2023-11-26T07:14:06.839 INFO:teuthology.orchestra.run.smithi031.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7484-g44c2237f-1jammy [121 kB] 2023-11-26T07:14:06.854 INFO:teuthology.orchestra.run.smithi031.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-7484-g44c2237f-1jammy [225 kB] 2023-11-26T07:14:06.907 INFO:teuthology.orchestra.run.smithi031.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-7484-g44c2237f-1jammy [34.0 kB] 2023-11-26T07:14:06.924 INFO:teuthology.orchestra.run.smithi031.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-7484-g44c2237f-1jammy [23.4 kB] 2023-11-26T07:14:06.925 INFO:teuthology.orchestra.run.smithi031.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7484-g44c2237f-1jammy [38.3 kB] 2023-11-26T07:14:06.926 INFO:teuthology.orchestra.run.smithi031.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7484-g44c2237f-1jammy [12.3 MB] 2023-11-26T07:14:07.196 INFO:teuthology.orchestra.run.smithi097.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7484-g44c2237f-1jammy [91.2 kB] 2023-11-26T07:14:07.777 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 186 MB in 29s (6,467 kB/s) 2023-11-26T07:14:07.920 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-26T07:14:07.989 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-11-26T07:14:07.993 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-26T07:14:08.044 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-26T07:14:08.380 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-26T07:14:08.393 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-26T07:14:08.430 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-26T07:14:08.900 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cmd2. 2023-11-26T07:14:08.913 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-26T07:14:08.958 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-26T07:14:09.311 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-26T07:14:09.321 INFO:teuthology.orchestra.run.smithi031.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/44c2237fe77713214148c1fd622c872f346f1b99/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7484-g44c2237f-1jammy [91.2 kB] 2023-11-26T07:14:09.325 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-26T07:14:09.446 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-26T07:14:09.790 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-26T07:14:09.804 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-26T07:14:09.848 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-26T07:14:09.928 INFO:teuthology.orchestra.run.smithi031.stdout:Fetched 186 MB in 31s (6,028 kB/s) 2023-11-26T07:14:10.091 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyperclip. 2023-11-26T07:14:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-11-26T07:14:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-11-26T07:14:10.193 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-26T07:14:10.206 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-11-26T07:14:10.207 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-26T07:14:10.251 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-26T07:14:10.492 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-wcwidth. 2023-11-26T07:14:10.506 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-11-26T07:14:10.576 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-26T07:14:10.730 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-26T07:14:10.744 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:10.895 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cmd2. 2023-11-26T07:14:10.909 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-11-26T07:14:10.953 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-11-26T07:14:11.183 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-26T07:14:11.197 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:11.232 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:11.569 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-26T07:14:11.583 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:11.627 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:11.955 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-26T07:14:11.970 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-26T07:14:12.013 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-26T07:14:12.040 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-11-26T07:14:12.056 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-11-26T07:14:12.139 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-26T07:14:12.383 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librados2. 2023-11-26T07:14:12.397 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../010-librados2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:12.441 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librados2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:12.509 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-11-26T07:14:12.525 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-11-26T07:14:12.568 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-26T07:14:12.844 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librbd1. 2023-11-26T07:14:12.858 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../011-librbd1_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:12.870 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-11-26T07:14:12.886 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-11-26T07:14:12.902 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librbd1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:12.929 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-26T07:14:13.299 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-11-26T07:14:13.313 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libcephfs2. 2023-11-26T07:14:13.314 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:13.328 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:13.365 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:13.371 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libcephfs2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:13.690 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rados. 2023-11-26T07:14:13.705 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../013-python3-rados_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:13.749 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rados (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:13.776 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-11-26T07:14:13.792 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:13.835 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:14.026 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-26T07:14:14.041 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:14.085 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-ceph-argparse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:14.205 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-11-26T07:14:14.220 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-11-26T07:14:14.254 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:14:14.362 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cephfs. 2023-11-26T07:14:14.377 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:14.421 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cephfs (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:14.698 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-26T07:14:14.713 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:14.748 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:14.807 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-11-26T07:14:14.822 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-11-26T07:14:14.865 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-26T07:14:15.026 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-prettytable. 2023-11-26T07:14:15.041 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2023-11-26T07:14:15.086 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-26T07:14:15.143 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librados2. 2023-11-26T07:14:15.158 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../010-librados2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:15.201 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librados2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:15.339 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rbd. 2023-11-26T07:14:15.353 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:15.397 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rbd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:15.604 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librbd1. 2023-11-26T07:14:15.619 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../011-librbd1_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:15.654 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librbd1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:15.775 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-26T07:14:15.790 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-26T07:14:15.833 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-26T07:14:16.065 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libcephfs2. 2023-11-26T07:14:16.080 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:16.111 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package librgw2. 2023-11-26T07:14:16.123 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libcephfs2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:16.125 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../020-librgw2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:16.186 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking librgw2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:16.417 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rados. 2023-11-26T07:14:16.432 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../013-python3-rados_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:16.475 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rados (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:16.622 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rgw. 2023-11-26T07:14:16.636 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../021-python3-rgw_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:16.680 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:16.812 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-11-26T07:14:16.827 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:16.869 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-ceph-argparse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:17.041 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-26T07:14:17.056 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../022-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-26T07:14:17.099 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-26T07:14:17.147 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cephfs. 2023-11-26T07:14:17.163 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:17.205 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cephfs (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:17.385 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libradosstriper1. 2023-11-26T07:14:17.400 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../023-libradosstriper1_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:17.435 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libradosstriper1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:17.516 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-ceph-common. 2023-11-26T07:14:17.532 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:17.574 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:17.855 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-common. 2023-11-26T07:14:17.861 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-prettytable. 2023-11-26T07:14:17.870 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../024-ceph-common_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:17.876 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2023-11-26T07:14:17.913 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:17.919 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-11-26T07:14:18.205 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rbd. 2023-11-26T07:14:18.220 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:18.263 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rbd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:18.633 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-11-26T07:14:18.648 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-11-26T07:14:18.691 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-26T07:14:18.977 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librgw2. 2023-11-26T07:14:18.991 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../020-librgw2_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:19.026 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librgw2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:19.244 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-base. 2023-11-26T07:14:19.259 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../025-ceph-base_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:19.312 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-base (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:19.470 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rgw. 2023-11-26T07:14:19.485 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../021-python3-rgw_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:19.528 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:19.890 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package liboath0:amd64. 2023-11-26T07:14:19.904 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../022-liboath0_2.6.7-3build1_amd64.deb ... 2023-11-26T07:14:19.948 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-11-26T07:14:20.040 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-26T07:14:20.054 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../026-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-26T07:14:20.098 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-26T07:14:20.267 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libradosstriper1. 2023-11-26T07:14:20.283 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../023-libradosstriper1_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:20.326 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libradosstriper1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:20.359 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cheroot. 2023-11-26T07:14:20.374 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../027-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-26T07:14:20.442 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-26T07:14:20.662 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-common. 2023-11-26T07:14:20.677 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../024-ceph-common_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:20.729 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-26T07:14:20.737 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:20.744 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../028-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-26T07:14:20.795 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-26T07:14:21.131 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-26T07:14:21.146 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../029-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-26T07:14:21.189 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-26T07:14:21.475 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-26T07:14:21.490 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../030-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-26T07:14:21.534 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-26T07:14:21.801 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-base. 2023-11-26T07:14:21.816 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../025-ceph-base_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:21.819 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-tempora. 2023-11-26T07:14:21.835 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../031-python3-tempora_4.1.2-1_all.deb ... 2023-11-26T07:14:21.869 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-26T07:14:21.894 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-base (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:22.147 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-portend. 2023-11-26T07:14:22.163 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../032-python3-portend_3.0.0-1_all.deb ... 2023-11-26T07:14:22.205 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-26T07:14:22.525 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-26T07:14:22.541 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../033-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-26T07:14:22.583 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-26T07:14:22.596 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-11-26T07:14:22.610 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../026-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-11-26T07:14:22.654 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-11-26T07:14:22.886 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-26T07:14:22.901 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../034-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-26T07:14:22.924 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cheroot. 2023-11-26T07:14:22.938 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../027-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-11-26T07:14:22.944 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-26T07:14:22.982 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-26T07:14:23.251 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-11-26T07:14:23.267 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../028-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-11-26T07:14:23.281 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-natsort. 2023-11-26T07:14:23.295 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../035-python3-natsort_8.0.2-1_all.deb ... 2023-11-26T07:14:23.309 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-11-26T07:14:23.339 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-26T07:14:23.629 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.text. 2023-11-26T07:14:23.644 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../029-python3-jaraco.text_3.6.0-2_all.deb ... 2023-11-26T07:14:23.651 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-logutils. 2023-11-26T07:14:23.666 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-8_all.deb ... 2023-11-26T07:14:23.687 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-11-26T07:14:23.709 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-26T07:14:23.990 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-11-26T07:14:24.005 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../030-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-11-26T07:14:24.045 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-26T07:14:24.060 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../037-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-26T07:14:24.073 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-11-26T07:14:24.103 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-26T07:14:24.350 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-tempora. 2023-11-26T07:14:24.366 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../031-python3-tempora_4.1.2-1_all.deb ... 2023-11-26T07:14:24.400 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-11-26T07:14:24.406 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-mako. 2023-11-26T07:14:24.421 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../038-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-26T07:14:24.489 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-26T07:14:24.801 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-26T07:14:24.816 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-26T07:14:24.859 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-26T07:14:24.870 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-portend. 2023-11-26T07:14:24.886 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../032-python3-portend_3.0.0-1_all.deb ... 2023-11-26T07:14:24.928 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-11-26T07:14:25.178 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-26T07:14:25.194 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-26T07:14:25.236 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-26T07:14:25.264 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-11-26T07:14:25.280 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../033-python3-zc.lockfile_2.0-1_all.deb ... 2023-11-26T07:14:25.322 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-11-26T07:14:25.573 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-webob. 2023-11-26T07:14:25.588 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-26T07:14:25.631 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-26T07:14:25.692 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cherrypy3. 2023-11-26T07:14:25.707 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../034-python3-cherrypy3_18.6.1-4_all.deb ... 2023-11-26T07:14:25.750 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-11-26T07:14:25.992 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-waitress. 2023-11-26T07:14:26.008 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../042-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-26T07:14:26.053 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-26T07:14:26.095 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-natsort. 2023-11-26T07:14:26.111 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../035-python3-natsort_8.0.2-1_all.deb ... 2023-11-26T07:14:26.153 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-11-26T07:14:26.362 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-tempita. 2023-11-26T07:14:26.377 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../043-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-26T07:14:26.420 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-26T07:14:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-logutils. 2023-11-26T07:14:26.513 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../036-python3-logutils_0.3.3-8_all.deb ... 2023-11-26T07:14:26.556 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-11-26T07:14:26.739 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-paste. 2023-11-26T07:14:26.755 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../044-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-26T07:14:26.789 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-26T07:14:26.867 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-markupsafe. 2023-11-26T07:14:26.882 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../037-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-11-26T07:14:26.917 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-11-26T07:14:27.133 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-26T07:14:27.148 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../045-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-26T07:14:27.186 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-mako. 2023-11-26T07:14:27.200 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-26T07:14:27.202 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../038-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-11-26T07:14:27.245 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-26T07:14:27.487 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-26T07:14:27.502 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../046-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-26T07:14:27.536 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-26T07:14:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-simplegeneric. 2023-11-26T07:14:27.613 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../039-python3-simplegeneric_0.8.1-3_all.deb ... 2023-11-26T07:14:27.657 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-11-26T07:14:27.865 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-webtest. 2023-11-26T07:14:27.880 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../047-python3-webtest_2.0.35-1_all.deb ... 2023-11-26T07:14:27.923 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-26T07:14:27.992 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-singledispatch. 2023-11-26T07:14:28.008 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../040-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-11-26T07:14:28.050 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-11-26T07:14:28.234 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pecan. 2023-11-26T07:14:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../048-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-26T07:14:28.292 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-26T07:14:28.336 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-webob. 2023-11-26T07:14:28.352 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../041-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-11-26T07:14:28.419 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-11-26T07:14:28.680 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-certifi. 2023-11-26T07:14:28.695 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../049-python3-certifi_2020.6.20-1_all.deb ... 2023-11-26T07:14:28.738 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-26T07:14:28.764 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-waitress. 2023-11-26T07:14:28.779 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../042-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-11-26T07:14:28.825 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-26T07:14:29.158 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-tempita. 2023-11-26T07:14:29.173 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../043-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-11-26T07:14:29.225 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-11-26T07:14:29.258 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-idna. 2023-11-26T07:14:29.273 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../050-python3-idna_3.3-1_all.deb ... 2023-11-26T07:14:29.316 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-26T07:14:29.552 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-paste. 2023-11-26T07:14:29.568 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../044-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-11-26T07:14:29.602 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-urllib3. 2023-11-26T07:14:29.610 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-11-26T07:14:29.617 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../051-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-11-26T07:14:29.652 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-26T07:14:29.906 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-11-26T07:14:29.920 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../045-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-11-26T07:14:29.955 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-11-26T07:14:29.980 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-requests. 2023-11-26T07:14:29.995 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../052-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-26T07:14:30.030 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-26T07:14:30.274 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-26T07:14:30.290 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../053-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-26T07:14:30.300 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pastedeploy. 2023-11-26T07:14:30.315 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../046-python3-pastedeploy_2.1.1-1_all.deb ... 2023-11-26T07:14:30.324 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-26T07:14:30.358 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-11-26T07:14:30.652 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-26T07:14:30.667 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../054-ceph-mgr-modules-core_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:30.685 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-webtest. 2023-11-26T07:14:30.701 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../047-python3-webtest_2.0.35-1_all.deb ... 2023-11-26T07:14:30.710 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:30.744 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-11-26T07:14:31.048 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-26T07:14:31.063 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../055-libsqlite3-mod-ceph_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:31.063 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pecan. 2023-11-26T07:14:31.079 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../048-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-11-26T07:14:31.098 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:31.146 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-11-26T07:14:31.376 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-openssl. 2023-11-26T07:14:31.390 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2023-11-26T07:14:31.434 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-26T07:14:31.509 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-certifi. 2023-11-26T07:14:31.524 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../049-python3-certifi_2020.6.20-1_all.deb ... 2023-11-26T07:14:31.567 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-11-26T07:14:31.762 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr. 2023-11-26T07:14:31.776 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../057-ceph-mgr_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:31.820 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:31.920 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-idna. 2023-11-26T07:14:31.935 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../050-python3-idna_3.3-1_all.deb ... 2023-11-26T07:14:31.978 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-idna (3.3-1) ... 2023-11-26T07:14:32.223 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mon. 2023-11-26T07:14:32.238 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../058-ceph-mon_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:32.264 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-urllib3. 2023-11-26T07:14:32.280 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../051-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-11-26T07:14:32.281 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mon (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:32.314 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-26T07:14:32.609 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-requests. 2023-11-26T07:14:32.624 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../052-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-11-26T07:14:32.659 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-26T07:14:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-werkzeug. 2023-11-26T07:14:33.010 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../053-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-11-26T07:14:33.053 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-26T07:14:33.102 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-osd. 2023-11-26T07:14:33.117 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../059-ceph-osd_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:33.159 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-osd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:33.372 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-11-26T07:14:33.387 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../054-ceph-mgr-modules-core_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:33.422 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:33.785 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-11-26T07:14:33.799 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../055-libsqlite3-mod-ceph_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:33.868 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:34.179 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-openssl. 2023-11-26T07:14:34.193 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2023-11-26T07:14:34.237 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-11-26T07:14:34.355 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph. 2023-11-26T07:14:34.370 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../060-ceph_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:34.404 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:34.557 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr. 2023-11-26T07:14:34.572 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../057-ceph-mgr_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:34.615 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:34.749 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-fuse. 2023-11-26T07:14:34.764 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../061-ceph-fuse_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:34.807 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:35.026 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mon. 2023-11-26T07:14:35.042 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../058-ceph-mon_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:35.085 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mon (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:35.168 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mds. 2023-11-26T07:14:35.184 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../062-ceph-mds_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:35.226 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mds (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:35.630 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephadm. 2023-11-26T07:14:35.645 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../063-cephadm_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:35.680 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:35.871 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-osd. 2023-11-26T07:14:35.886 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../059-ceph-osd_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:35.929 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-osd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:35.991 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-26T07:14:36.007 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-26T07:14:36.049 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-26T07:14:36.344 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-26T07:14:36.358 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-26T07:14:36.393 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-26T07:14:37.150 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph. 2023-11-26T07:14:37.165 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../060-ceph_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:37.207 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:37.532 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-babel. 2023-11-26T07:14:37.545 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-26T07:14:37.552 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-fuse. 2023-11-26T07:14:37.567 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../061-ceph-fuse_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:37.590 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-26T07:14:37.610 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:37.884 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-jinja2. 2023-11-26T07:14:37.898 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1_all.deb ... 2023-11-26T07:14:37.942 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-26T07:14:37.997 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mds. 2023-11-26T07:14:38.012 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../062-ceph-mds_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:38.063 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mds (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:38.304 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-26T07:14:38.318 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:38.362 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:38.424 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package cephadm. 2023-11-26T07:14:38.440 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../063-cephadm_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:38.474 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:38.732 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-26T07:14:38.747 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../069-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-26T07:14:38.785 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-asyncssh. 2023-11-26T07:14:38.799 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-26T07:14:38.801 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2023-11-26T07:14:38.843 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-11-26T07:14:39.102 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-26T07:14:39.116 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../070-python3-repoze.lru_0.7-2_all.deb ... 2023-11-26T07:14:39.138 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python-babel-localedata. 2023-11-26T07:14:39.154 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-11-26T07:14:39.160 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-26T07:14:39.195 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-26T07:14:39.680 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-routes. 2023-11-26T07:14:39.695 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../071-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-26T07:14:39.738 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-26T07:14:40.141 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-26T07:14:40.156 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:40.199 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:40.209 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-babel. 2023-11-26T07:14:40.223 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-11-26T07:14:40.267 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-11-26T07:14:40.620 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jinja2. 2023-11-26T07:14:40.627 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-26T07:14:40.634 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1_all.deb ... 2023-11-26T07:14:40.642 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../073-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-26T07:14:40.678 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-11-26T07:14:40.711 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-26T07:14:41.031 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-11-26T07:14:41.046 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:41.089 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:41.181 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-joblib. 2023-11-26T07:14:41.196 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../074-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-26T07:14:41.239 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-26T07:14:41.434 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-bcrypt. 2023-11-26T07:14:41.449 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../069-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-11-26T07:14:41.484 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-11-26T07:14:41.618 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-26T07:14:41.633 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../075-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-26T07:14:41.676 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-26T07:14:41.762 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-repoze.lru. 2023-11-26T07:14:41.778 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../070-python3-repoze.lru_0.7-2_all.deb ... 2023-11-26T07:14:41.820 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-11-26T07:14:41.970 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-sklearn. 2023-11-26T07:14:41.985 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../076-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-26T07:14:42.059 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-26T07:14:42.131 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-routes. 2023-11-26T07:14:42.147 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../071-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-11-26T07:14:42.233 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-11-26T07:14:42.576 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-11-26T07:14:42.592 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:42.634 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:42.745 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-26T07:14:42.759 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:42.803 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:43.062 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-11-26T07:14:43.077 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../073-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-11-26T07:14:43.120 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-26T07:14:43.591 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-joblib. 2023-11-26T07:14:43.606 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../074-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-11-26T07:14:43.640 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-11-26T07:14:43.740 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-cachetools. 2023-11-26T07:14:43.755 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../078-python3-cachetools_5.0.0-1_all.deb ... 2023-11-26T07:14:43.798 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-26T07:14:44.019 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-11-26T07:14:44.034 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../075-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-11-26T07:14:44.077 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-11-26T07:14:44.101 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-26T07:14:44.117 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-26T07:14:44.200 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-26T07:14:44.379 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-sklearn. 2023-11-26T07:14:44.394 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../076-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-11-26T07:14:44.471 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-26T07:14:44.545 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-26T07:14:44.561 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-26T07:14:44.604 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-26T07:14:44.931 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-rsa. 2023-11-26T07:14:44.947 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../081-python3-rsa_4.8-1_all.deb ... 2023-11-26T07:14:44.989 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-26T07:14:45.171 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-11-26T07:14:45.186 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:45.229 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:45.435 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-google-auth. 2023-11-26T07:14:45.451 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-3_all.deb ... 2023-11-26T07:14:45.518 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-26T07:14:45.931 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-26T07:14:45.947 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../083-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-26T07:14:45.989 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-26T07:14:46.257 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cachetools. 2023-11-26T07:14:46.273 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../078-python3-cachetools_5.0.0-1_all.deb ... 2023-11-26T07:14:46.307 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-11-26T07:14:46.334 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-websocket. 2023-11-26T07:14:46.349 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../084-python3-websocket_1.2.3-1_all.deb ... 2023-11-26T07:14:46.392 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-26T07:14:46.585 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyasn1. 2023-11-26T07:14:46.601 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.8-1_all.deb ... 2023-11-26T07:14:46.668 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-11-26T07:14:46.695 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-26T07:14:46.711 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../085-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-26T07:14:46.753 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-26T07:14:46.996 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-11-26T07:14:47.012 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-11-26T07:14:47.054 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-11-26T07:14:47.365 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rsa. 2023-11-26T07:14:47.381 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../081-python3-rsa_4.8-1_all.deb ... 2023-11-26T07:14:47.383 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-26T07:14:47.397 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../086-ceph-mgr-k8sevents_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:47.423 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rsa (4.8-1) ... 2023-11-26T07:14:47.441 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:47.744 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-26T07:14:47.752 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-google-auth. 2023-11-26T07:14:47.758 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../087-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-26T07:14:47.768 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-3_all.deb ... 2023-11-26T07:14:47.802 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-26T07:14:47.809 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-11-26T07:14:48.106 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-26T07:14:48.120 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../088-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-26T07:14:48.183 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-11-26T07:14:48.199 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../083-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-11-26T07:14:48.205 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-26T07:14:48.286 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-26T07:14:48.441 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package jq. 2023-11-26T07:14:48.456 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../089-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-26T07:14:48.499 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-26T07:14:48.603 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-websocket. 2023-11-26T07:14:48.618 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../084-python3-websocket_1.2.3-1_all.deb ... 2023-11-26T07:14:48.652 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-11-26T07:14:48.752 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package socat. 2023-11-26T07:14:48.767 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../090-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-26T07:14:48.810 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-26T07:14:48.973 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-kubernetes. 2023-11-26T07:14:48.989 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../085-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-11-26T07:14:49.022 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-26T07:14:49.113 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package xmlstarlet. 2023-11-26T07:14:49.129 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../091-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-26T07:14:49.171 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-26T07:14:49.686 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-11-26T07:14:49.701 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../086-ceph-mgr-k8sevents_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:49.744 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:49.776 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-test. 2023-11-26T07:14:49.791 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../092-ceph-test_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:49.825 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-test (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:50.064 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libonig5:amd64. 2023-11-26T07:14:50.078 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../087-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-11-26T07:14:50.114 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-26T07:14:50.484 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libjq1:amd64. 2023-11-26T07:14:50.498 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../088-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-11-26T07:14:50.542 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-26T07:14:50.803 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package jq. 2023-11-26T07:14:50.817 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../089-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-11-26T07:14:50.861 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-11-26T07:14:51.122 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package socat. 2023-11-26T07:14:51.137 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../090-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-11-26T07:14:51.172 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-11-26T07:14:51.484 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package xmlstarlet. 2023-11-26T07:14:51.500 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../091-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-11-26T07:14:51.567 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-11-26T07:14:51.712 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ceph-volume. 2023-11-26T07:14:51.728 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../093-ceph-volume_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:51.771 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ceph-volume (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:51.921 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-test. 2023-11-26T07:14:51.936 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../092-ceph-test_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:51.971 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-test (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:52.167 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-26T07:14:52.182 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../094-cephfs-mirror_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:52.225 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephfs-mirror (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:52.570 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-colorama. 2023-11-26T07:14:52.585 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../095-python3-colorama_0.4.4-1_all.deb ... 2023-11-26T07:14:52.619 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-26T07:14:52.847 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephfs-shell. 2023-11-26T07:14:52.862 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../096-cephfs-shell_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:52.914 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephfs-shell (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:53.208 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package cephfs-top. 2023-11-26T07:14:53.223 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../097-cephfs-top_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:53.266 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking cephfs-top (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:53.519 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-26T07:14:53.534 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../098-libcephfs-dev_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:53.577 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libcephfs-dev (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:53.791 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-volume. 2023-11-26T07:14:53.808 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../093-ceph-volume_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:53.850 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-volume (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:53.963 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package nvme-cli. 2023-11-26T07:14:53.979 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-26T07:14:54.013 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-26T07:14:54.212 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package cephfs-mirror. 2023-11-26T07:14:54.228 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../094-cephfs-mirror_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:54.271 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking cephfs-mirror (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:54.336 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-26T07:14:54.351 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-26T07:14:54.403 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-26T07:14:54.574 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-colorama. 2023-11-26T07:14:54.590 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../095-python3-colorama_0.4.4-1_all.deb ... 2023-11-26T07:14:54.656 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-11-26T07:14:54.764 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-26T07:14:54.778 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../101-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-26T07:14:54.822 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-26T07:14:54.910 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package cephfs-shell. 2023-11-26T07:14:54.925 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../096-cephfs-shell_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:54.968 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking cephfs-shell (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:55.159 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pastescript. 2023-11-26T07:14:55.173 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../102-python3-pastescript_2.0.2-4_all.deb ... 2023-11-26T07:14:55.217 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-26T07:14:55.229 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package cephfs-top. 2023-11-26T07:14:55.245 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../097-cephfs-top_18.0.0-7484-g44c2237f-1jammy_all.deb ... 2023-11-26T07:14:55.279 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking cephfs-top (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:55.520 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pluggy. 2023-11-26T07:14:55.535 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../103-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-26T07:14:55.578 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-26T07:14:55.782 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libcephfs-dev. 2023-11-26T07:14:55.798 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../098-libcephfs-dev_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:55.840 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libcephfs-dev (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:55.922 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-psutil. 2023-11-26T07:14:55.938 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../104-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-26T07:14:55.980 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-26T07:14:56.327 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package nvme-cli. 2023-11-26T07:14:56.343 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-11-26T07:14:56.383 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-py. 2023-11-26T07:14:56.385 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-11-26T07:14:56.399 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../105-python3-py_1.10.0-1_all.deb ... 2023-11-26T07:14:56.450 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-26T07:14:56.723 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-11-26T07:14:56.738 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-11-26T07:14:56.778 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pygments. 2023-11-26T07:14:56.794 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../106-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-26T07:14:56.812 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-11-26T07:14:56.837 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-26T07:14:57.201 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-iniconfig. 2023-11-26T07:14:57.216 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../101-python3-iniconfig_1.1.1-2_all.deb ... 2023-11-26T07:14:57.259 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-11-26T07:14:57.290 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-26T07:14:57.305 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-26T07:14:57.340 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-26T07:14:57.595 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pastescript. 2023-11-26T07:14:57.611 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../102-python3-pastescript_2.0.2-4_all.deb ... 2023-11-26T07:14:57.626 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-toml. 2023-11-26T07:14:57.642 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../108-python3-toml_0.10.2-1_all.deb ... 2023-11-26T07:14:57.653 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-11-26T07:14:57.684 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-26T07:14:57.995 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-pytest. 2023-11-26T07:14:58.011 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../109-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-26T07:14:58.015 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pluggy. 2023-11-26T07:14:58.031 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../103-python3-pluggy_0.13.0-7.1_all.deb ... 2023-11-26T07:14:58.053 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-26T07:14:58.073 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-11-26T07:14:58.348 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package python3-simplejson. 2023-11-26T07:14:58.364 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../110-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-26T07:14:58.368 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-psutil. 2023-11-26T07:14:58.384 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../104-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-11-26T07:14:58.440 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-26T07:14:58.460 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-11-26T07:14:58.684 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-26T07:14:58.700 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../111-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-26T07:14:58.742 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-26T07:14:58.804 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-py. 2023-11-26T07:14:58.821 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../105-python3-py_1.10.0-1_all.deb ... 2023-11-26T07:14:58.862 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-py (1.10.0-1) ... 2023-11-26T07:14:59.207 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pygments. 2023-11-26T07:14:59.224 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../106-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-11-26T07:14:59.274 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-11-26T07:14:59.396 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package radosgw. 2023-11-26T07:14:59.410 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../112-radosgw_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:14:59.454 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking radosgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:14:59.736 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyinotify. 2023-11-26T07:14:59.752 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-11-26T07:14:59.794 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-11-26T07:15:00.049 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package rbd-fuse. 2023-11-26T07:15:00.063 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../113-rbd-fuse_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:15:00.105 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-toml. 2023-11-26T07:15:00.107 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking rbd-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:00.121 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../108-python3-toml_0.10.2-1_all.deb ... 2023-11-26T07:15:00.163 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-11-26T07:15:00.365 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-26T07:15:00.433 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pytest. 2023-11-26T07:15:00.450 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../109-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-11-26T07:15:00.483 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-11-26T07:15:00.628 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-26T07:15:00.754 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-26T07:15:00.828 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-simplejson. 2023-11-26T07:15:00.845 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../110-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-11-26T07:15:00.886 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-11-26T07:15:01.012 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-26T07:15:01.181 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-11-26T07:15:01.197 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../111-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-11-26T07:15:01.239 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-11-26T07:15:01.421 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-26T07:15:01.952 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package radosgw. 2023-11-26T07:15:01.967 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../112-radosgw_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:15:01.998 INFO:teuthology.orchestra.run.smithi097.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-26T07:15:02.010 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking radosgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:02.011 INFO:teuthology.orchestra.run.smithi097.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-11-26T07:15:02.056 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:02.728 INFO:teuthology.orchestra.run.smithi097.stdout:Adding system user cephadm....done 2023-11-26T07:15:02.729 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package rbd-fuse. 2023-11-26T07:15:02.746 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../113-rbd-fuse_18.0.0-7484-g44c2237f-1jammy_amd64.deb ... 2023-11-26T07:15:02.779 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking rbd-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:03.083 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-11-26T07:15:03.238 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi031.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-11-26T07:15:08.168 INFO:teuthology.orchestra.run.smithi031.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Adding system user cephadm....done 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-26T07:15:08.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-26T07:15:08.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-11-26T07:15:08.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-11-26T07:15:08.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-11-26T07:15:08.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-11-26T07:15:08.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-11-26T07:15:08.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-py (1.10.0-1) ... 2023-11-26T07:15:08.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-11-26T07:15:08.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-11-26T07:15:08.170 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-26T07:15:08.193 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-26T07:15:08.193 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-26T07:15:08.193 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-26T07:15:08.193 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-26T07:15:08.193 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-ceph-argparse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:08.193 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-26T07:15:08.193 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-26T07:15:08.193 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-26T07:15:08.193 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-26T07:15:08.193 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-26T07:15:08.339 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-11-26T07:15:08.411 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-26T07:15:08.537 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-26T07:15:08.606 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-11-26T07:15:08.838 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-26T07:15:08.883 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-11-26T07:15:09.114 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-26T07:15:09.150 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-11-26T07:15:09.432 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-26T07:15:09.451 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-11-26T07:15:09.744 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-ceph-argparse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:10.020 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-11-26T07:15:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-11-26T07:15:10.251 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-26T07:15:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-11-26T07:15:10.510 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-26T07:15:10.644 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-26T07:15:10.682 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-11-26T07:15:10.770 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-26T07:15:10.808 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-11-26T07:15:11.079 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-26T07:15:11.201 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-11-26T07:15:11.301 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-11-26T07:15:11.481 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-26T07:15:11.578 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-11-26T07:15:11.740 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-26T07:15:11.862 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-11-26T07:15:12.134 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-26T07:15:12.188 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-11-26T07:15:12.435 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-26T07:15:12.752 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-26T07:15:12.924 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-11-26T07:15:13.112 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-26T07:15:13.192 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-11-26T07:15:13.318 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-11-26T07:15:13.363 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-26T07:15:13.469 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-11-26T07:15:13.489 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:13.790 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-idna (3.3-1) ... 2023-11-26T07:15:13.795 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-11-26T07:15:14.133 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-26T07:15:14.237 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-11-26T07:15:14.489 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-11-26T07:15:14.882 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-11-26T07:15:15.225 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-11-26T07:15:15.372 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:15.506 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-26T07:15:15.560 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-11-26T07:15:15.774 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-26T07:15:15.908 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-26T07:15:15.952 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-11-26T07:15:16.067 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-26T07:15:16.220 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-11-26T07:15:16.346 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:16.369 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-26T07:15:16.636 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-26T07:15:16.664 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-idna (3.3-1) ... 2023-11-26T07:15:16.921 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:17.030 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-26T07:15:17.032 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-11-26T07:15:17.373 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-26T07:15:17.632 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-26T07:15:17.925 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-26T07:15:18.218 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-26T07:15:18.321 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:18.469 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-26T07:15:18.472 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-toml (0.10.2-1) ... 2023-11-26T07:15:18.745 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-26T07:15:18.748 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-11-26T07:15:18.871 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-26T07:15:18.924 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-11-26T07:15:19.075 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-11-26T07:15:19.164 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-26T07:15:19.289 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-26T07:15:19.402 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-11-26T07:15:19.624 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-26T07:15:19.669 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-11-26T07:15:19.934 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-26T07:15:19.937 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:20.063 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-11-26T07:15:20.235 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-26T07:15:20.390 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-11-26T07:15:20.669 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-26T07:15:20.699 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-11-26T07:15:20.937 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-26T07:15:20.992 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-11-26T07:15:21.063 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-26T07:15:21.276 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-11-26T07:15:21.465 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-26T07:15:21.553 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-11-26T07:15:21.741 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-26T07:15:21.837 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-11-26T07:15:21.938 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-11-26T07:15:22.057 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:22.176 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-26T07:15:22.231 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-11-26T07:15:22.357 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-11-26T07:15:22.486 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-26T07:15:22.650 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-11-26T07:15:22.795 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-26T07:15:22.976 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-11-26T07:15:23.079 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-26T07:15:23.205 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-26T07:15:23.244 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-11-26T07:15:23.482 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-26T07:15:23.649 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-11-26T07:15:23.766 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-26T07:15:23.930 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-11-26T07:15:24.056 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-11-26T07:15:24.075 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-26T07:15:24.460 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-26T07:15:24.466 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-11-26T07:15:24.743 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-portend (3.0.0-1) ... 2023-11-26T07:15:24.770 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-26T07:15:25.034 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-11-26T07:15:25.130 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-26T07:15:25.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-11-26T07:15:25.231 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librados2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:25.503 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-11-26T07:15:25.574 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librgw2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:25.700 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:25.804 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-11-26T07:15:25.903 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libcephfs2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:26.029 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libradosstriper1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:26.171 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-26T07:15:26.439 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-11-26T07:15:26.473 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up librbd1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:26.565 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rsa (4.8-1) ... 2023-11-26T07:15:26.599 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-modules-core (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:26.734 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:26.850 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-11-26T07:15:26.976 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-26T07:15:26.976 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-26T07:15:27.167 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-11-26T07:15:27.444 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-11-26T07:15:27.618 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libcephfs-dev (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:27.744 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rados (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:27.870 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-11-26T07:15:27.871 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephfs-top (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:27.996 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-26T07:15:28.155 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-11-26T07:15:28.516 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-11-26T07:15:28.641 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librados2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:28.751 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librgw2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:28.877 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.080 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libcephfs2 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.223 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libradosstriper1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.285 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rbd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.340 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-11-26T07:15:29.385 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up rbd-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.553 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-rgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.626 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librbd1 (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.678 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up python3-cephfs (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.768 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-modules-core (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.804 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephfs-shell (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.894 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:29.930 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:30.116 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-26T07:15:30.116 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-11-26T07:15:30.509 INFO:teuthology.orchestra.run.smithi097.stdout:Adding group ceph....done 2023-11-26T07:15:30.721 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libcephfs-dev (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:30.872 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rados (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:30.885 INFO:teuthology.orchestra.run.smithi097.stdout:Adding system user ceph....done 2023-11-26T07:15:30.998 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up cephfs-top (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:31.045 INFO:teuthology.orchestra.run.smithi097.stdout:Setting system user ceph properties....done 2023-11-26T07:15:31.063 INFO:teuthology.orchestra.run.smithi097.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-26T07:15:31.116 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-11-26T07:15:31.204 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-26T07:15:31.575 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-26T07:15:32.248 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up cephfs-mirror (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:32.453 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rbd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:32.456 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-26T07:15:32.456 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-26T07:15:32.588 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up rbd-fuse (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:32.714 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:32.840 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cephfs (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:32.991 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up cephfs-shell (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:33.041 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-test (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:33.117 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-common (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:33.167 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up radosgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:33.696 INFO:teuthology.orchestra.run.smithi031.stdout:Adding group ceph....done 2023-11-26T07:15:33.696 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-26T07:15:33.696 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-26T07:15:34.063 INFO:teuthology.orchestra.run.smithi031.stdout:Adding system user ceph....done 2023-11-26T07:15:34.224 INFO:teuthology.orchestra.run.smithi031.stdout:Setting system user ceph properties....done 2023-11-26T07:15:34.246 INFO:teuthology.orchestra.run.smithi031.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-11-26T07:15:34.318 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-base (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:34.395 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-11-26T07:15:34.677 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-26T07:15:34.790 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-11-26T07:15:35.328 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mds (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:35.451 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up cephfs-mirror (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:35.569 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-26T07:15:35.569 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-26T07:15:35.669 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-26T07:15:35.669 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-11-26T07:15:36.206 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:36.261 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-test (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:36.395 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up radosgw (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:36.418 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-26T07:15:36.418 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-26T07:15:36.909 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-26T07:15:36.909 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-11-26T07:15:37.065 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-osd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:37.383 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-26T07:15:37.383 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-26T07:15:37.522 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-base (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:37.889 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-11-26T07:15:37.983 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:38.084 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:38.244 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mon (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:38.452 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-26T07:15:38.452 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-26T07:15:38.524 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mds (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:38.755 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-26T07:15:38.756 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-11-26T07:15:39.062 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:39.204 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:39.330 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-mgr-dashboard (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:39.409 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:39.473 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ceph-volume (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:39.654 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-26T07:15:39.655 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-11-26T07:15:39.851 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-26T07:15:40.269 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-osd (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:40.567 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-26T07:15:40.567 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-11-26T07:15:41.129 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:41.252 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-26T07:15:41.255 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:41.406 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mon (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:41.606 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-26T07:15:41.606 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-11-26T07:15:41.731 INFO:teuthology.orchestra.run.smithi097.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-26T07:15:41.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-26T07:15:41.810 INFO:teuthology.orchestra.run.smithi097.stdout:Reading package lists... 2023-11-26T07:15:41.976 INFO:teuthology.orchestra.run.smithi097.stdout:Building dependency tree... 2023-11-26T07:15:41.977 INFO:teuthology.orchestra.run.smithi097.stdout:Reading state information... 2023-11-26T07:15:42.128 INFO:teuthology.orchestra.run.smithi097.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-26T07:15:42.128 INFO:teuthology.orchestra.run.smithi097.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-26T07:15:42.129 INFO:teuthology.orchestra.run.smithi097.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-26T07:15:42.129 INFO:teuthology.orchestra.run.smithi097.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-26T07:15:42.129 INFO:teuthology.orchestra.run.smithi097.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-26T07:15:42.129 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages were automatically installed and are no longer required: 2023-11-26T07:15:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-26T07:15:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-26T07:15:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-26T07:15:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-26T07:15:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-26T07:15:42.130 INFO:teuthology.orchestra.run.smithi097.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-26T07:15:42.130 INFO:teuthology.orchestra.run.smithi097.stdout:The following additional packages will be installed: 2023-11-26T07:15:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-26T07:15:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-26T07:15:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-26T07:15:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-26T07:15:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-26T07:15:42.132 INFO:teuthology.orchestra.run.smithi097.stdout:Suggested packages: 2023-11-26T07:15:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-26T07:15:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-26T07:15:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: libengine-pkcs11-openssl 2023-11-26T07:15:42.181 INFO:teuthology.orchestra.run.smithi097.stdout:The following NEW packages will be installed: 2023-11-26T07:15:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-26T07:15:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-26T07:15:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-26T07:15:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-26T07:15:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-26T07:15:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-26T07:15:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-26T07:15:42.183 INFO:teuthology.orchestra.run.smithi097.stdout:The following packages will be upgraded: 2023-11-26T07:15:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: libssl3 2023-11-26T07:15:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-cephadm (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:42.244 INFO:teuthology.orchestra.run.smithi097.stdout:1 upgraded, 39 newly installed, 0 to remove and 263 not upgraded. 2023-11-26T07:15:42.244 INFO:teuthology.orchestra.run.smithi097.stdout:Need to get 54.6 MB of archives. 2023-11-26T07:15:42.245 INFO:teuthology.orchestra.run.smithi097.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-26T07:15:42.245 INFO:teuthology.orchestra.run.smithi097.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-26T07:15:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:42.365 INFO:teuthology.orchestra.run.smithi097.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-26T07:15:42.429 INFO:teuthology.orchestra.run.smithi097.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-26T07:15:42.437 INFO:teuthology.orchestra.run.smithi097.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-26T07:15:42.438 INFO:teuthology.orchestra.run.smithi097.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-26T07:15:42.439 INFO:teuthology.orchestra.run.smithi097.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-26T07:15:42.441 INFO:teuthology.orchestra.run.smithi097.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-26T07:15:42.441 INFO:teuthology.orchestra.run.smithi097.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-26T07:15:42.442 INFO:teuthology.orchestra.run.smithi097.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-26T07:15:42.443 INFO:teuthology.orchestra.run.smithi097.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-26T07:15:42.443 INFO:teuthology.orchestra.run.smithi097.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-26T07:15:42.449 INFO:teuthology.orchestra.run.smithi097.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-26T07:15:42.460 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-dashboard (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:42.463 INFO:teuthology.orchestra.run.smithi097.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-26T07:15:42.611 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-volume (18.0.0-7484-g44c2237f-1jammy) ... 2023-11-26T07:15:42.683 INFO:teuthology.orchestra.run.smithi097.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-26T07:15:42.686 INFO:teuthology.orchestra.run.smithi097.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-26T07:15:42.686 INFO:teuthology.orchestra.run.smithi097.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-26T07:15:42.689 INFO:teuthology.orchestra.run.smithi097.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-26T07:15:42.689 INFO:teuthology.orchestra.run.smithi097.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-26T07:15:42.694 INFO:teuthology.orchestra.run.smithi097.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-26T07:15:42.695 INFO:teuthology.orchestra.run.smithi097.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-26T07:15:42.703 INFO:teuthology.orchestra.run.smithi097.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-26T07:15:42.703 INFO:teuthology.orchestra.run.smithi097.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-26T07:15:42.724 INFO:teuthology.orchestra.run.smithi097.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-26T07:15:42.725 INFO:teuthology.orchestra.run.smithi097.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-26T07:15:42.735 INFO:teuthology.orchestra.run.smithi097.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-26T07:15:42.756 INFO:teuthology.orchestra.run.smithi097.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-26T07:15:42.759 INFO:teuthology.orchestra.run.smithi097.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-26T07:15:42.760 INFO:teuthology.orchestra.run.smithi097.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-26T07:15:42.761 INFO:teuthology.orchestra.run.smithi097.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-26T07:15:42.761 INFO:teuthology.orchestra.run.smithi097.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-26T07:15:42.773 INFO:teuthology.orchestra.run.smithi097.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-26T07:15:42.775 INFO:teuthology.orchestra.run.smithi097.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-26T07:15:42.923 INFO:teuthology.orchestra.run.smithi097.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-26T07:15:42.923 INFO:teuthology.orchestra.run.smithi097.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-26T07:15:42.923 INFO:teuthology.orchestra.run.smithi097.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-26T07:15:42.924 INFO:teuthology.orchestra.run.smithi097.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-26T07:15:42.924 INFO:teuthology.orchestra.run.smithi097.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-26T07:15:42.924 INFO:teuthology.orchestra.run.smithi097.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-26T07:15:42.937 INFO:teuthology.orchestra.run.smithi097.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-26T07:15:42.937 INFO:teuthology.orchestra.run.smithi097.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-26T07:15:42.972 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-26T07:15:43.315 INFO:teuthology.orchestra.run.smithi097.stdout:Preconfiguring packages ... 2023-11-26T07:15:43.467 INFO:teuthology.orchestra.run.smithi097.stdout:Fetched 54.6 MB in 1s (73.2 MB/s) 2023-11-26T07:15:43.642 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package flex. 2023-11-26T07:15:43.715 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139845 files and directories currently installed.) 2023-11-26T07:15:43.719 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-26T07:15:43.766 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-26T07:15:44.098 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-26T07:15:44.327 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-26T07:15:44.504 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-26T07:15:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-26T07:15:44.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-11-26T07:15:44.666 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2023-11-26T07:15:44.836 INFO:teuthology.orchestra.run.smithi031.stdout:Building dependency tree... 2023-11-26T07:15:44.837 INFO:teuthology.orchestra.run.smithi031.stdout:Reading state information... 2023-11-26T07:15:44.893 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:15:44.993 INFO:teuthology.orchestra.run.smithi031.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-11-26T07:15:44.993 INFO:teuthology.orchestra.run.smithi031.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-11-26T07:15:44.994 INFO:teuthology.orchestra.run.smithi031.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-11-26T07:15:44.994 INFO:teuthology.orchestra.run.smithi031.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-11-26T07:15:44.994 INFO:teuthology.orchestra.run.smithi031.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-11-26T07:15:44.994 INFO:teuthology.orchestra.run.smithi031.stdout:The following packages were automatically installed and are no longer required: 2023-11-26T07:15:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-11-26T07:15:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-11-26T07:15:44.994 INFO:teuthology.orchestra.run.smithi031.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-11-26T07:15:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-11-26T07:15:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-11-26T07:15:44.995 INFO:teuthology.orchestra.run.smithi031.stdout:Use 'sudo apt autoremove' to remove them. 2023-11-26T07:15:44.995 INFO:teuthology.orchestra.run.smithi031.stdout:The following additional packages will be installed: 2023-11-26T07:15:44.995 INFO:teuthology.orchestra.run.smithi031.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-11-26T07:15:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-11-26T07:15:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-11-26T07:15:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-11-26T07:15:44.996 INFO:teuthology.orchestra.run.smithi031.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-26T07:15:44.997 INFO:teuthology.orchestra.run.smithi031.stdout:Suggested packages: 2023-11-26T07:15:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-11-26T07:15:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-11-26T07:15:44.997 INFO:teuthology.orchestra.run.smithi031.stdout: libengine-pkcs11-openssl 2023-11-26T07:15:45.047 INFO:teuthology.orchestra.run.smithi031.stdout:The following NEW packages will be installed: 2023-11-26T07:15:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-11-26T07:15:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-11-26T07:15:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-11-26T07:15:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-11-26T07:15:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-11-26T07:15:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-11-26T07:15:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-11-26T07:15:45.050 INFO:teuthology.orchestra.run.smithi031.stdout:The following packages will be upgraded: 2023-11-26T07:15:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: libssl3 2023-11-26T07:15:45.105 INFO:teuthology.orchestra.run.smithi031.stdout:1 upgraded, 39 newly installed, 0 to remove and 263 not upgraded. 2023-11-26T07:15:45.105 INFO:teuthology.orchestra.run.smithi031.stdout:Need to get 54.6 MB of archives. 2023-11-26T07:15:45.105 INFO:teuthology.orchestra.run.smithi031.stdout:After this operation, 202 MB of additional disk space will be used. 2023-11-26T07:15:45.105 INFO:teuthology.orchestra.run.smithi031.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-11-26T07:15:45.206 INFO:teuthology.orchestra.run.smithi031.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-11-26T07:15:45.259 INFO:teuthology.orchestra.run.smithi031.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-11-26T07:15:45.267 INFO:teuthology.orchestra.run.smithi031.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-11-26T07:15:45.268 INFO:teuthology.orchestra.run.smithi031.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-11-26T07:15:45.269 INFO:teuthology.orchestra.run.smithi031.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-11-26T07:15:45.270 INFO:teuthology.orchestra.run.smithi031.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-11-26T07:15:45.271 INFO:teuthology.orchestra.run.smithi031.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-11-26T07:15:45.272 INFO:teuthology.orchestra.run.smithi031.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-11-26T07:15:45.273 INFO:teuthology.orchestra.run.smithi031.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-11-26T07:15:45.273 INFO:teuthology.orchestra.run.smithi031.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-11-26T07:15:45.279 INFO:teuthology.orchestra.run.smithi031.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-11-26T07:15:45.290 INFO:teuthology.orchestra.run.smithi031.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-11-26T07:15:45.455 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package bison. 2023-11-26T07:15:45.508 INFO:teuthology.orchestra.run.smithi031.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-11-26T07:15:45.510 INFO:teuthology.orchestra.run.smithi031.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-11-26T07:15:45.511 INFO:teuthology.orchestra.run.smithi031.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-11-26T07:15:45.514 INFO:teuthology.orchestra.run.smithi031.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-11-26T07:15:45.514 INFO:teuthology.orchestra.run.smithi031.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-11-26T07:15:45.518 INFO:teuthology.orchestra.run.smithi031.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-11-26T07:15:45.520 INFO:teuthology.orchestra.run.smithi031.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-11-26T07:15:45.527 INFO:teuthology.orchestra.run.smithi031.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-11-26T07:15:45.527 INFO:teuthology.orchestra.run.smithi031.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-11-26T07:15:45.528 INFO:teuthology.orchestra.run.smithi097.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139918 files and directories currently installed.) 2023-11-26T07:15:45.531 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-26T07:15:45.549 INFO:teuthology.orchestra.run.smithi031.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-11-26T07:15:45.550 INFO:teuthology.orchestra.run.smithi031.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-11-26T07:15:45.560 INFO:teuthology.orchestra.run.smithi031.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-11-26T07:15:45.581 INFO:teuthology.orchestra.run.smithi031.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-11-26T07:15:45.584 INFO:teuthology.orchestra.run.smithi031.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-11-26T07:15:45.585 INFO:teuthology.orchestra.run.smithi031.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-11-26T07:15:45.585 INFO:teuthology.orchestra.run.smithi031.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-11-26T07:15:45.586 INFO:teuthology.orchestra.run.smithi031.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-11-26T07:15:45.597 INFO:teuthology.orchestra.run.smithi031.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-11-26T07:15:45.599 INFO:teuthology.orchestra.run.smithi031.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-11-26T07:15:45.616 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-26T07:15:45.747 INFO:teuthology.orchestra.run.smithi031.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-11-26T07:15:45.747 INFO:teuthology.orchestra.run.smithi031.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-11-26T07:15:45.748 INFO:teuthology.orchestra.run.smithi031.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-11-26T07:15:45.748 INFO:teuthology.orchestra.run.smithi031.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-11-26T07:15:45.748 INFO:teuthology.orchestra.run.smithi031.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-11-26T07:15:45.748 INFO:teuthology.orchestra.run.smithi031.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-11-26T07:15:45.761 INFO:teuthology.orchestra.run.smithi031.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-11-26T07:15:45.762 INFO:teuthology.orchestra.run.smithi031.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-11-26T07:15:46.156 INFO:teuthology.orchestra.run.smithi031.stdout:Preconfiguring packages ... 2023-11-26T07:15:46.265 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-26T07:15:46.280 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-26T07:15:46.316 INFO:teuthology.orchestra.run.smithi031.stdout:Fetched 54.6 MB in 1s (77.5 MB/s) 2023-11-26T07:15:46.327 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-26T07:15:46.465 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package flex. 2023-11-26T07:15:46.541 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139845 files and directories currently installed.) 2023-11-26T07:15:46.545 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-11-26T07:15:46.588 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking flex (2.6.4-8build2) ... 2023-11-26T07:15:46.654 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package dump. 2023-11-26T07:15:46.670 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-26T07:15:46.704 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking dump (0.4b47-3) ... 2023-11-26T07:15:47.007 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package indent. 2023-11-26T07:15:47.023 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:47.066 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-26T07:15:47.277 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-26T07:15:47.318 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-26T07:15:47.334 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-26T07:15:47.377 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-26T07:15:47.446 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-11-26T07:15:47.738 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-26T07:15:47.754 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-26T07:15:47.813 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-26T07:15:47.854 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:15:48.049 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-26T07:15:48.065 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-26T07:15:48.099 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-26T07:15:48.371 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package bison. 2023-11-26T07:15:48.448 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139918 files and directories currently installed.) 2023-11-26T07:15:48.452 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-11-26T07:15:48.454 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-26T07:15:48.470 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-26T07:15:48.499 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-11-26T07:15:48.512 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-26T07:15:48.765 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-26T07:15:48.780 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-26T07:15:48.823 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-26T07:15:48.907 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-11-26T07:15:48.922 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-11-26T07:15:48.965 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-11-26T07:15:49.085 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libjson-perl. 2023-11-26T07:15:49.100 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-26T07:15:49.142 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-26T07:15:49.335 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package dump. 2023-11-26T07:15:49.350 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-11-26T07:15:49.393 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking dump (0.4b47-3) ... 2023-11-26T07:15:49.445 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-26T07:15:49.461 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-26T07:15:49.504 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-26T07:15:49.672 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package indent. 2023-11-26T07:15:49.687 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:49.730 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-26T07:15:50.015 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-11-26T07:15:50.031 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-11-26T07:15:50.074 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-26T07:15:50.394 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-11-26T07:15:50.410 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-11-26T07:15:50.444 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-26T07:15:50.763 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-11-26T07:15:50.779 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-11-26T07:15:50.821 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-11-26T07:15:50.881 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-26T07:15:50.897 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-26T07:15:50.947 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:15:51.199 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libfl2:amd64. 2023-11-26T07:15:51.216 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-11-26T07:15:51.258 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-11-26T07:15:51.276 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-26T07:15:51.292 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-26T07:15:51.334 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:15:51.535 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-11-26T07:15:51.551 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-11-26T07:15:51.593 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-26T07:15:51.645 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-26T07:15:51.661 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-26T07:15:51.703 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:15:51.812 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libjson-perl. 2023-11-26T07:15:51.829 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-11-26T07:15:51.862 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-11-26T07:15:52.039 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-26T07:15:52.055 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-26T07:15:52.097 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-26T07:15:52.292 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libllvm14:amd64. 2023-11-26T07:15:52.308 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-11-26T07:15:52.358 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-26T07:15:52.425 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-26T07:15:52.441 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-26T07:15:52.484 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-26T07:15:52.853 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-26T07:15:52.868 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:52.911 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:15:53.231 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-26T07:15:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:53.313 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:15:53.592 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-26T07:15:53.607 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:53.650 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:15:53.719 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-11-26T07:15:53.735 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-26T07:15:53.777 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:15:53.902 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-26T07:15:53.917 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-26T07:15:53.985 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:15:54.055 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libmbim-proxy. 2023-11-26T07:15:54.071 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-11-26T07:15:54.105 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:15:54.415 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-11-26T07:15:54.432 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-26T07:15:54.472 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-26T07:15:54.487 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-26T07:15:54.507 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:15:54.522 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-26T07:15:54.852 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libndp0:amd64. 2023-11-26T07:15:54.868 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-11-26T07:15:54.899 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package modemmanager. 2023-11-26T07:15:54.910 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-26T07:15:54.915 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-26T07:15:54.958 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:15:55.262 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libnm0:amd64. 2023-11-26T07:15:55.278 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-11-26T07:15:55.320 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-26T07:15:55.622 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package network-manager. 2023-11-26T07:15:55.631 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libpq5:amd64. 2023-11-26T07:15:55.638 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-26T07:15:55.648 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:55.681 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-26T07:15:55.690 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:15:56.043 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-11-26T07:15:56.059 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:56.126 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:15:56.396 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package ppp. 2023-11-26T07:15:56.412 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-26T07:15:56.420 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqmi-proxy. 2023-11-26T07:15:56.436 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:56.478 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:15:56.739 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-11-26T07:15:56.740 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-26T07:15:56.755 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-11-26T07:15:56.813 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:15:57.084 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package pptp-linux. 2023-11-26T07:15:57.098 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-26T07:15:57.143 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-26T07:15:57.429 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-26T07:15:57.443 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-26T07:15:57.479 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-26T07:15:57.710 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-11-26T07:15:57.726 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-11-26T07:15:57.740 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-26T07:15:57.755 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-26T07:15:57.768 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-11-26T07:15:57.798 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-client-common (238) ... 2023-11-26T07:15:58.112 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package modemmanager. 2023-11-26T07:15:58.126 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-26T07:15:58.129 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-11-26T07:15:58.142 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:58.170 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:15:58.185 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:15:58.790 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-common. 2023-11-26T07:15:58.801 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package network-manager. 2023-11-26T07:15:58.806 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-26T07:15:58.817 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-11-26T07:15:58.866 INFO:teuthology.orchestra.run.smithi097.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-26T07:15:58.897 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-common (238) ... 2023-11-26T07:15:58.912 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-11-26T07:15:59.397 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-14. 2023-11-26T07:15:59.413 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:15:59.458 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:15:59.508 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ppp. 2023-11-26T07:15:59.524 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-11-26T07:15:59.871 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-11-26T07:16:00.229 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package pptp-linux. 2023-11-26T07:16:00.243 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-11-26T07:16:00.287 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-11-26T07:16:00.533 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql. 2023-11-26T07:16:00.539 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package network-manager-pptp. 2023-11-26T07:16:00.548 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-26T07:16:00.554 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-11-26T07:16:00.589 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-11-26T07:16:00.591 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql (14+238) ... 2023-11-26T07:16:00.843 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-client. 2023-11-26T07:16:00.858 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-26T07:16:00.876 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-client-common. 2023-11-26T07:16:00.891 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-11-26T07:16:00.893 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-client (14+238) ... 2023-11-26T07:16:00.934 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-client-common (238) ... 2023-11-26T07:16:01.129 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-26T07:16:01.144 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-26T07:16:01.187 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-26T07:16:01.262 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-client-14. 2023-11-26T07:16:01.278 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:16:01.320 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:01.725 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package tcl8.6. 2023-11-26T07:16:01.741 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-26T07:16:01.783 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-26T07:16:01.917 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-common. 2023-11-26T07:16:01.932 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-11-26T07:16:01.994 INFO:teuthology.orchestra.run.smithi031.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-11-26T07:16:02.024 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-common (238) ... 2023-11-26T07:16:02.052 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package tcl. 2023-11-26T07:16:02.068 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-26T07:16:02.107 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-26T07:16:02.455 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package wpasupplicant. 2023-11-26T07:16:02.471 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-26T07:16:02.513 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-26T07:16:02.769 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-14. 2023-11-26T07:16:02.785 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-11-26T07:16:02.830 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:02.993 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-26T07:16:03.008 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-26T07:16:03.051 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-26T07:16:03.379 INFO:teuthology.orchestra.run.smithi097.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-26T07:16:03.395 INFO:teuthology.orchestra.run.smithi097.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-26T07:16:03.437 INFO:teuthology.orchestra.run.smithi097.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-26T07:16:03.747 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-client-common (238) ... 2023-11-26T07:16:03.929 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql. 2023-11-26T07:16:03.945 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-11-26T07:16:03.951 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-26T07:16:03.979 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql (14+238) ... 2023-11-26T07:16:04.077 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-26T07:16:04.223 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-client. 2023-11-26T07:16:04.237 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:04.238 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-11-26T07:16:04.281 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-client (14+238) ... 2023-11-26T07:16:04.338 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-26T07:16:04.464 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-26T07:16:04.559 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-contrib. 2023-11-26T07:16:04.573 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-11-26T07:16:04.589 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:04.617 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-contrib (14+238) ... 2023-11-26T07:16:04.970 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package tcl8.6. 2023-11-26T07:16:04.985 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-11-26T07:16:05.019 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-26T07:16:05.297 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package tcl. 2023-11-26T07:16:05.313 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-11-26T07:16:05.360 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-11-26T07:16:05.599 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-26T07:16:05.704 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-26T07:16:05.758 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package wpasupplicant. 2023-11-26T07:16:05.775 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-11-26T07:16:05.816 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-26T07:16:06.099 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-26T07:16:06.099 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-26T07:16:06.195 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-11-26T07:16:06.212 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-11-26T07:16:06.254 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-11-26T07:16:06.623 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package usb-modeswitch. 2023-11-26T07:16:06.640 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-11-26T07:16:06.690 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-26T07:16:06.773 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-26T07:16:06.924 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:16:06.975 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-client-common (238) ... 2023-11-26T07:16:07.050 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-26T07:16:07.176 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-26T07:16:07.187 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up flex (2.6.4-8build2) ... 2023-11-26T07:16:07.285 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-26T07:16:07.296 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-11-26T07:16:07.395 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-26T07:16:07.397 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:07.506 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-11-26T07:16:07.521 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-26T07:16:07.666 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-11-26T07:16:07.792 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:08.209 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-26T07:16:08.369 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up dump (0.4b47-3) ... 2023-11-26T07:16:08.463 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-26T07:16:08.520 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:16:08.630 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-26T07:16:08.755 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-26T07:16:08.825 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-11-26T07:16:08.881 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-26T07:16:08.922 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-11-26T07:16:09.041 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-26T07:16:09.130 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-26T07:16:09.200 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-26T07:16:09.307 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-26T07:16:09.307 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-11-26T07:16:09.327 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-26T07:16:09.503 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-26T07:16:09.662 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:16:09.788 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-26T07:16:09.956 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-client (14+238) ... 2023-11-26T07:16:09.975 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-11-26T07:16:10.082 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-26T07:16:10.109 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-11-26T07:16:10.236 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-11-26T07:16:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-11-26T07:16:10.479 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-11-26T07:16:10.580 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-11-26T07:16:10.681 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-11-26T07:16:10.734 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:10.734 INFO:teuthology.orchestra.run.smithi097.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-26T07:16:10.734 INFO:teuthology.orchestra.run.smithi097.stdout:which means they are currently configured by ifupdown: 2023-11-26T07:16:10.735 INFO:teuthology.orchestra.run.smithi097.stdout:- enp3s0f1 2023-11-26T07:16:10.735 INFO:teuthology.orchestra.run.smithi097.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-26T07:16:10.735 INFO:teuthology.orchestra.run.smithi097.stdout:remove their configuration from /etc/network/interfaces. 2023-11-26T07:16:10.735 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:10.868 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-26T07:16:11.270 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-26T07:16:11.428 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-11-26T07:16:11.563 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up dump (0.4b47-3) ... 2023-11-26T07:16:11.641 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-11-26T07:16:11.676 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-26T07:16:11.697 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:16:11.807 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-11-26T07:16:11.933 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up tcl (8.6.11+1build2) ... 2023-11-26T07:16:12.084 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libjson-perl (4.04000-1) ... 2023-11-26T07:16:12.210 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-11-26T07:16:12.300 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-11-26T07:16:12.370 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-11-26T07:16:12.454 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:16:12.479 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-11-26T07:16:12.595 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-26T07:16:12.647 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-11-26T07:16:12.712 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-26T07:16:12.756 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:16:12.813 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-common (238) ... 2023-11-26T07:16:12.874 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-11-26T07:16:13.042 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-client (14+238) ... 2023-11-26T07:16:13.210 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-11-26T07:16:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:13.762 INFO:teuthology.orchestra.run.smithi031.stdout:The following network interfaces were found in /etc/network/interfaces 2023-11-26T07:16:13.762 INFO:teuthology.orchestra.run.smithi031.stdout:which means they are currently configured by ifupdown: 2023-11-26T07:16:13.762 INFO:teuthology.orchestra.run.smithi031.stdout:- ens1f0 2023-11-26T07:16:13.762 INFO:teuthology.orchestra.run.smithi031.stdout:If you want to manage those interfaces with NetworkManager instead 2023-11-26T07:16:13.763 INFO:teuthology.orchestra.run.smithi031.stdout:remove their configuration from /etc/network/interfaces. 2023-11-26T07:16:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:13.902 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-11-26T07:16:13.957 INFO:teuthology.orchestra.run.smithi097.stdout:Adding user postgres to group ssl-cert 2023-11-26T07:16:14.229 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:14.229 INFO:teuthology.orchestra.run.smithi097.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-26T07:16:14.309 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-11-26T07:16:14.480 INFO:teuthology.orchestra.run.smithi097.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-26T07:16:14.480 INFO:teuthology.orchestra.run.smithi097.stdout:Removing obsolete dictionary files: 2023-11-26T07:16:14.693 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-11-26T07:16:15.164 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-26T07:16:15.346 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-11-26T07:16:15.464 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-11-26T07:16:15.573 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-11-26T07:16:15.640 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:15.699 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-common (238) ... 2023-11-26T07:16:16.014 INFO:teuthology.orchestra.run.smithi097.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-26T07:16:16.033 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-26T07:16:16.084 INFO:teuthology.orchestra.run.smithi097.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-26T07:16:16.084 INFO:teuthology.orchestra.run.smithi097.stdout:This user must also own the server process. 2023-11-26T07:16:16.084 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:16.084 INFO:teuthology.orchestra.run.smithi097.stdout:The database cluster will be initialized with locale "en_US". 2023-11-26T07:16:16.084 INFO:teuthology.orchestra.run.smithi097.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-26T07:16:16.084 INFO:teuthology.orchestra.run.smithi097.stdout:The default text search configuration will be set to "english". 2023-11-26T07:16:16.084 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:16.084 INFO:teuthology.orchestra.run.smithi097.stdout:Data page checksums are disabled. 2023-11-26T07:16:16.084 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:16.085 INFO:teuthology.orchestra.run.smithi097.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-26T07:16:16.085 INFO:teuthology.orchestra.run.smithi097.stdout:creating subdirectories ... ok 2023-11-26T07:16:16.085 INFO:teuthology.orchestra.run.smithi097.stdout:selecting dynamic shared memory implementation ... posix 2023-11-26T07:16:16.109 INFO:teuthology.orchestra.run.smithi097.stdout:selecting default max_connections ... 100 2023-11-26T07:16:16.144 INFO:teuthology.orchestra.run.smithi097.stdout:selecting default shared_buffers ... 128MB 2023-11-26T07:16:16.158 INFO:teuthology.orchestra.run.smithi097.stdout:selecting default time zone ... Etc/UTC 2023-11-26T07:16:16.159 INFO:teuthology.orchestra.run.smithi097.stdout:creating configuration files ... ok 2023-11-26T07:16:16.327 INFO:teuthology.orchestra.run.smithi097.stdout:running bootstrap script ... ok 2023-11-26T07:16:16.640 INFO:teuthology.orchestra.run.smithi031.stdout:Adding user postgres to group ssl-cert 2023-11-26T07:16:16.902 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:16.902 INFO:teuthology.orchestra.run.smithi031.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-11-26T07:16:16.932 INFO:teuthology.orchestra.run.smithi097.stdout:performing post-bootstrap initialization ... ok 2023-11-26T07:16:17.167 INFO:teuthology.orchestra.run.smithi031.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-11-26T07:16:17.167 INFO:teuthology.orchestra.run.smithi031.stdout:Removing obsolete dictionary files: 2023-11-26T07:16:17.523 INFO:teuthology.orchestra.run.smithi097.stdout:syncing data to disk ... ok 2023-11-26T07:16:17.841 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-11-26T07:16:18.021 INFO:teuthology.orchestra.run.smithi097.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-26T07:16:18.317 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-11-26T07:16:18.672 INFO:teuthology.orchestra.run.smithi031.stdout:Creating new PostgreSQL cluster 14/main ... 2023-11-26T07:16:18.694 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-11-26T07:16:18.757 INFO:teuthology.orchestra.run.smithi031.stdout:The files belonging to this database system will be owned by user "postgres". 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:This user must also own the server process. 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:The database cluster will be initialized with locale "en_US". 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:The default text search configuration will be set to "english". 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:Data page checksums are disabled. 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:creating subdirectories ... ok 2023-11-26T07:16:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:selecting dynamic shared memory implementation ... posix 2023-11-26T07:16:18.783 INFO:teuthology.orchestra.run.smithi031.stdout:selecting default max_connections ... 100 2023-11-26T07:16:18.819 INFO:teuthology.orchestra.run.smithi031.stdout:selecting default shared_buffers ... 128MB 2023-11-26T07:16:18.834 INFO:teuthology.orchestra.run.smithi031.stdout:selecting default time zone ... Etc/UTC 2023-11-26T07:16:18.834 INFO:teuthology.orchestra.run.smithi031.stdout:creating configuration files ... ok 2023-11-26T07:16:19.013 INFO:teuthology.orchestra.run.smithi031.stdout:running bootstrap script ... ok 2023-11-26T07:16:19.653 INFO:teuthology.orchestra.run.smithi031.stdout:performing post-bootstrap initialization ... ok 2023-11-26T07:16:20.303 INFO:teuthology.orchestra.run.smithi031.stdout:syncing data to disk ... ok 2023-11-26T07:16:20.819 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-11-26T07:16:21.081 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:16:21.207 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-26T07:16:21.332 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up postgresql (14+238) ... 2023-11-26T07:16:21.483 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:16:21.609 INFO:teuthology.orchestra.run.smithi097.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:16:21.887 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-26T07:16:21.887 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-26T07:16:22.595 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-26T07:16:22.914 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-26T07:16:23.057 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-26T07:16:24.075 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:16:24.209 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-contrib (14+238) ... 2023-11-26T07:16:24.335 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql (14+238) ... 2023-11-26T07:16:24.451 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-26T07:16:24.460 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-11-26T07:16:24.551 INFO:teuthology.orchestra.run.smithi097.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-26T07:16:24.620 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-11-26T07:16:24.889 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-11-26T07:16:24.890 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-11-26T07:16:25.244 INFO:teuthology.orchestra.run.smithi097.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-26T07:16:25.250 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:25.681 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-11-26T07:16:25.999 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-11-26T07:16:26.134 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-11-26T07:16:27.544 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-11-26T07:16:27.645 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-11-26T07:16:28.349 INFO:teuthology.orchestra.run.smithi031.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-11-26T07:16:28.354 DEBUG:teuthology.parallel:result is None 2023-11-26T07:16:28.354 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:16:28.568 DEBUG:teuthology.orchestra.run.smithi031:> dpkg-query -W -f '${Version}' ceph 2023-11-26T07:16:28.591 INFO:teuthology.orchestra.run.smithi031.stdout:18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:16:28.591 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:16:28.591 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484-g44c2237f-1jammy is installed. 2023-11-26T07:16:28.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=44c2237fe77713214148c1fd622c872f346f1b99 2023-11-26T07:16:28.806 DEBUG:teuthology.orchestra.run.smithi097:> dpkg-query -W -f '${Version}' ceph 2023-11-26T07:16:28.832 INFO:teuthology.orchestra.run.smithi097.stdout:18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:16:28.832 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484-g44c2237f-1jammy 2023-11-26T07:16:28.832 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484-g44c2237f-1jammy is installed. 2023-11-26T07:16:28.833 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-26T07:16:28.833 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:28.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-26T07:16:28.854 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:16:28.854 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-26T07:16:28.895 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-26T07:16:28.895 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:28.895 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2023-11-26T07:16:28.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-26T07:16:28.990 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:16:28.990 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2023-11-26T07:16:29.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-26T07:16:29.073 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-26T07:16:29.074 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:29.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-26T07:16:29.094 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-26T07:16:29.158 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:16:29.158 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-26T07:16:29.179 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-26T07:16:29.240 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-26T07:16:29.241 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:29.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2023-11-26T07:16:29.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-26T07:16:29.322 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:16:29.322 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2023-11-26T07:16:29.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-26T07:16:29.409 INFO:teuthology.run_tasks:Running task ceph... 2023-11-26T07:16:29.472 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-26T07:16:29.472 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2023-11-26T07:16:29.476 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /var/log/ceph 2023-11-26T07:16:29.496 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-26T07:16:29.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-26T07:16:29.537 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-26T07:16:29.561 INFO:tasks.ceph:Creating extra log directories... 2023-11-26T07:16:29.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-26T07:16:29.601 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-26T07:16:29.631 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-26T07:16:29.631 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-26T07:16:29.631 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-26T07:16:29.631 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:29.632 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-26T07:16:29.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-26T07:16:29.768 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:16:29.768 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-26T07:16:29.777 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-26T07:16:29.879 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-26T07:16:29.879 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-26T07:16:29.879 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}', 'email': None, 'first_in_suite': False, 'job_id': '7467348', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-mgr-vol-misc', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1863, 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-mgr-vol-misc', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFdUem5b4uJ1RP0qDFIJWqlOYtyO8Uys9f1w0JQufAK1nRJT1UvZ7YTajtMvazOYiXkxfaefbDWQOCeDXdf4mw=', 'smithi097.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRZs30f+Ls5Q5Etnx4X6xuZ6NMQ3Xg3Xl2Q7odrw+rGhijjK+x3/VA3x/9HKNd3t3oMpcyH0ELdOzKCFkCQ6XI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '44c2237fe77713214148c1fd622c872f346f1b99', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-26_06:51:12', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-26T07:16:29.880 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-26T07:16:29.884 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-26T07:16:29.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2023-11-26T07:16:29.937 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m0777 -- /var/run/ceph 2023-11-26T07:16:29.960 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:29.961 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2023-11-26T07:16:30.010 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-26T07:16:30.011 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2023-11-26T07:16:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-26T07:16:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:30.067 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 897 Links: 1 2023-11-26T07:16:30.067 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:30.067 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-11-26 07:09:47.281792749 +0000 2023-11-26T07:16:30.067 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-11-26 07:09:46.869796430 +0000 2023-11-26T07:16:30.067 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-11-26 07:09:46.869796430 +0000 2023-11-26T07:16:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-11-26 07:09:46.869796430 +0000 2023-11-26T07:16:30.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-26T07:16:30.133 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:16:30.133 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:16:30.133 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000390929 s, 1.3 MB/s 2023-11-26T07:16:30.135 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-26T07:16:30.189 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2023-11-26T07:16:30.243 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-26T07:16:30.243 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:30.243 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 912 Links: 1 2023-11-26T07:16:30.243 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:30.244 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-11-26 07:09:47.969786604 +0000 2023-11-26T07:16:30.244 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-11-26 07:09:47.553790320 +0000 2023-11-26T07:16:30.244 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-11-26 07:09:47.553790320 +0000 2023-11-26T07:16:30.244 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-11-26 07:09:47.553790320 +0000 2023-11-26T07:16:30.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-26T07:16:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:16:30.310 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:16:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000395945 s, 1.3 MB/s 2023-11-26T07:16:30.312 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-26T07:16:30.365 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2023-11-26T07:16:30.415 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-26T07:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 931 Links: 1 2023-11-26T07:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-11-26 07:09:48.633780674 +0000 2023-11-26T07:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-11-26 07:09:48.237784211 +0000 2023-11-26T07:16:30.416 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-11-26 07:09:48.237784211 +0000 2023-11-26T07:16:30.417 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-11-26 07:09:48.237784211 +0000 2023-11-26T07:16:30.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-26T07:16:30.478 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:16:30.478 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:16:30.478 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000374033 s, 1.4 MB/s 2023-11-26T07:16:30.480 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-26T07:16:30.533 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2023-11-26T07:16:30.583 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-26T07:16:30.583 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:30.584 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 949 Links: 1 2023-11-26T07:16:30.584 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:30.584 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-11-26 07:09:48.933777993 +0000 2023-11-26T07:16:30.584 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-11-26 07:09:48.929778029 +0000 2023-11-26T07:16:30.584 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-11-26 07:09:48.929778029 +0000 2023-11-26T07:16:30.584 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2023-11-26 07:09:48.929778029 +0000 2023-11-26T07:16:30.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-26T07:16:30.646 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:16:30.646 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:16:30.646 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000377898 s, 1.4 MB/s 2023-11-26T07:16:30.648 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-26T07:16:30.701 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:30.701 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:16:30.701 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2023-11-26T07:16:30.711 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-26T07:16:30.711 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2023-11-26T07:16:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-26T07:16:30.765 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:30.766 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 890 Links: 1 2023-11-26T07:16:30.766 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:30.766 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-11-26 07:09:44.468149690 +0000 2023-11-26T07:16:30.766 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-11-26 07:09:44.064153299 +0000 2023-11-26T07:16:30.766 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-11-26 07:09:44.064153299 +0000 2023-11-26T07:16:30.766 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2023-11-26 07:09:44.064153299 +0000 2023-11-26T07:16:30.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-26T07:16:30.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-11-26T07:16:30.828 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-11-26T07:16:30.828 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000396809 s, 1.3 MB/s 2023-11-26T07:16:30.830 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-26T07:16:30.883 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2023-11-26T07:16:30.937 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-26T07:16:30.937 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:30.937 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 906 Links: 1 2023-11-26T07:16:30.937 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:30.938 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-11-26 07:09:44.760147082 +0000 2023-11-26T07:16:30.938 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-11-26 07:09:44.756147118 +0000 2023-11-26T07:16:30.938 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-11-26 07:09:44.756147118 +0000 2023-11-26T07:16:30.938 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2023-11-26 07:09:44.756147118 +0000 2023-11-26T07:16:30.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-26T07:16:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-11-26T07:16:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-11-26T07:16:31.000 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000624625 s, 820 kB/s 2023-11-26T07:16:31.002 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-26T07:16:31.055 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2023-11-26T07:16:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-26T07:16:31.109 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:31.110 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 925 Links: 1 2023-11-26T07:16:31.110 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:31.110 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-11-26 07:09:45.804137757 +0000 2023-11-26T07:16:31.110 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-11-26 07:09:45.432141079 +0000 2023-11-26T07:16:31.110 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-11-26 07:09:45.432141079 +0000 2023-11-26T07:16:31.110 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2023-11-26 07:09:45.432141079 +0000 2023-11-26T07:16:31.111 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-26T07:16:31.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-11-26T07:16:31.172 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-11-26T07:16:31.172 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00038701 s, 1.3 MB/s 2023-11-26T07:16:31.174 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-26T07:16:31.227 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2023-11-26T07:16:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-26T07:16:31.281 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-26T07:16:31.281 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 943 Links: 1 2023-11-26T07:16:31.281 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-26T07:16:31.282 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-11-26 07:09:46.468131826 +0000 2023-11-26T07:16:31.282 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-11-26 07:09:46.056135506 +0000 2023-11-26T07:16:31.282 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-11-26 07:09:46.056135506 +0000 2023-11-26T07:16:31.282 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2023-11-26 07:09:46.056135506 +0000 2023-11-26T07:16:31.282 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-26T07:16:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-11-26T07:16:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-11-26T07:16:31.344 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00038542 s, 1.3 MB/s 2023-11-26T07:16:31.346 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-26T07:16:31.399 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:31.399 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-26T07:16:31.399 INFO:tasks.ceph:Generating config... 2023-11-26T07:16:31.401 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-26T07:16:31.401 INFO:tasks.ceph:[client] debug client = 20 2023-11-26T07:16:31.401 INFO:tasks.ceph:[client] debug ms = 1 2023-11-26T07:16:31.401 INFO:tasks.ceph:[client] pid file = 2023-11-26T07:16:31.401 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-26T07:16:31.401 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-26T07:16:31.401 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-26T07:16:31.402 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-26T07:16:31.402 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-26T07:16:31.402 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-26T07:16:31.402 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-26T07:16:31.402 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-26T07:16:31.402 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-26T07:16:31.402 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-26T07:16:31.403 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-26T07:16:31.403 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-26T07:16:31.403 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-26T07:16:31.403 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-26T07:16:31.403 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-26T07:16:31.403 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-26T07:16:31.404 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-26T07:16:31.404 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-26T07:16:31.404 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-26T07:16:31.404 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-26T07:16:31.404 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-26T07:16:31.404 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-26T07:16:31.404 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-26T07:16:31.405 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-26T07:16:31.405 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-26T07:16:31.405 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-26T07:16:31.405 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-26T07:16:31.405 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-26T07:16:31.405 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-26T07:16:31.405 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-26T07:16:31.406 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-26T07:16:31.406 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-26T07:16:31.406 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-26T07:16:31.406 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-26T07:16:31.406 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-26T07:16:31.406 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-26T07:16:31.406 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-26T07:16:31.407 INFO:tasks.ceph:Setting up mon.a... 2023-11-26T07:16:31.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-26T07:16:31.455 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2023-11-26T07:16:31.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-26T07:16:31.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-26T07:16:31.539 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.b', '172.21.15.97'), ('mon.c', '[v2:172.21.15.97:3301,v1:172.21.15.97:6790]')] 2023-11-26T07:16:31.540 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-26T07:16:31.541 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:31.541 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-26T07:16:31.590 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --add b 172.21.15.97 --addv c '[v2:172.21.15.97:3301,v1:172.21.15.97:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:31.685 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:31.686 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid 4c2b4df1-71b1-4841-8807-af144d2610e7 2023-11-26T07:16:31.686 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = pacific 2023-11-26T07:16:31.686 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2023-11-26T07:16:31.686 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 4c2b4df1-71b1-4841-8807-af144d2610e7 2023-11-26T07:16:31.686 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2023-11-26T07:16:31.686388+0000 2023-11-26T07:16:31.686 INFO:teuthology.orchestra.run.smithi031.stdout:created 2023-11-26T07:16:31.686388+0000 2023-11-26T07:16:31.687 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 16 (pacific) 2023-11-26T07:16:31.687 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2023-11-26T07:16:31.687 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2023-11-26T07:16:31.687 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2023-11-26T07:16:31.687 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] mon.c 2023-11-26T07:16:31.687 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-26T07:16:31.688 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-26T07:16:31.697 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4c2b4df1-71b1-4841-8807-af144d2610e7... 2023-11-26T07:16:31.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-26T07:16:31.741 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-26T07:16:31.784 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2023-11-26T07:16:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2023-11-26T07:16:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-26T07:16:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2023-11-26T07:16:31.784 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2023-11-26T07:16:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2023-11-26T07:16:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2023-11-26T07:16:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2023-11-26T07:16:31.785 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2023-11-26T07:16:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2023-11-26T07:16:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-26T07:16:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2023-11-26T07:16:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.786 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2023-11-26T07:16:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-26T07:16:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2023-11-26T07:16:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2023-11-26T07:16:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2023-11-26T07:16:31.787 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2023-11-26T07:16:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2023-11-26T07:16:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2023-11-26T07:16:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2023-11-26T07:16:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2023-11-26T07:16:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-26T07:16:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-26T07:16:31.788 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2023-11-26T07:16:31.789 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790]" 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on skipped message = False 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 4c2b4df1-71b1-4841-8807-af144d2610e7 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.790 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug trim objects = true 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2023-11-26T07:16:31.791 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: bdev async discard = True 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: bdev enable discard = True 2023-11-26T07:16:31.792 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore allocator = bitmap 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2023-11-26T07:16:31.793 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:16:31.794 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: debug client = 20 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: debug finisher = 20 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:16:31.795 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.796 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: mon op complaint time = 120 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-26T07:16:31.797 INFO:teuthology.orchestra.run.smithi031.stdout: client mount timeout = 600 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: debug client = 20 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = "" 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2023-11-26T07:16:31.798 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout:[mds] 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: debug mds = 20 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: debug mds balancer = 20 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug frag = True 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug scatterstat = True 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: mds op complaint time = 180 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: mds verify scatter = True 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2023-11-26T07:16:31.799 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2023-11-26T07:16:31.800 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2023-11-26T07:16:31.800 INFO:teuthology.orchestra.run.smithi097.stdout: chdir = "" 2023-11-26T07:16:31.800 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-26T07:16:31.800 INFO:teuthology.orchestra.run.smithi097.stdout: auth supported = cephx 2023-11-26T07:16:31.800 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: filestore xattr use omap = true 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: mon clock drift allowed = 1.000 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: osd crush chooseleaf type = 0 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: auth debug = true 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on old message = true 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on bug = true 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-26T07:16:31.801 INFO:teuthology.orchestra.run.smithi097.stdout: mon pg warn max object skew = 0 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: # disable pg_autoscaler by default for new pools 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default size = 2 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow primary affinity = true 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd allow pg remap = true 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on legacy crush tunables = false 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on crush straw calc version zero = false 2023-11-26T07:16:31.802 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on no sortbitwise = false 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on osd down out interval zero = false 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: mon warn on too few osds = false 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: mon_allow_pool_size_one = true 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: osd default data pool replay window = 5 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: mon allow pool delete = true 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.803 INFO:teuthology.orchestra.run.smithi097.stdout: mon cluster log file level = debug 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: debug asserts on shutdown = true 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: mon health detail to clog = false 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: mon host = "172.21.15.31,172.21.15.97,[v2:172.21.15.97:3301,v1:172.21.15.97:6790]" 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: ms die on skipped message = False 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = 4c2b4df1-71b1-4841-8807-af144d2610e7 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout:[osd] 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: osd journal size = 100 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub load threshold = 5.0 2023-11-26T07:16:31.804 INFO:teuthology.orchestra.run.smithi097.stdout: osd scrub max interval = 600 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: osd mclock profile = high_recovery_ops 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: osd recover clone overlap = true 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: osd recovery max chunk = 1048576 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug shutdown = true 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug op order = true 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug verify stray on activate = true 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug trim objects = true 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.805 INFO:teuthology.orchestra.run.smithi097.stdout: osd open classes on start = true 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug pg log writeout = true 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: osd deep scrub update digest min age = 30 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: osd map max advance = 10 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: journal zero on create = true 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: filestore ondisk finisher threads = 3 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: filestore apply finisher threads = 3 2023-11-26T07:16:31.806 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: bdev debug aio = true 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: osd debug misdirected ops = true 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: bdev async discard = True 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: bdev enable discard = True 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore allocator = bitmap 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore block size = 96636764160 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: bluestore fsck on mount = True 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluefs = 1/20 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: debug bluestore = 1/20 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: debug osd = 20 2023-11-26T07:16:31.807 INFO:teuthology.orchestra.run.smithi097.stdout: debug rocksdb = 4/10 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd backfillfull_ratio = 0.85 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd full ratio = 0.9 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd nearfull ratio = 0.8 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: osd failsafe full ratio = 0.95 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: osd objectstore = bluestore 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr] 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: debug mgr = 20 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-11-26T07:16:31.808 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: mgr/telemetry/nag = false 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: debug finisher = 20 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout:[mon] 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: debug mon = 20 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: debug paxos = 20 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: debug auth = 20 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: mon data avail warn = 5 2023-11-26T07:16:31.809 INFO:teuthology.orchestra.run.smithi097.stdout: mon mgr mkfs grace = 240 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min pgs per osd = 4 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd reporter subtree level = osd 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: mon osd prime pg temp = true 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: mon reweight min bytes per osd = 10 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: auth mon ticket ttl = 660 # 11m 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: auth service ticket ttl = 240 # 4m 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: # don't complain about insecure global_id in the test suite 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-26T07:16:31.810 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: # 1m isn't quite enough 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: mon_down_mkfs_grace = 2m 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: mon_warn_on_filestore_osds = false 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: mon op complaint time = 120 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout:[client] 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: rgw cache enabled = true 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable ops log = true 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: rgw enable usage log = true 2023-11-26T07:16:31.811 INFO:teuthology.orchestra.run.smithi097.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout: client mount timeout = 600 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout: debug client = 20 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout: pid file = "" 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.a] 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.b] 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout:[mon.c] 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-11-26T07:16:31.812 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout:[mds] 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds = 20 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout: debug mds balancer = 20 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout: debug ms = 1 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug frag = True 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout: mds debug scatterstat = True 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout: mds op complaint time = 180 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout: mds verify scatter = True 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout: osd op complaint time = 180 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout: rados mon op timeout = 900 2023-11-26T07:16:31.813 INFO:teuthology.orchestra.run.smithi097.stdout: rados osd op timeout = 900 2023-11-26T07:16:31.815 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-26T07:16:31.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-26T07:16:31.921 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-26T07:16:31.922 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:31.922 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-26T07:16:31.931 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:31.931 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-26T07:16:31.986 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:16:31.986 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:31.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-26T07:16:31.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-26T07:16:32.063 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:16:32.063 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:32.118 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:16:32.118 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:16:32.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-26T07:16:32.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-26T07:16:32.154 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:16:32.154 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:32.209 INFO:tasks.ceph:Setting up mon nodes... 2023-11-26T07:16:32.210 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-26T07:16:32.210 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-26T07:16:32.267 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-26T07:16:32.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-26T07:16:32.333 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-26T07:16:32.353 INFO:tasks.ceph:Setting up mds nodes... 2023-11-26T07:16:32.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-26T07:16:32.411 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-26T07:16:32.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-26T07:16:32.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-26T07:16:32.534 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-26T07:16:32.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-26T07:16:32.559 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-26T07:16:32.619 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-26T07:16:32.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-26T07:16:32.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-26T07:16:32.740 INFO:teuthology.orchestra.run.smithi097.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-26T07:16:32.744 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-26T07:16:32.764 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-26T07:16:32.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-26T07:16:32.809 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-26T07:16:32.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-26T07:16:32.913 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-26T07:16:32.931 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-26T07:16:32.973 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-26T07:16:32.990 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-26T07:16:33.078 INFO:teuthology.orchestra.run.smithi097.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-26T07:16:33.095 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-26T07:16:33.095 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi097.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-26T07:16:33.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-26T07:16:33.116 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:33.116 INFO:tasks.ceph:role: osd.0 2023-11-26T07:16:33.117 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:16:33.117 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-26T07:16:33.189 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:33.190 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:33.190 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:33.190 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:33.190 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:33.190 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:33.654 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-11-26T07:16:33.657 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-11-26T07:16:33.657 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-26T07:16:33.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-26T07:16:33.809 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:33.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:33.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-26T07:16:33.875 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:33.875 INFO:tasks.ceph:role: osd.1 2023-11-26T07:16:33.875 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:16:33.875 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-26T07:16:33.950 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:33.951 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:33.951 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:33.951 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:33.951 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:33.951 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:34.404 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-11-26T07:16:34.407 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-11-26T07:16:34.407 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-26T07:16:34.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-26T07:16:34.532 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:34.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:34.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-26T07:16:34.595 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:34.595 INFO:tasks.ceph:role: osd.2 2023-11-26T07:16:34.596 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:16:34.596 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-26T07:16:34.668 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:34.668 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:34.668 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:34.668 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:34.668 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:34.668 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:35.125 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-11-26T07:16:35.128 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-11-26T07:16:35.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-26T07:16:35.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-26T07:16:35.290 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:35.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:35.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-26T07:16:35.355 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:35.355 INFO:tasks.ceph:role: osd.3 2023-11-26T07:16:35.355 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:16:35.355 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-26T07:16:35.427 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:35.427 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:35.427 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:35.427 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:35.428 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:35.428 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:35.428 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:35.905 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-11-26T07:16:35.908 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-11-26T07:16:35.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-26T07:16:36.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-26T07:16:36.079 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:36.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:36.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:36.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:36.279+0000 7f862732d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-26T07:16:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:36.283+0000 7f862732d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-26T07:16:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:36.283+0000 7f862732d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-26T07:16:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:36.283+0000 7f862732d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-26T07:16:36.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:36.283+0000 7f862732d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-26T07:16:36.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:36.283+0000 7f862732d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-26T07:16:39.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-26T07:16:39.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:39.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:39.391+0000 7fe888a2b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-26T07:16:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:39.391+0000 7fe888a2b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-26T07:16:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:39.391+0000 7fe888a2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-26T07:16:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:39.395+0000 7fe888a2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-26T07:16:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:39.395+0000 7fe888a2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-26T07:16:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:39.395+0000 7fe888a2b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-26T07:16:42.402 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-26T07:16:42.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:42.523+0000 7fdda75a4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-26T07:16:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:42.523+0000 7fdda75a4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-26T07:16:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:42.523+0000 7fdda75a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-26T07:16:42.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:42.523+0000 7fdda75a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-26T07:16:42.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:42.523+0000 7fdda75a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-26T07:16:42.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:42.523+0000 7fdda75a4640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-26T07:16:45.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-26T07:16:45.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:45.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:45.647+0000 7f171e428640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-26T07:16:45.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:45.647+0000 7f171e428640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-26T07:16:45.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:45.647+0000 7f171e428640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-26T07:16:45.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:45.647+0000 7f171e428640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-26T07:16:45.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:45.647+0000 7f171e428640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-26T07:16:45.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:16:45.647+0000 7f171e428640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-26T07:16:48.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-26T07:16:48.667 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-26T07:16:48.689 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:48.689 INFO:tasks.ceph:role: osd.4 2023-11-26T07:16:48.690 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:16:48.690 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-26T07:16:48.765 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:48.765 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:48.766 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:48.766 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:48.766 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:48.766 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:48.766 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:49.222 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-11-26T07:16:49.225 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-11-26T07:16:49.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-26T07:16:49.329 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-26T07:16:49.347 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:49.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:49.348 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-26T07:16:49.410 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:49.410 INFO:tasks.ceph:role: osd.5 2023-11-26T07:16:49.410 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:16:49.410 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-26T07:16:49.481 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:49.481 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:49.481 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:49.482 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:49.482 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:49.482 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:49.482 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:49.962 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-11-26T07:16:49.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-11-26T07:16:49.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-26T07:16:50.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-26T07:16:50.136 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:50.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:50.137 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-26T07:16:50.206 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:50.206 INFO:tasks.ceph:role: osd.6 2023-11-26T07:16:50.206 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:16:50.207 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-26T07:16:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:50.278 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:50.279 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:50.762 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-11-26T07:16:50.765 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-11-26T07:16:50.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-26T07:16:50.902 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-26T07:16:50.919 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:50.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:50.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-26T07:16:50.989 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-26T07:16:50.990 INFO:tasks.ceph:role: osd.7 2023-11-26T07:16:50.990 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:16:50.990 DEBUG:teuthology.orchestra.run.smithi097:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-26T07:16:51.065 INFO:teuthology.orchestra.run.smithi097.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-26T07:16:51.065 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-26T07:16:51.065 INFO:teuthology.orchestra.run.smithi097.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-26T07:16:51.065 INFO:teuthology.orchestra.run.smithi097.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-26T07:16:51.066 INFO:teuthology.orchestra.run.smithi097.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-26T07:16:51.066 INFO:teuthology.orchestra.run.smithi097.stdout: = sunit=0 swidth=0 blks 2023-11-26T07:16:51.066 INFO:teuthology.orchestra.run.smithi097.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-26T07:16:51.066 INFO:teuthology.orchestra.run.smithi097.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-26T07:16:51.066 INFO:teuthology.orchestra.run.smithi097.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-26T07:16:51.066 INFO:teuthology.orchestra.run.smithi097.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-26T07:16:51.526 INFO:teuthology.orchestra.run.smithi097.stdout:Discarding blocks...Done. 2023-11-26T07:16:51.529 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi097.front.sepia.ceph.com -o noatime 2023-11-26T07:16:51.529 DEBUG:teuthology.orchestra.run.smithi097:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-26T07:16:51.638 DEBUG:teuthology.orchestra.run.smithi097:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-26T07:16:51.655 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: /sbin/restorecon: command not found 2023-11-26T07:16:51.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:16:51.656 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:51.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:51.786+0000 7f36ca08c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-26T07:16:51.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:51.790+0000 7f36ca08c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-26T07:16:51.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:51.790+0000 7f36ca08c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-26T07:16:51.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:51.790+0000 7f36ca08c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-26T07:16:51.794 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:51.790+0000 7f36ca08c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-26T07:16:51.801 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:51.798+0000 7f36ca08c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-26T07:16:54.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-26T07:16:54.828 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:54.924 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:54.918+0000 7f06080f3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-26T07:16:54.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:54.918+0000 7f06080f3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-26T07:16:54.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:54.918+0000 7f06080f3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-26T07:16:54.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:54.918+0000 7f06080f3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-26T07:16:54.925 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:54.918+0000 7f06080f3640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-26T07:16:54.926 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:54.922+0000 7f06080f3640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-26T07:16:57.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-26T07:16:57.941 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:16:58.039 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:58.034+0000 7f788ac4a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-26T07:16:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:58.034+0000 7f788ac4a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-26T07:16:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:58.034+0000 7f788ac4a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-26T07:16:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:58.034+0000 7f788ac4a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-26T07:16:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:58.034+0000 7f788ac4a640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-26T07:16:58.042 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:16:58.038+0000 7f788ac4a640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-26T07:16:59.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:16:59.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:17:01.024 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-26T07:17:01.045 DEBUG:teuthology.orchestra.run.smithi097:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:01.147 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:01.141+0000 7f4baa309640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-26T07:17:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:01.141+0000 7f4baa309640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-26T07:17:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:01.141+0000 7f4baa309640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-26T07:17:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:01.141+0000 7f4baa309640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-26T07:17:01.149 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:01.141+0000 7f4baa309640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-26T07:17:01.150 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:01.145+0000 7f4baa309640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-26T07:17:04.164 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-26T07:17:04.184 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-26T07:17:04.185 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:04.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-26T07:17:04.208 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:04.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-26T07:17:04.272 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:04.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-26T07:17:04.336 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:04.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-26T07:17:04.400 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:04.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-26T07:17:04.464 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:04.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-26T07:17:04.527 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:04.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-26T07:17:04.592 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:04.592 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-26T07:17:04.613 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:04.613 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-26T07:17:04.674 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:04.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-26T07:17:04.742 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:04.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-26T07:17:04.806 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:04.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-26T07:17:04.869 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:04.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-26T07:17:04.934 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:04.934 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-26T07:17:04.998 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:04.998 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-26T07:17:05.008 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:05.008 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-26T07:17:05.063 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:05.063 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-26T07:17:05.072 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:05.072 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-26T07:17:05.127 INFO:tasks.ceph:Adding keys to all mons... 2023-11-26T07:17:05.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-26T07:17:05.131 DEBUG:teuthology.orchestra.run.smithi097:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-26T07:17:05.178 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2023-11-26T07:17:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJlYhb/DxAAx1G3Ve23IdZ7cX9nPb2WhA== 2023-11-26T07:17:05.178 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2023-11-26T07:17:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJlRJmWGBAAuUAS68YFt9PaqDV8RFi+6w== 2023-11-26T07:17:05.178 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2023-11-26T07:17:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJlfuXoHxAAf7vKZr+LVZaN6HR3qiks5w== 2023-11-26T07:17:05.178 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2023-11-26T07:17:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDU8GJlc/bqEBAAueOQvSeK6MW+WOT746yQzA== 2023-11-26T07:17:05.179 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2023-11-26T07:17:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDX8GJlEaW6FxAA8q4rEKSJtIl/Hr2uCNSMUg== 2023-11-26T07:17:05.179 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2023-11-26T07:17:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDa8GJldV90HxAAQRgcLf8ABSU4Sfb3A1Xa4g== 2023-11-26T07:17:05.179 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2023-11-26T07:17:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDd8GJl2erPJhAAoC71eDwCOmskb+VCLYQyUw== 2023-11-26T07:17:05.179 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2023-11-26T07:17:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJlnh/pExAA6qyPM7So2BLbxbGbquLZZA== 2023-11-26T07:17:05.179 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2023-11-26T07:17:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJlIdH1JBAAmPlAQXq78ht4pJf6I7Cd5w== 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJlWmgkLBAAojAzX06O7kbwmj154qUGog== 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDj8GJllq08LxAAGuz9+Ou/ybWxdkN3PSMaEQ== 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDm8GJlww4eNxAAjRaBq45nrNwJPlopyOhXNg== 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDq8GJl2jxiAhAAhu2OULy84YRJYxTmShoVBg== 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDt8GJld8HNCBAA3deA4HKTqRggbVc73Y8efg== 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-11-26T07:17:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA== 2023-11-26T07:17:05.181 INFO:teuthology.orchestra.run.smithi031.stdout:[client.2] 2023-11-26T07:17:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJl/CN+NhAAeyry8gIOtxwLxCQb0t1eKw== 2023-11-26T07:17:05.181 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2023-11-26T07:17:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJlEBAQOhAAx5TsMN17L2fNm+j3yqUIOQ== 2023-11-26T07:17:05.181 INFO:teuthology.orchestra.run.smithi031.stdout:[client.3] 2023-11-26T07:17:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDR8GJlA9e7BBAATe3WYA79PgfLc5UHgKGkpw== 2023-11-26T07:17:05.192 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.y] 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQ8GJlYhb/DxAAx1G3Ve23IdZ7cX9nPb2WhA== 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a] 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQ8GJlRJmWGBAAuUAS68YFt9PaqDV8RFi+6w== 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c] 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQ8GJlfuXoHxAAf7vKZr+LVZaN6HR3qiks5w== 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.0] 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDU8GJlc/bqEBAAueOQvSeK6MW+WOT746yQzA== 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.1] 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDX8GJlEaW6FxAA8q4rEKSJtIl/Hr2uCNSMUg== 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.2] 2023-11-26T07:17:05.193 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDa8GJldV90HxAAQRgcLf8ABSU4Sfb3A1Xa4g== 2023-11-26T07:17:05.194 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.3] 2023-11-26T07:17:05.194 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDd8GJl2erPJhAAoC71eDwCOmskb+VCLYQyUw== 2023-11-26T07:17:05.194 INFO:teuthology.orchestra.run.smithi097.stdout:[mgr.x] 2023-11-26T07:17:05.194 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQ8GJlnh/pExAA6qyPM7So2BLbxbGbquLZZA== 2023-11-26T07:17:05.194 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b] 2023-11-26T07:17:05.194 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQ8GJlIdH1JBAAmPlAQXq78ht4pJf6I7Cd5w== 2023-11-26T07:17:05.194 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d] 2023-11-26T07:17:05.194 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQ8GJlWmgkLBAAojAzX06O7kbwmj154qUGog== 2023-11-26T07:17:05.194 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.4] 2023-11-26T07:17:05.194 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDj8GJllq08LxAAGuz9+Ou/ybWxdkN3PSMaEQ== 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.5] 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDm8GJlww4eNxAAjRaBq45nrNwJPlopyOhXNg== 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.6] 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDq8GJl2jxiAhAAhu2OULy84YRJYxTmShoVBg== 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout:[osd.7] 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDt8GJld8HNCBAA3deA4HKTqRggbVc73Y8efg== 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA== 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout:[client.2] 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQ8GJl/CN+NhAAeyry8gIOtxwLxCQb0t1eKw== 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2023-11-26T07:17:05.195 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDQ8GJlEBAQOhAAx5TsMN17L2fNm+j3yqUIOQ== 2023-11-26T07:17:05.196 INFO:teuthology.orchestra.run.smithi097.stdout:[client.3] 2023-11-26T07:17:05.196 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQDR8GJlA9e7BBAATe3WYA79PgfLc5UHgKGkpw== 2023-11-26T07:17:05.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:17:05.225 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:17:05.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:05.304 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:05.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:05.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:05.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.540 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.600 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:17:05.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-26T07:17:05.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:05.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:05.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:05.774 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-26T07:17:05.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:05.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:06.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:06.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-26T07:17:06.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:06.068 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:06.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:06.130 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:06.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:06.191 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:06.248 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:06.252 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-26T07:17:06.309 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-26T07:17:06.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-26T07:17:06.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-26T07:17:06.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-26T07:17:06.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-26T07:17:06.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-26T07:17:06.596 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-26T07:17:06.616 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-26T07:17:06.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-26T07:17:06.793 DEBUG:teuthology.orchestra.run.smithi097:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-26T07:17:06.813 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:06.817 DEBUG:teuthology.orchestra.run.smithi097:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-26T07:17:06.861 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-26T07:17:06.862 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-26T07:17:06.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-26T07:17:06.869 INFO:tasks.ceph.mon.a:Started 2023-11-26T07:17:06.870 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-26T07:17:06.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-26T07:17:06.904 INFO:tasks.ceph.mon.b:Started 2023-11-26T07:17:06.904 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-26T07:17:06.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-26T07:17:06.908 INFO:tasks.ceph.mon.c:Started 2023-11-26T07:17:06.908 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-26T07:17:06.908 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-26T07:17:06.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-26T07:17:06.912 INFO:tasks.ceph.mgr.y:Started 2023-11-26T07:17:06.912 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-26T07:17:06.912 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-26T07:17:06.915 INFO:tasks.ceph.mgr.x:Started 2023-11-26T07:17:06.916 INFO:tasks.ceph:Setting crush tunables to default 2023-11-26T07:17:06.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2023-11-26T07:17:07.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.058+0000 7ffba031d640 1 Processor -- start 2023-11-26T07:17:07.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.058+0000 7ffba031d640 1 -- start start 2023-11-26T07:17:07.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.058+0000 7ffba031d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb9815a710 0x7ffb980abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.058+0000 7ffba031d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb980b2710 con 0x7ffb980cdfb0 2023-11-26T07:17:07.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.058+0000 7ffba031d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb980b29f0 con 0x7ffb980cbd50 2023-11-26T07:17:07.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.058+0000 7ffba031d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb980cca80 con 0x7ffb9815a710 2023-11-26T07:17:07.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.058+0000 7ffb9e8ba640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ffb980cbd50 0x7ffb980cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:39710/0 (socket says 172.21.15.31:39710) 2023-11-26T07:17:07.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.058+0000 7ffb9d8b8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffb9815a710 0x7ffb980abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.058+0000 7ffb9e8ba640 1 -- 172.21.15.31:0/3592174196 learned_addr learned my addr 172.21.15.31:0/3592174196 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:07.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.058+0000 7ffb9e0b9640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffb980cdfb0 0x7ffb980ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50520/0 (socket says 172.21.15.31:50520) 2023-11-26T07:17:07.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.062+0000 7ffb9d8b8640 1 -- 172.21.15.31:0/3592174196 >> v2:172.21.15.31:3300/0 conn(0x7ffb9815a710 msgr2=0x7ffb980abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-26T07:17:07.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.062+0000 7ffb9d8b8640 1 -- 172.21.15.31:0/3592174196 >> v2:172.21.15.31:3300/0 conn(0x7ffb9815a710 msgr2=0x7ffb980abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-26T07:17:07.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.062+0000 7ffb9d8b8640 1 --2- 172.21.15.31:0/3592174196 >> v2:172.21.15.31:3300/0 conn(0x7ffb9815a710 0x7ffb980abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-26T07:17:07.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.062+0000 7ffb9d8b8640 1 --2- 172.21.15.31:0/3592174196 >> v2:172.21.15.31:3300/0 conn(0x7ffb9815a710 0x7ffb980abf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-26T07:17:07.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.234+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852440662 0 0) 0x7ffb980b2710 con 0x7ffb980cdfb0 2023-11-26T07:17:07.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.234+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb84003530 con 0x7ffb980cdfb0 2023-11-26T07:17:07.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3508650502 0 0) 0x7ffb84003530 con 0x7ffb980cdfb0 2023-11-26T07:17:07.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 >> v2:172.21.15.31:3300/0 conn(0x7ffb9815a710 msgr2=0x7ffb980abf90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-26T07:17:07.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 --2- 172.21.15.31:0/3592174196 >> v2:172.21.15.31:3300/0 conn(0x7ffb9815a710 0x7ffb980abf90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 >> v1:172.21.15.97:6789/0 conn(0x7ffb980cbd50 legacy=0x7ffb980cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb980b2cd0 con 0x7ffb980cdfb0 2023-11-26T07:17:07.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7ffb94001680 con 0x7ffb980cdfb0 2023-11-26T07:17:07.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 >> v1:172.21.15.31:6789/0 conn(0x7ffb980cdfb0 legacy=0x7ffb980ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb840039d0 0x7ffb84003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840047d0 0x7ffb84004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb840051e0 0x7ffb8400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb980cca80 con 0x7ffb840051e0 2023-11-26T07:17:07.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8400e340 con 0x7ffb840047d0 2023-11-26T07:17:07.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8400e500 con 0x7ffb840039d0 2023-11-26T07:17:07.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9e8ba640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb840051e0 0x7ffb8400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9e0b9640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb840039d0 0x7ffb84003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d8b8640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840047d0 0x7ffb84004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9e8ba640 1 -- 172.21.15.31:0/3592174196 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb840039d0 msgr2=0x7ffb84003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9e8ba640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb840039d0 0x7ffb84003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9e8ba640 1 -- 172.21.15.31:0/3592174196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840047d0 msgr2=0x7ffb84004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9e8ba640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840047d0 0x7ffb84004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9e8ba640 1 -- 172.21.15.31:0/3592174196 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb84003530 con 0x7ffb840051e0 2023-11-26T07:17:07.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.238+0000 7ffb9d8b8640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840047d0 0x7ffb84004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffb9e8ba640 1 -- 172.21.15.31:0/3592174196 reap_dead start 2023-11-26T07:17:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffb9e8ba640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb840051e0 0x7ffb8400dba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c190 tx=0x7ffb9001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24c75146a06bda1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb900a5b10 con 0x7ffb840051e0 2023-11-26T07:17:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb900a5c70 con 0x7ffb840051e0 2023-11-26T07:17:07.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffb9d0b7640 1 -- 172.21.15.31:0/3592174196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb900070d0 con 0x7ffb840051e0 2023-11-26T07:17:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffba031d640 1 -- 172.21.15.31:0/3592174196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb840051e0 msgr2=0x7ffb8400dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffba031d640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb840051e0 0x7ffb8400dba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000c190 tx=0x7ffb9001cc70 comp rx=0 tx=0).stop 2023-11-26T07:17:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffba031d640 1 -- 172.21.15.31:0/3592174196 shutdown_connections 2023-11-26T07:17:07.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffba031d640 1 --2- 172.21.15.31:0/3592174196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb840051e0 0x7ffb8400dba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffba031d640 1 -- 172.21.15.31:0/3592174196 >> 172.21.15.31:0/3592174196 conn(0x7ffb980b0cb0 msgr2=0x7ffb980b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:07.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffba031d640 1 -- 172.21.15.31:0/3592174196 shutdown_connections 2023-11-26T07:17:07.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffba031d640 1 -- 172.21.15.31:0/3592174196 wait complete. 2023-11-26T07:17:07.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffba031d640 1 Processor -- start 2023-11-26T07:17:07.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.242+0000 7ffba031d640 1 -- start start 2023-11-26T07:17:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffba031d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840051e0 0x7ffb981560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffba031d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb98099a30 0x7ffb980acec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffba031d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb98099e90 0x7ffb980ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffba031d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb98156630 con 0x7ffb98099a30 2023-11-26T07:17:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffba031d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9809c3c0 con 0x7ffb840051e0 2023-11-26T07:17:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffba031d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9815bc00 con 0x7ffb98099e90 2023-11-26T07:17:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9d8b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb98099a30 0x7ffb980acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9e0b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840051e0 0x7ffb981560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9d8b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb98099a30 0x7ffb980acec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50850/0 (socket says 172.21.15.31:50850) 2023-11-26T07:17:07.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9e0b9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840051e0 0x7ffb981560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51730/0 (socket says 172.21.15.31:51730) 2023-11-26T07:17:07.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9d8b8640 1 -- 172.21.15.31:0/1464585089 learned_addr learned my addr 172.21.15.31:0/1464585089 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:07.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9e8ba640 1 --2- 172.21.15.31:0/1464585089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb98099e90 0x7ffb980ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9d8b8640 1 -- 172.21.15.31:0/1464585089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb98099e90 msgr2=0x7ffb980ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9d8b8640 1 --2- 172.21.15.31:0/1464585089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb98099e90 0x7ffb980ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9d8b8640 1 -- 172.21.15.31:0/1464585089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840051e0 msgr2=0x7ffb981560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9d8b8640 1 --2- 172.21.15.31:0/1464585089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840051e0 0x7ffb981560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9d8b8640 1 -- 172.21.15.31:0/1464585089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9809c660 con 0x7ffb98099a30 2023-11-26T07:17:07.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.246+0000 7ffb9d8b8640 1 --2- 172.21.15.31:0/1464585089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb98099a30 0x7ffb980acec0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffb88014930 tx=0x7ffb88014df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:07.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.250+0000 7ffb8effd640 1 -- 172.21.15.31:0/1464585089 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88009ab0 con 0x7ffb98099a30 2023-11-26T07:17:07.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.250+0000 7ffb8effd640 1 -- 172.21.15.31:0/1464585089 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb88016040 con 0x7ffb98099a30 2023-11-26T07:17:07.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.250+0000 7ffb8effd640 1 -- 172.21.15.31:0/1464585089 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88009e50 con 0x7ffb98099a30 2023-11-26T07:17:07.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.250+0000 7ffba031d640 1 -- 172.21.15.31:0/1464585089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9809c940 con 0x7ffb98099a30 2023-11-26T07:17:07.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.250+0000 7ffba031d640 1 -- 172.21.15.31:0/1464585089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9809cf00 con 0x7ffb98099a30 2023-11-26T07:17:07.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.250+0000 7ffb8effd640 1 -- 172.21.15.31:0/1464585089 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffb88024740 con 0x7ffb98099a30 2023-11-26T07:17:07.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.250+0000 7ffb8effd640 1 -- 172.21.15.31:0/1464585089 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffb8801aad0 con 0x7ffb98099a30 2023-11-26T07:17:07.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.250+0000 7ffba031d640 1 -- 172.21.15.31:0/1464585089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb980671f0 con 0x7ffb98099a30 2023-11-26T07:17:07.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.266+0000 7ffb8effd640 1 -- 172.21.15.31:0/1464585089 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ffb88029060 con 0x7ffb98099a30 2023-11-26T07:17:07.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.334+0000 7ffba031d640 1 -- 172.21.15.31:0/1464585089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ffb9806b460 con 0x7ffb98099a30 2023-11-26T07:17:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.338+0000 7ffb8effd640 1 -- 172.21.15.31:0/1464585089 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ffb8801f070 con 0x7ffb98099a30 2023-11-26T07:17:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2023-11-26T07:17:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.338+0000 7ffba031d640 1 -- 172.21.15.31:0/1464585089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb98099a30 msgr2=0x7ffb980acec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.338+0000 7ffba031d640 1 --2- 172.21.15.31:0/1464585089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb98099a30 0x7ffb980acec0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffb88014930 tx=0x7ffb88014df0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.338+0000 7ffba031d640 1 -- 172.21.15.31:0/1464585089 shutdown_connections 2023-11-26T07:17:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.338+0000 7ffba031d640 1 --2- 172.21.15.31:0/1464585089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffb98099e90 0x7ffb980ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.338+0000 7ffba031d640 1 --2- 172.21.15.31:0/1464585089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb98099a30 0x7ffb980acec0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.338+0000 7ffba031d640 1 --2- 172.21.15.31:0/1464585089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb840051e0 0x7ffb981560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.338+0000 7ffba031d640 1 -- 172.21.15.31:0/1464585089 >> 172.21.15.31:0/1464585089 conn(0x7ffb980b0cb0 msgr2=0x7ffb980ac470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.338+0000 7ffba031d640 1 -- 172.21.15.31:0/1464585089 shutdown_connections 2023-11-26T07:17:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:07.338+0000 7ffba031d640 1 -- 172.21.15.31:0/1464585089 wait complete. 2023-11-26T07:17:07.362 INFO:tasks.ceph:check_enable_crimson: False 2023-11-26T07:17:07.363 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-26T07:17:07.363 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:07.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-26T07:17:07.384 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:07.384 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-26T07:17:07.450 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:07.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-26T07:17:07.454 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:07.453+0000 7f247b35f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-26T07:17:07.478 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:07.474+0000 7f1d2a377180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-26T07:17:07.498 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:07.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-26T07:17:07.565 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:07.566 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-26T07:17:07.587 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:07.587 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-26T07:17:07.650 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:07.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-26T07:17:07.718 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:07.718 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-26T07:17:07.785 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 0ed44b69-8020-41c4-a5da-ef18bcf6728f 0 2023-11-26T07:17:07.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f07a48fe640 1 Processor -- start 2023-11-26T07:17:07.960 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f07a48fe640 1 -- start start 2023-11-26T07:17:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f07a48fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07a015d8f0 0x7f07a015e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f07a48fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f07a48fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f07a00cf1b0 0x7f07a00d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f07a48fe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a00d4420 con 0x7f07a00cf1b0 2023-11-26T07:17:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f07a48fe640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a015ecc0 con 0x7f07a015d8f0 2023-11-26T07:17:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f07a48fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00cdfb0 con 0x7f07a00ced30 2023-11-26T07:17:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f079e59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07a015d8f0 0x7f07a015e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f0797fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.963 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f0797fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a00cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:35072/0 (socket says 172.21.15.97:35072) 2023-11-26T07:17:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f0797fff640 1 -- 172.21.15.97:0/150771562 learned_addr learned my addr 172.21.15.97:0/150771562 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f079e59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f07a015d8f0 0x7f07a015e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46524/0 (socket says 172.21.15.97:46524) 2023-11-26T07:17:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.957+0000 7f079ed9d640 1 --2- 172.21.15.97:0/150771562 >> v2:172.21.15.31:3300/0 conn(0x7f07a00cf1b0 0x7f07a00d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f0797fff640 1 -- 172.21.15.97:0/150771562 >> v2:172.21.15.97:3300/0 conn(0x7f07a015d8f0 msgr2=0x7f07a015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f0797fff640 1 --2- 172.21.15.97:0/150771562 >> v2:172.21.15.97:3300/0 conn(0x7f07a015d8f0 0x7f07a015e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.964 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f0797fff640 1 -- 172.21.15.97:0/150771562 >> v2:172.21.15.31:3300/0 conn(0x7f07a00cf1b0 msgr2=0x7f07a00d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:07.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f0797fff640 1 --2- 172.21.15.97:0/150771562 >> v2:172.21.15.31:3300/0 conn(0x7f07a00cf1b0 0x7f07a00d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f0797fff640 1 -- 172.21.15.97:0/150771562 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a00d4140 con 0x7f07a00ced30 2023-11-26T07:17:07.965 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f079e59c640 1 --2- 172.21.15.97:0/150771562 >> v2:172.21.15.97:3300/0 conn(0x7f07a015d8f0 0x7f07a015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:07.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f0797fff640 1 --2- 172.21.15.97:0/150771562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a00cfe00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f07a008b800 tx=0x7f078801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3519f7eba35fe657 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:07.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f079dd9b640 1 -- 172.21.15.97:0/150771562 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880aa5f0 con 0x7f07a00ced30 2023-11-26T07:17:07.966 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f079dd9b640 1 -- 172.21.15.97:0/150771562 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07880aa750 con 0x7f07a00ced30 2023-11-26T07:17:07.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f079dd9b640 1 -- 172.21.15.97:0/150771562 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880aaae0 con 0x7f07a00ced30 2023-11-26T07:17:07.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f07a48fe640 1 -- 172.21.15.97:0/150771562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 msgr2=0x7f07a00cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.961+0000 7f07a48fe640 1 --2- 172.21.15.97:0/150771562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a00cfe00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f07a008b800 tx=0x7f078801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 -- 172.21.15.97:0/150771562 shutdown_connections 2023-11-26T07:17:07.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 --2- 172.21.15.97:0/150771562 >> v2:172.21.15.31:3300/0 conn(0x7f07a00cf1b0 0x7f07a00d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.967 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 --2- 172.21.15.97:0/150771562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a00cfe00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 --2- 172.21.15.97:0/150771562 >> v2:172.21.15.97:3300/0 conn(0x7f07a015d8f0 0x7f07a015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 -- 172.21.15.97:0/150771562 >> 172.21.15.97:0/150771562 conn(0x7f07a009c750 msgr2=0x7f07a00d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:07.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 -- 172.21.15.97:0/150771562 shutdown_connections 2023-11-26T07:17:07.968 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 -- 172.21.15.97:0/150771562 wait complete. 2023-11-26T07:17:07.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 Processor -- start 2023-11-26T07:17:07.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 -- start start 2023-11-26T07:17:07.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a0150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a00cf1b0 0x7f07a00b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07a015d8f0 0x7f07a01509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:07.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b0a90 con 0x7f07a00cf1b0 2023-11-26T07:17:07.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b0d70 con 0x7f07a015d8f0 2023-11-26T07:17:07.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f07a48fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a00b1050 con 0x7f07a00ced30 2023-11-26T07:17:07.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079e59c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a0150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07a015d8f0 0x7f07a01509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079e59c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a0150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:35086/0 (socket says 172.21.15.97:35086) 2023-11-26T07:17:07.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07a015d8f0 0x7f07a01509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46538/0 (socket says 172.21.15.97:46538) 2023-11-26T07:17:07.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079e59c640 1 -- 172.21.15.97:0/1894962186 learned_addr learned my addr 172.21.15.97:0/1894962186 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f0797fff640 1 --2- 172.21.15.97:0/1894962186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a00cf1b0 0x7f07a00b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079e59c640 1 -- 172.21.15.97:0/1894962186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07a015d8f0 msgr2=0x7f07a01509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079e59c640 1 --2- 172.21.15.97:0/1894962186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07a015d8f0 0x7f07a01509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079e59c640 1 -- 172.21.15.97:0/1894962186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a00cf1b0 msgr2=0x7f07a00b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079e59c640 1 --2- 172.21.15.97:0/1894962186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a00cf1b0 0x7f07a00b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079e59c640 1 -- 172.21.15.97:0/1894962186 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0790009420 con 0x7f07a00ced30 2023-11-26T07:17:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.965+0000 7f079ed9d640 1 --2- 172.21.15.97:0/1894962186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07a015d8f0 0x7f07a01509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.969+0000 7f079e59c640 1 --2- 172.21.15.97:0/1894962186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a0150210 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f079000ebe0 tx=0x7f079000c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.969+0000 7f0796ffd640 1 -- 172.21.15.97:0/1894962186 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079000eee0 con 0x7f07a00ced30 2023-11-26T07:17:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.969+0000 7f0796ffd640 1 -- 172.21.15.97:0/1894962186 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079000cc30 con 0x7f07a00ced30 2023-11-26T07:17:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.969+0000 7f07a48fe640 1 -- 172.21.15.97:0/1894962186 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07880094d0 con 0x7f07a00ced30 2023-11-26T07:17:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.969+0000 7f0796ffd640 1 -- 172.21.15.97:0/1894962186 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790013470 con 0x7f07a00ced30 2023-11-26T07:17:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.969+0000 7f07a48fe640 1 -- 172.21.15.97:0/1894962186 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a00b1610 con 0x7f07a00ced30 2023-11-26T07:17:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.969+0000 7f0796ffd640 1 -- 172.21.15.97:0/1894962186 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0790005ca0 con 0x7f07a00ced30 2023-11-26T07:17:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.969+0000 7f0796ffd640 1 -- 172.21.15.97:0/1894962186 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0790013910 con 0x7f07a00ced30 2023-11-26T07:17:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.969+0000 7f07a48fe640 1 -- 172.21.15.97:0/1894962186 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a004e850 con 0x7f07a00ced30 2023-11-26T07:17:07.985 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:07.981+0000 7f0796ffd640 1 -- 172.21.15.97:0/1894962186 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f0790021060 con 0x7f07a00ced30 2023-11-26T07:17:08.050 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.049+0000 7f07a48fe640 1 -- 172.21.15.97:0/1894962186 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0ed44b69-8020-41c4-a5da-ef18bcf6728f", "id": 0} v 0) v1 -- 0x7f07a00b5d10 con 0x7f07a00ced30 2023-11-26T07:17:08.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.053+0000 7f0796ffd640 1 -- 172.21.15.97:0/1894962186 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0ed44b69-8020-41c4-a5da-ef18bcf6728f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0790029020 con 0x7f07a00ced30 2023-11-26T07:17:08.057 INFO:teuthology.orchestra.run.smithi097.stdout:0 2023-11-26T07:17:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.053+0000 7f07a48fe640 1 -- 172.21.15.97:0/1894962186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 msgr2=0x7f07a0150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.053+0000 7f07a48fe640 1 --2- 172.21.15.97:0/1894962186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a0150210 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f079000ebe0 tx=0x7f079000c520 comp rx=0 tx=0).stop 2023-11-26T07:17:08.058 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.057+0000 7f07a48fe640 1 -- 172.21.15.97:0/1894962186 shutdown_connections 2023-11-26T07:17:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.057+0000 7f07a48fe640 1 --2- 172.21.15.97:0/1894962186 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07a015d8f0 0x7f07a01509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.057+0000 7f07a48fe640 1 --2- 172.21.15.97:0/1894962186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07a00cf1b0 0x7f07a00b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.057+0000 7f07a48fe640 1 --2- 172.21.15.97:0/1894962186 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f07a00ced30 0x7f07a0150210 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.057+0000 7f07a48fe640 1 -- 172.21.15.97:0/1894962186 >> 172.21.15.97:0/1894962186 conn(0x7f07a009c750 msgr2=0x7f07a0159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.057+0000 7f07a48fe640 1 -- 172.21.15.97:0/1894962186 shutdown_connections 2023-11-26T07:17:08.059 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.057+0000 7f07a48fe640 1 -- 172.21.15.97:0/1894962186 wait complete. 2023-11-26T07:17:08.060 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:08.054+0000 7f1d2a377180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-26T07:17:08.065 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:08.061+0000 7f247b35f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-26T07:17:08.077 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 27567e0b-06aa-4c17-b136-1fc946605813 1 2023-11-26T07:17:08.213 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.209+0000 7f26658fe640 1 Processor -- start 2023-11-26T07:17:08.214 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.209+0000 7f26658fe640 1 -- start start 2023-11-26T07:17:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f26658fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f266015d950 0x7f26600d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f26658fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 0x7f266006c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f26658fe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26600cef60 con 0x7f26600ce9f0 2023-11-26T07:17:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f26658fe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26600cf240 con 0x7f266015d950 2023-11-26T07:17:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f26658fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266006b710 con 0x7f26600d1bf0 2023-11-26T07:17:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f265f7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 0x7f266006c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f265e7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f26600ce9f0 0x7f26600d14e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46516/0 (socket says 172.21.15.97:46516) 2023-11-26T07:17:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f265e7fc640 1 -- 172.21.15.97:0/3346827028 learned_addr learned my addr 172.21.15.97:0/3346827028 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-26T07:17:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f265effd640 1 --2- 172.21.15.97:0/3346827028 >> v2:172.21.15.31:3300/0 conn(0x7f266015d950 0x7f26600d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.216 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f265effd640 1 -- 172.21.15.97:0/3346827028 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 msgr2=0x7f266006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f265effd640 1 --2- 172.21.15.97:0/3346827028 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 0x7f266006c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f265effd640 1 -- 172.21.15.97:0/3346827028 >> v1:172.21.15.97:6789/0 conn(0x7f26600ce9f0 legacy=0x7f26600d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f265effd640 1 -- 172.21.15.97:0/3346827028 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266006b9f0 con 0x7f266015d950 2023-11-26T07:17:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f265f7fe640 1 --2- 172.21.15.97:0/3346827028 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 0x7f266006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:08.217 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.213+0000 7f265dffb640 1 -- 172.21.15.97:0/3346827028 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114496602 0 0) 0x7f26600cef60 con 0x7f26600ce9f0 2023-11-26T07:17:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f265effd640 1 --2- 172.21.15.97:0/3346827028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266015d950 0x7f26600d3ca0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f26500097c0 tx=0x7f265001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bcb20225a643a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f265dffb640 1 -- 172.21.15.97:0/3346827028 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26500ad660 con 0x7f266015d950 2023-11-26T07:17:08.219 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f265dffb640 1 -- 172.21.15.97:0/3346827028 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26500ad7c0 con 0x7f266015d950 2023-11-26T07:17:08.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f265dffb640 1 -- 172.21.15.97:0/3346827028 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26500adb50 con 0x7f266015d950 2023-11-26T07:17:08.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f26658fe640 1 -- 172.21.15.97:0/3346827028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266015d950 msgr2=0x7f26600d3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f26658fe640 1 --2- 172.21.15.97:0/3346827028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266015d950 0x7f26600d3ca0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f26500097c0 tx=0x7f265001c450 comp rx=0 tx=0).stop 2023-11-26T07:17:08.220 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f26658fe640 1 -- 172.21.15.97:0/3346827028 shutdown_connections 2023-11-26T07:17:08.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f26658fe640 1 --2- 172.21.15.97:0/3346827028 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 0x7f266006c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f26658fe640 1 --2- 172.21.15.97:0/3346827028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266015d950 0x7f26600d3ca0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f26658fe640 1 -- 172.21.15.97:0/3346827028 >> 172.21.15.97:0/3346827028 conn(0x7f266008afc0 msgr2=0x7f26600d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f26658fe640 1 -- 172.21.15.97:0/3346827028 shutdown_connections 2023-11-26T07:17:08.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f26658fe640 1 -- 172.21.15.97:0/3346827028 wait complete. 2023-11-26T07:17:08.221 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.217+0000 7f26658fe640 1 Processor -- start 2023-11-26T07:17:08.222 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f26658fe640 1 -- start start 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f26658fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26600ce9f0 0x7f26600ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f26658fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 0x7f266014b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f26658fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2660157e00 0x7f266014baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f26658fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2660158f60 con 0x7f2660157e00 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f26658fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2660158350 con 0x7f26600ce9f0 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f26658fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2660158630 con 0x7f26600d1bf0 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26600ce9f0 0x7f26600ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265e7fc640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 0x7f266014b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26600ce9f0 0x7f26600ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46554/0 (socket says 172.21.15.97:46554) 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265effd640 1 -- 172.21.15.97:0/1194364975 learned_addr learned my addr 172.21.15.97:0/1194364975 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:08.224 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265f7fe640 1 --2- 172.21.15.97:0/1194364975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2660157e00 0x7f266014baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265effd640 1 -- 172.21.15.97:0/1194364975 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 msgr2=0x7f266014b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265effd640 1 --2- 172.21.15.97:0/1194364975 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 0x7f266014b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265effd640 1 -- 172.21.15.97:0/1194364975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2660157e00 msgr2=0x7f266014baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265effd640 1 --2- 172.21.15.97:0/1194364975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2660157e00 0x7f266014baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.225 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265effd640 1 -- 172.21.15.97:0/1194364975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2658009a70 con 0x7f26600ce9f0 2023-11-26T07:17:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f265effd640 1 --2- 172.21.15.97:0/1194364975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26600ce9f0 0x7f26600ccf10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26500b0040 tx=0x7f26500191f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f2647fff640 1 -- 172.21.15.97:0/1194364975 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26500b6610 con 0x7f26600ce9f0 2023-11-26T07:17:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f2647fff640 1 -- 172.21.15.97:0/1194364975 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26500b6790 con 0x7f26600ce9f0 2023-11-26T07:17:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f2647fff640 1 -- 172.21.15.97:0/1194364975 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26500b6b40 con 0x7f26600ce9f0 2023-11-26T07:17:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f26658fe640 1 -- 172.21.15.97:0/1194364975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2650009420 con 0x7f26600ce9f0 2023-11-26T07:17:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f26658fe640 1 -- 172.21.15.97:0/1194364975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2660158bf0 con 0x7f26600ce9f0 2023-11-26T07:17:08.226 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f2647fff640 1 -- 172.21.15.97:0/1194364975 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f26500b4040 con 0x7f26600ce9f0 2023-11-26T07:17:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f2647fff640 1 -- 172.21.15.97:0/1194364975 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f26500c13c0 con 0x7f26600ce9f0 2023-11-26T07:17:08.227 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.221+0000 7f26658fe640 1 -- 172.21.15.97:0/1194364975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266004e850 con 0x7f26600ce9f0 2023-11-26T07:17:08.228 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.225+0000 7f2647fff640 1 -- 172.21.15.97:0/1194364975 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f26500b2070 con 0x7f26600ce9f0 2023-11-26T07:17:08.265 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:08.262+0000 7f1d2a377180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-26T07:17:08.274 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:08.269+0000 7f247b35f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-26T07:17:08.297 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.293+0000 7f26658fe640 1 -- 172.21.15.97:0/1194364975 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "27567e0b-06aa-4c17-b136-1fc946605813", "id": 1} v 0) v1 -- 0x7f26600b2b10 con 0x7f26600ce9f0 2023-11-26T07:17:08.306 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.301+0000 7f2647fff640 1 -- 172.21.15.97:0/1194364975 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "27567e0b-06aa-4c17-b136-1fc946605813", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f26500c2460 con 0x7f26600ce9f0 2023-11-26T07:17:08.306 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-26T07:17:08.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.305+0000 7f26658fe640 1 -- 172.21.15.97:0/1194364975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26600ce9f0 msgr2=0x7f26600ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.305+0000 7f26658fe640 1 --2- 172.21.15.97:0/1194364975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26600ce9f0 0x7f26600ccf10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f26500b0040 tx=0x7f26500191f0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.305+0000 7f26658fe640 1 -- 172.21.15.97:0/1194364975 shutdown_connections 2023-11-26T07:17:08.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.305+0000 7f26658fe640 1 --2- 172.21.15.97:0/1194364975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2660157e00 0x7f266014baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.307 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.305+0000 7f26658fe640 1 --2- 172.21.15.97:0/1194364975 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26600d1bf0 0x7f266014b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.305+0000 7f26658fe640 1 --2- 172.21.15.97:0/1194364975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26600ce9f0 0x7f26600ccf10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.305+0000 7f26658fe640 1 -- 172.21.15.97:0/1194364975 >> 172.21.15.97:0/1194364975 conn(0x7f266008afc0 msgr2=0x7f26600b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.305+0000 7f26658fe640 1 -- 172.21.15.97:0/1194364975 shutdown_connections 2023-11-26T07:17:08.308 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.305+0000 7f26658fe640 1 -- 172.21.15.97:0/1194364975 wait complete. 2023-11-26T07:17:08.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new bb48a666-f258-451b-93a0-90c77a44d49f 2 2023-11-26T07:17:08.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f9420765640 1 Processor -- start 2023-11-26T07:17:08.460 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f9420765640 1 -- start start 2023-11-26T07:17:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f9420765640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f941815d8f0 0x7f941815e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f9420765640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94180ced30 0x7f94180cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f9420765640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941815ecc0 con 0x7f94180cf1e0 2023-11-26T07:17:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f9420765640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94180d4420 con 0x7f941815d8f0 2023-11-26T07:17:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f9420765640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94180cdfb0 con 0x7f94180ced30 2023-11-26T07:17:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f941dd00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94180ced30 0x7f94180cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f941dd00640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94180ced30 0x7f94180cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46560/0 (socket says 172.21.15.97:46560) 2023-11-26T07:17:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f941dd00640 1 -- 172.21.15.97:0/3677562720 learned_addr learned my addr 172.21.15.97:0/3677562720 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:08.462 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.457+0000 7f941e501640 1 --2- 172.21.15.97:0/3677562720 >> v2:172.21.15.31:3300/0 conn(0x7f941815d8f0 0x7f941815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.461+0000 7f941e501640 1 -- 172.21.15.97:0/3677562720 >> v2:172.21.15.97:3300/0 conn(0x7f94180ced30 msgr2=0x7f94180cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.461+0000 7f941e501640 1 --2- 172.21.15.97:0/3677562720 >> v2:172.21.15.97:3300/0 conn(0x7f94180ced30 0x7f94180cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.461+0000 7f941e501640 1 -- 172.21.15.97:0/3677562720 >> v1:172.21.15.31:6789/0 conn(0x7f94180cf1e0 legacy=0x7f94180d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.461+0000 7f941e501640 1 -- 172.21.15.97:0/3677562720 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180d4140 con 0x7f941815d8f0 2023-11-26T07:17:08.463 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.461+0000 7f941dd00640 1 --2- 172.21.15.97:0/3677562720 >> v2:172.21.15.97:3300/0 conn(0x7f94180ced30 0x7f94180cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.461+0000 7f941e501640 1 --2- 172.21.15.97:0/3677562720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f941815d8f0 0x7f941815e780 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94140097c0 tx=0x7f941401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f00a35129c406a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.461+0000 7f941d4ff640 1 -- 172.21.15.97:0/3677562720 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94140ab610 con 0x7f941815d8f0 2023-11-26T07:17:08.465 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.461+0000 7f941d4ff640 1 -- 172.21.15.97:0/3677562720 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94140ab770 con 0x7f941815d8f0 2023-11-26T07:17:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.461+0000 7f941d4ff640 1 -- 172.21.15.97:0/3677562720 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94140abb00 con 0x7f941815d8f0 2023-11-26T07:17:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 -- 172.21.15.97:0/3677562720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f941815d8f0 msgr2=0x7f941815e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 --2- 172.21.15.97:0/3677562720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f941815d8f0 0x7f941815e780 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94140097c0 tx=0x7f941401c410 comp rx=0 tx=0).stop 2023-11-26T07:17:08.466 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 -- 172.21.15.97:0/3677562720 shutdown_connections 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 --2- 172.21.15.97:0/3677562720 >> v2:172.21.15.97:3300/0 conn(0x7f94180ced30 0x7f94180cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 --2- 172.21.15.97:0/3677562720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f941815d8f0 0x7f941815e780 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 -- 172.21.15.97:0/3677562720 >> 172.21.15.97:0/3677562720 conn(0x7f941809c750 msgr2=0x7f94180d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 -- 172.21.15.97:0/3677562720 shutdown_connections 2023-11-26T07:17:08.467 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 -- 172.21.15.97:0/3677562720 wait complete. 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 Processor -- start 2023-11-26T07:17:08.468 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 -- start start 2023-11-26T07:17:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94180ced30 0x7f94181f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94180cf1e0 0x7f941815a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f941815d8f0 0x7f941809d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9418148300 con 0x7f94180ced30 2023-11-26T07:17:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94181f8180 con 0x7f94180cf1e0 2023-11-26T07:17:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f9420765640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94181476f0 con 0x7f941815d8f0 2023-11-26T07:17:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f941dd00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94180cf1e0 0x7f941815a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f941e501640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94180ced30 0x7f94181f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f941ed02640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f941815d8f0 0x7f941809d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.470 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f941e501640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94180ced30 0x7f94181f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.97:48122/0 (socket says 172.21.15.97:48122) 2023-11-26T07:17:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f941e501640 1 -- 172.21.15.97:0/3588932235 learned_addr learned my addr 172.21.15.97:0/3588932235 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f941e501640 1 -- 172.21.15.97:0/3588932235 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f941815d8f0 msgr2=0x7f941809d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f941e501640 1 --2- 172.21.15.97:0/3588932235 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f941815d8f0 0x7f941809d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f941e501640 1 -- 172.21.15.97:0/3588932235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94180cf1e0 msgr2=0x7f941815a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f941e501640 1 --2- 172.21.15.97:0/3588932235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94180cf1e0 0x7f941815a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.471 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.465+0000 7f941e501640 1 -- 172.21.15.97:0/3588932235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94080094d0 con 0x7f94180ced30 2023-11-26T07:17:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.469+0000 7f941e501640 1 --2- 172.21.15.97:0/3588932235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94180ced30 0x7f94181f7c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f94140ad040 tx=0x7f941401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.469+0000 7f940effd640 1 -- 172.21.15.97:0/3588932235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94140bf610 con 0x7f94180ced30 2023-11-26T07:17:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.469+0000 7f940effd640 1 -- 172.21.15.97:0/3588932235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94140bf770 con 0x7f94180ced30 2023-11-26T07:17:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.469+0000 7f940effd640 1 -- 172.21.15.97:0/3588932235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94140bfb00 con 0x7f94180ced30 2023-11-26T07:17:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.469+0000 7f9420765640 1 -- 172.21.15.97:0/3588932235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9414009420 con 0x7f94180ced30 2023-11-26T07:17:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.469+0000 7f9420765640 1 -- 172.21.15.97:0/3588932235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9418147cb0 con 0x7f94180ced30 2023-11-26T07:17:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.469+0000 7f940effd640 1 -- 172.21.15.97:0/3588932235 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f94140b5400 con 0x7f94180ced30 2023-11-26T07:17:08.472 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.469+0000 7f940effd640 1 -- 172.21.15.97:0/3588932235 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f94140b4390 con 0x7f94180ced30 2023-11-26T07:17:08.473 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.469+0000 7f9420765640 1 -- 172.21.15.97:0/3588932235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94180cdc20 con 0x7f94180ced30 2023-11-26T07:17:08.474 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.473+0000 7f940effd640 1 -- 172.21.15.97:0/3588932235 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f94140b57a0 con 0x7f94180ced30 2023-11-26T07:17:08.540 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.537+0000 7f9420765640 1 -- 172.21.15.97:0/3588932235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bb48a666-f258-451b-93a0-90c77a44d49f", "id": 2} v 0) v1 -- 0x7f941804e850 con 0x7f94180ced30 2023-11-26T07:17:08.544 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.541+0000 7f940effd640 1 -- 172.21.15.97:0/3588932235 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bb48a666-f258-451b-93a0-90c77a44d49f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f94140b59c0 con 0x7f94180ced30 2023-11-26T07:17:08.544 INFO:teuthology.orchestra.run.smithi097.stdout:2 2023-11-26T07:17:08.545 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.541+0000 7f9420765640 1 -- 172.21.15.97:0/3588932235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94180ced30 msgr2=0x7f94181f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.541+0000 7f9420765640 1 --2- 172.21.15.97:0/3588932235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94180ced30 0x7f94181f7c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f94140ad040 tx=0x7f941401ac50 comp rx=0 tx=0).stop 2023-11-26T07:17:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.541+0000 7f9420765640 1 -- 172.21.15.97:0/3588932235 shutdown_connections 2023-11-26T07:17:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.541+0000 7f9420765640 1 --2- 172.21.15.97:0/3588932235 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f941815d8f0 0x7f941809d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.541+0000 7f9420765640 1 --2- 172.21.15.97:0/3588932235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94180cf1e0 0x7f941815a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.541+0000 7f9420765640 1 --2- 172.21.15.97:0/3588932235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94180ced30 0x7f94181f7c40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.541+0000 7f9420765640 1 -- 172.21.15.97:0/3588932235 >> 172.21.15.97:0/3588932235 conn(0x7f941809c750 msgr2=0x7f941815dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.541+0000 7f9420765640 1 -- 172.21.15.97:0/3588932235 shutdown_connections 2023-11-26T07:17:08.546 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.541+0000 7f9420765640 1 -- 172.21.15.97:0/3588932235 wait complete. 2023-11-26T07:17:08.564 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 5824e328-a264-4106-85ba-46428653e39b 3 2023-11-26T07:17:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.697+0000 7ff7cd780640 1 Processor -- start 2023-11-26T07:17:08.701 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.697+0000 7ff7cd780640 1 -- start start 2023-11-26T07:17:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.697+0000 7ff7cd780640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff7c80ce9f0 0x7ff7c80d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.697+0000 7ff7cd780640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80d1a20 0x7ff7c806c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.697+0000 7ff7cd780640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c80cef60 con 0x7ff7c815d950 2023-11-26T07:17:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.697+0000 7ff7cd780640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c80d43b0 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.697+0000 7ff7cd780640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80d1e90 con 0x7ff7c80d1a20 2023-11-26T07:17:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c6ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff7c815d950 0x7ff7c80d3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46522/0 (socket says 172.21.15.97:46522) 2023-11-26T07:17:08.703 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c6ffd640 1 -- 172.21.15.97:0/141733403 learned_addr learned my addr 172.21.15.97:0/141733403 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-26T07:17:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c67fc640 1 --2- 172.21.15.97:0/141733403 >> v2:172.21.15.31:3300/0 conn(0x7ff7c80ce9f0 0x7ff7c80d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c77fe640 1 --2- 172.21.15.97:0/141733403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80d1a20 0x7ff7c806c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c5ffb640 1 -- 172.21.15.97:0/141733403 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452490730 0 0) 0x7ff7c80cef60 con 0x7ff7c815d950 2023-11-26T07:17:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c5ffb640 1 -- 172.21.15.97:0/141733403 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b0003530 con 0x7ff7c815d950 2023-11-26T07:17:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c67fc640 1 -- 172.21.15.97:0/141733403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80d1a20 msgr2=0x7ff7c806c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c67fc640 1 --2- 172.21.15.97:0/141733403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80d1a20 0x7ff7c806c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c67fc640 1 -- 172.21.15.97:0/141733403 >> v1:172.21.15.97:6789/0 conn(0x7ff7c815d950 legacy=0x7ff7c80d3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.704 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c67fc640 1 -- 172.21.15.97:0/141733403 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c80cf240 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.705 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.701+0000 7ff7c67fc640 1 --2- 172.21.15.97:0/141733403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c80ce9f0 0x7ff7c80d14e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0097f0 tx=0x7ff7bc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be66b9ed9afd402 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7c5ffb640 1 -- 172.21.15.97:0/141733403 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0ab650 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7c5ffb640 1 -- 172.21.15.97:0/141733403 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc0ab7b0 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.706 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7c5ffb640 1 -- 172.21.15.97:0/141733403 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0abb40 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 -- 172.21.15.97:0/141733403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c80ce9f0 msgr2=0x7ff7c80d14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 --2- 172.21.15.97:0/141733403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c80ce9f0 0x7ff7c80d14e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0097f0 tx=0x7ff7bc01c450 comp rx=0 tx=0).stop 2023-11-26T07:17:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 -- 172.21.15.97:0/141733403 shutdown_connections 2023-11-26T07:17:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 --2- 172.21.15.97:0/141733403 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80d1a20 0x7ff7c806c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 --2- 172.21.15.97:0/141733403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c80ce9f0 0x7ff7c80d14e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 -- 172.21.15.97:0/141733403 >> 172.21.15.97:0/141733403 conn(0x7ff7c808afc0 msgr2=0x7ff7c80d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 -- 172.21.15.97:0/141733403 shutdown_connections 2023-11-26T07:17:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 -- 172.21.15.97:0/141733403 wait complete. 2023-11-26T07:17:08.708 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 Processor -- start 2023-11-26T07:17:08.709 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 -- start start 2023-11-26T07:17:08.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80ce9f0 0x7ff7c814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7c80d1a20 0x7ff7c80b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.711 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c80b0ea0 0x7ff7c814b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c80b1330 con 0x7ff7c80b0ea0 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c814b330 con 0x7ff7c80d1a20 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7cd780640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c814c850 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7c6ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80ce9f0 0x7ff7c814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7c6ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80ce9f0 0x7ff7c814ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:35126/0 (socket says 172.21.15.97:35126) 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7c6ffd640 1 -- 172.21.15.97:0/2657861664 learned_addr learned my addr 172.21.15.97:0/2657861664 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7c67fc640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7c80d1a20 0x7ff7c80b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.705+0000 7ff7c77fe640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c80b0ea0 0x7ff7c814b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7c6ffd640 1 -- 172.21.15.97:0/2657861664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7c80d1a20 msgr2=0x7ff7c80b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7c6ffd640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7c80d1a20 0x7ff7c80b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7c6ffd640 1 -- 172.21.15.97:0/2657861664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c80b0ea0 msgr2=0x7ff7c814b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7c6ffd640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c80b0ea0 0x7ff7c814b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7c6ffd640 1 -- 172.21.15.97:0/2657861664 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc009420 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7c67fc640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7c80d1a20 0x7ff7c80b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7c77fe640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c80b0ea0 0x7ff7c814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7c6ffd640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80ce9f0 0x7ff7c814ff60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800c450 tx=0x7ff7b800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7af7fe640 1 -- 172.21.15.97:0/2657861664 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8005cf0 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7af7fe640 1 -- 172.21.15.97:0/2657861664 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b8005e50 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7cd780640 1 -- 172.21.15.97:0/2657861664 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c814c290 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7af7fe640 1 -- 172.21.15.97:0/2657861664 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80072d0 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7cd780640 1 -- 172.21.15.97:0/2657861664 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c80b2e60 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7af7fe640 1 -- 172.21.15.97:0/2657861664 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff7b801c440 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7af7fe640 1 -- 172.21.15.97:0/2657861664 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7ff7b801b840 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.709+0000 7ff7cd780640 1 -- 172.21.15.97:0/2657861664 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c804e850 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.714 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.713+0000 7ff7af7fe640 1 -- 172.21.15.97:0/2657861664 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff7b8017070 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.781 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.777+0000 7ff7cd780640 1 -- 172.21.15.97:0/2657861664 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5824e328-a264-4106-85ba-46428653e39b", "id": 3} v 0) v1 -- 0x7ff7c80ca240 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.787 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.785+0000 7ff7af7fe640 1 -- 172.21.15.97:0/2657861664 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5824e328-a264-4106-85ba-46428653e39b", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff7b801b5a0 con 0x7ff7c80ce9f0 2023-11-26T07:17:08.788 INFO:teuthology.orchestra.run.smithi097.stdout:3 2023-11-26T07:17:08.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.785+0000 7ff7cd780640 1 -- 172.21.15.97:0/2657861664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80ce9f0 msgr2=0x7ff7c814ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.788 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.785+0000 7ff7cd780640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80ce9f0 0x7ff7c814ff60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800c450 tx=0x7ff7b800c910 comp rx=0 tx=0).stop 2023-11-26T07:17:08.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.785+0000 7ff7cd780640 1 -- 172.21.15.97:0/2657861664 shutdown_connections 2023-11-26T07:17:08.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.785+0000 7ff7cd780640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c80b0ea0 0x7ff7c814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.785+0000 7ff7cd780640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7c80d1a20 0x7ff7c80b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.785+0000 7ff7cd780640 1 --2- 172.21.15.97:0/2657861664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7c80ce9f0 0x7ff7c814ff60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.785+0000 7ff7cd780640 1 -- 172.21.15.97:0/2657861664 >> 172.21.15.97:0/2657861664 conn(0x7ff7c808afc0 msgr2=0x7ff7c80cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.785+0000 7ff7cd780640 1 -- 172.21.15.97:0/2657861664 shutdown_connections 2023-11-26T07:17:08.789 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.785+0000 7ff7cd780640 1 -- 172.21.15.97:0/2657861664 wait complete. 2023-11-26T07:17:08.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 85a5d6b2-f995-4478-8add-968a03c0baa9 4 2023-11-26T07:17:08.923 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:08.921+0000 7f247b35f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-26T07:17:08.930 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:08.926+0000 7f1d2a377180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-26T07:17:08.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7fade5da1640 1 Processor -- start 2023-11-26T07:17:08.944 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7fade5da1640 1 -- start start 2023-11-26T07:17:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7fade5da1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fade015d8f0 0x7fade015e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7fade5da1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade00ced30 0x7fade00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7fade5da1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fade00cf1b0 0x7fade00d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7fade5da1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade00d4420 con 0x7fade015d8f0 2023-11-26T07:17:08.946 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7fade5da1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade015ecc0 con 0x7fade00cf1b0 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7fade5da1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade00cdfb0 con 0x7fade00ced30 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7faddeffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade00ced30 0x7fade00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7faddffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fade00cf1b0 0x7fade00d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7faddeffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade00ced30 0x7fade00cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:35138/0 (socket says 172.21.15.97:35138) 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7faddeffd640 1 -- 172.21.15.97:0/1043590258 learned_addr learned my addr 172.21.15.97:0/1043590258 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.941+0000 7faddf7fe640 1 --2- 172.21.15.97:0/1043590258 >> v2:172.21.15.31:3300/0 conn(0x7fade015d8f0 0x7fade015e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7faddeffd640 1 -- 172.21.15.97:0/1043590258 >> v2:172.21.15.97:3300/0 conn(0x7fade00cf1b0 msgr2=0x7fade00d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7faddeffd640 1 --2- 172.21.15.97:0/1043590258 >> v2:172.21.15.97:3300/0 conn(0x7fade00cf1b0 0x7fade00d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7faddeffd640 1 -- 172.21.15.97:0/1043590258 >> v2:172.21.15.31:3300/0 conn(0x7fade015d8f0 msgr2=0x7fade015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7faddeffd640 1 --2- 172.21.15.97:0/1043590258 >> v2:172.21.15.31:3300/0 conn(0x7fade015d8f0 0x7fade015e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.947 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7faddeffd640 1 -- 172.21.15.97:0/1043590258 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade00d4140 con 0x7fade00ced30 2023-11-26T07:17:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7faddeffd640 1 --2- 172.21.15.97:0/1043590258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade00ced30 0x7fade00cfe00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fade00b21f0 tx=0x7fadd401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4668453a3591ff95 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7fadde7fc640 1 -- 172.21.15.97:0/1043590258 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40aa5f0 con 0x7fade00ced30 2023-11-26T07:17:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7fadde7fc640 1 -- 172.21.15.97:0/1043590258 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd40aa750 con 0x7fade00ced30 2023-11-26T07:17:08.949 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7fadde7fc640 1 -- 172.21.15.97:0/1043590258 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40aaae0 con 0x7fade00ced30 2023-11-26T07:17:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7fade5da1640 1 -- 172.21.15.97:0/1043590258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade00ced30 msgr2=0x7fade00cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.945+0000 7fade5da1640 1 --2- 172.21.15.97:0/1043590258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade00ced30 0x7fade00cfe00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fade00b21f0 tx=0x7fadd401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 -- 172.21.15.97:0/1043590258 shutdown_connections 2023-11-26T07:17:08.950 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 --2- 172.21.15.97:0/1043590258 >> v2:172.21.15.97:3300/0 conn(0x7fade00cf1b0 0x7fade00d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 --2- 172.21.15.97:0/1043590258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade00ced30 0x7fade00cfe00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 --2- 172.21.15.97:0/1043590258 >> v2:172.21.15.31:3300/0 conn(0x7fade015d8f0 0x7fade015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 -- 172.21.15.97:0/1043590258 >> 172.21.15.97:0/1043590258 conn(0x7fade009c750 msgr2=0x7fade00d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 -- 172.21.15.97:0/1043590258 shutdown_connections 2023-11-26T07:17:08.951 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 -- 172.21.15.97:0/1043590258 wait complete. 2023-11-26T07:17:08.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 Processor -- start 2023-11-26T07:17:08.952 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 -- start start 2023-11-26T07:17:08.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fade00ced30 0x7fade01504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade00cf1b0 0x7fade00b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade015d8f0 0x7fade015c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:08.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade01579c0 con 0x7fade00ced30 2023-11-26T07:17:08.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0150a10 con 0x7fade00cf1b0 2023-11-26T07:17:08.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7fade5da1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade00b1350 con 0x7fade015d8f0 2023-11-26T07:17:08.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7faddeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade00cf1b0 0x7fade00b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7faddffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade015d8f0 0x7fade015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.955 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7faddeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade00cf1b0 0x7fade00b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46588/0 (socket says 172.21.15.97:46588) 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7faddeffd640 1 -- 172.21.15.97:0/719922321 learned_addr learned my addr 172.21.15.97:0/719922321 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7faddf7fe640 1 --2- 172.21.15.97:0/719922321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fade00ced30 0x7fade01504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7faddffff640 1 -- 172.21.15.97:0/719922321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade00cf1b0 msgr2=0x7fade00b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7faddffff640 1 --2- 172.21.15.97:0/719922321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade00cf1b0 0x7fade00b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7faddffff640 1 -- 172.21.15.97:0/719922321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fade00ced30 msgr2=0x7fade01504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7faddffff640 1 --2- 172.21.15.97:0/719922321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fade00ced30 0x7fade01504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.949+0000 7faddffff640 1 -- 172.21.15.97:0/719922321 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40094d0 con 0x7fade015d8f0 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.953+0000 7faddffff640 1 --2- 172.21.15.97:0/719922321 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade015d8f0 0x7fade015c700 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fadd800eae0 tx=0x7fadd800b0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.953+0000 7fadc7fff640 1 -- 172.21.15.97:0/719922321 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8010040 con 0x7fade015d8f0 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.953+0000 7fadc7fff640 1 -- 172.21.15.97:0/719922321 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd800bc10 con 0x7fade015d8f0 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.953+0000 7fadc7fff640 1 -- 172.21.15.97:0/719922321 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8014400 con 0x7fade015d8f0 2023-11-26T07:17:08.956 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.953+0000 7fade5da1640 1 -- 172.21.15.97:0/719922321 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade00b1630 con 0x7fade015d8f0 2023-11-26T07:17:08.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.953+0000 7fade5da1640 1 -- 172.21.15.97:0/719922321 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade0157c10 con 0x7fade015d8f0 2023-11-26T07:17:08.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.953+0000 7fadc7fff640 1 -- 172.21.15.97:0/719922321 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fadd8014860 con 0x7fade015d8f0 2023-11-26T07:17:08.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.953+0000 7fadc7fff640 1 -- 172.21.15.97:0/719922321 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fadd801d400 con 0x7fade015d8f0 2023-11-26T07:17:08.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.953+0000 7fade5da1640 1 -- 172.21.15.97:0/719922321 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade004e850 con 0x7fade015d8f0 2023-11-26T07:17:08.957 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:08.953+0000 7fadc7fff640 1 -- 172.21.15.97:0/719922321 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fadd803bce0 con 0x7fade015d8f0 2023-11-26T07:17:08.988 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:08.985+0000 7f247b35f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-26T07:17:08.995 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:08.994+0000 7f1d2a377180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-26T07:17:09.024 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.021+0000 7fade5da1640 1 -- 172.21.15.97:0/719922321 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd new", "uuid": "85a5d6b2-f995-4478-8add-968a03c0baa9", "id": 4} v 0) v1 -- 0x7fade00d0cf0 con 0x7fade015d8f0 2023-11-26T07:17:09.030 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.029+0000 7fadc7fff640 1 -- 172.21.15.97:0/719922321 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "85a5d6b2-f995-4478-8add-968a03c0baa9", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fadd802a070 con 0x7fade015d8f0 2023-11-26T07:17:09.030 INFO:teuthology.orchestra.run.smithi097.stdout:4 2023-11-26T07:17:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.029+0000 7fade5da1640 1 -- 172.21.15.97:0/719922321 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade015d8f0 msgr2=0x7fade015c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.031 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.029+0000 7fade5da1640 1 --2- 172.21.15.97:0/719922321 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade015d8f0 0x7fade015c700 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fadd800eae0 tx=0x7fadd800b0b0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.029+0000 7fade5da1640 1 -- 172.21.15.97:0/719922321 shutdown_connections 2023-11-26T07:17:09.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.029+0000 7fade5da1640 1 --2- 172.21.15.97:0/719922321 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fade015d8f0 0x7fade015c700 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.029+0000 7fade5da1640 1 --2- 172.21.15.97:0/719922321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade00cf1b0 0x7fade00b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.029+0000 7fade5da1640 1 --2- 172.21.15.97:0/719922321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fade00ced30 0x7fade01504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.029+0000 7fade5da1640 1 -- 172.21.15.97:0/719922321 >> 172.21.15.97:0/719922321 conn(0x7fade009c750 msgr2=0x7fade015a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:09.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.029+0000 7fade5da1640 1 -- 172.21.15.97:0/719922321 shutdown_connections 2023-11-26T07:17:09.032 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.029+0000 7fade5da1640 1 -- 172.21.15.97:0/719922321 wait complete. 2023-11-26T07:17:09.050 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new 536f84f4-6ab7-494f-bfe9-0e88f03c1366 5 2023-11-26T07:17:09.053 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:09.049+0000 7f247b35f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-26T07:17:09.059 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:09.058+0000 7f1d2a377180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-26T07:17:09.143 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:09.141+0000 7f247b35f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-26T07:17:09.152 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:09.150+0000 7f1d2a377180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-26T07:17:09.184 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.181+0000 7fd9c8079640 1 Processor -- start 2023-11-26T07:17:09.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.181+0000 7fd9c8079640 1 -- start start 2023-11-26T07:17:09.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c8079640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00ced30 0x7fd9c00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c8079640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9c00cf1b0 0x7fd9c00d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c8079640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c00cdfb0 con 0x7fd9c015d8f0 2023-11-26T07:17:09.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c8079640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c00d4420 con 0x7fd9c00cf1b0 2023-11-26T07:17:09.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c8079640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00d4140 con 0x7fd9c00ced30 2023-11-26T07:17:09.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c5614640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00ced30 0x7fd9c00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c5614640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00ced30 0x7fd9c00cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:35158/0 (socket says 172.21.15.97:35158) 2023-11-26T07:17:09.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c5614640 1 -- 172.21.15.97:0/260910205 learned_addr learned my addr 172.21.15.97:0/260910205 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:09.187 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c6616640 1 --2- 172.21.15.97:0/260910205 >> v2:172.21.15.31:3300/0 conn(0x7fd9c00cf1b0 0x7fd9c00d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c5614640 1 -- 172.21.15.97:0/260910205 >> v2:172.21.15.31:3300/0 conn(0x7fd9c00cf1b0 msgr2=0x7fd9c00d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:09.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c5614640 1 --2- 172.21.15.97:0/260910205 >> v2:172.21.15.31:3300/0 conn(0x7fd9c00cf1b0 0x7fd9c00d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c5614640 1 -- 172.21.15.97:0/260910205 >> v1:172.21.15.31:6789/0 conn(0x7fd9c015d8f0 legacy=0x7fd9c015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.188 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c5614640 1 -- 172.21.15.97:0/260910205 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00ce250 con 0x7fd9c00ced30 2023-11-26T07:17:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c5614640 1 --2- 172.21.15.97:0/260910205 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00ced30 0x7fd9c00cfe00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4781e405a33c0a09 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.185+0000 7fd9c4e13640 1 -- 172.21.15.97:0/260910205 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00aa670 con 0x7fd9c00ced30 2023-11-26T07:17:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c4e13640 1 -- 172.21.15.97:0/260910205 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b00aa7d0 con 0x7fd9c00ced30 2023-11-26T07:17:09.190 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c4e13640 1 -- 172.21.15.97:0/260910205 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00aab60 con 0x7fd9c00ced30 2023-11-26T07:17:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 -- 172.21.15.97:0/260910205 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00ced30 msgr2=0x7fd9c00cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 --2- 172.21.15.97:0/260910205 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00ced30 0x7fd9c00cfe00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b001c510 comp rx=0 tx=0).stop 2023-11-26T07:17:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 -- 172.21.15.97:0/260910205 shutdown_connections 2023-11-26T07:17:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 --2- 172.21.15.97:0/260910205 >> v2:172.21.15.31:3300/0 conn(0x7fd9c00cf1b0 0x7fd9c00d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 --2- 172.21.15.97:0/260910205 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00ced30 0x7fd9c00cfe00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 -- 172.21.15.97:0/260910205 >> 172.21.15.97:0/260910205 conn(0x7fd9c009c750 msgr2=0x7fd9c00d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:09.191 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 -- 172.21.15.97:0/260910205 shutdown_connections 2023-11-26T07:17:09.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 -- 172.21.15.97:0/260910205 wait complete. 2023-11-26T07:17:09.192 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 Processor -- start 2023-11-26T07:17:09.193 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 -- start start 2023-11-26T07:17:09.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c00ced30 0x7fd9c01517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00cf1b0 0x7fd9c01502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.195 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c015d8f0 0x7fd9c0159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c015c8e0 con 0x7fd9c00cf1b0 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c015cb10 con 0x7fd9c00ced30 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c8079640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c015cdf0 con 0x7fd9c015d8f0 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c5e15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c00ced30 0x7fd9c01517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c6616640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c015d8f0 0x7fd9c0159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c5e15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c00ced30 0x7fd9c01517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46592/0 (socket says 172.21.15.97:46592) 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c5e15640 1 -- 172.21.15.97:0/3725820916 learned_addr learned my addr 172.21.15.97:0/3725820916 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.189+0000 7fd9c5614640 1 --2- 172.21.15.97:0/3725820916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00cf1b0 0x7fd9c01502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9c6616640 1 -- 172.21.15.97:0/3725820916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c00ced30 msgr2=0x7fd9c01517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9c6616640 1 --2- 172.21.15.97:0/3725820916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c00ced30 0x7fd9c01517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.196 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9c6616640 1 -- 172.21.15.97:0/3725820916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00cf1b0 msgr2=0x7fd9c01502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9c6616640 1 --2- 172.21.15.97:0/3725820916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00cf1b0 0x7fd9c01502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9c6616640 1 -- 172.21.15.97:0/3725820916 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00094d0 con 0x7fd9c015d8f0 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9c5e15640 1 --2- 172.21.15.97:0/3725820916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c00ced30 0x7fd9c01517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9c6616640 1 --2- 172.21.15.97:0/3725820916 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c015d8f0 0x7fd9c0159140 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800efc0 tx=0x7fd9b800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9b67fc640 1 -- 172.21.15.97:0/3725820916 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8009c00 con 0x7fd9c015d8f0 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9b67fc640 1 -- 172.21.15.97:0/3725820916 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b800f040 con 0x7fd9c015d8f0 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9b67fc640 1 -- 172.21.15.97:0/3725820916 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80146c0 con 0x7fd9c015d8f0 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9c8079640 1 -- 172.21.15.97:0/3725820916 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c01543b0 con 0x7fd9c015d8f0 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9c8079640 1 -- 172.21.15.97:0/3725820916 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c015d0d0 con 0x7fd9c015d8f0 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9b67fc640 1 -- 172.21.15.97:0/3725820916 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd9b8014b20 con 0x7fd9c015d8f0 2023-11-26T07:17:09.197 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9b67fc640 1 -- 172.21.15.97:0/3725820916 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fd9b8014dd0 con 0x7fd9c015d8f0 2023-11-26T07:17:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.193+0000 7fd9c8079640 1 -- 172.21.15.97:0/3725820916 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c01545d0 con 0x7fd9c015d8f0 2023-11-26T07:17:09.198 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.197+0000 7fd9b67fc640 1 -- 172.21.15.97:0/3725820916 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fd9b803bce0 con 0x7fd9c015d8f0 2023-11-26T07:17:09.206 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:09.205+0000 7f247b35f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-26T07:17:09.215 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:09.214+0000 7f1d2a377180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-26T07:17:09.267 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.265+0000 7fd9c8079640 1 -- 172.21.15.97:0/3725820916 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd new", "uuid": "536f84f4-6ab7-494f-bfe9-0e88f03c1366", "id": 5} v 0) v1 -- 0x7fd9c004e850 con 0x7fd9c015d8f0 2023-11-26T07:17:09.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.269+0000 7fd9b67fc640 1 -- 172.21.15.97:0/3725820916 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "536f84f4-6ab7-494f-bfe9-0e88f03c1366", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd9b802a070 con 0x7fd9c015d8f0 2023-11-26T07:17:09.273 INFO:teuthology.orchestra.run.smithi097.stdout:5 2023-11-26T07:17:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.273+0000 7fd9c8079640 1 -- 172.21.15.97:0/3725820916 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c015d8f0 msgr2=0x7fd9c0159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.273+0000 7fd9c8079640 1 --2- 172.21.15.97:0/3725820916 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c015d8f0 0x7fd9c0159140 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800efc0 tx=0x7fd9b800c490 comp rx=0 tx=0).stop 2023-11-26T07:17:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.273+0000 7fd9c8079640 1 -- 172.21.15.97:0/3725820916 shutdown_connections 2023-11-26T07:17:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.273+0000 7fd9c8079640 1 --2- 172.21.15.97:0/3725820916 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c015d8f0 0x7fd9c0159140 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.273+0000 7fd9c8079640 1 --2- 172.21.15.97:0/3725820916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00cf1b0 0x7fd9c01502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.273+0000 7fd9c8079640 1 --2- 172.21.15.97:0/3725820916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c00ced30 0x7fd9c01517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.273+0000 7fd9c8079640 1 -- 172.21.15.97:0/3725820916 >> 172.21.15.97:0/3725820916 conn(0x7fd9c009c750 msgr2=0x7fd9c0157070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.273+0000 7fd9c8079640 1 -- 172.21.15.97:0/3725820916 shutdown_connections 2023-11-26T07:17:09.275 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.273+0000 7fd9c8079640 1 -- 172.21.15.97:0/3725820916 wait complete. 2023-11-26T07:17:09.292 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new f7a51918-df92-41a1-8128-0c83b2456051 6 2023-11-26T07:17:09.300 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:09.297+0000 7f247b35f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-26T07:17:09.311 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:09.310+0000 7f1d2a377180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-26T07:17:09.362 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:09.361+0000 7f247b35f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-26T07:17:09.373 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:09.370+0000 7f1d2a377180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-26T07:17:09.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.425+0000 7fbc78279640 1 Processor -- start 2023-11-26T07:17:09.428 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.425+0000 7fbc78279640 1 -- start start 2023-11-26T07:17:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.425+0000 7fbc78279640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7015d950 0x7fbc700d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.425+0000 7fbc78279640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc700cef60 con 0x7fbc700d1bf0 2023-11-26T07:17:09.430 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.425+0000 7fbc78279640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc700cf240 con 0x7fbc700ce9f0 2023-11-26T07:17:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.425+0000 7fbc78279640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7006b710 con 0x7fbc7015d950 2023-11-26T07:17:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc75814640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fbc700ce9f0 0x7fbc700d14e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46538/0 (socket says 172.21.15.97:46538) 2023-11-26T07:17:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc75814640 1 -- 172.21.15.97:0/2536943356 learned_addr learned my addr 172.21.15.97:0/2536943356 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-26T07:17:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc76015640 1 --2- 172.21.15.97:0/2536943356 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7015d950 0x7fbc700d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.431 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc75013640 1 -- 172.21.15.97:0/2536943356 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756346713 0 0) 0x7fbc700cf240 con 0x7fbc700ce9f0 2023-11-26T07:17:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc75013640 1 -- 172.21.15.97:0/2536943356 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc5c003530 con 0x7fbc700ce9f0 2023-11-26T07:17:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc76015640 1 -- 172.21.15.97:0/2536943356 >> v1:172.21.15.97:6789/0 conn(0x7fbc700ce9f0 legacy=0x7fbc700d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc76015640 1 -- 172.21.15.97:0/2536943356 >> v1:172.21.15.31:6789/0 conn(0x7fbc700d1bf0 legacy=0x7fbc7006c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.432 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc76015640 1 -- 172.21.15.97:0/2536943356 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7006b9f0 con 0x7fbc7015d950 2023-11-26T07:17:09.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc76015640 1 --2- 172.21.15.97:0/2536943356 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7015d950 0x7fbc700d3ca0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0097f0 tx=0x7fbc6c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29ec82bd25452fef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:09.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc75013640 1 -- 172.21.15.97:0/2536943356 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0ac600 con 0x7fbc7015d950 2023-11-26T07:17:09.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc75013640 1 -- 172.21.15.97:0/2536943356 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6c0a8050 con 0x7fbc7015d950 2023-11-26T07:17:09.434 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.429+0000 7fbc75013640 1 -- 172.21.15.97:0/2536943356 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0ac960 con 0x7fbc7015d950 2023-11-26T07:17:09.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 -- 172.21.15.97:0/2536943356 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7015d950 msgr2=0x7fbc700d3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 --2- 172.21.15.97:0/2536943356 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7015d950 0x7fbc700d3ca0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0097f0 tx=0x7fbc6c01c490 comp rx=0 tx=0).stop 2023-11-26T07:17:09.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 -- 172.21.15.97:0/2536943356 shutdown_connections 2023-11-26T07:17:09.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 --2- 172.21.15.97:0/2536943356 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc7015d950 0x7fbc700d3ca0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 -- 172.21.15.97:0/2536943356 >> 172.21.15.97:0/2536943356 conn(0x7fbc7008afc0 msgr2=0x7fbc700d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:09.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 -- 172.21.15.97:0/2536943356 shutdown_connections 2023-11-26T07:17:09.435 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 -- 172.21.15.97:0/2536943356 wait complete. 2023-11-26T07:17:09.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 Processor -- start 2023-11-26T07:17:09.436 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 -- start start 2023-11-26T07:17:09.438 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700ce9f0 0x7fbc7014dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc700d1bf0 0x7fbc701507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc70154020 0x7fbc70154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc701549d0 con 0x7fbc700d1bf0 2023-11-26T07:17:09.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc700b4630 con 0x7fbc700ce9f0 2023-11-26T07:17:09.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc78279640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc70151550 con 0x7fbc70154020 2023-11-26T07:17:09.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc76015640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700ce9f0 0x7fbc7014dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc76816640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc70154020 0x7fbc70154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc76015640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700ce9f0 0x7fbc7014dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46600/0 (socket says 172.21.15.97:46600) 2023-11-26T07:17:09.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc76015640 1 -- 172.21.15.97:0/1265448428 learned_addr learned my addr 172.21.15.97:0/1265448428 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:09.439 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.433+0000 7fbc75814640 1 --2- 172.21.15.97:0/1265448428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc700d1bf0 0x7fbc701507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc76015640 1 -- 172.21.15.97:0/1265448428 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc70154020 msgr2=0x7fbc70154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc76015640 1 --2- 172.21.15.97:0/1265448428 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc70154020 0x7fbc70154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc76015640 1 -- 172.21.15.97:0/1265448428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc700d1bf0 msgr2=0x7fbc701507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc76015640 1 --2- 172.21.15.97:0/1265448428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc700d1bf0 0x7fbc701507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc76015640 1 -- 172.21.15.97:0/1265448428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c009420 con 0x7fbc700ce9f0 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc76015640 1 --2- 172.21.15.97:0/1265448428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700ce9f0 0x7fbc7014dba0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0097f0 tx=0x7fbc6c019230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc66ffd640 1 -- 172.21.15.97:0/1265448428 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c006e20 con 0x7fbc700ce9f0 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc66ffd640 1 -- 172.21.15.97:0/1265448428 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6c004030 con 0x7fbc700ce9f0 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc78279640 1 -- 172.21.15.97:0/1265448428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7006d520 con 0x7fbc700ce9f0 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc66ffd640 1 -- 172.21.15.97:0/1265448428 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0043a0 con 0x7fbc700ce9f0 2023-11-26T07:17:09.440 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc78279640 1 -- 172.21.15.97:0/1265448428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc700b3a20 con 0x7fbc700ce9f0 2023-11-26T07:17:09.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc66ffd640 1 -- 172.21.15.97:0/1265448428 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbc6c0b3040 con 0x7fbc700ce9f0 2023-11-26T07:17:09.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc66ffd640 1 -- 172.21.15.97:0/1265448428 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fbc6c0b6bf0 con 0x7fbc700ce9f0 2023-11-26T07:17:09.441 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc78279640 1 -- 172.21.15.97:0/1265448428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7004e850 con 0x7fbc700ce9f0 2023-11-26T07:17:09.442 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.437+0000 7fbc66ffd640 1 -- 172.21.15.97:0/1265448428 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fbc6c0c0070 con 0x7fbc700ce9f0 2023-11-26T07:17:09.507 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.505+0000 7fbc78279640 1 -- 172.21.15.97:0/1265448428 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f7a51918-df92-41a1-8128-0c83b2456051", "id": 6} v 0) v1 -- 0x7fbc700b1d30 con 0x7fbc700ce9f0 2023-11-26T07:17:09.512 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.509+0000 7fbc66ffd640 1 -- 172.21.15.97:0/1265448428 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f7a51918-df92-41a1-8128-0c83b2456051", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbc6c0b1070 con 0x7fbc700ce9f0 2023-11-26T07:17:09.512 INFO:teuthology.orchestra.run.smithi097.stdout:6 2023-11-26T07:17:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.509+0000 7fbc78279640 1 -- 172.21.15.97:0/1265448428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700ce9f0 msgr2=0x7fbc7014dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.509+0000 7fbc78279640 1 --2- 172.21.15.97:0/1265448428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700ce9f0 0x7fbc7014dba0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0097f0 tx=0x7fbc6c019230 comp rx=0 tx=0).stop 2023-11-26T07:17:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.509+0000 7fbc78279640 1 -- 172.21.15.97:0/1265448428 shutdown_connections 2023-11-26T07:17:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.509+0000 7fbc78279640 1 --2- 172.21.15.97:0/1265448428 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbc70154020 0x7fbc70154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.509+0000 7fbc78279640 1 --2- 172.21.15.97:0/1265448428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc700d1bf0 0x7fbc701507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.509+0000 7fbc78279640 1 --2- 172.21.15.97:0/1265448428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc700ce9f0 0x7fbc7014dba0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.509+0000 7fbc78279640 1 -- 172.21.15.97:0/1265448428 >> 172.21.15.97:0/1265448428 conn(0x7fbc7008afc0 msgr2=0x7fbc70155220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.509+0000 7fbc78279640 1 -- 172.21.15.97:0/1265448428 shutdown_connections 2023-11-26T07:17:09.514 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.509+0000 7fbc78279640 1 -- 172.21.15.97:0/1265448428 wait complete. 2023-11-26T07:17:09.525 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:09.521+0000 7f247b35f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-26T07:17:09.531 DEBUG:teuthology.orchestra.run.smithi097:> sudo ceph --cluster ceph osd new be2367b1-f10f-410c-8eb2-4e56f59b2792 7 2023-11-26T07:17:09.538 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:09.534+0000 7f1d2a377180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-26T07:17:09.596 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:09.593+0000 7f247b35f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-26T07:17:09.609 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:09.606+0000 7f1d2a377180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-26T07:17:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23f5f55640 1 Processor -- start 2023-11-26T07:17:09.710 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23f5f55640 1 -- start start 2023-11-26T07:17:09.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23f5f55640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00ced30 0x7f23f00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23f5f55640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f00cdfb0 con 0x7f23f015d8f0 2023-11-26T07:17:09.712 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23f5f55640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f00d4140 con 0x7f23f00cf1b0 2023-11-26T07:17:09.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23f5f55640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f00d4420 con 0x7f23f00ced30 2023-11-26T07:17:09.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23effff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f23f00cf1b0 0x7f23f00d1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.97:46554/0 (socket says 172.21.15.97:46554) 2023-11-26T07:17:09.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23effff640 1 -- 172.21.15.97:0/3994869151 learned_addr learned my addr 172.21.15.97:0/3994869151 (peer_addr_for_me v1:172.21.15.97:0/0) 2023-11-26T07:17:09.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23eeffd640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00ced30 0x7f23f00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194049301 0 0) 0x7f23f00d4140 con 0x7f23f00cf1b0 2023-11-26T07:17:09.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23d8003530 con 0x7f23f00cf1b0 2023-11-26T07:17:09.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208942106 0 0) 0x7f23f00cdfb0 con 0x7f23f015d8f0 2023-11-26T07:17:09.713 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23f00d4140 con 0x7f23f015d8f0 2023-11-26T07:17:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1177281862 0 0) 0x7f23d8003530 con 0x7f23f00cf1b0 2023-11-26T07:17:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00ced30 msgr2=0x7f23f00cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00ced30 0x7f23f00cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 >> v1:172.21.15.31:6789/0 conn(0x7f23f015d8f0 legacy=0x7f23f015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f00ce250 con 0x7f23f00cf1b0 2023-11-26T07:17:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 <== mon.1 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f23e8001a00 con 0x7f23f00cf1b0 2023-11-26T07:17:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 >> v1:172.21.15.97:6789/0 conn(0x7f23f00cf1b0 legacy=0x7f23f00d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8004b70 0x7f23d8004f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.715 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d8003770 0x7f23d8003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00ced30 0x7f23d8004130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f00d4420 con 0x7f23d8003770 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d800ba70 con 0x7f23d8004b70 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d800bc30 con 0x7f23f00ced30 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ef7fe640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8004b70 0x7f23d8004f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23effff640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00ced30 0x7f23d8004130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23eeffd640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d8003770 0x7f23d8003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ef7fe640 1 -- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00ced30 msgr2=0x7f23d8004130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ef7fe640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00ced30 0x7f23d8004130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ef7fe640 1 -- 172.21.15.97:0/3994869151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d8003770 msgr2=0x7f23d8003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ef7fe640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d8003770 0x7f23d8003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.716 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.709+0000 7f23ef7fe640 1 -- 172.21.15.97:0/3994869151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8003530 con 0x7f23d8004b70 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23ef7fe640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8004b70 0x7f23d8004f70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f23e00018d0 tx=0x7f23e001ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f58b00359b4d3df7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00052d0 con 0x7f23d8004b70 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e0005450 con 0x7f23d8004b70 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23ee7fc640 1 -- 172.21.15.97:0/3994869151 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00a8930 con 0x7f23d8004b70 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23f5f55640 1 -- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8004b70 msgr2=0x7f23d8004f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23f5f55640 1 --2- 172.21.15.97:0/3994869151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23d8004b70 0x7f23d8004f70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f23e00018d0 tx=0x7f23e001ecd0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23effff640 1 -- 172.21.15.97:0/3994869151 reap_dead start 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23f5f55640 1 -- 172.21.15.97:0/3994869151 shutdown_connections 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23f5f55640 1 -- 172.21.15.97:0/3994869151 >> 172.21.15.97:0/3994869151 conn(0x7f23f009c750 msgr2=0x7f23f00d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23f5f55640 1 -- 172.21.15.97:0/3994869151 shutdown_connections 2023-11-26T07:17:09.718 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.713+0000 7f23f5f55640 1 -- 172.21.15.97:0/3994869151 wait complete. 2023-11-26T07:17:09.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23f5f55640 1 Processor -- start 2023-11-26T07:17:09.719 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23f5f55640 1 -- start start 2023-11-26T07:17:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23f5f55640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d8004b70 0x7f23f009d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23f5f55640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f0158a30 0x7f23f014e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23f5f55640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00cc320 0x7f23f00cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23f5f55640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f00a0f80 con 0x7f23d8004b70 2023-11-26T07:17:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23f5f55640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f00ccce0 con 0x7f23f0158a30 2023-11-26T07:17:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23f5f55640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f009d700 con 0x7f23f00cc320 2023-11-26T07:17:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23effff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00cc320 0x7f23f00cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23eeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f0158a30 0x7f23f014e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.721 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23effff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00cc320 0x7f23f00cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.97:35216/0 (socket says 172.21.15.97:35216) 2023-11-26T07:17:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23eeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f0158a30 0x7f23f014e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46612/0 (socket says 172.21.15.97:46612) 2023-11-26T07:17:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23effff640 1 -- 172.21.15.97:0/2755589102 learned_addr learned my addr 172.21.15.97:0/2755589102 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-11-26T07:17:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23ef7fe640 1 --2- 172.21.15.97:0/2755589102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d8004b70 0x7f23f009d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23eeffd640 1 -- 172.21.15.97:0/2755589102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00cc320 msgr2=0x7f23f00cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23eeffd640 1 --2- 172.21.15.97:0/2755589102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00cc320 0x7f23f00cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23eeffd640 1 -- 172.21.15.97:0/2755589102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d8004b70 msgr2=0x7f23f009d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23eeffd640 1 --2- 172.21.15.97:0/2755589102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d8004b70 0x7f23f009d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23eeffd640 1 -- 172.21.15.97:0/2755589102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f00a0060 con 0x7f23f0158a30 2023-11-26T07:17:09.723 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23effff640 1 --2- 172.21.15.97:0/2755589102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00cc320 0x7f23f00cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23eeffd640 1 --2- 172.21.15.97:0/2755589102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f0158a30 0x7f23f014e6b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f23e4011930 tx=0x7f23e4011df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23d7fff640 1 -- 172.21.15.97:0/2755589102 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4004240 con 0x7f23f0158a30 2023-11-26T07:17:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23d7fff640 1 -- 172.21.15.97:0/2755589102 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e40043a0 con 0x7f23f0158a30 2023-11-26T07:17:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23d7fff640 1 -- 172.21.15.97:0/2755589102 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e400fa20 con 0x7f23f0158a30 2023-11-26T07:17:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.717+0000 7f23f5f55640 1 -- 172.21.15.97:0/2755589102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f00a0300 con 0x7f23f0158a30 2023-11-26T07:17:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.721+0000 7f23f5f55640 1 -- 172.21.15.97:0/2755589102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f00a08c0 con 0x7f23f0158a30 2023-11-26T07:17:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.721+0000 7f23d7fff640 1 -- 172.21.15.97:0/2755589102 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f23e4003620 con 0x7f23f0158a30 2023-11-26T07:17:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.721+0000 7f23d7fff640 1 -- 172.21.15.97:0/2755589102 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f23e401e450 con 0x7f23f0158a30 2023-11-26T07:17:09.724 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.721+0000 7f23f5f55640 1 -- 172.21.15.97:0/2755589102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f004e850 con 0x7f23f0158a30 2023-11-26T07:17:09.725 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.721+0000 7f23d7fff640 1 -- 172.21.15.97:0/2755589102 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f23e4023060 con 0x7f23f0158a30 2023-11-26T07:17:09.791 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.789+0000 7f23f5f55640 1 -- 172.21.15.97:0/2755589102 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd new", "uuid": "be2367b1-f10f-410c-8eb2-4e56f59b2792", "id": 7} v 0) v1 -- 0x7f23f00cf9d0 con 0x7f23f0158a30 2023-11-26T07:17:09.797 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.793+0000 7f23d7fff640 1 -- 172.21.15.97:0/2755589102 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "be2367b1-f10f-410c-8eb2-4e56f59b2792", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f23e402c020 con 0x7f23f0158a30 2023-11-26T07:17:09.797 INFO:teuthology.orchestra.run.smithi097.stdout:7 2023-11-26T07:17:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.793+0000 7f23f5f55640 1 -- 172.21.15.97:0/2755589102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f0158a30 msgr2=0x7f23f014e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.793+0000 7f23f5f55640 1 --2- 172.21.15.97:0/2755589102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f0158a30 0x7f23f014e6b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f23e4011930 tx=0x7f23e4011df0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.793+0000 7f23f5f55640 1 -- 172.21.15.97:0/2755589102 shutdown_connections 2023-11-26T07:17:09.798 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.793+0000 7f23f5f55640 1 --2- 172.21.15.97:0/2755589102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f23f00cc320 0x7f23f00cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.793+0000 7f23f5f55640 1 --2- 172.21.15.97:0/2755589102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23f0158a30 0x7f23f014e6b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.793+0000 7f23f5f55640 1 --2- 172.21.15.97:0/2755589102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23d8004b70 0x7f23f009d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.793+0000 7f23f5f55640 1 -- 172.21.15.97:0/2755589102 >> 172.21.15.97:0/2755589102 conn(0x7f23f009c750 msgr2=0x7f23f006b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.797+0000 7f23f5f55640 1 -- 172.21.15.97:0/2755589102 shutdown_connections 2023-11-26T07:17:09.799 INFO:teuthology.orchestra.run.smithi097.stderr:2023-11-26T07:17:09.797+0000 7f23f5f55640 1 -- 172.21.15.97:0/2755589102 wait complete. 2023-11-26T07:17:09.816 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-26T07:17:09.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-26T07:17:09.820 INFO:tasks.ceph.osd.0:Started 2023-11-26T07:17:09.820 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-26T07:17:09.820 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-26T07:17:09.823 INFO:tasks.ceph.osd.1:Started 2023-11-26T07:17:09.823 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-26T07:17:09.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-26T07:17:09.827 INFO:tasks.ceph.osd.2:Started 2023-11-26T07:17:09.827 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-26T07:17:09.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-26T07:17:09.830 INFO:tasks.ceph.osd.3:Started 2023-11-26T07:17:09.830 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-26T07:17:09.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-26T07:17:09.861 INFO:tasks.ceph.osd.4:Started 2023-11-26T07:17:09.861 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-26T07:17:09.861 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-26T07:17:09.864 INFO:tasks.ceph.osd.5:Started 2023-11-26T07:17:09.864 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-26T07:17:09.864 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-26T07:17:09.868 INFO:tasks.ceph.osd.6:Started 2023-11-26T07:17:09.868 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-26T07:17:09.868 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-26T07:17:09.871 INFO:tasks.ceph.osd.7:Started 2023-11-26T07:17:09.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:10.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.058+0000 7f783a11c640 1 Processor -- start 2023-11-26T07:17:10.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.058+0000 7f783a11c640 1 -- start start 2023-11-26T07:17:10.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f783a11c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f783415d5a0 0x7f78340b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f783a11c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f78340b1f40 0x7f7834070670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f783a11c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78340abf20 con 0x7f7834070bb0 2023-11-26T07:17:10.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f783a11c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7834071040 con 0x7f78340b1f40 2023-11-26T07:17:10.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f783a11c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78340b3dd0 con 0x7f783415d5a0 2023-11-26T07:17:10.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f7833fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7834070bb0 0x7f7834160fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50554/0 (socket says 172.21.15.31:50554) 2023-11-26T07:17:10.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f7833fff640 1 -- 172.21.15.31:0/3057777504 learned_addr learned my addr 172.21.15.31:0/3057777504 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:10.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f7832ffd640 1 --2- 172.21.15.31:0/3057777504 >> v2:172.21.15.97:3300/0 conn(0x7f78340b1f40 0x7f7834070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f78337fe640 1 --2- 172.21.15.31:0/3057777504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f783415d5a0 0x7f78340b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f78327fc640 1 -- 172.21.15.31:0/3057777504 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251824657 0 0) 0x7f78340abf20 con 0x7f7834070bb0 2023-11-26T07:17:10.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f78327fc640 1 -- 172.21.15.31:0/3057777504 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f781c003530 con 0x7f7834070bb0 2023-11-26T07:17:10.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f7832ffd640 1 -- 172.21.15.31:0/3057777504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f783415d5a0 msgr2=0x7f78340b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:10.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f7832ffd640 1 --2- 172.21.15.31:0/3057777504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f783415d5a0 0x7f78340b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f7832ffd640 1 -- 172.21.15.31:0/3057777504 >> v1:172.21.15.31:6789/0 conn(0x7f7834070bb0 legacy=0x7f7834160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.062+0000 7f7832ffd640 1 -- 172.21.15.31:0/3057777504 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78340b3100 con 0x7f78340b1f40 2023-11-26T07:17:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f7832ffd640 1 --2- 172.21.15.31:0/3057777504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78340b1f40 0x7f7834070670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78280097f0 tx=0x7f782801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a407b3695b1e46d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f78327fc640 1 -- 172.21.15.31:0/3057777504 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78280ab600 con 0x7f78340b1f40 2023-11-26T07:17:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f78327fc640 1 -- 172.21.15.31:0/3057777504 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78280a7050 con 0x7f78340b1f40 2023-11-26T07:17:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f78327fc640 1 -- 172.21.15.31:0/3057777504 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78280ab980 con 0x7f78340b1f40 2023-11-26T07:17:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f783a11c640 1 -- 172.21.15.31:0/3057777504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78340b1f40 msgr2=0x7f7834070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f783a11c640 1 --2- 172.21.15.31:0/3057777504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78340b1f40 0x7f7834070670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f78280097f0 tx=0x7f782801c3f0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f783a11c640 1 -- 172.21.15.31:0/3057777504 shutdown_connections 2023-11-26T07:17:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f783a11c640 1 --2- 172.21.15.31:0/3057777504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78340b1f40 0x7f7834070670 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f783a11c640 1 --2- 172.21.15.31:0/3057777504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f783415d5a0 0x7f78340b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f783a11c640 1 -- 172.21.15.31:0/3057777504 >> 172.21.15.31:0/3057777504 conn(0x7f78340b0c40 msgr2=0x7f78340b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:10.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f783a11c640 1 -- 172.21.15.31:0/3057777504 shutdown_connections 2023-11-26T07:17:10.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f783a11c640 1 -- 172.21.15.31:0/3057777504 wait complete. 2023-11-26T07:17:10.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f783a11c640 1 Processor -- start 2023-11-26T07:17:10.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.066+0000 7f783a11c640 1 -- start start 2023-11-26T07:17:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f783a11c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834070bb0 0x7f783414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f783a11c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f78340b1f40 0x7f78341541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f783a11c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f783415d5a0 0x7f783415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f783a11c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783414f5a0 con 0x7f783415d5a0 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f783a11c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78341518a0 con 0x7f7834070bb0 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f783a11c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783415fd90 con 0x7f78340b1f40 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f7833fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f783415d5a0 0x7f783415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f7833fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f783415d5a0 0x7f783415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50870/0 (socket says 172.21.15.31:50870) 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f7833fff640 1 -- 172.21.15.31:0/935550530 learned_addr learned my addr 172.21.15.31:0/935550530 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f78337fe640 1 --2- 172.21.15.31:0/935550530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834070bb0 0x7f783414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f7832ffd640 1 --2- 172.21.15.31:0/935550530 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f78340b1f40 0x7f78341541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f7833fff640 1 -- 172.21.15.31:0/935550530 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f78340b1f40 msgr2=0x7f78341541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f7833fff640 1 --2- 172.21.15.31:0/935550530 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f78340b1f40 0x7f78341541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f7833fff640 1 -- 172.21.15.31:0/935550530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834070bb0 msgr2=0x7f783414eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f7833fff640 1 --2- 172.21.15.31:0/935550530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834070bb0 0x7f783414eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f7833fff640 1 -- 172.21.15.31:0/935550530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7828009420 con 0x7f783415d5a0 2023-11-26T07:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f7833fff640 1 --2- 172.21.15.31:0/935550530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f783415d5a0 0x7f783415b6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f782c00cb40 tx=0x7f782c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f781bfff640 1 -- 172.21.15.31:0/935550530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c00aca0 con 0x7f783415d5a0 2023-11-26T07:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f781bfff640 1 -- 172.21.15.31:0/935550530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c00ae00 con 0x7f783415d5a0 2023-11-26T07:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f781bfff640 1 -- 172.21.15.31:0/935550530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c012620 con 0x7f783415d5a0 2023-11-26T07:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f783a11c640 1 -- 172.21.15.31:0/935550530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7834160070 con 0x7f783415d5a0 2023-11-26T07:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f783a11c640 1 -- 172.21.15.31:0/935550530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7834160630 con 0x7f783415d5a0 2023-11-26T07:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f781bfff640 1 -- 172.21.15.31:0/935550530 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f782c012a80 con 0x7f783415d5a0 2023-11-26T07:17:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f781bfff640 1 -- 172.21.15.31:0/935550530 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f782c012d30 con 0x7f783415d5a0 2023-11-26T07:17:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.070+0000 7f783a11c640 1 -- 172.21.15.31:0/935550530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783414f6e0 con 0x7f783415d5a0 2023-11-26T07:17:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.074+0000 7f781bfff640 1 -- 172.21.15.31:0/935550530 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f783414f6e0 con 0x7f783415d5a0 2023-11-26T07:17:10.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.130+0000 7f783a11c640 1 -- 172.21.15.31:0/935550530 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7834160820 con 0x7f783415d5a0 2023-11-26T07:17:10.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.130+0000 7f781bfff640 1 -- 172.21.15.31:0/935550530 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f782c017070 con 0x7f783415d5a0 2023-11-26T07:17:10.134 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:09.793096+0000","last_up_change":"0.000000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:10.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.134+0000 7f783a11c640 1 -- 172.21.15.31:0/935550530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f783415d5a0 msgr2=0x7f783415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:10.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.134+0000 7f783a11c640 1 --2- 172.21.15.31:0/935550530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f783415d5a0 0x7f783415b6b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f782c00cb40 tx=0x7f782c00a060 comp rx=0 tx=0).stop 2023-11-26T07:17:10.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.134+0000 7f783a11c640 1 -- 172.21.15.31:0/935550530 shutdown_connections 2023-11-26T07:17:10.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.134+0000 7f783a11c640 1 --2- 172.21.15.31:0/935550530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f783415d5a0 0x7f783415b6b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.134+0000 7f783a11c640 1 --2- 172.21.15.31:0/935550530 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f78340b1f40 0x7f78341541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.134+0000 7f783a11c640 1 --2- 172.21.15.31:0/935550530 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7834070bb0 0x7f783414eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:10.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.134+0000 7f783a11c640 1 -- 172.21.15.31:0/935550530 >> 172.21.15.31:0/935550530 conn(0x7f78340b0c40 msgr2=0x7f783406d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:10.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.134+0000 7f783a11c640 1 -- 172.21.15.31:0/935550530 shutdown_connections 2023-11-26T07:17:10.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:10.134+0000 7f783a11c640 1 -- 172.21.15.31:0/935550530 wait complete. 2023-11-26T07:17:10.154 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-26T07:17:10.155 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-26T07:17:10.265 INFO:tasks.ceph.mgr.x.smithi097.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-26T07:17:10.266 INFO:tasks.ceph.mgr.x.smithi097.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-26T07:17:10.266 INFO:tasks.ceph.mgr.x.smithi097.stderr: from numpy import show_config as show_numpy_config 2023-11-26T07:17:10.313 INFO:tasks.ceph.mgr.y.smithi031.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-26T07:17:10.314 INFO:tasks.ceph.mgr.y.smithi031.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-26T07:17:10.314 INFO:tasks.ceph.mgr.y.smithi031.stderr: from numpy import show_config as show_numpy_config 2023-11-26T07:17:10.320 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:10.317+0000 7f247b35f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-26T07:17:10.352 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:10.350+0000 7f1d2a377180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-26T07:17:10.455 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:10.453+0000 7f247b35f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:17:10.492 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-11-26T07:17:10.490+0000 7f31df4ea640 -1 Falling back to public interface 2023-11-26T07:17:10.492 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-11-26T07:17:10.490+0000 7f9ed7bc6640 -1 Falling back to public interface 2023-11-26T07:17:10.495 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:10.490+0000 7f1d2a377180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:17:10.508 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-11-26T07:17:10.506+0000 7fd22942b640 -1 Falling back to public interface 2023-11-26T07:17:10.516 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-11-26T07:17:10.514+0000 7efdffe2b640 -1 Falling back to public interface 2023-11-26T07:17:10.567 INFO:tasks.ceph.osd.6.smithi097.stderr:2023-11-26T07:17:10.565+0000 7f7cb9687640 -1 Falling back to public interface 2023-11-26T07:17:10.583 INFO:tasks.ceph.osd.7.smithi097.stderr:2023-11-26T07:17:10.581+0000 7fc5c4cea640 -1 Falling back to public interface 2023-11-26T07:17:10.584 INFO:tasks.ceph.osd.4.smithi097.stderr:2023-11-26T07:17:10.581+0000 7f0dbbd3c640 -1 Falling back to public interface 2023-11-26T07:17:10.584 INFO:tasks.ceph.osd.5.smithi097.stderr:2023-11-26T07:17:10.581+0000 7f8180fb6640 -1 Falling back to public interface 2023-11-26T07:17:10.585 INFO:tasks.ceph.osd.4.smithi097.stderr:2023-11-26T07:17:10.581+0000 7f0dbacff640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6813/0: (98) Address already in use 2023-11-26T07:17:10.588 INFO:tasks.ceph.osd.7.smithi097.stderr:2023-11-26T07:17:10.585+0000 7fc5c3cab640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6820/0: (98) Address already in use 2023-11-26T07:17:10.638 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:10.637+0000 7f247b35f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-26T07:17:10.692 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:10.690+0000 7f1d2a377180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-26T07:17:10.720 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:10.717+0000 7f247b35f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-26T07:17:10.770 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:10.766+0000 7f1d2a377180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-26T07:17:10.857 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:10.853+0000 7f247b35f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:17:10.916 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:10.914+0000 7f1d2a377180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:17:10.960 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:11.093 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.090+0000 7fb9dba32640 1 Processor -- start 2023-11-26T07:17:11.093 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.090+0000 7fb9dba32640 1 -- start start 2023-11-26T07:17:11.095 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.090+0000 7fb9dba32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 0x7fb9d40cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.095 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.090+0000 7fb9dba32640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d40d0ac0 con 0x7fb9d40d0260 2023-11-26T07:17:11.095 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.090+0000 7fb9dba32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d40b6a90 con 0x7fb9d40d2190 2023-11-26T07:17:11.095 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.090+0000 7fb9dba32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40d07e0 con 0x7fb9d4160c40 2023-11-26T07:17:11.096 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.090+0000 7fb9d8fcd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb9d40d0260 0x7fb9d40d1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:50568/0 (socket says 172.21.15.31:50568) 2023-11-26T07:17:11.096 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.090+0000 7fb9d8fcd640 1 -- 172.21.15.31:0/3611441785 learned_addr learned my addr 172.21.15.31:0/3611441785 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:11.096 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9d97ce640 1 --2- 172.21.15.31:0/3611441785 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 0x7fb9d40cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.096 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9cbfff640 1 -- 172.21.15.31:0/3611441785 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797205223 0 0) 0x7fb9d40d0ac0 con 0x7fb9d40d0260 2023-11-26T07:17:11.096 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9cbfff640 1 -- 172.21.15.31:0/3611441785 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c0003530 con 0x7fb9d40d0260 2023-11-26T07:17:11.096 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9cbfff640 1 -- 172.21.15.31:0/3611441785 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441429466 0 0) 0x7fb9d40b6a90 con 0x7fb9d40d2190 2023-11-26T07:17:11.096 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9cbfff640 1 -- 172.21.15.31:0/3611441785 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9d40d0ac0 con 0x7fb9d40d2190 2023-11-26T07:17:11.096 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9d97ce640 1 -- 172.21.15.31:0/3611441785 >> v1:172.21.15.97:6789/0 conn(0x7fb9d40d2190 legacy=0x7fb9d40b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.096 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9d97ce640 1 -- 172.21.15.31:0/3611441785 >> v1:172.21.15.31:6789/0 conn(0x7fb9d40d0260 legacy=0x7fb9d40d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.096 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9d97ce640 1 -- 172.21.15.31:0/3611441785 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d40b6440 con 0x7fb9d4160c40 2023-11-26T07:17:11.098 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9d97ce640 1 --2- 172.21.15.31:0/3611441785 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 0x7fb9d40cfd20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00097f0 tx=0x7fb9d001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86b5a917e60c7ea9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:11.099 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9cbfff640 1 -- 172.21.15.31:0/3611441785 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00ab600 con 0x7fb9d4160c40 2023-11-26T07:17:11.099 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9cbfff640 1 -- 172.21.15.31:0/3611441785 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d00a7050 con 0x7fb9d4160c40 2023-11-26T07:17:11.099 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.094+0000 7fb9cbfff640 1 -- 172.21.15.31:0/3611441785 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00ab980 con 0x7fb9d4160c40 2023-11-26T07:17:11.099 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 -- 172.21.15.31:0/3611441785 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 msgr2=0x7fb9d40cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.100 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 --2- 172.21.15.31:0/3611441785 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 0x7fb9d40cfd20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb9d00097f0 tx=0x7fb9d001c3a0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.100 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 -- 172.21.15.31:0/3611441785 shutdown_connections 2023-11-26T07:17:11.100 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 --2- 172.21.15.31:0/3611441785 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 0x7fb9d40cfd20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.100 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 -- 172.21.15.31:0/3611441785 >> 172.21.15.31:0/3611441785 conn(0x7fb9d408afc0 msgr2=0x7fb9d40d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:11.100 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 -- 172.21.15.31:0/3611441785 shutdown_connections 2023-11-26T07:17:11.100 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 -- 172.21.15.31:0/3611441785 wait complete. 2023-11-26T07:17:11.101 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 Processor -- start 2023-11-26T07:17:11.101 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 -- start start 2023-11-26T07:17:11.103 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d40d0260 0x7fb9d414c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.103 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40d2190 0x7fb9d41539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.103 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 0x7fb9d4157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d4153f10 con 0x7fb9d40d0260 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d4155a30 con 0x7fb9d40d2190 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9dba32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d4154100 con 0x7fb9d4160c40 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9d97ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d40d0260 0x7fb9d414c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9d97ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d40d0260 0x7fb9d414c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50884/0 (socket says 172.21.15.31:50884) 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9d97ce640 1 -- 172.21.15.31:0/3218408714 learned_addr learned my addr 172.21.15.31:0/3218408714 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9d8fcd640 1 --2- 172.21.15.31:0/3218408714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40d2190 0x7fb9d41539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.098+0000 7fb9d9fcf640 1 --2- 172.21.15.31:0/3218408714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 0x7fb9d4157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9d97ce640 1 -- 172.21.15.31:0/3218408714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 msgr2=0x7fb9d4157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9d97ce640 1 --2- 172.21.15.31:0/3218408714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 0x7fb9d4157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.104 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9d97ce640 1 -- 172.21.15.31:0/3218408714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40d2190 msgr2=0x7fb9d41539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.105 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9d97ce640 1 --2- 172.21.15.31:0/3218408714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40d2190 0x7fb9d41539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.105 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9d97ce640 1 -- 172.21.15.31:0/3218408714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d0009420 con 0x7fb9d40d0260 2023-11-26T07:17:11.105 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9d97ce640 1 --2- 172.21.15.31:0/3218408714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d40d0260 0x7fb9d414c290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0018fb0 tx=0x7fb9d001abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:11.105 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9c9ffb640 1 -- 172.21.15.31:0/3218408714 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00b7450 con 0x7fb9d40d0260 2023-11-26T07:17:11.105 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9c9ffb640 1 -- 172.21.15.31:0/3218408714 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d00b75d0 con 0x7fb9d40d0260 2023-11-26T07:17:11.105 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9c9ffb640 1 -- 172.21.15.31:0/3218408714 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00b7980 con 0x7fb9d40d0260 2023-11-26T07:17:11.105 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9dba32640 1 -- 172.21.15.31:0/3218408714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d415ded0 con 0x7fb9d40d0260 2023-11-26T07:17:11.106 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9dba32640 1 -- 172.21.15.31:0/3218408714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d415e490 con 0x7fb9d40d0260 2023-11-26T07:17:11.106 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9c9ffb640 1 -- 172.21.15.31:0/3218408714 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb9d00c0400 con 0x7fb9d40d0260 2023-11-26T07:17:11.106 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9c9ffb640 1 -- 172.21.15.31:0/3218408714 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb9d00b7de0 con 0x7fb9d40d0260 2023-11-26T07:17:11.106 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.102+0000 7fb9dba32640 1 -- 172.21.15.31:0/3218408714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d404e850 con 0x7fb9d40d0260 2023-11-26T07:17:11.107 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.106+0000 7fb9c9ffb640 1 -- 172.21.15.31:0/3218408714 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb9d00dece0 con 0x7fb9d40d0260 2023-11-26T07:17:11.165 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.162+0000 7fb9dba32640 1 -- 172.21.15.31:0/3218408714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9d40d3980 con 0x7fb9d40d0260 2023-11-26T07:17:11.166 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.162+0000 7fb9c9ffb640 1 -- 172.21.15.31:0/3218408714 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb9d00cd030 con 0x7fb9d40d0260 2023-11-26T07:17:11.166 INFO:teuthology.misc.health.smithi031.stdout: 2023-11-26T07:17:11.166 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:09.793096+0000","last_up_change":"0.000000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:11.167 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.166+0000 7fb9dba32640 1 -- 172.21.15.31:0/3218408714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d40d0260 msgr2=0x7fb9d414c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:11.167 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.166+0000 7fb9dba32640 1 --2- 172.21.15.31:0/3218408714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d40d0260 0x7fb9d414c290 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0018fb0 tx=0x7fb9d001abe0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.168 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.166+0000 7fb9dba32640 1 -- 172.21.15.31:0/3218408714 shutdown_connections 2023-11-26T07:17:11.168 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.166+0000 7fb9dba32640 1 --2- 172.21.15.31:0/3218408714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9d4160c40 0x7fb9d4157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.168 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.166+0000 7fb9dba32640 1 --2- 172.21.15.31:0/3218408714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40d2190 0x7fb9d41539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.168 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.166+0000 7fb9dba32640 1 --2- 172.21.15.31:0/3218408714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d40d0260 0x7fb9d414c290 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:11.168 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.166+0000 7fb9dba32640 1 -- 172.21.15.31:0/3218408714 >> 172.21.15.31:0/3218408714 conn(0x7fb9d408afc0 msgr2=0x7fb9d406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:11.168 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.166+0000 7fb9dba32640 1 -- 172.21.15.31:0/3218408714 shutdown_connections 2023-11-26T07:17:11.168 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:11.166+0000 7fb9dba32640 1 -- 172.21.15.31:0/3218408714 wait complete. 2023-11-26T07:17:11.184 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-26T07:17:11.536 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:11.533+0000 7f247b35f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-26T07:17:11.619 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:11.618+0000 7f1d2a377180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-26T07:17:11.622 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:11.621+0000 7f247b35f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-26T07:17:11.710 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:11.706+0000 7f1d2a377180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-26T07:17:11.811 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:11.809+0000 7f247b35f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-26T07:17:11.896 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:11.893+0000 7f247b35f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-26T07:17:11.909 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:11.906+0000 7f1d2a377180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-26T07:17:11.997 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:11.994+0000 7f1d2a377180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-26T07:17:12.223 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:12.221+0000 7f247b35f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-26T07:17:12.291 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:12.289+0000 7f247b35f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-26T07:17:12.344 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:12.342+0000 7f1d2a377180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-26T07:17:12.416 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:17:12.414+0000 7f1d2a377180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-26T07:17:13.316 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:13.313+0000 7f2422ada640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-26T07:17:13.316 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:17:13.313+0000 7f2422ada640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-26T07:17:14.340 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-11-26T07:17:14.338+0000 7fd22942b640 -1 osd.0 0 log_to_monitors true 2023-11-26T07:17:14.418 INFO:tasks.ceph.osd.4.smithi097.stderr:2023-11-26T07:17:14.413+0000 7f0dbbd3c640 -1 osd.4 0 log_to_monitors true 2023-11-26T07:17:14.458 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-11-26T07:17:14.454+0000 7efdffe2b640 -1 osd.3 0 log_to_monitors true 2023-11-26T07:17:14.489 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-11-26T07:17:14.486+0000 7f31df4ea640 -1 osd.1 0 log_to_monitors true 2023-11-26T07:17:14.508 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-11-26T07:17:14.506+0000 7f9ed7bc6640 -1 osd.2 0 log_to_monitors true 2023-11-26T07:17:14.581 INFO:tasks.ceph.osd.7.smithi097.stderr:2023-11-26T07:17:14.577+0000 7fc5c4cea640 -1 osd.7 0 log_to_monitors true 2023-11-26T07:17:14.609 INFO:tasks.ceph.osd.6.smithi097.stderr:2023-11-26T07:17:14.605+0000 7f7cb9687640 -1 osd.6 0 log_to_monitors true 2023-11-26T07:17:14.622 INFO:tasks.ceph.osd.5.smithi097.stderr:2023-11-26T07:17:14.617+0000 7f8180fb6640 -1 osd.5 0 log_to_monitors true 2023-11-26T07:17:17.990 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:18.206 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.202+0000 7f2d46f48640 1 Processor -- start 2023-11-26T07:17:18.206 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.202+0000 7f2d46f48640 1 -- start start 2023-11-26T07:17:18.206 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.202+0000 7f2d46f48640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d400b05a0 0x7f2d400d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:18.206 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.202+0000 7f2d46f48640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d400d14c0 0x7f2d400cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:18.206 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.202+0000 7f2d46f48640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d400b0a00 con 0x7f2d4015b370 2023-11-26T07:17:18.207 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.202+0000 7f2d46f48640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d400b5fe0 con 0x7f2d400d14c0 2023-11-26T07:17:18.207 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.202+0000 7f2d46f48640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d400cf290 con 0x7f2d400b05a0 2023-11-26T07:17:18.207 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.202+0000 7f2d46747640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d400d14c0 0x7f2d400cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:18.207 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.202+0000 7f2d46747640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d400d14c0 0x7f2d400cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42520/0 (socket says 172.21.15.31:42520) 2023-11-26T07:17:18.207 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.202+0000 7f2d46747640 1 -- 172.21.15.31:0/3779216192 learned_addr learned my addr 172.21.15.31:0/3779216192 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:18.211 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.206+0000 7f2d46747640 1 -- 172.21.15.31:0/3779216192 >> v2:172.21.15.97:3300/0 conn(0x7f2d400b05a0 msgr2=0x7f2d400d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:18.211 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.206+0000 7f2d46747640 1 --2- 172.21.15.31:0/3779216192 >> v2:172.21.15.97:3300/0 conn(0x7f2d400b05a0 0x7f2d400d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:18.211 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.206+0000 7f2d46747640 1 -- 172.21.15.31:0/3779216192 >> v1:172.21.15.97:6789/0 conn(0x7f2d4015b370 legacy=0x7f2d4015b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:18.211 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.206+0000 7f2d46747640 1 -- 172.21.15.31:0/3779216192 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d400cf570 con 0x7f2d400d14c0 2023-11-26T07:17:18.213 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.210+0000 7f2d46747640 1 --2- 172.21.15.31:0/3779216192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d400d14c0 0x7f2d400cfd70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2d400b3e50 tx=0x7f2d3801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75754646f6de6640 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:18.213 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.210+0000 7f2d44f44640 1 -- 172.21.15.31:0/3779216192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d380a57b0 con 0x7f2d400d14c0 2023-11-26T07:17:18.213 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.210+0000 7f2d44f44640 1 -- 172.21.15.31:0/3779216192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d380a5910 con 0x7f2d400d14c0 2023-11-26T07:17:18.213 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.210+0000 7f2d44f44640 1 -- 172.21.15.31:0/3779216192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d380a5ca0 con 0x7f2d400d14c0 2023-11-26T07:17:18.214 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.210+0000 7f2d46f48640 1 -- 172.21.15.31:0/3779216192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d400d14c0 msgr2=0x7f2d400cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:18.215 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.210+0000 7f2d46f48640 1 --2- 172.21.15.31:0/3779216192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d400d14c0 0x7f2d400cfd70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2d400b3e50 tx=0x7f2d3801c8d0 comp rx=0 tx=0).stop 2023-11-26T07:17:18.215 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.214+0000 7f2d46f48640 1 -- 172.21.15.31:0/3779216192 shutdown_connections 2023-11-26T07:17:18.215 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.214+0000 7f2d46f48640 1 --2- 172.21.15.31:0/3779216192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d400d14c0 0x7f2d400cfd70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:18.215 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.214+0000 7f2d46f48640 1 --2- 172.21.15.31:0/3779216192 >> v2:172.21.15.97:3300/0 conn(0x7f2d400b05a0 0x7f2d400d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:18.215 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.214+0000 7f2d46f48640 1 -- 172.21.15.31:0/3779216192 >> 172.21.15.31:0/3779216192 conn(0x7f2d4006e8d0 msgr2=0x7f2d4006ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:18.216 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.214+0000 7f2d46f48640 1 -- 172.21.15.31:0/3779216192 shutdown_connections 2023-11-26T07:17:18.216 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.214+0000 7f2d46f48640 1 -- 172.21.15.31:0/3779216192 wait complete. 2023-11-26T07:17:18.219 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.214+0000 7f2d46f48640 1 Processor -- start 2023-11-26T07:17:18.219 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.214+0000 7f2d46f48640 1 -- start start 2023-11-26T07:17:18.220 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d46f48640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d400b05a0 0x7f2d400b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:18.220 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d46f48640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d400d14c0 0x7f2d40144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:18.220 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d46f48640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4015b370 0x7f2d4015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:18.221 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d46f48640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400b3a40 con 0x7f2d400b05a0 2023-11-26T07:17:18.221 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d46f48640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4015d250 con 0x7f2d4015b370 2023-11-26T07:17:18.221 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d46f48640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4015d530 con 0x7f2d400d14c0 2023-11-26T07:17:18.221 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d45745640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d400d14c0 0x7f2d40144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:18.221 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d45745640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d400d14c0 0x7f2d40144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:60660/0 (socket says 172.21.15.31:60660) 2023-11-26T07:17:18.221 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d45745640 1 -- 172.21.15.31:0/2003688572 learned_addr learned my addr 172.21.15.31:0/2003688572 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:18.222 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d45f46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d400b05a0 0x7f2d400b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:18.222 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d46747640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4015b370 0x7f2d4015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:18.222 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d45745640 1 -- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4015b370 msgr2=0x7f2d4015ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:18.222 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d45745640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4015b370 0x7f2d4015ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:18.222 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d45745640 1 -- 172.21.15.31:0/2003688572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d400b05a0 msgr2=0x7f2d400b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:18.222 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d45745640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d400b05a0 0x7f2d400b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:18.222 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d45745640 1 -- 172.21.15.31:0/2003688572 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d38009b30 con 0x7f2d400d14c0 2023-11-26T07:17:18.223 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d45745640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d400d14c0 0x7f2d40144e70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000b550 tx=0x7f2d3000ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:18.223 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.218+0000 7f2d367fc640 1 -- 172.21.15.31:0/2003688572 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30010870 con 0x7f2d400d14c0 2023-11-26T07:17:18.223 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.222+0000 7f2d367fc640 1 -- 172.21.15.31:0/2003688572 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d300109f0 con 0x7f2d400d14c0 2023-11-26T07:17:18.223 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.222+0000 7f2d367fc640 1 -- 172.21.15.31:0/2003688572 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30010da0 con 0x7f2d400d14c0 2023-11-26T07:17:18.223 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.222+0000 7f2d46f48640 1 -- 172.21.15.31:0/2003688572 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4015d810 con 0x7f2d400d14c0 2023-11-26T07:17:18.224 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.222+0000 7f2d46f48640 1 -- 172.21.15.31:0/2003688572 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4015e010 con 0x7f2d400d14c0 2023-11-26T07:17:18.226 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.222+0000 7f2d46f48640 1 -- 172.21.15.31:0/2003688572 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d28005260 con 0x7f2d400d14c0 2023-11-26T07:17:18.227 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.226+0000 7f2d367fc640 1 -- 172.21.15.31:0/2003688572 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 4) v1 ==== 85796+0+0 (secure 0 0 0) 0x7f2d3000f3c0 con 0x7f2d400d14c0 2023-11-26T07:17:18.228 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.226+0000 7f2d367fc640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d240663e0 0x7f2d240688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:18.228 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.226+0000 7f2d367fc640 1 -- 172.21.15.31:0/2003688572 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f2d30082f20 con 0x7f2d400d14c0 2023-11-26T07:17:18.228 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.226+0000 7f2d45f46640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d240663e0 0x7f2d240688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:18.229 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.226+0000 7f2d45f46640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d240663e0 0x7f2d240688a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00b810 tx=0x7f2d3c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:18.244 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.242+0000 7f2d367fc640 1 -- 172.21.15.31:0/2003688572 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d30051300 con 0x7f2d400d14c0 2023-11-26T07:17:18.589 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.586+0000 7f2d46f48640 1 -- 172.21.15.31:0/2003688572 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d28005510 con 0x7f2d400d14c0 2023-11-26T07:17:18.591 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.590+0000 7f2d367fc640 1 -- 172.21.15.31:0/2003688572 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2d30014090 con 0x7f2d400d14c0 2023-11-26T07:17:18.594 INFO:teuthology.misc.health.smithi031.stdout: 2023-11-26T07:17:18.595 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":12,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:16.319984+0000","last_up_change":"0.000000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:18.595 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.590+0000 7f2d46f48640 1 -- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d240663e0 msgr2=0x7f2d240688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:18.596 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.590+0000 7f2d46f48640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d240663e0 0x7f2d240688a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00b810 tx=0x7f2d3c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:18.596 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.594+0000 7f2d46f48640 1 -- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d400d14c0 msgr2=0x7f2d40144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:18.596 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.594+0000 7f2d46f48640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d400d14c0 0x7f2d40144e70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000b550 tx=0x7f2d3000ba10 comp rx=0 tx=0).stop 2023-11-26T07:17:18.596 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.594+0000 7f2d46f48640 1 -- 172.21.15.31:0/2003688572 shutdown_connections 2023-11-26T07:17:18.596 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.594+0000 7f2d46f48640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d240663e0 0x7f2d240688a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:18.596 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.594+0000 7f2d46f48640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d4015b370 0x7f2d4015ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:18.596 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.594+0000 7f2d46f48640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d400d14c0 0x7f2d40144e70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:18.596 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.594+0000 7f2d46f48640 1 --2- 172.21.15.31:0/2003688572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d400b05a0 0x7f2d400b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:18.597 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.594+0000 7f2d46f48640 1 -- 172.21.15.31:0/2003688572 >> 172.21.15.31:0/2003688572 conn(0x7f2d4006e8d0 msgr2=0x7f2d400ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:18.597 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.594+0000 7f2d46f48640 1 -- 172.21.15.31:0/2003688572 shutdown_connections 2023-11-26T07:17:18.597 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:18.594+0000 7f2d46f48640 1 -- 172.21.15.31:0/2003688572 wait complete. 2023-11-26T07:17:18.627 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-26T07:17:18.785 INFO:tasks.ceph.osd.6.smithi097.stderr:2023-11-26T07:17:18.780+0000 7f7cb4e10640 -1 osd.6 0 waiting for initial osdmap 2023-11-26T07:17:18.799 INFO:tasks.ceph.osd.4.smithi097.stderr:2023-11-26T07:17:18.796+0000 7f0db74c5640 -1 osd.4 0 waiting for initial osdmap 2023-11-26T07:17:18.812 INFO:tasks.ceph.osd.6.smithi097.stderr:2023-11-26T07:17:18.808+0000 7f7cad45d640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:18.822 INFO:tasks.ceph.osd.7.smithi097.stderr:2023-11-26T07:17:18.820+0000 7fc5c0471640 -1 osd.7 0 waiting for initial osdmap 2023-11-26T07:17:18.826 INFO:tasks.ceph.osd.5.smithi097.stderr:2023-11-26T07:17:18.820+0000 7f817c73d640 -1 osd.5 0 waiting for initial osdmap 2023-11-26T07:17:18.838 INFO:tasks.ceph.osd.4.smithi097.stderr:2023-11-26T07:17:18.832+0000 7f0dafb12640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:18.847 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-11-26T07:17:18.846+0000 7fd224193640 -1 osd.0 0 waiting for initial osdmap 2023-11-26T07:17:18.856 INFO:tasks.ceph.osd.5.smithi097.stderr:2023-11-26T07:17:18.852+0000 7f8174d8a640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:18.856 INFO:tasks.ceph.osd.7.smithi097.stderr:2023-11-26T07:17:18.852+0000 7fc5b8abe640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:18.875 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-11-26T07:17:18.874+0000 7f9ed4b83640 -1 osd.2 0 waiting for initial osdmap 2023-11-26T07:17:18.877 INFO:tasks.ceph.osd.0.smithi031.stderr:2023-11-26T07:17:18.874+0000 7fd21c7e0640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:18.897 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-11-26T07:17:18.894+0000 7efdfaa47640 -1 osd.3 0 waiting for initial osdmap 2023-11-26T07:17:18.901 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-11-26T07:17:18.898+0000 7f31dac71640 -1 osd.1 0 waiting for initial osdmap 2023-11-26T07:17:18.912 INFO:tasks.ceph.osd.2.smithi031.stderr:2023-11-26T07:17:18.910+0000 7f9ecb99a640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:18.928 INFO:tasks.ceph.osd.1.smithi031.stderr:2023-11-26T07:17:18.926+0000 7f31d32be640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:18.932 INFO:tasks.ceph.osd.3.smithi031.stderr:2023-11-26T07:17:18.930+0000 7efdf3094640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-26T07:17:25.433 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:25.576 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5ae49c640 1 Processor -- start 2023-11-26T07:17:25.576 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5ae49c640 1 -- start start 2023-11-26T07:17:25.578 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5ae49c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc5a80d1a80 0x7fc5a80d1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.578 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5ae49c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80b6f20 0x7fc5a80b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.578 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5ae49c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80d2420 con 0x7fc5a8160c40 2023-11-26T07:17:25.578 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5ae49c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80d0ac0 con 0x7fc5a80d1a80 2023-11-26T07:17:25.578 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5ae49c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80b6a90 con 0x7fc5a80b6f20 2023-11-26T07:17:25.578 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5a7fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc5a8160c40 0x7fc5a80cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44030/0 (socket says 172.21.15.31:44030) 2023-11-26T07:17:25.578 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5a7fff640 1 -- 172.21.15.31:0/2648510480 learned_addr learned my addr 172.21.15.31:0/2648510480 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:25.578 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5aca39640 1 --2- 172.21.15.31:0/2648510480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80b6f20 0x7fc5a80b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.579 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5a77fe640 1 --2- 172.21.15.31:0/2648510480 >> v2:172.21.15.97:3300/0 conn(0x7fc5a80d1a80 0x7fc5a80d1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.579 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5a6ffd640 1 -- 172.21.15.31:0/2648510480 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515513579 0 0) 0x7fc5a80d2420 con 0x7fc5a8160c40 2023-11-26T07:17:25.579 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5a6ffd640 1 -- 172.21.15.31:0/2648510480 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc590003530 con 0x7fc5a8160c40 2023-11-26T07:17:25.579 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5aca39640 1 -- 172.21.15.31:0/2648510480 >> v2:172.21.15.97:3300/0 conn(0x7fc5a80d1a80 msgr2=0x7fc5a80d1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:25.579 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5a6ffd640 1 -- 172.21.15.31:0/2648510480 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3853307220 0 0) 0x7fc590003530 con 0x7fc5a8160c40 2023-11-26T07:17:25.580 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5aca39640 1 --2- 172.21.15.31:0/2648510480 >> v2:172.21.15.97:3300/0 conn(0x7fc5a80d1a80 0x7fc5a80d1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.580 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5aca39640 1 -- 172.21.15.31:0/2648510480 >> v1:172.21.15.31:6789/0 conn(0x7fc5a8160c40 legacy=0x7fc5a80cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:25.580 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5aca39640 1 -- 172.21.15.31:0/2648510480 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a80d07e0 con 0x7fc5a80b6f20 2023-11-26T07:17:25.580 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.573+0000 7fc5a77fe640 1 --2- 172.21.15.31:0/2648510480 >> v2:172.21.15.97:3300/0 conn(0x7fc5a80d1a80 0x7fc5a80d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:25.582 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5aca39640 1 --2- 172.21.15.31:0/2648510480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80b6f20 0x7fc5a80b73b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009d90 tx=0x7fc5a001c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f67dbb579739bcc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:25.582 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5a6ffd640 1 -- 172.21.15.31:0/2648510480 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598001680 con 0x7fc5a80b6f20 2023-11-26T07:17:25.582 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5a6ffd640 1 -- 172.21.15.31:0/2648510480 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a00a6450 con 0x7fc5a80b6f20 2023-11-26T07:17:25.582 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5a6ffd640 1 -- 172.21.15.31:0/2648510480 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00a67c0 con 0x7fc5a80b6f20 2023-11-26T07:17:25.583 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2648510480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80b6f20 msgr2=0x7fc5a80b73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:25.583 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5ae49c640 1 --2- 172.21.15.31:0/2648510480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80b6f20 0x7fc5a80b73b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009d90 tx=0x7fc5a001c750 comp rx=0 tx=0).stop 2023-11-26T07:17:25.583 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2648510480 shutdown_connections 2023-11-26T07:17:25.583 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5ae49c640 1 --2- 172.21.15.31:0/2648510480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80b6f20 0x7fc5a80b73b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.583 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5ae49c640 1 --2- 172.21.15.31:0/2648510480 >> v2:172.21.15.97:3300/0 conn(0x7fc5a80d1a80 0x7fc5a80d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.583 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2648510480 >> 172.21.15.31:0/2648510480 conn(0x7fc5a808afc0 msgr2=0x7fc5a80d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:25.584 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.577+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2648510480 shutdown_connections 2023-11-26T07:17:25.584 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2648510480 wait complete. 2023-11-26T07:17:25.584 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5ae49c640 1 Processor -- start 2023-11-26T07:17:25.585 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5ae49c640 1 -- start start 2023-11-26T07:17:25.586 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5ae49c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a80b6f20 0x7fc5a815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.586 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5ae49c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80d1a80 0x7fc5a81497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5ae49c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8160c40 0x7fc5a80c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5ae49c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8157990 con 0x7fc5a8160c40 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5ae49c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8157c30 con 0x7fc5a80b6f20 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5ae49c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8157f10 con 0x7fc5a80d1a80 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5aca39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8160c40 0x7fc5a80c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5aca39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8160c40 0x7fc5a80c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52106/0 (socket says 172.21.15.31:52106) 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5aca39640 1 -- 172.21.15.31:0/2193158235 learned_addr learned my addr 172.21.15.31:0/2193158235 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5a7fff640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a80b6f20 0x7fc5a815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5a77fe640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80d1a80 0x7fc5a81497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5aca39640 1 -- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80d1a80 msgr2=0x7fc5a81497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:25.587 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5aca39640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80d1a80 0x7fc5a81497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.588 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5aca39640 1 -- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a80b6f20 msgr2=0x7fc5a815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:25.588 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5aca39640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a80b6f20 0x7fc5a815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.588 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5aca39640 1 -- 172.21.15.31:0/2193158235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c009420 con 0x7fc5a8160c40 2023-11-26T07:17:25.588 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.581+0000 7fc5aca39640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8160c40 0x7fc5a80c92c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc5a001cff0 tx=0x7fc5a00194f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:25.589 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.585+0000 7fc5a4ff9640 1 -- 172.21.15.31:0/2193158235 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00b7070 con 0x7fc5a8160c40 2023-11-26T07:17:25.589 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.585+0000 7fc5a4ff9640 1 -- 172.21.15.31:0/2193158235 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0004250 con 0x7fc5a8160c40 2023-11-26T07:17:25.589 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.585+0000 7fc5a4ff9640 1 -- 172.21.15.31:0/2193158235 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00045e0 con 0x7fc5a8160c40 2023-11-26T07:17:25.589 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.585+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2193158235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a0009a70 con 0x7fc5a8160c40 2023-11-26T07:17:25.589 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.585+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2193158235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a80d05b0 con 0x7fc5a8160c40 2023-11-26T07:17:25.589 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.585+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2193158235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a804e850 con 0x7fc5a8160c40 2023-11-26T07:17:25.589 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.585+0000 7fc5a4ff9640 1 -- 172.21.15.31:0/2193158235 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc5a0004740 con 0x7fc5a8160c40 2023-11-26T07:17:25.590 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.585+0000 7fc5a4ff9640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc594066470 0x7fc594068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.591 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.585+0000 7fc5a4ff9640 1 -- 172.21.15.31:0/2193158235 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc5a011e1d0 con 0x7fc5a8160c40 2023-11-26T07:17:25.591 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.589+0000 7fc5a7fff640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc594066470 0x7fc594068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.592 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.589+0000 7fc5a7fff640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc594066470 0x7fc594068930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc598006ae0 tx=0x7fc598001d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:25.595 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.593+0000 7fc5a4ff9640 1 -- 172.21.15.31:0/2193158235 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc5a00eba50 con 0x7fc5a8160c40 2023-11-26T07:17:25.753 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.749+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2193158235 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5a815d0e0 con 0x7fc5a8160c40 2023-11-26T07:17:25.754 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.749+0000 7fc5a4ff9640 1 -- 172.21.15.31:0/2193158235 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12766 (secure 0 0 0) 0x7fc5a00ef070 con 0x7fc5a8160c40 2023-11-26T07:17:25.755 INFO:teuthology.misc.health.smithi031.stdout: 2023-11-26T07:17:25.755 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":16,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:22.986827+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:25.757 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc594066470 msgr2=0x7fc594068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:25.757 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc594066470 0x7fc594068930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc598006ae0 tx=0x7fc598001d30 comp rx=0 tx=0).stop 2023-11-26T07:17:25.757 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2193158235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8160c40 msgr2=0x7fc5a80c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:25.757 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8160c40 0x7fc5a80c92c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc5a001cff0 tx=0x7fc5a00194f0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.757 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2193158235 shutdown_connections 2023-11-26T07:17:25.757 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc594066470 0x7fc594068930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.757 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5a8160c40 0x7fc5a80c92c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.757 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5a80d1a80 0x7fc5a81497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.757 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 --2- 172.21.15.31:0/2193158235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a80b6f20 0x7fc5a815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.757 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2193158235 >> 172.21.15.31:0/2193158235 conn(0x7fc5a808afc0 msgr2=0x7fc5a80d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:25.758 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2193158235 shutdown_connections 2023-11-26T07:17:25.758 INFO:teuthology.misc.health.smithi031.stderr:2023-11-26T07:17:25.753+0000 7fc5ae49c640 1 -- 172.21.15.31:0/2193158235 wait complete. 2023-11-26T07:17:25.774 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-26T07:17:25.774 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-26T07:17:25.775 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:17:25.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:17:25.778 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:17:25.779 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:17:25.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:17:25.782 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:17:25.782 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:17:25.782 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:17:25.787 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:17:25.787 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:17:25.788 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:17:25.791 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:17:25.791 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-26T07:17:25.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:25.873 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-11-26T07:17:25.874 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-11-26T07:17:25.891 INFO:tasks.ceph.mds.d.smithi097.stdout:starting mds.d at 2023-11-26T07:17:25.892 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2023-11-26T07:17:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f738a3a1640 1 Processor -- start 2023-11-26T07:17:25.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f738a3a1640 1 -- start start 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f738a3a1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f738415d7b0 0x7f73840d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f738a3a1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f73840d2080 0x7f738406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f738a3a1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73840cefd0 con 0x7f73840cead0 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f738a3a1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73840cf2b0 con 0x7f738415d7b0 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f738a3a1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738406b3f0 con 0x7f73840d2080 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f7383fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f738415d7b0 0x7f73840d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f737bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f73840cead0 0x7f73840d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44040/0 (socket says 172.21.15.31:44040) 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f737bfff640 1 -- 172.21.15.31:0/2800499032 learned_addr learned my addr 172.21.15.31:0/2800499032 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f738893e640 1 --2- 172.21.15.31:0/2800499032 >> v2:172.21.15.97:3300/0 conn(0x7f73840d2080 0x7f738406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f7383fff640 1 -- 172.21.15.31:0/2800499032 >> v2:172.21.15.97:3300/0 conn(0x7f73840d2080 msgr2=0x7f738406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f7383fff640 1 --2- 172.21.15.31:0/2800499032 >> v2:172.21.15.97:3300/0 conn(0x7f73840d2080 0x7f738406bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f7383fff640 1 -- 172.21.15.31:0/2800499032 >> v1:172.21.15.31:6789/0 conn(0x7f73840cead0 legacy=0x7f73840d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:25.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.937+0000 7f7383fff640 1 -- 172.21.15.31:0/2800499032 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738406b6d0 con 0x7f738415d7b0 2023-11-26T07:17:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f7383fff640 1 --2- 172.21.15.31:0/2800499032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 0x7f73840d10d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f738408c870 tx=0x7f737401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcf12efa88eb8d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f73837fe640 1 -- 172.21.15.31:0/2800499032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73740ab600 con 0x7f738415d7b0 2023-11-26T07:17:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f73837fe640 1 -- 172.21.15.31:0/2800499032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73740a7050 con 0x7f738415d7b0 2023-11-26T07:17:25.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f73837fe640 1 -- 172.21.15.31:0/2800499032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73740ab980 con 0x7f738415d7b0 2023-11-26T07:17:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f738a3a1640 1 -- 172.21.15.31:0/2800499032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 msgr2=0x7f73840d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f738a3a1640 1 --2- 172.21.15.31:0/2800499032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 0x7f73840d10d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f738408c870 tx=0x7f737401c310 comp rx=0 tx=0).stop 2023-11-26T07:17:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f738a3a1640 1 -- 172.21.15.31:0/2800499032 shutdown_connections 2023-11-26T07:17:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f738a3a1640 1 --2- 172.21.15.31:0/2800499032 >> v2:172.21.15.97:3300/0 conn(0x7f73840d2080 0x7f738406bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f738a3a1640 1 --2- 172.21.15.31:0/2800499032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 0x7f73840d10d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f738a3a1640 1 -- 172.21.15.31:0/2800499032 >> 172.21.15.31:0/2800499032 conn(0x7f73840d33b0 msgr2=0x7f73840d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f738a3a1640 1 -- 172.21.15.31:0/2800499032 shutdown_connections 2023-11-26T07:17:25.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.941+0000 7f738a3a1640 1 -- 172.21.15.31:0/2800499032 wait complete. 2023-11-26T07:17:25.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738a3a1640 1 Processor -- start 2023-11-26T07:17:25.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738a3a1640 1 -- start start 2023-11-26T07:17:25.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738a3a1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73840cead0 0x7f73840c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738a3a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73840d2080 0x7f738415c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738a3a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 0x7f73840b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738a3a1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840b39f0 con 0x7f738415d7b0 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738a3a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73840c9e20 con 0x7f73840d2080 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738a3a1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840b3bd0 con 0x7f73840cead0 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738893e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 0x7f73840b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738893e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 0x7f73840b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52160/0 (socket says 172.21.15.31:52160) 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738893e640 1 -- 172.21.15.31:0/2345055184 learned_addr learned my addr 172.21.15.31:0/2345055184 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f737bfff640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73840d2080 0x7f738415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f7383fff640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73840cead0 0x7f73840c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738893e640 1 -- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73840cead0 msgr2=0x7f73840c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738893e640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73840cead0 0x7f73840c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738893e640 1 -- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73840d2080 msgr2=0x7f738415c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738893e640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73840d2080 0x7f738415c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738893e640 1 -- 172.21.15.31:0/2345055184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7374009420 con 0x7f738415d7b0 2023-11-26T07:17:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738893e640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 0x7f73840b2e10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f737c007cf0 tx=0x7f737c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f73817fa640 1 -- 172.21.15.31:0/2345055184 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c014420 con 0x7f738415d7b0 2023-11-26T07:17:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f73817fa640 1 -- 172.21.15.31:0/2345055184 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c0145a0 con 0x7f738415d7b0 2023-11-26T07:17:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f73817fa640 1 -- 172.21.15.31:0/2345055184 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c014950 con 0x7f738415d7b0 2023-11-26T07:17:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738a3a1640 1 -- 172.21.15.31:0/2345055184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73840b3eb0 con 0x7f738415d7b0 2023-11-26T07:17:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.945+0000 7f738a3a1640 1 -- 172.21.15.31:0/2345055184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738415c2e0 con 0x7f738415d7b0 2023-11-26T07:17:25.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.949+0000 7f738a3a1640 1 -- 172.21.15.31:0/2345055184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738415be40 con 0x7f738415d7b0 2023-11-26T07:17:25.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.949+0000 7f73817fa640 1 -- 172.21.15.31:0/2345055184 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f737c014ab0 con 0x7f738415d7b0 2023-11-26T07:17:25.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.949+0000 7f73817fa640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73640664a0 0x7f7364068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:25.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.949+0000 7f73817fa640 1 -- 172.21.15.31:0/2345055184 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f737c0841f0 con 0x7f738415d7b0 2023-11-26T07:17:25.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.949+0000 7f7383fff640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73640664a0 0x7f7364068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:25.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.949+0000 7f7383fff640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73640664a0 0x7f7364068960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73740ad000 tx=0x7f7374008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:25.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:25.953+0000 7f73817fa640 1 -- 172.21.15.31:0/2345055184 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f737c051a60 con 0x7f738415d7b0 2023-11-26T07:17:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.113+0000 7f738a3a1640 1 -- 172.21.15.31:0/2345055184 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f738415c500 con 0x7f738415d7b0 2023-11-26T07:17:26.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.113+0000 7f73817fa640 1 -- 172.21.15.31:0/2345055184 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12766 (secure 0 0 0) 0x7f737c055080 con 0x7f738415d7b0 2023-11-26T07:17:26.118 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:26.118 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:22.986827+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:26.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 -- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73640664a0 msgr2=0x7f7364068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73640664a0 0x7f7364068960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73740ad000 tx=0x7f7374008040 comp rx=0 tx=0).stop 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 -- 172.21.15.31:0/2345055184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 msgr2=0x7f73840b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 0x7f73840b2e10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f737c007cf0 tx=0x7f737c00e4b0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 -- 172.21.15.31:0/2345055184 shutdown_connections 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73640664a0 0x7f7364068960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f738415d7b0 0x7f73840b2e10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73840d2080 0x7f738415c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 --2- 172.21.15.31:0/2345055184 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73840cead0 0x7f73840c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 -- 172.21.15.31:0/2345055184 >> 172.21.15.31:0/2345055184 conn(0x7f73840d33b0 msgr2=0x7f73840b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 -- 172.21.15.31:0/2345055184 shutdown_connections 2023-11-26T07:17:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.117+0000 7f738a3a1640 1 -- 172.21.15.31:0/2345055184 wait complete. 2023-11-26T07:17:26.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:26.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.281+0000 7f96f74ea640 1 Processor -- start 2023-11-26T07:17:26.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.281+0000 7f96f74ea640 1 -- start start 2023-11-26T07:17:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f74ea640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f96f015a710 0x7f96f00abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f74ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96f00cbd50 0x7f96f00cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f74ea640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f00b2710 con 0x7f96f00cdfb0 2023-11-26T07:17:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f74ea640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f00cca80 con 0x7f96f015a710 2023-11-26T07:17:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f74ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00b29f0 con 0x7f96f00cbd50 2023-11-26T07:17:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f4a85640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f96f015a710 0x7f96f00abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f5286640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f96f00cdfb0 0x7f96f00ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44046/0 (socket says 172.21.15.31:44046) 2023-11-26T07:17:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f4a85640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f96f015a710 0x7f96f00abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52164/0 (socket says 172.21.15.31:52164) 2023-11-26T07:17:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f5286640 1 -- 172.21.15.31:0/3912058991 learned_addr learned my addr 172.21.15.31:0/3912058991 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:26.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f5a87640 1 --2- 172.21.15.31:0/3912058991 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96f00cbd50 0x7f96f00cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f4a85640 1 -- 172.21.15.31:0/3912058991 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96f00cbd50 msgr2=0x7f96f00cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f4a85640 1 --2- 172.21.15.31:0/3912058991 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96f00cbd50 0x7f96f00cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f4a85640 1 -- 172.21.15.31:0/3912058991 >> v1:172.21.15.31:6789/0 conn(0x7f96f00cdfb0 legacy=0x7f96f00ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96f4a85640 1 -- 172.21.15.31:0/3912058991 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f00b2cd0 con 0x7f96f015a710 2023-11-26T07:17:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.285+0000 7f96e7fff640 1 -- 172.21.15.31:0/3912058991 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567285537 0 0) 0x7f96f00b2710 con 0x7f96f00cdfb0 2023-11-26T07:17:26.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f4a85640 1 --2- 172.21.15.31:0/3912058991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f015a710 0x7f96f00abf90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f96e000ba60 tx=0x7f96e001eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77ce169f17636b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:26.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96e7fff640 1 -- 172.21.15.31:0/3912058991 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00a7c10 con 0x7f96f015a710 2023-11-26T07:17:26.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96e7fff640 1 -- 172.21.15.31:0/3912058991 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e00a7d70 con 0x7f96f015a710 2023-11-26T07:17:26.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96e7fff640 1 -- 172.21.15.31:0/3912058991 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00af660 con 0x7f96f015a710 2023-11-26T07:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f74ea640 1 -- 172.21.15.31:0/3912058991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f015a710 msgr2=0x7f96f00abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f74ea640 1 --2- 172.21.15.31:0/3912058991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f015a710 0x7f96f00abf90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f96e000ba60 tx=0x7f96e001eb20 comp rx=0 tx=0).stop 2023-11-26T07:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f74ea640 1 -- 172.21.15.31:0/3912058991 shutdown_connections 2023-11-26T07:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f74ea640 1 --2- 172.21.15.31:0/3912058991 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96f00cbd50 0x7f96f00cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f74ea640 1 --2- 172.21.15.31:0/3912058991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f015a710 0x7f96f00abf90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f74ea640 1 -- 172.21.15.31:0/3912058991 >> 172.21.15.31:0/3912058991 conn(0x7f96f00b0cb0 msgr2=0x7f96f00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f74ea640 1 -- 172.21.15.31:0/3912058991 shutdown_connections 2023-11-26T07:17:26.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f74ea640 1 -- 172.21.15.31:0/3912058991 wait complete. 2023-11-26T07:17:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f74ea640 1 Processor -- start 2023-11-26T07:17:26.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.289+0000 7f96f74ea640 1 -- start start 2023-11-26T07:17:26.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f74ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f00cbd50 0x7f96f01504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f74ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00cdfb0 0x7f96f0159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f74ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96f015a710 0x7f96f014fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f74ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0152650 con 0x7f96f00cbd50 2023-11-26T07:17:26.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f74ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0150ac0 con 0x7f96f00cdfb0 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f74ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f0151db0 con 0x7f96f015a710 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f5286640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f00cbd50 0x7f96f01504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f5286640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f00cbd50 0x7f96f01504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52176/0 (socket says 172.21.15.31:52176) 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f5286640 1 -- 172.21.15.31:0/683136684 learned_addr learned my addr 172.21.15.31:0/683136684 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f5a87640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96f015a710 0x7f96f014fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f4a85640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00cdfb0 0x7f96f0159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f5286640 1 -- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96f015a710 msgr2=0x7f96f014fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f5286640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96f015a710 0x7f96f014fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f5286640 1 -- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00cdfb0 msgr2=0x7f96f0159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f5286640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00cdfb0 0x7f96f0159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f5286640 1 -- 172.21.15.31:0/683136684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e000b6c0 con 0x7f96f00cbd50 2023-11-26T07:17:26.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f5286640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f00cbd50 0x7f96f01504e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0011f0 tx=0x7f96ec00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:26.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96e5ffb640 1 -- 172.21.15.31:0/683136684 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0074e0 con 0x7f96f00cbd50 2023-11-26T07:17:26.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96e5ffb640 1 -- 172.21.15.31:0/683136684 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec007640 con 0x7f96f00cbd50 2023-11-26T07:17:26.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96e5ffb640 1 -- 172.21.15.31:0/683136684 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec004a10 con 0x7f96f00cbd50 2023-11-26T07:17:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f74ea640 1 -- 172.21.15.31:0/683136684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f0152090 con 0x7f96f00cbd50 2023-11-26T07:17:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f74ea640 1 -- 172.21.15.31:0/683136684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f015a250 con 0x7f96f00cbd50 2023-11-26T07:17:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.293+0000 7f96f74ea640 1 -- 172.21.15.31:0/683136684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f00671f0 con 0x7f96f00cbd50 2023-11-26T07:17:26.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.297+0000 7f96e5ffb640 1 -- 172.21.15.31:0/683136684 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f96ec004b70 con 0x7f96f00cbd50 2023-11-26T07:17:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.297+0000 7f96e5ffb640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f96d40664c0 0x7f96d4068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.297+0000 7f96e5ffb640 1 -- 172.21.15.31:0/683136684 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f96ec0830a0 con 0x7f96f00cbd50 2023-11-26T07:17:26.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.297+0000 7f96f4a85640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f96d40664c0 0x7f96d4068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.297+0000 7f96f4a85640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f96d40664c0 0x7f96d4068980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f96e00b1000 tx=0x7f96e00a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:26.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.301+0000 7f96e5ffb640 1 -- 172.21.15.31:0/683136684 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f96ec0509e0 con 0x7f96f00cbd50 2023-11-26T07:17:26.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 -- 172.21.15.31:0/683136684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96f006b460 con 0x7f96f00cbd50 2023-11-26T07:17:26.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96e5ffb640 1 -- 172.21.15.31:0/683136684 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96ec054000 con 0x7f96f00cbd50 2023-11-26T07:17:26.465 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:17:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 -- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f96d40664c0 msgr2=0x7f96d4068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f96d40664c0 0x7f96d4068980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f96e00b1000 tx=0x7f96e00a9070 comp rx=0 tx=0).stop 2023-11-26T07:17:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 -- 172.21.15.31:0/683136684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f00cbd50 msgr2=0x7f96f01504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f00cbd50 0x7f96f01504e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0011f0 tx=0x7f96ec00c9b0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 -- 172.21.15.31:0/683136684 shutdown_connections 2023-11-26T07:17:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f96d40664c0 0x7f96d4068980 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f96f015a710 0x7f96f014fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96f00cdfb0 0x7f96f0159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 --2- 172.21.15.31:0/683136684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f00cbd50 0x7f96f01504e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 -- 172.21.15.31:0/683136684 >> 172.21.15.31:0/683136684 conn(0x7f96f00b0cb0 msgr2=0x7f96f01649e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:26.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 -- 172.21.15.31:0/683136684 shutdown_connections 2023-11-26T07:17:26.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.461+0000 7f96f74ea640 1 -- 172.21.15.31:0/683136684 wait complete. 2023-11-26T07:17:26.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:26.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f432118d640 1 Processor -- start 2023-11-26T07:17:26.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f432118d640 1 -- start start 2023-11-26T07:17:26.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f432118d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f431c15d7b0 0x7f431c0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f432118d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f431c0cead0 0x7f431c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f432118d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c0cefd0 con 0x7f431c0d1eb0 2023-11-26T07:17:26.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f432118d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c0d2340 con 0x7f431c0cead0 2023-11-26T07:17:26.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f432118d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0cf2b0 con 0x7f431c15d7b0 2023-11-26T07:17:26.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f4313fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f431c0cead0 0x7f431c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f431b59e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f431c0d1eb0 0x7f431c0cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44052/0 (socket says 172.21.15.31:44052) 2023-11-26T07:17:26.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f431b59e640 1 -- 172.21.15.31:0/3099169945 learned_addr learned my addr 172.21.15.31:0/3099169945 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:26.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f431ad9d640 1 --2- 172.21.15.31:0/3099169945 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f431c15d7b0 0x7f431c0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f4313fff640 1 -- 172.21.15.31:0/3099169945 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f431c15d7b0 msgr2=0x7f431c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:26.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f4313fff640 1 --2- 172.21.15.31:0/3099169945 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f431c15d7b0 0x7f431c0d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f4313fff640 1 -- 172.21.15.31:0/3099169945 >> v1:172.21.15.31:6789/0 conn(0x7f431c0d1eb0 legacy=0x7f431c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.625+0000 7f4313fff640 1 -- 172.21.15.31:0/3099169945 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c06b3f0 con 0x7f431c0cead0 2023-11-26T07:17:26.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f4313fff640 1 --2- 172.21.15.31:0/3099169945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c0cead0 0x7f431c0d1970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f43040097c0 tx=0x7f430401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b57dc2cfd115bc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:26.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f431a59c640 1 -- 172.21.15.31:0/3099169945 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43040ab600 con 0x7f431c0cead0 2023-11-26T07:17:26.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f431a59c640 1 -- 172.21.15.31:0/3099169945 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43040a7050 con 0x7f431c0cead0 2023-11-26T07:17:26.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f431a59c640 1 -- 172.21.15.31:0/3099169945 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43040ab980 con 0x7f431c0cead0 2023-11-26T07:17:26.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f432118d640 1 -- 172.21.15.31:0/3099169945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c0cead0 msgr2=0x7f431c0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f432118d640 1 --2- 172.21.15.31:0/3099169945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c0cead0 0x7f431c0d1970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f43040097c0 tx=0x7f430401c330 comp rx=0 tx=0).stop 2023-11-26T07:17:26.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f432118d640 1 -- 172.21.15.31:0/3099169945 shutdown_connections 2023-11-26T07:17:26.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f432118d640 1 --2- 172.21.15.31:0/3099169945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c0cead0 0x7f431c0d1970 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f432118d640 1 --2- 172.21.15.31:0/3099169945 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f431c15d7b0 0x7f431c0d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f432118d640 1 -- 172.21.15.31:0/3099169945 >> 172.21.15.31:0/3099169945 conn(0x7f431c0d33b0 msgr2=0x7f431c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:26.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f432118d640 1 -- 172.21.15.31:0/3099169945 shutdown_connections 2023-11-26T07:17:26.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.629+0000 7f432118d640 1 -- 172.21.15.31:0/3099169945 wait complete. 2023-11-26T07:17:26.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f432118d640 1 Processor -- start 2023-11-26T07:17:26.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f432118d640 1 -- start start 2023-11-26T07:17:26.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f432118d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c0cead0 0x7f431c153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f432118d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f431c0d1eb0 0x7f431c15c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f432118d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c15d7b0 0x7f431c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f432118d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0b92e0 con 0x7f431c15d7b0 2023-11-26T07:17:26.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f432118d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0b8240 con 0x7f431c0cead0 2023-11-26T07:17:26.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f432118d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0b8520 con 0x7f431c0d1eb0 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f431b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c15d7b0 0x7f431c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f431b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c15d7b0 0x7f431c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52198/0 (socket says 172.21.15.31:52198) 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f431b59e640 1 -- 172.21.15.31:0/3354525035 learned_addr learned my addr 172.21.15.31:0/3354525035 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f4313fff640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f431c0d1eb0 0x7f431c15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f431ad9d640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c0cead0 0x7f431c153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f431b59e640 1 -- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f431c0d1eb0 msgr2=0x7f431c15c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f431b59e640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f431c0d1eb0 0x7f431c15c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f431b59e640 1 -- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c0cead0 msgr2=0x7f431c153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f431b59e640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c0cead0 0x7f431c153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f431b59e640 1 -- 172.21.15.31:0/3354525035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4304009420 con 0x7f431c15d7b0 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f431b59e640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c15d7b0 0x7f431c0b5f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f431400caa0 tx=0x7f431400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:26.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f43137fe640 1 -- 172.21.15.31:0/3354525035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400ab40 con 0x7f431c15d7b0 2023-11-26T07:17:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f43137fe640 1 -- 172.21.15.31:0/3354525035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431400aca0 con 0x7f431c15d7b0 2023-11-26T07:17:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f43137fe640 1 -- 172.21.15.31:0/3354525035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314012600 con 0x7f431c15d7b0 2023-11-26T07:17:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f432118d640 1 -- 172.21.15.31:0/3354525035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c0b8800 con 0x7f431c15d7b0 2023-11-26T07:17:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.633+0000 7f432118d640 1 -- 172.21.15.31:0/3354525035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c0b8dc0 con 0x7f431c15d7b0 2023-11-26T07:17:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.637+0000 7f432118d640 1 -- 172.21.15.31:0/3354525035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c0b8fe0 con 0x7f431c15d7b0 2023-11-26T07:17:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.637+0000 7f43137fe640 1 -- 172.21.15.31:0/3354525035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4314007070 con 0x7f431c15d7b0 2023-11-26T07:17:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.637+0000 7f43137fe640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42fc066470 0x7f42fc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.637+0000 7f43137fe640 1 -- 172.21.15.31:0/3354525035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4314083940 con 0x7f431c15d7b0 2023-11-26T07:17:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.637+0000 7f431ad9d640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42fc066470 0x7f42fc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:26.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.637+0000 7f431ad9d640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42fc066470 0x7f42fc068930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f430c009b60 tx=0x7f430c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.641+0000 7f43137fe640 1 -- 172.21.15.31:0/3354525035 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4314051090 con 0x7f431c15d7b0 2023-11-26T07:17:26.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.829+0000 7f432118d640 1 -- 172.21.15.31:0/3354525035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f431c04e850 con 0x7f431c15d7b0 2023-11-26T07:17:26.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f43137fe640 1 -- 172.21.15.31:0/3354525035 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f4314017070 con 0x7f431c15d7b0 2023-11-26T07:17:26.836 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:26.836 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1312969281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1312969281},{"type":"v1","addr":"172.21.15.31:6834","nonce":1312969281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-26T07:17:26.836 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2023-11-26T07:17:26.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 -- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42fc066470 msgr2=0x7f42fc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42fc066470 0x7f42fc068930 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f430c009b60 tx=0x7f430c009190 comp rx=0 tx=0).stop 2023-11-26T07:17:26.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 -- 172.21.15.31:0/3354525035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c15d7b0 msgr2=0x7f431c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:26.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c15d7b0 0x7f431c0b5f40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f431400caa0 tx=0x7f431400cf60 comp rx=0 tx=0).stop 2023-11-26T07:17:26.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 -- 172.21.15.31:0/3354525035 shutdown_connections 2023-11-26T07:17:26.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42fc066470 0x7f42fc068930 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c15d7b0 0x7f431c0b5f40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f431c0d1eb0 0x7f431c15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 --2- 172.21.15.31:0/3354525035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f431c0cead0 0x7f431c153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:26.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 -- 172.21.15.31:0/3354525035 >> 172.21.15.31:0/3354525035 conn(0x7f431c0d33b0 msgr2=0x7f431c1563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:26.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 -- 172.21.15.31:0/3354525035 shutdown_connections 2023-11-26T07:17:26.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:26.833+0000 7f432118d640 1 -- 172.21.15.31:0/3354525035 wait complete. 2023-11-26T07:17:28.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:29.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1987e16640 1 Processor -- start 2023-11-26T07:17:29.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1987e16640 1 -- start start 2023-11-26T07:17:29.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1987e16640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f198015d7b0 0x7f19800d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1987e16640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19800cefd0 con 0x7f19800cead0 2023-11-26T07:17:29.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1987e16640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19800cf2b0 con 0x7f19800d2080 2023-11-26T07:17:29.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1987e16640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198006b3f0 con 0x7f198015d7b0 2023-11-26T07:17:29.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1985bb2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f198015d7b0 0x7f19800d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f19853b1640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f19800cead0 0x7f19800d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44064/0 (socket says 172.21.15.31:44064) 2023-11-26T07:17:29.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f19853b1640 1 -- 172.21.15.31:0/1772723458 learned_addr learned my addr 172.21.15.31:0/1772723458 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:29.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1985bb2640 1 --2- 172.21.15.31:0/1772723458 >> v2:172.21.15.31:3300/0 conn(0x7f198015d7b0 0x7f19800d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52212/0 (socket says 172.21.15.31:52212) 2023-11-26T07:17:29.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1985bb2640 1 -- 172.21.15.31:0/1772723458 >> v1:172.21.15.97:6789/0 conn(0x7f19800d2080 legacy=0x7f198006bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:29.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1985bb2640 1 -- 172.21.15.31:0/1772723458 >> v1:172.21.15.31:6789/0 conn(0x7f19800cead0 legacy=0x7f19800d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.009+0000 7f1985bb2640 1 -- 172.21.15.31:0/1772723458 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198006b6d0 con 0x7f198015d7b0 2023-11-26T07:17:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1985bb2640 1 --2- 172.21.15.31:0/1772723458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198015d7b0 0x7f19800d10d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f197c0097c0 tx=0x7f197c01c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dd36af86f0f59a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1984bb0640 1 -- 172.21.15.31:0/1772723458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0ac640 con 0x7f198015d7b0 2023-11-26T07:17:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1984bb0640 1 -- 172.21.15.31:0/1772723458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c0ac7a0 con 0x7f198015d7b0 2023-11-26T07:17:29.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1984bb0640 1 -- 172.21.15.31:0/1772723458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0acb30 con 0x7f198015d7b0 2023-11-26T07:17:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1987e16640 1 -- 172.21.15.31:0/1772723458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198015d7b0 msgr2=0x7f19800d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1987e16640 1 --2- 172.21.15.31:0/1772723458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198015d7b0 0x7f19800d10d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f197c0097c0 tx=0x7f197c01c2f0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1987e16640 1 -- 172.21.15.31:0/1772723458 shutdown_connections 2023-11-26T07:17:29.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1987e16640 1 --2- 172.21.15.31:0/1772723458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f198015d7b0 0x7f19800d10d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1987e16640 1 -- 172.21.15.31:0/1772723458 >> 172.21.15.31:0/1772723458 conn(0x7f19800d33b0 msgr2=0x7f19800d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1987e16640 1 -- 172.21.15.31:0/1772723458 shutdown_connections 2023-11-26T07:17:29.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.013+0000 7f1987e16640 1 -- 172.21.15.31:0/1772723458 wait complete. 2023-11-26T07:17:29.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1987e16640 1 Processor -- start 2023-11-26T07:17:29.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1987e16640 1 -- start start 2023-11-26T07:17:29.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1987e16640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19800cead0 0x7f1980150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1987e16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19800d2080 0x7f19800c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1987e16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198015d7b0 0x7f1980155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1987e16640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980150ba0 con 0x7f19800d2080 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1987e16640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19800b76a0 con 0x7f198015d7b0 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1987e16640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19800b7940 con 0x7f19800cead0 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19853b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19800d2080 0x7f19800c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19853b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19800d2080 0x7f19800c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52226/0 (socket says 172.21.15.31:52226) 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19853b1640 1 -- 172.21.15.31:0/715380296 learned_addr learned my addr 172.21.15.31:0/715380296 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1985bb2640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19800cead0 0x7f1980150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19863b3640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198015d7b0 0x7f1980155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19853b1640 1 -- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19800cead0 msgr2=0x7f1980150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19853b1640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19800cead0 0x7f1980150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19853b1640 1 -- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198015d7b0 msgr2=0x7f1980155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19853b1640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198015d7b0 0x7f1980155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19853b1640 1 -- 172.21.15.31:0/715380296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c009420 con 0x7f19800d2080 2023-11-26T07:17:29.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1985bb2640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19800cead0 0x7f1980150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:29.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19853b1640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19800d2080 0x7f19800c7db0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1970001ec0 tx=0x7f197000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19767fc640 1 -- 172.21.15.31:0/715380296 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970005cf0 con 0x7f19800d2080 2023-11-26T07:17:29.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19767fc640 1 -- 172.21.15.31:0/715380296 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1970005e50 con 0x7f19800d2080 2023-11-26T07:17:29.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f19767fc640 1 -- 172.21.15.31:0/715380296 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19700072d0 con 0x7f19800d2080 2023-11-26T07:17:29.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1987e16640 1 -- 172.21.15.31:0/715380296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19800b7c20 con 0x7f19800d2080 2023-11-26T07:17:29.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.017+0000 7f1987e16640 1 -- 172.21.15.31:0/715380296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19800c8720 con 0x7f19800d2080 2023-11-26T07:17:29.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.021+0000 7f1987e16640 1 -- 172.21.15.31:0/715380296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198004e850 con 0x7f19800d2080 2023-11-26T07:17:29.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.021+0000 7f19767fc640 1 -- 172.21.15.31:0/715380296 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1970007430 con 0x7f19800d2080 2023-11-26T07:17:29.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.021+0000 7f19767fc640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1964066470 0x7f1964068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.021+0000 7f19767fc640 1 -- 172.21.15.31:0/715380296 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1970082fc0 con 0x7f19800d2080 2023-11-26T07:17:29.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.021+0000 7f1985bb2640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1964066470 0x7f1964068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.021+0000 7f1985bb2640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1964066470 0x7f1964068930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f197c0ae000 tx=0x7f197c0a6070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.025+0000 7f19767fc640 1 -- 172.21.15.31:0/715380296 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1970050830 con 0x7f19800d2080 2023-11-26T07:17:29.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 -- 172.21.15.31:0/715380296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1980153dc0 con 0x7f19800d2080 2023-11-26T07:17:29.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f19767fc640 1 -- 172.21.15.31:0/715380296 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3691 (secure 0 0 0) 0x7f1970017090 con 0x7f19800d2080 2023-11-26T07:17:29.221 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:29.222 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4099326271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4099326271},{"type":"v1","addr":"172.21.15.31:6835","nonce":4099326271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2404164953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2404164953},{"type":"v1","addr":"172.21.15.97:6836","nonce":2404164953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1312969281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1312969281},{"type":"v1","addr":"172.21.15.31:6834","nonce":1312969281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/527056216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":527056216},{"type":"v1","addr":"172.21.15.97:6837","nonce":527056216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-26T07:17:29.222 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 3 2023-11-26T07:17:29.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 -- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1964066470 msgr2=0x7f1964068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1964066470 0x7f1964068930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f197c0ae000 tx=0x7f197c0a6070 comp rx=0 tx=0).stop 2023-11-26T07:17:29.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 -- 172.21.15.31:0/715380296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19800d2080 msgr2=0x7f19800c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19800d2080 0x7f19800c7db0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1970001ec0 tx=0x7f197000c890 comp rx=0 tx=0).stop 2023-11-26T07:17:29.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 -- 172.21.15.31:0/715380296 shutdown_connections 2023-11-26T07:17:29.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1964066470 0x7f1964068930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f198015d7b0 0x7f1980155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19800d2080 0x7f19800c7db0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 --2- 172.21.15.31:0/715380296 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19800cead0 0x7f1980150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 -- 172.21.15.31:0/715380296 >> 172.21.15.31:0/715380296 conn(0x7f19800d33b0 msgr2=0x7f19800b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 -- 172.21.15.31:0/715380296 shutdown_connections 2023-11-26T07:17:29.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.217+0000 7f1987e16640 1 -- 172.21.15.31:0/715380296 wait complete. 2023-11-26T07:17:29.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:29.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b2d0f640 1 Processor -- start 2023-11-26T07:17:29.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b2d0f640 1 -- start start 2023-11-26T07:17:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b2d0f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72ac15d7b0 0x7f72ac0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b2d0f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac0d2080 0x7f72ac06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b2d0f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ac0cefd0 con 0x7f72ac0cead0 2023-11-26T07:17:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b2d0f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ac0cf2b0 con 0x7f72ac15d7b0 2023-11-26T07:17:29.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b2d0f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac06b3f0 con 0x7f72ac0d2080 2023-11-26T07:17:29.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b12ac640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac0d2080 0x7f72ac06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b12ac640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac0d2080 0x7f72ac06bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:42952/0 (socket says 172.21.15.31:42952) 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72abfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f72ac0cead0 0x7f72ac0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:43406/0 (socket says 172.21.15.31:43406) 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b12ac640 1 -- 172.21.15.31:0/3661704516 learned_addr learned my addr 172.21.15.31:0/3661704516 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.381+0000 7f72b0aab640 1 --2- 172.21.15.31:0/3661704516 >> v2:172.21.15.97:3300/0 conn(0x7f72ac15d7b0 0x7f72ac0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72ab7fe640 1 -- 172.21.15.31:0/3661704516 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503082004 0 0) 0x7f72ac0cefd0 con 0x7f72ac0cead0 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72ab7fe640 1 -- 172.21.15.31:0/3661704516 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f729c003530 con 0x7f72ac0cead0 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b0aab640 1 -- 172.21.15.31:0/3661704516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac0d2080 msgr2=0x7f72ac06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b0aab640 1 --2- 172.21.15.31:0/3661704516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac0d2080 0x7f72ac06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b0aab640 1 -- 172.21.15.31:0/3661704516 >> v1:172.21.15.97:6789/0 conn(0x7f72ac0cead0 legacy=0x7f72ac0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b0aab640 1 -- 172.21.15.31:0/3661704516 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac06b6d0 con 0x7f72ac15d7b0 2023-11-26T07:17:29.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b12ac640 1 --2- 172.21.15.31:0/3661704516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac0d2080 0x7f72ac06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:29.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b0aab640 1 --2- 172.21.15.31:0/3661704516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ac15d7b0 0x7f72ac0d10d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f72a00097c0 tx=0x7f72a001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff9e7a73fc92660 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72ab7fe640 1 -- 172.21.15.31:0/3661704516 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0004230 con 0x7f72ac15d7b0 2023-11-26T07:17:29.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72ab7fe640 1 -- 172.21.15.31:0/3661704516 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a00a7050 con 0x7f72ac15d7b0 2023-11-26T07:17:29.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72ab7fe640 1 -- 172.21.15.31:0/3661704516 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0003620 con 0x7f72ac15d7b0 2023-11-26T07:17:29.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b2d0f640 1 -- 172.21.15.31:0/3661704516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ac15d7b0 msgr2=0x7f72ac0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b2d0f640 1 --2- 172.21.15.31:0/3661704516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ac15d7b0 0x7f72ac0d10d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f72a00097c0 tx=0x7f72a001c330 comp rx=0 tx=0).stop 2023-11-26T07:17:29.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b2d0f640 1 -- 172.21.15.31:0/3661704516 shutdown_connections 2023-11-26T07:17:29.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b2d0f640 1 --2- 172.21.15.31:0/3661704516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac0d2080 0x7f72ac06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b2d0f640 1 --2- 172.21.15.31:0/3661704516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ac15d7b0 0x7f72ac0d10d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b2d0f640 1 -- 172.21.15.31:0/3661704516 >> 172.21.15.31:0/3661704516 conn(0x7f72ac0d33b0 msgr2=0x7f72ac0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b2d0f640 1 -- 172.21.15.31:0/3661704516 shutdown_connections 2023-11-26T07:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.385+0000 7f72b2d0f640 1 -- 172.21.15.31:0/3661704516 wait complete. 2023-11-26T07:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b2d0f640 1 Processor -- start 2023-11-26T07:17:29.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b2d0f640 1 -- start start 2023-11-26T07:17:29.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b2d0f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ac0cead0 0x7f72ac142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b2d0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ac0d2080 0x7f72ac0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b2d0f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac15d7b0 0x7f72ac154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b2d0f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac143140 con 0x7f72ac0d2080 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b2d0f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0ccec0 con 0x7f72ac0cead0 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b2d0f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0cd1a0 con 0x7f72ac15d7b0 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ac0d2080 0x7f72ac0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ac0d2080 0x7f72ac0c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52240/0 (socket says 172.21.15.31:52240) 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72abfff640 1 -- 172.21.15.31:0/1892602720 learned_addr learned my addr 172.21.15.31:0/1892602720 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b0aab640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ac0cead0 0x7f72ac142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b12ac640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac15d7b0 0x7f72ac154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72abfff640 1 -- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac15d7b0 msgr2=0x7f72ac154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72abfff640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac15d7b0 0x7f72ac154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72abfff640 1 -- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ac0cead0 msgr2=0x7f72ac142c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72abfff640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ac0cead0 0x7f72ac142c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72abfff640 1 -- 172.21.15.31:0/1892602720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a4009b30 con 0x7f72ac0d2080 2023-11-26T07:17:29.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72abfff640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ac0d2080 0x7f72ac0c95f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f729800c430 tx=0x7f729800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72a97fa640 1 -- 172.21.15.31:0/1892602720 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980074e0 con 0x7f72ac0d2080 2023-11-26T07:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72a97fa640 1 -- 172.21.15.31:0/1892602720 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7298007660 con 0x7f72ac0d2080 2023-11-26T07:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72a97fa640 1 -- 172.21.15.31:0/1892602720 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298004a30 con 0x7f72ac0d2080 2023-11-26T07:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.389+0000 7f72b2d0f640 1 -- 172.21.15.31:0/1892602720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a0009420 con 0x7f72ac0d2080 2023-11-26T07:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.393+0000 7f72b2d0f640 1 -- 172.21.15.31:0/1892602720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac154550 con 0x7f72ac0d2080 2023-11-26T07:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.393+0000 7f72b2d0f640 1 -- 172.21.15.31:0/1892602720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ac04e850 con 0x7f72ac0d2080 2023-11-26T07:17:29.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.393+0000 7f72a97fa640 1 -- 172.21.15.31:0/1892602720 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7298004b90 con 0x7f72ac0d2080 2023-11-26T07:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.393+0000 7f72a97fa640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7290066470 0x7f7290068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.393+0000 7f72a97fa640 1 -- 172.21.15.31:0/1892602720 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7298083530 con 0x7f72ac0d2080 2023-11-26T07:17:29.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.393+0000 7f72b0aab640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7290066470 0x7f7290068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.397+0000 7f72b0aab640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7290066470 0x7f7290068930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72a00ad000 tx=0x7f72a00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.397+0000 7f72a97fa640 1 -- 172.21.15.31:0/1892602720 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7298050cf0 con 0x7f72ac0d2080 2023-11-26T07:17:29.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 -- 172.21.15.31:0/1892602720 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72ac08ce20 con 0x7f72ac0d2080 2023-11-26T07:17:29.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72a97fa640 1 -- 172.21.15.31:0/1892602720 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12766 (secure 0 0 0) 0x7f7298054310 con 0x7f72ac0d2080 2023-11-26T07:17:29.561 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:29.561 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:22.986827+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 -- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7290066470 msgr2=0x7f7290068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7290066470 0x7f7290068930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72a00ad000 tx=0x7f72a00a6000 comp rx=0 tx=0).stop 2023-11-26T07:17:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 -- 172.21.15.31:0/1892602720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ac0d2080 msgr2=0x7f72ac0c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ac0d2080 0x7f72ac0c95f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f729800c430 tx=0x7f729800c8f0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 -- 172.21.15.31:0/1892602720 shutdown_connections 2023-11-26T07:17:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7290066470 0x7f7290068930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72ac15d7b0 0x7f72ac154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72ac0d2080 0x7f72ac0c95f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 --2- 172.21.15.31:0/1892602720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ac0cead0 0x7f72ac142c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 -- 172.21.15.31:0/1892602720 >> 172.21.15.31:0/1892602720 conn(0x7f72ac0d33b0 msgr2=0x7f72ac0d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 -- 172.21.15.31:0/1892602720 shutdown_connections 2023-11-26T07:17:29.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.557+0000 7f72b2d0f640 1 -- 172.21.15.31:0/1892602720 wait complete. 2023-11-26T07:17:29.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:29.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc74bce3640 1 Processor -- start 2023-11-26T07:17:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc74bce3640 1 -- start start 2023-11-26T07:17:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc74bce3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7440b3c60 0x7fc7440b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc74bce3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc744162190 0x7fc7441625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc74bce3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7440b2590 con 0x7fc7440b4040 2023-11-26T07:17:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc74bce3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7440b2870 con 0x7fc7440b3c60 2023-11-26T07:17:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc74bce3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440b2b50 con 0x7fc744162190 2023-11-26T07:17:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc749a7f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7440b3c60 0x7fc7440b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc749a7f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc7440b3c60 0x7fc7440b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52254/0 (socket says 172.21.15.31:52254) 2023-11-26T07:17:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc749a7f640 1 -- 172.21.15.31:0/340793225 learned_addr learned my addr 172.21.15.31:0/340793225 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc74a280640 1 --2- 172.21.15.31:0/340793225 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc744162190 0x7fc7441625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc749a7f640 1 -- 172.21.15.31:0/340793225 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc744162190 msgr2=0x7fc7441625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc749a7f640 1 --2- 172.21.15.31:0/340793225 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc744162190 0x7fc7441625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc749a7f640 1 -- 172.21.15.31:0/340793225 >> v1:172.21.15.97:6789/0 conn(0x7fc7440b4040 legacy=0x7fc7440ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.725+0000 7fc749a7f640 1 -- 172.21.15.31:0/340793225 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440b2e30 con 0x7fc7440b3c60 2023-11-26T07:17:29.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc749a7f640 1 --2- 172.21.15.31:0/340793225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7440b3c60 0x7fc7440b1fb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc740009870 tx=0x7fc74001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecc5591b6de49b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc748a7d640 1 -- 172.21.15.31:0/340793225 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7400a95f0 con 0x7fc7440b3c60 2023-11-26T07:17:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc748a7d640 1 -- 172.21.15.31:0/340793225 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7400a9750 con 0x7fc7440b3c60 2023-11-26T07:17:29.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc748a7d640 1 -- 172.21.15.31:0/340793225 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7400a9ae0 con 0x7fc7440b3c60 2023-11-26T07:17:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc74bce3640 1 -- 172.21.15.31:0/340793225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7440b3c60 msgr2=0x7fc7440b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc74bce3640 1 --2- 172.21.15.31:0/340793225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7440b3c60 0x7fc7440b1fb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc740009870 tx=0x7fc74001c510 comp rx=0 tx=0).stop 2023-11-26T07:17:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc74bce3640 1 -- 172.21.15.31:0/340793225 shutdown_connections 2023-11-26T07:17:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc74bce3640 1 --2- 172.21.15.31:0/340793225 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc744162190 0x7fc7441625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc74bce3640 1 --2- 172.21.15.31:0/340793225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7440b3c60 0x7fc7440b1fb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc74bce3640 1 -- 172.21.15.31:0/340793225 >> 172.21.15.31:0/340793225 conn(0x7fc7440b0cb0 msgr2=0x7fc7440b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc74bce3640 1 -- 172.21.15.31:0/340793225 shutdown_connections 2023-11-26T07:17:29.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.729+0000 7fc74bce3640 1 -- 172.21.15.31:0/340793225 wait complete. 2023-11-26T07:17:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 Processor -- start 2023-11-26T07:17:29.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 -- start start 2023-11-26T07:17:29.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7440b3c60 0x7fc744162fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc744162190 0x7fc74415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc744158e10 0x7fc7441592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7441634f0 con 0x7fc744162190 2023-11-26T07:17:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc744099e50 con 0x7fc7440b3c60 2023-11-26T07:17:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74409a130 con 0x7fc744158e10 2023-11-26T07:17:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74927e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc744162190 0x7fc74415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc749a7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7440b3c60 0x7fc744162fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74927e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc744162190 0x7fc74415ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52262/0 (socket says 172.21.15.31:52262) 2023-11-26T07:17:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74a280640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc744158e10 0x7fc7441592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74927e640 1 -- 172.21.15.31:0/47150648 learned_addr learned my addr 172.21.15.31:0/47150648 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:29.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74927e640 1 -- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc744158e10 msgr2=0x7fc7441592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74927e640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc744158e10 0x7fc7441592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74927e640 1 -- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7440b3c60 msgr2=0x7fc744162fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74927e640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7440b3c60 0x7fc744162fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74927e640 1 -- 172.21.15.31:0/47150648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7400094d0 con 0x7fc744162190 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc749a7f640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7440b3c60 0x7fc744162fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74927e640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc744162190 0x7fc74415ad40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c500 tx=0x7fc73400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc73a7fc640 1 -- 172.21.15.31:0/47150648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734005220 con 0x7fc744162190 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 -- 172.21.15.31:0/47150648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74409a410 con 0x7fc744162190 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc73a7fc640 1 -- 172.21.15.31:0/47150648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc734005380 con 0x7fc744162190 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 -- 172.21.15.31:0/47150648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74409a9d0 con 0x7fc744162190 2023-11-26T07:17:29.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc73a7fc640 1 -- 172.21.15.31:0/47150648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734007270 con 0x7fc744162190 2023-11-26T07:17:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.733+0000 7fc74bce3640 1 -- 172.21.15.31:0/47150648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7440671f0 con 0x7fc744162190 2023-11-26T07:17:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.737+0000 7fc73a7fc640 1 -- 172.21.15.31:0/47150648 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc7340073d0 con 0x7fc744162190 2023-11-26T07:17:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.737+0000 7fc73a7fc640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc7280664a0 0x7fc728068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:29.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.737+0000 7fc73a7fc640 1 -- 172.21.15.31:0/47150648 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc7340833b0 con 0x7fc744162190 2023-11-26T07:17:29.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.737+0000 7fc749a7f640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc7280664a0 0x7fc728068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:29.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.737+0000 7fc749a7f640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc7280664a0 0x7fc728068960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7400ae000 tx=0x7fc7400b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:29.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.741+0000 7fc73a7fc640 1 -- 172.21.15.31:0/47150648 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc734050ba0 con 0x7fc744162190 2023-11-26T07:17:29.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 -- 172.21.15.31:0/47150648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc74406b460 con 0x7fc744162190 2023-11-26T07:17:29.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc73a7fc640 1 -- 172.21.15.31:0/47150648 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7340541c0 con 0x7fc744162190 2023-11-26T07:17:29.905 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:17:29.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 -- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc7280664a0 msgr2=0x7fc728068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc7280664a0 0x7fc728068960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7400ae000 tx=0x7fc7400b3040 comp rx=0 tx=0).stop 2023-11-26T07:17:29.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 -- 172.21.15.31:0/47150648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc744162190 msgr2=0x7fc74415ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:29.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc744162190 0x7fc74415ad40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c500 tx=0x7fc73400c9c0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 -- 172.21.15.31:0/47150648 shutdown_connections 2023-11-26T07:17:29.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc7280664a0 0x7fc728068960 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc744158e10 0x7fc7441592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc744162190 0x7fc74415ad40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 --2- 172.21.15.31:0/47150648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7440b3c60 0x7fc744162fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:29.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 -- 172.21.15.31:0/47150648 >> 172.21.15.31:0/47150648 conn(0x7fc7440b0cb0 msgr2=0x7fc7440b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:29.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 -- 172.21.15.31:0/47150648 shutdown_connections 2023-11-26T07:17:29.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:29.901+0000 7fc74bce3640 1 -- 172.21.15.31:0/47150648 wait complete. 2023-11-26T07:17:29.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:17:29.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:17:29.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:17:30.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a37f5640 1 Processor -- start 2023-11-26T07:17:30.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a37f5640 1 -- start start 2023-11-26T07:17:30.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a37f5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f119c15d7b0 0x7f119c0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a37f5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f119c0cead0 0x7f119c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a37f5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119c0d22c0 con 0x7f119c0d1eb0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a37f5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119c0cefd0 con 0x7f119c0cead0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a37f5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f119c0cf2b0 con 0x7f119c15d7b0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a0d90640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f119c0cead0 0x7f119c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a1d92640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f119c0d1eb0 0x7f119c0cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44080/0 (socket says 172.21.15.31:44080) 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a1d92640 1 -- 172.21.15.31:0/2704090239 learned_addr learned my addr 172.21.15.31:0/2704090239 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a1591640 1 --2- 172.21.15.31:0/2704090239 >> v2:172.21.15.97:3300/0 conn(0x7f119c15d7b0 0x7f119c0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a0d90640 1 -- 172.21.15.31:0/2704090239 >> v2:172.21.15.97:3300/0 conn(0x7f119c15d7b0 msgr2=0x7f119c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a0d90640 1 --2- 172.21.15.31:0/2704090239 >> v2:172.21.15.97:3300/0 conn(0x7f119c15d7b0 0x7f119c0d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a0d90640 1 -- 172.21.15.31:0/2704090239 >> v1:172.21.15.31:6789/0 conn(0x7f119c0d1eb0 legacy=0x7f119c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f11a0d90640 1 -- 172.21.15.31:0/2704090239 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c06b3f0 con 0x7f119c0cead0 2023-11-26T07:17:30.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.065+0000 7f1193fff640 1 -- 172.21.15.31:0/2704090239 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075952805 0 0) 0x7f119c0d22c0 con 0x7f119c0d1eb0 2023-11-26T07:17:30.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f11a0d90640 1 --2- 172.21.15.31:0/2704090239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c0cead0 0x7f119c0d1970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f119c0d3310 tx=0x7f118c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752c985130d77c7f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f1193fff640 1 -- 172.21.15.31:0/2704090239 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c0ac640 con 0x7f119c0cead0 2023-11-26T07:17:30.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f1193fff640 1 -- 172.21.15.31:0/2704090239 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c0ac7a0 con 0x7f119c0cead0 2023-11-26T07:17:30.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f1193fff640 1 -- 172.21.15.31:0/2704090239 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c0acb30 con 0x7f119c0cead0 2023-11-26T07:17:30.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f11a37f5640 1 -- 172.21.15.31:0/2704090239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c0cead0 msgr2=0x7f119c0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f11a37f5640 1 --2- 172.21.15.31:0/2704090239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c0cead0 0x7f119c0d1970 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f119c0d3310 tx=0x7f118c01c310 comp rx=0 tx=0).stop 2023-11-26T07:17:30.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f11a37f5640 1 -- 172.21.15.31:0/2704090239 shutdown_connections 2023-11-26T07:17:30.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f11a37f5640 1 --2- 172.21.15.31:0/2704090239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c0cead0 0x7f119c0d1970 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f11a37f5640 1 --2- 172.21.15.31:0/2704090239 >> v2:172.21.15.97:3300/0 conn(0x7f119c15d7b0 0x7f119c0d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f11a37f5640 1 -- 172.21.15.31:0/2704090239 >> 172.21.15.31:0/2704090239 conn(0x7f119c0d33b0 msgr2=0x7f119c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f11a37f5640 1 -- 172.21.15.31:0/2704090239 shutdown_connections 2023-11-26T07:17:30.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.069+0000 7f11a37f5640 1 -- 172.21.15.31:0/2704090239 wait complete. 2023-11-26T07:17:30.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a37f5640 1 Processor -- start 2023-11-26T07:17:30.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a37f5640 1 -- start start 2023-11-26T07:17:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a37f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f119c0cead0 0x7f119c152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a37f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f119c0d1eb0 0x7f119c0b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a37f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c15d7b0 0x7f119c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a37f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0c6ee0 con 0x7f119c15d7b0 2023-11-26T07:17:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a37f5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0c71c0 con 0x7f119c0d1eb0 2023-11-26T07:17:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a37f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0c74a0 con 0x7f119c0cead0 2023-11-26T07:17:30.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a1d92640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c15d7b0 0x7f119c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a1d92640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c15d7b0 0x7f119c15b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52282/0 (socket says 172.21.15.31:52282) 2023-11-26T07:17:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a1d92640 1 -- 172.21.15.31:0/2613005577 learned_addr learned my addr 172.21.15.31:0/2613005577 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a0d90640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f119c0d1eb0 0x7f119c0b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a1591640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f119c0cead0 0x7f119c152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a1d92640 1 -- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f119c0cead0 msgr2=0x7f119c152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a1d92640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f119c0cead0 0x7f119c152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a1d92640 1 -- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f119c0d1eb0 msgr2=0x7f119c0b87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a1d92640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f119c0d1eb0 0x7f119c0b87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a1d92640 1 -- 172.21.15.31:0/2613005577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118c009420 con 0x7f119c15d7b0 2023-11-26T07:17:30.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a1d92640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c15d7b0 0x7f119c15b950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f119400c780 tx=0x7f119400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f1191ffb640 1 -- 172.21.15.31:0/2613005577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194004d70 con 0x7f119c15d7b0 2023-11-26T07:17:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f1191ffb640 1 -- 172.21.15.31:0/2613005577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119400a640 con 0x7f119c15d7b0 2023-11-26T07:17:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f1191ffb640 1 -- 172.21.15.31:0/2613005577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119400aa70 con 0x7f119c15d7b0 2023-11-26T07:17:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a37f5640 1 -- 172.21.15.31:0/2613005577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119c0cb760 con 0x7f119c15d7b0 2023-11-26T07:17:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.073+0000 7f11a37f5640 1 -- 172.21.15.31:0/2613005577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c0cbd20 con 0x7f119c15d7b0 2023-11-26T07:17:30.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.077+0000 7f11a37f5640 1 -- 172.21.15.31:0/2613005577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c04e850 con 0x7f119c15d7b0 2023-11-26T07:17:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.077+0000 7f1191ffb640 1 -- 172.21.15.31:0/2613005577 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f119400abd0 con 0x7f119c15d7b0 2023-11-26T07:17:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.077+0000 7f1191ffb640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1180066470 0x7f1180068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.077+0000 7f1191ffb640 1 -- 172.21.15.31:0/2613005577 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1194083020 con 0x7f119c15d7b0 2023-11-26T07:17:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.077+0000 7f11a1591640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1180066470 0x7f1180068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.077+0000 7f11a1591640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1180066470 0x7f1180068930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11980079c0 tx=0x7f1198008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.081+0000 7f1191ffb640 1 -- 172.21.15.31:0/2613005577 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1194050800 con 0x7f119c15d7b0 2023-11-26T07:17:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.241+0000 7f11a37f5640 1 -- 172.21.15.31:0/2613005577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f119c0d11c0 con 0x7f119c15d7b0 2023-11-26T07:17:30.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f1191ffb640 1 -- 172.21.15.31:0/2613005577 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1194018070 con 0x7f119c15d7b0 2023-11-26T07:17:30.248 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:30.248 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:17:30.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:17:30.248 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-11-26T07:17:30.248 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-11-26T07:17:30.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:17:30.248 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:17:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 -- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1180066470 msgr2=0x7f1180068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1180066470 0x7f1180068930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f11980079c0 tx=0x7f1198008040 comp rx=0 tx=0).stop 2023-11-26T07:17:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 -- 172.21.15.31:0/2613005577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c15d7b0 msgr2=0x7f119c15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c15d7b0 0x7f119c15b950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f119400c780 tx=0x7f119400cc40 comp rx=0 tx=0).stop 2023-11-26T07:17:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 -- 172.21.15.31:0/2613005577 shutdown_connections 2023-11-26T07:17:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1180066470 0x7f1180068930 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f119c15d7b0 0x7f119c15b950 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f119c0d1eb0 0x7f119c0b87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 --2- 172.21.15.31:0/2613005577 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f119c0cead0 0x7f119c152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 -- 172.21.15.31:0/2613005577 >> 172.21.15.31:0/2613005577 conn(0x7f119c0d33b0 msgr2=0x7f119c0d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 -- 172.21.15.31:0/2613005577 shutdown_connections 2023-11-26T07:17:30.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.245+0000 7f11a37f5640 1 -- 172.21.15.31:0/2613005577 wait complete. 2023-11-26T07:17:30.268 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:17:30.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:17:30.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.409+0000 7f3cb32aa640 1 Processor -- start 2023-11-26T07:17:30.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.409+0000 7f3cb32aa640 1 -- start start 2023-11-26T07:17:30.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb32aa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3cac0b4040 0x7f3cac0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb32aa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3cac162190 0x7f3cac1625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb32aa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cac0b2590 con 0x7f3cac0b3c60 2023-11-26T07:17:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb32aa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac0b2870 con 0x7f3cac0b4040 2023-11-26T07:17:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb32aa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac0b2b50 con 0x7f3cac162190 2023-11-26T07:17:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb0845640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3cac0b4040 0x7f3cac0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb0845640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3cac0b4040 0x7f3cac0b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52286/0 (socket says 172.21.15.31:52286) 2023-11-26T07:17:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb0845640 1 -- 172.21.15.31:0/1905909723 learned_addr learned my addr 172.21.15.31:0/1905909723 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb1847640 1 --2- 172.21.15.31:0/1905909723 >> v2:172.21.15.97:3300/0 conn(0x7f3cac162190 0x7f3cac1625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb0845640 1 -- 172.21.15.31:0/1905909723 >> v2:172.21.15.97:3300/0 conn(0x7f3cac162190 msgr2=0x7f3cac1625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:30.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb0845640 1 --2- 172.21.15.31:0/1905909723 >> v2:172.21.15.97:3300/0 conn(0x7f3cac162190 0x7f3cac1625a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb0845640 1 -- 172.21.15.31:0/1905909723 >> v1:172.21.15.97:6789/0 conn(0x7f3cac0b3c60 legacy=0x7f3cac0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb0845640 1 -- 172.21.15.31:0/1905909723 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0b2e30 con 0x7f3cac0b4040 2023-11-26T07:17:30.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.413+0000 7f3cb1847640 1 --2- 172.21.15.31:0/1905909723 >> v2:172.21.15.97:3300/0 conn(0x7f3cac162190 0x7f3cac1625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:30.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb0845640 1 --2- 172.21.15.31:0/1905909723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac0b4040 0x7f3cac0b1fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0b1cd0 tx=0x7f3c9c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44992324317e6d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3ca3fff640 1 -- 172.21.15.31:0/1905909723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0aa610 con 0x7f3cac0b4040 2023-11-26T07:17:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3ca3fff640 1 -- 172.21.15.31:0/1905909723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c0a6040 con 0x7f3cac0b4040 2023-11-26T07:17:30.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3ca3fff640 1 -- 172.21.15.31:0/1905909723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0aa990 con 0x7f3cac0b4040 2023-11-26T07:17:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb32aa640 1 -- 172.21.15.31:0/1905909723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac0b4040 msgr2=0x7f3cac0b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb32aa640 1 --2- 172.21.15.31:0/1905909723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac0b4040 0x7f3cac0b1fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0b1cd0 tx=0x7f3c9c01c490 comp rx=0 tx=0).stop 2023-11-26T07:17:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb32aa640 1 -- 172.21.15.31:0/1905909723 shutdown_connections 2023-11-26T07:17:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb32aa640 1 --2- 172.21.15.31:0/1905909723 >> v2:172.21.15.97:3300/0 conn(0x7f3cac162190 0x7f3cac1625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb32aa640 1 --2- 172.21.15.31:0/1905909723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac0b4040 0x7f3cac0b1fb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb32aa640 1 -- 172.21.15.31:0/1905909723 >> 172.21.15.31:0/1905909723 conn(0x7f3cac0b0cb0 msgr2=0x7f3cac0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb32aa640 1 -- 172.21.15.31:0/1905909723 shutdown_connections 2023-11-26T07:17:30.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb32aa640 1 -- 172.21.15.31:0/1905909723 wait complete. 2023-11-26T07:17:30.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb32aa640 1 Processor -- start 2023-11-26T07:17:30.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.417+0000 7f3cb32aa640 1 -- start start 2023-11-26T07:17:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb32aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cac0b3c60 0x7f3cac1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb32aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3cac162190 0x7f3cac1635f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb32aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac163b30 0x7f3cac15f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb32aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac15eea0 con 0x7f3cac163b30 2023-11-26T07:17:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb32aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac163fc0 con 0x7f3cac0b3c60 2023-11-26T07:17:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb32aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac15ade0 con 0x7f3cac162190 2023-11-26T07:17:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1847640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac163b30 0x7f3cac15f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1847640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac163b30 0x7f3cac15f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52290/0 (socket says 172.21.15.31:52290) 2023-11-26T07:17:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1847640 1 -- 172.21.15.31:0/781476457 learned_addr learned my addr 172.21.15.31:0/781476457 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:30.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb0845640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3cac162190 0x7f3cac1635f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1046640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cac0b3c60 0x7f3cac1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1847640 1 -- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3cac162190 msgr2=0x7f3cac1635f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1847640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3cac162190 0x7f3cac1635f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1847640 1 -- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cac0b3c60 msgr2=0x7f3cac1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1847640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cac0b3c60 0x7f3cac1553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1847640 1 -- 172.21.15.31:0/781476457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0094d0 con 0x7f3cac163b30 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1046640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cac0b3c60 0x7f3cac1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb1847640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac163b30 0x7f3cac15f2c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400cd50 tx=0x7f3ca400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3ca1ffb640 1 -- 172.21.15.31:0/781476457 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400f680 con 0x7f3cac163b30 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3ca1ffb640 1 -- 172.21.15.31:0/781476457 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca400f800 con 0x7f3cac163b30 2023-11-26T07:17:30.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb32aa640 1 -- 172.21.15.31:0/781476457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac15b0c0 con 0x7f3cac163b30 2023-11-26T07:17:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3ca1ffb640 1 -- 172.21.15.31:0/781476457 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400fbb0 con 0x7f3cac163b30 2023-11-26T07:17:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb32aa640 1 -- 172.21.15.31:0/781476457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac1558e0 con 0x7f3cac163b30 2023-11-26T07:17:30.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3cb32aa640 1 -- 172.21.15.31:0/781476457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac0671f0 con 0x7f3cac163b30 2023-11-26T07:17:30.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.421+0000 7f3ca1ffb640 1 -- 172.21.15.31:0/781476457 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3ca400b040 con 0x7f3cac163b30 2023-11-26T07:17:30.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.425+0000 7f3ca1ffb640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c90066470 0x7f3c90068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:30.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.425+0000 7f3ca1ffb640 1 -- 172.21.15.31:0/781476457 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3ca40844b0 con 0x7f3cac163b30 2023-11-26T07:17:30.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.425+0000 7f3cb1046640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c90066470 0x7f3c90068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:30.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.425+0000 7f3cb1046640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c90066470 0x7f3c90068930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8003ec0 tx=0x7f3ca8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:30.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.425+0000 7f3ca1ffb640 1 -- 172.21.15.31:0/781476457 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3ca4051c70 con 0x7f3cac163b30 2023-11-26T07:17:30.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:30.589+0000 7f3cb32aa640 1 -- 172.21.15.31:0/781476457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3cac15efe0 con 0x7f3cac163b30 2023-11-26T07:17:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.093+0000 7f3ca1ffb640 1 -- 172.21.15.31:0/781476457 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3cac15efe0 con 0x7f3cac163b30 2023-11-26T07:17:31.097 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-11-26T07:17:31.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.093+0000 7f3cb32aa640 1 -- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c90066470 msgr2=0x7f3c90068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.093+0000 7f3cb32aa640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c90066470 0x7f3c90068930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8003ec0 tx=0x7f3ca8009040 comp rx=0 tx=0).stop 2023-11-26T07:17:31.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.093+0000 7f3cb32aa640 1 -- 172.21.15.31:0/781476457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac163b30 msgr2=0x7f3cac15f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.093+0000 7f3cb32aa640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac163b30 0x7f3cac15f2c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400cd50 tx=0x7f3ca400a650 comp rx=0 tx=0).stop 2023-11-26T07:17:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.097+0000 7f3cb32aa640 1 -- 172.21.15.31:0/781476457 shutdown_connections 2023-11-26T07:17:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.097+0000 7f3cb32aa640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c90066470 0x7f3c90068930 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.097+0000 7f3cb32aa640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cac163b30 0x7f3cac15f2c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.097+0000 7f3cb32aa640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3cac162190 0x7f3cac1635f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.097+0000 7f3cb32aa640 1 --2- 172.21.15.31:0/781476457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cac0b3c60 0x7f3cac1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.097+0000 7f3cb32aa640 1 -- 172.21.15.31:0/781476457 >> 172.21.15.31:0/781476457 conn(0x7f3cac0b0cb0 msgr2=0x7f3cac0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.097+0000 7f3cb32aa640 1 -- 172.21.15.31:0/781476457 shutdown_connections 2023-11-26T07:17:31.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.097+0000 7f3cb32aa640 1 -- 172.21.15.31:0/781476457 wait complete. 2023-11-26T07:17:31.127 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:17:31.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.273+0000 7f89f8eba640 1 Processor -- start 2023-11-26T07:17:31.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f8eba640 1 -- start start 2023-11-26T07:17:31.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f8eba640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f89f415d2a0 0x7f89f40d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f8eba640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f40b6250 con 0x7f89f40b5350 2023-11-26T07:17:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f8eba640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f4161cf0 con 0x7f89f40d2c20 2023-11-26T07:17:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f8eba640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f40ce8c0 con 0x7f89f415d2a0 2023-11-26T07:17:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f37fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f89f415d2a0 0x7f89f40d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f37fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f89f415d2a0 0x7f89f40d26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52306/0 (socket says 172.21.15.31:52306) 2023-11-26T07:17:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f37fe640 1 -- 172.21.15.31:0/3821035001 learned_addr learned my addr 172.21.15.31:0/3821035001 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f3fff640 1 --1- 172.21.15.31:0/3821035001 >> v1:172.21.15.31:6789/0 conn(0x7f89f40b5350 0x7f89f40d10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44094/0 (socket says 172.21.15.31:44094) 2023-11-26T07:17:31.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f37fe640 1 -- 172.21.15.31:0/3821035001 >> v1:172.21.15.97:6789/0 conn(0x7f89f40d2c20 legacy=0x7f89f40b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f37fe640 1 -- 172.21.15.31:0/3821035001 >> v1:172.21.15.31:6789/0 conn(0x7f89f40b5350 legacy=0x7f89f40d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.277+0000 7f89f37fe640 1 -- 172.21.15.31:0/3821035001 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f4161170 con 0x7f89f415d2a0 2023-11-26T07:17:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f37fe640 1 --2- 172.21.15.31:0/3821035001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f415d2a0 0x7f89f40d26e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009870 tx=0x7f89e401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5620ce3b2f056ea5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f27fc640 1 -- 172.21.15.31:0/3821035001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40aa670 con 0x7f89f415d2a0 2023-11-26T07:17:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f27fc640 1 -- 172.21.15.31:0/3821035001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e40aa7d0 con 0x7f89f415d2a0 2023-11-26T07:17:31.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f27fc640 1 -- 172.21.15.31:0/3821035001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40aab60 con 0x7f89f415d2a0 2023-11-26T07:17:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f8eba640 1 -- 172.21.15.31:0/3821035001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f415d2a0 msgr2=0x7f89f40d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f8eba640 1 --2- 172.21.15.31:0/3821035001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f415d2a0 0x7f89f40d26e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009870 tx=0x7f89e401c510 comp rx=0 tx=0).stop 2023-11-26T07:17:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f8eba640 1 -- 172.21.15.31:0/3821035001 shutdown_connections 2023-11-26T07:17:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f8eba640 1 --2- 172.21.15.31:0/3821035001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f415d2a0 0x7f89f40d26e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f8eba640 1 -- 172.21.15.31:0/3821035001 >> 172.21.15.31:0/3821035001 conn(0x7f89f40668a0 msgr2=0x7f89f40cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f8eba640 1 -- 172.21.15.31:0/3821035001 shutdown_connections 2023-11-26T07:17:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f8eba640 1 -- 172.21.15.31:0/3821035001 wait complete. 2023-11-26T07:17:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f8eba640 1 Processor -- start 2023-11-26T07:17:31.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.281+0000 7f89f8eba640 1 -- start start 2023-11-26T07:17:31.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f8eba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f40b5350 0x7f89f40b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f8eba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f40d2c20 0x7f89f41598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f8eba640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89f415d2a0 0x7f89f4159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f8eba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40cc750 con 0x7f89f40d2c20 2023-11-26T07:17:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f8eba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40cca30 con 0x7f89f40b5350 2023-11-26T07:17:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f8eba640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40ccd10 con 0x7f89f415d2a0 2023-11-26T07:17:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f40d2c20 0x7f89f41598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f40d2c20 0x7f89f41598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52320/0 (socket says 172.21.15.31:52320) 2023-11-26T07:17:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f2ffd640 1 -- 172.21.15.31:0/1583652306 learned_addr learned my addr 172.21.15.31:0/1583652306 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f3fff640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89f415d2a0 0x7f89f4159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f2ffd640 1 -- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89f415d2a0 msgr2=0x7f89f4159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f37fe640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f40b5350 0x7f89f40b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f2ffd640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89f415d2a0 0x7f89f4159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f2ffd640 1 -- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f40b5350 msgr2=0x7f89f40b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:31.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f2ffd640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f40b5350 0x7f89f40b7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:31.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f2ffd640 1 -- 172.21.15.31:0/1583652306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e40094d0 con 0x7f89f40d2c20 2023-11-26T07:17:31.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f37fe640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f40b5350 0x7f89f40b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:31.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f3fff640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89f415d2a0 0x7f89f4159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:31.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f2ffd640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f40d2c20 0x7f89f41598d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89e800c4d0 tx=0x7f89e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:31.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89dbfff640 1 -- 172.21.15.31:0/1583652306 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80052d0 con 0x7f89f40d2c20 2023-11-26T07:17:31.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89dbfff640 1 -- 172.21.15.31:0/1583652306 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8005430 con 0x7f89f40d2c20 2023-11-26T07:17:31.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89dbfff640 1 -- 172.21.15.31:0/1583652306 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8012830 con 0x7f89f40d2c20 2023-11-26T07:17:31.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f8eba640 1 -- 172.21.15.31:0/1583652306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f40ccff0 con 0x7f89f40d2c20 2023-11-26T07:17:31.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f8eba640 1 -- 172.21.15.31:0/1583652306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f40b8640 con 0x7f89f40d2c20 2023-11-26T07:17:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.285+0000 7f89f8eba640 1 -- 172.21.15.31:0/1583652306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f4074990 con 0x7f89f40d2c20 2023-11-26T07:17:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.289+0000 7f89dbfff640 1 -- 172.21.15.31:0/1583652306 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f89e8012990 con 0x7f89f40d2c20 2023-11-26T07:17:31.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.289+0000 7f89dbfff640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89e0066470 0x7f89e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:31.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.289+0000 7f89dbfff640 1 -- 172.21.15.31:0/1583652306 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f89e80837c0 con 0x7f89f40d2c20 2023-11-26T07:17:31.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.289+0000 7f89f37fe640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89e0066470 0x7f89e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:31.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.289+0000 7f89f37fe640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89e0066470 0x7f89e0068930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89e40ae000 tx=0x7f89e4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:31.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.293+0000 7f89dbfff640 1 -- 172.21.15.31:0/1583652306 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f89e8050e10 con 0x7f89f40d2c20 2023-11-26T07:17:31.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:31.449+0000 7f89f8eba640 1 -- 172.21.15.31:0/1583652306 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f89f40b4360 con 0x7f89f40d2c20 2023-11-26T07:17:32.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89dbfff640 1 -- 172.21.15.31:0/1583652306 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f89e8054430 con 0x7f89f40d2c20 2023-11-26T07:17:32.104 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-11-26T07:17:32.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 -- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89e0066470 msgr2=0x7f89e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89e0066470 0x7f89e0068930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89e40ae000 tx=0x7f89e4008040 comp rx=0 tx=0).stop 2023-11-26T07:17:32.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 -- 172.21.15.31:0/1583652306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f40d2c20 msgr2=0x7f89f41598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f40d2c20 0x7f89f41598d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89e800c4d0 tx=0x7f89e800c990 comp rx=0 tx=0).stop 2023-11-26T07:17:32.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 -- 172.21.15.31:0/1583652306 shutdown_connections 2023-11-26T07:17:32.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89e0066470 0x7f89e0068930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f89f415d2a0 0x7f89f4159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89f40d2c20 0x7f89f41598d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 --2- 172.21.15.31:0/1583652306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89f40b5350 0x7f89f40b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 -- 172.21.15.31:0/1583652306 >> 172.21.15.31:0/1583652306 conn(0x7f89f40668a0 msgr2=0x7f89f40b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:32.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 -- 172.21.15.31:0/1583652306 shutdown_connections 2023-11-26T07:17:32.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.101+0000 7f89f8eba640 1 -- 172.21.15.31:0/1583652306 wait complete. 2023-11-26T07:17:32.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:17:32.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cc06e3640 1 Processor -- start 2023-11-26T07:17:32.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cc06e3640 1 -- start start 2023-11-26T07:17:32.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cc06e3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6cb8154600 0x7f6cb8161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:32.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cc06e3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6cb806c640 0x7f6cb806cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:32.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cc06e3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb806d520 con 0x7f6cb815dae0 2023-11-26T07:17:32.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cc06e3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb806cff0 con 0x7f6cb8154600 2023-11-26T07:17:32.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cc06e3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb8162070 con 0x7f6cb806c640 2023-11-26T07:17:32.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cbe47f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6cb8154600 0x7f6cb8161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:32.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cbe47f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6cb8154600 0x7f6cb8161b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52334/0 (socket says 172.21.15.31:52334) 2023-11-26T07:17:32.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cbe47f640 1 -- 172.21.15.31:0/1163868286 learned_addr learned my addr 172.21.15.31:0/1163868286 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:32.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cbec80640 1 --2- 172.21.15.31:0/1163868286 >> v2:172.21.15.97:3300/0 conn(0x7f6cb806c640 0x7f6cb806cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:32.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cbe47f640 1 -- 172.21.15.31:0/1163868286 >> v2:172.21.15.97:3300/0 conn(0x7f6cb806c640 msgr2=0x7f6cb806cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:32.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cbe47f640 1 --2- 172.21.15.31:0/1163868286 >> v2:172.21.15.97:3300/0 conn(0x7f6cb806c640 0x7f6cb806cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cbe47f640 1 -- 172.21.15.31:0/1163868286 >> v1:172.21.15.97:6789/0 conn(0x7f6cb815dae0 legacy=0x7f6cb80ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.297+0000 7f6cbe47f640 1 -- 172.21.15.31:0/1163868286 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb815e060 con 0x7f6cb8154600 2023-11-26T07:17:32.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cbe47f640 1 --2- 172.21.15.31:0/1163868286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 0x7f6cb8161b30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80ceff0 tx=0x7f6cb401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6334869f8e14369c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:32.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cbd47d640 1 -- 172.21.15.31:0/1163868286 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40aa430 con 0x7f6cb8154600 2023-11-26T07:17:32.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cbd47d640 1 -- 172.21.15.31:0/1163868286 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb40aa590 con 0x7f6cb8154600 2023-11-26T07:17:32.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cbd47d640 1 -- 172.21.15.31:0/1163868286 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40aa920 con 0x7f6cb8154600 2023-11-26T07:17:32.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1163868286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 msgr2=0x7f6cb8161b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cc06e3640 1 --2- 172.21.15.31:0/1163868286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 0x7f6cb8161b30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80ceff0 tx=0x7f6cb401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1163868286 shutdown_connections 2023-11-26T07:17:32.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cc06e3640 1 --2- 172.21.15.31:0/1163868286 >> v2:172.21.15.97:3300/0 conn(0x7f6cb806c640 0x7f6cb806cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cc06e3640 1 --2- 172.21.15.31:0/1163868286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 0x7f6cb8161b30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1163868286 >> 172.21.15.31:0/1163868286 conn(0x7f6cb80b0db0 msgr2=0x7f6cb80b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:32.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1163868286 shutdown_connections 2023-11-26T07:17:32.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.301+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1163868286 wait complete. 2023-11-26T07:17:32.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cc06e3640 1 Processor -- start 2023-11-26T07:17:32.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cc06e3640 1 -- start start 2023-11-26T07:17:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cc06e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cb806c640 0x7f6cb814ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cc06e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 0x7f6cb814a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cc06e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cb815dae0 0x7f6cb80b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cc06e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb8153250 con 0x7f6cb8154600 2023-11-26T07:17:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cc06e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb81526d0 con 0x7f6cb806c640 2023-11-26T07:17:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cc06e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb81529b0 con 0x7f6cb815dae0 2023-11-26T07:17:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbdc7e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 0x7f6cb814a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbdc7e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 0x7f6cb814a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52338/0 (socket says 172.21.15.31:52338) 2023-11-26T07:17:32.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbdc7e640 1 -- 172.21.15.31:0/1507488752 learned_addr learned my addr 172.21.15.31:0/1507488752 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbec80640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cb815dae0 0x7f6cb80b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbe47f640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cb806c640 0x7f6cb814ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbdc7e640 1 -- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cb815dae0 msgr2=0x7f6cb80b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbdc7e640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cb815dae0 0x7f6cb80b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbdc7e640 1 -- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cb806c640 msgr2=0x7f6cb814ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbdc7e640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cb806c640 0x7f6cb814ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbdc7e640 1 -- 172.21.15.31:0/1507488752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40094d0 con 0x7f6cb8154600 2023-11-26T07:17:32.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cbdc7e640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 0x7f6cb814a560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800c3c0 tx=0x7f6ca800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:32.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6caeffd640 1 -- 172.21.15.31:0/1507488752 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80052d0 con 0x7f6cb8154600 2023-11-26T07:17:32.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6caeffd640 1 -- 172.21.15.31:0/1507488752 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca8005430 con 0x7f6cb8154600 2023-11-26T07:17:32.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6caeffd640 1 -- 172.21.15.31:0/1507488752 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8003840 con 0x7f6cb8154600 2023-11-26T07:17:32.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1507488752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb8152c90 con 0x7f6cb8154600 2023-11-26T07:17:32.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.305+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1507488752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb814b360 con 0x7f6cb8154600 2023-11-26T07:17:32.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.309+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1507488752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb804e850 con 0x7f6cb8154600 2023-11-26T07:17:32.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.309+0000 7f6caeffd640 1 -- 172.21.15.31:0/1507488752 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6ca80039a0 con 0x7f6cb8154600 2023-11-26T07:17:32.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.309+0000 7f6caeffd640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c9c066470 0x7f6c9c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:32.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.309+0000 7f6caeffd640 1 -- 172.21.15.31:0/1507488752 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6ca8083380 con 0x7f6cb8154600 2023-11-26T07:17:32.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.309+0000 7f6cbe47f640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c9c066470 0x7f6c9c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:32.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.309+0000 7f6cbe47f640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c9c066470 0x7f6c9c068930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40ae000 tx=0x7f6cb40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:32.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.313+0000 7f6caeffd640 1 -- 172.21.15.31:0/1507488752 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ca8050850 con 0x7f6cb8154600 2023-11-26T07:17:32.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:32.505+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1507488752 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6cb80acc80 con 0x7f6cb8154600 2023-11-26T07:17:33.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.101+0000 7f6caeffd640 1 -- 172.21.15.31:0/1507488752 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6ca8053e70 con 0x7f6cb8154600 2023-11-26T07:17:33.107 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 2 and data pool 3 2023-11-26T07:17:33.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c9c066470 msgr2=0x7f6c9c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c9c066470 0x7f6c9c068930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40ae000 tx=0x7f6cb40a7000 comp rx=0 tx=0).stop 2023-11-26T07:17:33.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1507488752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 msgr2=0x7f6cb814a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 0x7f6cb814a560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800c3c0 tx=0x7f6ca800c880 comp rx=0 tx=0).stop 2023-11-26T07:17:33.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1507488752 shutdown_connections 2023-11-26T07:17:33.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c9c066470 0x7f6c9c068930 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6cb815dae0 0x7f6cb80b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cb8154600 0x7f6cb814a560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 --2- 172.21.15.31:0/1507488752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cb806c640 0x7f6cb814ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1507488752 >> 172.21.15.31:0/1507488752 conn(0x7f6cb80b0db0 msgr2=0x7f6cb81538d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:33.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1507488752 shutdown_connections 2023-11-26T07:17:33.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.105+0000 7f6cc06e3640 1 -- 172.21.15.31:0/1507488752 wait complete. 2023-11-26T07:17:33.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:33.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.305+0000 7f6c5405a640 1 Processor -- start 2023-11-26T07:17:33.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.305+0000 7f6c5405a640 1 -- start start 2023-11-26T07:17:33.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c5405a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c4c15d7b0 0x7f6c4c0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c5405a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c4c0d2080 0x7f6c4c06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c5405a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4c0cefd0 con 0x7f6c4c0cead0 2023-11-26T07:17:33.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c5405a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4c0cf2b0 con 0x7f6c4c15d7b0 2023-11-26T07:17:33.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c5405a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4c06b3f0 con 0x7f6c4c0d2080 2023-11-26T07:17:33.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c51df6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c4c15d7b0 0x7f6c4c0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c515f5640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6c4c0cead0 0x7f6c4c0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44106/0 (socket says 172.21.15.31:44106) 2023-11-26T07:17:33.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c515f5640 1 -- 172.21.15.31:0/3078709188 learned_addr learned my addr 172.21.15.31:0/3078709188 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:33.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c525f7640 1 --2- 172.21.15.31:0/3078709188 >> v2:172.21.15.97:3300/0 conn(0x7f6c4c0d2080 0x7f6c4c06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c50df4640 1 -- 172.21.15.31:0/3078709188 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890606425 0 0) 0x7f6c4c0cefd0 con 0x7f6c4c0cead0 2023-11-26T07:17:33.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c50df4640 1 -- 172.21.15.31:0/3078709188 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c38003530 con 0x7f6c4c0cead0 2023-11-26T07:17:33.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c51df6640 1 -- 172.21.15.31:0/3078709188 >> v2:172.21.15.97:3300/0 conn(0x7f6c4c0d2080 msgr2=0x7f6c4c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:33.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c51df6640 1 --2- 172.21.15.31:0/3078709188 >> v2:172.21.15.97:3300/0 conn(0x7f6c4c0d2080 0x7f6c4c06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c51df6640 1 -- 172.21.15.31:0/3078709188 >> v1:172.21.15.31:6789/0 conn(0x7f6c4c0cead0 legacy=0x7f6c4c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.309+0000 7f6c51df6640 1 -- 172.21.15.31:0/3078709188 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c06b6d0 con 0x7f6c4c15d7b0 2023-11-26T07:17:33.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c51df6640 1 --2- 172.21.15.31:0/3078709188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c15d7b0 0x7f6c4c0d10d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c08c870 tx=0x7f6c4801c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22c98a1f3021ad8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:33.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c50df4640 1 -- 172.21.15.31:0/3078709188 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480ab600 con 0x7f6c4c15d7b0 2023-11-26T07:17:33.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c50df4640 1 -- 172.21.15.31:0/3078709188 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c480a7050 con 0x7f6c4c15d7b0 2023-11-26T07:17:33.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c50df4640 1 -- 172.21.15.31:0/3078709188 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480ab980 con 0x7f6c4c15d7b0 2023-11-26T07:17:33.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c5405a640 1 -- 172.21.15.31:0/3078709188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c15d7b0 msgr2=0x7f6c4c0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c5405a640 1 --2- 172.21.15.31:0/3078709188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c15d7b0 0x7f6c4c0d10d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c08c870 tx=0x7f6c4801c310 comp rx=0 tx=0).stop 2023-11-26T07:17:33.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c5405a640 1 -- 172.21.15.31:0/3078709188 shutdown_connections 2023-11-26T07:17:33.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c5405a640 1 --2- 172.21.15.31:0/3078709188 >> v2:172.21.15.97:3300/0 conn(0x7f6c4c0d2080 0x7f6c4c06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c5405a640 1 --2- 172.21.15.31:0/3078709188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c15d7b0 0x7f6c4c0d10d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c5405a640 1 -- 172.21.15.31:0/3078709188 >> 172.21.15.31:0/3078709188 conn(0x7f6c4c0d33b0 msgr2=0x7f6c4c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:33.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c5405a640 1 -- 172.21.15.31:0/3078709188 shutdown_connections 2023-11-26T07:17:33.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c5405a640 1 -- 172.21.15.31:0/3078709188 wait complete. 2023-11-26T07:17:33.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c5405a640 1 Processor -- start 2023-11-26T07:17:33.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.313+0000 7f6c5405a640 1 -- start start 2023-11-26T07:17:33.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c5405a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c0cead0 0x7f6c4c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c5405a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4c0d2080 0x7f6c4c0c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c5405a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c4c15d7b0 0x7f6c4c15b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c5405a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c0c8d30 con 0x7f6c4c0cead0 2023-11-26T07:17:33.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c5405a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c0c8120 con 0x7f6c4c0d2080 2023-11-26T07:17:33.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c5405a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c0c8390 con 0x7f6c4c15d7b0 2023-11-26T07:17:33.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c51df6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c0cead0 0x7f6c4c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c51df6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c0cead0 0x7f6c4c0cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52342/0 (socket says 172.21.15.31:52342) 2023-11-26T07:17:33.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c51df6640 1 -- 172.21.15.31:0/928439827 learned_addr learned my addr 172.21.15.31:0/928439827 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:33.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c515f5640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4c0d2080 0x7f6c4c0c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c525f7640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c4c15d7b0 0x7f6c4c15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c51df6640 1 -- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c4c15d7b0 msgr2=0x7f6c4c15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c51df6640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c4c15d7b0 0x7f6c4c15b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c51df6640 1 -- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4c0d2080 msgr2=0x7f6c4c0c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c51df6640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4c0d2080 0x7f6c4c0c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c51df6640 1 -- 172.21.15.31:0/928439827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48009420 con 0x7f6c4c0cead0 2023-11-26T07:17:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c51df6640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c0cead0 0x7f6c4c0cb3a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c480ae040 tx=0x7f6c480190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c427fc640 1 -- 172.21.15.31:0/928439827 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480b4410 con 0x7f6c4c0cead0 2023-11-26T07:17:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c427fc640 1 -- 172.21.15.31:0/928439827 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c480b4570 con 0x7f6c4c0cead0 2023-11-26T07:17:33.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c427fc640 1 -- 172.21.15.31:0/928439827 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480b4920 con 0x7f6c4c0cead0 2023-11-26T07:17:33.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c5405a640 1 -- 172.21.15.31:0/928439827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c0c8670 con 0x7f6c4c0cead0 2023-11-26T07:17:33.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c5405a640 1 -- 172.21.15.31:0/928439827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c0cb8e0 con 0x7f6c4c0cead0 2023-11-26T07:17:33.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c5405a640 1 -- 172.21.15.31:0/928439827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c0c8e70 con 0x7f6c4c0cead0 2023-11-26T07:17:33.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.317+0000 7f6c427fc640 1 -- 172.21.15.31:0/928439827 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6c480b2040 con 0x7f6c4c0cead0 2023-11-26T07:17:33.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.321+0000 7f6c427fc640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 0x7f6c30068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.321+0000 7f6c427fc640 1 -- 172.21.15.31:0/928439827 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c4811dc80 con 0x7f6c4c0cead0 2023-11-26T07:17:33.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.321+0000 7f6c515f5640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 0x7f6c30068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.321+0000 7f6c515f5640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 0x7f6c30068930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0c6ad0 tx=0x7f6c3c007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:33.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.325+0000 7f6c427fc640 1 -- 172.21.15.31:0/928439827 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c480eb240 con 0x7f6c4c0cead0 2023-11-26T07:17:33.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.481+0000 7f6c5405a640 1 -- 172.21.15.31:0/928439827 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c4c0cbad0 con 0x7f6c4c0cead0 2023-11-26T07:17:33.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c427fc640 1 -- 172.21.15.31:0/928439827 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16434 (secure 0 0 0) 0x7f6c480b0090 con 0x7f6c4c0cead0 2023-11-26T07:17:33.488 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:33.488 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:32.510427+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:33.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 -- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 msgr2=0x7f6c30068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 0x7f6c30068930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0c6ad0 tx=0x7f6c3c007d40 comp rx=0 tx=0).stop 2023-11-26T07:17:33.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 -- 172.21.15.31:0/928439827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c0cead0 msgr2=0x7f6c4c0cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c0cead0 0x7f6c4c0cb3a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c480ae040 tx=0x7f6c480190b0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 -- 172.21.15.31:0/928439827 shutdown_connections 2023-11-26T07:17:33.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 0x7f6c30068930 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c4c15d7b0 0x7f6c4c15b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c4c0d2080 0x7f6c4c0c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 --2- 172.21.15.31:0/928439827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4c0cead0 0x7f6c4c0cb3a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 -- 172.21.15.31:0/928439827 >> 172.21.15.31:0/928439827 conn(0x7f6c4c0d33b0 msgr2=0x7f6c4c0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:33.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 -- 172.21.15.31:0/928439827 shutdown_connections 2023-11-26T07:17:33.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.485+0000 7f6c5405a640 1 -- 172.21.15.31:0/928439827 wait complete. 2023-11-26T07:17:33.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:33.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.645+0000 7eff1eb03640 1 Processor -- start 2023-11-26T07:17:33.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1eb03640 1 -- start start 2023-11-26T07:17:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1eb03640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff180724e0 0x7eff1813efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1eb03640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff1813f520 0x7eff1813f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1eb03640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1815a7c0 con 0x7eff18072100 2023-11-26T07:17:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1eb03640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1815b440 con 0x7eff180724e0 2023-11-26T07:17:33.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1eb03640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1813d3f0 con 0x7eff1813f520 2023-11-26T07:17:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1d300640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff180724e0 0x7eff1813efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1d300640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff180724e0 0x7eff1813efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52354/0 (socket says 172.21.15.31:52354) 2023-11-26T07:17:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1d300640 1 -- 172.21.15.31:0/2941652554 learned_addr learned my addr 172.21.15.31:0/2941652554 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1e302640 1 --2- 172.21.15.31:0/2941652554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff1813f520 0x7eff1813f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1d300640 1 -- 172.21.15.31:0/2941652554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff1813f520 msgr2=0x7eff1813f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1d300640 1 --2- 172.21.15.31:0/2941652554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff1813f520 0x7eff1813f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1d300640 1 -- 172.21.15.31:0/2941652554 >> v1:172.21.15.97:6789/0 conn(0x7eff18072100 legacy=0x7eff1815e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:33.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.649+0000 7eff1d300640 1 -- 172.21.15.31:0/2941652554 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1815aaa0 con 0x7eff180724e0 2023-11-26T07:17:33.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1d300640 1 --2- 172.21.15.31:0/2941652554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff180724e0 0x7eff1813efe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff0800bb10 tx=0x7eff0801eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6ab196a3b85df9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1caff640 1 -- 172.21.15.31:0/2941652554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff080a7ba0 con 0x7eff180724e0 2023-11-26T07:17:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1caff640 1 -- 172.21.15.31:0/2941652554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff080a7d00 con 0x7eff180724e0 2023-11-26T07:17:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1caff640 1 -- 172.21.15.31:0/2941652554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff080af660 con 0x7eff180724e0 2023-11-26T07:17:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1eb03640 1 -- 172.21.15.31:0/2941652554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff180724e0 msgr2=0x7eff1813efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1eb03640 1 --2- 172.21.15.31:0/2941652554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff180724e0 0x7eff1813efe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7eff0800bb10 tx=0x7eff0801eb10 comp rx=0 tx=0).stop 2023-11-26T07:17:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1eb03640 1 -- 172.21.15.31:0/2941652554 shutdown_connections 2023-11-26T07:17:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1eb03640 1 --2- 172.21.15.31:0/2941652554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff1813f520 0x7eff1813f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1eb03640 1 --2- 172.21.15.31:0/2941652554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff180724e0 0x7eff1813efe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1eb03640 1 -- 172.21.15.31:0/2941652554 >> 172.21.15.31:0/2941652554 conn(0x7eff180b0e00 msgr2=0x7eff180b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:33.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1eb03640 1 -- 172.21.15.31:0/2941652554 shutdown_connections 2023-11-26T07:17:33.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1eb03640 1 -- 172.21.15.31:0/2941652554 wait complete. 2023-11-26T07:17:33.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1eb03640 1 Processor -- start 2023-11-26T07:17:33.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.653+0000 7eff1eb03640 1 -- start start 2023-11-26T07:17:33.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1eb03640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18072100 0x7eff180b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1eb03640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff180724e0 0x7eff181353c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1eb03640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff1813f520 0x7eff1815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1eb03640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff181379c0 con 0x7eff1813f520 2023-11-26T07:17:33.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1eb03640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff18136260 con 0x7eff18072100 2023-11-26T07:17:33.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1eb03640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff18135ca0 con 0x7eff180724e0 2023-11-26T07:17:33.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1e302640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff1813f520 0x7eff1815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1e302640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff1813f520 0x7eff1815b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52356/0 (socket says 172.21.15.31:52356) 2023-11-26T07:17:33.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1e302640 1 -- 172.21.15.31:0/3257587852 learned_addr learned my addr 172.21.15.31:0/3257587852 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:33.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1db01640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18072100 0x7eff180b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1d300640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff180724e0 0x7eff181353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1e302640 1 -- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff180724e0 msgr2=0x7eff181353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1e302640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff180724e0 0x7eff181353c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1e302640 1 -- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18072100 msgr2=0x7eff180b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1e302640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18072100 0x7eff180b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1e302640 1 -- 172.21.15.31:0/3257587852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0800b770 con 0x7eff1813f520 2023-11-26T07:17:33.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1d300640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff180724e0 0x7eff181353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:33.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1db01640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18072100 0x7eff180b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:33.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1e302640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff1813f520 0x7eff1815b900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff10007e60 tx=0x7eff1000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:33.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff0e7fc640 1 -- 172.21.15.31:0/3257587852 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10014420 con 0x7eff1813f520 2023-11-26T07:17:33.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff0e7fc640 1 -- 172.21.15.31:0/3257587852 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff100145a0 con 0x7eff1813f520 2023-11-26T07:17:33.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff0e7fc640 1 -- 172.21.15.31:0/3257587852 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10014950 con 0x7eff1813f520 2023-11-26T07:17:33.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1eb03640 1 -- 172.21.15.31:0/3257587852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff18135f80 con 0x7eff1813f520 2023-11-26T07:17:33.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1eb03640 1 -- 172.21.15.31:0/3257587852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff18135900 con 0x7eff1813f520 2023-11-26T07:17:33.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff1eb03640 1 -- 172.21.15.31:0/3257587852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff18137b30 con 0x7eff1813f520 2023-11-26T07:17:33.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.657+0000 7eff0e7fc640 1 -- 172.21.15.31:0/3257587852 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7eff10014ab0 con 0x7eff1813f520 2023-11-26T07:17:33.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.661+0000 7eff0e7fc640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efefc0664a0 0x7efefc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:33.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.661+0000 7eff0e7fc640 1 -- 172.21.15.31:0/3257587852 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff100846e0 con 0x7eff1813f520 2023-11-26T07:17:33.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.661+0000 7eff1db01640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efefc0664a0 0x7efefc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:33.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.661+0000 7eff1db01640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efefc0664a0 0x7efefc068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7eff14003ec0 tx=0x7eff14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:33.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.665+0000 7eff0e7fc640 1 -- 172.21.15.31:0/3257587852 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7eff10051bd0 con 0x7eff1813f520 2023-11-26T07:17:33.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.825+0000 7eff1eb03640 1 -- 172.21.15.31:0/3257587852 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff18074a00 con 0x7eff1813f520 2023-11-26T07:17:33.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.825+0000 7eff0e7fc640 1 -- 172.21.15.31:0/3257587852 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16434 (secure 0 0 0) 0x7eff100551f0 con 0x7eff1813f520 2023-11-26T07:17:33.830 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:33.830 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:32.510427+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 -- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efefc0664a0 msgr2=0x7efefc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efefc0664a0 0x7efefc068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7eff14003ec0 tx=0x7eff14009040 comp rx=0 tx=0).stop 2023-11-26T07:17:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 -- 172.21.15.31:0/3257587852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff1813f520 msgr2=0x7eff1815b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff1813f520 0x7eff1815b900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7eff10007e60 tx=0x7eff1000e4b0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 -- 172.21.15.31:0/3257587852 shutdown_connections 2023-11-26T07:17:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efefc0664a0 0x7efefc068960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff1813f520 0x7eff1815b900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff180724e0 0x7eff181353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 --2- 172.21.15.31:0/3257587852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff18072100 0x7eff180b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:33.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 -- 172.21.15.31:0/3257587852 >> 172.21.15.31:0/3257587852 conn(0x7eff180b0e00 msgr2=0x7eff18070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:33.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 -- 172.21.15.31:0/3257587852 shutdown_connections 2023-11-26T07:17:33.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.829+0000 7eff1eb03640 1 -- 172.21.15.31:0/3257587852 wait complete. 2023-11-26T07:17:33.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:17:33.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8ebe879640 1 Processor -- start 2023-11-26T07:17:33.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8ebe879640 1 -- start start 2023-11-26T07:17:34.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8ebe879640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80ce4b0 0x7f8eb80caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:34.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8ebe879640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8eb806c520 0x7f8eb806c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:34.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8ebe879640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb806ceb0 con 0x7f8eb815d610 2023-11-26T07:17:34.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8ebe879640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb80ce910 con 0x7f8eb806c520 2023-11-26T07:17:34.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8ebe879640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80cbfe0 con 0x7f8eb80ce4b0 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb7fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8eb815d610 0x7f8eb80cb8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46164/0 (socket says 172.21.15.31:46164) 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb7fff640 1 -- 172.21.15.31:0/3373066325 learned_addr learned my addr 172.21.15.31:0/3373066325 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221326130 0 0) 0x7f8ea8001120 con 0x7f8eb815d610 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb77fe640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80ce4b0 0x7f8eb80caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8ebce16640 1 --2- 172.21.15.31:0/3373066325 >> v2:172.21.15.97:3300/0 conn(0x7f8eb806c520 0x7f8eb806c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea0003530 con 0x7f8eb815d610 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2198931794 0 0) 0x7f8ea0003530 con 0x7f8eb815d610 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80ce4b0 msgr2=0x7f8eb80caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80ce4b0 0x7f8eb80caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 >> v2:172.21.15.97:3300/0 conn(0x7f8eb806c520 msgr2=0x7f8eb806c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 --2- 172.21.15.31:0/3373066325 >> v2:172.21.15.97:3300/0 conn(0x7f8eb806c520 0x7f8eb806c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb806b970 con 0x7f8eb815d610 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f8ea8001810 con 0x7f8eb815d610 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 >> v1:172.21.15.31:6789/0 conn(0x7f8eb815d610 legacy=0x7f8eb80cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea0003980 0x7f8ea0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea00047d0 0x7f8ea0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80ce4b0 0x7f8ea0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80ce910 con 0x7f8ea00047d0 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80cbfe0 con 0x7f8ea0003980 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.993+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea000b990 con 0x7f8eb80ce4b0 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb77fe640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea00047d0 0x7f8ea0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb7fff640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea0003980 0x7f8ea0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8ebce16640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80ce4b0 0x7f8ea0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:34.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb77fe640 1 -- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80ce4b0 msgr2=0x7f8ea0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:34.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb77fe640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80ce4b0 0x7f8ea0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:34.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb77fe640 1 -- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea0003980 msgr2=0x7f8ea0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:34.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb77fe640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ea0003980 0x7f8ea0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:34.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb77fe640 1 -- 172.21.15.31:0/3373066325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea0003530 con 0x7f8ea00047d0 2023-11-26T07:17:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb77fe640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea00047d0 0x7f8ea0004c50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009ae0 tx=0x7f8eac01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4449313bb15afa57 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0a9450 con 0x7f8ea00047d0 2023-11-26T07:17:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac0a95b0 con 0x7f8ea00047d0 2023-11-26T07:17:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:33.997+0000 7f8eb6ffd640 1 -- 172.21.15.31:0/3373066325 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0a9960 con 0x7f8ea00047d0 2023-11-26T07:17:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 -- 172.21.15.31:0/3373066325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea00047d0 msgr2=0x7f8ea0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 --2- 172.21.15.31:0/3373066325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ea00047d0 0x7f8ea0004c50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8eac009ae0 tx=0x7f8eac01ca50 comp rx=0 tx=0).stop 2023-11-26T07:17:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebce16640 1 -- 172.21.15.31:0/3373066325 reap_dead start 2023-11-26T07:17:34.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 -- 172.21.15.31:0/3373066325 shutdown_connections 2023-11-26T07:17:34.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 -- 172.21.15.31:0/3373066325 >> 172.21.15.31:0/3373066325 conn(0x7f8eb80b2f10 msgr2=0x7f8eb80b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:34.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 -- 172.21.15.31:0/3373066325 shutdown_connections 2023-11-26T07:17:34.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 -- 172.21.15.31:0/3373066325 wait complete. 2023-11-26T07:17:34.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 Processor -- start 2023-11-26T07:17:34.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 -- start start 2023-11-26T07:17:34.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea00047d0 0x7f8eb80b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:34.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb814de60 0x7f8eb81f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:34.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d17d0 0x7f8eb80d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:34.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80a8500 con 0x7f8eb80d17d0 2023-11-26T07:17:34.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80d2190 con 0x7f8eb814de60 2023-11-26T07:17:34.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebe879640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb81f7550 con 0x7f8ea00047d0 2023-11-26T07:17:34.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebce16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d17d0 0x7f8eb80d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:34.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebce16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d17d0 0x7f8eb80d1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59204/0 (socket says 172.21.15.31:59204) 2023-11-26T07:17:34.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8eb77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb814de60 0x7f8eb81f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:34.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8ebce16640 1 -- 172.21.15.31:0/2570874033 learned_addr learned my addr 172.21.15.31:0/2570874033 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:34.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.001+0000 7f8eb7fff640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea00047d0 0x7f8eb80b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:34.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8ebce16640 1 -- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea00047d0 msgr2=0x7f8eb80b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:34.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8ebce16640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea00047d0 0x7f8eb80b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:34.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8ebce16640 1 -- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb814de60 msgr2=0x7f8eb81f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:34.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8ebce16640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb814de60 0x7f8eb81f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:34.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8ebce16640 1 -- 172.21.15.31:0/2570874033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb814e2c0 con 0x7f8eb80d17d0 2023-11-26T07:17:34.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8ebce16640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d17d0 0x7f8eb80d1c50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8eb81610e0 tx=0x7f8eb000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:34.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8eb4ff9640 1 -- 172.21.15.31:0/2570874033 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb000e040 con 0x7f8eb80d17d0 2023-11-26T07:17:34.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8eb4ff9640 1 -- 172.21.15.31:0/2570874033 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb000f040 con 0x7f8eb80d17d0 2023-11-26T07:17:34.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8eb4ff9640 1 -- 172.21.15.31:0/2570874033 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0015670 con 0x7f8eb80d17d0 2023-11-26T07:17:34.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8ebe879640 1 -- 172.21.15.31:0/2570874033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb80b5430 con 0x7f8eb80d17d0 2023-11-26T07:17:34.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8ebe879640 1 -- 172.21.15.31:0/2570874033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb80b59f0 con 0x7f8eb80d17d0 2023-11-26T07:17:34.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8ebe879640 1 -- 172.21.15.31:0/2570874033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb804e850 con 0x7f8eb80d17d0 2023-11-26T07:17:34.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8eb4ff9640 1 -- 172.21.15.31:0/2570874033 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8eb00157d0 con 0x7f8eb80d17d0 2023-11-26T07:17:34.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8eb4ff9640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8ea4066470 0x7f8ea4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:34.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.005+0000 7f8eb4ff9640 1 -- 172.21.15.31:0/2570874033 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8eb00845d0 con 0x7f8eb80d17d0 2023-11-26T07:17:34.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.009+0000 7f8eb7fff640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8ea4066470 0x7f8ea4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:34.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.009+0000 7f8eb7fff640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8ea4066470 0x7f8ea4068930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80012c0 tx=0x7f8ea8007a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:34.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.009+0000 7f8eb4ff9640 1 -- 172.21.15.31:0/2570874033 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8eb0051920 con 0x7f8eb80d17d0 2023-11-26T07:17:34.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:34.205+0000 7f8ebe879640 1 -- 172.21.15.31:0/2570874033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8eb80d12b0 con 0x7f8eb80d17d0 2023-11-26T07:17:35.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8eb4ff9640 1 -- 172.21.15.31:0/2570874033 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8eb0054f40 con 0x7f8eb80d17d0 2023-11-26T07:17:35.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 -- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8ea4066470 msgr2=0x7f8ea4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8ea4066470 0x7f8ea4068930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ea80012c0 tx=0x7f8ea8007a80 comp rx=0 tx=0).stop 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 -- 172.21.15.31:0/2570874033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d17d0 msgr2=0x7f8eb80d1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d17d0 0x7f8eb80d1c50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8eb81610e0 tx=0x7f8eb000f580 comp rx=0 tx=0).stop 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 -- 172.21.15.31:0/2570874033 shutdown_connections 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8ea4066470 0x7f8ea4068930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d17d0 0x7f8eb80d1c50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb814de60 0x7f8eb81f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 --2- 172.21.15.31:0/2570874033 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ea00047d0 0x7f8eb80b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 -- 172.21.15.31:0/2570874033 >> 172.21.15.31:0/2570874033 conn(0x7f8eb80b2f10 msgr2=0x7f8eb80d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 -- 172.21.15.31:0/2570874033 shutdown_connections 2023-11-26T07:17:35.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.129+0000 7f8ebe879640 1 -- 172.21.15.31:0/2570874033 wait complete. 2023-11-26T07:17:35.157 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:17:35.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:17:35.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f445229c640 1 Processor -- start 2023-11-26T07:17:35.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f445229c640 1 -- start start 2023-11-26T07:17:35.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f445229c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 0x7f444c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:35.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f445229c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f444c0cbd90 0x7f444c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:35.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f445229c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c159630 con 0x7f444c0ce5d0 2023-11-26T07:17:35.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f445229c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c15bba0 con 0x7f444c0cbd90 2023-11-26T07:17:35.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f445229c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0cdd50 con 0x7f444c0cb9b0 2023-11-26T07:17:35.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f4450839640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f444c0ce5d0 0x7f444c15bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46176/0 (socket says 172.21.15.31:46176) 2023-11-26T07:17:35.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f4450839640 1 -- 172.21.15.31:0/885163485 learned_addr learned my addr 172.21.15.31:0/885163485 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:35.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f444b7fe640 1 --2- 172.21.15.31:0/885163485 >> v2:172.21.15.97:3300/0 conn(0x7f444c0cbd90 0x7f444c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f444bfff640 1 --2- 172.21.15.31:0/885163485 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 0x7f444c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f444affd640 1 -- 172.21.15.31:0/885163485 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161672149 0 0) 0x7f444c159630 con 0x7f444c0ce5d0 2023-11-26T07:17:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f444affd640 1 -- 172.21.15.31:0/885163485 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4434003530 con 0x7f444c0ce5d0 2023-11-26T07:17:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f444affd640 1 -- 172.21.15.31:0/885163485 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3212143535 0 0) 0x7f4434003530 con 0x7f444c0ce5d0 2023-11-26T07:17:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f444b7fe640 1 -- 172.21.15.31:0/885163485 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 msgr2=0x7f444c06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f444b7fe640 1 --2- 172.21.15.31:0/885163485 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 0x7f444c06e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f444b7fe640 1 -- 172.21.15.31:0/885163485 >> v1:172.21.15.31:6789/0 conn(0x7f444c0ce5d0 legacy=0x7f444c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f444b7fe640 1 -- 172.21.15.31:0/885163485 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c15f6d0 con 0x7f444c0cbd90 2023-11-26T07:17:35.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.305+0000 7f444bfff640 1 --2- 172.21.15.31:0/885163485 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 0x7f444c06e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:35.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f444b7fe640 1 --2- 172.21.15.31:0/885163485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444c0cbd90 0x7f444c15f190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f44400097c0 tx=0x7f444001c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed5c9e9181ee102b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:35.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f444affd640 1 -- 172.21.15.31:0/885163485 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444001a00 con 0x7f444c0cbd90 2023-11-26T07:17:35.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f444affd640 1 -- 172.21.15.31:0/885163485 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44400a7050 con 0x7f444c0cbd90 2023-11-26T07:17:35.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f444affd640 1 -- 172.21.15.31:0/885163485 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400ab620 con 0x7f444c0cbd90 2023-11-26T07:17:35.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f445229c640 1 -- 172.21.15.31:0/885163485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444c0cbd90 msgr2=0x7f444c15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:35.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f445229c640 1 --2- 172.21.15.31:0/885163485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444c0cbd90 0x7f444c15f190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f44400097c0 tx=0x7f444001c370 comp rx=0 tx=0).stop 2023-11-26T07:17:35.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f445229c640 1 -- 172.21.15.31:0/885163485 shutdown_connections 2023-11-26T07:17:35.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f445229c640 1 --2- 172.21.15.31:0/885163485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444c0cbd90 0x7f444c15f190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:35.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f445229c640 1 --2- 172.21.15.31:0/885163485 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 0x7f444c06e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:35.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f445229c640 1 -- 172.21.15.31:0/885163485 >> 172.21.15.31:0/885163485 conn(0x7f444c0b0cb0 msgr2=0x7f444c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:35.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f445229c640 1 -- 172.21.15.31:0/885163485 shutdown_connections 2023-11-26T07:17:35.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.309+0000 7f445229c640 1 -- 172.21.15.31:0/885163485 wait complete. 2023-11-26T07:17:35.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f445229c640 1 Processor -- start 2023-11-26T07:17:35.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f445229c640 1 -- start start 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f445229c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 0x7f444c153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f445229c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c0cbd90 0x7f444c1531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f445229c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444c0ce5d0 0x7f444c15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f445229c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0b3fa0 con 0x7f444c0cbd90 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f445229c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c153700 con 0x7f444c0ce5d0 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f445229c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0b3cc0 con 0x7f444c0cb9b0 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c0cbd90 0x7f444c1531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f444b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c0cbd90 0x7f444c1531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59206/0 (socket says 172.21.15.31:59206) 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f444b7fe640 1 -- 172.21.15.31:0/1551879582 learned_addr learned my addr 172.21.15.31:0/1551879582 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f444bfff640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 0x7f444c153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:35.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f4450839640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444c0ce5d0 0x7f444c15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:35.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f444b7fe640 1 -- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 msgr2=0x7f444c153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:35.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f444b7fe640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 0x7f444c153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:35.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f444b7fe640 1 -- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444c0ce5d0 msgr2=0x7f444c15a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:35.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f444b7fe640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444c0ce5d0 0x7f444c15a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:35.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f444b7fe640 1 -- 172.21.15.31:0/1551879582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c009420 con 0x7f444c0cbd90 2023-11-26T07:17:35.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f444b7fe640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c0cbd90 0x7f444c1531c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44400ae040 tx=0x7f4440019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:35.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f4448ff9640 1 -- 172.21.15.31:0/1551879582 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400b4430 con 0x7f444c0cbd90 2023-11-26T07:17:35.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f4448ff9640 1 -- 172.21.15.31:0/1551879582 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44400b45b0 con 0x7f444c0cbd90 2023-11-26T07:17:35.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f445229c640 1 -- 172.21.15.31:0/1551879582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4440009420 con 0x7f444c0cbd90 2023-11-26T07:17:35.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f4448ff9640 1 -- 172.21.15.31:0/1551879582 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400b4960 con 0x7f444c0cbd90 2023-11-26T07:17:35.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.313+0000 7f445229c640 1 -- 172.21.15.31:0/1551879582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c0ace20 con 0x7f444c0cbd90 2023-11-26T07:17:35.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.317+0000 7f445229c640 1 -- 172.21.15.31:0/1551879582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c0671f0 con 0x7f444c0cbd90 2023-11-26T07:17:35.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.317+0000 7f4448ff9640 1 -- 172.21.15.31:0/1551879582 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f44400b2040 con 0x7f444c0cbd90 2023-11-26T07:17:35.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.317+0000 7f4448ff9640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4438066470 0x7f4438068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:35.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.317+0000 7f4448ff9640 1 -- 172.21.15.31:0/1551879582 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f444011dd40 con 0x7f444c0cbd90 2023-11-26T07:17:35.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.317+0000 7f444bfff640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4438066470 0x7f4438068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:35.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.317+0000 7f444bfff640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4438066470 0x7f4438068930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f444c1fa2c0 tx=0x7f443c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:35.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.321+0000 7f4448ff9640 1 -- 172.21.15.31:0/1551879582 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f44400eb120 con 0x7f444c0cbd90 2023-11-26T07:17:35.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:35.509+0000 7f445229c640 1 -- 172.21.15.31:0/1551879582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f444c0ac9a0 con 0x7f444c0cbd90 2023-11-26T07:17:36.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f4448ff9640 1 -- 172.21.15.31:0/1551879582 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f444c0ac9a0 con 0x7f444c0cbd90 2023-11-26T07:17:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f445229c640 1 -- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4438066470 msgr2=0x7f4438068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f445229c640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4438066470 0x7f4438068930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f444c1fa2c0 tx=0x7f443c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f445229c640 1 -- 172.21.15.31:0/1551879582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c0cbd90 msgr2=0x7f444c1531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f445229c640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c0cbd90 0x7f444c1531c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44400ae040 tx=0x7f4440019110 comp rx=0 tx=0).stop 2023-11-26T07:17:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f445229c640 1 -- 172.21.15.31:0/1551879582 shutdown_connections 2023-11-26T07:17:36.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f445229c640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4438066470 0x7f4438068930 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f445229c640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f444c0ce5d0 0x7f444c15a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f445229c640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f444c0cbd90 0x7f444c1531c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f445229c640 1 --2- 172.21.15.31:0/1551879582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f444c0cb9b0 0x7f444c153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.137+0000 7f445229c640 1 -- 172.21.15.31:0/1551879582 >> 172.21.15.31:0/1551879582 conn(0x7f444c0b0cb0 msgr2=0x7f444c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.141+0000 7f445229c640 1 -- 172.21.15.31:0/1551879582 shutdown_connections 2023-11-26T07:17:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.141+0000 7f445229c640 1 -- 172.21.15.31:0/1551879582 wait complete. 2023-11-26T07:17:36.165 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:17:36.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:36.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2240ad6640 1 Processor -- start 2023-11-26T07:17:36.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2240ad6640 1 -- start start 2023-11-26T07:17:36.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2240ad6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c15d7b0 0x7f223c0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2240ad6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f223c0cead0 0x7f223c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2240ad6640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223c0cefd0 con 0x7f223c0d1eb0 2023-11-26T07:17:36.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2240ad6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223c0d2340 con 0x7f223c0cead0 2023-11-26T07:17:36.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2240ad6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0cf2b0 con 0x7f223c15d7b0 2023-11-26T07:17:36.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f223ad9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f223c0d1eb0 0x7f223c0cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46178/0 (socket says 172.21.15.31:46178) 2023-11-26T07:17:36.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f223ad9d640 1 -- 172.21.15.31:0/2696273799 learned_addr learned my addr 172.21.15.31:0/2696273799 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:36.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2233fff640 1 --2- 172.21.15.31:0/2696273799 >> v2:172.21.15.97:3300/0 conn(0x7f223c0cead0 0x7f223c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f223a59c640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c15d7b0 0x7f223c0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485483649 0 0) 0x7f223c0cefd0 con 0x7f223c0d1eb0 2023-11-26T07:17:36.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2228003530 con 0x7f223c0d1eb0 2023-11-26T07:17:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4231988898 0 0) 0x7f2228003530 con 0x7f223c0d1eb0 2023-11-26T07:17:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c15d7b0 msgr2=0x7f223c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c15d7b0 0x7f223c0d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 >> v2:172.21.15.97:3300/0 conn(0x7f223c0cead0 msgr2=0x7f223c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 --2- 172.21.15.31:0/2696273799 >> v2:172.21.15.97:3300/0 conn(0x7f223c0cead0 0x7f223c0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c06b3f0 con 0x7f223c0d1eb0 2023-11-26T07:17:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f2234001a00 con 0x7f223c0d1eb0 2023-11-26T07:17:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 >> v1:172.21.15.31:6789/0 conn(0x7f223c0d1eb0 legacy=0x7f223c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.309+0000 7f2239d9b640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c15d7b0 0x7f2228003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f2239d9b640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2228003f00 0x7f22280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f2239d9b640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 0x7f2228005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0d2340 con 0x7f2228004d10 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0cf2b0 con 0x7f2228003f00 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222800ba70 con 0x7f223c15d7b0 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f223ad9d640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 0x7f2228005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f2233fff640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2228003f00 0x7f22280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f223a59c640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c15d7b0 0x7f2228003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f223ad9d640 1 -- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c15d7b0 msgr2=0x7f2228003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f223ad9d640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c15d7b0 0x7f2228003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f223ad9d640 1 -- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2228003f00 msgr2=0x7f22280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f223ad9d640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2228003f00 0x7f22280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f223ad9d640 1 -- 172.21.15.31:0/2696273799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2228003530 con 0x7f2228004d10 2023-11-26T07:17:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f223ad9d640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 0x7f2228005190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f223400a860 tx=0x7f223401d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a7db4511ba5be5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340ad630 con 0x7f2228004d10 2023-11-26T07:17:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22340a8070 con 0x7f2228004d10 2023-11-26T07:17:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.313+0000 7f2239d9b640 1 -- 172.21.15.31:0/2696273799 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340ada30 con 0x7f2228004d10 2023-11-26T07:17:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 -- 172.21.15.31:0/2696273799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 msgr2=0x7f2228005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 --2- 172.21.15.31:0/2696273799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 0x7f2228005190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f223400a860 tx=0x7f223401d330 comp rx=0 tx=0).stop 2023-11-26T07:17:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f223ad9d640 1 -- 172.21.15.31:0/2696273799 reap_dead start 2023-11-26T07:17:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 -- 172.21.15.31:0/2696273799 shutdown_connections 2023-11-26T07:17:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 -- 172.21.15.31:0/2696273799 >> 172.21.15.31:0/2696273799 conn(0x7f223c0d33b0 msgr2=0x7f223c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 -- 172.21.15.31:0/2696273799 shutdown_connections 2023-11-26T07:17:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 -- 172.21.15.31:0/2696273799 wait complete. 2023-11-26T07:17:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 Processor -- start 2023-11-26T07:17:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 -- start start 2023-11-26T07:17:36.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 0x7f223c1f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f223c0cbdc0 0x7f223c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c0cc950 0x7f223c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0cc240 con 0x7f2228004d10 2023-11-26T07:17:36.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c09d3c0 con 0x7f223c0cbdc0 2023-11-26T07:17:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f2240ad6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0cd310 con 0x7f223c0cc950 2023-11-26T07:17:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f223a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 0x7f223c1f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f223a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 0x7f223c1f5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59212/0 (socket says 172.21.15.31:59212) 2023-11-26T07:17:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f223a59c640 1 -- 172.21.15.31:0/2760249704 learned_addr learned my addr 172.21.15.31:0/2760249704 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f223a59c640 1 -- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c0cc950 msgr2=0x7f223c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f223a59c640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c0cc950 0x7f223c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f223a59c640 1 -- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f223c0cbdc0 msgr2=0x7f223c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f223a59c640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f223c0cbdc0 0x7f223c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.317+0000 7f223a59c640 1 -- 172.21.15.31:0/2760249704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c0b73a0 con 0x7f2228004d10 2023-11-26T07:17:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f223a59c640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 0x7f223c1f5f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f222c0119d0 tx=0x7f222c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:36.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f2232ffd640 1 -- 172.21.15.31:0/2760249704 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00fa70 con 0x7f2228004d10 2023-11-26T07:17:36.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f2232ffd640 1 -- 172.21.15.31:0/2760249704 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c00fbd0 con 0x7f2228004d10 2023-11-26T07:17:36.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f2232ffd640 1 -- 172.21.15.31:0/2760249704 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c014440 con 0x7f2228004d10 2023-11-26T07:17:36.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f2240ad6640 1 -- 172.21.15.31:0/2760249704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223c0b6790 con 0x7f2228004d10 2023-11-26T07:17:36.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f2240ad6640 1 -- 172.21.15.31:0/2760249704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223c0b6d50 con 0x7f2228004d10 2023-11-26T07:17:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f2240ad6640 1 -- 172.21.15.31:0/2760249704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223c04e850 con 0x7f2228004d10 2023-11-26T07:17:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f2232ffd640 1 -- 172.21.15.31:0/2760249704 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f222c0145a0 con 0x7f2228004d10 2023-11-26T07:17:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f2232ffd640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f221c0664a0 0x7f221c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f2232ffd640 1 -- 172.21.15.31:0/2760249704 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f222c084610 con 0x7f2228004d10 2023-11-26T07:17:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.321+0000 7f2233fff640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f221c0664a0 0x7f221c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.325+0000 7f2233fff640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f221c0664a0 0x7f221c068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2224006b20 tx=0x7f2224006ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.325+0000 7f2232ffd640 1 -- 172.21.15.31:0/2760249704 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f222c051be0 con 0x7f2228004d10 2023-11-26T07:17:36.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 -- 172.21.15.31:0/2760249704 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f223c0b3500 con 0x7f2228004d10 2023-11-26T07:17:36.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2232ffd640 1 -- 172.21.15.31:0/2760249704 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f222c055200 con 0x7f2228004d10 2023-11-26T07:17:36.525 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:36.525 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4099326271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4099326271},{"type":"v1","addr":"172.21.15.31:6835","nonce":4099326271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2404164953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2404164953},{"type":"v1","addr":"172.21.15.97:6836","nonce":2404164953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1312969281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1312969281},{"type":"v1","addr":"172.21.15.31:6834","nonce":1312969281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:32.510772+0000","modified":"2023-11-26T07:17:35.514610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/527056216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":527056216},{"type":"v1","addr":"172.21.15.97:6837","nonce":527056216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:17:36.525 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-11-26T07:17:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 -- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f221c0664a0 msgr2=0x7f221c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f221c0664a0 0x7f221c068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2224006b20 tx=0x7f2224006ab0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 -- 172.21.15.31:0/2760249704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 msgr2=0x7f223c1f5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 0x7f223c1f5f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f222c0119d0 tx=0x7f222c011e90 comp rx=0 tx=0).stop 2023-11-26T07:17:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 -- 172.21.15.31:0/2760249704 shutdown_connections 2023-11-26T07:17:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f221c0664a0 0x7f221c068960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f223c0cc950 0x7f223c0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f223c0cbdc0 0x7f223c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 --2- 172.21.15.31:0/2760249704 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2228004d10 0x7f223c1f5f40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 -- 172.21.15.31:0/2760249704 >> 172.21.15.31:0/2760249704 conn(0x7f223c0d33b0 msgr2=0x7f223c15c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:36.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 -- 172.21.15.31:0/2760249704 shutdown_connections 2023-11-26T07:17:36.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.521+0000 7f2240ad6640 1 -- 172.21.15.31:0/2760249704 wait complete. 2023-11-26T07:17:36.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:36.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd304caa640 1 Processor -- start 2023-11-26T07:17:36.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd304caa640 1 -- start start 2023-11-26T07:17:36.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd304caa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd300146240 0x7fd300161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd304caa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3000b4920 0x7fd30015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd304caa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30006c840 con 0x7fd300146860 2023-11-26T07:17:36.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd304caa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd300146620 con 0x7fd3000b4920 2023-11-26T07:17:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd304caa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd300162000 con 0x7fd300146240 2023-11-26T07:17:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd2fed9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3000b4920 0x7fd30015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd2fed9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3000b4920 0x7fd30015daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59224/0 (socket says 172.21.15.31:59224) 2023-11-26T07:17:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd2f7fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd300146860 0x7fd3000b4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46190/0 (socket says 172.21.15.31:46190) 2023-11-26T07:17:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd2fed9d640 1 -- 172.21.15.31:0/1745418526 learned_addr learned my addr 172.21.15.31:0/1745418526 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd2fe59c640 1 --2- 172.21.15.31:0/1745418526 >> v2:172.21.15.97:3300/0 conn(0x7fd300146240 0x7fd300161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd2fed9d640 1 -- 172.21.15.31:0/1745418526 >> v2:172.21.15.97:3300/0 conn(0x7fd300146240 msgr2=0x7fd300161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:36.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd2fed9d640 1 --2- 172.21.15.31:0/1745418526 >> v2:172.21.15.97:3300/0 conn(0x7fd300146240 0x7fd300161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd2fed9d640 1 -- 172.21.15.31:0/1745418526 >> v1:172.21.15.31:6789/0 conn(0x7fd300146860 legacy=0x7fd3000b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd2fdd9b640 1 -- 172.21.15.31:0/1745418526 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159176702 0 0) 0x7fd30006c840 con 0x7fd300146860 2023-11-26T07:17:36.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.685+0000 7fd2fed9d640 1 -- 172.21.15.31:0/1745418526 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30006bc30 con 0x7fd3000b4920 2023-11-26T07:17:36.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd2fed9d640 1 --2- 172.21.15.31:0/1745418526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 0x7fd30015daf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd30006e3a0 tx=0x7fd2f801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb8f63f9d7d105d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:36.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd2fdd9b640 1 -- 172.21.15.31:0/1745418526 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80a5860 con 0x7fd3000b4920 2023-11-26T07:17:36.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd2fdd9b640 1 -- 172.21.15.31:0/1745418526 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f80a59c0 con 0x7fd3000b4920 2023-11-26T07:17:36.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd2fdd9b640 1 -- 172.21.15.31:0/1745418526 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80a5d50 con 0x7fd3000b4920 2023-11-26T07:17:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd304caa640 1 -- 172.21.15.31:0/1745418526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 msgr2=0x7fd30015daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd304caa640 1 --2- 172.21.15.31:0/1745418526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 0x7fd30015daf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd30006e3a0 tx=0x7fd2f801c9a0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd304caa640 1 -- 172.21.15.31:0/1745418526 shutdown_connections 2023-11-26T07:17:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd304caa640 1 --2- 172.21.15.31:0/1745418526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 0x7fd30015daf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd304caa640 1 --2- 172.21.15.31:0/1745418526 >> v2:172.21.15.97:3300/0 conn(0x7fd300146240 0x7fd300161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd304caa640 1 -- 172.21.15.31:0/1745418526 >> 172.21.15.31:0/1745418526 conn(0x7fd30006e950 msgr2=0x7fd30006ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd304caa640 1 -- 172.21.15.31:0/1745418526 shutdown_connections 2023-11-26T07:17:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.689+0000 7fd304caa640 1 -- 172.21.15.31:0/1745418526 wait complete. 2023-11-26T07:17:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd304caa640 1 Processor -- start 2023-11-26T07:17:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd304caa640 1 -- start start 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd304caa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 0x7fd30013f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd304caa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd300146240 0x7fd300138360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd304caa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd300146860 0x7fd30013d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd304caa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3001388a0 con 0x7fd3000b4920 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd304caa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd300135970 con 0x7fd300146860 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd304caa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30013e3e0 con 0x7fd300146240 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 0x7fd30013f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 0x7fd30013f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59228/0 (socket says 172.21.15.31:59228) 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fe59c640 1 -- 172.21.15.31:0/2463929882 learned_addr learned my addr 172.21.15.31:0/2463929882 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2f7fff640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd300146240 0x7fd300138360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fed9d640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd300146860 0x7fd30013d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fe59c640 1 -- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd300146240 msgr2=0x7fd300138360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fe59c640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd300146240 0x7fd300138360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fe59c640 1 -- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd300146860 msgr2=0x7fd30013d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fe59c640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd300146860 0x7fd30013d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fe59c640 1 -- 172.21.15.31:0/2463929882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f8009b30 con 0x7fd3000b4920 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2f7fff640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd300146240 0x7fd300138360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fed9d640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd300146860 0x7fd30013d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2fe59c640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 0x7fd30013f850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000e930 tx=0x7fd2f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2f6ffd640 1 -- 172.21.15.31:0/2463929882 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0009690 con 0x7fd3000b4920 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2f6ffd640 1 -- 172.21.15.31:0/2463929882 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f000ca00 con 0x7fd3000b4920 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd304caa640 1 -- 172.21.15.31:0/2463929882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30013e6c0 con 0x7fd3000b4920 2023-11-26T07:17:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd2f6ffd640 1 -- 172.21.15.31:0/2463929882 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f000cd70 con 0x7fd3000b4920 2023-11-26T07:17:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.693+0000 7fd304caa640 1 -- 172.21.15.31:0/2463929882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30013d650 con 0x7fd3000b4920 2023-11-26T07:17:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.697+0000 7fd304caa640 1 -- 172.21.15.31:0/2463929882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30004e850 con 0x7fd3000b4920 2023-11-26T07:17:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.697+0000 7fd2f6ffd640 1 -- 172.21.15.31:0/2463929882 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd2f0013470 con 0x7fd3000b4920 2023-11-26T07:17:36.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.697+0000 7fd2f6ffd640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd2e00664a0 0x7fd2e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:36.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.697+0000 7fd2f6ffd640 1 -- 172.21.15.31:0/2463929882 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2f00841c0 con 0x7fd3000b4920 2023-11-26T07:17:36.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.697+0000 7fd2f7fff640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd2e00664a0 0x7fd2e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:36.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.697+0000 7fd2f7fff640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd2e00664a0 0x7fd2e0068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8007fd0 tx=0x7fd2e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:36.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.701+0000 7fd2f6ffd640 1 -- 172.21.15.31:0/2463929882 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2f0051660 con 0x7fd3000b4920 2023-11-26T07:17:36.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 -- 172.21.15.31:0/2463929882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3000b0ab0 con 0x7fd3000b4920 2023-11-26T07:17:36.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd2f6ffd640 1 -- 172.21.15.31:0/2463929882 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fd2f0054c80 con 0x7fd3000b4920 2023-11-26T07:17:36.865 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:36.865 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 -- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd2e00664a0 msgr2=0x7fd2e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd2e00664a0 0x7fd2e0068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8007fd0 tx=0x7fd2e8009040 comp rx=0 tx=0).stop 2023-11-26T07:17:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 -- 172.21.15.31:0/2463929882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 msgr2=0x7fd30013f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 0x7fd30013f850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000e930 tx=0x7fd2f000edf0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 -- 172.21.15.31:0/2463929882 shutdown_connections 2023-11-26T07:17:36.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd2e00664a0 0x7fd2e0068960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd300146860 0x7fd30013d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd300146240 0x7fd300138360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 --2- 172.21.15.31:0/2463929882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3000b4920 0x7fd30013f850 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:36.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 -- 172.21.15.31:0/2463929882 >> 172.21.15.31:0/2463929882 conn(0x7fd30006e950 msgr2=0x7fd3000b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:36.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 -- 172.21.15.31:0/2463929882 shutdown_connections 2023-11-26T07:17:36.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:36.861+0000 7fd304caa640 1 -- 172.21.15.31:0/2463929882 wait complete. 2023-11-26T07:17:36.887 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:17:36.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:36.904 INFO:tasks.ceph.mds.b.smithi097.stderr:2023-11-26T07:17:36.900+0000 7f4eee0d8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:17:37.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.025+0000 7f59bde9c640 1 Processor -- start 2023-11-26T07:17:37.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.025+0000 7f59bde9c640 1 -- start start 2023-11-26T07:17:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.029+0000 7f59bde9c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59b80ce5c0 0x7f59b80cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.029+0000 7f59bde9c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b80b0a70 con 0x7f59b806a5b0 2023-11-26T07:17:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.029+0000 7f59bde9c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b80b4100 con 0x7f59b815d8b0 2023-11-26T07:17:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.029+0000 7f59bde9c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b80cddb0 con 0x7f59b80ce5c0 2023-11-26T07:17:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.029+0000 7f59b6ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59b80ce5c0 0x7f59b80cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.029+0000 7f59b6ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59b80ce5c0 0x7f59b80cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59238/0 (socket says 172.21.15.31:59238) 2023-11-26T07:17:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.029+0000 7f59b6ffd640 1 -- 172.21.15.31:0/603494483 learned_addr learned my addr 172.21.15.31:0/603494483 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:37.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.029+0000 7f59b6ffd640 1 -- 172.21.15.31:0/603494483 >> v1:172.21.15.97:6789/0 conn(0x7f59b815d8b0 legacy=0x7f59b80b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.029+0000 7f59b6ffd640 1 -- 172.21.15.31:0/603494483 >> v1:172.21.15.31:6789/0 conn(0x7f59b806a5b0 legacy=0x7f59b806aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.029+0000 7f59b6ffd640 1 -- 172.21.15.31:0/603494483 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b80b43e0 con 0x7f59b80ce5c0 2023-11-26T07:17:37.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59b6ffd640 1 --2- 172.21.15.31:0/603494483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80ce5c0 0x7f59b80cea20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009870 tx=0x7f59ac01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27a0ee20ce9be2f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59b67fc640 1 -- 172.21.15.31:0/603494483 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0a8a30 con 0x7f59b80ce5c0 2023-11-26T07:17:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59b67fc640 1 -- 172.21.15.31:0/603494483 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac0a8b90 con 0x7f59b80ce5c0 2023-11-26T07:17:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59b67fc640 1 -- 172.21.15.31:0/603494483 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0b0440 con 0x7f59b80ce5c0 2023-11-26T07:17:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 -- 172.21.15.31:0/603494483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80ce5c0 msgr2=0x7f59b80cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 --2- 172.21.15.31:0/603494483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80ce5c0 0x7f59b80cea20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009870 tx=0x7f59ac01ed30 comp rx=0 tx=0).stop 2023-11-26T07:17:37.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 -- 172.21.15.31:0/603494483 shutdown_connections 2023-11-26T07:17:37.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 --2- 172.21.15.31:0/603494483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80ce5c0 0x7f59b80cea20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 -- 172.21.15.31:0/603494483 >> 172.21.15.31:0/603494483 conn(0x7f59b80b5570 msgr2=0x7f59b80b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:37.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 -- 172.21.15.31:0/603494483 shutdown_connections 2023-11-26T07:17:37.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 -- 172.21.15.31:0/603494483 wait complete. 2023-11-26T07:17:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 Processor -- start 2023-11-26T07:17:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 -- start start 2023-11-26T07:17:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b806a5b0 0x7f59b814ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.033+0000 7f59bde9c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80ce5c0 0x7f59b8160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59bde9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b815d8b0 0x7f59b80cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59bde9c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8158e20 con 0x7f59b806a5b0 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59bde9c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b814b180 con 0x7f59b815d8b0 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59bde9c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8160fb0 con 0x7f59b80ce5c0 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b806a5b0 0x7f59b814ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b806a5b0 0x7f59b814ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59244/0 (socket says 172.21.15.31:59244) 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b77fe640 1 -- 172.21.15.31:0/3680973334 learned_addr learned my addr 172.21.15.31:0/3680973334 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b7fff640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b815d8b0 0x7f59b80cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b6ffd640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80ce5c0 0x7f59b8160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b77fe640 1 -- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80ce5c0 msgr2=0x7f59b8160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b77fe640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80ce5c0 0x7f59b8160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b77fe640 1 -- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b815d8b0 msgr2=0x7f59b80cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b77fe640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b815d8b0 0x7f59b80cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b77fe640 1 -- 172.21.15.31:0/3680973334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac0094d0 con 0x7f59b806a5b0 2023-11-26T07:17:37.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b6ffd640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80ce5c0 0x7f59b8160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b7fff640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b815d8b0 0x7f59b80cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59b77fe640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b806a5b0 0x7f59b814ac40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c3f0 tx=0x7f59a800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f599ffff640 1 -- 172.21.15.31:0/3680973334 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a80052d0 con 0x7f59b806a5b0 2023-11-26T07:17:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f599ffff640 1 -- 172.21.15.31:0/3680973334 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a8005430 con 0x7f59b806a5b0 2023-11-26T07:17:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59bde9c640 1 -- 172.21.15.31:0/3680973334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b81581a0 con 0x7f59b806a5b0 2023-11-26T07:17:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f599ffff640 1 -- 172.21.15.31:0/3680973334 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8003840 con 0x7f59b806a5b0 2023-11-26T07:17:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59bde9c640 1 -- 172.21.15.31:0/3680973334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b8158760 con 0x7f59b806a5b0 2023-11-26T07:17:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f59bde9c640 1 -- 172.21.15.31:0/3680973334 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b80cf6a0 con 0x7f59b806a5b0 2023-11-26T07:17:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f599ffff640 1 -- 172.21.15.31:0/3680973334 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f59a80039a0 con 0x7f59b806a5b0 2023-11-26T07:17:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.037+0000 7f599ffff640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f59a4066470 0x7f59a4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.041+0000 7f599ffff640 1 -- 172.21.15.31:0/3680973334 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59a80833c0 con 0x7f59b806a5b0 2023-11-26T07:17:37.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.041+0000 7f59b6ffd640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f59a4066470 0x7f59a4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.041+0000 7f59b6ffd640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f59a4066470 0x7f59a4068930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0b2000 tx=0x7f59ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.045+0000 7f599ffff640 1 -- 172.21.15.31:0/3680973334 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f59a8050840 con 0x7f59b806a5b0 2023-11-26T07:17:37.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.197+0000 7f59bde9c640 1 -- 172.21.15.31:0/3680973334 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59b80b52f0 con 0x7f59a4066470 2023-11-26T07:17:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.209+0000 7f599ffff640 1 -- 172.21.15.31:0/3680973334 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264614 (secure 0 0 0) 0x7f59ac0b05c0 con 0x7f59a4066470 2023-11-26T07:17:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:37.216 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:17:37.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 -- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f59a4066470 msgr2=0x7f59a4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f59a4066470 0x7f59a4068930 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0b2000 tx=0x7f59ac008040 comp rx=0 tx=0).stop 2023-11-26T07:17:37.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 -- 172.21.15.31:0/3680973334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b806a5b0 msgr2=0x7f59b814ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b806a5b0 0x7f59b814ac40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59a800c3f0 tx=0x7f59a800c8b0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 -- 172.21.15.31:0/3680973334 shutdown_connections 2023-11-26T07:17:37.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f59a4066470 0x7f59a4068930 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b815d8b0 0x7f59b80cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80ce5c0 0x7f59b8160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 --2- 172.21.15.31:0/3680973334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b806a5b0 0x7f59b814ac40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 -- 172.21.15.31:0/3680973334 >> 172.21.15.31:0/3680973334 conn(0x7f59b80b5570 msgr2=0x7f59b806d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:37.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 -- 172.21.15.31:0/3680973334 shutdown_connections 2023-11-26T07:17:37.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.213+0000 7f59bde9c640 1 -- 172.21.15.31:0/3680973334 wait complete. 2023-11-26T07:17:37.235 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-26T07:17:36.307256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.210503"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326879+0000","last_change":"2023-11-26T07:17:33.157517+0000","last_active":"2023-11-26T07:17:34.326879+0000","last_peered":"2023-11-26T07:17:34.326879+0000","last_clean":"2023-11-26T07:17:34.326879+0000","last_became_active":"2023-11-26T07:17:33.157236+0000","last_became_peered":"2023-11-26T07:17:33.157236+0000","last_unstale":"2023-11-26T07:17:34.326879+0000","last_undegraded":"2023-11-26T07:17:34.326879+0000","last_fullsized":"2023-11-26T07:17:34.326879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:50.214046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327264+0000","last_change":"2023-11-26T07:17:33.153106+0000","last_active":"2023-11-26T07:17:34.327264+0000","last_peered":"2023-11-26T07:17:34.327264+0000","last_clean":"2023-11-26T07:17:34.327264+0000","last_became_active":"2023-11-26T07:17:33.152758+0000","last_became_peered":"2023-11-26T07:17:33.152758+0000","last_unstale":"2023-11-26T07:17:34.327264+0000","last_undegraded":"2023-11-26T07:17:34.327264+0000","last_fullsized":"2023-11-26T07:17:34.327264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328087+0000","last_change":"2023-11-26T07:17:33.119394+0000","last_active":"2023-11-26T07:17:34.328087+0000","last_peered":"2023-11-26T07:17:34.328087+0000","last_clean":"2023-11-26T07:17:34.328087+0000","last_became_active":"2023-11-26T07:17:33.118798+0000","last_became_peered":"2023-11-26T07:17:33.118798+0000","last_unstale":"2023-11-26T07:17:34.328087+0000","last_undegraded":"2023-11-26T07:17:34.328087+0000","last_fullsized":"2023-11-26T07:17:34.328087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:11.113481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326896+0000","last_change":"2023-11-26T07:17:33.147341+0000","last_active":"2023-11-26T07:17:34.326896+0000","last_peered":"2023-11-26T07:17:34.326896+0000","last_clean":"2023-11-26T07:17:34.326896+0000","last_became_active":"2023-11-26T07:17:33.147066+0000","last_became_peered":"2023-11-26T07:17:33.147066+0000","last_unstale":"2023-11-26T07:17:34.326896+0000","last_undegraded":"2023-11-26T07:17:34.326896+0000","last_fullsized":"2023-11-26T07:17:34.326896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:59:35.145697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326372+0000","last_change":"2023-11-26T07:17:33.155667+0000","last_active":"2023-11-26T07:17:34.326372+0000","last_peered":"2023-11-26T07:17:34.326372+0000","last_clean":"2023-11-26T07:17:34.326372+0000","last_became_active":"2023-11-26T07:17:33.155374+0000","last_became_peered":"2023-11-26T07:17:33.155374+0000","last_unstale":"2023-11-26T07:17:34.326372+0000","last_undegraded":"2023-11-26T07:17:34.326372+0000","last_fullsized":"2023-11-26T07:17:34.326372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327032+0000","last_change":"2023-11-26T07:17:33.149613+0000","last_active":"2023-11-26T07:17:34.327032+0000","last_peered":"2023-11-26T07:17:34.327032+0000","last_clean":"2023-11-26T07:17:34.327032+0000","last_became_active":"2023-11-26T07:17:33.143662+0000","last_became_peered":"2023-11-26T07:17:33.143662+0000","last_unstale":"2023-11-26T07:17:34.327032+0000","last_undegraded":"2023-11-26T07:17:34.327032+0000","last_fullsized":"2023-11-26T07:17:34.327032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326468+0000","last_change":"2023-11-26T07:17:33.157519+0000","last_active":"2023-11-26T07:17:34.326468+0000","last_peered":"2023-11-26T07:17:34.326468+0000","last_clean":"2023-11-26T07:17:34.326468+0000","last_became_active":"2023-11-26T07:17:33.157220+0000","last_became_peered":"2023-11-26T07:17:33.157220+0000","last_unstale":"2023-11-26T07:17:34.326468+0000","last_undegraded":"2023-11-26T07:17:34.326468+0000","last_fullsized":"2023-11-26T07:17:34.326468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:15:29.617633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130474+0000","last_change":"2023-11-26T07:17:33.136788+0000","last_active":"2023-11-26T07:17:34.130474+0000","last_peered":"2023-11-26T07:17:34.130474+0000","last_clean":"2023-11-26T07:17:34.130474+0000","last_became_active":"2023-11-26T07:17:33.136185+0000","last_became_peered":"2023-11-26T07:17:33.136185+0000","last_unstale":"2023-11-26T07:17:34.130474+0000","last_undegraded":"2023-11-26T07:17:34.130474+0000","last_fullsized":"2023-11-26T07:17:34.130474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:11.113481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326962+0000","last_change":"2023-11-26T07:17:33.154122+0000","last_active":"2023-11-26T07:17:34.326962+0000","last_peered":"2023-11-26T07:17:34.326962+0000","last_clean":"2023-11-26T07:17:34.326962+0000","last_became_active":"2023-11-26T07:17:33.153841+0000","last_became_peered":"2023-11-26T07:17:33.153841+0000","last_unstale":"2023-11-26T07:17:34.326962+0000","last_undegraded":"2023-11-26T07:17:34.326962+0000","last_fullsized":"2023-11-26T07:17:34.326962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:05.773940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128871+0000","last_change":"2023-11-26T07:17:33.134966+0000","last_active":"2023-11-26T07:17:34.128871+0000","last_peered":"2023-11-26T07:17:34.128871+0000","last_clean":"2023-11-26T07:17:34.128871+0000","last_became_active":"2023-11-26T07:17:33.134672+0000","last_became_peered":"2023-11-26T07:17:33.134672+0000","last_unstale":"2023-11-26T07:17:34.128871+0000","last_undegraded":"2023-11-26T07:17:34.128871+0000","last_fullsized":"2023-11-26T07:17:34.128871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128543+0000","last_change":"2023-11-26T07:17:33.134943+0000","last_active":"2023-11-26T07:17:34.128543+0000","last_peered":"2023-11-26T07:17:34.128543+0000","last_clean":"2023-11-26T07:17:34.128543+0000","last_became_active":"2023-11-26T07:17:33.134570+0000","last_became_peered":"2023-11-26T07:17:33.134570+0000","last_unstale":"2023-11-26T07:17:34.128543+0000","last_undegraded":"2023-11-26T07:17:34.128543+0000","last_fullsized":"2023-11-26T07:17:34.128543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128407+0000","last_change":"2023-11-26T07:17:33.136068+0000","last_active":"2023-11-26T07:17:34.128407+0000","last_peered":"2023-11-26T07:17:34.128407+0000","last_clean":"2023-11-26T07:17:34.128407+0000","last_became_active":"2023-11-26T07:17:33.135783+0000","last_became_peered":"2023-11-26T07:17:33.135783+0000","last_unstale":"2023-11-26T07:17:34.128407+0000","last_undegraded":"2023-11-26T07:17:34.128407+0000","last_fullsized":"2023-11-26T07:17:34.128407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:30.841394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130800+0000","last_change":"2023-11-26T07:17:33.145916+0000","last_active":"2023-11-26T07:17:34.130800+0000","last_peered":"2023-11-26T07:17:34.130800+0000","last_clean":"2023-11-26T07:17:34.130800+0000","last_became_active":"2023-11-26T07:17:33.141115+0000","last_became_peered":"2023-11-26T07:17:33.141115+0000","last_unstale":"2023-11-26T07:17:34.130800+0000","last_undegraded":"2023-11-26T07:17:34.130800+0000","last_fullsized":"2023-11-26T07:17:34.130800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327928+0000","last_change":"2023-11-26T07:17:33.153356+0000","last_active":"2023-11-26T07:17:34.327928+0000","last_peered":"2023-11-26T07:17:34.327928+0000","last_clean":"2023-11-26T07:17:34.327928+0000","last_became_active":"2023-11-26T07:17:33.153016+0000","last_became_peered":"2023-11-26T07:17:33.153016+0000","last_unstale":"2023-11-26T07:17:34.327928+0000","last_undegraded":"2023-11-26T07:17:34.327928+0000","last_fullsized":"2023-11-26T07:17:34.327928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:27.264806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326905+0000","last_change":"2023-11-26T07:17:33.138021+0000","last_active":"2023-11-26T07:17:34.326905+0000","last_peered":"2023-11-26T07:17:34.326905+0000","last_clean":"2023-11-26T07:17:34.326905+0000","last_became_active":"2023-11-26T07:17:33.137694+0000","last_became_peered":"2023-11-26T07:17:33.137694+0000","last_unstale":"2023-11-26T07:17:34.326905+0000","last_undegraded":"2023-11-26T07:17:34.326905+0000","last_fullsized":"2023-11-26T07:17:34.326905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.325959+0000","last_change":"2023-11-26T07:17:33.125499+0000","last_active":"2023-11-26T07:17:34.325959+0000","last_peered":"2023-11-26T07:17:34.325959+0000","last_clean":"2023-11-26T07:17:34.325959+0000","last_became_active":"2023-11-26T07:17:33.125132+0000","last_became_peered":"2023-11-26T07:17:33.125132+0000","last_unstale":"2023-11-26T07:17:34.325959+0000","last_undegraded":"2023-11-26T07:17:34.325959+0000","last_fullsized":"2023-11-26T07:17:34.325959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326610+0000","last_change":"2023-11-26T07:17:33.147715+0000","last_active":"2023-11-26T07:17:34.326610+0000","last_peered":"2023-11-26T07:17:34.326610+0000","last_clean":"2023-11-26T07:17:34.326610+0000","last_became_active":"2023-11-26T07:17:33.146632+0000","last_became_peered":"2023-11-26T07:17:33.146632+0000","last_unstale":"2023-11-26T07:17:34.326610+0000","last_undegraded":"2023-11-26T07:17:34.326610+0000","last_fullsized":"2023-11-26T07:17:34.326610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:54.079736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326760+0000","last_change":"2023-11-26T07:17:33.141795+0000","last_active":"2023-11-26T07:17:34.326760+0000","last_peered":"2023-11-26T07:17:34.326760+0000","last_clean":"2023-11-26T07:17:34.326760+0000","last_became_active":"2023-11-26T07:17:33.141529+0000","last_became_peered":"2023-11-26T07:17:33.141529+0000","last_unstale":"2023-11-26T07:17:34.326760+0000","last_undegraded":"2023-11-26T07:17:34.326760+0000","last_fullsized":"2023-11-26T07:17:34.326760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:35.443597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327208+0000","last_change":"2023-11-26T07:17:33.156644+0000","last_active":"2023-11-26T07:17:34.327208+0000","last_peered":"2023-11-26T07:17:34.327208+0000","last_clean":"2023-11-26T07:17:34.327208+0000","last_became_active":"2023-11-26T07:17:33.156430+0000","last_became_peered":"2023-11-26T07:17:33.156430+0000","last_unstale":"2023-11-26T07:17:34.327208+0000","last_undegraded":"2023-11-26T07:17:34.327208+0000","last_fullsized":"2023-11-26T07:17:34.327208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:54.433587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127997+0000","last_change":"2023-11-26T07:17:33.133505+0000","last_active":"2023-11-26T07:17:34.127997+0000","last_peered":"2023-11-26T07:17:34.127997+0000","last_clean":"2023-11-26T07:17:34.127997+0000","last_became_active":"2023-11-26T07:17:33.133134+0000","last_became_peered":"2023-11-26T07:17:33.133134+0000","last_unstale":"2023-11-26T07:17:34.127997+0000","last_undegraded":"2023-11-26T07:17:34.127997+0000","last_fullsized":"2023-11-26T07:17:34.127997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328056+0000","last_change":"2023-11-26T07:17:33.144471+0000","last_active":"2023-11-26T07:17:34.328056+0000","last_peered":"2023-11-26T07:17:34.328056+0000","last_clean":"2023-11-26T07:17:34.328056+0000","last_became_active":"2023-11-26T07:17:33.144183+0000","last_became_peered":"2023-11-26T07:17:33.144183+0000","last_unstale":"2023-11-26T07:17:34.328056+0000","last_undegraded":"2023-11-26T07:17:34.328056+0000","last_fullsized":"2023-11-26T07:17:34.328056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:52.817990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327032+0000","last_change":"2023-11-26T07:17:33.149400+0000","last_active":"2023-11-26T07:17:34.327032+0000","last_peered":"2023-11-26T07:17:34.327032+0000","last_clean":"2023-11-26T07:17:34.327032+0000","last_became_active":"2023-11-26T07:17:33.144991+0000","last_became_peered":"2023-11-26T07:17:33.144991+0000","last_unstale":"2023-11-26T07:17:34.327032+0000","last_undegraded":"2023-11-26T07:17:34.327032+0000","last_fullsized":"2023-11-26T07:17:34.327032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:15:29.617633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130588+0000","last_change":"2023-11-26T07:17:33.125477+0000","last_active":"2023-11-26T07:17:34.130588+0000","last_peered":"2023-11-26T07:17:34.130588+0000","last_clean":"2023-11-26T07:17:34.130588+0000","last_became_active":"2023-11-26T07:17:33.125109+0000","last_became_peered":"2023-11-26T07:17:33.125109+0000","last_unstale":"2023-11-26T07:17:34.130588+0000","last_undegraded":"2023-11-26T07:17:34.130588+0000","last_fullsized":"2023-11-26T07:17:34.130588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326026+0000","last_change":"2023-11-26T07:17:33.138762+0000","last_active":"2023-11-26T07:17:34.326026+0000","last_peered":"2023-11-26T07:17:34.326026+0000","last_clean":"2023-11-26T07:17:34.326026+0000","last_became_active":"2023-11-26T07:17:33.138466+0000","last_became_peered":"2023-11-26T07:17:33.138466+0000","last_unstale":"2023-11-26T07:17:34.326026+0000","last_undegraded":"2023-11-26T07:17:34.326026+0000","last_fullsized":"2023-11-26T07:17:34.326026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130957+0000","last_change":"2023-11-26T07:17:33.128236+0000","last_active":"2023-11-26T07:17:34.130957+0000","last_peered":"2023-11-26T07:17:34.130957+0000","last_clean":"2023-11-26T07:17:34.130957+0000","last_became_active":"2023-11-26T07:17:33.127900+0000","last_became_peered":"2023-11-26T07:17:33.127900+0000","last_unstale":"2023-11-26T07:17:34.130957+0000","last_undegraded":"2023-11-26T07:17:34.130957+0000","last_fullsized":"2023-11-26T07:17:34.130957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128993+0000","last_change":"2023-11-26T07:17:33.144760+0000","last_active":"2023-11-26T07:17:34.128993+0000","last_peered":"2023-11-26T07:17:34.128993+0000","last_clean":"2023-11-26T07:17:34.128993+0000","last_became_active":"2023-11-26T07:17:33.144272+0000","last_became_peered":"2023-11-26T07:17:33.144272+0000","last_unstale":"2023-11-26T07:17:34.128993+0000","last_undegraded":"2023-11-26T07:17:34.128993+0000","last_fullsized":"2023-11-26T07:17:34.128993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327140+0000","last_change":"2023-11-26T07:17:33.151793+0000","last_active":"2023-11-26T07:17:34.327140+0000","last_peered":"2023-11-26T07:17:34.327140+0000","last_clean":"2023-11-26T07:17:34.327140+0000","last_became_active":"2023-11-26T07:17:33.151517+0000","last_became_peered":"2023-11-26T07:17:33.151517+0000","last_unstale":"2023-11-26T07:17:34.327140+0000","last_undegraded":"2023-11-26T07:17:34.327140+0000","last_fullsized":"2023-11-26T07:17:34.327140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131015+0000","last_change":"2023-11-26T07:17:33.128368+0000","last_active":"2023-11-26T07:17:34.131015+0000","last_peered":"2023-11-26T07:17:34.131015+0000","last_clean":"2023-11-26T07:17:34.131015+0000","last_became_active":"2023-11-26T07:17:33.128068+0000","last_became_peered":"2023-11-26T07:17:33.128068+0000","last_unstale":"2023-11-26T07:17:34.131015+0000","last_undegraded":"2023-11-26T07:17:34.131015+0000","last_fullsized":"2023-11-26T07:17:34.131015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:54.079736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328159+0000","last_change":"2023-11-26T07:17:33.124145+0000","last_active":"2023-11-26T07:17:34.328159+0000","last_peered":"2023-11-26T07:17:34.328159+0000","last_clean":"2023-11-26T07:17:34.328159+0000","last_became_active":"2023-11-26T07:17:33.123870+0000","last_became_peered":"2023-11-26T07:17:33.123870+0000","last_unstale":"2023-11-26T07:17:34.328159+0000","last_undegraded":"2023-11-26T07:17:34.328159+0000","last_fullsized":"2023-11-26T07:17:34.328159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327220+0000","last_change":"2023-11-26T07:17:33.138523+0000","last_active":"2023-11-26T07:17:34.327220+0000","last_peered":"2023-11-26T07:17:34.327220+0000","last_clean":"2023-11-26T07:17:34.327220+0000","last_became_active":"2023-11-26T07:17:33.138213+0000","last_became_peered":"2023-11-26T07:17:33.138213+0000","last_unstale":"2023-11-26T07:17:34.327220+0000","last_undegraded":"2023-11-26T07:17:34.327220+0000","last_fullsized":"2023-11-26T07:17:34.327220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:12.772210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326933+0000","last_change":"2023-11-26T07:17:33.128197+0000","last_active":"2023-11-26T07:17:34.326933+0000","last_peered":"2023-11-26T07:17:34.326933+0000","last_clean":"2023-11-26T07:17:34.326933+0000","last_became_active":"2023-11-26T07:17:33.127911+0000","last_became_peered":"2023-11-26T07:17:33.127911+0000","last_unstale":"2023-11-26T07:17:34.326933+0000","last_undegraded":"2023-11-26T07:17:34.326933+0000","last_fullsized":"2023-11-26T07:17:34.326933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:17.558267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127456+0000","last_change":"2023-11-26T07:17:33.140214+0000","last_active":"2023-11-26T07:17:34.127456+0000","last_peered":"2023-11-26T07:17:34.127456+0000","last_clean":"2023-11-26T07:17:34.127456+0000","last_became_active":"2023-11-26T07:17:33.139927+0000","last_became_peered":"2023-11-26T07:17:33.139927+0000","last_unstale":"2023-11-26T07:17:34.127456+0000","last_undegraded":"2023-11-26T07:17:34.127456+0000","last_fullsized":"2023-11-26T07:17:34.127456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128003+0000","last_change":"2023-11-26T07:17:33.151273+0000","last_active":"2023-11-26T07:17:34.128003+0000","last_peered":"2023-11-26T07:17:34.128003+0000","last_clean":"2023-11-26T07:17:34.128003+0000","last_became_active":"2023-11-26T07:17:33.150182+0000","last_became_peered":"2023-11-26T07:17:33.150182+0000","last_unstale":"2023-11-26T07:17:34.128003+0000","last_undegraded":"2023-11-26T07:17:34.128003+0000","last_fullsized":"2023-11-26T07:17:34.128003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:02.280784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131100+0000","last_change":"2023-11-26T07:17:32.135703+0000","last_active":"2023-11-26T07:17:34.131100+0000","last_peered":"2023-11-26T07:17:34.131100+0000","last_clean":"2023-11-26T07:17:34.131100+0000","last_became_active":"2023-11-26T07:17:32.135392+0000","last_became_peered":"2023-11-26T07:17:32.135392+0000","last_unstale":"2023-11-26T07:17:34.131100+0000","last_undegraded":"2023-11-26T07:17:34.131100+0000","last_fullsized":"2023-11-26T07:17:34.131100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328230+0000","last_change":"2023-11-26T07:17:33.123539+0000","last_active":"2023-11-26T07:17:34.328230+0000","last_peered":"2023-11-26T07:17:34.328230+0000","last_clean":"2023-11-26T07:17:34.328230+0000","last_became_active":"2023-11-26T07:17:33.123240+0000","last_became_peered":"2023-11-26T07:17:33.123240+0000","last_unstale":"2023-11-26T07:17:34.328230+0000","last_undegraded":"2023-11-26T07:17:34.328230+0000","last_fullsized":"2023-11-26T07:17:34.328230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127853+0000","last_change":"2023-11-26T07:17:32.124304+0000","last_active":"2023-11-26T07:17:34.127853+0000","last_peered":"2023-11-26T07:17:34.127853+0000","last_clean":"2023-11-26T07:17:34.127853+0000","last_became_active":"2023-11-26T07:17:32.122374+0000","last_became_peered":"2023-11-26T07:17:32.122374+0000","last_unstale":"2023-11-26T07:17:34.127853+0000","last_undegraded":"2023-11-26T07:17:34.127853+0000","last_fullsized":"2023-11-26T07:17:34.127853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:53:41.643510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130553+0000","last_change":"2023-11-26T07:17:32.132756+0000","last_active":"2023-11-26T07:17:34.130553+0000","last_peered":"2023-11-26T07:17:34.130553+0000","last_clean":"2023-11-26T07:17:34.130553+0000","last_became_active":"2023-11-26T07:17:32.132329+0000","last_became_peered":"2023-11-26T07:17:32.132329+0000","last_unstale":"2023-11-26T07:17:34.130553+0000","last_undegraded":"2023-11-26T07:17:34.130553+0000","last_fullsized":"2023-11-26T07:17:34.130553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130543+0000","last_change":"2023-11-26T07:17:33.124808+0000","last_active":"2023-11-26T07:17:34.130543+0000","last_peered":"2023-11-26T07:17:34.130543+0000","last_clean":"2023-11-26T07:17:34.130543+0000","last_became_active":"2023-11-26T07:17:33.124508+0000","last_became_peered":"2023-11-26T07:17:33.124508+0000","last_unstale":"2023-11-26T07:17:34.130543+0000","last_undegraded":"2023-11-26T07:17:34.130543+0000","last_fullsized":"2023-11-26T07:17:34.130543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326062+0000","last_change":"2023-11-26T07:17:32.136207+0000","last_active":"2023-11-26T07:17:34.326062+0000","last_peered":"2023-11-26T07:17:34.326062+0000","last_clean":"2023-11-26T07:17:34.326062+0000","last_became_active":"2023-11-26T07:17:32.135850+0000","last_became_peered":"2023-11-26T07:17:32.135850+0000","last_unstale":"2023-11-26T07:17:34.326062+0000","last_undegraded":"2023-11-26T07:17:34.326062+0000","last_fullsized":"2023-11-26T07:17:34.326062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:48.178664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326761+0000","last_change":"2023-11-26T07:17:33.153245+0000","last_active":"2023-11-26T07:17:34.326761+0000","last_peered":"2023-11-26T07:17:34.326761+0000","last_clean":"2023-11-26T07:17:34.326761+0000","last_became_active":"2023-11-26T07:17:33.151566+0000","last_became_peered":"2023-11-26T07:17:33.151566+0000","last_unstale":"2023-11-26T07:17:34.326761+0000","last_undegraded":"2023-11-26T07:17:34.326761+0000","last_fullsized":"2023-11-26T07:17:34.326761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.129164+0000","last_change":"2023-11-26T07:17:32.126580+0000","last_active":"2023-11-26T07:17:34.129164+0000","last_peered":"2023-11-26T07:17:34.129164+0000","last_clean":"2023-11-26T07:17:34.129164+0000","last_became_active":"2023-11-26T07:17:32.125693+0000","last_became_peered":"2023-11-26T07:17:32.125693+0000","last_unstale":"2023-11-26T07:17:34.129164+0000","last_undegraded":"2023-11-26T07:17:34.129164+0000","last_fullsized":"2023-11-26T07:17:34.129164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327410+0000","last_change":"2023-11-26T07:17:33.145310+0000","last_active":"2023-11-26T07:17:34.327410+0000","last_peered":"2023-11-26T07:17:34.327410+0000","last_clean":"2023-11-26T07:17:34.327410+0000","last_became_active":"2023-11-26T07:17:33.145010+0000","last_became_peered":"2023-11-26T07:17:33.145010+0000","last_unstale":"2023-11-26T07:17:34.327410+0000","last_undegraded":"2023-11-26T07:17:34.327410+0000","last_fullsized":"2023-11-26T07:17:34.327410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:23:10.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327442+0000","last_change":"2023-11-26T07:17:32.130729+0000","last_active":"2023-11-26T07:17:34.327442+0000","last_peered":"2023-11-26T07:17:34.327442+0000","last_clean":"2023-11-26T07:17:34.327442+0000","last_became_active":"2023-11-26T07:17:32.130438+0000","last_became_peered":"2023-11-26T07:17:32.130438+0000","last_unstale":"2023-11-26T07:17:34.327442+0000","last_undegraded":"2023-11-26T07:17:34.327442+0000","last_fullsized":"2023-11-26T07:17:34.327442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:24.546723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328264+0000","last_change":"2023-11-26T07:17:33.136875+0000","last_active":"2023-11-26T07:17:34.328264+0000","last_peered":"2023-11-26T07:17:34.328264+0000","last_clean":"2023-11-26T07:17:34.328264+0000","last_became_active":"2023-11-26T07:17:33.136544+0000","last_became_peered":"2023-11-26T07:17:33.136544+0000","last_unstale":"2023-11-26T07:17:34.328264+0000","last_undegraded":"2023-11-26T07:17:34.328264+0000","last_fullsized":"2023-11-26T07:17:34.328264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327150+0000","last_change":"2023-11-26T07:17:32.127762+0000","last_active":"2023-11-26T07:17:34.327150+0000","last_peered":"2023-11-26T07:17:34.327150+0000","last_clean":"2023-11-26T07:17:34.327150+0000","last_became_active":"2023-11-26T07:17:32.127380+0000","last_became_peered":"2023-11-26T07:17:32.127380+0000","last_unstale":"2023-11-26T07:17:34.327150+0000","last_undegraded":"2023-11-26T07:17:34.327150+0000","last_fullsized":"2023-11-26T07:17:34.327150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128525+0000","last_change":"2023-11-26T07:17:33.147339+0000","last_active":"2023-11-26T07:17:34.128525+0000","last_peered":"2023-11-26T07:17:34.128525+0000","last_clean":"2023-11-26T07:17:34.128525+0000","last_became_active":"2023-11-26T07:17:33.147047+0000","last_became_peered":"2023-11-26T07:17:33.147047+0000","last_unstale":"2023-11-26T07:17:34.128525+0000","last_undegraded":"2023-11-26T07:17:34.128525+0000","last_fullsized":"2023-11-26T07:17:34.128525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:35.443597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128083+0000","last_change":"2023-11-26T07:17:32.133337+0000","last_active":"2023-11-26T07:17:34.128083+0000","last_peered":"2023-11-26T07:17:34.128083+0000","last_clean":"2023-11-26T07:17:34.128083+0000","last_became_active":"2023-11-26T07:17:32.132169+0000","last_became_peered":"2023-11-26T07:17:32.132169+0000","last_unstale":"2023-11-26T07:17:34.128083+0000","last_undegraded":"2023-11-26T07:17:34.128083+0000","last_fullsized":"2023-11-26T07:17:34.128083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:21.575382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326848+0000","last_change":"2023-11-26T07:17:33.150427+0000","last_active":"2023-11-26T07:17:34.326848+0000","last_peered":"2023-11-26T07:17:34.326848+0000","last_clean":"2023-11-26T07:17:34.326848+0000","last_became_active":"2023-11-26T07:17:33.150029+0000","last_became_peered":"2023-11-26T07:17:33.150029+0000","last_unstale":"2023-11-26T07:17:34.326848+0000","last_undegraded":"2023-11-26T07:17:34.326848+0000","last_fullsized":"2023-11-26T07:17:34.326848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:23:10.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328227+0000","last_change":"2023-11-26T07:17:32.133555+0000","last_active":"2023-11-26T07:17:34.328227+0000","last_peered":"2023-11-26T07:17:34.328227+0000","last_clean":"2023-11-26T07:17:34.328227+0000","last_became_active":"2023-11-26T07:17:32.132862+0000","last_became_peered":"2023-11-26T07:17:32.132862+0000","last_unstale":"2023-11-26T07:17:34.328227+0000","last_undegraded":"2023-11-26T07:17:34.328227+0000","last_fullsized":"2023-11-26T07:17:34.328227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:19.749525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328005+0000","last_change":"2023-11-26T07:17:33.144778+0000","last_active":"2023-11-26T07:17:34.328005+0000","last_peered":"2023-11-26T07:17:34.328005+0000","last_clean":"2023-11-26T07:17:34.328005+0000","last_became_active":"2023-11-26T07:17:33.144469+0000","last_became_peered":"2023-11-26T07:17:33.144469+0000","last_unstale":"2023-11-26T07:17:34.328005+0000","last_undegraded":"2023-11-26T07:17:34.328005+0000","last_fullsized":"2023-11-26T07:17:34.328005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:10.505099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327149+0000","last_change":"2023-11-26T07:17:32.140716+0000","last_active":"2023-11-26T07:17:34.327149+0000","last_peered":"2023-11-26T07:17:34.327149+0000","last_clean":"2023-11-26T07:17:34.327149+0000","last_became_active":"2023-11-26T07:17:32.139344+0000","last_became_peered":"2023-11-26T07:17:32.139344+0000","last_unstale":"2023-11-26T07:17:34.327149+0000","last_undegraded":"2023-11-26T07:17:34.327149+0000","last_fullsized":"2023-11-26T07:17:34.327149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326526+0000","last_change":"2023-11-26T07:17:33.152939+0000","last_active":"2023-11-26T07:17:34.326526+0000","last_peered":"2023-11-26T07:17:34.326526+0000","last_clean":"2023-11-26T07:17:34.326526+0000","last_became_active":"2023-11-26T07:17:33.152626+0000","last_became_peered":"2023-11-26T07:17:33.152626+0000","last_unstale":"2023-11-26T07:17:34.326526+0000","last_undegraded":"2023-11-26T07:17:34.326526+0000","last_fullsized":"2023-11-26T07:17:34.326526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:13.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127755+0000","last_change":"2023-11-26T07:17:32.127951+0000","last_active":"2023-11-26T07:17:34.127755+0000","last_peered":"2023-11-26T07:17:34.127755+0000","last_clean":"2023-11-26T07:17:34.127755+0000","last_became_active":"2023-11-26T07:17:32.127195+0000","last_became_peered":"2023-11-26T07:17:32.127195+0000","last_unstale":"2023-11-26T07:17:34.127755+0000","last_undegraded":"2023-11-26T07:17:34.127755+0000","last_fullsized":"2023-11-26T07:17:34.127755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.551435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127675+0000","last_change":"2023-11-26T07:17:33.132518+0000","last_active":"2023-11-26T07:17:34.127675+0000","last_peered":"2023-11-26T07:17:34.127675+0000","last_clean":"2023-11-26T07:17:34.127675+0000","last_became_active":"2023-11-26T07:17:33.131905+0000","last_became_peered":"2023-11-26T07:17:33.131905+0000","last_unstale":"2023-11-26T07:17:34.127675+0000","last_undegraded":"2023-11-26T07:17:34.127675+0000","last_fullsized":"2023-11-26T07:17:34.127675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326584+0000","last_change":"2023-11-26T07:17:32.129880+0000","last_active":"2023-11-26T07:17:34.326584+0000","last_peered":"2023-11-26T07:17:34.326584+0000","last_clean":"2023-11-26T07:17:34.326584+0000","last_became_active":"2023-11-26T07:17:32.129575+0000","last_became_peered":"2023-11-26T07:17:32.129575+0000","last_unstale":"2023-11-26T07:17:34.326584+0000","last_undegraded":"2023-11-26T07:17:34.326584+0000","last_fullsized":"2023-11-26T07:17:34.326584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:27.401076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326605+0000","last_change":"2023-11-26T07:17:33.152780+0000","last_active":"2023-11-26T07:17:34.326605+0000","last_peered":"2023-11-26T07:17:34.326605+0000","last_clean":"2023-11-26T07:17:34.326605+0000","last_became_active":"2023-11-26T07:17:33.152468+0000","last_became_peered":"2023-11-26T07:17:33.152468+0000","last_unstale":"2023-11-26T07:17:34.326605+0000","last_undegraded":"2023-11-26T07:17:34.326605+0000","last_fullsized":"2023-11-26T07:17:34.326605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:56:25.969462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128485+0000","last_change":"2023-11-26T07:17:32.132211+0000","last_active":"2023-11-26T07:17:34.128485+0000","last_peered":"2023-11-26T07:17:34.128485+0000","last_clean":"2023-11-26T07:17:34.128485+0000","last_became_active":"2023-11-26T07:17:32.131233+0000","last_became_peered":"2023-11-26T07:17:32.131233+0000","last_unstale":"2023-11-26T07:17:34.128485+0000","last_undegraded":"2023-11-26T07:17:34.128485+0000","last_fullsized":"2023-11-26T07:17:34.128485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:21.662285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326787+0000","last_change":"2023-11-26T07:17:33.154639+0000","last_active":"2023-11-26T07:17:34.326787+0000","last_peered":"2023-11-26T07:17:34.326787+0000","last_clean":"2023-11-26T07:17:34.326787+0000","last_became_active":"2023-11-26T07:17:33.153245+0000","last_became_peered":"2023-11-26T07:17:33.153245+0000","last_unstale":"2023-11-26T07:17:34.326787+0000","last_undegraded":"2023-11-26T07:17:34.326787+0000","last_fullsized":"2023-11-26T07:17:34.326787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:12.772210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326854+0000","last_change":"2023-11-26T07:17:32.138855+0000","last_active":"2023-11-26T07:17:34.326854+0000","last_peered":"2023-11-26T07:17:34.326854+0000","last_clean":"2023-11-26T07:17:34.326854+0000","last_became_active":"2023-11-26T07:17:32.138566+0000","last_became_peered":"2023-11-26T07:17:32.138566+0000","last_unstale":"2023-11-26T07:17:34.326854+0000","last_undegraded":"2023-11-26T07:17:34.326854+0000","last_fullsized":"2023-11-26T07:17:34.326854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:29.834563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327040+0000","last_change":"2023-11-26T07:17:33.152760+0000","last_active":"2023-11-26T07:17:34.327040+0000","last_peered":"2023-11-26T07:17:34.327040+0000","last_clean":"2023-11-26T07:17:34.327040+0000","last_became_active":"2023-11-26T07:17:33.152455+0000","last_became_peered":"2023-11-26T07:17:33.152455+0000","last_unstale":"2023-11-26T07:17:34.327040+0000","last_undegraded":"2023-11-26T07:17:34.327040+0000","last_fullsized":"2023-11-26T07:17:34.327040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:40.786432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327244+0000","last_change":"2023-11-26T07:17:32.132634+0000","last_active":"2023-11-26T07:17:34.327244+0000","last_peered":"2023-11-26T07:17:34.327244+0000","last_clean":"2023-11-26T07:17:34.327244+0000","last_became_active":"2023-11-26T07:17:32.132192+0000","last_became_peered":"2023-11-26T07:17:32.132192+0000","last_unstale":"2023-11-26T07:17:34.327244+0000","last_undegraded":"2023-11-26T07:17:34.327244+0000","last_fullsized":"2023-11-26T07:17:34.327244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:54:55.533333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327181+0000","last_change":"2023-11-26T07:17:33.147935+0000","last_active":"2023-11-26T07:17:34.327181+0000","last_peered":"2023-11-26T07:17:34.327181+0000","last_clean":"2023-11-26T07:17:34.327181+0000","last_became_active":"2023-11-26T07:17:33.147595+0000","last_became_peered":"2023-11-26T07:17:33.147595+0000","last_unstale":"2023-11-26T07:17:34.327181+0000","last_undegraded":"2023-11-26T07:17:34.327181+0000","last_fullsized":"2023-11-26T07:17:34.327181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:56:25.969462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326702+0000","last_change":"2023-11-26T07:17:32.128101+0000","last_active":"2023-11-26T07:17:34.326702+0000","last_peered":"2023-11-26T07:17:34.326702+0000","last_clean":"2023-11-26T07:17:34.326702+0000","last_became_active":"2023-11-26T07:17:32.127821+0000","last_became_peered":"2023-11-26T07:17:32.127821+0000","last_unstale":"2023-11-26T07:17:34.326702+0000","last_undegraded":"2023-11-26T07:17:34.326702+0000","last_fullsized":"2023-11-26T07:17:34.326702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:30:24.091629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326978+0000","last_change":"2023-11-26T07:17:33.153014+0000","last_active":"2023-11-26T07:17:34.326978+0000","last_peered":"2023-11-26T07:17:34.326978+0000","last_clean":"2023-11-26T07:17:34.326978+0000","last_became_active":"2023-11-26T07:17:33.152675+0000","last_became_peered":"2023-11-26T07:17:33.152675+0000","last_unstale":"2023-11-26T07:17:34.326978+0000","last_undegraded":"2023-11-26T07:17:34.326978+0000","last_fullsized":"2023-11-26T07:17:34.326978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:06.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-26T07:17:33.140703+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:17:33.140703+0000","last_peered":"2023-11-26T07:17:33.140703+0000","last_clean":"2023-11-26T07:17:33.140703+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:17:33.140703+0000","last_undegraded":"2023-11-26T07:17:33.140703+0000","last_fullsized":"2023-11-26T07:17:33.140703+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.325979+0000","last_change":"2023-11-26T07:17:32.127453+0000","last_active":"2023-11-26T07:17:34.325979+0000","last_peered":"2023-11-26T07:17:34.325979+0000","last_clean":"2023-11-26T07:17:34.325979+0000","last_became_active":"2023-11-26T07:17:32.127171+0000","last_became_peered":"2023-11-26T07:17:32.127171+0000","last_unstale":"2023-11-26T07:17:34.325979+0000","last_undegraded":"2023-11-26T07:17:34.325979+0000","last_fullsized":"2023-11-26T07:17:34.325979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:10.106650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130743+0000","last_change":"2023-11-26T07:17:33.132790+0000","last_active":"2023-11-26T07:17:34.130743+0000","last_peered":"2023-11-26T07:17:34.130743+0000","last_clean":"2023-11-26T07:17:34.130743+0000","last_became_active":"2023-11-26T07:17:33.132415+0000","last_became_peered":"2023-11-26T07:17:33.132415+0000","last_unstale":"2023-11-26T07:17:34.130743+0000","last_undegraded":"2023-11-26T07:17:34.130743+0000","last_fullsized":"2023-11-26T07:17:34.130743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326218+0000","last_change":"2023-11-26T07:17:32.134663+0000","last_active":"2023-11-26T07:17:34.326218+0000","last_peered":"2023-11-26T07:17:34.326218+0000","last_clean":"2023-11-26T07:17:34.326218+0000","last_became_active":"2023-11-26T07:17:32.134372+0000","last_became_peered":"2023-11-26T07:17:32.134372+0000","last_unstale":"2023-11-26T07:17:34.326218+0000","last_undegraded":"2023-11-26T07:17:34.326218+0000","last_fullsized":"2023-11-26T07:17:34.326218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:34.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327893+0000","last_change":"2023-11-26T07:17:33.131023+0000","last_active":"2023-11-26T07:17:34.327893+0000","last_peered":"2023-11-26T07:17:34.327893+0000","last_clean":"2023-11-26T07:17:34.327893+0000","last_became_active":"2023-11-26T07:17:33.130665+0000","last_became_peered":"2023-11-26T07:17:33.130665+0000","last_unstale":"2023-11-26T07:17:34.327893+0000","last_undegraded":"2023-11-26T07:17:34.327893+0000","last_fullsized":"2023-11-26T07:17:34.327893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:54:56.540164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327009+0000","last_change":"2023-11-26T07:17:32.141091+0000","last_active":"2023-11-26T07:17:34.327009+0000","last_peered":"2023-11-26T07:17:34.327009+0000","last_clean":"2023-11-26T07:17:34.327009+0000","last_became_active":"2023-11-26T07:17:32.139978+0000","last_became_peered":"2023-11-26T07:17:32.139978+0000","last_unstale":"2023-11-26T07:17:34.327009+0000","last_undegraded":"2023-11-26T07:17:34.327009+0000","last_fullsized":"2023-11-26T07:17:34.327009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:01.273952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326756+0000","last_change":"2023-11-26T07:17:33.137737+0000","last_active":"2023-11-26T07:17:34.326756+0000","last_peered":"2023-11-26T07:17:34.326756+0000","last_clean":"2023-11-26T07:17:34.326756+0000","last_became_active":"2023-11-26T07:17:33.137350+0000","last_became_peered":"2023-11-26T07:17:33.137350+0000","last_unstale":"2023-11-26T07:17:34.326756+0000","last_undegraded":"2023-11-26T07:17:34.326756+0000","last_fullsized":"2023-11-26T07:17:34.326756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327251+0000","last_change":"2023-11-26T07:17:32.134855+0000","last_active":"2023-11-26T07:17:34.327251+0000","last_peered":"2023-11-26T07:17:34.327251+0000","last_clean":"2023-11-26T07:17:34.327251+0000","last_became_active":"2023-11-26T07:17:32.134587+0000","last_became_peered":"2023-11-26T07:17:32.134587+0000","last_unstale":"2023-11-26T07:17:34.327251+0000","last_undegraded":"2023-11-26T07:17:34.327251+0000","last_fullsized":"2023-11-26T07:17:34.327251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128590+0000","last_change":"2023-11-26T07:17:33.122483+0000","last_active":"2023-11-26T07:17:34.128590+0000","last_peered":"2023-11-26T07:17:34.128590+0000","last_clean":"2023-11-26T07:17:34.128590+0000","last_became_active":"2023-11-26T07:17:33.122042+0000","last_became_peered":"2023-11-26T07:17:33.122042+0000","last_unstale":"2023-11-26T07:17:34.128590+0000","last_undegraded":"2023-11-26T07:17:34.128590+0000","last_fullsized":"2023-11-26T07:17:34.128590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:17.558267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327027+0000","last_change":"2023-11-26T07:17:32.124516+0000","last_active":"2023-11-26T07:17:34.327027+0000","last_peered":"2023-11-26T07:17:34.327027+0000","last_clean":"2023-11-26T07:17:34.327027+0000","last_became_active":"2023-11-26T07:17:32.123978+0000","last_became_peered":"2023-11-26T07:17:32.123978+0000","last_unstale":"2023-11-26T07:17:34.327027+0000","last_undegraded":"2023-11-26T07:17:34.327027+0000","last_fullsized":"2023-11-26T07:17:34.327027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:34.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326612+0000","last_change":"2023-11-26T07:17:33.139055+0000","last_active":"2023-11-26T07:17:34.326612+0000","last_peered":"2023-11-26T07:17:34.326612+0000","last_clean":"2023-11-26T07:17:34.326612+0000","last_became_active":"2023-11-26T07:17:33.138235+0000","last_became_peered":"2023-11-26T07:17:33.138235+0000","last_unstale":"2023-11-26T07:17:34.326612+0000","last_undegraded":"2023-11-26T07:17:34.326612+0000","last_fullsized":"2023-11-26T07:17:34.326612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327313+0000","last_change":"2023-11-26T07:17:32.135542+0000","last_active":"2023-11-26T07:17:34.327313+0000","last_peered":"2023-11-26T07:17:34.327313+0000","last_clean":"2023-11-26T07:17:34.327313+0000","last_became_active":"2023-11-26T07:17:32.135078+0000","last_became_peered":"2023-11-26T07:17:32.135078+0000","last_unstale":"2023-11-26T07:17:34.327313+0000","last_undegraded":"2023-11-26T07:17:34.327313+0000","last_fullsized":"2023-11-26T07:17:34.327313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:04.536431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327077+0000","last_change":"2023-11-26T07:17:33.149589+0000","last_active":"2023-11-26T07:17:34.327077+0000","last_peered":"2023-11-26T07:17:34.327077+0000","last_clean":"2023-11-26T07:17:34.327077+0000","last_became_active":"2023-11-26T07:17:33.149228+0000","last_became_peered":"2023-11-26T07:17:33.149228+0000","last_unstale":"2023-11-26T07:17:34.327077+0000","last_undegraded":"2023-11-26T07:17:34.327077+0000","last_fullsized":"2023-11-26T07:17:34.327077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327286+0000","last_change":"2023-11-26T07:17:32.118350+0000","last_active":"2023-11-26T07:17:34.327286+0000","last_peered":"2023-11-26T07:17:34.327286+0000","last_clean":"2023-11-26T07:17:34.327286+0000","last_became_active":"2023-11-26T07:17:32.118040+0000","last_became_peered":"2023-11-26T07:17:32.118040+0000","last_unstale":"2023-11-26T07:17:34.327286+0000","last_undegraded":"2023-11-26T07:17:34.327286+0000","last_fullsized":"2023-11-26T07:17:34.327286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:21.575382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131017+0000","last_change":"2023-11-26T07:17:33.150796+0000","last_active":"2023-11-26T07:17:34.131017+0000","last_peered":"2023-11-26T07:17:34.131017+0000","last_clean":"2023-11-26T07:17:34.131017+0000","last_became_active":"2023-11-26T07:17:33.150543+0000","last_became_peered":"2023-11-26T07:17:33.150543+0000","last_unstale":"2023-11-26T07:17:34.131017+0000","last_undegraded":"2023-11-26T07:17:34.131017+0000","last_fullsized":"2023-11-26T07:17:34.131017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:59:35.145697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.129145+0000","last_change":"2023-11-26T07:17:32.134890+0000","last_active":"2023-11-26T07:17:34.129145+0000","last_peered":"2023-11-26T07:17:34.129145+0000","last_clean":"2023-11-26T07:17:34.129145+0000","last_became_active":"2023-11-26T07:17:32.134621+0000","last_became_peered":"2023-11-26T07:17:32.134621+0000","last_unstale":"2023-11-26T07:17:34.129145+0000","last_undegraded":"2023-11-26T07:17:34.129145+0000","last_fullsized":"2023-11-26T07:17:34.129145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:09.498267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328230+0000","last_change":"2023-11-26T07:17:33.135948+0000","last_active":"2023-11-26T07:17:34.328230+0000","last_peered":"2023-11-26T07:17:34.328230+0000","last_clean":"2023-11-26T07:17:34.328230+0000","last_became_active":"2023-11-26T07:17:33.135636+0000","last_became_peered":"2023-11-26T07:17:33.135636+0000","last_unstale":"2023-11-26T07:17:34.328230+0000","last_undegraded":"2023-11-26T07:17:34.328230+0000","last_fullsized":"2023-11-26T07:17:34.328230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326291+0000","last_change":"2023-11-26T07:17:32.134899+0000","last_active":"2023-11-26T07:17:34.326291+0000","last_peered":"2023-11-26T07:17:34.326291+0000","last_clean":"2023-11-26T07:17:34.326291+0000","last_became_active":"2023-11-26T07:17:32.134566+0000","last_became_peered":"2023-11-26T07:17:32.134566+0000","last_unstale":"2023-11-26T07:17:34.326291+0000","last_undegraded":"2023-11-26T07:17:34.326291+0000","last_fullsized":"2023-11-26T07:17:34.326291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:29.834563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127563+0000","last_change":"2023-11-26T07:17:33.123386+0000","last_active":"2023-11-26T07:17:34.127563+0000","last_peered":"2023-11-26T07:17:34.127563+0000","last_clean":"2023-11-26T07:17:34.127563+0000","last_became_active":"2023-11-26T07:17:33.120220+0000","last_became_peered":"2023-11-26T07:17:33.120220+0000","last_unstale":"2023-11-26T07:17:34.127563+0000","last_undegraded":"2023-11-26T07:17:34.127563+0000","last_fullsized":"2023-11-26T07:17:34.127563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:20.756357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326867+0000","last_change":"2023-11-26T07:17:32.138806+0000","last_active":"2023-11-26T07:17:34.326867+0000","last_peered":"2023-11-26T07:17:34.326867+0000","last_clean":"2023-11-26T07:17:34.326867+0000","last_became_active":"2023-11-26T07:17:32.138612+0000","last_became_peered":"2023-11-26T07:17:32.138612+0000","last_unstale":"2023-11-26T07:17:34.326867+0000","last_undegraded":"2023-11-26T07:17:34.326867+0000","last_fullsized":"2023-11-26T07:17:34.326867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130595+0000","last_change":"2023-11-26T07:17:33.139703+0000","last_active":"2023-11-26T07:17:34.130595+0000","last_peered":"2023-11-26T07:17:34.130595+0000","last_clean":"2023-11-26T07:17:34.130595+0000","last_became_active":"2023-11-26T07:17:33.139347+0000","last_became_peered":"2023-11-26T07:17:33.139347+0000","last_unstale":"2023-11-26T07:17:34.130595+0000","last_undegraded":"2023-11-26T07:17:34.130595+0000","last_fullsized":"2023-11-26T07:17:34.130595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327161+0000","last_change":"2023-11-26T07:17:32.141098+0000","last_active":"2023-11-26T07:17:34.327161+0000","last_peered":"2023-11-26T07:17:34.327161+0000","last_clean":"2023-11-26T07:17:34.327161+0000","last_became_active":"2023-11-26T07:17:32.137814+0000","last_became_peered":"2023-11-26T07:17:32.137814+0000","last_unstale":"2023-11-26T07:17:34.327161+0000","last_undegraded":"2023-11-26T07:17:34.327161+0000","last_fullsized":"2023-11-26T07:17:34.327161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:53:41.643510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130835+0000","last_change":"2023-11-26T07:17:33.124135+0000","last_active":"2023-11-26T07:17:34.130835+0000","last_peered":"2023-11-26T07:17:34.130835+0000","last_clean":"2023-11-26T07:17:34.130835+0000","last_became_active":"2023-11-26T07:17:33.123814+0000","last_became_peered":"2023-11-26T07:17:33.123814+0000","last_unstale":"2023-11-26T07:17:34.130835+0000","last_undegraded":"2023-11-26T07:17:34.130835+0000","last_fullsized":"2023-11-26T07:17:34.130835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326630+0000","last_change":"2023-11-26T07:17:32.140282+0000","last_active":"2023-11-26T07:17:34.326630+0000","last_peered":"2023-11-26T07:17:34.326630+0000","last_clean":"2023-11-26T07:17:34.326630+0000","last_became_active":"2023-11-26T07:17:32.139388+0000","last_became_peered":"2023-11-26T07:17:32.139388+0000","last_unstale":"2023-11-26T07:17:34.326630+0000","last_undegraded":"2023-11-26T07:17:34.326630+0000","last_fullsized":"2023-11-26T07:17:34.326630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:24.546723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127842+0000","last_change":"2023-11-26T07:17:33.140095+0000","last_active":"2023-11-26T07:17:34.127842+0000","last_peered":"2023-11-26T07:17:34.127842+0000","last_clean":"2023-11-26T07:17:34.127842+0000","last_became_active":"2023-11-26T07:17:33.139715+0000","last_became_peered":"2023-11-26T07:17:33.139715+0000","last_unstale":"2023-11-26T07:17:34.127842+0000","last_undegraded":"2023-11-26T07:17:34.127842+0000","last_fullsized":"2023-11-26T07:17:34.127842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326469+0000","last_change":"2023-11-26T07:17:33.132899+0000","last_active":"2023-11-26T07:17:34.326469+0000","last_peered":"2023-11-26T07:17:34.326469+0000","last_clean":"2023-11-26T07:17:34.326469+0000","last_became_active":"2023-11-26T07:17:33.132617+0000","last_became_peered":"2023-11-26T07:17:33.132617+0000","last_unstale":"2023-11-26T07:17:34.326469+0000","last_undegraded":"2023-11-26T07:17:34.326469+0000","last_fullsized":"2023-11-26T07:17:34.326469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326817+0000","last_change":"2023-11-26T07:17:32.138620+0000","last_active":"2023-11-26T07:17:34.326817+0000","last_peered":"2023-11-26T07:17:34.326817+0000","last_clean":"2023-11-26T07:17:34.326817+0000","last_became_active":"2023-11-26T07:17:32.138135+0000","last_became_peered":"2023-11-26T07:17:32.138135+0000","last_unstale":"2023-11-26T07:17:34.326817+0000","last_undegraded":"2023-11-26T07:17:34.326817+0000","last_fullsized":"2023-11-26T07:17:34.326817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327036+0000","last_change":"2023-11-26T07:17:33.136866+0000","last_active":"2023-11-26T07:17:34.327036+0000","last_peered":"2023-11-26T07:17:34.327036+0000","last_clean":"2023-11-26T07:17:34.327036+0000","last_became_active":"2023-11-26T07:17:33.136571+0000","last_became_peered":"2023-11-26T07:17:33.136571+0000","last_unstale":"2023-11-26T07:17:34.327036+0000","last_undegraded":"2023-11-26T07:17:34.327036+0000","last_fullsized":"2023-11-26T07:17:34.327036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:13.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327073+0000","last_change":"2023-11-26T07:17:32.135405+0000","last_active":"2023-11-26T07:17:34.327073+0000","last_peered":"2023-11-26T07:17:34.327073+0000","last_clean":"2023-11-26T07:17:34.327073+0000","last_became_active":"2023-11-26T07:17:32.134722+0000","last_became_peered":"2023-11-26T07:17:32.134722+0000","last_unstale":"2023-11-26T07:17:34.327073+0000","last_undegraded":"2023-11-26T07:17:34.327073+0000","last_fullsized":"2023-11-26T07:17:34.327073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328275+0000","last_change":"2023-11-26T07:17:33.138935+0000","last_active":"2023-11-26T07:17:34.328275+0000","last_peered":"2023-11-26T07:17:34.328275+0000","last_clean":"2023-11-26T07:17:34.328275+0000","last_became_active":"2023-11-26T07:17:33.138684+0000","last_became_peered":"2023-11-26T07:17:33.138684+0000","last_unstale":"2023-11-26T07:17:34.328275+0000","last_undegraded":"2023-11-26T07:17:34.328275+0000","last_fullsized":"2023-11-26T07:17:34.328275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131063+0000","last_change":"2023-11-26T07:17:32.130834+0000","last_active":"2023-11-26T07:17:34.131063+0000","last_peered":"2023-11-26T07:17:34.131063+0000","last_clean":"2023-11-26T07:17:34.131063+0000","last_became_active":"2023-11-26T07:17:32.130525+0000","last_became_peered":"2023-11-26T07:17:32.130525+0000","last_unstale":"2023-11-26T07:17:34.131063+0000","last_undegraded":"2023-11-26T07:17:34.131063+0000","last_fullsized":"2023-11-26T07:17:34.131063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:09.498267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131026+0000","last_change":"2023-11-26T07:17:33.146148+0000","last_active":"2023-11-26T07:17:34.131026+0000","last_peered":"2023-11-26T07:17:34.131026+0000","last_clean":"2023-11-26T07:17:34.131026+0000","last_became_active":"2023-11-26T07:17:33.145838+0000","last_became_peered":"2023-11-26T07:17:33.145838+0000","last_unstale":"2023-11-26T07:17:34.131026+0000","last_undegraded":"2023-11-26T07:17:34.131026+0000","last_fullsized":"2023-11-26T07:17:34.131026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328317+0000","last_change":"2023-11-26T07:17:32.135921+0000","last_active":"2023-11-26T07:17:34.328317+0000","last_peered":"2023-11-26T07:17:34.328317+0000","last_clean":"2023-11-26T07:17:34.328317+0000","last_became_active":"2023-11-26T07:17:32.135634+0000","last_became_peered":"2023-11-26T07:17:32.135634+0000","last_unstale":"2023-11-26T07:17:34.328317+0000","last_undegraded":"2023-11-26T07:17:34.328317+0000","last_fullsized":"2023-11-26T07:17:34.328317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.551435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:37.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:37.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0872613640 1 Processor -- start 2023-11-26T07:17:37.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0872613640 1 -- start start 2023-11-26T07:17:37.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0872613640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f086c143ae0 0x7f086c0b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0872613640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f086c0adcc0 0x7f086c0ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0872613640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086c0b2480 con 0x7f086c06dfd0 2023-11-26T07:17:37.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0872613640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f086c070670 con 0x7f086c0adcc0 2023-11-26T07:17:37.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0872613640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086c070950 con 0x7f086c143ae0 2023-11-26T07:17:37.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0870bb0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f086c0adcc0 0x7f086c0ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f086b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f086c06dfd0 0x7f086c06e410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46210/0 (socket says 172.21.15.31:46210) 2023-11-26T07:17:37.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f086b7fe640 1 -- 172.21.15.31:0/2139770135 learned_addr learned my addr 172.21.15.31:0/2139770135 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:37.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0870bb0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f086c0adcc0 0x7f086c0ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59248/0 (socket says 172.21.15.31:59248) 2023-11-26T07:17:37.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f086bfff640 1 --2- 172.21.15.31:0/2139770135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f086c143ae0 0x7f086c0b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0870bb0640 1 -- 172.21.15.31:0/2139770135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f086c143ae0 msgr2=0x7f086c0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:37.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0870bb0640 1 --2- 172.21.15.31:0/2139770135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f086c143ae0 0x7f086c0b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f0870bb0640 1 -- 172.21.15.31:0/2139770135 >> v1:172.21.15.31:6789/0 conn(0x7f086c06dfd0 legacy=0x7f086c06e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f0870bb0640 1 -- 172.21.15.31:0/2139770135 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c070c30 con 0x7f086c0adcc0 2023-11-26T07:17:37.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.421+0000 7f086affd640 1 -- 172.21.15.31:0/2139770135 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562528556 0 0) 0x7f086c0b2480 con 0x7f086c06dfd0 2023-11-26T07:17:37.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f0870bb0640 1 --2- 172.21.15.31:0/2139770135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c0adcc0 0x7f086c0ae110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0864009d90 tx=0x7f086401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e69b999a48a37d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f086affd640 1 -- 172.21.15.31:0/2139770135 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08640a59e0 con 0x7f086c0adcc0 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f086affd640 1 -- 172.21.15.31:0/2139770135 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08640a5b40 con 0x7f086c0adcc0 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f086affd640 1 -- 172.21.15.31:0/2139770135 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08640ad440 con 0x7f086c0adcc0 2023-11-26T07:17:37.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f0872613640 1 -- 172.21.15.31:0/2139770135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c0adcc0 msgr2=0x7f086c0ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f0872613640 1 --2- 172.21.15.31:0/2139770135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c0adcc0 0x7f086c0ae110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0864009d90 tx=0x7f086401c920 comp rx=0 tx=0).stop 2023-11-26T07:17:37.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f0872613640 1 -- 172.21.15.31:0/2139770135 shutdown_connections 2023-11-26T07:17:37.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f0872613640 1 --2- 172.21.15.31:0/2139770135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c0adcc0 0x7f086c0ae110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f0872613640 1 --2- 172.21.15.31:0/2139770135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f086c143ae0 0x7f086c0b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f0872613640 1 -- 172.21.15.31:0/2139770135 >> 172.21.15.31:0/2139770135 conn(0x7f086c0b0c40 msgr2=0x7f086c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:37.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f0872613640 1 -- 172.21.15.31:0/2139770135 shutdown_connections 2023-11-26T07:17:37.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.425+0000 7f0872613640 1 -- 172.21.15.31:0/2139770135 wait complete. 2023-11-26T07:17:37.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0872613640 1 Processor -- start 2023-11-26T07:17:37.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0872613640 1 -- start start 2023-11-26T07:17:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0872613640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c06dfd0 0x7f086c09c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0872613640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f086c0adcc0 0x7f086c144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0872613640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f086c143ae0 0x7f086c139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0872613640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c159fe0 con 0x7f086c06dfd0 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0872613640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c1597d0 con 0x7f086c0adcc0 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0872613640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086c13bfd0 con 0x7f086c143ae0 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f086bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c06dfd0 0x7f086c09c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f086bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c06dfd0 0x7f086c09c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59260/0 (socket says 172.21.15.31:59260) 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f086bfff640 1 -- 172.21.15.31:0/3750264109 learned_addr learned my addr 172.21.15.31:0/3750264109 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0870bb0640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f086c143ae0 0x7f086c139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f086b7fe640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f086c0adcc0 0x7f086c144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f086bfff640 1 -- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f086c143ae0 msgr2=0x7f086c139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f086bfff640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f086c143ae0 0x7f086c139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f086bfff640 1 -- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f086c0adcc0 msgr2=0x7f086c144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f086bfff640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f086c0adcc0 0x7f086c144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f086bfff640 1 -- 172.21.15.31:0/3750264109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0864009a70 con 0x7f086c06dfd0 2023-11-26T07:17:37.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f086bfff640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c06dfd0 0x7f086c09c2c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f085c00d620 tx=0x7f085c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0868ff9640 1 -- 172.21.15.31:0/3750264109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c00b620 con 0x7f086c06dfd0 2023-11-26T07:17:37.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0868ff9640 1 -- 172.21.15.31:0/3750264109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c00b780 con 0x7f086c06dfd0 2023-11-26T07:17:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0868ff9640 1 -- 172.21.15.31:0/3750264109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c00bb30 con 0x7f086c06dfd0 2023-11-26T07:17:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.429+0000 7f0872613640 1 -- 172.21.15.31:0/3750264109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086c13b9c0 con 0x7f086c06dfd0 2023-11-26T07:17:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.433+0000 7f0872613640 1 -- 172.21.15.31:0/3750264109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086c159d00 con 0x7f086c06dfd0 2023-11-26T07:17:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.433+0000 7f0872613640 1 -- 172.21.15.31:0/3750264109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086c04e850 con 0x7f086c06dfd0 2023-11-26T07:17:37.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.433+0000 7f0868ff9640 1 -- 172.21.15.31:0/3750264109 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f085c00bc90 con 0x7f086c06dfd0 2023-11-26T07:17:37.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.433+0000 7f0868ff9640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0858066470 0x7f0858068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.433+0000 7f0868ff9640 1 -- 172.21.15.31:0/3750264109 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f085c0842d0 con 0x7f086c06dfd0 2023-11-26T07:17:37.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.433+0000 7f086b7fe640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0858066470 0x7f0858068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.437+0000 7f086b7fe640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0858066470 0x7f0858068930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f086c165010 tx=0x7f0860007d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.437+0000 7f0868ff9640 1 -- 172.21.15.31:0/3750264109 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f085c051760 con 0x7f086c06dfd0 2023-11-26T07:17:37.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.593+0000 7f0872613640 1 -- 172.21.15.31:0/3750264109 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f086c0725d0 con 0x7f0858066470 2023-11-26T07:17:37.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.597+0000 7f0868ff9640 1 -- 172.21.15.31:0/3750264109 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264614 (secure 0 0 0) 0x7f086c0725d0 con 0x7f0858066470 2023-11-26T07:17:37.605 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:17:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 -- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0858066470 msgr2=0x7f0858068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0858066470 0x7f0858068930 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f086c165010 tx=0x7f0860007d60 comp rx=0 tx=0).stop 2023-11-26T07:17:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 -- 172.21.15.31:0/3750264109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c06dfd0 msgr2=0x7f086c09c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c06dfd0 0x7f086c09c2c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f085c00d620 tx=0x7f085c00dae0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 -- 172.21.15.31:0/3750264109 shutdown_connections 2023-11-26T07:17:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0858066470 0x7f0858068930 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f086c143ae0 0x7f086c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f086c0adcc0 0x7f086c144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 --2- 172.21.15.31:0/3750264109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086c06dfd0 0x7f086c09c2c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 -- 172.21.15.31:0/3750264109 >> 172.21.15.31:0/3750264109 conn(0x7f086c0b0c40 msgr2=0x7f086c0aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 -- 172.21.15.31:0/3750264109 shutdown_connections 2023-11-26T07:17:37.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.601+0000 7f0872613640 1 -- 172.21.15.31:0/3750264109 wait complete. 2023-11-26T07:17:37.625 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-26T07:17:36.307256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.210503"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326879+0000","last_change":"2023-11-26T07:17:33.157517+0000","last_active":"2023-11-26T07:17:34.326879+0000","last_peered":"2023-11-26T07:17:34.326879+0000","last_clean":"2023-11-26T07:17:34.326879+0000","last_became_active":"2023-11-26T07:17:33.157236+0000","last_became_peered":"2023-11-26T07:17:33.157236+0000","last_unstale":"2023-11-26T07:17:34.326879+0000","last_undegraded":"2023-11-26T07:17:34.326879+0000","last_fullsized":"2023-11-26T07:17:34.326879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:50.214046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327264+0000","last_change":"2023-11-26T07:17:33.153106+0000","last_active":"2023-11-26T07:17:34.327264+0000","last_peered":"2023-11-26T07:17:34.327264+0000","last_clean":"2023-11-26T07:17:34.327264+0000","last_became_active":"2023-11-26T07:17:33.152758+0000","last_became_peered":"2023-11-26T07:17:33.152758+0000","last_unstale":"2023-11-26T07:17:34.327264+0000","last_undegraded":"2023-11-26T07:17:34.327264+0000","last_fullsized":"2023-11-26T07:17:34.327264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328087+0000","last_change":"2023-11-26T07:17:33.119394+0000","last_active":"2023-11-26T07:17:34.328087+0000","last_peered":"2023-11-26T07:17:34.328087+0000","last_clean":"2023-11-26T07:17:34.328087+0000","last_became_active":"2023-11-26T07:17:33.118798+0000","last_became_peered":"2023-11-26T07:17:33.118798+0000","last_unstale":"2023-11-26T07:17:34.328087+0000","last_undegraded":"2023-11-26T07:17:34.328087+0000","last_fullsized":"2023-11-26T07:17:34.328087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:11.113481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326896+0000","last_change":"2023-11-26T07:17:33.147341+0000","last_active":"2023-11-26T07:17:34.326896+0000","last_peered":"2023-11-26T07:17:34.326896+0000","last_clean":"2023-11-26T07:17:34.326896+0000","last_became_active":"2023-11-26T07:17:33.147066+0000","last_became_peered":"2023-11-26T07:17:33.147066+0000","last_unstale":"2023-11-26T07:17:34.326896+0000","last_undegraded":"2023-11-26T07:17:34.326896+0000","last_fullsized":"2023-11-26T07:17:34.326896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:59:35.145697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326372+0000","last_change":"2023-11-26T07:17:33.155667+0000","last_active":"2023-11-26T07:17:34.326372+0000","last_peered":"2023-11-26T07:17:34.326372+0000","last_clean":"2023-11-26T07:17:34.326372+0000","last_became_active":"2023-11-26T07:17:33.155374+0000","last_became_peered":"2023-11-26T07:17:33.155374+0000","last_unstale":"2023-11-26T07:17:34.326372+0000","last_undegraded":"2023-11-26T07:17:34.326372+0000","last_fullsized":"2023-11-26T07:17:34.326372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327032+0000","last_change":"2023-11-26T07:17:33.149613+0000","last_active":"2023-11-26T07:17:34.327032+0000","last_peered":"2023-11-26T07:17:34.327032+0000","last_clean":"2023-11-26T07:17:34.327032+0000","last_became_active":"2023-11-26T07:17:33.143662+0000","last_became_peered":"2023-11-26T07:17:33.143662+0000","last_unstale":"2023-11-26T07:17:34.327032+0000","last_undegraded":"2023-11-26T07:17:34.327032+0000","last_fullsized":"2023-11-26T07:17:34.327032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326468+0000","last_change":"2023-11-26T07:17:33.157519+0000","last_active":"2023-11-26T07:17:34.326468+0000","last_peered":"2023-11-26T07:17:34.326468+0000","last_clean":"2023-11-26T07:17:34.326468+0000","last_became_active":"2023-11-26T07:17:33.157220+0000","last_became_peered":"2023-11-26T07:17:33.157220+0000","last_unstale":"2023-11-26T07:17:34.326468+0000","last_undegraded":"2023-11-26T07:17:34.326468+0000","last_fullsized":"2023-11-26T07:17:34.326468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:15:29.617633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130474+0000","last_change":"2023-11-26T07:17:33.136788+0000","last_active":"2023-11-26T07:17:34.130474+0000","last_peered":"2023-11-26T07:17:34.130474+0000","last_clean":"2023-11-26T07:17:34.130474+0000","last_became_active":"2023-11-26T07:17:33.136185+0000","last_became_peered":"2023-11-26T07:17:33.136185+0000","last_unstale":"2023-11-26T07:17:34.130474+0000","last_undegraded":"2023-11-26T07:17:34.130474+0000","last_fullsized":"2023-11-26T07:17:34.130474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:11.113481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326962+0000","last_change":"2023-11-26T07:17:33.154122+0000","last_active":"2023-11-26T07:17:34.326962+0000","last_peered":"2023-11-26T07:17:34.326962+0000","last_clean":"2023-11-26T07:17:34.326962+0000","last_became_active":"2023-11-26T07:17:33.153841+0000","last_became_peered":"2023-11-26T07:17:33.153841+0000","last_unstale":"2023-11-26T07:17:34.326962+0000","last_undegraded":"2023-11-26T07:17:34.326962+0000","last_fullsized":"2023-11-26T07:17:34.326962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:05.773940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128871+0000","last_change":"2023-11-26T07:17:33.134966+0000","last_active":"2023-11-26T07:17:34.128871+0000","last_peered":"2023-11-26T07:17:34.128871+0000","last_clean":"2023-11-26T07:17:34.128871+0000","last_became_active":"2023-11-26T07:17:33.134672+0000","last_became_peered":"2023-11-26T07:17:33.134672+0000","last_unstale":"2023-11-26T07:17:34.128871+0000","last_undegraded":"2023-11-26T07:17:34.128871+0000","last_fullsized":"2023-11-26T07:17:34.128871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128543+0000","last_change":"2023-11-26T07:17:33.134943+0000","last_active":"2023-11-26T07:17:34.128543+0000","last_peered":"2023-11-26T07:17:34.128543+0000","last_clean":"2023-11-26T07:17:34.128543+0000","last_became_active":"2023-11-26T07:17:33.134570+0000","last_became_peered":"2023-11-26T07:17:33.134570+0000","last_unstale":"2023-11-26T07:17:34.128543+0000","last_undegraded":"2023-11-26T07:17:34.128543+0000","last_fullsized":"2023-11-26T07:17:34.128543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128407+0000","last_change":"2023-11-26T07:17:33.136068+0000","last_active":"2023-11-26T07:17:34.128407+0000","last_peered":"2023-11-26T07:17:34.128407+0000","last_clean":"2023-11-26T07:17:34.128407+0000","last_became_active":"2023-11-26T07:17:33.135783+0000","last_became_peered":"2023-11-26T07:17:33.135783+0000","last_unstale":"2023-11-26T07:17:34.128407+0000","last_undegraded":"2023-11-26T07:17:34.128407+0000","last_fullsized":"2023-11-26T07:17:34.128407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:30.841394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130800+0000","last_change":"2023-11-26T07:17:33.145916+0000","last_active":"2023-11-26T07:17:34.130800+0000","last_peered":"2023-11-26T07:17:34.130800+0000","last_clean":"2023-11-26T07:17:34.130800+0000","last_became_active":"2023-11-26T07:17:33.141115+0000","last_became_peered":"2023-11-26T07:17:33.141115+0000","last_unstale":"2023-11-26T07:17:34.130800+0000","last_undegraded":"2023-11-26T07:17:34.130800+0000","last_fullsized":"2023-11-26T07:17:34.130800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327928+0000","last_change":"2023-11-26T07:17:33.153356+0000","last_active":"2023-11-26T07:17:34.327928+0000","last_peered":"2023-11-26T07:17:34.327928+0000","last_clean":"2023-11-26T07:17:34.327928+0000","last_became_active":"2023-11-26T07:17:33.153016+0000","last_became_peered":"2023-11-26T07:17:33.153016+0000","last_unstale":"2023-11-26T07:17:34.327928+0000","last_undegraded":"2023-11-26T07:17:34.327928+0000","last_fullsized":"2023-11-26T07:17:34.327928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:27.264806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326905+0000","last_change":"2023-11-26T07:17:33.138021+0000","last_active":"2023-11-26T07:17:34.326905+0000","last_peered":"2023-11-26T07:17:34.326905+0000","last_clean":"2023-11-26T07:17:34.326905+0000","last_became_active":"2023-11-26T07:17:33.137694+0000","last_became_peered":"2023-11-26T07:17:33.137694+0000","last_unstale":"2023-11-26T07:17:34.326905+0000","last_undegraded":"2023-11-26T07:17:34.326905+0000","last_fullsized":"2023-11-26T07:17:34.326905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.325959+0000","last_change":"2023-11-26T07:17:33.125499+0000","last_active":"2023-11-26T07:17:34.325959+0000","last_peered":"2023-11-26T07:17:34.325959+0000","last_clean":"2023-11-26T07:17:34.325959+0000","last_became_active":"2023-11-26T07:17:33.125132+0000","last_became_peered":"2023-11-26T07:17:33.125132+0000","last_unstale":"2023-11-26T07:17:34.325959+0000","last_undegraded":"2023-11-26T07:17:34.325959+0000","last_fullsized":"2023-11-26T07:17:34.325959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326610+0000","last_change":"2023-11-26T07:17:33.147715+0000","last_active":"2023-11-26T07:17:34.326610+0000","last_peered":"2023-11-26T07:17:34.326610+0000","last_clean":"2023-11-26T07:17:34.326610+0000","last_became_active":"2023-11-26T07:17:33.146632+0000","last_became_peered":"2023-11-26T07:17:33.146632+0000","last_unstale":"2023-11-26T07:17:34.326610+0000","last_undegraded":"2023-11-26T07:17:34.326610+0000","last_fullsized":"2023-11-26T07:17:34.326610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:54.079736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326760+0000","last_change":"2023-11-26T07:17:33.141795+0000","last_active":"2023-11-26T07:17:34.326760+0000","last_peered":"2023-11-26T07:17:34.326760+0000","last_clean":"2023-11-26T07:17:34.326760+0000","last_became_active":"2023-11-26T07:17:33.141529+0000","last_became_peered":"2023-11-26T07:17:33.141529+0000","last_unstale":"2023-11-26T07:17:34.326760+0000","last_undegraded":"2023-11-26T07:17:34.326760+0000","last_fullsized":"2023-11-26T07:17:34.326760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:35.443597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327208+0000","last_change":"2023-11-26T07:17:33.156644+0000","last_active":"2023-11-26T07:17:34.327208+0000","last_peered":"2023-11-26T07:17:34.327208+0000","last_clean":"2023-11-26T07:17:34.327208+0000","last_became_active":"2023-11-26T07:17:33.156430+0000","last_became_peered":"2023-11-26T07:17:33.156430+0000","last_unstale":"2023-11-26T07:17:34.327208+0000","last_undegraded":"2023-11-26T07:17:34.327208+0000","last_fullsized":"2023-11-26T07:17:34.327208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:54.433587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127997+0000","last_change":"2023-11-26T07:17:33.133505+0000","last_active":"2023-11-26T07:17:34.127997+0000","last_peered":"2023-11-26T07:17:34.127997+0000","last_clean":"2023-11-26T07:17:34.127997+0000","last_became_active":"2023-11-26T07:17:33.133134+0000","last_became_peered":"2023-11-26T07:17:33.133134+0000","last_unstale":"2023-11-26T07:17:34.127997+0000","last_undegraded":"2023-11-26T07:17:34.127997+0000","last_fullsized":"2023-11-26T07:17:34.127997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328056+0000","last_change":"2023-11-26T07:17:33.144471+0000","last_active":"2023-11-26T07:17:34.328056+0000","last_peered":"2023-11-26T07:17:34.328056+0000","last_clean":"2023-11-26T07:17:34.328056+0000","last_became_active":"2023-11-26T07:17:33.144183+0000","last_became_peered":"2023-11-26T07:17:33.144183+0000","last_unstale":"2023-11-26T07:17:34.328056+0000","last_undegraded":"2023-11-26T07:17:34.328056+0000","last_fullsized":"2023-11-26T07:17:34.328056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:52.817990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327032+0000","last_change":"2023-11-26T07:17:33.149400+0000","last_active":"2023-11-26T07:17:34.327032+0000","last_peered":"2023-11-26T07:17:34.327032+0000","last_clean":"2023-11-26T07:17:34.327032+0000","last_became_active":"2023-11-26T07:17:33.144991+0000","last_became_peered":"2023-11-26T07:17:33.144991+0000","last_unstale":"2023-11-26T07:17:34.327032+0000","last_undegraded":"2023-11-26T07:17:34.327032+0000","last_fullsized":"2023-11-26T07:17:34.327032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:15:29.617633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130588+0000","last_change":"2023-11-26T07:17:33.125477+0000","last_active":"2023-11-26T07:17:34.130588+0000","last_peered":"2023-11-26T07:17:34.130588+0000","last_clean":"2023-11-26T07:17:34.130588+0000","last_became_active":"2023-11-26T07:17:33.125109+0000","last_became_peered":"2023-11-26T07:17:33.125109+0000","last_unstale":"2023-11-26T07:17:34.130588+0000","last_undegraded":"2023-11-26T07:17:34.130588+0000","last_fullsized":"2023-11-26T07:17:34.130588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326026+0000","last_change":"2023-11-26T07:17:33.138762+0000","last_active":"2023-11-26T07:17:34.326026+0000","last_peered":"2023-11-26T07:17:34.326026+0000","last_clean":"2023-11-26T07:17:34.326026+0000","last_became_active":"2023-11-26T07:17:33.138466+0000","last_became_peered":"2023-11-26T07:17:33.138466+0000","last_unstale":"2023-11-26T07:17:34.326026+0000","last_undegraded":"2023-11-26T07:17:34.326026+0000","last_fullsized":"2023-11-26T07:17:34.326026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130957+0000","last_change":"2023-11-26T07:17:33.128236+0000","last_active":"2023-11-26T07:17:34.130957+0000","last_peered":"2023-11-26T07:17:34.130957+0000","last_clean":"2023-11-26T07:17:34.130957+0000","last_became_active":"2023-11-26T07:17:33.127900+0000","last_became_peered":"2023-11-26T07:17:33.127900+0000","last_unstale":"2023-11-26T07:17:34.130957+0000","last_undegraded":"2023-11-26T07:17:34.130957+0000","last_fullsized":"2023-11-26T07:17:34.130957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128993+0000","last_change":"2023-11-26T07:17:33.144760+0000","last_active":"2023-11-26T07:17:34.128993+0000","last_peered":"2023-11-26T07:17:34.128993+0000","last_clean":"2023-11-26T07:17:34.128993+0000","last_became_active":"2023-11-26T07:17:33.144272+0000","last_became_peered":"2023-11-26T07:17:33.144272+0000","last_unstale":"2023-11-26T07:17:34.128993+0000","last_undegraded":"2023-11-26T07:17:34.128993+0000","last_fullsized":"2023-11-26T07:17:34.128993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327140+0000","last_change":"2023-11-26T07:17:33.151793+0000","last_active":"2023-11-26T07:17:34.327140+0000","last_peered":"2023-11-26T07:17:34.327140+0000","last_clean":"2023-11-26T07:17:34.327140+0000","last_became_active":"2023-11-26T07:17:33.151517+0000","last_became_peered":"2023-11-26T07:17:33.151517+0000","last_unstale":"2023-11-26T07:17:34.327140+0000","last_undegraded":"2023-11-26T07:17:34.327140+0000","last_fullsized":"2023-11-26T07:17:34.327140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131015+0000","last_change":"2023-11-26T07:17:33.128368+0000","last_active":"2023-11-26T07:17:34.131015+0000","last_peered":"2023-11-26T07:17:34.131015+0000","last_clean":"2023-11-26T07:17:34.131015+0000","last_became_active":"2023-11-26T07:17:33.128068+0000","last_became_peered":"2023-11-26T07:17:33.128068+0000","last_unstale":"2023-11-26T07:17:34.131015+0000","last_undegraded":"2023-11-26T07:17:34.131015+0000","last_fullsized":"2023-11-26T07:17:34.131015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:54.079736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328159+0000","last_change":"2023-11-26T07:17:33.124145+0000","last_active":"2023-11-26T07:17:34.328159+0000","last_peered":"2023-11-26T07:17:34.328159+0000","last_clean":"2023-11-26T07:17:34.328159+0000","last_became_active":"2023-11-26T07:17:33.123870+0000","last_became_peered":"2023-11-26T07:17:33.123870+0000","last_unstale":"2023-11-26T07:17:34.328159+0000","last_undegraded":"2023-11-26T07:17:34.328159+0000","last_fullsized":"2023-11-26T07:17:34.328159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327220+0000","last_change":"2023-11-26T07:17:33.138523+0000","last_active":"2023-11-26T07:17:34.327220+0000","last_peered":"2023-11-26T07:17:34.327220+0000","last_clean":"2023-11-26T07:17:34.327220+0000","last_became_active":"2023-11-26T07:17:33.138213+0000","last_became_peered":"2023-11-26T07:17:33.138213+0000","last_unstale":"2023-11-26T07:17:34.327220+0000","last_undegraded":"2023-11-26T07:17:34.327220+0000","last_fullsized":"2023-11-26T07:17:34.327220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:12.772210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326933+0000","last_change":"2023-11-26T07:17:33.128197+0000","last_active":"2023-11-26T07:17:34.326933+0000","last_peered":"2023-11-26T07:17:34.326933+0000","last_clean":"2023-11-26T07:17:34.326933+0000","last_became_active":"2023-11-26T07:17:33.127911+0000","last_became_peered":"2023-11-26T07:17:33.127911+0000","last_unstale":"2023-11-26T07:17:34.326933+0000","last_undegraded":"2023-11-26T07:17:34.326933+0000","last_fullsized":"2023-11-26T07:17:34.326933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:17.558267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127456+0000","last_change":"2023-11-26T07:17:33.140214+0000","last_active":"2023-11-26T07:17:34.127456+0000","last_peered":"2023-11-26T07:17:34.127456+0000","last_clean":"2023-11-26T07:17:34.127456+0000","last_became_active":"2023-11-26T07:17:33.139927+0000","last_became_peered":"2023-11-26T07:17:33.139927+0000","last_unstale":"2023-11-26T07:17:34.127456+0000","last_undegraded":"2023-11-26T07:17:34.127456+0000","last_fullsized":"2023-11-26T07:17:34.127456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128003+0000","last_change":"2023-11-26T07:17:33.151273+0000","last_active":"2023-11-26T07:17:34.128003+0000","last_peered":"2023-11-26T07:17:34.128003+0000","last_clean":"2023-11-26T07:17:34.128003+0000","last_became_active":"2023-11-26T07:17:33.150182+0000","last_became_peered":"2023-11-26T07:17:33.150182+0000","last_unstale":"2023-11-26T07:17:34.128003+0000","last_undegraded":"2023-11-26T07:17:34.128003+0000","last_fullsized":"2023-11-26T07:17:34.128003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:02.280784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131100+0000","last_change":"2023-11-26T07:17:32.135703+0000","last_active":"2023-11-26T07:17:34.131100+0000","last_peered":"2023-11-26T07:17:34.131100+0000","last_clean":"2023-11-26T07:17:34.131100+0000","last_became_active":"2023-11-26T07:17:32.135392+0000","last_became_peered":"2023-11-26T07:17:32.135392+0000","last_unstale":"2023-11-26T07:17:34.131100+0000","last_undegraded":"2023-11-26T07:17:34.131100+0000","last_fullsized":"2023-11-26T07:17:34.131100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328230+0000","last_change":"2023-11-26T07:17:33.123539+0000","last_active":"2023-11-26T07:17:34.328230+0000","last_peered":"2023-11-26T07:17:34.328230+0000","last_clean":"2023-11-26T07:17:34.328230+0000","last_became_active":"2023-11-26T07:17:33.123240+0000","last_became_peered":"2023-11-26T07:17:33.123240+0000","last_unstale":"2023-11-26T07:17:34.328230+0000","last_undegraded":"2023-11-26T07:17:34.328230+0000","last_fullsized":"2023-11-26T07:17:34.328230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127853+0000","last_change":"2023-11-26T07:17:32.124304+0000","last_active":"2023-11-26T07:17:34.127853+0000","last_peered":"2023-11-26T07:17:34.127853+0000","last_clean":"2023-11-26T07:17:34.127853+0000","last_became_active":"2023-11-26T07:17:32.122374+0000","last_became_peered":"2023-11-26T07:17:32.122374+0000","last_unstale":"2023-11-26T07:17:34.127853+0000","last_undegraded":"2023-11-26T07:17:34.127853+0000","last_fullsized":"2023-11-26T07:17:34.127853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:53:41.643510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130553+0000","last_change":"2023-11-26T07:17:32.132756+0000","last_active":"2023-11-26T07:17:34.130553+0000","last_peered":"2023-11-26T07:17:34.130553+0000","last_clean":"2023-11-26T07:17:34.130553+0000","last_became_active":"2023-11-26T07:17:32.132329+0000","last_became_peered":"2023-11-26T07:17:32.132329+0000","last_unstale":"2023-11-26T07:17:34.130553+0000","last_undegraded":"2023-11-26T07:17:34.130553+0000","last_fullsized":"2023-11-26T07:17:34.130553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130543+0000","last_change":"2023-11-26T07:17:33.124808+0000","last_active":"2023-11-26T07:17:34.130543+0000","last_peered":"2023-11-26T07:17:34.130543+0000","last_clean":"2023-11-26T07:17:34.130543+0000","last_became_active":"2023-11-26T07:17:33.124508+0000","last_became_peered":"2023-11-26T07:17:33.124508+0000","last_unstale":"2023-11-26T07:17:34.130543+0000","last_undegraded":"2023-11-26T07:17:34.130543+0000","last_fullsized":"2023-11-26T07:17:34.130543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326062+0000","last_change":"2023-11-26T07:17:32.136207+0000","last_active":"2023-11-26T07:17:34.326062+0000","last_peered":"2023-11-26T07:17:34.326062+0000","last_clean":"2023-11-26T07:17:34.326062+0000","last_became_active":"2023-11-26T07:17:32.135850+0000","last_became_peered":"2023-11-26T07:17:32.135850+0000","last_unstale":"2023-11-26T07:17:34.326062+0000","last_undegraded":"2023-11-26T07:17:34.326062+0000","last_fullsized":"2023-11-26T07:17:34.326062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:48.178664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326761+0000","last_change":"2023-11-26T07:17:33.153245+0000","last_active":"2023-11-26T07:17:34.326761+0000","last_peered":"2023-11-26T07:17:34.326761+0000","last_clean":"2023-11-26T07:17:34.326761+0000","last_became_active":"2023-11-26T07:17:33.151566+0000","last_became_peered":"2023-11-26T07:17:33.151566+0000","last_unstale":"2023-11-26T07:17:34.326761+0000","last_undegraded":"2023-11-26T07:17:34.326761+0000","last_fullsized":"2023-11-26T07:17:34.326761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.129164+0000","last_change":"2023-11-26T07:17:32.126580+0000","last_active":"2023-11-26T07:17:34.129164+0000","last_peered":"2023-11-26T07:17:34.129164+0000","last_clean":"2023-11-26T07:17:34.129164+0000","last_became_active":"2023-11-26T07:17:32.125693+0000","last_became_peered":"2023-11-26T07:17:32.125693+0000","last_unstale":"2023-11-26T07:17:34.129164+0000","last_undegraded":"2023-11-26T07:17:34.129164+0000","last_fullsized":"2023-11-26T07:17:34.129164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327410+0000","last_change":"2023-11-26T07:17:33.145310+0000","last_active":"2023-11-26T07:17:34.327410+0000","last_peered":"2023-11-26T07:17:34.327410+0000","last_clean":"2023-11-26T07:17:34.327410+0000","last_became_active":"2023-11-26T07:17:33.145010+0000","last_became_peered":"2023-11-26T07:17:33.145010+0000","last_unstale":"2023-11-26T07:17:34.327410+0000","last_undegraded":"2023-11-26T07:17:34.327410+0000","last_fullsized":"2023-11-26T07:17:34.327410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:23:10.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327442+0000","last_change":"2023-11-26T07:17:32.130729+0000","last_active":"2023-11-26T07:17:34.327442+0000","last_peered":"2023-11-26T07:17:34.327442+0000","last_clean":"2023-11-26T07:17:34.327442+0000","last_became_active":"2023-11-26T07:17:32.130438+0000","last_became_peered":"2023-11-26T07:17:32.130438+0000","last_unstale":"2023-11-26T07:17:34.327442+0000","last_undegraded":"2023-11-26T07:17:34.327442+0000","last_fullsized":"2023-11-26T07:17:34.327442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:24.546723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328264+0000","last_change":"2023-11-26T07:17:33.136875+0000","last_active":"2023-11-26T07:17:34.328264+0000","last_peered":"2023-11-26T07:17:34.328264+0000","last_clean":"2023-11-26T07:17:34.328264+0000","last_became_active":"2023-11-26T07:17:33.136544+0000","last_became_peered":"2023-11-26T07:17:33.136544+0000","last_unstale":"2023-11-26T07:17:34.328264+0000","last_undegraded":"2023-11-26T07:17:34.328264+0000","last_fullsized":"2023-11-26T07:17:34.328264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327150+0000","last_change":"2023-11-26T07:17:32.127762+0000","last_active":"2023-11-26T07:17:34.327150+0000","last_peered":"2023-11-26T07:17:34.327150+0000","last_clean":"2023-11-26T07:17:34.327150+0000","last_became_active":"2023-11-26T07:17:32.127380+0000","last_became_peered":"2023-11-26T07:17:32.127380+0000","last_unstale":"2023-11-26T07:17:34.327150+0000","last_undegraded":"2023-11-26T07:17:34.327150+0000","last_fullsized":"2023-11-26T07:17:34.327150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128525+0000","last_change":"2023-11-26T07:17:33.147339+0000","last_active":"2023-11-26T07:17:34.128525+0000","last_peered":"2023-11-26T07:17:34.128525+0000","last_clean":"2023-11-26T07:17:34.128525+0000","last_became_active":"2023-11-26T07:17:33.147047+0000","last_became_peered":"2023-11-26T07:17:33.147047+0000","last_unstale":"2023-11-26T07:17:34.128525+0000","last_undegraded":"2023-11-26T07:17:34.128525+0000","last_fullsized":"2023-11-26T07:17:34.128525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:35.443597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128083+0000","last_change":"2023-11-26T07:17:32.133337+0000","last_active":"2023-11-26T07:17:34.128083+0000","last_peered":"2023-11-26T07:17:34.128083+0000","last_clean":"2023-11-26T07:17:34.128083+0000","last_became_active":"2023-11-26T07:17:32.132169+0000","last_became_peered":"2023-11-26T07:17:32.132169+0000","last_unstale":"2023-11-26T07:17:34.128083+0000","last_undegraded":"2023-11-26T07:17:34.128083+0000","last_fullsized":"2023-11-26T07:17:34.128083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:21.575382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326848+0000","last_change":"2023-11-26T07:17:33.150427+0000","last_active":"2023-11-26T07:17:34.326848+0000","last_peered":"2023-11-26T07:17:34.326848+0000","last_clean":"2023-11-26T07:17:34.326848+0000","last_became_active":"2023-11-26T07:17:33.150029+0000","last_became_peered":"2023-11-26T07:17:33.150029+0000","last_unstale":"2023-11-26T07:17:34.326848+0000","last_undegraded":"2023-11-26T07:17:34.326848+0000","last_fullsized":"2023-11-26T07:17:34.326848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:23:10.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328227+0000","last_change":"2023-11-26T07:17:32.133555+0000","last_active":"2023-11-26T07:17:34.328227+0000","last_peered":"2023-11-26T07:17:34.328227+0000","last_clean":"2023-11-26T07:17:34.328227+0000","last_became_active":"2023-11-26T07:17:32.132862+0000","last_became_peered":"2023-11-26T07:17:32.132862+0000","last_unstale":"2023-11-26T07:17:34.328227+0000","last_undegraded":"2023-11-26T07:17:34.328227+0000","last_fullsized":"2023-11-26T07:17:34.328227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:19.749525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328005+0000","last_change":"2023-11-26T07:17:33.144778+0000","last_active":"2023-11-26T07:17:34.328005+0000","last_peered":"2023-11-26T07:17:34.328005+0000","last_clean":"2023-11-26T07:17:34.328005+0000","last_became_active":"2023-11-26T07:17:33.144469+0000","last_became_peered":"2023-11-26T07:17:33.144469+0000","last_unstale":"2023-11-26T07:17:34.328005+0000","last_undegraded":"2023-11-26T07:17:34.328005+0000","last_fullsized":"2023-11-26T07:17:34.328005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:10.505099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327149+0000","last_change":"2023-11-26T07:17:32.140716+0000","last_active":"2023-11-26T07:17:34.327149+0000","last_peered":"2023-11-26T07:17:34.327149+0000","last_clean":"2023-11-26T07:17:34.327149+0000","last_became_active":"2023-11-26T07:17:32.139344+0000","last_became_peered":"2023-11-26T07:17:32.139344+0000","last_unstale":"2023-11-26T07:17:34.327149+0000","last_undegraded":"2023-11-26T07:17:34.327149+0000","last_fullsized":"2023-11-26T07:17:34.327149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326526+0000","last_change":"2023-11-26T07:17:33.152939+0000","last_active":"2023-11-26T07:17:34.326526+0000","last_peered":"2023-11-26T07:17:34.326526+0000","last_clean":"2023-11-26T07:17:34.326526+0000","last_became_active":"2023-11-26T07:17:33.152626+0000","last_became_peered":"2023-11-26T07:17:33.152626+0000","last_unstale":"2023-11-26T07:17:34.326526+0000","last_undegraded":"2023-11-26T07:17:34.326526+0000","last_fullsized":"2023-11-26T07:17:34.326526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:13.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127755+0000","last_change":"2023-11-26T07:17:32.127951+0000","last_active":"2023-11-26T07:17:34.127755+0000","last_peered":"2023-11-26T07:17:34.127755+0000","last_clean":"2023-11-26T07:17:34.127755+0000","last_became_active":"2023-11-26T07:17:32.127195+0000","last_became_peered":"2023-11-26T07:17:32.127195+0000","last_unstale":"2023-11-26T07:17:34.127755+0000","last_undegraded":"2023-11-26T07:17:34.127755+0000","last_fullsized":"2023-11-26T07:17:34.127755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.551435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127675+0000","last_change":"2023-11-26T07:17:33.132518+0000","last_active":"2023-11-26T07:17:34.127675+0000","last_peered":"2023-11-26T07:17:34.127675+0000","last_clean":"2023-11-26T07:17:34.127675+0000","last_became_active":"2023-11-26T07:17:33.131905+0000","last_became_peered":"2023-11-26T07:17:33.131905+0000","last_unstale":"2023-11-26T07:17:34.127675+0000","last_undegraded":"2023-11-26T07:17:34.127675+0000","last_fullsized":"2023-11-26T07:17:34.127675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326584+0000","last_change":"2023-11-26T07:17:32.129880+0000","last_active":"2023-11-26T07:17:34.326584+0000","last_peered":"2023-11-26T07:17:34.326584+0000","last_clean":"2023-11-26T07:17:34.326584+0000","last_became_active":"2023-11-26T07:17:32.129575+0000","last_became_peered":"2023-11-26T07:17:32.129575+0000","last_unstale":"2023-11-26T07:17:34.326584+0000","last_undegraded":"2023-11-26T07:17:34.326584+0000","last_fullsized":"2023-11-26T07:17:34.326584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:27.401076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326605+0000","last_change":"2023-11-26T07:17:33.152780+0000","last_active":"2023-11-26T07:17:34.326605+0000","last_peered":"2023-11-26T07:17:34.326605+0000","last_clean":"2023-11-26T07:17:34.326605+0000","last_became_active":"2023-11-26T07:17:33.152468+0000","last_became_peered":"2023-11-26T07:17:33.152468+0000","last_unstale":"2023-11-26T07:17:34.326605+0000","last_undegraded":"2023-11-26T07:17:34.326605+0000","last_fullsized":"2023-11-26T07:17:34.326605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:56:25.969462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128485+0000","last_change":"2023-11-26T07:17:32.132211+0000","last_active":"2023-11-26T07:17:34.128485+0000","last_peered":"2023-11-26T07:17:34.128485+0000","last_clean":"2023-11-26T07:17:34.128485+0000","last_became_active":"2023-11-26T07:17:32.131233+0000","last_became_peered":"2023-11-26T07:17:32.131233+0000","last_unstale":"2023-11-26T07:17:34.128485+0000","last_undegraded":"2023-11-26T07:17:34.128485+0000","last_fullsized":"2023-11-26T07:17:34.128485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:21.662285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326787+0000","last_change":"2023-11-26T07:17:33.154639+0000","last_active":"2023-11-26T07:17:34.326787+0000","last_peered":"2023-11-26T07:17:34.326787+0000","last_clean":"2023-11-26T07:17:34.326787+0000","last_became_active":"2023-11-26T07:17:33.153245+0000","last_became_peered":"2023-11-26T07:17:33.153245+0000","last_unstale":"2023-11-26T07:17:34.326787+0000","last_undegraded":"2023-11-26T07:17:34.326787+0000","last_fullsized":"2023-11-26T07:17:34.326787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:12.772210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326854+0000","last_change":"2023-11-26T07:17:32.138855+0000","last_active":"2023-11-26T07:17:34.326854+0000","last_peered":"2023-11-26T07:17:34.326854+0000","last_clean":"2023-11-26T07:17:34.326854+0000","last_became_active":"2023-11-26T07:17:32.138566+0000","last_became_peered":"2023-11-26T07:17:32.138566+0000","last_unstale":"2023-11-26T07:17:34.326854+0000","last_undegraded":"2023-11-26T07:17:34.326854+0000","last_fullsized":"2023-11-26T07:17:34.326854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:29.834563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327040+0000","last_change":"2023-11-26T07:17:33.152760+0000","last_active":"2023-11-26T07:17:34.327040+0000","last_peered":"2023-11-26T07:17:34.327040+0000","last_clean":"2023-11-26T07:17:34.327040+0000","last_became_active":"2023-11-26T07:17:33.152455+0000","last_became_peered":"2023-11-26T07:17:33.152455+0000","last_unstale":"2023-11-26T07:17:34.327040+0000","last_undegraded":"2023-11-26T07:17:34.327040+0000","last_fullsized":"2023-11-26T07:17:34.327040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:40.786432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327244+0000","last_change":"2023-11-26T07:17:32.132634+0000","last_active":"2023-11-26T07:17:34.327244+0000","last_peered":"2023-11-26T07:17:34.327244+0000","last_clean":"2023-11-26T07:17:34.327244+0000","last_became_active":"2023-11-26T07:17:32.132192+0000","last_became_peered":"2023-11-26T07:17:32.132192+0000","last_unstale":"2023-11-26T07:17:34.327244+0000","last_undegraded":"2023-11-26T07:17:34.327244+0000","last_fullsized":"2023-11-26T07:17:34.327244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:54:55.533333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327181+0000","last_change":"2023-11-26T07:17:33.147935+0000","last_active":"2023-11-26T07:17:34.327181+0000","last_peered":"2023-11-26T07:17:34.327181+0000","last_clean":"2023-11-26T07:17:34.327181+0000","last_became_active":"2023-11-26T07:17:33.147595+0000","last_became_peered":"2023-11-26T07:17:33.147595+0000","last_unstale":"2023-11-26T07:17:34.327181+0000","last_undegraded":"2023-11-26T07:17:34.327181+0000","last_fullsized":"2023-11-26T07:17:34.327181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:56:25.969462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326702+0000","last_change":"2023-11-26T07:17:32.128101+0000","last_active":"2023-11-26T07:17:34.326702+0000","last_peered":"2023-11-26T07:17:34.326702+0000","last_clean":"2023-11-26T07:17:34.326702+0000","last_became_active":"2023-11-26T07:17:32.127821+0000","last_became_peered":"2023-11-26T07:17:32.127821+0000","last_unstale":"2023-11-26T07:17:34.326702+0000","last_undegraded":"2023-11-26T07:17:34.326702+0000","last_fullsized":"2023-11-26T07:17:34.326702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:30:24.091629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326978+0000","last_change":"2023-11-26T07:17:33.153014+0000","last_active":"2023-11-26T07:17:34.326978+0000","last_peered":"2023-11-26T07:17:34.326978+0000","last_clean":"2023-11-26T07:17:34.326978+0000","last_became_active":"2023-11-26T07:17:33.152675+0000","last_became_peered":"2023-11-26T07:17:33.152675+0000","last_unstale":"2023-11-26T07:17:34.326978+0000","last_undegraded":"2023-11-26T07:17:34.326978+0000","last_fullsized":"2023-11-26T07:17:34.326978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:06.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-26T07:17:33.140703+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:17:33.140703+0000","last_peered":"2023-11-26T07:17:33.140703+0000","last_clean":"2023-11-26T07:17:33.140703+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:17:33.140703+0000","last_undegraded":"2023-11-26T07:17:33.140703+0000","last_fullsized":"2023-11-26T07:17:33.140703+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.325979+0000","last_change":"2023-11-26T07:17:32.127453+0000","last_active":"2023-11-26T07:17:34.325979+0000","last_peered":"2023-11-26T07:17:34.325979+0000","last_clean":"2023-11-26T07:17:34.325979+0000","last_became_active":"2023-11-26T07:17:32.127171+0000","last_became_peered":"2023-11-26T07:17:32.127171+0000","last_unstale":"2023-11-26T07:17:34.325979+0000","last_undegraded":"2023-11-26T07:17:34.325979+0000","last_fullsized":"2023-11-26T07:17:34.325979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:10.106650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130743+0000","last_change":"2023-11-26T07:17:33.132790+0000","last_active":"2023-11-26T07:17:34.130743+0000","last_peered":"2023-11-26T07:17:34.130743+0000","last_clean":"2023-11-26T07:17:34.130743+0000","last_became_active":"2023-11-26T07:17:33.132415+0000","last_became_peered":"2023-11-26T07:17:33.132415+0000","last_unstale":"2023-11-26T07:17:34.130743+0000","last_undegraded":"2023-11-26T07:17:34.130743+0000","last_fullsized":"2023-11-26T07:17:34.130743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326218+0000","last_change":"2023-11-26T07:17:32.134663+0000","last_active":"2023-11-26T07:17:34.326218+0000","last_peered":"2023-11-26T07:17:34.326218+0000","last_clean":"2023-11-26T07:17:34.326218+0000","last_became_active":"2023-11-26T07:17:32.134372+0000","last_became_peered":"2023-11-26T07:17:32.134372+0000","last_unstale":"2023-11-26T07:17:34.326218+0000","last_undegraded":"2023-11-26T07:17:34.326218+0000","last_fullsized":"2023-11-26T07:17:34.326218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:34.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327893+0000","last_change":"2023-11-26T07:17:33.131023+0000","last_active":"2023-11-26T07:17:34.327893+0000","last_peered":"2023-11-26T07:17:34.327893+0000","last_clean":"2023-11-26T07:17:34.327893+0000","last_became_active":"2023-11-26T07:17:33.130665+0000","last_became_peered":"2023-11-26T07:17:33.130665+0000","last_unstale":"2023-11-26T07:17:34.327893+0000","last_undegraded":"2023-11-26T07:17:34.327893+0000","last_fullsized":"2023-11-26T07:17:34.327893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:54:56.540164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327009+0000","last_change":"2023-11-26T07:17:32.141091+0000","last_active":"2023-11-26T07:17:34.327009+0000","last_peered":"2023-11-26T07:17:34.327009+0000","last_clean":"2023-11-26T07:17:34.327009+0000","last_became_active":"2023-11-26T07:17:32.139978+0000","last_became_peered":"2023-11-26T07:17:32.139978+0000","last_unstale":"2023-11-26T07:17:34.327009+0000","last_undegraded":"2023-11-26T07:17:34.327009+0000","last_fullsized":"2023-11-26T07:17:34.327009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:01.273952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326756+0000","last_change":"2023-11-26T07:17:33.137737+0000","last_active":"2023-11-26T07:17:34.326756+0000","last_peered":"2023-11-26T07:17:34.326756+0000","last_clean":"2023-11-26T07:17:34.326756+0000","last_became_active":"2023-11-26T07:17:33.137350+0000","last_became_peered":"2023-11-26T07:17:33.137350+0000","last_unstale":"2023-11-26T07:17:34.326756+0000","last_undegraded":"2023-11-26T07:17:34.326756+0000","last_fullsized":"2023-11-26T07:17:34.326756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327251+0000","last_change":"2023-11-26T07:17:32.134855+0000","last_active":"2023-11-26T07:17:34.327251+0000","last_peered":"2023-11-26T07:17:34.327251+0000","last_clean":"2023-11-26T07:17:34.327251+0000","last_became_active":"2023-11-26T07:17:32.134587+0000","last_became_peered":"2023-11-26T07:17:32.134587+0000","last_unstale":"2023-11-26T07:17:34.327251+0000","last_undegraded":"2023-11-26T07:17:34.327251+0000","last_fullsized":"2023-11-26T07:17:34.327251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128590+0000","last_change":"2023-11-26T07:17:33.122483+0000","last_active":"2023-11-26T07:17:34.128590+0000","last_peered":"2023-11-26T07:17:34.128590+0000","last_clean":"2023-11-26T07:17:34.128590+0000","last_became_active":"2023-11-26T07:17:33.122042+0000","last_became_peered":"2023-11-26T07:17:33.122042+0000","last_unstale":"2023-11-26T07:17:34.128590+0000","last_undegraded":"2023-11-26T07:17:34.128590+0000","last_fullsized":"2023-11-26T07:17:34.128590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:17.558267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327027+0000","last_change":"2023-11-26T07:17:32.124516+0000","last_active":"2023-11-26T07:17:34.327027+0000","last_peered":"2023-11-26T07:17:34.327027+0000","last_clean":"2023-11-26T07:17:34.327027+0000","last_became_active":"2023-11-26T07:17:32.123978+0000","last_became_peered":"2023-11-26T07:17:32.123978+0000","last_unstale":"2023-11-26T07:17:34.327027+0000","last_undegraded":"2023-11-26T07:17:34.327027+0000","last_fullsized":"2023-11-26T07:17:34.327027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:34.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326612+0000","last_change":"2023-11-26T07:17:33.139055+0000","last_active":"2023-11-26T07:17:34.326612+0000","last_peered":"2023-11-26T07:17:34.326612+0000","last_clean":"2023-11-26T07:17:34.326612+0000","last_became_active":"2023-11-26T07:17:33.138235+0000","last_became_peered":"2023-11-26T07:17:33.138235+0000","last_unstale":"2023-11-26T07:17:34.326612+0000","last_undegraded":"2023-11-26T07:17:34.326612+0000","last_fullsized":"2023-11-26T07:17:34.326612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327313+0000","last_change":"2023-11-26T07:17:32.135542+0000","last_active":"2023-11-26T07:17:34.327313+0000","last_peered":"2023-11-26T07:17:34.327313+0000","last_clean":"2023-11-26T07:17:34.327313+0000","last_became_active":"2023-11-26T07:17:32.135078+0000","last_became_peered":"2023-11-26T07:17:32.135078+0000","last_unstale":"2023-11-26T07:17:34.327313+0000","last_undegraded":"2023-11-26T07:17:34.327313+0000","last_fullsized":"2023-11-26T07:17:34.327313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:04.536431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327077+0000","last_change":"2023-11-26T07:17:33.149589+0000","last_active":"2023-11-26T07:17:34.327077+0000","last_peered":"2023-11-26T07:17:34.327077+0000","last_clean":"2023-11-26T07:17:34.327077+0000","last_became_active":"2023-11-26T07:17:33.149228+0000","last_became_peered":"2023-11-26T07:17:33.149228+0000","last_unstale":"2023-11-26T07:17:34.327077+0000","last_undegraded":"2023-11-26T07:17:34.327077+0000","last_fullsized":"2023-11-26T07:17:34.327077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327286+0000","last_change":"2023-11-26T07:17:32.118350+0000","last_active":"2023-11-26T07:17:34.327286+0000","last_peered":"2023-11-26T07:17:34.327286+0000","last_clean":"2023-11-26T07:17:34.327286+0000","last_became_active":"2023-11-26T07:17:32.118040+0000","last_became_peered":"2023-11-26T07:17:32.118040+0000","last_unstale":"2023-11-26T07:17:34.327286+0000","last_undegraded":"2023-11-26T07:17:34.327286+0000","last_fullsized":"2023-11-26T07:17:34.327286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:21.575382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131017+0000","last_change":"2023-11-26T07:17:33.150796+0000","last_active":"2023-11-26T07:17:34.131017+0000","last_peered":"2023-11-26T07:17:34.131017+0000","last_clean":"2023-11-26T07:17:34.131017+0000","last_became_active":"2023-11-26T07:17:33.150543+0000","last_became_peered":"2023-11-26T07:17:33.150543+0000","last_unstale":"2023-11-26T07:17:34.131017+0000","last_undegraded":"2023-11-26T07:17:34.131017+0000","last_fullsized":"2023-11-26T07:17:34.131017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:59:35.145697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.129145+0000","last_change":"2023-11-26T07:17:32.134890+0000","last_active":"2023-11-26T07:17:34.129145+0000","last_peered":"2023-11-26T07:17:34.129145+0000","last_clean":"2023-11-26T07:17:34.129145+0000","last_became_active":"2023-11-26T07:17:32.134621+0000","last_became_peered":"2023-11-26T07:17:32.134621+0000","last_unstale":"2023-11-26T07:17:34.129145+0000","last_undegraded":"2023-11-26T07:17:34.129145+0000","last_fullsized":"2023-11-26T07:17:34.129145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:09.498267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328230+0000","last_change":"2023-11-26T07:17:33.135948+0000","last_active":"2023-11-26T07:17:34.328230+0000","last_peered":"2023-11-26T07:17:34.328230+0000","last_clean":"2023-11-26T07:17:34.328230+0000","last_became_active":"2023-11-26T07:17:33.135636+0000","last_became_peered":"2023-11-26T07:17:33.135636+0000","last_unstale":"2023-11-26T07:17:34.328230+0000","last_undegraded":"2023-11-26T07:17:34.328230+0000","last_fullsized":"2023-11-26T07:17:34.328230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326291+0000","last_change":"2023-11-26T07:17:32.134899+0000","last_active":"2023-11-26T07:17:34.326291+0000","last_peered":"2023-11-26T07:17:34.326291+0000","last_clean":"2023-11-26T07:17:34.326291+0000","last_became_active":"2023-11-26T07:17:32.134566+0000","last_became_peered":"2023-11-26T07:17:32.134566+0000","last_unstale":"2023-11-26T07:17:34.326291+0000","last_undegraded":"2023-11-26T07:17:34.326291+0000","last_fullsized":"2023-11-26T07:17:34.326291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:29.834563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127563+0000","last_change":"2023-11-26T07:17:33.123386+0000","last_active":"2023-11-26T07:17:34.127563+0000","last_peered":"2023-11-26T07:17:34.127563+0000","last_clean":"2023-11-26T07:17:34.127563+0000","last_became_active":"2023-11-26T07:17:33.120220+0000","last_became_peered":"2023-11-26T07:17:33.120220+0000","last_unstale":"2023-11-26T07:17:34.127563+0000","last_undegraded":"2023-11-26T07:17:34.127563+0000","last_fullsized":"2023-11-26T07:17:34.127563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:20.756357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326867+0000","last_change":"2023-11-26T07:17:32.138806+0000","last_active":"2023-11-26T07:17:34.326867+0000","last_peered":"2023-11-26T07:17:34.326867+0000","last_clean":"2023-11-26T07:17:34.326867+0000","last_became_active":"2023-11-26T07:17:32.138612+0000","last_became_peered":"2023-11-26T07:17:32.138612+0000","last_unstale":"2023-11-26T07:17:34.326867+0000","last_undegraded":"2023-11-26T07:17:34.326867+0000","last_fullsized":"2023-11-26T07:17:34.326867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130595+0000","last_change":"2023-11-26T07:17:33.139703+0000","last_active":"2023-11-26T07:17:34.130595+0000","last_peered":"2023-11-26T07:17:34.130595+0000","last_clean":"2023-11-26T07:17:34.130595+0000","last_became_active":"2023-11-26T07:17:33.139347+0000","last_became_peered":"2023-11-26T07:17:33.139347+0000","last_unstale":"2023-11-26T07:17:34.130595+0000","last_undegraded":"2023-11-26T07:17:34.130595+0000","last_fullsized":"2023-11-26T07:17:34.130595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327161+0000","last_change":"2023-11-26T07:17:32.141098+0000","last_active":"2023-11-26T07:17:34.327161+0000","last_peered":"2023-11-26T07:17:34.327161+0000","last_clean":"2023-11-26T07:17:34.327161+0000","last_became_active":"2023-11-26T07:17:32.137814+0000","last_became_peered":"2023-11-26T07:17:32.137814+0000","last_unstale":"2023-11-26T07:17:34.327161+0000","last_undegraded":"2023-11-26T07:17:34.327161+0000","last_fullsized":"2023-11-26T07:17:34.327161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:53:41.643510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130835+0000","last_change":"2023-11-26T07:17:33.124135+0000","last_active":"2023-11-26T07:17:34.130835+0000","last_peered":"2023-11-26T07:17:34.130835+0000","last_clean":"2023-11-26T07:17:34.130835+0000","last_became_active":"2023-11-26T07:17:33.123814+0000","last_became_peered":"2023-11-26T07:17:33.123814+0000","last_unstale":"2023-11-26T07:17:34.130835+0000","last_undegraded":"2023-11-26T07:17:34.130835+0000","last_fullsized":"2023-11-26T07:17:34.130835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326630+0000","last_change":"2023-11-26T07:17:32.140282+0000","last_active":"2023-11-26T07:17:34.326630+0000","last_peered":"2023-11-26T07:17:34.326630+0000","last_clean":"2023-11-26T07:17:34.326630+0000","last_became_active":"2023-11-26T07:17:32.139388+0000","last_became_peered":"2023-11-26T07:17:32.139388+0000","last_unstale":"2023-11-26T07:17:34.326630+0000","last_undegraded":"2023-11-26T07:17:34.326630+0000","last_fullsized":"2023-11-26T07:17:34.326630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:24.546723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127842+0000","last_change":"2023-11-26T07:17:33.140095+0000","last_active":"2023-11-26T07:17:34.127842+0000","last_peered":"2023-11-26T07:17:34.127842+0000","last_clean":"2023-11-26T07:17:34.127842+0000","last_became_active":"2023-11-26T07:17:33.139715+0000","last_became_peered":"2023-11-26T07:17:33.139715+0000","last_unstale":"2023-11-26T07:17:34.127842+0000","last_undegraded":"2023-11-26T07:17:34.127842+0000","last_fullsized":"2023-11-26T07:17:34.127842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326469+0000","last_change":"2023-11-26T07:17:33.132899+0000","last_active":"2023-11-26T07:17:34.326469+0000","last_peered":"2023-11-26T07:17:34.326469+0000","last_clean":"2023-11-26T07:17:34.326469+0000","last_became_active":"2023-11-26T07:17:33.132617+0000","last_became_peered":"2023-11-26T07:17:33.132617+0000","last_unstale":"2023-11-26T07:17:34.326469+0000","last_undegraded":"2023-11-26T07:17:34.326469+0000","last_fullsized":"2023-11-26T07:17:34.326469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326817+0000","last_change":"2023-11-26T07:17:32.138620+0000","last_active":"2023-11-26T07:17:34.326817+0000","last_peered":"2023-11-26T07:17:34.326817+0000","last_clean":"2023-11-26T07:17:34.326817+0000","last_became_active":"2023-11-26T07:17:32.138135+0000","last_became_peered":"2023-11-26T07:17:32.138135+0000","last_unstale":"2023-11-26T07:17:34.326817+0000","last_undegraded":"2023-11-26T07:17:34.326817+0000","last_fullsized":"2023-11-26T07:17:34.326817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327036+0000","last_change":"2023-11-26T07:17:33.136866+0000","last_active":"2023-11-26T07:17:34.327036+0000","last_peered":"2023-11-26T07:17:34.327036+0000","last_clean":"2023-11-26T07:17:34.327036+0000","last_became_active":"2023-11-26T07:17:33.136571+0000","last_became_peered":"2023-11-26T07:17:33.136571+0000","last_unstale":"2023-11-26T07:17:34.327036+0000","last_undegraded":"2023-11-26T07:17:34.327036+0000","last_fullsized":"2023-11-26T07:17:34.327036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:13.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327073+0000","last_change":"2023-11-26T07:17:32.135405+0000","last_active":"2023-11-26T07:17:34.327073+0000","last_peered":"2023-11-26T07:17:34.327073+0000","last_clean":"2023-11-26T07:17:34.327073+0000","last_became_active":"2023-11-26T07:17:32.134722+0000","last_became_peered":"2023-11-26T07:17:32.134722+0000","last_unstale":"2023-11-26T07:17:34.327073+0000","last_undegraded":"2023-11-26T07:17:34.327073+0000","last_fullsized":"2023-11-26T07:17:34.327073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328275+0000","last_change":"2023-11-26T07:17:33.138935+0000","last_active":"2023-11-26T07:17:34.328275+0000","last_peered":"2023-11-26T07:17:34.328275+0000","last_clean":"2023-11-26T07:17:34.328275+0000","last_became_active":"2023-11-26T07:17:33.138684+0000","last_became_peered":"2023-11-26T07:17:33.138684+0000","last_unstale":"2023-11-26T07:17:34.328275+0000","last_undegraded":"2023-11-26T07:17:34.328275+0000","last_fullsized":"2023-11-26T07:17:34.328275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131063+0000","last_change":"2023-11-26T07:17:32.130834+0000","last_active":"2023-11-26T07:17:34.131063+0000","last_peered":"2023-11-26T07:17:34.131063+0000","last_clean":"2023-11-26T07:17:34.131063+0000","last_became_active":"2023-11-26T07:17:32.130525+0000","last_became_peered":"2023-11-26T07:17:32.130525+0000","last_unstale":"2023-11-26T07:17:34.131063+0000","last_undegraded":"2023-11-26T07:17:34.131063+0000","last_fullsized":"2023-11-26T07:17:34.131063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:09.498267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131026+0000","last_change":"2023-11-26T07:17:33.146148+0000","last_active":"2023-11-26T07:17:34.131026+0000","last_peered":"2023-11-26T07:17:34.131026+0000","last_clean":"2023-11-26T07:17:34.131026+0000","last_became_active":"2023-11-26T07:17:33.145838+0000","last_became_peered":"2023-11-26T07:17:33.145838+0000","last_unstale":"2023-11-26T07:17:34.131026+0000","last_undegraded":"2023-11-26T07:17:34.131026+0000","last_fullsized":"2023-11-26T07:17:34.131026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328317+0000","last_change":"2023-11-26T07:17:32.135921+0000","last_active":"2023-11-26T07:17:34.328317+0000","last_peered":"2023-11-26T07:17:34.328317+0000","last_clean":"2023-11-26T07:17:34.328317+0000","last_became_active":"2023-11-26T07:17:32.135634+0000","last_became_peered":"2023-11-26T07:17:32.135634+0000","last_unstale":"2023-11-26T07:17:34.328317+0000","last_undegraded":"2023-11-26T07:17:34.328317+0000","last_fullsized":"2023-11-26T07:17:34.328317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.551435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.161}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.333}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:37.629 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:17:37.629 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-26T07:17:37.629 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-26T07:17:37.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-26T07:17:37.630 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-26T07:17:37.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6e6406640 1 Processor -- start 2023-11-26T07:17:37.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6e6406640 1 -- start start 2023-11-26T07:17:37.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6e6406640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe6e00716a0 0x7fe6e0071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6e6406640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e0071b30 con 0x7fe6e015d3e0 2023-11-26T07:17:37.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6e6406640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e0070920 con 0x7fe6e0072470 2023-11-26T07:17:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6e6406640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e00728e0 con 0x7fe6e00716a0 2023-11-26T07:17:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6dffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe6e015d3e0 0x7fe6e006c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46224/0 (socket says 172.21.15.31:46224) 2023-11-26T07:17:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6dffff640 1 -- 172.21.15.31:0/3650930923 learned_addr learned my addr 172.21.15.31:0/3650930923 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6df7fe640 1 --2- 172.21.15.31:0/3650930923 >> v2:172.21.15.97:3300/0 conn(0x7fe6e00716a0 0x7fe6e0071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1981875927 0 0) 0x7fe6e0071b30 con 0x7fe6e015d3e0 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.805+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6c8003530 con 0x7fe6e015d3e0 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 54782003 0 0) 0x7fe6c8003530 con 0x7fe6e015d3e0 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 >> v2:172.21.15.97:3300/0 conn(0x7fe6e00716a0 msgr2=0x7fe6e0071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 --2- 172.21.15.31:0/3650930923 >> v2:172.21.15.97:3300/0 conn(0x7fe6e00716a0 0x7fe6e0071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 >> v1:172.21.15.97:6789/0 conn(0x7fe6e0072470 legacy=0x7fe6e00b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e015d7c0 con 0x7fe6e015d3e0 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fe6d0001680 con 0x7fe6e015d3e0 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 >> v1:172.21.15.31:6789/0 conn(0x7fe6e015d3e0 legacy=0x7fe6e006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c8003980 0x7fe6c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6c80047d0 0x7fe6c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8005190 0x7fe6c800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00728e0 con 0x7fe6c8003980 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c800c030 con 0x7fe6c8005190 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c800c1f0 con 0x7fe6c80047d0 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6dffff640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c8003980 0x7fe6c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6df7fe640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6c80047d0 0x7fe6c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6e49a3640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8005190 0x7fe6c800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6dffff640 1 -- 172.21.15.31:0/3650930923 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6c80047d0 msgr2=0x7fe6c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6dffff640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6c80047d0 0x7fe6c8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6dffff640 1 -- 172.21.15.31:0/3650930923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8005190 msgr2=0x7fe6c800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:37.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6dffff640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8005190 0x7fe6c800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6dffff640 1 -- 172.21.15.31:0/3650930923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8003530 con 0x7fe6c8003980 2023-11-26T07:17:37.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6df7fe640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6c80047d0 0x7fe6c8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:37.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6e49a3640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8005190 0x7fe6c800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:37.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6e49a3640 1 -- 172.21.15.31:0/3650930923 reap_dead start 2023-11-26T07:17:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.809+0000 7fe6dffff640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c8003980 0x7fe6c8003da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000c1c0 tx=0x7fe6d001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2883f637bd243f66 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0005ae0 con 0x7fe6c8003980 2023-11-26T07:17:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d0005c40 con 0x7fe6c8003980 2023-11-26T07:17:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6deffd640 1 -- 172.21.15.31:0/3650930923 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d00a5a20 con 0x7fe6c8003980 2023-11-26T07:17:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 -- 172.21.15.31:0/3650930923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c8003980 msgr2=0x7fe6c8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c8003980 0x7fe6c8003da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000c1c0 tx=0x7fe6d001cbf0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 -- 172.21.15.31:0/3650930923 shutdown_connections 2023-11-26T07:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 --2- 172.21.15.31:0/3650930923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6c8003980 0x7fe6c8003da0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 -- 172.21.15.31:0/3650930923 >> 172.21.15.31:0/3650930923 conn(0x7fe6e00b0c40 msgr2=0x7fe6e00b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 -- 172.21.15.31:0/3650930923 shutdown_connections 2023-11-26T07:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 -- 172.21.15.31:0/3650930923 wait complete. 2023-11-26T07:17:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 Processor -- start 2023-11-26T07:17:37.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 -- start start 2023-11-26T07:17:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8003980 0x7fe6e01483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6e015c490 0x7fe6e0160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6e015c910 0x7fe6e0160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0148f20 con 0x7fe6e015c910 2023-11-26T07:17:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e015ae40 con 0x7fe6c8003980 2023-11-26T07:17:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.813+0000 7fe6e6406640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0155b40 con 0x7fe6e015c490 2023-11-26T07:17:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6df7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6e015c490 0x7fe6e0160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6dffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8003980 0x7fe6e01483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6e49a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6e015c910 0x7fe6e0160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6dffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8003980 0x7fe6e01483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59450/0 (socket says 172.21.15.31:59450) 2023-11-26T07:17:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6df7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6e015c490 0x7fe6e0160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:47956/0 (socket says 172.21.15.31:47956) 2023-11-26T07:17:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6dffff640 1 -- 172.21.15.31:0/2889269038 learned_addr learned my addr 172.21.15.31:0/2889269038 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6e49a3640 1 -- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6e015c490 msgr2=0x7fe6e0160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6e49a3640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6e015c490 0x7fe6e0160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6e49a3640 1 -- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8003980 msgr2=0x7fe6e01483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6e49a3640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8003980 0x7fe6e01483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6e49a3640 1 -- 172.21.15.31:0/2889269038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0155d90 con 0x7fe6e015c910 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6df7fe640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6e015c490 0x7fe6e0160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6dffff640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8003980 0x7fe6e01483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6e49a3640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6e015c910 0x7fe6e0160b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe6e006bef0 tx=0x7fe6d800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6dcff9640 1 -- 172.21.15.31:0/2889269038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d800eed0 con 0x7fe6e015c910 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6dcff9640 1 -- 172.21.15.31:0/2889269038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d8003980 con 0x7fe6e015c910 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6e6406640 1 -- 172.21.15.31:0/2889269038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e0156070 con 0x7fe6e015c910 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6dcff9640 1 -- 172.21.15.31:0/2889269038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8003d10 con 0x7fe6e015c910 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6e6406640 1 -- 172.21.15.31:0/2889269038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e0156630 con 0x7fe6e015c910 2023-11-26T07:17:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6e6406640 1 -- 172.21.15.31:0/2889269038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e004e850 con 0x7fe6e015c910 2023-11-26T07:17:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.817+0000 7fe6dcff9640 1 -- 172.21.15.31:0/2889269038 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe6d80050d0 con 0x7fe6e015c910 2023-11-26T07:17:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.821+0000 7fe6dcff9640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe6cc0664a0 0x7fe6cc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.821+0000 7fe6dcff9640 1 -- 172.21.15.31:0/2889269038 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6d80839a0 con 0x7fe6e015c910 2023-11-26T07:17:37.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.821+0000 7fe6dffff640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe6cc0664a0 0x7fe6cc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:37.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.821+0000 7fe6dffff640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe6cc0664a0 0x7fe6cc068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0001940 tx=0x7fe6d00b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:37.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:37.825+0000 7fe6dcff9640 1 -- 172.21.15.31:0/2889269038 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe6d8050ec0 con 0x7fe6e015c910 2023-11-26T07:17:38.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.017+0000 7fe6e6406640 1 -- 172.21.15.31:0/2889269038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe6e00ce7e0 con 0x7fe6e015c910 2023-11-26T07:17:38.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.021+0000 7fe6dcff9640 1 -- 172.21.15.31:0/2889269038 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164432 (secure 0 0 0) 0x7fe6d80544e0 con 0x7fe6e015c910 2023-11-26T07:17:38.027 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:38.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 -- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe6cc0664a0 msgr2=0x7fe6cc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe6cc0664a0 0x7fe6cc068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0001940 tx=0x7fe6d00b1040 comp rx=0 tx=0).stop 2023-11-26T07:17:38.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 -- 172.21.15.31:0/2889269038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6e015c910 msgr2=0x7fe6e0160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6e015c910 0x7fe6e0160b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe6e006bef0 tx=0x7fe6d800eb90 comp rx=0 tx=0).stop 2023-11-26T07:17:38.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 -- 172.21.15.31:0/2889269038 shutdown_connections 2023-11-26T07:17:38.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe6cc0664a0 0x7fe6cc068960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6e015c910 0x7fe6e0160b40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe6e015c490 0x7fe6e0160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 --2- 172.21.15.31:0/2889269038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe6c8003980 0x7fe6e01483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 -- 172.21.15.31:0/2889269038 >> 172.21.15.31:0/2889269038 conn(0x7fe6e00b0c40 msgr2=0x7fe6e00b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 -- 172.21.15.31:0/2889269038 shutdown_connections 2023-11-26T07:17:38.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.025+0000 7fe6e6406640 1 -- 172.21.15.31:0/2889269038 wait complete. 2023-11-26T07:17:38.047 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3008114855},{"type":"v1","addr":"172.21.15.97:6833","nonce":3008114855}]},"active_addr":"172.21.15.97:6833/3008114855","active_change":"2023-11-26T07:17:12.293862+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":4191921753}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":4223698495}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3784615915}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3104042147}]}]} 2023-11-26T07:17:38.050 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-26T07:17:38.050 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-26T07:17:38.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:38.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.189+0000 7f6fc7e9c640 1 Processor -- start 2023-11-26T07:17:38.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.189+0000 7f6fc7e9c640 1 -- start start 2023-11-26T07:17:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.189+0000 7f6fc7e9c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6fc0144cf0 0x7f6fc0070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.189+0000 7f6fc7e9c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fc00b44b0 0x7f6fc00cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.189+0000 7f6fc7e9c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc00ce5b0 con 0x7f6fc00ce160 2023-11-26T07:17:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.189+0000 7f6fc7e9c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc00b5d10 con 0x7f6fc00b44b0 2023-11-26T07:17:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.189+0000 7f6fc7e9c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc00b4c70 con 0x7f6fc0144cf0 2023-11-26T07:17:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.189+0000 7f6fc5437640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fc00b44b0 0x7f6fc00cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.189+0000 7f6fc5437640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6fc00b44b0 0x7f6fc00cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59296/0 (socket says 172.21.15.31:59296) 2023-11-26T07:17:38.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.189+0000 7f6fc5437640 1 -- 172.21.15.31:0/3922316890 learned_addr learned my addr 172.21.15.31:0/3922316890 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:38.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc5c38640 1 --2- 172.21.15.31:0/3922316890 >> v2:172.21.15.97:3300/0 conn(0x7f6fc0144cf0 0x7f6fc0070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc5437640 1 -- 172.21.15.31:0/3922316890 >> v2:172.21.15.97:3300/0 conn(0x7f6fc0144cf0 msgr2=0x7f6fc0070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc5437640 1 --2- 172.21.15.31:0/3922316890 >> v2:172.21.15.97:3300/0 conn(0x7f6fc0144cf0 0x7f6fc0070ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc5437640 1 -- 172.21.15.31:0/3922316890 >> v1:172.21.15.31:6789/0 conn(0x7f6fc00ce160 legacy=0x7f6fc015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc5437640 1 -- 172.21.15.31:0/3922316890 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00b4f50 con 0x7f6fc00b44b0 2023-11-26T07:17:38.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc5437640 1 --2- 172.21.15.31:0/3922316890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc00b44b0 0x7f6fc00cdc20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6fc006f970 tx=0x7f6fb001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16239ae89ecc1ad server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc4c36640 1 -- 172.21.15.31:0/3922316890 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00aa430 con 0x7f6fc00b44b0 2023-11-26T07:17:38.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc4c36640 1 -- 172.21.15.31:0/3922316890 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb00aa590 con 0x7f6fc00b44b0 2023-11-26T07:17:38.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc4c36640 1 -- 172.21.15.31:0/3922316890 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00aa920 con 0x7f6fc00b44b0 2023-11-26T07:17:38.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/3922316890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc00b44b0 msgr2=0x7f6fc00cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc7e9c640 1 --2- 172.21.15.31:0/3922316890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc00b44b0 0x7f6fc00cdc20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6fc006f970 tx=0x7f6fb001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/3922316890 shutdown_connections 2023-11-26T07:17:38.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc7e9c640 1 --2- 172.21.15.31:0/3922316890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc00b44b0 0x7f6fc00cdc20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc7e9c640 1 --2- 172.21.15.31:0/3922316890 >> v2:172.21.15.97:3300/0 conn(0x7f6fc0144cf0 0x7f6fc0070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.193+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/3922316890 >> 172.21.15.31:0/3922316890 conn(0x7f6fc00b2ea0 msgr2=0x7f6fc00b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/3922316890 shutdown_connections 2023-11-26T07:17:38.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/3922316890 wait complete. 2023-11-26T07:17:38.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 Processor -- start 2023-11-26T07:17:38.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 -- start start 2023-11-26T07:17:38.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc00b44b0 0x7f6fc0142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6fc00ce160 0x7f6fc00b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc0144cf0 0x7f6fc015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc015c680 con 0x7f6fc0144cf0 2023-11-26T07:17:38.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc01432a0 con 0x7f6fc00b44b0 2023-11-26T07:17:38.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc015c030 con 0x7f6fc00ce160 2023-11-26T07:17:38.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc6439640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc0144cf0 0x7f6fc015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc6439640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc0144cf0 0x7f6fc015cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59300/0 (socket says 172.21.15.31:59300) 2023-11-26T07:17:38.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc6439640 1 -- 172.21.15.31:0/279674201 learned_addr learned my addr 172.21.15.31:0/279674201 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:38.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc5437640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6fc00ce160 0x7f6fc00b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc5c38640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc00b44b0 0x7f6fc0142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc6439640 1 -- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6fc00ce160 msgr2=0x7f6fc00b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc6439640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6fc00ce160 0x7f6fc00b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc6439640 1 -- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc00b44b0 msgr2=0x7f6fc0142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc6439640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc00b44b0 0x7f6fc0142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc6439640 1 -- 172.21.15.31:0/279674201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb00094d0 con 0x7f6fc0144cf0 2023-11-26T07:17:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc6439640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc0144cf0 0x7f6fc015cb10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800c770 tx=0x7f6fb800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fb67fc640 1 -- 172.21.15.31:0/279674201 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8003b80 con 0x7f6fc0144cf0 2023-11-26T07:17:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fb67fc640 1 -- 172.21.15.31:0/279674201 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb8003ce0 con 0x7f6fc0144cf0 2023-11-26T07:17:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.197+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/279674201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc015c310 con 0x7f6fc0144cf0 2023-11-26T07:17:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.201+0000 7f6fb67fc640 1 -- 172.21.15.31:0/279674201 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb800aa90 con 0x7f6fc0144cf0 2023-11-26T07:17:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.201+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/279674201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc0142f90 con 0x7f6fc0144cf0 2023-11-26T07:17:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.201+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/279674201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc013e810 con 0x7f6fc0144cf0 2023-11-26T07:17:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.201+0000 7f6fb67fc640 1 -- 172.21.15.31:0/279674201 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6fb800abf0 con 0x7f6fc0144cf0 2023-11-26T07:17:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.201+0000 7f6fb67fc640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fa4066470 0x7f6fa4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.201+0000 7f6fb67fc640 1 -- 172.21.15.31:0/279674201 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fb8083950 con 0x7f6fc0144cf0 2023-11-26T07:17:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.201+0000 7f6fc5c38640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fa4066470 0x7f6fa4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.205+0000 7f6fc5c38640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fa4066470 0x7f6fa4068930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0166210 tx=0x7f6fbc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.205+0000 7f6fb67fc640 1 -- 172.21.15.31:0/279674201 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6fb8050f10 con 0x7f6fc0144cf0 2023-11-26T07:17:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/279674201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fc00ce5b0 con 0x7f6fc0144cf0 2023-11-26T07:17:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fb67fc640 1 -- 172.21.15.31:0/279674201 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f6fc00ce5b0 con 0x7f6fc0144cf0 2023-11-26T07:17:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:38.369 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fa4066470 msgr2=0x7f6fa4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fa4066470 0x7f6fa4068930 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0166210 tx=0x7f6fbc008040 comp rx=0 tx=0).stop 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/279674201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc0144cf0 msgr2=0x7f6fc015cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc0144cf0 0x7f6fc015cb10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800c770 tx=0x7f6fb800cc30 comp rx=0 tx=0).stop 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/279674201 shutdown_connections 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fa4066470 0x7f6fa4068930 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fc0144cf0 0x7f6fc015cb10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6fc00ce160 0x7f6fc00b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 --2- 172.21.15.31:0/279674201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6fc00b44b0 0x7f6fc0142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/279674201 >> 172.21.15.31:0/279674201 conn(0x7f6fc00b2ea0 msgr2=0x7f6fc013e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/279674201 shutdown_connections 2023-11-26T07:17:38.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.365+0000 7f6fc7e9c640 1 -- 172.21.15.31:0/279674201 wait complete. 2023-11-26T07:17:38.389 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-26T07:17:38.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:38.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea3a68640 1 Processor -- start 2023-11-26T07:17:38.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea3a68640 1 -- start start 2023-11-26T07:17:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea3a68640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4e9c0ac630 0x7f4e9c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea3a68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 0x7f4e9c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea3a68640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c145060 con 0x7f4e9c144c80 2023-11-26T07:17:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea3a68640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9c071fd0 con 0x7f4e9c0ac630 2023-11-26T07:17:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea3a68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c071450 con 0x7f4e9c070c20 2023-11-26T07:17:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea1804640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f4e9c144c80 0x7f4e9c0abf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:56950/0 (socket says 172.21.15.31:56950) 2023-11-26T07:17:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea1804640 1 -- 172.21.15.31:0/164370053 learned_addr learned my addr 172.21.15.31:0/164370053 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea1003640 1 --2- 172.21.15.31:0/164370053 >> v2:172.21.15.31:3300/0 conn(0x7f4e9c0ac630 0x7f4e9c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea2005640 1 --2- 172.21.15.31:0/164370053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 0x7f4e9c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea1003640 1 -- 172.21.15.31:0/164370053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 msgr2=0x7f4e9c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea1003640 1 --2- 172.21.15.31:0/164370053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 0x7f4e9c06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea1003640 1 -- 172.21.15.31:0/164370053 >> v1:172.21.15.97:6789/0 conn(0x7f4e9c144c80 legacy=0x7f4e9c0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea1003640 1 -- 172.21.15.31:0/164370053 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c071730 con 0x7f4e9c0ac630 2023-11-26T07:17:38.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.529+0000 7f4ea0802640 1 -- 172.21.15.31:0/164370053 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009013730 0 0) 0x7f4e9c145060 con 0x7f4e9c144c80 2023-11-26T07:17:38.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea1003640 1 --2- 172.21.15.31:0/164370053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c0ac630 0x7f4e9c0706e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0097c0 tx=0x7f4e8c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3072a1caa9ee1907 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea0802640 1 -- 172.21.15.31:0/164370053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0ab650 con 0x7f4e9c0ac630 2023-11-26T07:17:38.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea0802640 1 -- 172.21.15.31:0/164370053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c0ab7b0 con 0x7f4e9c0ac630 2023-11-26T07:17:38.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea0802640 1 -- 172.21.15.31:0/164370053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0abb40 con 0x7f4e9c0ac630 2023-11-26T07:17:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea3a68640 1 -- 172.21.15.31:0/164370053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c0ac630 msgr2=0x7f4e9c0706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea3a68640 1 --2- 172.21.15.31:0/164370053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c0ac630 0x7f4e9c0706e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0097c0 tx=0x7f4e8c01c450 comp rx=0 tx=0).stop 2023-11-26T07:17:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea3a68640 1 -- 172.21.15.31:0/164370053 shutdown_connections 2023-11-26T07:17:38.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea3a68640 1 --2- 172.21.15.31:0/164370053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 0x7f4e9c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea3a68640 1 --2- 172.21.15.31:0/164370053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c0ac630 0x7f4e9c0706e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea3a68640 1 -- 172.21.15.31:0/164370053 >> 172.21.15.31:0/164370053 conn(0x7f4e9c0b0c40 msgr2=0x7f4e9c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea3a68640 1 -- 172.21.15.31:0/164370053 shutdown_connections 2023-11-26T07:17:38.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.533+0000 7f4ea3a68640 1 -- 172.21.15.31:0/164370053 wait complete. 2023-11-26T07:17:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea3a68640 1 Processor -- start 2023-11-26T07:17:38.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea3a68640 1 -- start start 2023-11-26T07:17:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea3a68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 0x7f4e9c1f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea3a68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e9c0ac630 0x7f4e9c13ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea3a68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c144c80 0x7f4e9c13d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea3a68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c13b230 con 0x7f4e9c144c80 2023-11-26T07:17:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea3a68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c145430 con 0x7f4e9c0ac630 2023-11-26T07:17:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea3a68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c142030 con 0x7f4e9c070c20 2023-11-26T07:17:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea2005640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c144c80 0x7f4e9c13d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea2005640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c144c80 0x7f4e9c13d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59328/0 (socket says 172.21.15.31:59328) 2023-11-26T07:17:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea2005640 1 -- 172.21.15.31:0/2315530858 learned_addr learned my addr 172.21.15.31:0/2315530858 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea1003640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e9c0ac630 0x7f4e9c13ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea1804640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 0x7f4e9c1f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea2005640 1 -- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 msgr2=0x7f4e9c1f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea2005640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 0x7f4e9c1f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea2005640 1 -- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e9c0ac630 msgr2=0x7f4e9c13ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea2005640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e9c0ac630 0x7f4e9c13ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea2005640 1 -- 172.21.15.31:0/2315530858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c009420 con 0x7f4e9c144c80 2023-11-26T07:17:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea1804640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 0x7f4e9c1f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea2005640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c144c80 0x7f4e9c13d800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400efc0 tx=0x7f4e9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4e927fc640 1 -- 172.21.15.31:0/2315530858 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94014480 con 0x7f4e9c144c80 2023-11-26T07:17:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4e927fc640 1 -- 172.21.15.31:0/2315530858 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9400f040 con 0x7f4e9c144c80 2023-11-26T07:17:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea3a68640 1 -- 172.21.15.31:0/2315530858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c141790 con 0x7f4e9c144c80 2023-11-26T07:17:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4e927fc640 1 -- 172.21.15.31:0/2315530858 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94014820 con 0x7f4e9c144c80 2023-11-26T07:17:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.537+0000 7f4ea3a68640 1 -- 172.21.15.31:0/2315530858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c141d50 con 0x7f4e9c144c80 2023-11-26T07:17:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.541+0000 7f4ea3a68640 1 -- 172.21.15.31:0/2315530858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c0b1ba0 con 0x7f4e9c144c80 2023-11-26T07:17:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.541+0000 7f4e927fc640 1 -- 172.21.15.31:0/2315530858 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4e94014980 con 0x7f4e9c144c80 2023-11-26T07:17:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.541+0000 7f4e927fc640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4e800664a0 0x7f4e80068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.541+0000 7f4e927fc640 1 -- 172.21.15.31:0/2315530858 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e940849c0 con 0x7f4e9c144c80 2023-11-26T07:17:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.541+0000 7f4ea1804640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4e800664a0 0x7f4e80068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.541+0000 7f4ea1804640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4e800664a0 0x7f4e80068960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e98007fd0 tx=0x7f4e98009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.545+0000 7f4e927fc640 1 -- 172.21.15.31:0/2315530858 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e94051ed0 con 0x7f4e9c144c80 2023-11-26T07:17:38.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.701+0000 7f4ea3a68640 1 -- 172.21.15.31:0/2315530858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e9c04e850 con 0x7f4e9c144c80 2023-11-26T07:17:38.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4e927fc640 1 -- 172.21.15.31:0/2315530858 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f4e94018090 con 0x7f4e9c144c80 2023-11-26T07:17:38.708 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:38.708 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 -- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4e800664a0 msgr2=0x7f4e80068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4e800664a0 0x7f4e80068960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e98007fd0 tx=0x7f4e98009040 comp rx=0 tx=0).stop 2023-11-26T07:17:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 -- 172.21.15.31:0/2315530858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c144c80 msgr2=0x7f4e9c13d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c144c80 0x7f4e9c13d800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400efc0 tx=0x7f4e9400c490 comp rx=0 tx=0).stop 2023-11-26T07:17:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 -- 172.21.15.31:0/2315530858 shutdown_connections 2023-11-26T07:17:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4e800664a0 0x7f4e80068960 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e9c144c80 0x7f4e9c13d800 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e9c0ac630 0x7f4e9c13ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 --2- 172.21.15.31:0/2315530858 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e9c070c20 0x7f4e9c1f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 -- 172.21.15.31:0/2315530858 >> 172.21.15.31:0/2315530858 conn(0x7f4e9c0b0c40 msgr2=0x7f4e9c1425b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 -- 172.21.15.31:0/2315530858 shutdown_connections 2023-11-26T07:17:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.705+0000 7f4ea3a68640 1 -- 172.21.15.31:0/2315530858 wait complete. 2023-11-26T07:17:38.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-26T07:17:38.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-26T07:17:38.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-26T07:17:38.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-26T07:17:38.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-26T07:17:38.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-26T07:17:38.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-26T07:17:38.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-26T07:17:38.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.973+0000 7f8b64ad6640 1 Processor -- start 2023-11-26T07:17:38.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.973+0000 7f39cd4ad640 1 Processor -- start 2023-11-26T07:17:38.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.973+0000 7f39cd4ad640 1 -- start start 2023-11-26T07:17:38.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.973+0000 7f8b64ad6640 1 -- start start 2023-11-26T07:17:38.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.973+0000 7f8b64ad6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b6014af40 0x7f8b60070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.973+0000 7f8b64ad6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 0x7f8b600b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b5dd9b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 0x7f8b600b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b5ed9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8b60071af0 0x7f8b600b3fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:56964/0 (socket says 172.21.15.31:56964) 2023-11-26T07:17:38.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b5ed9d640 1 -- 172.21.15.31:0/751188045 learned_addr learned my addr 172.21.15.31:0/751188045 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:38.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b64ad6640 1 -- 172.21.15.31:0/751188045 --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b600b2350 con 0x7f8b60071af0 2023-11-26T07:17:38.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b64ad6640 1 -- 172.21.15.31:0/751188045 --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b60070a50 con 0x7f8b6014af40 2023-11-26T07:17:38.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b64ad6640 1 -- 172.21.15.31:0/751188045 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b60070d30 con 0x7f8b6014b320 2023-11-26T07:17:38.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39cd4ad640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39c80ae650 0x7f39c8167de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39cd4ad640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c80acda0 con 0x7f39c80ac8a0 2023-11-26T07:17:38.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39cd4ad640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c80ad080 con 0x7f39c8143a90 2023-11-26T07:17:38.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39cd4ad640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80ad360 con 0x7f39c80ae650 2023-11-26T07:17:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c6ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f39c8143a90 0x7f39c8072290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:56978/0 (socket says 172.21.15.31:56978) 2023-11-26T07:17:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c6ffd640 1 -- 172.21.15.31:0/2216374363 learned_addr learned my addr 172.21.15.31:0/2216374363 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c77fe640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39c80ae650 0x7f39c8167de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887575337 0 0) 0x7f39c80ad080 con 0x7f39c8143a90 2023-11-26T07:17:38.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b5d59a640 1 -- 172.21.15.31:0/751188045 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614492983 0 0) 0x7f8b600b2350 con 0x7f8b60071af0 2023-11-26T07:17:38.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b5d59a640 1 -- 172.21.15.31:0/751188045 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b48003530 con 0x7f8b60071af0 2023-11-26T07:17:38.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b5dd9b640 1 -- 172.21.15.31:0/751188045 >> v2:172.21.15.31:3300/0 conn(0x7f8b6014af40 msgr2=0x7f8b60070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b5dd9b640 1 --2- 172.21.15.31:0/751188045 >> v2:172.21.15.31:3300/0 conn(0x7f8b6014af40 0x7f8b60070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b4003530 con 0x7f39c8143a90 2023-11-26T07:17:38.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1666501945 0 0) 0x7f39b4003530 con 0x7f39c8143a90 2023-11-26T07:17:38.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39c80ae650 msgr2=0x7f39c8167de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39c80ae650 0x7f39c8167de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 >> v1:172.21.15.31:6789/0 conn(0x7f39c80ac8a0 legacy=0x7f39c80adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c80ad640 con 0x7f39c8143a90 2023-11-26T07:17:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 <== mon.1 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f39b8001680 con 0x7f39c8143a90 2023-11-26T07:17:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 >> v1:172.21.15.97:6789/0 conn(0x7f39c8143a90 legacy=0x7f39c8072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b4003980 0x7f39b4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39b40047d0 0x7f39b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39b4005190 0x7f39b400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c80ad360 con 0x7f39b4003980 2023-11-26T07:17:38.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b400bf80 con 0x7f39b4005190 2023-11-26T07:17:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b400c140 con 0x7f39b40047d0 2023-11-26T07:17:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f39c6ffd640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b4003980 0x7f39b4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f39c77fe640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39b4005190 0x7f39b400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f39c6ffd640 1 -- 172.21.15.31:0/2216374363 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39b40047d0 msgr2=0x7f39b4004c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:17:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f39c6ffd640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39b40047d0 0x7f39b4004c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f39c6ffd640 1 -- 172.21.15.31:0/2216374363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39b4005190 msgr2=0x7f39b400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f39c6ffd640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39b4005190 0x7f39b400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f39c6ffd640 1 -- 172.21.15.31:0/2216374363 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4003530 con 0x7f39b4003980 2023-11-26T07:17:38.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f226ca79640 1 Processor -- start 2023-11-26T07:17:38.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f226ca79640 1 -- start start 2023-11-26T07:17:38.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b5dd9b640 1 -- 172.21.15.31:0/751188045 >> v1:172.21.15.97:6789/0 conn(0x7f8b60071af0 legacy=0x7f8b600b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.977+0000 7f8b5dd9b640 1 -- 172.21.15.31:0/751188045 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60071010 con 0x7f8b6014b320 2023-11-26T07:17:38.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f226ca79640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22681595f0 0x7f22680b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f226ca79640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2268070470 0x7f2268070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f226ca79640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22680b4ef0 con 0x7f226815cf50 2023-11-26T07:17:38.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f226ca79640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22680b3bf0 con 0x7f2268070470 2023-11-26T07:17:38.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f226ca79640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22680cd910 con 0x7f22681595f0 2023-11-26T07:17:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f225dd9b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22681595f0 0x7f22680b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f226659c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f226815cf50 0x7f22680cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:56992/0 (socket says 172.21.15.31:56992) 2023-11-26T07:17:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f226659c640 1 -- 172.21.15.31:0/1283304885 learned_addr learned my addr 172.21.15.31:0/1283304885 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.981+0000 7f2266d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2268070470 0x7f2268070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f2265d9b640 1 -- 172.21.15.31:0/1283304885 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993153575 0 0) 0x7f22680b4ef0 con 0x7f226815cf50 2023-11-26T07:17:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f2265d9b640 1 -- 172.21.15.31:0/1283304885 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2254005740 con 0x7f226815cf50 2023-11-26T07:17:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f225dd9b640 1 -- 172.21.15.31:0/1283304885 >> v2:172.21.15.97:3300/0 conn(0x7f2268070470 msgr2=0x7f2268070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f225dd9b640 1 --2- 172.21.15.31:0/1283304885 >> v2:172.21.15.97:3300/0 conn(0x7f2268070470 0x7f2268070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f225dd9b640 1 -- 172.21.15.31:0/1283304885 >> v1:172.21.15.97:6789/0 conn(0x7f226815cf50 legacy=0x7f22680cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f225dd9b640 1 -- 172.21.15.31:0/1283304885 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2268070e60 con 0x7f22681595f0 2023-11-26T07:17:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f39c6ffd640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b4003980 0x7f39b4003da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c190 tx=0x7f39b801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee5e861945fb3ad server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f8b5dd9b640 1 --2- 172.21.15.31:0/751188045 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 0x7f8b600b2f60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8b500097c0 tx=0x7f8b5001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67b4ce18ffbc10d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f8b5d59a640 1 -- 172.21.15.31:0/751188045 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b500ab650 con 0x7f8b6014b320 2023-11-26T07:17:38.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f8b5d59a640 1 -- 172.21.15.31:0/751188045 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b500ab7b0 con 0x7f8b6014b320 2023-11-26T07:17:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f8b5d59a640 1 -- 172.21.15.31:0/751188045 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b500abb40 con 0x7f8b6014b320 2023-11-26T07:17:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80a5b10 con 0x7f39b4003980 2023-11-26T07:17:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f7a99aba640 1 Processor -- start 2023-11-26T07:17:38.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b80a5c70 con 0x7f39b4003980 2023-11-26T07:17:38.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f7a99aba640 1 -- start start 2023-11-26T07:17:38.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f8b64ad6640 1 -- 172.21.15.31:0/751188045 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 msgr2=0x7f8b600b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f8b64ad6640 1 --2- 172.21.15.31:0/751188045 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 0x7f8b600b2f60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8b500097c0 tx=0x7f8b5001c490 comp rx=0 tx=0).stop 2023-11-26T07:17:38.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2216374363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b4003980 msgr2=0x7f39b4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f39c67fc640 1 -- 172.21.15.31:0/2216374363 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80a5b10 con 0x7f39b4003980 2023-11-26T07:17:38.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f39cd4ad640 1 --2- 172.21.15.31:0/2216374363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39b4003980 0x7f39b4003da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39b800c190 tx=0x7f39b801cc10 comp rx=0 tx=0).stop 2023-11-26T07:17:38.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.985+0000 7f39c77fe640 1 -- 172.21.15.31:0/2216374363 reap_dead start 2023-11-26T07:17:38.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2216374363 shutdown_connections 2023-11-26T07:17:38.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2216374363 >> 172.21.15.31:0/2216374363 conn(0x7f39c80b0c20 msgr2=0x7f39c80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f8b64ad6640 1 -- 172.21.15.31:0/751188045 shutdown_connections 2023-11-26T07:17:38.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f8b64ad6640 1 --2- 172.21.15.31:0/751188045 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 0x7f8b600b2f60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f8b64ad6640 1 --2- 172.21.15.31:0/751188045 >> v2:172.21.15.31:3300/0 conn(0x7f8b6014af40 0x7f8b60070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f8b64ad6640 1 -- 172.21.15.31:0/751188045 >> 172.21.15.31:0/751188045 conn(0x7f8b600b0d90 msgr2=0x7f8b600b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:38.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2216374363 shutdown_connections 2023-11-26T07:17:38.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a99aba640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a8c00aab0 0x7f7a8c053ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:38.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a99aba640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8c005ec0 con 0x7f7a8c053060 2023-11-26T07:17:38.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a99aba640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8c006160 con 0x7f7a8c0df260 2023-11-26T07:17:38.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a99aba640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8c006440 con 0x7f7a8c00aab0 2023-11-26T07:17:38.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f8b64ad6640 1 -- 172.21.15.31:0/751188045 shutdown_connections 2023-11-26T07:17:38.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f8b64ad6640 1 -- 172.21.15.31:0/751188045 wait complete. 2023-11-26T07:17:38.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a98ab8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a8c00aab0 0x7f7a8c053ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:38.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a93fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7a8c053060 0x7f7a8c007200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46250/0 (socket says 172.21.15.31:46250) 2023-11-26T07:17:38.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a93fff640 1 -- 172.21.15.31:0/2199344529 learned_addr learned my addr 172.21.15.31:0/2199344529 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:38.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f225dd9b640 1 --2- 172.21.15.31:0/1283304885 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22681595f0 0x7f22680b49b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f225000a960 tx=0x7f225001c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc2e397f4ccc4fb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:38.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2216374363 wait complete. 2023-11-26T07:17:38.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f2265d9b640 1 -- 172.21.15.31:0/1283304885 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22500a6620 con 0x7f22681595f0 2023-11-26T07:17:38.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f2265d9b640 1 -- 172.21.15.31:0/1283304885 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22500a6780 con 0x7f22681595f0 2023-11-26T07:17:38.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f2265d9b640 1 -- 172.21.15.31:0/1283304885 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22500a6b10 con 0x7f22681595f0 2023-11-26T07:17:38.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a98ab8640 1 -- 172.21.15.31:0/2199344529 >> v1:172.21.15.97:6789/0 conn(0x7f7a8c0df260 legacy=0x7f7a8c04fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a98ab8640 1 -- 172.21.15.31:0/2199344529 >> v1:172.21.15.31:6789/0 conn(0x7f7a8c053060 legacy=0x7f7a8c007200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a937fe640 1 -- 172.21.15.31:0/2199344529 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148821120 0 0) 0x7f7a8c006160 con 0x7f7a8c0df260 2023-11-26T07:17:38.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f7a98ab8640 1 -- 172.21.15.31:0/2199344529 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c006720 con 0x7f7a8c00aab0 2023-11-26T07:17:38.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f226ca79640 1 -- 172.21.15.31:0/1283304885 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22681595f0 msgr2=0x7f22680b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:38.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.989+0000 7f226ca79640 1 --2- 172.21.15.31:0/1283304885 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22681595f0 0x7f22680b49b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f225000a960 tx=0x7f225001c750 comp rx=0 tx=0).stop 2023-11-26T07:17:38.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f226ca79640 1 -- 172.21.15.31:0/1283304885 shutdown_connections 2023-11-26T07:17:38.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f226ca79640 1 --2- 172.21.15.31:0/1283304885 >> v2:172.21.15.97:3300/0 conn(0x7f2268070470 0x7f2268070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:38.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f226ca79640 1 --2- 172.21.15.31:0/1283304885 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f22681595f0 0x7f22680b49b0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f225000a960 tx=0x7f225001c750 comp rx=0 tx=0).stop 2023-11-26T07:17:38.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f226ca79640 1 -- 172.21.15.31:0/1283304885 >> 172.21.15.31:0/1283304885 conn(0x7f22680b0d90 msgr2=0x7f22680b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f8b64ad6640 1 Processor -- start 2023-11-26T07:17:39.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f8b64ad6640 1 -- start start 2023-11-26T07:17:39.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f2b8756f640 1 Processor -- start 2023-11-26T07:17:39.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f39cd4ad640 1 Processor -- start 2023-11-26T07:17:39.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f39cd4ad640 1 -- start start 2023-11-26T07:17:39.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f226ca79640 1 -- 172.21.15.31:0/1283304885 shutdown_connections 2023-11-26T07:17:39.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f226ca79640 1 -- 172.21.15.31:0/1283304885 wait complete. 2023-11-26T07:17:39.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f8b64ad6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b60071af0 0x7f8b600ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f7a98ab8640 1 --2- 172.21.15.31:0/2199344529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c00aab0 0x7f7a8c053ee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7a94050190 tx=0x7f7a94096b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e74c8d6708821fc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f2b8756f640 1 -- start start 2023-11-26T07:17:39.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f8b64ad6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6014af40 0x7f8b600c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f8b64ad6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 0x7f8b6013ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f8b5dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6014af40 0x7f8b600c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f8b5dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6014af40 0x7f8b600c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59520/0 (socket says 172.21.15.31:59520) 2023-11-26T07:17:39.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b5dd9b640 1 -- 172.21.15.31:0/1083332818 learned_addr learned my addr 172.21.15.31:0/1083332818 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b5e59c640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b60071af0 0x7f8b600ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b60148430 con 0x7f8b60071af0 2023-11-26T07:17:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39cd4ad640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39b4003980 0x7f39c813f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226ca79640 1 Processor -- start 2023-11-26T07:17:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f7a937fe640 1 -- 172.21.15.31:0/2199344529 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9411f9a0 con 0x7f7a8c00aab0 2023-11-26T07:17:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f7a937fe640 1 -- 172.21.15.31:0/2199344529 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9411fb00 con 0x7f7a8c00aab0 2023-11-26T07:17:39.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f7a937fe640 1 -- 172.21.15.31:0/2199344529 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94127440 con 0x7f7a8c00aab0 2023-11-26T07:17:39.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.993+0000 7f2b8756f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b8006c960 0x7f2b80165ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7f710c8640 1 Processor -- start 2023-11-26T07:17:39.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39cd4ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c8141800 0x7f39c81568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39cd4ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c815c040 0x7f39c815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39cd4ad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c815ca00 con 0x7f39c815c040 2023-11-26T07:17:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39cd4ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8156de0 con 0x7f39c8141800 2023-11-26T07:17:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39cd4ad640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c8140380 con 0x7f39b4003980 2023-11-26T07:17:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7f710c8640 1 -- start start 2023-11-26T07:17:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c815c040 0x7f39c815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39be7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c8141800 0x7f39c81568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39be7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c8141800 0x7f39c81568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59536/0 (socket says 172.21.15.31:59536) 2023-11-26T07:17:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39be7fc640 1 -- 172.21.15.31:0/2990608427 learned_addr learned my addr 172.21.15.31:0/2990608427 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39be7fc640 1 -- 172.21.15.31:0/2990608427 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39b4003980 msgr2=0x7f39c813f6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:17:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39be7fc640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39b4003980 0x7f39c813f6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39be7fc640 1 -- 172.21.15.31:0/2990608427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c815c040 msgr2=0x7f39c815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39be7fc640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39c815c040 0x7f39c815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39be7fc640 1 -- 172.21.15.31:0/2990608427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c813fdc0 con 0x7f39c8141800 2023-11-26T07:17:39.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226ca79640 1 -- start start 2023-11-26T07:17:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7a99aba640 1 -- 172.21.15.31:0/2199344529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c00aab0 msgr2=0x7f7a8c053ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7a99aba640 1 --2- 172.21.15.31:0/2199344529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c00aab0 0x7f7a8c053ee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7a94050190 tx=0x7f7a94096b40 comp rx=0 tx=0).stop 2023-11-26T07:17:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7a99aba640 1 -- 172.21.15.31:0/2199344529 shutdown_connections 2023-11-26T07:17:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7a99aba640 1 --2- 172.21.15.31:0/2199344529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c00aab0 0x7f7a8c053ee0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7a99aba640 1 -- 172.21.15.31:0/2199344529 >> 172.21.15.31:0/2199344529 conn(0x7f7a8c009f90 msgr2=0x7f7a8c00a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8656d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2b8014b090 0x7f2b8006c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:57018/0 (socket says 172.21.15.31:57018) 2023-11-26T07:17:39.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b85d6c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b8006c960 0x7f2b80165ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b85d6c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b8006c960 0x7f2b80165ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48046/0 (socket says 172.21.15.31:48046) 2023-11-26T07:17:39.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8756f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8014b470 con 0x7f2b80070400 2023-11-26T07:17:39.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8756f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b80165a10 con 0x7f2b8014b090 2023-11-26T07:17:39.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b86d6e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2b80070400 0x7f2b800cc6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46252/0 (socket says 172.21.15.31:46252) 2023-11-26T07:17:39.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8756f640 1 -- 172.21.15.31:0/954415669 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b80071a70 con 0x7f2b8006c960 2023-11-26T07:17:39.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b86d6e640 1 -- 172.21.15.31:0/954415669 learned_addr learned my addr 172.21.15.31:0/954415669 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:39.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7a99aba640 1 -- 172.21.15.31:0/2199344529 shutdown_connections 2023-11-26T07:17:39.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215986120 0 0) 0x7f2b80165a10 con 0x7f2b8014b090 2023-11-26T07:17:39.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b6c003530 con 0x7f2b8014b090 2023-11-26T07:17:39.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39be7fc640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c8141800 0x7f39c81568a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39b000c3c0 tx=0x7f39b000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39bffff640 1 -- 172.21.15.31:0/2990608427 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b00052d0 con 0x7f39c8141800 2023-11-26T07:17:39.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226ca79640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2268070470 0x7f226814e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226ca79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f226815cf50 0x7f226814cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 12631693 0 0) 0x7f2b6c003530 con 0x7f2b8014b090 2023-11-26T07:17:39.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b8006c960 msgr2=0x7f2b80165ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b8006c960 0x7f2b80165ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 >> v1:172.21.15.31:6789/0 conn(0x7f2b80070400 legacy=0x7f2b800cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b80070fe0 con 0x7f2b8014b090 2023-11-26T07:17:39.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 <== mon.1 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f2b7c001680 con 0x7f2b8014b090 2023-11-26T07:17:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7a99aba640 1 -- 172.21.15.31:0/2199344529 wait complete. 2023-11-26T07:17:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226ca79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226814d2d0 0x7f2268157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226ca79640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2268153030 con 0x7f226815cf50 2023-11-26T07:17:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226ca79640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2268152720 con 0x7f226814d2d0 2023-11-26T07:17:39.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226ca79640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226814d740 con 0x7f2268070470 2023-11-26T07:17:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226659c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2268070470 0x7f226814e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226659c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2268070470 0x7f226814e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48060/0 (socket says 172.21.15.31:48060) 2023-11-26T07:17:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226659c640 1 -- 172.21.15.31:0/3850924629 learned_addr learned my addr 172.21.15.31:0/3850924629 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7f710c8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f6404fa50 0x7f7f6404fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b60147780 con 0x7f8b6014af40 2023-11-26T07:17:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600cae10 con 0x7f8b6014b320 2023-11-26T07:17:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b5ed9d640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 0x7f8b6013ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39bffff640 1 -- 172.21.15.31:0/2990608427 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b0005430 con 0x7f39c8141800 2023-11-26T07:17:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c81400a0 con 0x7f39c8141800 2023-11-26T07:17:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c80cd5b0 con 0x7f39c8141800 2023-11-26T07:17:39.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2266d9d640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226814d2d0 0x7f2268157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226659c640 1 -- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226814d2d0 msgr2=0x7f2268157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226659c640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f226814d2d0 0x7f2268157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 >> v1:172.21.15.97:6789/0 conn(0x7f2b8014b090 legacy=0x7f2b8006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b6c003980 0x7f2b6c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6c0047d0 0x7f2b6c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c005190 0x7f2b6c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b80071a70 con 0x7f2b6c003980 2023-11-26T07:17:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c00c000 con 0x7f2b6c0047d0 2023-11-26T07:17:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c00c1c0 con 0x7f2b6c005190 2023-11-26T07:17:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f2b86d6e640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c005190 0x7f2b6c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f7f710c8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 0x7f7f640071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7f710c8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7f6400c080 0x7f7f640e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7f710c8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f64005ec0 con 0x7f7f6400c080 2023-11-26T07:17:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7f710c8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f64006d20 con 0x7f7f6404fa50 2023-11-26T07:17:39.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7f710c8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f64054360 con 0x7f7f640df360 2023-11-26T07:17:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b5ed9d640 1 -- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6014af40 msgr2=0x7f8b600c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b5ed9d640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6014af40 0x7f8b600c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b5ed9d640 1 -- 172.21.15.31:0/1083332818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b60071af0 msgr2=0x7f8b600ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b5ed9d640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b60071af0 0x7f8b600ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b5ed9d640 1 -- 172.21.15.31:0/1083332818 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b50009420 con 0x7f8b6014b320 2023-11-26T07:17:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b5dd9b640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6014af40 0x7f8b600c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b5ed9d640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 0x7f8b6013ff20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c3c0 tx=0x7f8b5400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b4effd640 1 -- 172.21.15.31:0/1083332818 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54005cf0 con 0x7f8b6014b320 2023-11-26T07:17:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b4effd640 1 -- 172.21.15.31:0/1083332818 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54005e50 con 0x7f8b6014b320 2023-11-26T07:17:39.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b4effd640 1 -- 172.21.15.31:0/1083332818 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b540072d0 con 0x7f8b6014b320 2023-11-26T07:17:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b60207080 con 0x7f8b6014b320 2023-11-26T07:17:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b60147d70 con 0x7f8b6014b320 2023-11-26T07:17:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226659c640 1 -- 172.21.15.31:0/3850924629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f226815cf50 msgr2=0x7f226814cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226659c640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f226815cf50 0x7f226814cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:38.997+0000 7f226659c640 1 -- 172.21.15.31:0/3850924629 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225000a5c0 con 0x7f2268070470 2023-11-26T07:17:39.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b600b48c0 con 0x7f8b6014b320 2023-11-26T07:17:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f2b8656d640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b6c003980 0x7f2b6c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f2b85d6c640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6c0047d0 0x7f2b6c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7f708c7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7f6400c080 0x7f7f640e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7f6b7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 0x7f7f640071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f39bffff640 1 -- 172.21.15.31:0/2990608427 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b0007260 con 0x7f39c8141800 2023-11-26T07:17:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f39ac000fd0 con 0x7f39c8141800 2023-11-26T07:17:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f2b85d6c640 1 -- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c005190 msgr2=0x7f2b6c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f2b85d6c640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b6c005190 0x7f2b6c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f2b85d6c640 1 -- 172.21.15.31:0/954415669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b6c003980 msgr2=0x7f2b6c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7f6b7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 0x7f7f640071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48074/0 (socket says 172.21.15.31:48074) 2023-11-26T07:17:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7f708c7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7f6400c080 0x7f7f640e2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59382/0 (socket says 172.21.15.31:59382) 2023-11-26T07:17:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7f708c7640 1 -- 172.21.15.31:0/489200417 learned_addr learned my addr 172.21.15.31:0/489200417 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7a99aba640 1 Processor -- start 2023-11-26T07:17:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f226659c640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2268070470 0x7f226814e8c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f225800d5b0 tx=0x7f225800da70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f2b85d6c640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b6c003980 0x7f2b6c003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f2b85d6c640 1 -- 172.21.15.31:0/954415669 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b80070fe0 con 0x7f2b6c0047d0 2023-11-26T07:17:39.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7f6bfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7f6404fa50 0x7f7f6404fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7a99aba640 1 -- start start 2023-11-26T07:17:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7fd1f3fff640 1 Processor -- start 2023-11-26T07:17:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f2b8656d640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b6c003980 0x7f2b6c003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:39.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7fd1f3fff640 1 -- start start 2023-11-26T07:17:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f39bffff640 1 -- 172.21.15.31:0/2990608427 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f39b00073c0 con 0x7f39c8141800 2023-11-26T07:17:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f7a99aba640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a8c00aab0 0x7f7a8c048830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f8b4effd640 1 -- 172.21.15.31:0/1083332818 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8b54007430 con 0x7f8b6014b320 2023-11-26T07:17:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a99aba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c053060 0x7f7a8c045af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a99aba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c0df260 0x7f7a8c046030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a99aba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0e1120 con 0x7f7a8c0df260 2023-11-26T07:17:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a99aba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c04a7e0 con 0x7f7a8c053060 2023-11-26T07:17:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a99aba640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c04ada0 con 0x7f7a8c00aab0 2023-11-26T07:17:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f225f7fe640 1 -- 172.21.15.31:0/3850924629 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225800f910 con 0x7f2268070470 2023-11-26T07:17:39.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2268158510 con 0x7f2268070470 2023-11-26T07:17:39.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2268152c50 con 0x7f2268070470 2023-11-26T07:17:39.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.001+0000 7f225f7fe640 1 -- 172.21.15.31:0/3850924629 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225800fa70 con 0x7f2268070470 2023-11-26T07:17:39.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f225f7fe640 1 -- 172.21.15.31:0/3850924629 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225800f910 con 0x7f2268070470 2023-11-26T07:17:39.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c053060 0x7f7a8c045af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a93fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c053060 0x7f7a8c045af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59574/0 (socket says 172.21.15.31:59574) 2023-11-26T07:17:39.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a93fff640 1 -- 172.21.15.31:0/2499008137 learned_addr learned my addr 172.21.15.31:0/2499008137 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f1b6ebaa640 1 Processor -- start 2023-11-26T07:17:39.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f8b4effd640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b400664c0 0x7f8b40068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f8b4effd640 1 -- 172.21.15.31:0/1083332818 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b54083cf0 con 0x7f8b6014b320 2023-11-26T07:17:39.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f8b5e59c640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b400664c0 0x7f8b40068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f8b4effd640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] conn(0x7f8b4006e6e0 0x7f8b40070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f8b4effd640 1 -- 172.21.15.31:0/1083332818 --> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b400711b0 con 0x7f8b4006e6e0 2023-11-26T07:17:39.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f8b4effd640 1 -- 172.21.15.31:0/1083332818 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b54084190 con 0x7f8b6014b320 2023-11-26T07:17:39.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f1b6ebaa640 1 -- start start 2023-11-26T07:17:39.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f39bffff640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f39a80664f0 0x7f39a80689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f8b5dd9b640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] conn(0x7f8b4006e6e0 0x7f8b40070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7fd1f3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1ec0e3260 0x7fd1ec056fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7fd1f3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1ec0543b0 0x7fd1ec03c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7fd1f3fff640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0e3640 con 0x7fd1ec03e8d0 2023-11-26T07:17:39.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7fd1f3fff640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec0578c0 con 0x7fd1ec0e3260 2023-11-26T07:17:39.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7fd1f3fff640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec057ba0 con 0x7fd1ec0543b0 2023-11-26T07:17:39.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a93fff640 1 -- 172.21.15.31:0/2499008137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a8c00aab0 msgr2=0x7f7a8c048830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a93fff640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a8c00aab0 0x7f7a8c048830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a93fff640 1 -- 172.21.15.31:0/2499008137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c0df260 msgr2=0x7f7a8c046030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-26T07:17:39.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a93fff640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8c0df260 0x7f7a8c046030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a93fff640 1 -- 172.21.15.31:0/2499008137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9404f7e0 con 0x7f7a8c053060 2023-11-26T07:17:39.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f39bffff640 1 -- 172.21.15.31:0/2990608427 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39b0082ec0 con 0x7f39c8141800 2023-11-26T07:17:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f39bffff640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] conn(0x7f39a806e710 0x7f39a8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f39bffff640 1 -- 172.21.15.31:0/2990608427 --> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f39a80711e0 con 0x7f39a806e710 2023-11-26T07:17:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f39bffff640 1 -- 172.21.15.31:0/2990608427 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39b0083240 con 0x7f39c8141800 2023-11-26T07:17:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f224c000fd0 con 0x7f2268070470 2023-11-26T07:17:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f8b5e59c640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b400664c0 0x7f8b40068980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800a400 tx=0x7f8b5800d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f39c6ffd640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f39a80664f0 0x7f39a80689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7fd1f27fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd1ec03e8d0 0x7fd1ec053ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:57032/0 (socket says 172.21.15.31:57032) 2023-11-26T07:17:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7fd1f27fc640 1 -- 172.21.15.31:0/4252366815 learned_addr learned my addr 172.21.15.31:0/4252366815 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a93fff640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c053060 0x7f7a8c045af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009e70 tx=0x7f7a8800c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7fd1f37fe640 1 --2- 172.21.15.31:0/4252366815 >> v2:172.21.15.97:3300/0 conn(0x7fd1ec0543b0 0x7fd1ec03c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a917fa640 1 -- 172.21.15.31:0/2499008137 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a880078f0 con 0x7f7a8c053060 2023-11-26T07:17:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a917fa640 1 -- 172.21.15.31:0/2499008137 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a88007a50 con 0x7f7a8c053060 2023-11-26T07:17:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f39c77fe640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] conn(0x7f39a806e710 0x7f39a8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a917fa640 1 -- 172.21.15.31:0/2499008137 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88007de0 con 0x7f7a8c053060 2023-11-26T07:17:39.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c0492a0 con 0x7f7a8c053060 2023-11-26T07:17:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8c0e0d60 con 0x7f7a8c053060 2023-11-26T07:17:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f225f7fe640 1 -- 172.21.15.31:0/3850924629 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f225800fbe0 con 0x7f2268070470 2023-11-26T07:17:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f39c6ffd640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f39a80664f0 0x7f39a80689b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39b80ab000 tx=0x7f39b80a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f39c77fe640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] conn(0x7f39a806e710 0x7f39a8070b30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f8b5dd9b640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] conn(0x7f8b4006e6e0 0x7f8b40070b00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7fd1f37fe640 1 -- 172.21.15.31:0/4252366815 >> v2:172.21.15.31:3300/0 conn(0x7fd1ec0e3260 msgr2=0x7fd1ec056fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7fd1f37fe640 1 --2- 172.21.15.31:0/4252366815 >> v2:172.21.15.31:3300/0 conn(0x7fd1ec0e3260 0x7fd1ec056fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7fd1f37fe640 1 -- 172.21.15.31:0/4252366815 >> v1:172.21.15.97:6789/0 conn(0x7fd1ec03e8d0 legacy=0x7fd1ec053ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7fd1f37fe640 1 -- 172.21.15.31:0/4252366815 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec057e80 con 0x7fd1ec0543b0 2023-11-26T07:17:39.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a8c006ce0 con 0x7f7a8c053060 2023-11-26T07:17:39.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b6ebaa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1b680b4040 0x7f1b680b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b6ebaa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b68162190 0x7f1b681625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b6ebaa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b680b2590 con 0x7f1b680b3c60 2023-11-26T07:17:39.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b6ebaa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b680b2870 con 0x7f1b680b4040 2023-11-26T07:17:39.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b6ebaa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b680b2b50 con 0x7f1b68162190 2023-11-26T07:17:39.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f225f7fe640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2248066490 0x7f2248068950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b67fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1b680b4040 0x7f1b680b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b67fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1b680b4040 0x7f1b680b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59406/0 (socket says 172.21.15.31:59406) 2023-11-26T07:17:39.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b67fff640 1 -- 172.21.15.31:0/2835537830 learned_addr learned my addr 172.21.15.31:0/2835537830 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b6d147640 1 --2- 172.21.15.31:0/2835537830 >> v2:172.21.15.97:3300/0 conn(0x7f1b68162190 0x7f1b681625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f225f7fe640 1 -- 172.21.15.31:0/3850924629 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2258009040 con 0x7f2268070470 2023-11-26T07:17:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f39bffff640 1 -- 172.21.15.31:0/2990608427 <== osd.1 v2:172.21.15.31:6800/845128234 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f39a80711e0 con 0x7f39a806e710 2023-11-26T07:17:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f2b85d6c640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6c0047d0 0x7f2b6c004c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2b80072660 tx=0x7f2b7001c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4499384abc19b1ff server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f225f7fe640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] conn(0x7f224806e540 0x7f2248070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f225f7fe640 1 -- 172.21.15.31:0/3850924629 --> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2248071010 con 0x7f224806e540 2023-11-26T07:17:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f225f7fe640 1 -- 172.21.15.31:0/3850924629 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2258086340 con 0x7f2268070470 2023-11-26T07:17:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f225dd9b640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2248066490 0x7f2248068950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7f6bfff640 1 -- 172.21.15.31:0/489200417 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 msgr2=0x7f7f640071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7f6bfff640 1 --2- 172.21.15.31:0/489200417 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 0x7f7f640071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7f6bfff640 1 -- 172.21.15.31:0/489200417 >> v2:172.21.15.31:3300/0 conn(0x7f7f6400c080 msgr2=0x7f7f640e2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7f6bfff640 1 --2- 172.21.15.31:0/489200417 >> v2:172.21.15.31:3300/0 conn(0x7f7f6400c080 0x7f7f640e2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.005+0000 7f7f6bfff640 1 -- 172.21.15.31:0/489200417 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f64050370 con 0x7f7f6404fa50 2023-11-26T07:17:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b67fff640 1 -- 172.21.15.31:0/2835537830 >> v2:172.21.15.97:3300/0 conn(0x7f1b68162190 msgr2=0x7f1b681625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b67fff640 1 --2- 172.21.15.31:0/2835537830 >> v2:172.21.15.97:3300/0 conn(0x7f1b68162190 0x7f1b681625a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b67fff640 1 -- 172.21.15.31:0/2835537830 >> v1:172.21.15.97:6789/0 conn(0x7f1b680b3c60 legacy=0x7f1b680ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f1b67fff640 1 -- 172.21.15.31:0/2835537830 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b680b2e30 con 0x7f1b680b4040 2023-11-26T07:17:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b700a9410 con 0x7f2b6c0047d0 2023-11-26T07:17:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b700a9570 con 0x7f2b6c0047d0 2023-11-26T07:17:39.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f2b8556b640 1 -- 172.21.15.31:0/954415669 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b700a9960 con 0x7f2b6c0047d0 2023-11-26T07:17:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f7f6bfff640 1 --2- 172.21.15.31:0/489200417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6404fa50 0x7f7f6404fe30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b380 tx=0x7f7f6001c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=982d107c8c78e120 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f7a917fa640 1 -- 172.21.15.31:0/2499008137 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7a88009070 con 0x7f7a8c053060 2023-11-26T07:17:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f2266d9d640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] conn(0x7f224806e540 0x7f2248070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f8b4effd640 1 -- 172.21.15.31:0/1083332818 <== osd.0 v2:172.21.15.31:6816/4191585440 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8b400711b0 con 0x7f8b4006e6e0 2023-11-26T07:17:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f225dd9b640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2248066490 0x7f2248068950 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f22500ae000 tx=0x7f2250004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f2b8756f640 1 -- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6c0047d0 msgr2=0x7f2b6c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f2b8756f640 1 --2- 172.21.15.31:0/954415669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b6c0047d0 0x7f2b6c004c50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2b80072660 tx=0x7f2b7001c930 comp rx=0 tx=0).stop 2023-11-26T07:17:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f2b86d6e640 1 -- 172.21.15.31:0/954415669 reap_dead start 2023-11-26T07:17:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.009+0000 7f2b8756f640 1 -- 172.21.15.31:0/954415669 shutdown_connections 2023-11-26T07:17:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f2b8756f640 1 -- 172.21.15.31:0/954415669 >> 172.21.15.31:0/954415669 conn(0x7f2b8006df30 msgr2=0x7f2b8006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f2b8756f640 1 -- 172.21.15.31:0/954415669 shutdown_connections 2023-11-26T07:17:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7a917fa640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a78066470 0x7f7a78068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7fd1f37fe640 1 --2- 172.21.15.31:0/4252366815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1ec0543b0 0x7fd1ec03c070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4050190 tx=0x7fd1f4099520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f33d2f7aa15433 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7a917fa640 1 -- 172.21.15.31:0/2499008137 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a880864e0 con 0x7f7a8c053060 2023-11-26T07:17:39.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f2b8756f640 1 -- 172.21.15.31:0/954415669 wait complete. 2023-11-26T07:17:39.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f2266d9d640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] conn(0x7f224806e540 0x7f2248070960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7a98ab8640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a78066470 0x7f7a78068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7a917fa640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] conn(0x7f7a7806e690 0x7f7a78070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7fd1f1ffb640 1 -- 172.21.15.31:0/4252366815 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4129640 con 0x7fd1ec0543b0 2023-11-26T07:17:39.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7fd1f1ffb640 1 -- 172.21.15.31:0/4252366815 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4125050 con 0x7fd1ec0543b0 2023-11-26T07:17:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7fd1f1ffb640 1 -- 172.21.15.31:0/4252366815 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4129a20 con 0x7fd1ec0543b0 2023-11-26T07:17:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f6affd640 1 -- 172.21.15.31:0/489200417 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600ab600 con 0x7f7f6404fa50 2023-11-26T07:17:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f6affd640 1 -- 172.21.15.31:0/489200417 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f600a7050 con 0x7f7f6404fa50 2023-11-26T07:17:39.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f6affd640 1 -- 172.21.15.31:0/489200417 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600ab9c0 con 0x7f7f6404fa50 2023-11-26T07:17:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7a992b9640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] conn(0x7f7a7806e690 0x7f7a78070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f710c8640 1 -- 172.21.15.31:0/489200417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6404fa50 msgr2=0x7f7f6404fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f710c8640 1 --2- 172.21.15.31:0/489200417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6404fa50 0x7f7f6404fe30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400b380 tx=0x7f7f6001c3b0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7fd1f3fff640 1 -- 172.21.15.31:0/4252366815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1ec0543b0 msgr2=0x7fd1ec03c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7fd1f3fff640 1 --2- 172.21.15.31:0/4252366815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1ec0543b0 0x7fd1ec03c070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4050190 tx=0x7fd1f4099520 comp rx=0 tx=0).stop 2023-11-26T07:17:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7a98ab8640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a78066470 0x7f7a78068930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00d830 tx=0x7f7a9406a400 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7a917fa640 1 -- 172.21.15.31:0/2499008137 --> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a78071160 con 0x7f7a7806e690 2023-11-26T07:17:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7a917fa640 1 -- 172.21.15.31:0/2499008137 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a88086890 con 0x7f7a8c053060 2023-11-26T07:17:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7a992b9640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] conn(0x7f7a7806e690 0x7f7a78070ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f710c8640 1 -- 172.21.15.31:0/489200417 shutdown_connections 2023-11-26T07:17:39.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f710c8640 1 --2- 172.21.15.31:0/489200417 >> v2:172.21.15.31:3300/0 conn(0x7f7f6400c080 0x7f7f640e2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f710c8640 1 --2- 172.21.15.31:0/489200417 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 0x7f7f640071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f710c8640 1 --2- 172.21.15.31:0/489200417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6404fa50 0x7f7f6404fe30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f710c8640 1 -- 172.21.15.31:0/489200417 >> 172.21.15.31:0/489200417 conn(0x7f7f64009f90 msgr2=0x7f7f6400a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f710c8640 1 -- 172.21.15.31:0/489200417 shutdown_connections 2023-11-26T07:17:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f7f710c8640 1 -- 172.21.15.31:0/489200417 wait complete. 2023-11-26T07:17:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7fd1f3fff640 1 -- 172.21.15.31:0/4252366815 shutdown_connections 2023-11-26T07:17:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7fd1f3fff640 1 --2- 172.21.15.31:0/4252366815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1ec0543b0 0x7fd1ec03c070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7fd1f3fff640 1 --2- 172.21.15.31:0/4252366815 >> v2:172.21.15.31:3300/0 conn(0x7fd1ec0e3260 0x7fd1ec056fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7fd1f3fff640 1 -- 172.21.15.31:0/4252366815 >> 172.21.15.31:0/4252366815 conn(0x7fd1ec058ed0 msgr2=0x7fd1ec0592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f1b67fff640 1 --2- 172.21.15.31:0/2835537830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b680b4040 0x7f1b680b1fb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b680b1cd0 tx=0x7f1b5401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8243b23bcbc848dc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f1b677fe640 1 -- 172.21.15.31:0/2835537830 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b540aa650 con 0x7f1b680b4040 2023-11-26T07:17:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f1b677fe640 1 -- 172.21.15.31:0/2835537830 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b540aa7b0 con 0x7f1b680b4040 2023-11-26T07:17:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f1b677fe640 1 -- 172.21.15.31:0/2835537830 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b540aab40 con 0x7f1b680b4040 2023-11-26T07:17:39.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f7a917fa640 1 -- 172.21.15.31:0/2499008137 <== osd.3 v2:172.21.15.31:6824/3143140069 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7a78071160 con 0x7f7a7806e690 2023-11-26T07:17:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7fd1f3fff640 1 -- 172.21.15.31:0/4252366815 shutdown_connections 2023-11-26T07:17:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7fd1f3fff640 1 -- 172.21.15.31:0/4252366815 wait complete. 2023-11-26T07:17:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.013+0000 7f2b8756f640 1 Processor -- start 2023-11-26T07:17:39.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f2b8756f640 1 -- start start 2023-11-26T07:17:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2835537830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b680b4040 msgr2=0x7f1b680b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f1b6ebaa640 1 --2- 172.21.15.31:0/2835537830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b680b4040 0x7f1b680b1fb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b680b1cd0 tx=0x7f1b5401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2835537830 shutdown_connections 2023-11-26T07:17:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f1b6ebaa640 1 --2- 172.21.15.31:0/2835537830 >> v2:172.21.15.97:3300/0 conn(0x7f1b68162190 0x7f1b681625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f1b6ebaa640 1 --2- 172.21.15.31:0/2835537830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b680b4040 0x7f1b680b1fb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2835537830 >> 172.21.15.31:0/2835537830 conn(0x7f1b680b0cb0 msgr2=0x7f1b680b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2835537830 shutdown_connections 2023-11-26T07:17:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f2b8756f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b6c0047d0 0x7f2b80164290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f7f710c8640 1 Processor -- start 2023-11-26T07:17:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7fd1f3fff640 1 Processor -- start 2023-11-26T07:17:39.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f2b8756f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b8016aac0 0x7f2b8016af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f2b8756f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b800b1c60 0x7f2b800b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f2b8756f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8016b480 con 0x7f2b6c0047d0 2023-11-26T07:17:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f2b8756f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b800b2620 con 0x7f2b800b1c60 2023-11-26T07:17:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f2b8756f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b80145020 con 0x7f2b8016aac0 2023-11-26T07:17:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f2b86d6e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b800b1c60 0x7f2b800b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f2b86d6e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b800b1c60 0x7f2b800b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59598/0 (socket says 172.21.15.31:59598) 2023-11-26T07:17:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f2b86d6e640 1 -- 172.21.15.31:0/3755204495 learned_addr learned my addr 172.21.15.31:0/3755204495 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f7f710c8640 1 -- start start 2023-11-26T07:17:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7fd1f3fff640 1 -- start start 2023-11-26T07:17:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2835537830 wait complete. 2023-11-26T07:17:39.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.017+0000 7f225f7fe640 1 -- 172.21.15.31:0/3850924629 <== osd.2 v2:172.21.15.31:6802/2371417753 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2248071010 con 0x7f224806e540 2023-11-26T07:17:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b86d6e640 1 -- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b8016aac0 msgr2=0x7f2b8016af40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-26T07:17:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b86d6e640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b8016aac0 0x7f2b8016af40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b86d6e640 1 -- 172.21.15.31:0/3755204495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b6c0047d0 msgr2=0x7f2b80164290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b86d6e640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b6c0047d0 0x7f2b80164290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b86d6e640 1 -- 172.21.15.31:0/3755204495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b800b2840 con 0x7f2b800b1c60 2023-11-26T07:17:39.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f1b6ebaa640 1 Processor -- start 2023-11-26T07:17:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1ec03e8d0 0x7fd1ec0e37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f3fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1ec0543b0 0x7fd1ec0e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0e3260 0x7fd1ec17c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f3fff640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0cf4f0 con 0x7fd1ec0e3260 2023-11-26T07:17:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f3fff640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0e5df0 con 0x7fd1ec03e8d0 2023-11-26T07:17:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f3fff640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0e60d0 con 0x7fd1ec0543b0 2023-11-26T07:17:39.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f710c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f6400c080 0x7f7f640d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1ec03e8d0 0x7fd1ec0e37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f27fc640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1ec0543b0 0x7fd1ec0e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f27fc640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1ec0543b0 0x7fd1ec0e3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48106/0 (socket says 172.21.15.31:48106) 2023-11-26T07:17:39.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f27fc640 1 -- 172.21.15.31:0/2303662172 learned_addr learned my addr 172.21.15.31:0/2303662172 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f710c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6404fa50 0x7f7f640d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f6400c080 0x7f7f640d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f6400c080 0x7f7f640d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59430/0 (socket says 172.21.15.31:59430) 2023-11-26T07:17:39.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f710c8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 0x7f7f640d6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f37fe640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0e3260 0x7fd1ec17c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f1b6ebaa640 1 -- start start 2023-11-26T07:17:39.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b86d6e640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b800b1c60 0x7f2b800b20e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b800cee40 tx=0x7f2b7800f5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f6bfff640 1 -- 172.21.15.31:0/2515986859 learned_addr learned my addr 172.21.15.31:0/2515986859 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640d69c0 con 0x7f7f6400c080 2023-11-26T07:17:39.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640d4150 con 0x7f7f6404fa50 2023-11-26T07:17:39.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f640d4430 con 0x7f7f640df360 2023-11-26T07:17:39.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f27fc640 1 -- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1ec03e8d0 msgr2=0x7fd1ec0e37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f27fc640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1ec03e8d0 0x7fd1ec0e37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b76ffd640 1 -- 172.21.15.31:0/3755204495 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7800fd90 con 0x7f2b800b1c60 2023-11-26T07:17:39.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b76ffd640 1 -- 172.21.15.31:0/3755204495 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b780133f0 con 0x7f2b800b1c60 2023-11-26T07:17:39.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b76ffd640 1 -- 172.21.15.31:0/3755204495 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b780137a0 con 0x7f2b800b1c60 2023-11-26T07:17:39.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b8756f640 1 -- 172.21.15.31:0/3755204495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b800b2b20 con 0x7f2b800b1c60 2023-11-26T07:17:39.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f27fc640 1 -- 172.21.15.31:0/2303662172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0e3260 msgr2=0x7fd1ec17c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f27fc640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0e3260 0x7fd1ec17c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7fd1f27fc640 1 -- 172.21.15.31:0/2303662172 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f404f7e0 con 0x7fd1ec0543b0 2023-11-26T07:17:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f2b8756f640 1 -- 172.21.15.31:0/3755204495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b800b30e0 con 0x7f2b800b1c60 2023-11-26T07:17:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f708c7640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 0x7f7f640d6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f708c7640 1 -- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6404fa50 msgr2=0x7f7f640d01e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-26T07:17:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f708c7640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f6404fa50 0x7f7f640d01e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f708c7640 1 -- 172.21.15.31:0/2515986859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f6400c080 msgr2=0x7f7f640d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f708c7640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f6400c080 0x7f7f640d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f7f708c7640 1 -- 172.21.15.31:0/2515986859 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f60009420 con 0x7f7f640df360 2023-11-26T07:17:39.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7fd1f2ffd640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1ec03e8d0 0x7fd1ec0e37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:39.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7fd1f37fe640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1ec0e3260 0x7fd1ec17c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:39.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f2b8756f640 1 -- 172.21.15.31:0/3755204495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b8006c250 con 0x7f2b800b1c60 2023-11-26T07:17:39.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f7f708c7640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 0x7f7f640d6480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c04fb60 tx=0x7f7f6c068000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7fd1f27fc640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1ec0543b0 0x7fd1ec0e3d10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4005890 tx=0x7fd1e400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7fd1db7fe640 1 -- 172.21.15.31:0/2303662172 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4005cf0 con 0x7fd1ec0543b0 2023-11-26T07:17:39.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7fd1db7fe640 1 -- 172.21.15.31:0/2303662172 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e4005e50 con 0x7fd1ec0543b0 2023-11-26T07:17:39.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7fd1db7fe640 1 -- 172.21.15.31:0/2303662172 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40072b0 con 0x7fd1ec0543b0 2023-11-26T07:17:39.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec0e63b0 con 0x7fd1ec0543b0 2023-11-26T07:17:39.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec0e6970 con 0x7fd1ec0543b0 2023-11-26T07:17:39.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.021+0000 7f1b6ebaa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b680b3c60 0x7f1b68158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b6ebaa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b68162190 0x7f1b68162ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b6ebaa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b681595e0 0x7f1b6809a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b6ebaa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681592f0 con 0x7f1b681595e0 2023-11-26T07:17:39.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b6ebaa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6815a5d0 con 0x7f1b68162190 2023-11-26T07:17:39.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b6ebaa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6815a870 con 0x7f1b680b3c60 2023-11-26T07:17:39.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b67fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b68162190 0x7f1b68162ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b6d147640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b681595e0 0x7f1b6809a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b67fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b68162190 0x7f1b68162ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59634/0 (socket says 172.21.15.31:59634) 2023-11-26T07:17:39.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b67fff640 1 -- 172.21.15.31:0/2838322297 learned_addr learned my addr 172.21.15.31:0/2838322297 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1ec058440 con 0x7fd1ec0543b0 2023-11-26T07:17:39.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f2b76ffd640 1 -- 172.21.15.31:0/3755204495 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2b78013900 con 0x7f2b800b1c60 2023-11-26T07:17:39.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f7f68ff9640 1 -- 172.21.15.31:0/2515986859 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c06bcb0 con 0x7f7f640df360 2023-11-26T07:17:39.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f7f68ff9640 1 -- 172.21.15.31:0/2515986859 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c098070 con 0x7f7f640df360 2023-11-26T07:17:39.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f7f68ff9640 1 -- 172.21.15.31:0/2515986859 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c094a90 con 0x7f7f640df360 2023-11-26T07:17:39.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f640d4710 con 0x7f7f640df360 2023-11-26T07:17:39.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f640d4cd0 con 0x7f7f640df360 2023-11-26T07:17:39.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b6c946640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b680b3c60 0x7f1b68158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b67fff640 1 -- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b680b3c60 msgr2=0x7f1b68158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b67fff640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b680b3c60 0x7f1b68158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b67fff640 1 -- 172.21.15.31:0/2838322297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b681595e0 msgr2=0x7f1b6809a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b67fff640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b681595e0 0x7f1b6809a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.025+0000 7f1b67fff640 1 -- 172.21.15.31:0/2838322297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b540094d0 con 0x7f1b68162190 2023-11-26T07:17:39.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f1b67fff640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b68162190 0x7f1b68162ee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1b540190e0 tx=0x7f1b5401ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f2b76ffd640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2b640664c0 0x7f2b64068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f64054850 con 0x7f7f640df360 2023-11-26T07:17:39.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f2b76ffd640 1 -- 172.21.15.31:0/3755204495 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b78084be0 con 0x7f2b800b1c60 2023-11-26T07:17:39.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f1b657fa640 1 -- 172.21.15.31:0/2838322297 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b540a9bf0 con 0x7f1b68162190 2023-11-26T07:17:39.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f2b76ffd640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] conn(0x7f2b6406e6e0 0x7f2b64070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f2b76ffd640 1 -- 172.21.15.31:0/3755204495 --> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b640711b0 con 0x7f2b6406e6e0 2023-11-26T07:17:39.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f2b76ffd640 1 -- 172.21.15.31:0/3755204495 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b78084f50 con 0x7f2b800b1c60 2023-11-26T07:17:39.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7fd1db7fe640 1 -- 172.21.15.31:0/2303662172 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd1e4007410 con 0x7fd1ec0543b0 2023-11-26T07:17:39.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f2b8656d640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2b640664c0 0x7f2b64068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f2b85d6c640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] conn(0x7f2b6406e6e0 0x7f2b64070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f2b85d6c640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] conn(0x7f2b6406e6e0 0x7f2b64070b00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f2b8656d640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2b640664c0 0x7f2b64068980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c001130 tx=0x7f2b7c0079f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f7f68ff9640 1 -- 172.21.15.31:0/2515986859 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7f6c09a030 con 0x7f7f640df360 2023-11-26T07:17:39.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6815ab50 con 0x7f1b68162190 2023-11-26T07:17:39.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f1b657fa640 1 -- 172.21.15.31:0/2838322297 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b540aae90 con 0x7f1b68162190 2023-11-26T07:17:39.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f1b657fa640 1 -- 172.21.15.31:0/2838322297 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b540a9bf0 con 0x7f1b68162190 2023-11-26T07:17:39.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6815b110 con 0x7f1b68162190 2023-11-26T07:17:39.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7fd1db7fe640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1e0066470 0x7fd1e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7fd1db7fe640 1 -- 172.21.15.31:0/2303662172 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1e4083330 con 0x7fd1ec0543b0 2023-11-26T07:17:39.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7fd1db7fe640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] conn(0x7fd1e006e610 0x7fd1e0070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7fd1db7fe640 1 -- 172.21.15.31:0/2303662172 --> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1e00710e0 con 0x7fd1e006e610 2023-11-26T07:17:39.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7fd1db7fe640 1 -- 172.21.15.31:0/2303662172 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1e40859f0 con 0x7fd1ec0543b0 2023-11-26T07:17:39.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f2b76ffd640 1 -- 172.21.15.31:0/3755204495 <== osd.4 v2:172.21.15.97:6809/362016202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2b640711b0 con 0x7f2b6406e6e0 2023-11-26T07:17:39.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7fd1f2ffd640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1e0066470 0x7fd1e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.029+0000 7fd1f37fe640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] conn(0x7fd1e006e610 0x7fd1e0070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1b6806b460 con 0x7f1b68162190 2023-11-26T07:17:39.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f7f68ff9640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7f500664c0 0x7f7f50068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f7f68ff9640 1 -- 172.21.15.31:0/2515986859 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f6c1063e0 con 0x7f7f640df360 2023-11-26T07:17:39.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f7f68ff9640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] conn(0x7f7f5006e6e0 0x7f7f50070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7fd1f2ffd640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1e0066470 0x7fd1e0068930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8006fd0 tx=0x7fd1e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f7f68ff9640 1 -- 172.21.15.31:0/2515986859 --> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f500711b0 con 0x7f7f5006e6e0 2023-11-26T07:17:39.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f7f68ff9640 1 -- 172.21.15.31:0/2515986859 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f6c1089c0 con 0x7f7f640df360 2023-11-26T07:17:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7fd1f37fe640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] conn(0x7fd1e006e610 0x7fd1e0070a30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f7f6bfff640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7f500664c0 0x7f7f50068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.033+0000 7f1b657fa640 1 -- 172.21.15.31:0/2838322297 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1b540a9d50 con 0x7f1b68162190 2023-11-26T07:17:39.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f7f6b7fe640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] conn(0x7f7f5006e6e0 0x7f7f50070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7fd1db7fe640 1 -- 172.21.15.31:0/2303662172 <== osd.6 v2:172.21.15.97:6800/28530448 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd1e00710e0 con 0x7fd1e006e610 2023-11-26T07:17:39.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f1b657fa640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b4c0664d0 0x7f1b4c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f1b657fa640 1 -- 172.21.15.31:0/2838322297 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b5411e760 con 0x7f1b68162190 2023-11-26T07:17:39.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f1b657fa640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] conn(0x7f1b4c06e6f0 0x7f1b4c070b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f1b657fa640 1 -- 172.21.15.31:0/2838322297 --> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b4c0711c0 con 0x7f1b4c06e6f0 2023-11-26T07:17:39.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f1b657fa640 1 -- 172.21.15.31:0/2838322297 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b54120d80 con 0x7f1b68162190 2023-11-26T07:17:39.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f7f6bfff640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7f500664c0 0x7f7f50068980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f60000c10 tx=0x7f7f600a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f1b6d147640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] conn(0x7f1b4c06e6f0 0x7f1b4c070b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f7f6b7fe640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] conn(0x7f7f5006e6e0 0x7f7f50070b00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f1b6c946640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b4c0664d0 0x7f1b4c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f1b6d147640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] conn(0x7f1b4c06e6f0 0x7f1b4c070b10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f7f68ff9640 1 -- 172.21.15.31:0/2515986859 <== osd.5 v2:172.21.15.97:6811/3669978555 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7f500711b0 con 0x7f7f5006e6e0 2023-11-26T07:17:39.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.037+0000 7f1b6c946640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b4c0664d0 0x7f1b4c068990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1b68099d50 tx=0x7f1b5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 --> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7a8c049490 con 0x7f7a7806e690 2023-11-26T07:17:39.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a917fa640 1 -- 172.21.15.31:0/2499008137 <== osd.3 v2:172.21.15.31:6824/3143140069 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7a8c049490 con 0x7f7a7806e690 2023-11-26T07:17:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 >> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] conn(0x7f7a7806e690 msgr2=0x7f7a78070ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a99aba640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] conn(0x7f7a7806e690 0x7f7a78070ab0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 --> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f39ac002cc0 con 0x7f39a806e710 2023-11-26T07:17:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f1b657fa640 1 -- 172.21.15.31:0/2838322297 <== osd.7 v2:172.21.15.97:6808/4245711455 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1b4c0711c0 con 0x7f1b4c06e6f0 2023-11-26T07:17:39.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a78066470 msgr2=0x7f7a78068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a99aba640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a78066470 0x7f7a78068930 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00d830 tx=0x7f7a9406a400 comp rx=0 tx=0).stop 2023-11-26T07:17:39.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f39bffff640 1 -- 172.21.15.31:0/2990608427 <== osd.1 v2:172.21.15.31:6800/845128234 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f39ac002cc0 con 0x7f39a806e710 2023-11-26T07:17:39.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c053060 msgr2=0x7f7a8c045af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a99aba640 1 --2- 172.21.15.31:0/2499008137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a8c053060 0x7f7a8c045af0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7a88009e70 tx=0x7f7a8800c430 comp rx=0 tx=0).stop 2023-11-26T07:17:39.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a992b9640 1 -- 172.21.15.31:0/2499008137 reap_dead start 2023-11-26T07:17:39.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 shutdown_connections 2023-11-26T07:17:39.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 --> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8b60074990 con 0x7f8b4006e6e0 2023-11-26T07:17:39.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 >> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] conn(0x7f39a806e710 msgr2=0x7f39a8070b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f39cd4ad640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] conn(0x7f39a806e710 0x7f39a8070b30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.041+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 >> 172.21.15.31:0/2499008137 conn(0x7f7a8c009f90 msgr2=0x7f7a8c0df740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 shutdown_connections 2023-11-26T07:17:39.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f7a99aba640 1 -- 172.21.15.31:0/2499008137 wait complete. 2023-11-26T07:17:39.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f8b4effd640 1 -- 172.21.15.31:0/1083332818 <== osd.0 v2:172.21.15.31:6816/4191585440 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8b60074990 con 0x7f8b4006e6e0 2023-11-26T07:17:39.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f39a80664f0 msgr2=0x7f39a80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f39cd4ad640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f39a80664f0 0x7f39a80689b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f39b80ab000 tx=0x7f39b80a7070 comp rx=0 tx=0).stop 2023-11-26T07:17:39.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c8141800 msgr2=0x7f39c81568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f39cd4ad640 1 --2- 172.21.15.31:0/2990608427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39c8141800 0x7f39c81568a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f39b000c3c0 tx=0x7f39b000c880 comp rx=0 tx=0).stop 2023-11-26T07:17:39.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f39c77fe640 1 -- 172.21.15.31:0/2990608427 reap_dead start 2023-11-26T07:17:39.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 shutdown_connections 2023-11-26T07:17:39.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 >> 172.21.15.31:0/2990608427 conn(0x7f39c80b0c20 msgr2=0x7f39c80adf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 >> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] conn(0x7f8b4006e6e0 msgr2=0x7f8b40070b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f8b64ad6640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] conn(0x7f8b4006e6e0 0x7f8b40070b00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 shutdown_connections 2023-11-26T07:17:39.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f39cd4ad640 1 -- 172.21.15.31:0/2990608427 wait complete. 2023-11-26T07:17:39.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b400664c0 msgr2=0x7f8b40068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f8b64ad6640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b400664c0 0x7f8b40068980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800a400 tx=0x7f8b5800d040 comp rx=0 tx=0).stop 2023-11-26T07:17:39.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 msgr2=0x7f8b6013ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f8b64ad6640 1 --2- 172.21.15.31:0/1083332818 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b6014b320 0x7f8b6013ff20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400c3c0 tx=0x7f8b5400c880 comp rx=0 tx=0).stop 2023-11-26T07:17:39.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f8b5ed9d640 1 -- 172.21.15.31:0/1083332818 reap_dead start 2023-11-26T07:17:39.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 shutdown_connections 2023-11-26T07:17:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.045+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 >> 172.21.15.31:0/1083332818 conn(0x7f8b600b0d90 msgr2=0x7f8b600c51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.049+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 shutdown_connections 2023-11-26T07:17:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.049+0000 7f8b64ad6640 1 -- 172.21.15.31:0/1083332818 wait complete. 2023-11-26T07:17:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.053+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 --> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f224c002cc0 con 0x7f224806e540 2023-11-26T07:17:39.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.053+0000 7f225f7fe640 1 -- 172.21.15.31:0/3850924629 <== osd.2 v2:172.21.15.31:6802/2371417753 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f224c002cc0 con 0x7f224806e540 2023-11-26T07:17:39.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 >> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] conn(0x7f224806e540 msgr2=0x7f2248070960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f226ca79640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] conn(0x7f224806e540 0x7f2248070960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2248066490 msgr2=0x7f2248068950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f226ca79640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2248066490 0x7f2248068950 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f22500ae000 tx=0x7f2250004740 comp rx=0 tx=0).stop 2023-11-26T07:17:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2268070470 msgr2=0x7f226814e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f226ca79640 1 --2- 172.21.15.31:0/3850924629 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2268070470 0x7f226814e8c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f225800d5b0 tx=0x7f225800da70 comp rx=0 tx=0).stop 2023-11-26T07:17:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f2266d9d640 1 -- 172.21.15.31:0/3850924629 reap_dead start 2023-11-26T07:17:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 shutdown_connections 2023-11-26T07:17:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 >> 172.21.15.31:0/3850924629 conn(0x7f22680b0d90 msgr2=0x7f2268071280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 shutdown_connections 2023-11-26T07:17:39.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.057+0000 7f226ca79640 1 -- 172.21.15.31:0/3850924629 wait complete. 2023-11-26T07:17:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.065+0000 7f2b8756f640 1 -- 172.21.15.31:0/3755204495 --> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b80067520 con 0x7f2b6406e6e0 2023-11-26T07:17:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.065+0000 7f2b76ffd640 1 -- 172.21.15.31:0/3755204495 <== osd.4 v2:172.21.15.97:6809/362016202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2b80067520 con 0x7f2b6406e6e0 2023-11-26T07:17:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.065+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 --> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd1ec0e6210 con 0x7fd1e006e610 2023-11-26T07:17:39.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.065+0000 7fd1db7fe640 1 -- 172.21.15.31:0/2303662172 <== osd.6 v2:172.21.15.97:6800/28530448 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd1ec0e6210 con 0x7fd1e006e610 2023-11-26T07:17:39.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.065+0000 7f2b74ff9640 1 -- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] conn(0x7f2b6406e6e0 msgr2=0x7f2b64070b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.065+0000 7f2b74ff9640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] conn(0x7f2b6406e6e0 0x7f2b64070b00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.065+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] conn(0x7fd1e006e610 msgr2=0x7fd1e0070a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.065+0000 7fd1f3fff640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] conn(0x7fd1e006e610 0x7fd1e0070a30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.065+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 --> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f640e3990 con 0x7f7f5006e6e0 2023-11-26T07:17:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1e0066470 msgr2=0x7fd1e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7fd1f3fff640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1e0066470 0x7fd1e0068930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8006fd0 tx=0x7fd1e8008040 comp rx=0 tx=0).stop 2023-11-26T07:17:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1ec0543b0 msgr2=0x7fd1ec0e3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7fd1f3fff640 1 --2- 172.21.15.31:0/2303662172 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1ec0543b0 0x7fd1ec0e3d10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4005890 tx=0x7fd1e400c960 comp rx=0 tx=0).stop 2023-11-26T07:17:39.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7fd1f37fe640 1 -- 172.21.15.31:0/2303662172 reap_dead start 2023-11-26T07:17:39.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f2b74ff9640 1 -- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2b640664c0 msgr2=0x7f2b64068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f2b74ff9640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2b640664c0 0x7f2b64068980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c001130 tx=0x7f2b7c0079f0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f2b74ff9640 1 -- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b800b1c60 msgr2=0x7f2b800b20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f2b74ff9640 1 --2- 172.21.15.31:0/3755204495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b800b1c60 0x7f2b800b20e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2b800cee40 tx=0x7f2b7800f5a0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f2b86d6e640 1 -- 172.21.15.31:0/3755204495 reap_dead start 2023-11-26T07:17:39.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f7f68ff9640 1 -- 172.21.15.31:0/2515986859 <== osd.5 v2:172.21.15.97:6811/3669978555 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f580095a0 con 0x7f7f5006e6e0 2023-11-26T07:17:39.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 shutdown_connections 2023-11-26T07:17:39.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 >> 172.21.15.31:0/2303662172 conn(0x7fd1ec058ed0 msgr2=0x7fd1ec005b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f2b74ff9640 1 -- 172.21.15.31:0/3755204495 shutdown_connections 2023-11-26T07:17:39.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f2b74ff9640 1 -- 172.21.15.31:0/3755204495 >> 172.21.15.31:0/3755204495 conn(0x7f2b8006df30 msgr2=0x7f2b800b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 shutdown_connections 2023-11-26T07:17:39.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7fd1f3fff640 1 -- 172.21.15.31:0/2303662172 wait complete. 2023-11-26T07:17:39.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f2b74ff9640 1 -- 172.21.15.31:0/3755204495 shutdown_connections 2023-11-26T07:17:39.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f2b74ff9640 1 -- 172.21.15.31:0/3755204495 wait complete. 2023-11-26T07:17:39.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 --> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1b6804e850 con 0x7f1b4c06e6f0 2023-11-26T07:17:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] conn(0x7f7f5006e6e0 msgr2=0x7f7f50070b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.069+0000 7f7f710c8640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] conn(0x7f7f5006e6e0 0x7f7f50070b00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.073+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7f500664c0 msgr2=0x7f7f50068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.073+0000 7f7f710c8640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7f500664c0 0x7f7f50068980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7f60000c10 tx=0x7f7f600a6000 comp rx=0 tx=0).stop 2023-11-26T07:17:39.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.073+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 msgr2=0x7f7f640d6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.073+0000 7f7f710c8640 1 --2- 172.21.15.31:0/2515986859 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7f640df360 0x7f7f640d6480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c04fb60 tx=0x7f7f6c068000 comp rx=0 tx=0).stop 2023-11-26T07:17:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.073+0000 7f7f708c7640 1 -- 172.21.15.31:0/2515986859 reap_dead start 2023-11-26T07:17:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.073+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 shutdown_connections 2023-11-26T07:17:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.073+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 >> 172.21.15.31:0/2515986859 conn(0x7f7f64009f90 msgr2=0x7f7f640d6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.073+0000 7f1b657fa640 1 -- 172.21.15.31:0/2838322297 <== osd.7 v2:172.21.15.97:6808/4245711455 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1b6804e850 con 0x7f1b4c06e6f0 2023-11-26T07:17:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.073+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] conn(0x7f1b4c06e6f0 msgr2=0x7f1b4c070b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.073+0000 7f1b6ebaa640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] conn(0x7f1b4c06e6f0 0x7f1b4c070b10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 shutdown_connections 2023-11-26T07:17:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b4c0664d0 msgr2=0x7f1b4c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f1b6ebaa640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b4c0664d0 0x7f1b4c068990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1b68099d50 tx=0x7f1b5c009040 comp rx=0 tx=0).stop 2023-11-26T07:17:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b68162190 msgr2=0x7f1b68162ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f1b6ebaa640 1 --2- 172.21.15.31:0/2838322297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b68162190 0x7f1b68162ee0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1b540190e0 tx=0x7f1b5401ad10 comp rx=0 tx=0).stop 2023-11-26T07:17:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f7f710c8640 1 -- 172.21.15.31:0/2515986859 wait complete. 2023-11-26T07:17:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f1b6d147640 1 -- 172.21.15.31:0/2838322297 reap_dead start 2023-11-26T07:17:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 shutdown_connections 2023-11-26T07:17:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 >> 172.21.15.31:0/2838322297 conn(0x7f1b680b0cb0 msgr2=0x7f1b680ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 shutdown_connections 2023-11-26T07:17:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.077+0000 7f1b6ebaa640 1 -- 172.21.15.31:0/2838322297 wait complete. 2023-11-26T07:17:39.100 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-11-26T07:17:39.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-26T07:17:39.101 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-11-26T07:17:39.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-26T07:17:39.102 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2023-11-26T07:17:39.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-26T07:17:39.103 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-11-26T07:17:39.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-26T07:17:39.106 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-11-26T07:17:39.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-26T07:17:39.110 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-11-26T07:17:39.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-26T07:17:39.112 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-11-26T07:17:39.112 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-11-26T07:17:39.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-26T07:17:39.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-26T07:17:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.321+0000 7fe22cf2a640 1 Processor -- start 2023-11-26T07:17:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.321+0000 7fe22cf2a640 1 -- start start 2023-11-26T07:17:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe22cf2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe22815cf50 0x7fe2280b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe22cf2a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe228070470 0x7fe228070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe225d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe2281595f0 0x7fe2280cdda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:57046/0 (socket says 172.21.15.31:57046) 2023-11-26T07:17:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe226d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe228070470 0x7fe228070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe226d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe228070470 0x7fe228070920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59436/0 (socket says 172.21.15.31:59436) 2023-11-26T07:17:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe22cf2a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2280b4ef0 con 0x7fe2281595f0 2023-11-26T07:17:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe22cf2a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2280b3bf0 con 0x7fe228070470 2023-11-26T07:17:39.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe22cf2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280cd910 con 0x7fe22815cf50 2023-11-26T07:17:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe225d9b640 1 -- 172.21.15.31:0/3169307180 learned_addr learned my addr 172.21.15.31:0/3169307180 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe22659c640 1 --2- 172.21.15.31:0/3169307180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe22815cf50 0x7fe2280b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe226d9d640 1 -- 172.21.15.31:0/3169307180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe22815cf50 msgr2=0x7fe2280b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe226d9d640 1 --2- 172.21.15.31:0/3169307180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe22815cf50 0x7fe2280b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe226d9d640 1 -- 172.21.15.31:0/3169307180 >> v1:172.21.15.97:6789/0 conn(0x7fe2281595f0 legacy=0x7fe2280cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.325+0000 7fe226d9d640 1 -- 172.21.15.31:0/3169307180 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe228070e60 con 0x7fe228070470 2023-11-26T07:17:39.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.329+0000 7fe226d9d640 1 --2- 172.21.15.31:0/3169307180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228070470 0x7fe228070920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe220009d90 tx=0x7fe22001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ee7f0379f34bcf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6c15edc640 1 Processor -- start 2023-11-26T07:17:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7fe22559a640 1 -- 172.21.15.31:0/3169307180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200a5790 con 0x7fe228070470 2023-11-26T07:17:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6c15edc640 1 -- start start 2023-11-26T07:17:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7fe22559a640 1 -- 172.21.15.31:0/3169307180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2200a58f0 con 0x7fe228070470 2023-11-26T07:17:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7fe22559a640 1 -- 172.21.15.31:0/3169307180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200a5c80 con 0x7fe228070470 2023-11-26T07:17:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6e49565640 1 Processor -- start 2023-11-26T07:17:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6e49565640 1 -- start start 2023-11-26T07:17:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6c15edc640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c080df360 0x7f6c080071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6c15edc640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c08005ec0 con 0x7f6c0800c080 2023-11-26T07:17:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6c15edc640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c08006d20 con 0x7f6c0804f590 2023-11-26T07:17:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6c14eda640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c080df360 0x7f6c080071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6c15edc640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c08054360 con 0x7f6c080df360 2023-11-26T07:17:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6c14eda640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c080df360 0x7f6c080071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59646/0 (socket says 172.21.15.31:59646) 2023-11-26T07:17:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6c14eda640 1 -- 172.21.15.31:0/2706911996 learned_addr learned my addr 172.21.15.31:0/2706911996 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7fe22cf2a640 1 -- 172.21.15.31:0/3169307180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228070470 msgr2=0x7fe228070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7fe22cf2a640 1 --2- 172.21.15.31:0/3169307180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228070470 0x7fe228070920 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe220009d90 tx=0x7fe22001c930 comp rx=0 tx=0).stop 2023-11-26T07:17:39.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.333+0000 7f6c0ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6c0800c080 0x7f6c08037ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46268/0 (socket says 172.21.15.31:46268) 2023-11-26T07:17:39.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7fe22cf2a640 1 -- 172.21.15.31:0/3169307180 shutdown_connections 2023-11-26T07:17:39.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7fe22cf2a640 1 --2- 172.21.15.31:0/3169307180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228070470 0x7fe228070920 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7fe22cf2a640 1 --2- 172.21.15.31:0/3169307180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe22815cf50 0x7fe2280b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7fe22cf2a640 1 -- 172.21.15.31:0/3169307180 >> 172.21.15.31:0/3169307180 conn(0x7fe2280b0d90 msgr2=0x7fe2280b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7fe22cf2a640 1 -- 172.21.15.31:0/3169307180 shutdown_connections 2023-11-26T07:17:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7fe22cf2a640 1 -- 172.21.15.31:0/3169307180 wait complete. 2023-11-26T07:17:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6e49565640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 0x7f6e440cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6e49565640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6e440cdf80 0x7f6e44071e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6e427fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6e440cdf80 0x7f6e44071e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218970620 0 0) 0x7f6c08006d20 con 0x7f6c0804f590 2023-11-26T07:17:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6e49565640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440727c0 con 0x7f6e44072350 2023-11-26T07:17:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6e49565640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e440cedf0 con 0x7f6e440cdf80 2023-11-26T07:17:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6e49565640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440ce550 con 0x7f6e4413ce70 2023-11-26T07:17:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6e437fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6e44072350 0x7f6e4413f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:57068/0 (socket says 172.21.15.31:57068) 2023-11-26T07:17:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6e42ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 0x7f6e440cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6e427fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6e440cdf80 0x7f6e44071e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59452/0 (socket says 172.21.15.31:59452) 2023-11-26T07:17:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6e427fc640 1 -- 172.21.15.31:0/2452109285 learned_addr learned my addr 172.21.15.31:0/2452109285 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c00003530 con 0x7f6c0804f590 2023-11-26T07:17:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7fe22cf2a640 1 Processor -- start 2023-11-26T07:17:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6e41ffb640 1 -- 172.21.15.31:0/2452109285 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710115770 0 0) 0x7f6e440727c0 con 0x7f6e44072350 2023-11-26T07:17:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6e41ffb640 1 -- 172.21.15.31:0/2452109285 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e2c003530 con 0x7f6e44072350 2023-11-26T07:17:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7fe22cf2a640 1 -- start start 2023-11-26T07:17:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442421002 0 0) 0x7f6c08005ec0 con 0x7f6c0800c080 2023-11-26T07:17:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c08006d20 con 0x7f6c0800c080 2023-11-26T07:17:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1820195002 0 0) 0x7f6c00003530 con 0x7f6c0804f590 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 >> v2:172.21.15.97:3300/0 conn(0x7f6c080df360 msgr2=0x7f6c080071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 --2- 172.21.15.31:0/2706911996 >> v2:172.21.15.97:3300/0 conn(0x7f6c080df360 0x7f6c080071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 >> v1:172.21.15.31:6789/0 conn(0x7f6c0800c080 legacy=0x7f6c08037ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c08006110 con 0x7f6c0804f590 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 <== mon.1 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f6c1004ed70 con 0x7f6c0804f590 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 >> v1:172.21.15.97:6789/0 conn(0x7f6c0804f590 legacy=0x7f6c080e9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 --2- 172.21.15.31:0/2706911996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c00003b10 0x7f6c00003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 --2- 172.21.15.31:0/2706911996 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c000047d0 0x7f6c00004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 --2- 172.21.15.31:0/2706911996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 0x7f6c0000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c08054360 con 0x7f6c00005190 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0000bed0 con 0x7f6c00003b10 2023-11-26T07:17:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0000c090 con 0x7f6c000047d0 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c156db640 1 --2- 172.21.15.31:0/2706911996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 0x7f6c0000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.337+0000 7f6c0ffff640 1 --2- 172.21.15.31:0/2706911996 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c000047d0 0x7f6c00004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6c156db640 1 -- 172.21.15.31:0/2706911996 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c000047d0 msgr2=0x7f6c00004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6c156db640 1 --2- 172.21.15.31:0/2706911996 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c000047d0 0x7f6c00004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6c156db640 1 -- 172.21.15.31:0/2706911996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c00003b10 msgr2=0x7f6c00003ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6c156db640 1 --2- 172.21.15.31:0/2706911996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c00003b10 0x7f6c00003ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6c156db640 1 -- 172.21.15.31:0/2706911996 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c08006110 con 0x7f6c00005190 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6e42ffd640 1 -- 172.21.15.31:0/2452109285 >> v2:172.21.15.31:3300/0 conn(0x7f6e440cdf80 msgr2=0x7f6e44071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6e42ffd640 1 --2- 172.21.15.31:0/2452109285 >> v2:172.21.15.31:3300/0 conn(0x7f6e440cdf80 0x7f6e44071e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6e42ffd640 1 -- 172.21.15.31:0/2452109285 >> v1:172.21.15.97:6789/0 conn(0x7f6e44072350 legacy=0x7f6e4413f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7f6e42ffd640 1 -- 172.21.15.31:0/2452109285 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440ce830 con 0x7f6e4413ce70 2023-11-26T07:17:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7fe22cf2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe228070470 0x7fe2280af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7fe22cf2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2281595f0 0x7fe22813d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7fe22cf2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22815cf50 0x7fe228153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7fe22659c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe228070470 0x7fe2280af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7fe22659c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe228070470 0x7fe2280af970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48180/0 (socket says 172.21.15.31:48180) 2023-11-26T07:17:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7fe22659c640 1 -- 172.21.15.31:0/789675963 learned_addr learned my addr 172.21.15.31:0/789675963 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.341+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22813de70 con 0x7fe2281595f0 2023-11-26T07:17:39.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe226d9d640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22815cf50 0x7fe228153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe228154e80 con 0x7fe22815cf50 2023-11-26T07:17:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe228154050 con 0x7fe228070470 2023-11-26T07:17:39.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe225d9b640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2281595f0 0x7fe22813d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe22659c640 1 -- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22815cf50 msgr2=0x7fe228153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe22659c640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22815cf50 0x7fe228153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe22659c640 1 -- 172.21.15.31:0/789675963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2281595f0 msgr2=0x7fe22813d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe22659c640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2281595f0 0x7fe22813d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe22659c640 1 -- 172.21.15.31:0/789675963 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220009a70 con 0x7fe228070470 2023-11-26T07:17:39.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe226d9d640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22815cf50 0x7fe228153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe225d9b640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2281595f0 0x7fe22813d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe22659c640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe228070470 0x7fe2280af970 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00c820 tx=0x7fe21c00cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe216ffd640 1 -- 172.21.15.31:0/789675963 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c007a30 con 0x7fe228070470 2023-11-26T07:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe216ffd640 1 -- 172.21.15.31:0/789675963 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21c007b90 con 0x7fe228070470 2023-11-26T07:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe216ffd640 1 -- 172.21.15.31:0/789675963 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00ce10 con 0x7fe228070470 2023-11-26T07:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7f6c156db640 1 -- 172.21.15.31:0/2706911996 reap_dead start 2023-11-26T07:17:39.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe228154300 con 0x7fe228070470 2023-11-26T07:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2281548c0 con 0x7fe228070470 2023-11-26T07:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7f6c156db640 1 --2- 172.21.15.31:0/2706911996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 0x7f6c0000b990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6c100643d0 tx=0x7f6c10099c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b59d2477eac78c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10123880 con 0x7f6c00005190 2023-11-26T07:17:39.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7f070438d640 1 Processor -- start 2023-11-26T07:17:39.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7f6e42ffd640 1 --2- 172.21.15.31:0/2452109285 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 0x7f6e440cda40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097c0 tx=0x7f6e3401c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83c7ee72e6d1d750 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.345+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c10123a20 con 0x7f6c00005190 2023-11-26T07:17:39.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f070438d640 1 -- start start 2023-11-26T07:17:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6c0f7fe640 1 -- 172.21.15.31:0/2706911996 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10123dd0 con 0x7f6c00005190 2023-11-26T07:17:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6c15edc640 1 -- 172.21.15.31:0/2706911996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 msgr2=0x7f6c0000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6c15edc640 1 --2- 172.21.15.31:0/2706911996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 0x7f6c0000b990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6c100643d0 tx=0x7f6c10099c30 comp rx=0 tx=0).stop 2023-11-26T07:17:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6c15edc640 1 -- 172.21.15.31:0/2706911996 shutdown_connections 2023-11-26T07:17:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6c15edc640 1 --2- 172.21.15.31:0/2706911996 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 0x7f6c0000b990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6c15edc640 1 -- 172.21.15.31:0/2706911996 >> 172.21.15.31:0/2706911996 conn(0x7f6c08009f90 msgr2=0x7f6c0800a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6e41ffb640 1 -- 172.21.15.31:0/2452109285 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340ab600 con 0x7f6e4413ce70 2023-11-26T07:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6e41ffb640 1 -- 172.21.15.31:0/2452109285 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e340a7050 con 0x7f6e4413ce70 2023-11-26T07:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6e41ffb640 1 -- 172.21.15.31:0/2452109285 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340ab980 con 0x7f6e4413ce70 2023-11-26T07:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6e49565640 1 -- 172.21.15.31:0/2452109285 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 msgr2=0x7f6e440cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6e49565640 1 --2- 172.21.15.31:0/2452109285 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 0x7f6e440cda40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097c0 tx=0x7f6e3401c330 comp rx=0 tx=0).stop 2023-11-26T07:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f070438d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06fc0cbc50 0x7f06fc0cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f070438d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15b570 0x7f06fc0721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7fe216ffd640 1 -- 172.21.15.31:0/789675963 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe21c0113f0 con 0x7fe228070470 2023-11-26T07:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f0702129640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06fc0cbc50 0x7f06fc0cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f070438d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc06c790 con 0x7f06fc072710 2023-11-26T07:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f070438d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc0cc570 con 0x7f06fc0cbc50 2023-11-26T07:17:39.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f070438d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0ce690 con 0x7f06fc15b570 2023-11-26T07:17:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f0701928640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15b570 0x7f06fc0721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7fe216ffd640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2080664c0 0x7fe208068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f0702129640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06fc0cbc50 0x7f06fc0cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59474/0 (socket says 172.21.15.31:59474) 2023-11-26T07:17:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f0702129640 1 -- 172.21.15.31:0/2125480845 learned_addr learned my addr 172.21.15.31:0/2125480845 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe228074990 con 0x7fe228070470 2023-11-26T07:17:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6e49565640 1 -- 172.21.15.31:0/2452109285 shutdown_connections 2023-11-26T07:17:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6e49565640 1 --2- 172.21.15.31:0/2452109285 >> v2:172.21.15.31:3300/0 conn(0x7f6e440cdf80 0x7f6e44071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6e49565640 1 --2- 172.21.15.31:0/2452109285 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 0x7f6e440cda40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6e49565640 1 -- 172.21.15.31:0/2452109285 >> 172.21.15.31:0/2452109285 conn(0x7f6e440b0d90 msgr2=0x7f6e440b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7fe225d9b640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2080664c0 0x7fe208068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f070292a640 1 --1- 172.21.15.31:0/2125480845 >> v1:172.21.15.31:6789/0 conn(0x7f06fc072710 0x7f06fc0ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46278/0 (socket says 172.21.15.31:46278) 2023-11-26T07:17:39.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f0702129640 1 -- 172.21.15.31:0/2125480845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15b570 msgr2=0x7f06fc0721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f0702129640 1 --2- 172.21.15.31:0/2125480845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15b570 0x7f06fc0721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f0702129640 1 -- 172.21.15.31:0/2125480845 >> v1:172.21.15.31:6789/0 conn(0x7f06fc072710 legacy=0x7f06fc0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f0702129640 1 -- 172.21.15.31:0/2125480845 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc0ce970 con 0x7f06fc0cbc50 2023-11-26T07:17:39.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6c15edc640 1 -- 172.21.15.31:0/2706911996 shutdown_connections 2023-11-26T07:17:39.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.349+0000 7f6c15edc640 1 -- 172.21.15.31:0/2706911996 wait complete. 2023-11-26T07:17:39.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6c15edc640 1 Processor -- start 2023-11-26T07:17:39.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6e49565640 1 -- 172.21.15.31:0/2452109285 shutdown_connections 2023-11-26T07:17:39.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7fe216ffd640 1 -- 172.21.15.31:0/789675963 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe21c0544b0 con 0x7fe228070470 2023-11-26T07:17:39.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6e49565640 1 -- 172.21.15.31:0/2452109285 wait complete. 2023-11-26T07:17:39.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6c15edc640 1 -- start start 2023-11-26T07:17:39.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7fe225d9b640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2080664c0 0x7fe208068980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe2281f72d0 tx=0x7fe218007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6753780640 1 Processor -- start 2023-11-26T07:17:39.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6753780640 1 -- start start 2023-11-26T07:17:39.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6c15edc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 0x7f6c080d7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.353+0000 7f6e49565640 1 Processor -- start 2023-11-26T07:17:39.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c15edc640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c08175af0 0x7f6c081776a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c15edc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c081752b0 0x7f6c080d9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c14eda640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 0x7f6c080d7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c14eda640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 0x7f6c080d7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59478/0 (socket says 172.21.15.31:59478) 2023-11-26T07:17:39.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c14eda640 1 -- 172.21.15.31:0/357215124 learned_addr learned my addr 172.21.15.31:0/357215124 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e49565640 1 -- start start 2023-11-26T07:17:39.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6753780640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f674c15d8f0 0x7f674c15e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c08049fc0 con 0x7f6c00005190 2023-11-26T07:17:39.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c080d7e30 con 0x7f6c081752b0 2023-11-26T07:17:39.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c08049620 con 0x7f6c08175af0 2023-11-26T07:17:39.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c0ffff640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c08175af0 0x7f6c081776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c156db640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c081752b0 0x7f6c080d9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6753780640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f674c0d0510 0x7f674c0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6753780640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674c15ecc0 con 0x7f674c0ced30 2023-11-26T07:17:39.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6753780640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674c0d4420 con 0x7f674c15d8f0 2023-11-26T07:17:39.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6753780640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674c0cdfb0 con 0x7f674c0d0510 2023-11-26T07:17:39.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c0ffff640 1 -- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c081752b0 msgr2=0x7f6c080d9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c0ffff640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c081752b0 0x7f6c080d9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c0ffff640 1 -- 172.21.15.31:0/357215124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 msgr2=0x7f6c080d7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c0ffff640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 0x7f6c080d7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c0ffff640 1 -- 172.21.15.31:0/357215124 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c08049900 con 0x7f6c08175af0 2023-11-26T07:17:39.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c156db640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c081752b0 0x7f6c080d9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:39.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6751d1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f674c0d0510 0x7f674c0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6751d1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f674c0d0510 0x7f674c0d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59688/0 (socket says 172.21.15.31:59688) 2023-11-26T07:17:39.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6751d1d640 1 -- 172.21.15.31:0/130843153 learned_addr learned my addr 172.21.15.31:0/130843153 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f675151c640 1 --2- 172.21.15.31:0/130843153 >> v2:172.21.15.31:3300/0 conn(0x7f674c15d8f0 0x7f674c15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c0ffff640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c08175af0 0x7f6c081776a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6c04005fd0 tx=0x7f6c04001760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6751d1d640 1 -- 172.21.15.31:0/130843153 >> v2:172.21.15.31:3300/0 conn(0x7f674c15d8f0 msgr2=0x7f674c15e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6751d1d640 1 --2- 172.21.15.31:0/130843153 >> v2:172.21.15.31:3300/0 conn(0x7f674c15d8f0 0x7f674c15e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6751d1d640 1 -- 172.21.15.31:0/130843153 >> v1:172.21.15.31:6789/0 conn(0x7f674c0ced30 legacy=0x7f674c0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6751d1d640 1 -- 172.21.15.31:0/130843153 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c0d4140 con 0x7f674c0d0510 2023-11-26T07:17:39.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e49565640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e44072350 0x7f6e4415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e49565640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cdf80 0x7f6e44138950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f0702129640 1 --2- 172.21.15.31:0/2125480845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc0cbc50 0x7f06fc0cc030 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009e50 tx=0x7f06f401d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0ac9fd947c726d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e42ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e44072350 0x7f6e4415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e42ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e44072350 0x7f6e4415bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59696/0 (socket says 172.21.15.31:59696) 2023-11-26T07:17:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e427fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cdf80 0x7f6e44138950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e427fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cdf80 0x7f6e44138950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59504/0 (socket says 172.21.15.31:59504) 2023-11-26T07:17:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e42ffd640 1 -- 172.21.15.31:0/3852165479 learned_addr learned my addr 172.21.15.31:0/3852165479 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e49565640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 0x7f6e4413ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44138e90 con 0x7f6e440cdf80 2023-11-26T07:17:39.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4413b340 con 0x7f6e44072350 2023-11-26T07:17:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415c1c0 con 0x7f6e4413ce70 2023-11-26T07:17:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7f6c0d7fa640 1 -- 172.21.15.31:0/357215124 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04010510 con 0x7f6c08175af0 2023-11-26T07:17:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6c0d7fa640 1 -- 172.21.15.31:0/357215124 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c04010690 con 0x7f6c08175af0 2023-11-26T07:17:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6c0d7fa640 1 -- 172.21.15.31:0/357215124 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04010a40 con 0x7f6c08175af0 2023-11-26T07:17:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.357+0000 7fe216ffd640 1 -- 172.21.15.31:0/789675963 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe21c0507f0 con 0x7fe228070470 2023-11-26T07:17:39.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c08049be0 con 0x7f6c08175af0 2023-11-26T07:17:39.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f0701127640 1 -- 172.21.15.31:0/2125480845 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f40a5840 con 0x7f06fc0cbc50 2023-11-26T07:17:39.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f0701127640 1 -- 172.21.15.31:0/2125480845 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f40a59a0 con 0x7f06fc0cbc50 2023-11-26T07:17:39.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f0701127640 1 -- 172.21.15.31:0/2125480845 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f40a5840 con 0x7f06fc0cbc50 2023-11-26T07:17:39.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e437fe640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 0x7f6e4413ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c08177be0 con 0x7f6c08175af0 2023-11-26T07:17:39.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f070438d640 1 -- 172.21.15.31:0/2125480845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc0cbc50 msgr2=0x7f06fc0cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f070438d640 1 --2- 172.21.15.31:0/2125480845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc0cbc50 0x7f06fc0cc030 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009e50 tx=0x7f06f401d400 comp rx=0 tx=0).stop 2023-11-26T07:17:39.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e437fe640 1 -- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e44072350 msgr2=0x7f6e4415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e437fe640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e44072350 0x7f6e4415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e437fe640 1 -- 172.21.15.31:0/3852165479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cdf80 msgr2=0x7f6e44138950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e437fe640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cdf80 0x7f6e44138950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e437fe640 1 -- 172.21.15.31:0/3852165479 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e34009420 con 0x7f6e4413ce70 2023-11-26T07:17:39.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e437fe640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 0x7f6e4413ae00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00c700 tx=0x7f6e3c00cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e2b7fe640 1 -- 172.21.15.31:0/3852165479 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00ce10 con 0x7f6e4413ce70 2023-11-26T07:17:39.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e2b7fe640 1 -- 172.21.15.31:0/3852165479 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c00a640 con 0x7f6e4413ce70 2023-11-26T07:17:39.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e2b7fe640 1 -- 172.21.15.31:0/3852165479 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00aa10 con 0x7f6e4413ce70 2023-11-26T07:17:39.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0804a100 con 0x7f6c08175af0 2023-11-26T07:17:39.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4415c4a0 con 0x7f6e4413ce70 2023-11-26T07:17:39.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4415ca60 con 0x7f6e4413ce70 2023-11-26T07:17:39.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6751d1d640 1 --2- 172.21.15.31:0/130843153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674c0d0510 0x7f674c0d1000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6744009e50 tx=0x7f674401c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2cc2ec30af6018c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6743fff640 1 -- 172.21.15.31:0/130843153 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67440a5860 con 0x7f674c0d0510 2023-11-26T07:17:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6743fff640 1 -- 172.21.15.31:0/130843153 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67440a59c0 con 0x7f674c0d0510 2023-11-26T07:17:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.361+0000 7f6743fff640 1 -- 172.21.15.31:0/130843153 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67440a5d50 con 0x7f674c0d0510 2023-11-26T07:17:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6753780640 1 -- 172.21.15.31:0/130843153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674c0d0510 msgr2=0x7f674c0d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6753780640 1 --2- 172.21.15.31:0/130843153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674c0d0510 0x7f674c0d1000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6744009e50 tx=0x7f674401c9c0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6753780640 1 -- 172.21.15.31:0/130843153 shutdown_connections 2023-11-26T07:17:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6753780640 1 --2- 172.21.15.31:0/130843153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674c0d0510 0x7f674c0d1000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6753780640 1 --2- 172.21.15.31:0/130843153 >> v2:172.21.15.31:3300/0 conn(0x7f674c15d8f0 0x7f674c15e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6753780640 1 -- 172.21.15.31:0/130843153 >> 172.21.15.31:0/130843153 conn(0x7f674c09c750 msgr2=0x7f674c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e44074990 con 0x7f6e4413ce70 2023-11-26T07:17:39.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f070438d640 1 -- 172.21.15.31:0/2125480845 shutdown_connections 2023-11-26T07:17:39.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f070438d640 1 --2- 172.21.15.31:0/2125480845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15b570 0x7f06fc0721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f070438d640 1 --2- 172.21.15.31:0/2125480845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc0cbc50 0x7f06fc0cc030 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f070438d640 1 -- 172.21.15.31:0/2125480845 >> 172.21.15.31:0/2125480845 conn(0x7f06fc0b0cb0 msgr2=0x7f06fc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6c0d7fa640 1 -- 172.21.15.31:0/357215124 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6c04010ba0 con 0x7f6c08175af0 2023-11-26T07:17:39.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f070438d640 1 -- 172.21.15.31:0/2125480845 shutdown_connections 2023-11-26T07:17:39.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6753780640 1 -- 172.21.15.31:0/130843153 shutdown_connections 2023-11-26T07:17:39.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6c0d7fa640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf4066540 0x7f6bf4068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6c0d7fa640 1 -- 172.21.15.31:0/357215124 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c04083840 con 0x7f6c08175af0 2023-11-26T07:17:39.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6753780640 1 -- 172.21.15.31:0/130843153 wait complete. 2023-11-26T07:17:39.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f070438d640 1 -- 172.21.15.31:0/2125480845 wait complete. 2023-11-26T07:17:39.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.365+0000 7f6e2b7fe640 1 -- 172.21.15.31:0/3852165479 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6e3c00ab70 con 0x7f6e4413ce70 2023-11-26T07:17:39.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6e2b7fe640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6e30066470 0x7f6e30068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f070438d640 1 Processor -- start 2023-11-26T07:17:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6753780640 1 Processor -- start 2023-11-26T07:17:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f070438d640 1 -- start start 2023-11-26T07:17:39.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6c14eda640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf4066540 0x7f6bf4068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6e42ffd640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6e30066470 0x7f6e30068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6753780640 1 -- start start 2023-11-26T07:17:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6e2b7fe640 1 -- 172.21.15.31:0/3852165479 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e3c083620 con 0x7f6e4413ce70 2023-11-26T07:17:39.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6c14eda640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf4066540 0x7f6bf4068a00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c080e0b00 tx=0x7f6bfc00a350 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6e42ffd640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6e30066470 0x7f6e30068930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097c0 tx=0x7f6e340a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6753780640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674c0ced30 0x7f674c0b6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6753780640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674c0d0510 0x7f674c0b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6753780640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674c15d8f0 0x7f674c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6753780640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0b7330 con 0x7f674c15d8f0 2023-11-26T07:17:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6753780640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0b7610 con 0x7f674c0d0510 2023-11-26T07:17:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6753780640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0cbe10 con 0x7f674c0ced30 2023-11-26T07:17:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f675151c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674c0ced30 0x7f674c0b6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6750d1b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674c0d0510 0x7f674c0b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f675151c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674c0ced30 0x7f674c0b6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48220/0 (socket says 172.21.15.31:48220) 2023-11-26T07:17:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f675151c640 1 -- 172.21.15.31:0/2064814894 learned_addr learned my addr 172.21.15.31:0/2064814894 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6751d1d640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674c15d8f0 0x7f674c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6750d1b640 1 -- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674c0ced30 msgr2=0x7f674c0b6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f070438d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc072710 0x7f06fc09c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6750d1b640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674c0ced30 0x7f674c0b6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6750d1b640 1 -- 172.21.15.31:0/2064814894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674c15d8f0 msgr2=0x7f674c0b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6750d1b640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674c15d8f0 0x7f674c0b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6750d1b640 1 -- 172.21.15.31:0/2064814894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6744009b30 con 0x7f674c0d0510 2023-11-26T07:17:39.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f070438d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc0cbc50 0x7f06fc1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.369+0000 7f6751d1d640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674c15d8f0 0x7f674c0b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f675151c640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674c0ced30 0x7f674c0b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0702129640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc072710 0x7f06fc09c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f6750d1b640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674c0d0510 0x7f674c0b8e40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6748000f70 tx=0x7f674800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0702129640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc072710 0x7f06fc09c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48232/0 (socket says 172.21.15.31:48232) 2023-11-26T07:17:39.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f6e2b7fe640 1 -- 172.21.15.31:0/3852165479 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6e3c050b40 con 0x7f6e4413ce70 2023-11-26T07:17:39.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f6741ffb640 1 -- 172.21.15.31:0/2064814894 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67480052d0 con 0x7f674c0d0510 2023-11-26T07:17:39.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f6741ffb640 1 -- 172.21.15.31:0/2064814894 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6748005430 con 0x7f674c0d0510 2023-11-26T07:17:39.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0702129640 1 -- 172.21.15.31:0/272992349 learned_addr learned my addr 172.21.15.31:0/272992349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0701928640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc0cbc50 0x7f06fc1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f070438d640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc15b570 0x7f06fc0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f6741ffb640 1 -- 172.21.15.31:0/2064814894 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748011870 con 0x7f674c0d0510 2023-11-26T07:17:39.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f6753780640 1 -- 172.21.15.31:0/2064814894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674c0cc0f0 con 0x7f674c0d0510 2023-11-26T07:17:39.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f6753780640 1 -- 172.21.15.31:0/2064814894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c151c40 con 0x7f674c0d0510 2023-11-26T07:17:39.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f070292a640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc15b570 0x7f06fc0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f070438d640 1 -- 172.21.15.31:0/272992349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc1f5530 con 0x7f06fc15b570 2023-11-26T07:17:39.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f070438d640 1 -- 172.21.15.31:0/272992349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc1f5810 con 0x7f06fc0cbc50 2023-11-26T07:17:39.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f070438d640 1 -- 172.21.15.31:0/272992349 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc1f5af0 con 0x7f06fc072710 2023-11-26T07:17:39.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0702129640 1 -- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc0cbc50 msgr2=0x7f06fc1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0702129640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc0cbc50 0x7f06fc1541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0702129640 1 -- 172.21.15.31:0/272992349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc15b570 msgr2=0x7f06fc0aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0702129640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc15b570 0x7f06fc0aebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0702129640 1 -- 172.21.15.31:0/272992349 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f4009b30 con 0x7f06fc072710 2023-11-26T07:17:39.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0701928640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc0cbc50 0x7f06fc1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:39.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f070292a640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc15b570 0x7f06fc0aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:39.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f0702129640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc072710 0x7f06fc09c5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06f401d240 tx=0x7f06f4019740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.373+0000 7f6753780640 1 -- 172.21.15.31:0/2064814894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674c0cc330 con 0x7f674c0d0510 2023-11-26T07:17:39.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.377+0000 7f6c0d7fa640 1 -- 172.21.15.31:0/357215124 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c0804a100 con 0x7f6c08175af0 2023-11-26T07:17:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.377+0000 7f6741ffb640 1 -- 172.21.15.31:0/2064814894 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f67480119d0 con 0x7f674c0d0510 2023-11-26T07:17:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.377+0000 7f06f2ffd640 1 -- 172.21.15.31:0/272992349 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4003db0 con 0x7f06fc072710 2023-11-26T07:17:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.377+0000 7f06f2ffd640 1 -- 172.21.15.31:0/272992349 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f4004040 con 0x7f06fc072710 2023-11-26T07:17:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.377+0000 7f06f2ffd640 1 -- 172.21.15.31:0/272992349 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f40043f0 con 0x7f06fc072710 2023-11-26T07:17:39.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.377+0000 7f070438d640 1 -- 172.21.15.31:0/272992349 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc1f5dd0 con 0x7f06fc072710 2023-11-26T07:17:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.377+0000 7f070438d640 1 -- 172.21.15.31:0/272992349 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc099a30 con 0x7f06fc072710 2023-11-26T07:17:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.377+0000 7f6741ffb640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67300664a0 0x7f6730068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.377+0000 7f6741ffb640 1 -- 172.21.15.31:0/2064814894 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6748083290 con 0x7f674c0d0510 2023-11-26T07:17:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.377+0000 7f675151c640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67300664a0 0x7f6730068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.381+0000 7f070438d640 1 -- 172.21.15.31:0/272992349 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc09cae0 con 0x7f06fc072710 2023-11-26T07:17:39.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.381+0000 7f675151c640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67300664a0 0x7f6730068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f674c0b2d50 tx=0x7f673c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.381+0000 7f06f2ffd640 1 -- 172.21.15.31:0/272992349 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f06f4004550 con 0x7f06fc072710 2023-11-26T07:17:39.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.381+0000 7f6741ffb640 1 -- 172.21.15.31:0/2064814894 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f674c0cc330 con 0x7f674c0d0510 2023-11-26T07:17:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.381+0000 7f06f2ffd640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06e0066540 0x7f06e0068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.381+0000 7f06f2ffd640 1 -- 172.21.15.31:0/272992349 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06f411f090 con 0x7f06fc072710 2023-11-26T07:17:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.385+0000 7f0701928640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06e0066540 0x7f06e0068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.385+0000 7f06f2ffd640 1 -- 172.21.15.31:0/272992349 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f06f40ec500 con 0x7f06fc072710 2023-11-26T07:17:39.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.389+0000 7f0701928640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06e0066540 0x7f06e0068a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f06fc151c90 tx=0x7f06f8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.421+0000 7faee6432640 1 Processor -- start 2023-11-26T07:17:39.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.421+0000 7faee6432640 1 -- start start 2023-11-26T07:17:39.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.421+0000 7f9c6e017640 1 Processor -- start 2023-11-26T07:17:39.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.421+0000 7faee6432640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faee00b3ef0 0x7faee00ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.421+0000 7faee6432640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee00cbd20 con 0x7faee00b1fb0 2023-11-26T07:17:39.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.421+0000 7f9c6e017640 1 -- start start 2023-11-26T07:17:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.421+0000 7faee6432640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee00cc810 con 0x7faee015cc80 2023-11-26T07:17:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.421+0000 7faee6432640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee0072960 con 0x7faee00b3ef0 2023-11-26T07:17:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.421+0000 7faedffff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7faee015cc80 0x7faee006df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:57074/0 (socket says 172.21.15.31:57074) 2023-11-26T07:17:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.421+0000 7faedffff640 1 -- 172.21.15.31:0/119118582 learned_addr learned my addr 172.21.15.31:0/119118582 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:39.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c6e017640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9c6815d8f0 0x7f9c6815e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c6e017640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9c680ced30 0x7f9c680cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c6e017640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9c680cf1b0 0x7f9c680d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c6e017640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c680d4420 con 0x7f9c6815d8f0 2023-11-26T07:17:39.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c6e017640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c6815ecc0 con 0x7f9c680cf1b0 2023-11-26T07:17:39.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c6e017640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c680cdfb0 con 0x7f9c680ced30 2023-11-26T07:17:39.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563316076 0 0) 0x7faee00cc810 con 0x7faee015cc80 2023-11-26T07:17:39.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c67fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9c680cf1b0 0x7f9c680d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c67fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9c680cf1b0 0x7f9c680d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59720/0 (socket says 172.21.15.31:59720) 2023-11-26T07:17:39.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c67fff640 1 -- 172.21.15.31:0/1224106192 learned_addr learned my addr 172.21.15.31:0/1224106192 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faec8003530 con 0x7faee015cc80 2023-11-26T07:17:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c66ffd640 1 --2- 172.21.15.31:0/1224106192 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9c680ced30 0x7f9c680cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4218911574 0 0) 0x7faec8003530 con 0x7faee015cc80 2023-11-26T07:17:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 >> v2:172.21.15.31:3300/0 conn(0x7faee00b3ef0 msgr2=0x7faee00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 --2- 172.21.15.31:0/119118582 >> v2:172.21.15.31:3300/0 conn(0x7faee00b3ef0 0x7faee00ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 >> v1:172.21.15.31:6789/0 conn(0x7faee00b1fb0 legacy=0x7faee00b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee00cbf70 con 0x7faee015cc80 2023-11-26T07:17:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 <== mon.1 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7faed4001680 con 0x7faee015cc80 2023-11-26T07:17:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 >> v1:172.21.15.97:6789/0 conn(0x7faee015cc80 legacy=0x7faee006df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 --2- 172.21.15.31:0/119118582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec8003980 0x7faec8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 --2- 172.21.15.31:0/119118582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec80047d0 0x7faec8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 --2- 172.21.15.31:0/119118582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faec8005390 0x7faec8014590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedf7fe640 1 --2- 172.21.15.31:0/119118582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec80047d0 0x7faec8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee0072960 con 0x7faec8003980 2023-11-26T07:17:39.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec8014ad0 con 0x7faec80047d0 2023-11-26T07:17:39.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec8014c10 con 0x7faec8005390 2023-11-26T07:17:39.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedf7fe640 1 -- 172.21.15.31:0/119118582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faec8005390 msgr2=0x7faec8014590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:17:39.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedf7fe640 1 --2- 172.21.15.31:0/119118582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faec8005390 0x7faec8014590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedf7fe640 1 -- 172.21.15.31:0/119118582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec8003980 msgr2=0x7faec8003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7faedf7fe640 1 --2- 172.21.15.31:0/119118582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec8003980 0x7faec8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.429+0000 7faedf7fe640 1 -- 172.21.15.31:0/119118582 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee00cbf70 con 0x7faec80047d0 2023-11-26T07:17:39.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.429+0000 7faee49cf640 1 -- 172.21.15.31:0/119118582 reap_dead start 2023-11-26T07:17:39.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c67fff640 1 -- 172.21.15.31:0/1224106192 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9c680ced30 msgr2=0x7f9c680cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c67fff640 1 --2- 172.21.15.31:0/1224106192 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9c680ced30 0x7f9c680cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c67fff640 1 -- 172.21.15.31:0/1224106192 >> v2:172.21.15.31:3300/0 conn(0x7f9c6815d8f0 msgr2=0x7f9c6815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c67fff640 1 --2- 172.21.15.31:0/1224106192 >> v2:172.21.15.31:3300/0 conn(0x7f9c6815d8f0 0x7f9c6815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.425+0000 7f9c67fff640 1 -- 172.21.15.31:0/1224106192 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c680d4140 con 0x7f9c680cf1b0 2023-11-26T07:17:39.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.429+0000 7f9c67fff640 1 --2- 172.21.15.31:0/1224106192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 0x7f9c680d0340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9c680b21f0 tx=0x7f9c5801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f158667603d29e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f376b29f640 1 Processor -- start 2023-11-26T07:17:39.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f376b29f640 1 -- start start 2023-11-26T07:17:39.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c667fc640 1 -- 172.21.15.31:0/1224106192 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c580aa610 con 0x7f9c680cf1b0 2023-11-26T07:17:39.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c667fc640 1 -- 172.21.15.31:0/1224106192 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c580a6040 con 0x7f9c680cf1b0 2023-11-26T07:17:39.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c667fc640 1 -- 172.21.15.31:0/1224106192 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c580aa9d0 con 0x7f9c680cf1b0 2023-11-26T07:17:39.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7faedf7fe640 1 --2- 172.21.15.31:0/119118582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec80047d0 0x7faec8004c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faed0001650 tx=0x7faed001c300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3952740b43382bec server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed0005270 con 0x7faec80047d0 2023-11-26T07:17:39.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7faedeffd640 1 -- 172.21.15.31:0/119118582 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed00a7070 con 0x7faec80047d0 2023-11-26T07:17:39.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c6e017640 1 -- 172.21.15.31:0/1224106192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 msgr2=0x7f9c680d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c6e017640 1 --2- 172.21.15.31:0/1224106192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 0x7f9c680d0340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9c680b21f0 tx=0x7f9c5801c510 comp rx=0 tx=0).stop 2023-11-26T07:17:39.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7faee6432640 1 -- 172.21.15.31:0/119118582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec80047d0 msgr2=0x7faec8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7faee6432640 1 --2- 172.21.15.31:0/119118582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec80047d0 0x7faec8004c50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faed0001650 tx=0x7faed001c300 comp rx=0 tx=0).stop 2023-11-26T07:17:39.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7faee6432640 1 -- 172.21.15.31:0/119118582 shutdown_connections 2023-11-26T07:17:39.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7faee6432640 1 --2- 172.21.15.31:0/119118582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faec80047d0 0x7faec8004c50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7faee6432640 1 -- 172.21.15.31:0/119118582 >> 172.21.15.31:0/119118582 conn(0x7faee00b0cb0 msgr2=0x7faee00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c6e017640 1 -- 172.21.15.31:0/1224106192 shutdown_connections 2023-11-26T07:17:39.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c6e017640 1 --2- 172.21.15.31:0/1224106192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 0x7f9c680d0340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c6e017640 1 --2- 172.21.15.31:0/1224106192 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9c680ced30 0x7f9c680cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c6e017640 1 --2- 172.21.15.31:0/1224106192 >> v2:172.21.15.31:3300/0 conn(0x7f9c6815d8f0 0x7f9c6815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c6e017640 1 -- 172.21.15.31:0/1224106192 >> 172.21.15.31:0/1224106192 conn(0x7f9c6809c750 msgr2=0x7f9c680d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c6e017640 1 -- 172.21.15.31:0/1224106192 shutdown_connections 2023-11-26T07:17:39.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f9c6e017640 1 -- 172.21.15.31:0/1224106192 wait complete. 2023-11-26T07:17:39.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f376b29f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c0e3260 0x7f375c056fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7f376b29f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f375c03e8d0 0x7f375c053ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f376b29f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c03d080 con 0x7f375c058960 2023-11-26T07:17:39.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f376a29d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c0e3260 0x7f375c056fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f376a29d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c0e3260 0x7f375c056fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48256/0 (socket says 172.21.15.31:48256) 2023-11-26T07:17:39.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f376a29d640 1 -- 172.21.15.31:0/3592729518 learned_addr learned my addr 172.21.15.31:0/3592729518 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f376aa9e640 1 --1- 172.21.15.31:0/3592729518 >> v1:172.21.15.97:6789/0 conn(0x7f375c058960 0x7f375c0061d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:57084/0 (socket says 172.21.15.31:57084) 2023-11-26T07:17:39.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f3769a9c640 1 --2- 172.21.15.31:0/3592729518 >> v2:172.21.15.97:3300/0 conn(0x7f375c03e8d0 0x7f375c053ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f376b29f640 1 -- 172.21.15.31:0/3592729518 --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c0e3640 con 0x7f375c03e8d0 2023-11-26T07:17:39.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f376929b640 1 -- 172.21.15.31:0/3592729518 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364836006 0 0) 0x7f37640687d0 con 0x7f375c058960 2023-11-26T07:17:39.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f376b29f640 1 -- 172.21.15.31:0/3592729518 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c0578c0 con 0x7f375c0e3260 2023-11-26T07:17:39.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7faee6432640 1 -- 172.21.15.31:0/119118582 shutdown_connections 2023-11-26T07:17:39.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.433+0000 7faee6432640 1 -- 172.21.15.31:0/119118582 wait complete. 2023-11-26T07:17:39.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7faee6432640 1 Processor -- start 2023-11-26T07:17:39.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7faee6432640 1 -- start start 2023-11-26T07:17:39.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f376929b640 1 -- 172.21.15.31:0/3592729518 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3750003530 con 0x7f375c058960 2023-11-26T07:17:39.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f3769a9c640 1 -- 172.21.15.31:0/3592729518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c0e3260 msgr2=0x7f375c056fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:39.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f9c6e017640 1 Processor -- start 2023-11-26T07:17:39.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f3769a9c640 1 --2- 172.21.15.31:0/3592729518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c0e3260 0x7f375c056fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f376929b640 1 -- 172.21.15.31:0/3592729518 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3295106570 0 0) 0x7f3750003530 con 0x7f375c058960 2023-11-26T07:17:39.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f3769a9c640 1 -- 172.21.15.31:0/3592729518 >> v1:172.21.15.97:6789/0 conn(0x7f375c058960 legacy=0x7f375c0061d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.437+0000 7f3769a9c640 1 -- 172.21.15.31:0/3592729518 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c057ba0 con 0x7f375c03e8d0 2023-11-26T07:17:39.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee6432640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faec80047d0 0x7faee015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7f9c6e017640 1 -- start start 2023-11-26T07:17:39.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee6432640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee01588d0 0x7faee01519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faedffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faec80047d0 0x7faee015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faedffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faec80047d0 0x7faee015b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48272/0 (socket says 172.21.15.31:48272) 2023-11-26T07:17:39.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faedf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee01588d0 0x7faee01519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faedf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee01588d0 0x7faee01519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59542/0 (socket says 172.21.15.31:59542) 2023-11-26T07:17:39.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faedffff640 1 -- 172.21.15.31:0/43625794 learned_addr learned my addr 172.21.15.31:0/43625794 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee6432640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faee00ccea0 0x7faee00cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee0158d50 con 0x7faee01588d0 2023-11-26T07:17:39.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee00cd860 con 0x7faee00ccea0 2023-11-26T07:17:39.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee00aec20 con 0x7faec80047d0 2023-11-26T07:17:39.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee49cf640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faee00ccea0 0x7faee00cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7f9c6e017640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c680ced30 0x7f9c680b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee49cf640 1 -- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faec80047d0 msgr2=0x7faee015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee49cf640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faec80047d0 0x7faee015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee49cf640 1 -- 172.21.15.31:0/43625794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee01588d0 msgr2=0x7faee01519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee49cf640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee01588d0 0x7faee01519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee49cf640 1 -- 172.21.15.31:0/43625794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee0151f10 con 0x7faee00ccea0 2023-11-26T07:17:39.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faedf7fe640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee01588d0 0x7faee01519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:39.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faee49cf640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faee00ccea0 0x7faee00cd320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faee0072c70 tx=0x7faed800ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.441+0000 7faedffff640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faec80047d0 0x7faee015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:39.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7faedcff9640 1 -- 172.21.15.31:0/43625794 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed800aa20 con 0x7faee00ccea0 2023-11-26T07:17:39.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c680ced30 0x7f9c680b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7faedcff9640 1 -- 172.21.15.31:0/43625794 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed800ab80 con 0x7faee00ccea0 2023-11-26T07:17:39.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee0152310 con 0x7faee00ccea0 2023-11-26T07:17:39.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f3769a9c640 1 --2- 172.21.15.31:0/3592729518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 0x7f375c053ca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f37600097c0 tx=0x7f376001ebd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b1c68af9fc912cf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee01550f0 con 0x7faee00ccea0 2023-11-26T07:17:39.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c6e017640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 0x7f9c682029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c6e017640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9c6815d8f0 0x7f9c680b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c6e017640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c68202f10 con 0x7f9c680ced30 2023-11-26T07:17:39.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c6e017640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c680b72d0 con 0x7f9c680cf1b0 2023-11-26T07:17:39.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c6e017640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c680b75b0 con 0x7f9c6815d8f0 2023-11-26T07:17:39.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c66ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 0x7f9c682029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c66ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 0x7f9c682029d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59746/0 (socket says 172.21.15.31:59746) 2023-11-26T07:17:39.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c66ffd640 1 -- 172.21.15.31:0/61462487 learned_addr learned my addr 172.21.15.31:0/61462487 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c67fff640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9c6815d8f0 0x7f9c680b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c66ffd640 1 -- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9c6815d8f0 msgr2=0x7f9c680b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c66ffd640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9c6815d8f0 0x7f9c680b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c66ffd640 1 -- 172.21.15.31:0/61462487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c680ced30 msgr2=0x7f9c680b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f376929b640 1 -- 172.21.15.31:0/3592729518 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376404f0a0 con 0x7f375c03e8d0 2023-11-26T07:17:39.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f376929b640 1 -- 172.21.15.31:0/3592729518 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37600a85d0 con 0x7f375c03e8d0 2023-11-26T07:17:39.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f376929b640 1 -- 172.21.15.31:0/3592729518 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600a8960 con 0x7f375c03e8d0 2023-11-26T07:17:39.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7faedcff9640 1 -- 172.21.15.31:0/43625794 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8012400 con 0x7faee00ccea0 2023-11-26T07:17:39.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c66ffd640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c680ced30 0x7f9c680b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c66ffd640 1 -- 172.21.15.31:0/61462487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c580094d0 con 0x7f9c680cf1b0 2023-11-26T07:17:39.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c66ffd640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 0x7f9c682029d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c60007e40 tx=0x7f9c6000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c4ffff640 1 -- 172.21.15.31:0/61462487 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60014620 con 0x7f9c680cf1b0 2023-11-26T07:17:39.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c4ffff640 1 -- 172.21.15.31:0/61462487 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6000ee90 con 0x7f9c680cf1b0 2023-11-26T07:17:39.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c4ffff640 1 -- 172.21.15.31:0/61462487 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c600149a0 con 0x7f9c680cf1b0 2023-11-26T07:17:39.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f376b29f640 1 -- 172.21.15.31:0/3592729518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 msgr2=0x7f375c053ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f376b29f640 1 --2- 172.21.15.31:0/3592729518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 0x7f375c053ca0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f37600097c0 tx=0x7f376001ebd0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7f376b29f640 1 -- 172.21.15.31:0/3592729518 shutdown_connections 2023-11-26T07:17:39.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7f376b29f640 1 --2- 172.21.15.31:0/3592729518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 0x7f375c053ca0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7f376b29f640 1 --2- 172.21.15.31:0/3592729518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c0e3260 0x7f375c056fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7f376b29f640 1 -- 172.21.15.31:0/3592729518 >> 172.21.15.31:0/3592729518 conn(0x7f375c058ed0 msgr2=0x7f375c0592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7f376b29f640 1 -- 172.21.15.31:0/3592729518 shutdown_connections 2023-11-26T07:17:39.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7f376b29f640 1 -- 172.21.15.31:0/3592729518 wait complete. 2023-11-26T07:17:39.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.445+0000 7f9c677fe640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c680ced30 0x7f9c680b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:39.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7f9c6e017640 1 -- 172.21.15.31:0/61462487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c680b7890 con 0x7f9c680cf1b0 2023-11-26T07:17:39.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7f9c6e017640 1 -- 172.21.15.31:0/61462487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c680b7e50 con 0x7f9c680cf1b0 2023-11-26T07:17:39.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7faedcff9640 1 -- 172.21.15.31:0/43625794 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faed8012560 con 0x7faee00ccea0 2023-11-26T07:17:39.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7faedcff9640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faecc0664a0 0x7faecc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7faedcff9640 1 -- 172.21.15.31:0/43625794 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faed8083740 con 0x7faee00ccea0 2023-11-26T07:17:39.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7f376b29f640 1 Processor -- start 2023-11-26T07:17:39.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.449+0000 7f9c6e017640 1 -- 172.21.15.31:0/61462487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6804e850 con 0x7f9c680cf1b0 2023-11-26T07:17:39.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376b29f640 1 -- start start 2023-11-26T07:17:39.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7faedffff640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faecc0664a0 0x7faecc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec0005260 con 0x7faee00ccea0 2023-11-26T07:17:39.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7faedffff640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faecc0664a0 0x7faecc068960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faed4001ec0 tx=0x7faed4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376b29f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 0x7f375c0e5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376b29f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c058960 0x7f375c050ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376a29d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 0x7f375c0e5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376a29d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 0x7f375c0e5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59762/0 (socket says 172.21.15.31:59762) 2023-11-26T07:17:39.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f3769a9c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c058960 0x7f375c050ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376a29d640 1 -- 172.21.15.31:0/4263990158 learned_addr learned my addr 172.21.15.31:0/4263990158 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376b29f640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c0e3260 0x7f375c1884a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c052730 con 0x7f375c0e3260 2023-11-26T07:17:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c051ab0 con 0x7f375c03e8d0 2023-11-26T07:17:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c051d90 con 0x7f375c058960 2023-11-26T07:17:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f9c4ffff640 1 -- 172.21.15.31:0/61462487 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9c60014b00 con 0x7f9c680cf1b0 2023-11-26T07:17:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f9c4ffff640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c54066540 0x7f9c54068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f9c4ffff640 1 -- 172.21.15.31:0/61462487 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c60084730 con 0x7f9c680cf1b0 2023-11-26T07:17:39.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.453+0000 7f376aa9e640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c0e3260 0x7f375c1884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.457+0000 7f376a29d640 1 -- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c058960 msgr2=0x7f375c050ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.457+0000 7f376a29d640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c058960 0x7f375c050ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.457+0000 7f376a29d640 1 -- 172.21.15.31:0/4263990158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c0e3260 msgr2=0x7f375c1884a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.457+0000 7f376a29d640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c0e3260 0x7f375c1884a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.457+0000 7f376a29d640 1 -- 172.21.15.31:0/4263990158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3760009420 con 0x7f375c03e8d0 2023-11-26T07:17:39.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.457+0000 7f376a29d640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 0x7f375c0e5ee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f375400e7f0 tx=0x7f375400ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.457+0000 7f9c677fe640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c54066540 0x7f9c54068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.457+0000 7f9c677fe640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c54066540 0x7f9c54068a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00a8b0 tx=0x7f9c5c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.457+0000 7faedcff9640 1 -- 172.21.15.31:0/43625794 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faed8050c60 con 0x7faee00ccea0 2023-11-26T07:17:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.461+0000 7f9c4ffff640 1 -- 172.21.15.31:0/61462487 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9c60051b90 con 0x7f9c680cf1b0 2023-11-26T07:17:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.461+0000 7f375affd640 1 -- 172.21.15.31:0/4263990158 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754009580 con 0x7f375c03e8d0 2023-11-26T07:17:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.461+0000 7f375affd640 1 -- 172.21.15.31:0/4263990158 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375400ee40 con 0x7f375c03e8d0 2023-11-26T07:17:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.461+0000 7f375affd640 1 -- 172.21.15.31:0/4263990158 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375400cb80 con 0x7f375c03e8d0 2023-11-26T07:17:39.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.461+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c052070 con 0x7f375c03e8d0 2023-11-26T07:17:39.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.461+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c189300 con 0x7f375c03e8d0 2023-11-26T07:17:39.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.469+0000 7f375affd640 1 -- 172.21.15.31:0/4263990158 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f375400cce0 con 0x7f375c03e8d0 2023-11-26T07:17:39.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.469+0000 7f375affd640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3748066570 0x7f3748068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:39.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.469+0000 7f375affd640 1 -- 172.21.15.31:0/4263990158 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3754083d60 con 0x7f375c03e8d0 2023-11-26T07:17:39.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.469+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c0056f0 con 0x7f375c03e8d0 2023-11-26T07:17:39.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.469+0000 7f3769a9c640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3748066570 0x7f3748068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:39.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.473+0000 7f3769a9c640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3748066570 0x7f3748068a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f37600b2000 tx=0x7f3760008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.473+0000 7f375affd640 1 -- 172.21.15.31:0/4263990158 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3754051300 con 0x7f375c03e8d0 2023-11-26T07:17:39.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.673+0000 7f070438d640 1 -- 172.21.15.31:0/272992349 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f06fc09cae0 con 0x7f06fc072710 2023-11-26T07:17:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.673+0000 7f06f2ffd640 1 -- 172.21.15.31:0/272992349 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f06f40efb20 con 0x7f06fc072710 2023-11-26T07:17:39.677 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-11-26T07:17:39.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 -- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06e0066540 msgr2=0x7f06e0068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06e0066540 0x7f06e0068a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f06fc151c90 tx=0x7f06f8009190 comp rx=0 tx=0).stop 2023-11-26T07:17:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 -- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc072710 msgr2=0x7f06fc09c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc072710 0x7f06fc09c5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f06f401d240 tx=0x7f06f4019740 comp rx=0 tx=0).stop 2023-11-26T07:17:39.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 -- 172.21.15.31:0/272992349 shutdown_connections 2023-11-26T07:17:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06e0066540 0x7f06e0068a00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc15b570 0x7f06fc0aebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc0cbc50 0x7f06fc1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 --2- 172.21.15.31:0/272992349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc072710 0x7f06fc09c5a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 -- 172.21.15.31:0/272992349 >> 172.21.15.31:0/272992349 conn(0x7f06fc0b0cb0 msgr2=0x7f06fc0ac020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 -- 172.21.15.31:0/272992349 shutdown_connections 2023-11-26T07:17:39.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.677+0000 7f06f0ff9640 1 -- 172.21.15.31:0/272992349 wait complete. 2023-11-26T07:17:39.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.693+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe22815ccc0 con 0x7fe228070470 2023-11-26T07:17:39.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.693+0000 7fe216ffd640 1 -- 172.21.15.31:0/789675963 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe21c053e10 con 0x7fe228070470 2023-11-26T07:17:39.699 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2023-11-26T07:17:39.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.697+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2080664c0 msgr2=0x7fe208068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.697+0000 7fe22cf2a640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2080664c0 0x7fe208068980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe2281f72d0 tx=0x7fe218007e70 comp rx=0 tx=0).stop 2023-11-26T07:17:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.697+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe228070470 msgr2=0x7fe2280af970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.697+0000 7fe22cf2a640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe228070470 0x7fe2280af970 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00c820 tx=0x7fe21c00cce0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.701+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 shutdown_connections 2023-11-26T07:17:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.701+0000 7fe22cf2a640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2080664c0 0x7fe208068980 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.701+0000 7fe22cf2a640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22815cf50 0x7fe228153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.701+0000 7fe22cf2a640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2281595f0 0x7fe22813d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.701+0000 7fe22cf2a640 1 --2- 172.21.15.31:0/789675963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe228070470 0x7fe2280af970 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.701+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 >> 172.21.15.31:0/789675963 conn(0x7fe2280b0d90 msgr2=0x7fe228166380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.701+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 shutdown_connections 2023-11-26T07:17:39.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.701+0000 7fe22cf2a640 1 -- 172.21.15.31:0/789675963 wait complete. 2023-11-26T07:17:39.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2023-11-26T07:17:39.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.709+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6e4413f8e0 con 0x7f6e4413ce70 2023-11-26T07:17:39.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.713+0000 7f6e2b7fe640 1 -- 172.21.15.31:0/3852165479 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e3c054160 con 0x7f6e4413ce70 2023-11-26T07:17:39.716 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-11-26T07:17:39.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6c0804a100 con 0x7f6c08175af0 2023-11-26T07:17:39.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6e30066470 msgr2=0x7f6e30068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6e30066470 0x7f6e30068930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097c0 tx=0x7f6e340a6000 comp rx=0 tx=0).stop 2023-11-26T07:17:39.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 msgr2=0x7f6e4413ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 0x7f6e4413ae00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00c700 tx=0x7f6e3c00cbc0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 shutdown_connections 2023-11-26T07:17:39.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6e30066470 0x7f6e30068930 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097c0 tx=0x7f6e340a6000 comp rx=0 tx=0).stop 2023-11-26T07:17:39.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e4413ce70 0x7f6e4413ae00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cdf80 0x7f6e44138950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 --2- 172.21.15.31:0/3852165479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e44072350 0x7f6e4415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 >> 172.21.15.31:0/3852165479 conn(0x7f6e440b0d90 msgr2=0x7f6e440ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c0d7fa640 1 -- 172.21.15.31:0/357215124 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c0804a100 con 0x7f6c08175af0 2023-11-26T07:17:39.722 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2023-11-26T07:17:39.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 shutdown_connections 2023-11-26T07:17:39.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6e49565640 1 -- 172.21.15.31:0/3852165479 wait complete. 2023-11-26T07:17:39.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf4066540 msgr2=0x7f6bf4068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf4066540 0x7f6bf4068a00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c080e0b00 tx=0x7f6bfc00a350 comp rx=0 tx=0).stop 2023-11-26T07:17:39.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c08175af0 msgr2=0x7f6c081776a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c08175af0 0x7f6c081776a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6c04005fd0 tx=0x7f6c04001760 comp rx=0 tx=0).stop 2023-11-26T07:17:39.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 shutdown_connections 2023-11-26T07:17:39.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf4066540 0x7f6bf4068a00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c081752b0 0x7f6c080d9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c08175af0 0x7f6c081776a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 --2- 172.21.15.31:0/357215124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c00005190 0x7f6c080d7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.717+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 >> 172.21.15.31:0/357215124 conn(0x7f6c08009f90 msgr2=0x7f6c080e0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.721+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 shutdown_connections 2023-11-26T07:17:39.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.721+0000 7f6c15edc640 1 -- 172.21.15.31:0/357215124 wait complete. 2023-11-26T07:17:39.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.721+0000 7f6753780640 1 -- 172.21.15.31:0/2064814894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f674c0cc330 con 0x7f674c0d0510 2023-11-26T07:17:39.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.721+0000 7f6741ffb640 1 -- 172.21.15.31:0/2064814894 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6748017020 con 0x7f674c0d0510 2023-11-26T07:17:39.727 INFO:teuthology.orchestra.run.smithi031.stdout:55834574852 2023-11-26T07:17:39.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 -- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67300664a0 msgr2=0x7f6730068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67300664a0 0x7f6730068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f674c0b2d50 tx=0x7f673c009190 comp rx=0 tx=0).stop 2023-11-26T07:17:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 -- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674c0d0510 msgr2=0x7f674c0b8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674c0d0510 0x7f674c0b8e40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6748000f70 tx=0x7f674800c820 comp rx=0 tx=0).stop 2023-11-26T07:17:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 -- 172.21.15.31:0/2064814894 shutdown_connections 2023-11-26T07:17:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67300664a0 0x7f6730068960 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674c15d8f0 0x7f674c0b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674c0d0510 0x7f674c0b8e40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 --2- 172.21.15.31:0/2064814894 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674c0ced30 0x7f674c0b6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 -- 172.21.15.31:0/2064814894 >> 172.21.15.31:0/2064814894 conn(0x7f674c09c750 msgr2=0x7f674c06c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 -- 172.21.15.31:0/2064814894 shutdown_connections 2023-11-26T07:17:39.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.725+0000 7f6753780640 1 -- 172.21.15.31:0/2064814894 wait complete. 2023-11-26T07:17:39.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-11-26T07:17:39.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2023-11-26T07:17:39.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-11-26T07:17:39.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2023-11-26T07:17:39.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.777+0000 7f9c6e017640 1 -- 172.21.15.31:0/61462487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9c680cf9d0 con 0x7f9c680cf1b0 2023-11-26T07:17:39.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.777+0000 7f9c4ffff640 1 -- 172.21.15.31:0/61462487 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9c600551b0 con 0x7f9c680cf1b0 2023-11-26T07:17:39.782 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2023-11-26T07:17:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 -- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c54066540 msgr2=0x7f9c54068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c54066540 0x7f9c54068a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00a8b0 tx=0x7f9c5c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 -- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 msgr2=0x7f9c682029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 0x7f9c682029d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c60007e40 tx=0x7f9c6000e4b0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 -- 172.21.15.31:0/61462487 shutdown_connections 2023-11-26T07:17:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c54066540 0x7f9c54068a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9c6815d8f0 0x7f9c680b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c680cf1b0 0x7f9c682029d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 --2- 172.21.15.31:0/61462487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c680ced30 0x7f9c680b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 -- 172.21.15.31:0/61462487 >> 172.21.15.31:0/61462487 conn(0x7f9c6809c750 msgr2=0x7f9c68159400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 -- 172.21.15.31:0/61462487 shutdown_connections 2023-11-26T07:17:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.781+0000 7f9c6e017640 1 -- 172.21.15.31:0/61462487 wait complete. 2023-11-26T07:17:39.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.793+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7faec0005510 con 0x7faee00ccea0 2023-11-26T07:17:39.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.793+0000 7faedcff9640 1 -- 172.21.15.31:0/43625794 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faed8017030 con 0x7faee00ccea0 2023-11-26T07:17:39.799 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-11-26T07:17:39.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faecc0664a0 msgr2=0x7faecc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faecc0664a0 0x7faecc068960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faed4001ec0 tx=0x7faed4009040 comp rx=0 tx=0).stop 2023-11-26T07:17:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faee00ccea0 msgr2=0x7faee00cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faee00ccea0 0x7faee00cd320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7faee0072c70 tx=0x7faed800ccd0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 shutdown_connections 2023-11-26T07:17:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faecc0664a0 0x7faecc068960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faee00ccea0 0x7faee00cd320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faee01588d0 0x7faee01519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 --2- 172.21.15.31:0/43625794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faec80047d0 0x7faee015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 >> 172.21.15.31:0/43625794 conn(0x7faee00b0cb0 msgr2=0x7faee0070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 shutdown_connections 2023-11-26T07:17:39.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.797+0000 7faee6432640 1 -- 172.21.15.31:0/43625794 wait complete. 2023-11-26T07:17:39.804 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-11-26T07:17:39.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.813+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f375c0534b0 con 0x7f375c03e8d0 2023-11-26T07:17:39.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.813+0000 7f375affd640 1 -- 172.21.15.31:0/4263990158 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3754021020 con 0x7f375c03e8d0 2023-11-26T07:17:39.817 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2023-11-26T07:17:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3748066570 msgr2=0x7f3748068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3748066570 0x7f3748068a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f37600b2000 tx=0x7f3760008040 comp rx=0 tx=0).stop 2023-11-26T07:17:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 msgr2=0x7f375c0e5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:39.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 0x7f375c0e5ee0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f375400e7f0 tx=0x7f375400ebf0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 shutdown_connections 2023-11-26T07:17:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3748066570 0x7f3748068a30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c0e3260 0x7f375c1884a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f375c058960 0x7f375c050ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 --2- 172.21.15.31:0/4263990158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f375c03e8d0 0x7f375c0e5ee0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 >> 172.21.15.31:0/4263990158 conn(0x7f375c058ed0 msgr2=0x7f375c03dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 shutdown_connections 2023-11-26T07:17:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:39.817+0000 7f376b29f640 1 -- 172.21.15.31:0/4263990158 wait complete. 2023-11-26T07:17:39.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-11-26T07:17:39.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-11-26T07:17:40.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-26T07:17:40.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-26T07:17:40.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-26T07:17:40.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-26T07:17:40.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-26T07:17:40.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-26T07:17:40.829 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-26T07:17:40.841 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-26T07:17:40.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.889+0000 7f66a3fff640 1 Processor -- start 2023-11-26T07:17:40.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.889+0000 7f66a3fff640 1 -- start start 2023-11-26T07:17:40.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.889+0000 7f66a3fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66a4153800 0x7f66a4071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a3fff640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a4072760 con 0x7f66a40b3cd0 2023-11-26T07:17:40.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a3fff640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a40ae970 con 0x7f66a406d8a0 2023-11-26T07:17:40.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a3fff640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40721a0 con 0x7f66a4153800 2023-11-26T07:17:40.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f66a406d8a0 0x7f66a406dcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:57100/0 (socket says 172.21.15.31:57100) 2023-11-26T07:17:40.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a37fe640 1 -- 172.21.15.31:0/1921376438 learned_addr learned my addr 172.21.15.31:0/1921376438 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:40.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a2ffd640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66a4153800 0x7f66a4071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441828796 0 0) 0x7f66a40ae970 con 0x7f66a406d8a0 2023-11-26T07:17:40.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f668c003530 con 0x7f66a406d8a0 2023-11-26T07:17:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 <== mon.1 v1:172.21.15.97:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3997034324 0 0) 0x7f668c003530 con 0x7f66a406d8a0 2023-11-26T07:17:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66a4153800 msgr2=0x7f66a4071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66a4153800 0x7f66a4071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 >> v1:172.21.15.31:6789/0 conn(0x7f66a40b3cd0 legacy=0x7f66a415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 --> v1:172.21.15.97:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a4072480 con 0x7f66a406d8a0 2023-11-26T07:17:40.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 <== mon.1 v1:172.21.15.97:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f6698001680 con 0x7f66a406d8a0 2023-11-26T07:17:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 >> v1:172.21.15.97:6789/0 conn(0x7f66a406d8a0 legacy=0x7f66a406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f668c003980 0x7f668c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c0047d0 0x7f668c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f668c0051e0 0x7f668c007970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40721a0 con 0x7f668c003980 2023-11-26T07:17:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c0080c0 con 0x7f668c0047d0 2023-11-26T07:17:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668c008280 con 0x7f668c0051e0 2023-11-26T07:17:40.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.893+0000 7f66a27fc640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c0047d0 0x7f668c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.897+0000 7f66a27fc640 1 -- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f668c0051e0 msgr2=0x7f668c007970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:17:40.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.897+0000 7f66a27fc640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f668c0051e0 0x7f668c007970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.897+0000 7f66a27fc640 1 -- 172.21.15.31:0/1921376438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f668c003980 msgr2=0x7f668c003d80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-26T07:17:40.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.897+0000 7f66a2ffd640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f668c003980 0x7f668c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.897+0000 7f66a27fc640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f668c003980 0x7f668c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.897+0000 7f66a27fc640 1 -- 172.21.15.31:0/1921376438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a4072480 con 0x7f668c0047d0 2023-11-26T07:17:40.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.901+0000 7f66a2ffd640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f668c003980 0x7f668c003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:40.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.901+0000 7f66a27fc640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c0047d0 0x7f668c004c50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6694001650 tx=0x7f669401c2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0b736f4b617b238 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.901+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694005270 con 0x7f668c0047d0 2023-11-26T07:17:40.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.901+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66940a7070 con 0x7f668c0047d0 2023-11-26T07:17:40.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.901+0000 7f66a1ffb640 1 -- 172.21.15.31:0/1921376438 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694003620 con 0x7f668c0047d0 2023-11-26T07:17:40.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.905+0000 7f66a3fff640 1 -- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c0047d0 msgr2=0x7f668c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.905+0000 7f66a3fff640 1 --2- 172.21.15.31:0/1921376438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c0047d0 0x7f668c004c50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6694001650 tx=0x7f669401c2b0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.905+0000 7f66a37fe640 1 -- 172.21.15.31:0/1921376438 reap_dead start 2023-11-26T07:17:40.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.905+0000 7f66a3fff640 1 -- 172.21.15.31:0/1921376438 shutdown_connections 2023-11-26T07:17:40.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.905+0000 7f66a3fff640 1 -- 172.21.15.31:0/1921376438 >> 172.21.15.31:0/1921376438 conn(0x7f66a40b0c20 msgr2=0x7f66a40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:40.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.905+0000 7f66a3fff640 1 -- 172.21.15.31:0/1921376438 shutdown_connections 2023-11-26T07:17:40.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.905+0000 7f66a3fff640 1 -- 172.21.15.31:0/1921376438 wait complete. 2023-11-26T07:17:40.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.909+0000 7f66a3fff640 1 Processor -- start 2023-11-26T07:17:40.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.909+0000 7f66a3fff640 1 -- start start 2023-11-26T07:17:40.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.909+0000 7f66a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a414e260 0x7f66a40adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.909+0000 7f66a3fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66a41f6560 0x7f66a41511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a414e640 0x7f66a4151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a414e260 0x7f66a40adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a414e260 0x7f66a40adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59582/0 (socket says 172.21.15.31:59582) 2023-11-26T07:17:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a2ffd640 1 -- 172.21.15.31:0/4207571912 learned_addr learned my addr 172.21.15.31:0/4207571912 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a27fc640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66a41f6560 0x7f66a41511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a3fff640 1 -- 172.21.15.31:0/4207571912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a41f6a90 con 0x7f66a414e260 2023-11-26T07:17:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a3fff640 1 -- 172.21.15.31:0/4207571912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a41f26c0 con 0x7f66a414e640 2023-11-26T07:17:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a3fff640 1 -- 172.21.15.31:0/4207571912 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a41f29a0 con 0x7f66a41f6560 2023-11-26T07:17:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a37fe640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a414e640 0x7f66a4151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a27fc640 1 -- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a414e640 msgr2=0x7f66a4151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a27fc640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a414e640 0x7f66a4151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a27fc640 1 -- 172.21.15.31:0/4207571912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a414e260 msgr2=0x7f66a40adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a27fc640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a414e260 0x7f66a40adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a27fc640 1 -- 172.21.15.31:0/4207571912 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a41f2c80 con 0x7f66a41f6560 2023-11-26T07:17:40.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a37fe640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a414e640 0x7f66a4151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:40.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a27fc640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66a41f6560 0x7f66a41511f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6694009000 tx=0x7f668c005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f668bfff640 1 -- 172.21.15.31:0/4207571912 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66940b2070 con 0x7f66a41f6560 2023-11-26T07:17:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a3fff640 1 -- 172.21.15.31:0/4207571912 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a41f2f60 con 0x7f66a41f6560 2023-11-26T07:17:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.913+0000 7f66a3fff640 1 -- 172.21.15.31:0/4207571912 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a4153e80 con 0x7f66a41f6560 2023-11-26T07:17:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.917+0000 7f668bfff640 1 -- 172.21.15.31:0/4207571912 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6694007030 con 0x7f66a41f6560 2023-11-26T07:17:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.917+0000 7f668bfff640 1 -- 172.21.15.31:0/4207571912 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66940b2070 con 0x7f66a41f6560 2023-11-26T07:17:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.917+0000 7f6689ffb640 1 -- 172.21.15.31:0/4207571912 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6684005260 con 0x7f66a41f6560 2023-11-26T07:17:40.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.921+0000 7f668bfff640 1 -- 172.21.15.31:0/4207571912 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f66940a9020 con 0x7f66a41f6560 2023-11-26T07:17:40.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.921+0000 7f668bfff640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6690066530 0x7f66900689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.921+0000 7f668bfff640 1 -- 172.21.15.31:0/4207571912 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66940ee9d0 con 0x7f66a41f6560 2023-11-26T07:17:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.921+0000 7f66a2ffd640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6690066530 0x7f66900689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.925+0000 7f66a2ffd640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6690066530 0x7f66900689f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f669c0073f0 tx=0x7f669c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.925+0000 7f668bfff640 1 -- 172.21.15.31:0/4207571912 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f66940eada0 con 0x7f66a41f6560 2023-11-26T07:17:40.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.949+0000 7fb7ab7ec640 1 Processor -- start 2023-11-26T07:17:40.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.953+0000 7fb7ab7ec640 1 -- start start 2023-11-26T07:17:40.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.953+0000 7fb7ab7ec640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a40b4690 0x7fb7a40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.953+0000 7fb7ab7ec640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a41620d0 con 0x7fb7a40b25d0 2023-11-26T07:17:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.953+0000 7fb7ab7ec640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a40b2c70 con 0x7fb7a4162960 2023-11-26T07:17:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.953+0000 7fb7ab7ec640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b3510 con 0x7fb7a40b4690 2023-11-26T07:17:40.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.953+0000 7fb7a9588640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb7a4162960 0x7fb7a40b3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:57104/0 (socket says 172.21.15.31:57104) 2023-11-26T07:17:40.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.953+0000 7fb7a9588640 1 -- 172.21.15.31:0/2169405021 learned_addr learned my addr 172.21.15.31:0/2169405021 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:40.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb7a8d87640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a40b4690 0x7fb7a40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716893401 0 0) 0x7fb7a41620d0 con 0x7fb7a40b25d0 2023-11-26T07:17:40.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb790003530 con 0x7fb7a40b25d0 2023-11-26T07:17:40.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2625752510 0 0) 0x7fb790003530 con 0x7fb7a40b25d0 2023-11-26T07:17:40.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a40b4690 msgr2=0x7fb7a40b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a40b4690 0x7fb7a40b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 >> v1:172.21.15.97:6789/0 conn(0x7fb7a4162960 legacy=0x7fb7a40b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a40b2f50 con 0x7fb7a40b25d0 2023-11-26T07:17:40.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fb794001680 con 0x7fb7a40b25d0 2023-11-26T07:17:40.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 >> v1:172.21.15.31:6789/0 conn(0x7fb7a40b25d0 legacy=0x7fb7a4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 0x7fb790003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7900047d0 0x7fb790004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a40b4690 0x7fb79000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b3510 con 0x7fb7900039d0 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb790005540 con 0x7fb7900047d0 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79001cee0 con 0x7fb7a40b4690 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb7a9588640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 0x7fb790003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb7a8d87640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7900047d0 0x7fb790004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb7a9d89640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a40b4690 0x7fb79000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb7a9588640 1 -- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a40b4690 msgr2=0x7fb79000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb7a9588640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a40b4690 0x7fb79000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb7a9588640 1 -- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7900047d0 msgr2=0x7fb790004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb7a9588640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7900047d0 0x7fb790004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.957+0000 7fb7a9588640 1 -- 172.21.15.31:0/2169405021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a41620d0 con 0x7fb7900039d0 2023-11-26T07:17:40.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.965+0000 7fca9c0d6640 1 Processor -- start 2023-11-26T07:17:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.965+0000 7fca9c0d6640 1 -- start start 2023-11-26T07:17:40.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.965+0000 7fb7a9588640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 0x7fb790003e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb79c002200 tx=0x7fb79c01cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5340cc603cc1aa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0a56f0 con 0x7fb7900039d0 2023-11-26T07:17:40.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fb79bfff640 1 -- 172.21.15.31:0/2169405021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c0a5850 con 0x7fb7900039d0 2023-11-26T07:17:40.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fb7ab7ec640 1 -- 172.21.15.31:0/2169405021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 msgr2=0x7fb790003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fb7ab7ec640 1 --2- 172.21.15.31:0/2169405021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 0x7fb790003e10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb79c002200 tx=0x7fb79c01cb20 comp rx=0 tx=0).stop 2023-11-26T07:17:40.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fb7a9d89640 1 -- 172.21.15.31:0/2169405021 reap_dead start 2023-11-26T07:17:40.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca9c0d6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca940d1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca9c0d6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca940d08d0 0x7fca940b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca9c0d6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9415d8d0 con 0x7fca940d2da0 2023-11-26T07:17:40.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca9c0d6640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9406c640 con 0x7fca940d08d0 2023-11-26T07:17:40.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca9c0d6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca940d1d00 con 0x7fca9415d4f0 2023-11-26T07:17:40.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca99e72640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca940d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca99e72640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca940d1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48342/0 (socket says 172.21.15.31:48342) 2023-11-26T07:17:40.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca99e72640 1 -- 172.21.15.31:0/1413108758 learned_addr learned my addr 172.21.15.31:0/1413108758 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:40.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fe7c0e46640 1 Processor -- start 2023-11-26T07:17:40.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fb7ab7ec640 1 -- 172.21.15.31:0/2169405021 shutdown_connections 2023-11-26T07:17:40.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fb7ab7ec640 1 -- 172.21.15.31:0/2169405021 >> 172.21.15.31:0/2169405021 conn(0x7fb7a406c340 msgr2=0x7fb7a406c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:40.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca99e72640 1 -- 172.21.15.31:0/1413108758 >> v2:172.21.15.31:3300/0 conn(0x7fca940d08d0 msgr2=0x7fca940b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca9a673640 1 --2- 172.21.15.31:0/1413108758 >> v2:172.21.15.31:3300/0 conn(0x7fca940d08d0 0x7fca940b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fe7c0e46640 1 -- start start 2023-11-26T07:17:40.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fb7ab7ec640 1 -- 172.21.15.31:0/2169405021 shutdown_connections 2023-11-26T07:17:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fb7ab7ec640 1 -- 172.21.15.31:0/2169405021 wait complete. 2023-11-26T07:17:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.973+0000 7fe7c0e46640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7bc0b3c60 0x7fe7bc0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.973+0000 7fe7c0e46640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7bc0b4040 0x7fe7bc0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.973+0000 7fe7c0e46640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0b2590 con 0x7fe7bc162190 2023-11-26T07:17:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.973+0000 7fe7c0e46640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7bc0b2870 con 0x7fe7bc0b3c60 2023-11-26T07:17:40.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.973+0000 7fe7c0e46640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7bc0b2b50 con 0x7fe7bc0b4040 2023-11-26T07:17:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca99e72640 1 --2- 172.21.15.31:0/1413108758 >> v2:172.21.15.31:3300/0 conn(0x7fca940d08d0 0x7fca940b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca99e72640 1 -- 172.21.15.31:0/1413108758 >> v1:172.21.15.97:6789/0 conn(0x7fca940d2da0 legacy=0x7fca940d01c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-26T07:17:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca99e72640 1 -- 172.21.15.31:0/1413108758 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca940d1fe0 con 0x7fca9415d4f0 2023-11-26T07:17:40.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.969+0000 7fca9a673640 1 --2- 172.21.15.31:0/1413108758 >> v2:172.21.15.31:3300/0 conn(0x7fca940d08d0 0x7fca940b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.973+0000 7fe7bad9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe7bc162190 0x7fe7bc165d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46312/0 (socket says 172.21.15.31:46312) 2023-11-26T07:17:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.973+0000 7fe7bad9d640 1 -- 172.21.15.31:0/3982714103 learned_addr learned my addr 172.21.15.31:0/3982714103 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.973+0000 7fb7ab7ec640 1 Processor -- start 2023-11-26T07:17:40.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.973+0000 7fe7b9d9b640 1 --2- 172.21.15.31:0/3982714103 >> v2:172.21.15.97:3300/0 conn(0x7fe7bc0b4040 0x7fe7bc0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.973+0000 7fe7b959a640 1 -- 172.21.15.31:0/3982714103 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864511816 0 0) 0x7fe7bc0b2590 con 0x7fe7bc162190 2023-11-26T07:17:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7ab7ec640 1 -- start start 2023-11-26T07:17:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fe7b959a640 1 -- 172.21.15.31:0/3982714103 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a4003530 con 0x7fe7bc162190 2023-11-26T07:17:40.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fe7ba59c640 1 --2- 172.21.15.31:0/3982714103 >> v2:172.21.15.31:3300/0 conn(0x7fe7bc0b3c60 0x7fe7bc0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7ab7ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 0x7fb7a4144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7ab7ec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7a40cd290 0x7fb7a41312c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7ab7ec640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a4131800 0x7fb7a41650c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7a9588640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 0x7fb7a4144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7a9588640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 0x7fb7a4144ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59602/0 (socket says 172.21.15.31:59602) 2023-11-26T07:17:40.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7a9d89640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a4131800 0x7fb7a41650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7a9d89640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a4131800 0x7fb7a41650c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48344/0 (socket says 172.21.15.31:48344) 2023-11-26T07:17:40.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7a8d87640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7a40cd290 0x7fb7a41312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7a8d87640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7a40cd290 0x7fb7a41312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59800/0 (socket says 172.21.15.31:59800) 2023-11-26T07:17:40.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7a9588640 1 -- 172.21.15.31:0/1234782851 learned_addr learned my addr 172.21.15.31:0/1234782851 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:40.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fca99e72640 1 --2- 172.21.15.31:0/1413108758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca940d1600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca90009870 tx=0x7fca9001c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5af756365c9e3329 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7ab7ec640 1 -- 172.21.15.31:0/1234782851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4145220 con 0x7fb7900039d0 2023-11-26T07:17:40.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7ab7ec640 1 -- 172.21.15.31:0/1234782851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40cce00 con 0x7fb7a40cd290 2023-11-26T07:17:40.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fb7ab7ec640 1 -- 172.21.15.31:0/1234782851 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4131db0 con 0x7fb7a4131800 2023-11-26T07:17:40.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb7a8d87640 1 -- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a4131800 msgr2=0x7fb7a41650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb7a8d87640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a4131800 0x7fb7a41650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb7a8d87640 1 -- 172.21.15.31:0/1234782851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 msgr2=0x7fb7a4144ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb7a8d87640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 0x7fb7a4144ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb7a8d87640 1 -- 172.21.15.31:0/1234782851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a4165c10 con 0x7fb7a40cd290 2023-11-26T07:17:40.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb7a9d89640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a4131800 0x7fb7a41650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:40.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fe7b9d9b640 1 -- 172.21.15.31:0/3982714103 >> v2:172.21.15.31:3300/0 conn(0x7fe7bc0b3c60 msgr2=0x7fe7bc0b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:40.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fe7b9d9b640 1 --2- 172.21.15.31:0/3982714103 >> v2:172.21.15.31:3300/0 conn(0x7fe7bc0b3c60 0x7fe7bc0b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fe7b9d9b640 1 -- 172.21.15.31:0/3982714103 >> v1:172.21.15.31:6789/0 conn(0x7fe7bc162190 legacy=0x7fe7bc165d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.977+0000 7fe7b9d9b640 1 -- 172.21.15.31:0/3982714103 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc0b2e30 con 0x7fe7bc0b4040 2023-11-26T07:17:40.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fe7ba59c640 1 --2- 172.21.15.31:0/3982714103 >> v2:172.21.15.31:3300/0 conn(0x7fe7bc0b3c60 0x7fe7bc0b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:40.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb7a8d87640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7a40cd290 0x7fb7a41312c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000e640 tx=0x7fb7a000ea40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fe7b9d9b640 1 --2- 172.21.15.31:0/3982714103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b4040 0x7fe7bc0ce5e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0b5200 tx=0x7fe7ac01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f8ac205bf7cf03 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fca98e70640 1 -- 172.21.15.31:0/1413108758 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca900aa620 con 0x7fca9415d4f0 2023-11-26T07:17:40.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fca98e70640 1 -- 172.21.15.31:0/1413108758 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca900aa780 con 0x7fca9415d4f0 2023-11-26T07:17:40.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb799ffb640 1 -- 172.21.15.31:0/1234782851 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0004030 con 0x7fb7a40cd290 2023-11-26T07:17:40.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fe7b959a640 1 -- 172.21.15.31:0/3982714103 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0ac620 con 0x7fe7bc0b4040 2023-11-26T07:17:40.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fca98e70640 1 -- 172.21.15.31:0/1413108758 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca900aab10 con 0x7fca9415d4f0 2023-11-26T07:17:40.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb799ffb640 1 -- 172.21.15.31:0/1234782851 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a0004190 con 0x7fb7a40cd290 2023-11-26T07:17:40.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fe7b959a640 1 -- 172.21.15.31:0/3982714103 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac0ac780 con 0x7fe7bc0b4040 2023-11-26T07:17:40.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fe7b959a640 1 -- 172.21.15.31:0/3982714103 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0acb10 con 0x7fe7bc0b4040 2023-11-26T07:17:40.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb7ab7ec640 1 -- 172.21.15.31:0/1234782851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a4165ef0 con 0x7fb7a40cd290 2023-11-26T07:17:40.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb7ab7ec640 1 -- 172.21.15.31:0/1234782851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a40cd710 con 0x7fb7a40cd290 2023-11-26T07:17:40.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fb799ffb640 1 -- 172.21.15.31:0/1234782851 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0004520 con 0x7fb7a40cd290 2023-11-26T07:17:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fe7c0e46640 1 -- 172.21.15.31:0/3982714103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b4040 msgr2=0x7fe7bc0ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.981+0000 7fe7c0e46640 1 --2- 172.21.15.31:0/3982714103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b4040 0x7fe7bc0ce5e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0b5200 tx=0x7fe7ac01c450 comp rx=0 tx=0).stop 2023-11-26T07:17:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fe7c0e46640 1 -- 172.21.15.31:0/3982714103 shutdown_connections 2023-11-26T07:17:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fe7c0e46640 1 --2- 172.21.15.31:0/3982714103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b4040 0x7fe7bc0ce5e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fe7c0e46640 1 --2- 172.21.15.31:0/3982714103 >> v2:172.21.15.31:3300/0 conn(0x7fe7bc0b3c60 0x7fe7bc0b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fe7c0e46640 1 -- 172.21.15.31:0/3982714103 >> 172.21.15.31:0/3982714103 conn(0x7fe7bc0b0cb0 msgr2=0x7fe7bc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fca9c0d6640 1 -- 172.21.15.31:0/1413108758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 msgr2=0x7fca940d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fca9c0d6640 1 --2- 172.21.15.31:0/1413108758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca940d1600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fca90009870 tx=0x7fca9001c500 comp rx=0 tx=0).stop 2023-11-26T07:17:40.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fb78f7fe640 1 -- 172.21.15.31:0/1234782851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a40cc8f0 con 0x7fb7a40cd290 2023-11-26T07:17:40.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fca9c0d6640 1 -- 172.21.15.31:0/1413108758 shutdown_connections 2023-11-26T07:17:40.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fca9c0d6640 1 --2- 172.21.15.31:0/1413108758 >> v2:172.21.15.31:3300/0 conn(0x7fca940d08d0 0x7fca940b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fca9c0d6640 1 --2- 172.21.15.31:0/1413108758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca940d1600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fca9c0d6640 1 -- 172.21.15.31:0/1413108758 >> 172.21.15.31:0/1413108758 conn(0x7fca940666a0 msgr2=0x7fca94066ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:40.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fca9c0d6640 1 -- 172.21.15.31:0/1413108758 shutdown_connections 2023-11-26T07:17:40.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fe7c0e46640 1 -- 172.21.15.31:0/3982714103 shutdown_connections 2023-11-26T07:17:40.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fca9c0d6640 1 -- 172.21.15.31:0/1413108758 wait complete. 2023-11-26T07:17:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fb799ffb640 1 -- 172.21.15.31:0/1234782851 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb7a0015690 con 0x7fb7a40cd290 2023-11-26T07:17:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fb799ffb640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb788066470 0x7fb788068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fb799ffb640 1 -- 172.21.15.31:0/1234782851 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7a00846b0 con 0x7fb7a40cd290 2023-11-26T07:17:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca9c0d6640 1 Processor -- start 2023-11-26T07:17:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fb7a9588640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb788066470 0x7fb788068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.985+0000 7fe7c0e46640 1 -- 172.21.15.31:0/3982714103 wait complete. 2023-11-26T07:17:40.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7c0e46640 1 Processor -- start 2023-11-26T07:17:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca9c0d6640 1 -- start start 2023-11-26T07:17:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7c0e46640 1 -- start start 2023-11-26T07:17:40.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca9c0d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca940d08d0 0x7fca9414af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7c0e46640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b3c60 0x7fe7bc152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fb7a9588640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb788066470 0x7fb788068930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb79c002200 tx=0x7fb79c001450 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:40.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca9c0d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca940d2da0 0x7fca940cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca9c0d6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca9414de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7c0e46640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7bc162190 0x7fe7bc159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca9c0d6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9414b460 con 0x7fca940d08d0 2023-11-26T07:17:40.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7ba59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b3c60 0x7fe7bc152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7ba59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b3c60 0x7fe7bc152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59804/0 (socket says 172.21.15.31:59804) 2023-11-26T07:17:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7ba59c640 1 -- 172.21.15.31:0/2068402939 learned_addr learned my addr 172.21.15.31:0/2068402939 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7c0e46640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc09b780 0x7fe7bc0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc15a640 con 0x7fe7bc09b780 2023-11-26T07:17:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc159ac0 con 0x7fe7bc0b3c60 2023-11-26T07:17:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc159da0 con 0x7fe7bc162190 2023-11-26T07:17:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7bad9d640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc09b780 0x7fe7bc0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:40.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fb799ffb640 1 -- 172.21.15.31:0/1234782851 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb7a40cc8f0 con 0x7fb7a40cd290 2023-11-26T07:17:40.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7ba59c640 1 -- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7bc162190 msgr2=0x7fe7bc159190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-26T07:17:40.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7ba59c640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7bc162190 0x7fe7bc159190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7ba59c640 1 -- 172.21.15.31:0/2068402939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc09b780 msgr2=0x7fe7bc0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:40.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7ba59c640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc09b780 0x7fe7bc0acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:40.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7ba59c640 1 -- 172.21.15.31:0/2068402939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac009420 con 0x7fe7bc0b3c60 2023-11-26T07:17:40.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7ba59c640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b3c60 0x7fe7bc152580 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000c7a0 tx=0x7fe7b000cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca9c0d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9414b690 con 0x7fca940d2da0 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca9c0d6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9414b970 con 0x7fca9415d4f0 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca99e72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca940d08d0 0x7fca9414af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca99e72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca940d08d0 0x7fca9414af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59608/0 (socket says 172.21.15.31:59608) 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca99e72640 1 -- 172.21.15.31:0/2401739864 learned_addr learned my addr 172.21.15.31:0/2401739864 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca99671640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca940d2da0 0x7fca940cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.989+0000 7fca9a673640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca9414de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca9a673640 1 -- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca940d2da0 msgr2=0x7fca940cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca9a673640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca940d2da0 0x7fca940cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca9a673640 1 -- 172.21.15.31:0/2401739864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca940d08d0 msgr2=0x7fca9414af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca9a673640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca940d08d0 0x7fca9414af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca9a673640 1 -- 172.21.15.31:0/2401739864 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca900094d0 con 0x7fca9415d4f0 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca99671640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca940d2da0 0x7fca940cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca9a673640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca9414de60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca9406c320 tx=0x7fca8c00a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca8a7fc640 1 -- 172.21.15.31:0/2401739864 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c003ba0 con 0x7fca9415d4f0 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca8a7fc640 1 -- 172.21.15.31:0/2401739864 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c003d00 con 0x7fca9415d4f0 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca8a7fc640 1 -- 172.21.15.31:0/2401739864 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c004260 con 0x7fca9415d4f0 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca9c0d6640 1 -- 172.21.15.31:0/2401739864 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9414bc50 con 0x7fca9415d4f0 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fca9c0d6640 1 -- 172.21.15.31:0/2401739864 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9414c210 con 0x7fca9415d4f0 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7aaffd640 1 -- 172.21.15.31:0/2068402939 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000aa60 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7aaffd640 1 -- 172.21.15.31:0/2068402939 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b000abc0 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7aaffd640 1 -- 172.21.15.31:0/2068402939 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b00113f0 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc15a080 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.993+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc152ac0 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.997+0000 7fca9c0d6640 1 -- 172.21.15.31:0/2401739864 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca940ca2e0 con 0x7fca9415d4f0 2023-11-26T07:17:41.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.997+0000 7fca8a7fc640 1 -- 172.21.15.31:0/2401739864 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fca8c0043c0 con 0x7fca9415d4f0 2023-11-26T07:17:41.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.997+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a0005260 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:40.997+0000 7fca8a7fc640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fca78066540 0x7fca78068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.001+0000 7fca8a7fc640 1 -- 172.21.15.31:0/2401739864 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca8c084770 con 0x7fca9415d4f0 2023-11-26T07:17:41.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.001+0000 7fca99e72640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fca78066540 0x7fca78068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.001+0000 7fe7aaffd640 1 -- 172.21.15.31:0/2068402939 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe7b0011550 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.001+0000 7fe7aaffd640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe79c066540 0x7fe79c068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.001+0000 7fca99e72640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fca78066540 0x7fca78068a00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fca940cd5a0 tx=0x7fca900a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.001+0000 7fe7aaffd640 1 -- 172.21.15.31:0/2068402939 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7b0083fc0 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.005+0000 7fe7b9d9b640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe79c066540 0x7fe79c068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.005+0000 7fca8a7fc640 1 -- 172.21.15.31:0/2401739864 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fca8c051bd0 con 0x7fca9415d4f0 2023-11-26T07:17:41.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.005+0000 7fe7b9d9b640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe79c066540 0x7fe79c068a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0ae000 tx=0x7fe7ac0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.009+0000 7fe7aaffd640 1 -- 172.21.15.31:0/2068402939 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe7b00514d0 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.065+0000 7fe16a232640 1 Processor -- start 2023-11-26T07:17:41.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.065+0000 7fe16a232640 1 -- start start 2023-11-26T07:17:41.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.069+0000 7fe16a232640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640cc860 0x7fe164169fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.069+0000 7fe16a232640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe164070ac0 con 0x7fe1640b4350 2023-11-26T07:17:41.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.069+0000 7fe16a232640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16408abd0 con 0x7fe1640b2aa0 2023-11-26T07:17:41.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.069+0000 7fe16a232640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16416a4e0 con 0x7fe1640cc860 2023-11-26T07:17:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.069+0000 7fe1637fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640cc860 0x7fe164169fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.069+0000 7fe1637fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640cc860 0x7fe164169fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48364/0 (socket says 172.21.15.31:48364) 2023-11-26T07:17:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.069+0000 7fe1637fe640 1 -- 172.21.15.31:0/581960216 learned_addr learned my addr 172.21.15.31:0/581960216 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.069+0000 7fe1637fe640 1 -- 172.21.15.31:0/581960216 >> v1:172.21.15.97:6789/0 conn(0x7fe1640b2aa0 legacy=0x7fe1640b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.069+0000 7fe1637fe640 1 -- 172.21.15.31:0/581960216 >> v1:172.21.15.31:6789/0 conn(0x7fe1640b4350 legacy=0x7fe164070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.069+0000 7fe1637fe640 1 -- 172.21.15.31:0/581960216 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1640b3060 con 0x7fe1640cc860 2023-11-26T07:17:41.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.073+0000 7f66d610e640 1 Processor -- start 2023-11-26T07:17:41.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.073+0000 7f66d610e640 1 -- start start 2023-11-26T07:17:41.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.073+0000 7fe1637fe640 1 --2- 172.21.15.31:0/581960216 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640cc860 0x7fe164169fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009e80 tx=0x7fe15c01c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc01ad17d445792 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.073+0000 7fe1627fc640 1 -- 172.21.15.31:0/581960216 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0a5880 con 0x7fe1640cc860 2023-11-26T07:17:41.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.073+0000 7fe1627fc640 1 -- 172.21.15.31:0/581960216 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c0a59e0 con 0x7fe1640cc860 2023-11-26T07:17:41.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.073+0000 7fe1627fc640 1 -- 172.21.15.31:0/581960216 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0a5d70 con 0x7fe1640cc860 2023-11-26T07:17:41.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.073+0000 7f66d610e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f66d01593c0 0x7f66d0070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.073+0000 7f66d610e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f66d00709b0 0x7f66d00b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66d610e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d0070e40 con 0x7f66d006e920 2023-11-26T07:17:41.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66d610e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d00cd860 con 0x7f66d01593c0 2023-11-26T07:17:41.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66d610e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d0159820 con 0x7f66d00709b0 2023-11-26T07:17:41.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7fe16a232640 1 -- 172.21.15.31:0/581960216 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640cc860 msgr2=0x7fe164169fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7fe16a232640 1 --2- 172.21.15.31:0/581960216 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640cc860 0x7fe164169fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009e80 tx=0x7fe15c01c9e0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7fe16a232640 1 -- 172.21.15.31:0/581960216 shutdown_connections 2023-11-26T07:17:41.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7fe16a232640 1 --2- 172.21.15.31:0/581960216 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640cc860 0x7fe164169fa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7fe16a232640 1 -- 172.21.15.31:0/581960216 >> 172.21.15.31:0/581960216 conn(0x7fe16406c340 msgr2=0x7fe16406c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7fe16a232640 1 -- 172.21.15.31:0/581960216 shutdown_connections 2023-11-26T07:17:41.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7fe16a232640 1 -- 172.21.15.31:0/581960216 wait complete. 2023-11-26T07:17:41.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66cf7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f66d006e920 0x7f66d00b48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46324/0 (socket says 172.21.15.31:46324) 2023-11-26T07:17:41.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66cf7fe640 1 -- 172.21.15.31:0/1760048146 learned_addr learned my addr 172.21.15.31:0/1760048146 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:41.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66cffff640 1 --2- 172.21.15.31:0/1760048146 >> v2:172.21.15.97:3300/0 conn(0x7f66d00709b0 0x7f66d00b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671626968 0 0) 0x7f66d0070e40 con 0x7f66d006e920 2023-11-26T07:17:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66b8003530 con 0x7f66d006e920 2023-11-26T07:17:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2590084040 0 0) 0x7f66b8003530 con 0x7f66d006e920 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 >> v2:172.21.15.97:3300/0 conn(0x7f66d00709b0 msgr2=0x7f66d00b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 --2- 172.21.15.31:0/1760048146 >> v2:172.21.15.97:3300/0 conn(0x7f66d00709b0 0x7f66d00b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 >> v2:172.21.15.31:3300/0 conn(0x7f66d01593c0 msgr2=0x7f66d0070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 --2- 172.21.15.31:0/1760048146 >> v2:172.21.15.31:3300/0 conn(0x7f66d01593c0 0x7f66d0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d00b50c0 con 0x7f66d006e920 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f66c0001680 con 0x7f66d006e920 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 >> v1:172.21.15.31:6789/0 conn(0x7f66d006e920 legacy=0x7f66d00b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b8003980 0x7f66b8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b80047d0 0x7f66b8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66b8009980 0x7f66b8005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d00cd860 con 0x7f66b80047d0 2023-11-26T07:17:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0159820 con 0x7f66b8003980 2023-11-26T07:17:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b8014970 con 0x7f66b8009980 2023-11-26T07:17:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.077+0000 7fe16a232640 1 Processor -- start 2023-11-26T07:17:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe16a232640 1 -- start start 2023-11-26T07:17:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7f66cf7fe640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b8003980 0x7f66b8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7f66ceffd640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b80047d0 0x7f66b8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7f66cffff640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66b8009980 0x7f66b8005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7f66cffff640 1 -- 172.21.15.31:0/1760048146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b8003980 msgr2=0x7f66b8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7f66cffff640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66b8003980 0x7f66b8003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7f66cffff640 1 -- 172.21.15.31:0/1760048146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b80047d0 msgr2=0x7f66b8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe16a232640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640b2aa0 0x7fe1642069b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe16a232640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1640b4350 0x7fe1641639f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe16a232640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1640cc860 0x7fe164165640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe1637fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640b2aa0 0x7fe1642069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe1637fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640b2aa0 0x7fe1642069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48394/0 (socket says 172.21.15.31:48394) 2023-11-26T07:17:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe162ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1640b4350 0x7fe1641639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe163fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1640cc860 0x7fe164165640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe162ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1640b4350 0x7fe1641639f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59828/0 (socket says 172.21.15.31:59828) 2023-11-26T07:17:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe163fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1640cc860 0x7fe164165640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59648/0 (socket says 172.21.15.31:59648) 2023-11-26T07:17:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe163fff640 1 -- 172.21.15.31:0/1631584661 learned_addr learned my addr 172.21.15.31:0/1631584661 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164163f30 con 0x7fe1640cc860 2023-11-26T07:17:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164172e90 con 0x7fe1640b4350 2023-11-26T07:17:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe164164140 con 0x7fe1640b2aa0 2023-11-26T07:17:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe163fff640 1 -- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640b2aa0 msgr2=0x7fe1642069b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe163fff640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640b2aa0 0x7fe1642069b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe163fff640 1 -- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1640b4350 msgr2=0x7fe1641639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe163fff640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1640b4350 0x7fe1641639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7fe163fff640 1 -- 172.21.15.31:0/1631584661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c009b30 con 0x7fe1640cc860 2023-11-26T07:17:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7fe163fff640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1640cc860 0x7fe164165640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe15800c410 tx=0x7fe15800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7fe14bfff640 1 -- 172.21.15.31:0/1631584661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158003600 con 0x7fe1640cc860 2023-11-26T07:17:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7fe14bfff640 1 -- 172.21.15.31:0/1631584661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158003760 con 0x7fe1640cc860 2023-11-26T07:17:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7f66cffff640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66b80047d0 0x7f66b8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.081+0000 7f66cffff640 1 -- 172.21.15.31:0/1760048146 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d00b50c0 con 0x7f66b8009980 2023-11-26T07:17:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7f66cffff640 1 -- 172.21.15.31:0/1760048146 reap_dead start 2023-11-26T07:17:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7fe14bfff640 1 -- 172.21.15.31:0/1631584661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158003b10 con 0x7fe1640cc860 2023-11-26T07:17:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe164164420 con 0x7fe1640cc860 2023-11-26T07:17:41.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1641649e0 con 0x7fe1640cc860 2023-11-26T07:17:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7f66cffff640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66b8009980 0x7f66b8005260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f66c400ba40 tx=0x7f66c401c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c401f7d7aa684456 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4006c70 con 0x7f66b8009980 2023-11-26T07:17:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c40a9070 con 0x7f66b8009980 2023-11-26T07:17:41.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.085+0000 7f66ce7fc640 1 -- 172.21.15.31:0/1760048146 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4005be0 con 0x7f66b8009980 2023-11-26T07:17:41.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.089+0000 7fe14bfff640 1 -- 172.21.15.31:0/1631584661 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe158003c70 con 0x7fe1640cc860 2023-11-26T07:17:41.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.089+0000 7fe14bfff640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe1500664c0 0x7fe150068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.093+0000 7fe14bfff640 1 -- 172.21.15.31:0/1631584661 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe158083070 con 0x7fe1640cc860 2023-11-26T07:17:41.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.093+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe164074990 con 0x7fe1640cc860 2023-11-26T07:17:41.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.093+0000 7f66d610e640 1 -- 172.21.15.31:0/1760048146 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66b8009980 msgr2=0x7f66b8005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.093+0000 7f66d610e640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66b8009980 0x7f66b8005260 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f66c400ba40 tx=0x7f66c401c9e0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.093+0000 7fe1637fe640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe1500664c0 0x7fe150068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.093+0000 7f66d610e640 1 -- 172.21.15.31:0/1760048146 shutdown_connections 2023-11-26T07:17:41.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.093+0000 7f66d610e640 1 --2- 172.21.15.31:0/1760048146 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66b8009980 0x7f66b8005260 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.093+0000 7f66d610e640 1 -- 172.21.15.31:0/1760048146 >> 172.21.15.31:0/1760048146 conn(0x7f66d006c340 msgr2=0x7f66d006c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.093+0000 7f66d610e640 1 -- 172.21.15.31:0/1760048146 shutdown_connections 2023-11-26T07:17:41.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.093+0000 7f66d610e640 1 -- 172.21.15.31:0/1760048146 wait complete. 2023-11-26T07:17:41.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7fe1637fe640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe1500664c0 0x7fe150068980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0ae000 tx=0x7fe15c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66d610e640 1 Processor -- start 2023-11-26T07:17:41.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66d610e640 1 -- start start 2023-11-26T07:17:41.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66d610e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d0135f90 0x7f66d0150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66d610e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d01efae0 0x7f66d014dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66d610e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66d01eee40 0x7f66d01ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66d610e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d0136370 con 0x7f66d01efae0 2023-11-26T07:17:41.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66d610e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d01ef800 con 0x7f66d0135f90 2023-11-26T07:17:41.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66d610e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d014fa10 con 0x7f66d01eee40 2023-11-26T07:17:41.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66cf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d0135f90 0x7f66d0150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66cffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66d01eee40 0x7f66d01ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66cf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d0135f90 0x7f66d0150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59844/0 (socket says 172.21.15.31:59844) 2023-11-26T07:17:41.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66cf7fe640 1 -- 172.21.15.31:0/3991369746 learned_addr learned my addr 172.21.15.31:0/3991369746 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66cffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66d01eee40 0x7f66d01ef2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48396/0 (socket says 172.21.15.31:48396) 2023-11-26T07:17:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.097+0000 7f66ceffd640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d01efae0 0x7f66d014dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66cf7fe640 1 -- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66d01eee40 msgr2=0x7f66d01ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66cf7fe640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66d01eee40 0x7f66d01ef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66cf7fe640 1 -- 172.21.15.31:0/3991369746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d01efae0 msgr2=0x7f66d014dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66cf7fe640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d01efae0 0x7f66d014dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66cf7fe640 1 -- 172.21.15.31:0/3991369746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d014fcf0 con 0x7f66d0135f90 2023-11-26T07:17:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66cf7fe640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d0135f90 0x7f66d0150ab0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66c0010c90 tx=0x7f66c000e4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66b7fff640 1 -- 172.21.15.31:0/3991369746 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c001a030 con 0x7f66d0135f90 2023-11-26T07:17:41.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66b7fff640 1 -- 172.21.15.31:0/3991369746 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c000edf0 con 0x7f66d0135f90 2023-11-26T07:17:41.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66b7fff640 1 -- 172.21.15.31:0/3991369746 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0015670 con 0x7f66d0135f90 2023-11-26T07:17:41.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7fe14bfff640 1 -- 172.21.15.31:0/1631584661 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe158017020 con 0x7fe1640cc860 2023-11-26T07:17:41.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66d610e640 1 -- 172.21.15.31:0/3991369746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d014ffd0 con 0x7f66d0135f90 2023-11-26T07:17:41.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.101+0000 7f66d610e640 1 -- 172.21.15.31:0/3991369746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d0150590 con 0x7f66d0135f90 2023-11-26T07:17:41.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.105+0000 7f66b7fff640 1 -- 172.21.15.31:0/3991369746 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f66c001d070 con 0x7f66d0135f90 2023-11-26T07:17:41.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.105+0000 7f66d610e640 1 -- 172.21.15.31:0/3991369746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d0150ff0 con 0x7f66d0135f90 2023-11-26T07:17:41.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.105+0000 7f66b7fff640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f66bc0664c0 0x7f66bc068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.109+0000 7f66b7fff640 1 -- 172.21.15.31:0/3991369746 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66c001b070 con 0x7f66d0135f90 2023-11-26T07:17:41.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.109+0000 7f66ceffd640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f66bc0664c0 0x7f66bc068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.109+0000 7f66ceffd640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f66bc0664c0 0x7f66bc068980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f66d0149630 tx=0x7f66c8010040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.113+0000 7f66b7fff640 1 -- 172.21.15.31:0/3991369746 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f66c0054180 con 0x7f66d0135f90 2023-11-26T07:17:41.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.133+0000 7fc8284fe640 1 Processor -- start 2023-11-26T07:17:41.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.133+0000 7fc8284fe640 1 -- start start 2023-11-26T07:17:41.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.137+0000 7fc8284fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8200b17d0 0x7fc82015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.137+0000 7fc8284fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b1fb0 0x7fc82006df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.137+0000 7fc8284fe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8200cbd20 con 0x7fc820072280 2023-11-26T07:17:41.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.137+0000 7fc8284fe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8200cc810 con 0x7fc8200b17d0 2023-11-26T07:17:41.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.137+0000 7fc8284fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82015cf60 con 0x7fc8200b1fb0 2023-11-26T07:17:41.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.141+0000 7fc825a99640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b1fb0 0x7fc82006df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.141+0000 7fc825a99640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b1fb0 0x7fc82006df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48400/0 (socket says 172.21.15.31:48400) 2023-11-26T07:17:41.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.141+0000 7fc825a99640 1 -- 172.21.15.31:0/438080006 learned_addr learned my addr 172.21.15.31:0/438080006 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:41.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.141+0000 7fc825a99640 1 -- 172.21.15.31:0/438080006 >> v2:172.21.15.31:3300/0 conn(0x7fc8200b17d0 msgr2=0x7fc82015ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.141+0000 7fc825a99640 1 --2- 172.21.15.31:0/438080006 >> v2:172.21.15.31:3300/0 conn(0x7fc8200b17d0 0x7fc82015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.141+0000 7fc825a99640 1 -- 172.21.15.31:0/438080006 >> v1:172.21.15.31:6789/0 conn(0x7fc820072280 legacy=0x7fc8200b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.141+0000 7fc825a99640 1 -- 172.21.15.31:0/438080006 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8200cbf70 con 0x7fc8200b1fb0 2023-11-26T07:17:41.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.145+0000 7fc825a99640 1 --2- 172.21.15.31:0/438080006 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b1fb0 0x7fc82006df60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009870 tx=0x7fc81c01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51046e0aaf98883c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.145+0000 7f15709c1640 1 Processor -- start 2023-11-26T07:17:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.145+0000 7f15709c1640 1 -- start start 2023-11-26T07:17:41.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.145+0000 7fc825298640 1 -- 172.21.15.31:0/438080006 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0aa5f0 con 0x7fc8200b1fb0 2023-11-26T07:17:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.145+0000 7fc825298640 1 -- 172.21.15.31:0/438080006 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c0aa750 con 0x7fc8200b1fb0 2023-11-26T07:17:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7fc825298640 1 -- 172.21.15.31:0/438080006 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0aaae0 con 0x7fc8200b1fb0 2023-11-26T07:17:41.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7f15709c1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f156c0b4040 0x7f156c0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7f15709c1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0b2590 con 0x7f156c162190 2023-11-26T07:17:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7f15709c1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0b2870 con 0x7f156c0b3c60 2023-11-26T07:17:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7f15709c1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0b2b50 con 0x7f156c0b4040 2023-11-26T07:17:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7f1569d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f156c0b4040 0x7f156c0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7f1569d9b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f156c0b4040 0x7f156c0b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:59852/0 (socket says 172.21.15.31:59852) 2023-11-26T07:17:41.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7f1569d9b640 1 -- 172.21.15.31:0/4072949484 learned_addr learned my addr 172.21.15.31:0/4072949484 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.153+0000 7fc8284fe640 1 -- 172.21.15.31:0/438080006 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b1fb0 msgr2=0x7fc82006df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.153+0000 7fc8284fe640 1 --2- 172.21.15.31:0/438080006 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b1fb0 0x7fc82006df60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009870 tx=0x7fc81c01c4f0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.153+0000 7fc8284fe640 1 -- 172.21.15.31:0/438080006 shutdown_connections 2023-11-26T07:17:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.153+0000 7fc8284fe640 1 --2- 172.21.15.31:0/438080006 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b1fb0 0x7fc82006df60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.153+0000 7fc8284fe640 1 --2- 172.21.15.31:0/438080006 >> v2:172.21.15.31:3300/0 conn(0x7fc8200b17d0 0x7fc82015ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.153+0000 7fc8284fe640 1 -- 172.21.15.31:0/438080006 >> 172.21.15.31:0/438080006 conn(0x7fc8200b0cb0 msgr2=0x7fc8200b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7f1569d9b640 1 -- 172.21.15.31:0/4072949484 >> v1:172.21.15.97:6789/0 conn(0x7f156c0b3c60 legacy=0x7f156c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7f1569d9b640 1 -- 172.21.15.31:0/4072949484 >> v1:172.21.15.31:6789/0 conn(0x7f156c162190 legacy=0x7f156c165d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.149+0000 7f1569d9b640 1 -- 172.21.15.31:0/4072949484 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0b2e30 con 0x7f156c0b4040 2023-11-26T07:17:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.157+0000 7fc8284fe640 1 -- 172.21.15.31:0/438080006 shutdown_connections 2023-11-26T07:17:41.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.157+0000 7fc8284fe640 1 -- 172.21.15.31:0/438080006 wait complete. 2023-11-26T07:17:41.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.157+0000 7f1569d9b640 1 --2- 172.21.15.31:0/4072949484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c0b4040 0x7f156c0b1fb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1560009870 tx=0x7f156001c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c49ad4340f5ac9f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.157+0000 7f156959a640 1 -- 172.21.15.31:0/4072949484 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15600aa610 con 0x7f156c0b4040 2023-11-26T07:17:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.157+0000 7f156959a640 1 -- 172.21.15.31:0/4072949484 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15600a6040 con 0x7f156c0b4040 2023-11-26T07:17:41.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.157+0000 7fc8284fe640 1 Processor -- start 2023-11-26T07:17:41.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.157+0000 7fc8284fe640 1 -- start start 2023-11-26T07:17:41.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7f15709c1640 1 -- 172.21.15.31:0/4072949484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c0b4040 msgr2=0x7f156c0b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7f15709c1640 1 --2- 172.21.15.31:0/4072949484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c0b4040 0x7f156c0b1fb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1560009870 tx=0x7f156001c490 comp rx=0 tx=0).stop 2023-11-26T07:17:41.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7f15709c1640 1 -- 172.21.15.31:0/4072949484 shutdown_connections 2023-11-26T07:17:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7f15709c1640 1 --2- 172.21.15.31:0/4072949484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c0b4040 0x7f156c0b1fb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7f15709c1640 1 -- 172.21.15.31:0/4072949484 >> 172.21.15.31:0/4072949484 conn(0x7f156c0b0cb0 msgr2=0x7f156c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc8284fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc820072280 0x7fc82014a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc8284fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8200b17d0 0x7fc820152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc8284fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8200b1fb0 0x7fc820152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc82629a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc820072280 0x7fc82014a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc82629a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc820072280 0x7fc82014a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48406/0 (socket says 172.21.15.31:48406) 2023-11-26T07:17:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc825a99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8200b17d0 0x7fc820152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc8284fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82014aa80 con 0x7fc8200b17d0 2023-11-26T07:17:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc825a99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8200b17d0 0x7fc820152050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59666/0 (socket says 172.21.15.31:59666) 2023-11-26T07:17:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc8284fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82014b9f0 con 0x7fc8200b1fb0 2023-11-26T07:17:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82014bcd0 con 0x7fc820072280 2023-11-26T07:17:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc825a99640 1 -- 172.21.15.31:0/650269827 learned_addr learned my addr 172.21.15.31:0/650269827 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:41.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc826a9b640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8200b1fb0 0x7fc820152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7f15709c1640 1 -- 172.21.15.31:0/4072949484 shutdown_connections 2023-11-26T07:17:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7f15709c1640 1 -- 172.21.15.31:0/4072949484 wait complete. 2023-11-26T07:17:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc82629a640 1 -- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8200b1fb0 msgr2=0x7fc820152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc82629a640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8200b1fb0 0x7fc820152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc82629a640 1 -- 172.21.15.31:0/650269827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8200b17d0 msgr2=0x7fc820152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc82629a640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8200b17d0 0x7fc820152050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.161+0000 7fc82629a640 1 -- 172.21.15.31:0/650269827 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0094d0 con 0x7fc820072280 2023-11-26T07:17:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.165+0000 7f15709c1640 1 Processor -- start 2023-11-26T07:17:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.165+0000 7f15709c1640 1 -- start start 2023-11-26T07:17:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.165+0000 7fc82629a640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc820072280 0x7fc82014a540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc81800bd70 tx=0x7fc81800be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7f15709c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b3c60 0x7f156c14b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7f15709c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c162190 0x7f156c159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7f156a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b3c60 0x7f156c14b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7f15709c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c203040 0x7f156c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7f15709c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c14be40 con 0x7f156c0b3c60 2023-11-26T07:17:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7fc816ffd640 1 -- 172.21.15.31:0/650269827 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818014440 con 0x7fc820072280 2023-11-26T07:17:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82014bfb0 con 0x7fc820072280 2023-11-26T07:17:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82014df30 con 0x7fc820072280 2023-11-26T07:17:41.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7fc816ffd640 1 -- 172.21.15.31:0/650269827 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8180145a0 con 0x7fc820072280 2023-11-26T07:17:41.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7fc816ffd640 1 -- 172.21.15.31:0/650269827 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818014440 con 0x7fc820072280 2023-11-26T07:17:41.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7f156a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b3c60 0x7f156c14b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59680/0 (socket says 172.21.15.31:59680) 2023-11-26T07:17:41.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7f156a59c640 1 -- 172.21.15.31:0/3930444382 learned_addr learned my addr 172.21.15.31:0/3930444382 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:41.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7f1569d9b640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c162190 0x7f156c159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.169+0000 7f156ad9d640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c203040 0x7f156c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c15aab0 con 0x7f156c203040 2023-11-26T07:17:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c159c40 con 0x7f156c162190 2023-11-26T07:17:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82004e850 con 0x7fc820072280 2023-11-26T07:17:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f156a59c640 1 -- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c162190 msgr2=0x7f156c159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f156a59c640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c162190 0x7f156c159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f156a59c640 1 -- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c203040 msgr2=0x7f156c0ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f156a59c640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c203040 0x7f156c0ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f156a59c640 1 -- 172.21.15.31:0/3930444382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15600094d0 con 0x7f156c0b3c60 2023-11-26T07:17:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f156ad9d640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c203040 0x7f156c0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:17:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f1569d9b640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c162190 0x7f156c159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f156a59c640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b3c60 0x7f156c14b900 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f155c00c480 tx=0x7f155c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7fc816ffd640 1 -- 172.21.15.31:0/650269827 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc81801d3b0 con 0x7fc820072280 2023-11-26T07:17:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f155affd640 1 -- 172.21.15.31:0/3930444382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c012830 con 0x7f156c0b3c60 2023-11-26T07:17:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7fc816ffd640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066500 0x7fc8040689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7fc816ffd640 1 -- 172.21.15.31:0/650269827 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc818083c70 con 0x7fc820072280 2023-11-26T07:17:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.173+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c159ea0 con 0x7f156c0b3c60 2023-11-26T07:17:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.177+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c15a460 con 0x7f156c0b3c60 2023-11-26T07:17:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.177+0000 7fc825a99640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066500 0x7fc8040689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.177+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c0671f0 con 0x7f156c0b3c60 2023-11-26T07:17:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.177+0000 7f155affd640 1 -- 172.21.15.31:0/3930444382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c0129b0 con 0x7f156c0b3c60 2023-11-26T07:17:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.177+0000 7f155affd640 1 -- 172.21.15.31:0/3930444382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c012d60 con 0x7f156c0b3c60 2023-11-26T07:17:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.177+0000 7fc825a99640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066500 0x7fc8040689c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc81c01c530 tx=0x7fc81c0b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.177+0000 7fc816ffd640 1 -- 172.21.15.31:0/650269827 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc818051100 con 0x7fc820072280 2023-11-26T07:17:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.181+0000 7f155affd640 1 -- 172.21.15.31:0/3930444382 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f155c01c460 con 0x7f156c0b3c60 2023-11-26T07:17:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.181+0000 7f155affd640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f154c066570 0x7f154c068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.181+0000 7f155affd640 1 -- 172.21.15.31:0/3930444382 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f155c083c10 con 0x7f156c0b3c60 2023-11-26T07:17:41.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.189+0000 7f1569d9b640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f154c066570 0x7f154c068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.189+0000 7f1569d9b640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f154c066570 0x7f154c068a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15600ae000 tx=0x7f15600b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.193+0000 7f155affd640 1 -- 172.21.15.31:0/3930444382 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f155c051100 con 0x7f156c0b3c60 2023-11-26T07:17:41.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.233+0000 7f6689ffb640 1 -- 172.21.15.31:0/4207571912 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f66840057e0 con 0x7f66a41f6560 2023-11-26T07:17:41.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f668bfff640 1 -- 172.21.15.31:0/4207571912 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66940eaf40 con 0x7f66a41f6560 2023-11-26T07:17:41.239 INFO:teuthology.orchestra.run.smithi031.stdout:55834574856 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f6689ffb640 1 -- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6690066530 msgr2=0x7f66900689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f6689ffb640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6690066530 0x7f66900689f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f669c0073f0 tx=0x7f669c00b040 comp rx=0 tx=0).stop 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f6689ffb640 1 -- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66a41f6560 msgr2=0x7f66a41511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f6689ffb640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66a41f6560 0x7f66a41511f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6694009000 tx=0x7f668c005800 comp rx=0 tx=0).stop 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f6689ffb640 1 -- 172.21.15.31:0/4207571912 shutdown_connections 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f6689ffb640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6690066530 0x7f66900689f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f6689ffb640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66a414e640 0x7f66a4151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f6689ffb640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66a41f6560 0x7f66a41511f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f6689ffb640 1 --2- 172.21.15.31:0/4207571912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66a414e260 0x7f66a40adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.237+0000 7f6689ffb640 1 -- 172.21.15.31:0/4207571912 >> 172.21.15.31:0/4207571912 conn(0x7f66a40b0c20 msgr2=0x7f66a40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.249+0000 7f6689ffb640 1 -- 172.21.15.31:0/4207571912 shutdown_connections 2023-11-26T07:17:41.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.249+0000 7f6689ffb640 1 -- 172.21.15.31:0/4207571912 wait complete. 2023-11-26T07:17:41.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2023-11-26T07:17:41.287 DEBUG:teuthology.parallel:result is None 2023-11-26T07:17:41.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 -- 172.21.15.31:0/2401739864 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fca94074990 con 0x7fca9415d4f0 2023-11-26T07:17:41.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca8a7fc640 1 -- 172.21.15.31:0/2401739864 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca8c0551f0 con 0x7fca9415d4f0 2023-11-26T07:17:41.316 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-11-26T07:17:41.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 -- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fca78066540 msgr2=0x7fca78068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fca78066540 0x7fca78068a00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fca940cd5a0 tx=0x7fca900a7000 comp rx=0 tx=0).stop 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 -- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 msgr2=0x7fca9414de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca9414de60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fca9406c320 tx=0x7fca8c00a710 comp rx=0 tx=0).stop 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 -- 172.21.15.31:0/2401739864 shutdown_connections 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fca78066540 0x7fca78068a00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca9415d4f0 0x7fca9414de60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca940d2da0 0x7fca940cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 --2- 172.21.15.31:0/2401739864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca940d08d0 0x7fca9414af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 -- 172.21.15.31:0/2401739864 >> 172.21.15.31:0/2401739864 conn(0x7fca940666a0 msgr2=0x7fca94168830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 -- 172.21.15.31:0/2401739864 shutdown_connections 2023-11-26T07:17:41.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.313+0000 7fca9c0d6640 1 -- 172.21.15.31:0/2401739864 wait complete. 2023-11-26T07:17:41.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-11-26T07:17:41.339 DEBUG:teuthology.parallel:result is None 2023-11-26T07:17:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 -- 172.21.15.31:0/1234782851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb7a40cc8f0 con 0x7fb7a40cd290 2023-11-26T07:17:41.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb799ffb640 1 -- 172.21.15.31:0/1234782851 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb7a40cc8f0 con 0x7fb7a40cd290 2023-11-26T07:17:41.341 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-11-26T07:17:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 -- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb788066470 msgr2=0x7fb788068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb788066470 0x7fb788068930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb79c002200 tx=0x7fb79c001450 comp rx=0 tx=0).stop 2023-11-26T07:17:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 -- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7a40cd290 msgr2=0x7fb7a41312c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7a40cd290 0x7fb7a41312c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000e640 tx=0x7fb7a000ea40 comp rx=0 tx=0).stop 2023-11-26T07:17:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 -- 172.21.15.31:0/1234782851 shutdown_connections 2023-11-26T07:17:41.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb788066470 0x7fb788068930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7a4131800 0x7fb7a41650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7a40cd290 0x7fb7a41312c0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000e640 tx=0x7fb7a000ea40 comp rx=0 tx=0).stop 2023-11-26T07:17:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 --2- 172.21.15.31:0/1234782851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7900039d0 0x7fb7a4144ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 -- 172.21.15.31:0/1234782851 >> 172.21.15.31:0/1234782851 conn(0x7fb7a406c340 msgr2=0x7fb7a40cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 -- 172.21.15.31:0/1234782851 shutdown_connections 2023-11-26T07:17:41.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.337+0000 7fb78f7fe640 1 -- 172.21.15.31:0/1234782851 wait complete. 2023-11-26T07:17:41.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.345+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe7a0005510 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.349+0000 7fe7aaffd640 1 -- 172.21.15.31:0/2068402939 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe7b00514d0 con 0x7fe7bc0b3c60 2023-11-26T07:17:41.353 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-11-26T07:17:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.349+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe79c066540 msgr2=0x7fe79c068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.349+0000 7fe7c0e46640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe79c066540 0x7fe79c068a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0ae000 tx=0x7fe7ac0a7000 comp rx=0 tx=0).stop 2023-11-26T07:17:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.353+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b3c60 msgr2=0x7fe7bc152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.353+0000 7fe7c0e46640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b3c60 0x7fe7bc152580 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000c7a0 tx=0x7fe7b000cc60 comp rx=0 tx=0).stop 2023-11-26T07:17:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.353+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 shutdown_connections 2023-11-26T07:17:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.353+0000 7fe7c0e46640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe79c066540 0x7fe79c068a00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.353+0000 7fe7c0e46640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7bc09b780 0x7fe7bc0acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.353+0000 7fe7c0e46640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7bc162190 0x7fe7bc159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.353+0000 7fe7c0e46640 1 --2- 172.21.15.31:0/2068402939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7bc0b3c60 0x7fe7bc152580 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.353+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 >> 172.21.15.31:0/2068402939 conn(0x7fe7bc0b0cb0 msgr2=0x7fe7bc0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.353+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 shutdown_connections 2023-11-26T07:17:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.353+0000 7fe7c0e46640 1 -- 172.21.15.31:0/2068402939 wait complete. 2023-11-26T07:17:41.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-11-26T07:17:41.365 DEBUG:teuthology.parallel:result is None 2023-11-26T07:17:41.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-11-26T07:17:41.389 DEBUG:teuthology.parallel:result is None 2023-11-26T07:17:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.409+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe164072080 con 0x7fe1640cc860 2023-11-26T07:17:41.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.409+0000 7fe14bfff640 1 -- 172.21.15.31:0/1631584661 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe1580504f0 con 0x7fe1640cc860 2023-11-26T07:17:41.413 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe1500664c0 msgr2=0x7fe150068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe1500664c0 0x7fe150068980 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0ae000 tx=0x7fe15c0a7070 comp rx=0 tx=0).stop 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1640cc860 msgr2=0x7fe164165640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1640cc860 0x7fe164165640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe15800c410 tx=0x7fe15800c8d0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 shutdown_connections 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe1500664c0 0x7fe150068980 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1640cc860 0x7fe164165640 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe1640b4350 0x7fe1641639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 --2- 172.21.15.31:0/1631584661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe1640b2aa0 0x7fe1642069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 >> 172.21.15.31:0/1631584661 conn(0x7fe16406c340 msgr2=0x7fe16416efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 shutdown_connections 2023-11-26T07:17:41.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.413+0000 7fe16a232640 1 -- 172.21.15.31:0/1631584661 wait complete. 2023-11-26T07:17:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc8200cb2c0 con 0x7fc820072280 2023-11-26T07:17:41.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc816ffd640 1 -- 172.21.15.31:0/650269827 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc818019090 con 0x7fc820072280 2023-11-26T07:17:41.428 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2023-11-26T07:17:41.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066500 msgr2=0x7fc8040689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066500 0x7fc8040689c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc81c01c530 tx=0x7fc81c0b5040 comp rx=0 tx=0).stop 2023-11-26T07:17:41.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc820072280 msgr2=0x7fc82014a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc820072280 0x7fc82014a540 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc81800bd70 tx=0x7fc81800be60 comp rx=0 tx=0).stop 2023-11-26T07:17:41.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 shutdown_connections 2023-11-26T07:17:41.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066500 0x7fc8040689c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8200b1fb0 0x7fc820152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8200b17d0 0x7fc820152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 --2- 172.21.15.31:0/650269827 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc820072280 0x7fc82014a540 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 >> 172.21.15.31:0/650269827 conn(0x7fc8200b0cb0 msgr2=0x7fc82015be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 shutdown_connections 2023-11-26T07:17:41.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.425+0000 7fc8284fe640 1 -- 172.21.15.31:0/650269827 wait complete. 2023-11-26T07:17:41.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-11-26T07:17:41.436 DEBUG:teuthology.parallel:result is None 2023-11-26T07:17:41.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-11-26T07:17:41.450 DEBUG:teuthology.parallel:result is None 2023-11-26T07:17:41.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.457+0000 7f66d610e640 1 -- 172.21.15.31:0/3991369746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f66d0074990 con 0x7f66d0135f90 2023-11-26T07:17:41.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.457+0000 7f66b7fff640 1 -- 172.21.15.31:0/3991369746 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66c00577a0 con 0x7f66d0135f90 2023-11-26T07:17:41.462 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2023-11-26T07:17:41.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 -- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f66bc0664c0 msgr2=0x7f66bc068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f66bc0664c0 0x7f66bc068980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f66d0149630 tx=0x7f66c8010040 comp rx=0 tx=0).stop 2023-11-26T07:17:41.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 -- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d0135f90 msgr2=0x7f66d0150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d0135f90 0x7f66d0150ab0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66c0010c90 tx=0x7f66c000e4c0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 -- 172.21.15.31:0/3991369746 shutdown_connections 2023-11-26T07:17:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f66bc0664c0 0x7f66bc068980 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66d01eee40 0x7f66d01ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66d01efae0 0x7f66d014dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 --2- 172.21.15.31:0/3991369746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66d0135f90 0x7f66d0150ab0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 -- 172.21.15.31:0/3991369746 >> 172.21.15.31:0/3991369746 conn(0x7f66d006c340 msgr2=0x7f66d00aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 -- 172.21.15.31:0/3991369746 shutdown_connections 2023-11-26T07:17:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.461+0000 7f66d610e640 1 -- 172.21.15.31:0/3991369746 wait complete. 2023-11-26T07:17:41.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2023-11-26T07:17:41.487 DEBUG:teuthology.parallel:result is None 2023-11-26T07:17:41.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.497+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f156c06b460 con 0x7f156c0b3c60 2023-11-26T07:17:41.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.497+0000 7f155affd640 1 -- 172.21.15.31:0/3930444382 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f155c051100 con 0x7f156c0b3c60 2023-11-26T07:17:41.503 INFO:teuthology.orchestra.run.smithi031.stdout:55834574855 2023-11-26T07:17:41.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f154c066570 msgr2=0x7f154c068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f154c066570 0x7f154c068a30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f15600ae000 tx=0x7f15600b5040 comp rx=0 tx=0).stop 2023-11-26T07:17:41.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b3c60 msgr2=0x7f156c14b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b3c60 0x7f156c14b900 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f155c00c480 tx=0x7f155c00c940 comp rx=0 tx=0).stop 2023-11-26T07:17:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 shutdown_connections 2023-11-26T07:17:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f154c066570 0x7f154c068a30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c203040 0x7f156c0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c162190 0x7f156c159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 --2- 172.21.15.31:0/3930444382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b3c60 0x7f156c14b900 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 >> 172.21.15.31:0/3930444382 conn(0x7f156c0b0cb0 msgr2=0x7f156c0aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 shutdown_connections 2023-11-26T07:17:41.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.501+0000 7f15709c1640 1 -- 172.21.15.31:0/3930444382 wait complete. 2023-11-26T07:17:41.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-11-26T07:17:41.524 DEBUG:teuthology.parallel:result is None 2023-11-26T07:17:41.524 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-26T07:17:41.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f721bbec640 1 Processor -- start 2023-11-26T07:17:41.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f721bbec640 1 -- start start 2023-11-26T07:17:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f721bbec640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f72140cead0 0x7f72140d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f721bbec640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140d1eb0 0x7f721406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f721bbec640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72140d2300 con 0x7f721415d7b0 2023-11-26T07:17:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f721bbec640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72140d17e0 con 0x7f72140cead0 2023-11-26T07:17:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f721bbec640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72140cefd0 con 0x7f72140d1eb0 2023-11-26T07:17:41.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7219988640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f721415d7b0 0x7f72140d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46344/0 (socket says 172.21.15.31:46344) 2023-11-26T07:17:41.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7219988640 1 -- 172.21.15.31:0/3600767749 learned_addr learned my addr 172.21.15.31:0/3600767749 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7219187640 1 --2- 172.21.15.31:0/3600767749 >> v2:172.21.15.97:3300/0 conn(0x7f72140cead0 0x7f72140d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f721a189640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140d1eb0 0x7f721406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1170980921 0 0) 0x7f7210001120 con 0x7f721415d7b0 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7200003530 con 0x7f721415d7b0 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283182710 0 0) 0x7f7200003530 con 0x7f721415d7b0 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140d1eb0 msgr2=0x7f721406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140d1eb0 0x7f721406bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 >> v2:172.21.15.97:3300/0 conn(0x7f72140cead0 msgr2=0x7f72140d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 --2- 172.21.15.31:0/3600767749 >> v2:172.21.15.97:3300/0 conn(0x7f72140cead0 0x7f72140d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72140cf2b0 con 0x7f721415d7b0 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f7210001810 con 0x7f721415d7b0 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 >> v1:172.21.15.31:6789/0 conn(0x7f721415d7b0 legacy=0x7f72140d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.705+0000 7f7218986640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 0x7f7200003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7218986640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140d1eb0 0x7f72000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7218986640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7200004d10 0x7f7200005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140d17e0 con 0x7f7200003980 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140cefd0 con 0x7f7200004d10 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720000ba10 con 0x7f72140d1eb0 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7219988640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 0x7f7200003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7219187640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140d1eb0 0x7f72000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f721a189640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7200004d10 0x7f7200005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7219988640 1 -- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140d1eb0 msgr2=0x7f72000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7219988640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140d1eb0 0x7f72000047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7219988640 1 -- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7200004d10 msgr2=0x7f7200005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7219988640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7200004d10 0x7f7200005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7219988640 1 -- 172.21.15.31:0/3600767749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7200003530 con 0x7f7200003980 2023-11-26T07:17:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f721a189640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7200004d10 0x7f7200005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7219988640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 0x7f7200003da0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f721000c210 tx=0x7f721001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47f7f8acb227a0d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72100a5910 con 0x7f7200003980 2023-11-26T07:17:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72100a5a90 con 0x7f7200003980 2023-11-26T07:17:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.709+0000 7f7218986640 1 -- 172.21.15.31:0/3600767749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72100a5e40 con 0x7f7200003980 2023-11-26T07:17:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 -- 172.21.15.31:0/3600767749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 msgr2=0x7f7200003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 --2- 172.21.15.31:0/3600767749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 0x7f7200003da0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f721000c210 tx=0x7f721001cc30 comp rx=0 tx=0).stop 2023-11-26T07:17:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721a189640 1 -- 172.21.15.31:0/3600767749 reap_dead start 2023-11-26T07:17:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 -- 172.21.15.31:0/3600767749 shutdown_connections 2023-11-26T07:17:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 -- 172.21.15.31:0/3600767749 >> 172.21.15.31:0/3600767749 conn(0x7f72140d33b0 msgr2=0x7f72140d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 -- 172.21.15.31:0/3600767749 shutdown_connections 2023-11-26T07:17:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 -- 172.21.15.31:0/3600767749 wait complete. 2023-11-26T07:17:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 Processor -- start 2023-11-26T07:17:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 -- start start 2023-11-26T07:17:41.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 0x7f7214154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140c8610 0x7f72140c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7214153e90 0x7f72140cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7214154e20 con 0x7f7200003980 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140c7570 con 0x7f7214153e90 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721bbec640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72140c7850 con 0x7f72140c8610 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f7219988640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 0x7f7214154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f7219988640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 0x7f7214154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59710/0 (socket says 172.21.15.31:59710) 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f7219988640 1 -- 172.21.15.31:0/1859320488 learned_addr learned my addr 172.21.15.31:0/1859320488 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f7219187640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140c8610 0x7f72140c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.713+0000 7f721a189640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7214153e90 0x7f72140cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f7219988640 1 -- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140c8610 msgr2=0x7f72140c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f7219988640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140c8610 0x7f72140c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f7219988640 1 -- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7214153e90 msgr2=0x7f72140cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f7219988640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7214153e90 0x7f72140cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f7219988640 1 -- 172.21.15.31:0/1859320488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72140c7b30 con 0x7f7200003980 2023-11-26T07:17:41.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f7219988640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 0x7f7214154640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7210007040 tx=0x7f7200005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f720a7fc640 1 -- 172.21.15.31:0/1859320488 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72100047f0 con 0x7f7200003980 2023-11-26T07:17:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f720a7fc640 1 -- 172.21.15.31:0/1859320488 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72100b3070 con 0x7f7200003980 2023-11-26T07:17:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f721bbec640 1 -- 172.21.15.31:0/1859320488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72140c7e10 con 0x7f7200003980 2023-11-26T07:17:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f720a7fc640 1 -- 172.21.15.31:0/1859320488 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210004b70 con 0x7f7200003980 2023-11-26T07:17:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f721bbec640 1 -- 172.21.15.31:0/1859320488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7214154310 con 0x7f7200003980 2023-11-26T07:17:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f721bbec640 1 -- 172.21.15.31:0/1859320488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721404e850 con 0x7f7200003980 2023-11-26T07:17:41.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f720a7fc640 1 -- 172.21.15.31:0/1859320488 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7210004cd0 con 0x7f7200003980 2023-11-26T07:17:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f720a7fc640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f71f80664a0 0x7f71f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.717+0000 7f720a7fc640 1 -- 172.21.15.31:0/1859320488 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72100b5070 con 0x7f7200003980 2023-11-26T07:17:41.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.721+0000 7f7219187640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f71f80664a0 0x7f71f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:41.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.721+0000 7f7219187640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f71f80664a0 0x7f71f8068960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f72140d0820 tx=0x7f7204008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:41.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.721+0000 7f720a7fc640 1 -- 172.21.15.31:0/1859320488 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f72100eb0b0 con 0x7f7200003980 2023-11-26T07:17:41.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.877+0000 7f721bbec640 1 -- 172.21.15.31:0/1859320488 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72140c82c0 con 0x7f71f80664a0 2023-11-26T07:17:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.885+0000 7f720a7fc640 1 -- 172.21.15.31:0/1859320488 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264561 (secure 0 0 0) 0x7f7204006be0 con 0x7f71f80664a0 2023-11-26T07:17:41.889 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 -- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f71f80664a0 msgr2=0x7f71f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f71f80664a0 0x7f71f8068960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f72140d0820 tx=0x7f7204008040 comp rx=0 tx=0).stop 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 -- 172.21.15.31:0/1859320488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 msgr2=0x7f7214154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 0x7f7214154640 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7210007040 tx=0x7f7200005800 comp rx=0 tx=0).stop 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 -- 172.21.15.31:0/1859320488 shutdown_connections 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f71f80664a0 0x7f71f8068960 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7214153e90 0x7f72140cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72140c8610 0x7f72140c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 --2- 172.21.15.31:0/1859320488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7200003980 0x7f7214154640 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 -- 172.21.15.31:0/1859320488 >> 172.21.15.31:0/1859320488 conn(0x7f72140d33b0 msgr2=0x7f7214158950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 -- 172.21.15.31:0/1859320488 shutdown_connections 2023-11-26T07:17:41.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:41.889+0000 7f721bbec640 1 -- 172.21.15.31:0/1859320488 wait complete. 2023-11-26T07:17:41.911 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-26T07:17:40.308219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001755"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326879+0000","last_change":"2023-11-26T07:17:33.157517+0000","last_active":"2023-11-26T07:17:34.326879+0000","last_peered":"2023-11-26T07:17:34.326879+0000","last_clean":"2023-11-26T07:17:34.326879+0000","last_became_active":"2023-11-26T07:17:33.157236+0000","last_became_peered":"2023-11-26T07:17:33.157236+0000","last_unstale":"2023-11-26T07:17:34.326879+0000","last_undegraded":"2023-11-26T07:17:34.326879+0000","last_fullsized":"2023-11-26T07:17:34.326879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:50.214046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327264+0000","last_change":"2023-11-26T07:17:33.153106+0000","last_active":"2023-11-26T07:17:34.327264+0000","last_peered":"2023-11-26T07:17:34.327264+0000","last_clean":"2023-11-26T07:17:34.327264+0000","last_became_active":"2023-11-26T07:17:33.152758+0000","last_became_peered":"2023-11-26T07:17:33.152758+0000","last_unstale":"2023-11-26T07:17:34.327264+0000","last_undegraded":"2023-11-26T07:17:34.327264+0000","last_fullsized":"2023-11-26T07:17:34.327264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328087+0000","last_change":"2023-11-26T07:17:33.119394+0000","last_active":"2023-11-26T07:17:34.328087+0000","last_peered":"2023-11-26T07:17:34.328087+0000","last_clean":"2023-11-26T07:17:34.328087+0000","last_became_active":"2023-11-26T07:17:33.118798+0000","last_became_peered":"2023-11-26T07:17:33.118798+0000","last_unstale":"2023-11-26T07:17:34.328087+0000","last_undegraded":"2023-11-26T07:17:34.328087+0000","last_fullsized":"2023-11-26T07:17:34.328087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:11.113481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326896+0000","last_change":"2023-11-26T07:17:33.147341+0000","last_active":"2023-11-26T07:17:34.326896+0000","last_peered":"2023-11-26T07:17:34.326896+0000","last_clean":"2023-11-26T07:17:34.326896+0000","last_became_active":"2023-11-26T07:17:33.147066+0000","last_became_peered":"2023-11-26T07:17:33.147066+0000","last_unstale":"2023-11-26T07:17:34.326896+0000","last_undegraded":"2023-11-26T07:17:34.326896+0000","last_fullsized":"2023-11-26T07:17:34.326896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:59:35.145697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326372+0000","last_change":"2023-11-26T07:17:33.155667+0000","last_active":"2023-11-26T07:17:34.326372+0000","last_peered":"2023-11-26T07:17:34.326372+0000","last_clean":"2023-11-26T07:17:34.326372+0000","last_became_active":"2023-11-26T07:17:33.155374+0000","last_became_peered":"2023-11-26T07:17:33.155374+0000","last_unstale":"2023-11-26T07:17:34.326372+0000","last_undegraded":"2023-11-26T07:17:34.326372+0000","last_fullsized":"2023-11-26T07:17:34.326372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327032+0000","last_change":"2023-11-26T07:17:33.149613+0000","last_active":"2023-11-26T07:17:34.327032+0000","last_peered":"2023-11-26T07:17:34.327032+0000","last_clean":"2023-11-26T07:17:34.327032+0000","last_became_active":"2023-11-26T07:17:33.143662+0000","last_became_peered":"2023-11-26T07:17:33.143662+0000","last_unstale":"2023-11-26T07:17:34.327032+0000","last_undegraded":"2023-11-26T07:17:34.327032+0000","last_fullsized":"2023-11-26T07:17:34.327032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326468+0000","last_change":"2023-11-26T07:17:33.157519+0000","last_active":"2023-11-26T07:17:34.326468+0000","last_peered":"2023-11-26T07:17:34.326468+0000","last_clean":"2023-11-26T07:17:34.326468+0000","last_became_active":"2023-11-26T07:17:33.157220+0000","last_became_peered":"2023-11-26T07:17:33.157220+0000","last_unstale":"2023-11-26T07:17:34.326468+0000","last_undegraded":"2023-11-26T07:17:34.326468+0000","last_fullsized":"2023-11-26T07:17:34.326468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:15:29.617633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130474+0000","last_change":"2023-11-26T07:17:33.136788+0000","last_active":"2023-11-26T07:17:34.130474+0000","last_peered":"2023-11-26T07:17:34.130474+0000","last_clean":"2023-11-26T07:17:34.130474+0000","last_became_active":"2023-11-26T07:17:33.136185+0000","last_became_peered":"2023-11-26T07:17:33.136185+0000","last_unstale":"2023-11-26T07:17:34.130474+0000","last_undegraded":"2023-11-26T07:17:34.130474+0000","last_fullsized":"2023-11-26T07:17:34.130474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:11.113481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326962+0000","last_change":"2023-11-26T07:17:33.154122+0000","last_active":"2023-11-26T07:17:34.326962+0000","last_peered":"2023-11-26T07:17:34.326962+0000","last_clean":"2023-11-26T07:17:34.326962+0000","last_became_active":"2023-11-26T07:17:33.153841+0000","last_became_peered":"2023-11-26T07:17:33.153841+0000","last_unstale":"2023-11-26T07:17:34.326962+0000","last_undegraded":"2023-11-26T07:17:34.326962+0000","last_fullsized":"2023-11-26T07:17:34.326962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:05.773940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128871+0000","last_change":"2023-11-26T07:17:33.134966+0000","last_active":"2023-11-26T07:17:34.128871+0000","last_peered":"2023-11-26T07:17:34.128871+0000","last_clean":"2023-11-26T07:17:34.128871+0000","last_became_active":"2023-11-26T07:17:33.134672+0000","last_became_peered":"2023-11-26T07:17:33.134672+0000","last_unstale":"2023-11-26T07:17:34.128871+0000","last_undegraded":"2023-11-26T07:17:34.128871+0000","last_fullsized":"2023-11-26T07:17:34.128871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128543+0000","last_change":"2023-11-26T07:17:33.134943+0000","last_active":"2023-11-26T07:17:34.128543+0000","last_peered":"2023-11-26T07:17:34.128543+0000","last_clean":"2023-11-26T07:17:34.128543+0000","last_became_active":"2023-11-26T07:17:33.134570+0000","last_became_peered":"2023-11-26T07:17:33.134570+0000","last_unstale":"2023-11-26T07:17:34.128543+0000","last_undegraded":"2023-11-26T07:17:34.128543+0000","last_fullsized":"2023-11-26T07:17:34.128543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128407+0000","last_change":"2023-11-26T07:17:33.136068+0000","last_active":"2023-11-26T07:17:34.128407+0000","last_peered":"2023-11-26T07:17:34.128407+0000","last_clean":"2023-11-26T07:17:34.128407+0000","last_became_active":"2023-11-26T07:17:33.135783+0000","last_became_peered":"2023-11-26T07:17:33.135783+0000","last_unstale":"2023-11-26T07:17:34.128407+0000","last_undegraded":"2023-11-26T07:17:34.128407+0000","last_fullsized":"2023-11-26T07:17:34.128407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:30.841394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130800+0000","last_change":"2023-11-26T07:17:33.145916+0000","last_active":"2023-11-26T07:17:34.130800+0000","last_peered":"2023-11-26T07:17:34.130800+0000","last_clean":"2023-11-26T07:17:34.130800+0000","last_became_active":"2023-11-26T07:17:33.141115+0000","last_became_peered":"2023-11-26T07:17:33.141115+0000","last_unstale":"2023-11-26T07:17:34.130800+0000","last_undegraded":"2023-11-26T07:17:34.130800+0000","last_fullsized":"2023-11-26T07:17:34.130800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327928+0000","last_change":"2023-11-26T07:17:33.153356+0000","last_active":"2023-11-26T07:17:34.327928+0000","last_peered":"2023-11-26T07:17:34.327928+0000","last_clean":"2023-11-26T07:17:34.327928+0000","last_became_active":"2023-11-26T07:17:33.153016+0000","last_became_peered":"2023-11-26T07:17:33.153016+0000","last_unstale":"2023-11-26T07:17:34.327928+0000","last_undegraded":"2023-11-26T07:17:34.327928+0000","last_fullsized":"2023-11-26T07:17:34.327928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:27.264806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326905+0000","last_change":"2023-11-26T07:17:33.138021+0000","last_active":"2023-11-26T07:17:34.326905+0000","last_peered":"2023-11-26T07:17:34.326905+0000","last_clean":"2023-11-26T07:17:34.326905+0000","last_became_active":"2023-11-26T07:17:33.137694+0000","last_became_peered":"2023-11-26T07:17:33.137694+0000","last_unstale":"2023-11-26T07:17:34.326905+0000","last_undegraded":"2023-11-26T07:17:34.326905+0000","last_fullsized":"2023-11-26T07:17:34.326905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.325959+0000","last_change":"2023-11-26T07:17:33.125499+0000","last_active":"2023-11-26T07:17:34.325959+0000","last_peered":"2023-11-26T07:17:34.325959+0000","last_clean":"2023-11-26T07:17:34.325959+0000","last_became_active":"2023-11-26T07:17:33.125132+0000","last_became_peered":"2023-11-26T07:17:33.125132+0000","last_unstale":"2023-11-26T07:17:34.325959+0000","last_undegraded":"2023-11-26T07:17:34.325959+0000","last_fullsized":"2023-11-26T07:17:34.325959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326610+0000","last_change":"2023-11-26T07:17:33.147715+0000","last_active":"2023-11-26T07:17:34.326610+0000","last_peered":"2023-11-26T07:17:34.326610+0000","last_clean":"2023-11-26T07:17:34.326610+0000","last_became_active":"2023-11-26T07:17:33.146632+0000","last_became_peered":"2023-11-26T07:17:33.146632+0000","last_unstale":"2023-11-26T07:17:34.326610+0000","last_undegraded":"2023-11-26T07:17:34.326610+0000","last_fullsized":"2023-11-26T07:17:34.326610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:54.079736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326760+0000","last_change":"2023-11-26T07:17:33.141795+0000","last_active":"2023-11-26T07:17:34.326760+0000","last_peered":"2023-11-26T07:17:34.326760+0000","last_clean":"2023-11-26T07:17:34.326760+0000","last_became_active":"2023-11-26T07:17:33.141529+0000","last_became_peered":"2023-11-26T07:17:33.141529+0000","last_unstale":"2023-11-26T07:17:34.326760+0000","last_undegraded":"2023-11-26T07:17:34.326760+0000","last_fullsized":"2023-11-26T07:17:34.326760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:35.443597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327208+0000","last_change":"2023-11-26T07:17:33.156644+0000","last_active":"2023-11-26T07:17:34.327208+0000","last_peered":"2023-11-26T07:17:34.327208+0000","last_clean":"2023-11-26T07:17:34.327208+0000","last_became_active":"2023-11-26T07:17:33.156430+0000","last_became_peered":"2023-11-26T07:17:33.156430+0000","last_unstale":"2023-11-26T07:17:34.327208+0000","last_undegraded":"2023-11-26T07:17:34.327208+0000","last_fullsized":"2023-11-26T07:17:34.327208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:54.433587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127997+0000","last_change":"2023-11-26T07:17:33.133505+0000","last_active":"2023-11-26T07:17:34.127997+0000","last_peered":"2023-11-26T07:17:34.127997+0000","last_clean":"2023-11-26T07:17:34.127997+0000","last_became_active":"2023-11-26T07:17:33.133134+0000","last_became_peered":"2023-11-26T07:17:33.133134+0000","last_unstale":"2023-11-26T07:17:34.127997+0000","last_undegraded":"2023-11-26T07:17:34.127997+0000","last_fullsized":"2023-11-26T07:17:34.127997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328056+0000","last_change":"2023-11-26T07:17:33.144471+0000","last_active":"2023-11-26T07:17:34.328056+0000","last_peered":"2023-11-26T07:17:34.328056+0000","last_clean":"2023-11-26T07:17:34.328056+0000","last_became_active":"2023-11-26T07:17:33.144183+0000","last_became_peered":"2023-11-26T07:17:33.144183+0000","last_unstale":"2023-11-26T07:17:34.328056+0000","last_undegraded":"2023-11-26T07:17:34.328056+0000","last_fullsized":"2023-11-26T07:17:34.328056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:52.817990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327032+0000","last_change":"2023-11-26T07:17:33.149400+0000","last_active":"2023-11-26T07:17:34.327032+0000","last_peered":"2023-11-26T07:17:34.327032+0000","last_clean":"2023-11-26T07:17:34.327032+0000","last_became_active":"2023-11-26T07:17:33.144991+0000","last_became_peered":"2023-11-26T07:17:33.144991+0000","last_unstale":"2023-11-26T07:17:34.327032+0000","last_undegraded":"2023-11-26T07:17:34.327032+0000","last_fullsized":"2023-11-26T07:17:34.327032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:15:29.617633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130588+0000","last_change":"2023-11-26T07:17:33.125477+0000","last_active":"2023-11-26T07:17:34.130588+0000","last_peered":"2023-11-26T07:17:34.130588+0000","last_clean":"2023-11-26T07:17:34.130588+0000","last_became_active":"2023-11-26T07:17:33.125109+0000","last_became_peered":"2023-11-26T07:17:33.125109+0000","last_unstale":"2023-11-26T07:17:34.130588+0000","last_undegraded":"2023-11-26T07:17:34.130588+0000","last_fullsized":"2023-11-26T07:17:34.130588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326026+0000","last_change":"2023-11-26T07:17:33.138762+0000","last_active":"2023-11-26T07:17:34.326026+0000","last_peered":"2023-11-26T07:17:34.326026+0000","last_clean":"2023-11-26T07:17:34.326026+0000","last_became_active":"2023-11-26T07:17:33.138466+0000","last_became_peered":"2023-11-26T07:17:33.138466+0000","last_unstale":"2023-11-26T07:17:34.326026+0000","last_undegraded":"2023-11-26T07:17:34.326026+0000","last_fullsized":"2023-11-26T07:17:34.326026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130957+0000","last_change":"2023-11-26T07:17:33.128236+0000","last_active":"2023-11-26T07:17:34.130957+0000","last_peered":"2023-11-26T07:17:34.130957+0000","last_clean":"2023-11-26T07:17:34.130957+0000","last_became_active":"2023-11-26T07:17:33.127900+0000","last_became_peered":"2023-11-26T07:17:33.127900+0000","last_unstale":"2023-11-26T07:17:34.130957+0000","last_undegraded":"2023-11-26T07:17:34.130957+0000","last_fullsized":"2023-11-26T07:17:34.130957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128993+0000","last_change":"2023-11-26T07:17:33.144760+0000","last_active":"2023-11-26T07:17:34.128993+0000","last_peered":"2023-11-26T07:17:34.128993+0000","last_clean":"2023-11-26T07:17:34.128993+0000","last_became_active":"2023-11-26T07:17:33.144272+0000","last_became_peered":"2023-11-26T07:17:33.144272+0000","last_unstale":"2023-11-26T07:17:34.128993+0000","last_undegraded":"2023-11-26T07:17:34.128993+0000","last_fullsized":"2023-11-26T07:17:34.128993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327140+0000","last_change":"2023-11-26T07:17:33.151793+0000","last_active":"2023-11-26T07:17:34.327140+0000","last_peered":"2023-11-26T07:17:34.327140+0000","last_clean":"2023-11-26T07:17:34.327140+0000","last_became_active":"2023-11-26T07:17:33.151517+0000","last_became_peered":"2023-11-26T07:17:33.151517+0000","last_unstale":"2023-11-26T07:17:34.327140+0000","last_undegraded":"2023-11-26T07:17:34.327140+0000","last_fullsized":"2023-11-26T07:17:34.327140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131015+0000","last_change":"2023-11-26T07:17:33.128368+0000","last_active":"2023-11-26T07:17:34.131015+0000","last_peered":"2023-11-26T07:17:34.131015+0000","last_clean":"2023-11-26T07:17:34.131015+0000","last_became_active":"2023-11-26T07:17:33.128068+0000","last_became_peered":"2023-11-26T07:17:33.128068+0000","last_unstale":"2023-11-26T07:17:34.131015+0000","last_undegraded":"2023-11-26T07:17:34.131015+0000","last_fullsized":"2023-11-26T07:17:34.131015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:54.079736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328159+0000","last_change":"2023-11-26T07:17:33.124145+0000","last_active":"2023-11-26T07:17:34.328159+0000","last_peered":"2023-11-26T07:17:34.328159+0000","last_clean":"2023-11-26T07:17:34.328159+0000","last_became_active":"2023-11-26T07:17:33.123870+0000","last_became_peered":"2023-11-26T07:17:33.123870+0000","last_unstale":"2023-11-26T07:17:34.328159+0000","last_undegraded":"2023-11-26T07:17:34.328159+0000","last_fullsized":"2023-11-26T07:17:34.328159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327220+0000","last_change":"2023-11-26T07:17:33.138523+0000","last_active":"2023-11-26T07:17:34.327220+0000","last_peered":"2023-11-26T07:17:34.327220+0000","last_clean":"2023-11-26T07:17:34.327220+0000","last_became_active":"2023-11-26T07:17:33.138213+0000","last_became_peered":"2023-11-26T07:17:33.138213+0000","last_unstale":"2023-11-26T07:17:34.327220+0000","last_undegraded":"2023-11-26T07:17:34.327220+0000","last_fullsized":"2023-11-26T07:17:34.327220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:12.772210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326933+0000","last_change":"2023-11-26T07:17:33.128197+0000","last_active":"2023-11-26T07:17:34.326933+0000","last_peered":"2023-11-26T07:17:34.326933+0000","last_clean":"2023-11-26T07:17:34.326933+0000","last_became_active":"2023-11-26T07:17:33.127911+0000","last_became_peered":"2023-11-26T07:17:33.127911+0000","last_unstale":"2023-11-26T07:17:34.326933+0000","last_undegraded":"2023-11-26T07:17:34.326933+0000","last_fullsized":"2023-11-26T07:17:34.326933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:17.558267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127456+0000","last_change":"2023-11-26T07:17:33.140214+0000","last_active":"2023-11-26T07:17:34.127456+0000","last_peered":"2023-11-26T07:17:34.127456+0000","last_clean":"2023-11-26T07:17:34.127456+0000","last_became_active":"2023-11-26T07:17:33.139927+0000","last_became_peered":"2023-11-26T07:17:33.139927+0000","last_unstale":"2023-11-26T07:17:34.127456+0000","last_undegraded":"2023-11-26T07:17:34.127456+0000","last_fullsized":"2023-11-26T07:17:34.127456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128003+0000","last_change":"2023-11-26T07:17:33.151273+0000","last_active":"2023-11-26T07:17:34.128003+0000","last_peered":"2023-11-26T07:17:34.128003+0000","last_clean":"2023-11-26T07:17:34.128003+0000","last_became_active":"2023-11-26T07:17:33.150182+0000","last_became_peered":"2023-11-26T07:17:33.150182+0000","last_unstale":"2023-11-26T07:17:34.128003+0000","last_undegraded":"2023-11-26T07:17:34.128003+0000","last_fullsized":"2023-11-26T07:17:34.128003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:02.280784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131100+0000","last_change":"2023-11-26T07:17:32.135703+0000","last_active":"2023-11-26T07:17:34.131100+0000","last_peered":"2023-11-26T07:17:34.131100+0000","last_clean":"2023-11-26T07:17:34.131100+0000","last_became_active":"2023-11-26T07:17:32.135392+0000","last_became_peered":"2023-11-26T07:17:32.135392+0000","last_unstale":"2023-11-26T07:17:34.131100+0000","last_undegraded":"2023-11-26T07:17:34.131100+0000","last_fullsized":"2023-11-26T07:17:34.131100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328230+0000","last_change":"2023-11-26T07:17:33.123539+0000","last_active":"2023-11-26T07:17:34.328230+0000","last_peered":"2023-11-26T07:17:34.328230+0000","last_clean":"2023-11-26T07:17:34.328230+0000","last_became_active":"2023-11-26T07:17:33.123240+0000","last_became_peered":"2023-11-26T07:17:33.123240+0000","last_unstale":"2023-11-26T07:17:34.328230+0000","last_undegraded":"2023-11-26T07:17:34.328230+0000","last_fullsized":"2023-11-26T07:17:34.328230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127853+0000","last_change":"2023-11-26T07:17:32.124304+0000","last_active":"2023-11-26T07:17:34.127853+0000","last_peered":"2023-11-26T07:17:34.127853+0000","last_clean":"2023-11-26T07:17:34.127853+0000","last_became_active":"2023-11-26T07:17:32.122374+0000","last_became_peered":"2023-11-26T07:17:32.122374+0000","last_unstale":"2023-11-26T07:17:34.127853+0000","last_undegraded":"2023-11-26T07:17:34.127853+0000","last_fullsized":"2023-11-26T07:17:34.127853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:53:41.643510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130553+0000","last_change":"2023-11-26T07:17:32.132756+0000","last_active":"2023-11-26T07:17:34.130553+0000","last_peered":"2023-11-26T07:17:34.130553+0000","last_clean":"2023-11-26T07:17:34.130553+0000","last_became_active":"2023-11-26T07:17:32.132329+0000","last_became_peered":"2023-11-26T07:17:32.132329+0000","last_unstale":"2023-11-26T07:17:34.130553+0000","last_undegraded":"2023-11-26T07:17:34.130553+0000","last_fullsized":"2023-11-26T07:17:34.130553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130543+0000","last_change":"2023-11-26T07:17:33.124808+0000","last_active":"2023-11-26T07:17:34.130543+0000","last_peered":"2023-11-26T07:17:34.130543+0000","last_clean":"2023-11-26T07:17:34.130543+0000","last_became_active":"2023-11-26T07:17:33.124508+0000","last_became_peered":"2023-11-26T07:17:33.124508+0000","last_unstale":"2023-11-26T07:17:34.130543+0000","last_undegraded":"2023-11-26T07:17:34.130543+0000","last_fullsized":"2023-11-26T07:17:34.130543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326062+0000","last_change":"2023-11-26T07:17:32.136207+0000","last_active":"2023-11-26T07:17:34.326062+0000","last_peered":"2023-11-26T07:17:34.326062+0000","last_clean":"2023-11-26T07:17:34.326062+0000","last_became_active":"2023-11-26T07:17:32.135850+0000","last_became_peered":"2023-11-26T07:17:32.135850+0000","last_unstale":"2023-11-26T07:17:34.326062+0000","last_undegraded":"2023-11-26T07:17:34.326062+0000","last_fullsized":"2023-11-26T07:17:34.326062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:48.178664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326761+0000","last_change":"2023-11-26T07:17:33.153245+0000","last_active":"2023-11-26T07:17:34.326761+0000","last_peered":"2023-11-26T07:17:34.326761+0000","last_clean":"2023-11-26T07:17:34.326761+0000","last_became_active":"2023-11-26T07:17:33.151566+0000","last_became_peered":"2023-11-26T07:17:33.151566+0000","last_unstale":"2023-11-26T07:17:34.326761+0000","last_undegraded":"2023-11-26T07:17:34.326761+0000","last_fullsized":"2023-11-26T07:17:34.326761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.129164+0000","last_change":"2023-11-26T07:17:32.126580+0000","last_active":"2023-11-26T07:17:34.129164+0000","last_peered":"2023-11-26T07:17:34.129164+0000","last_clean":"2023-11-26T07:17:34.129164+0000","last_became_active":"2023-11-26T07:17:32.125693+0000","last_became_peered":"2023-11-26T07:17:32.125693+0000","last_unstale":"2023-11-26T07:17:34.129164+0000","last_undegraded":"2023-11-26T07:17:34.129164+0000","last_fullsized":"2023-11-26T07:17:34.129164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327410+0000","last_change":"2023-11-26T07:17:33.145310+0000","last_active":"2023-11-26T07:17:34.327410+0000","last_peered":"2023-11-26T07:17:34.327410+0000","last_clean":"2023-11-26T07:17:34.327410+0000","last_became_active":"2023-11-26T07:17:33.145010+0000","last_became_peered":"2023-11-26T07:17:33.145010+0000","last_unstale":"2023-11-26T07:17:34.327410+0000","last_undegraded":"2023-11-26T07:17:34.327410+0000","last_fullsized":"2023-11-26T07:17:34.327410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:23:10.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327442+0000","last_change":"2023-11-26T07:17:32.130729+0000","last_active":"2023-11-26T07:17:34.327442+0000","last_peered":"2023-11-26T07:17:34.327442+0000","last_clean":"2023-11-26T07:17:34.327442+0000","last_became_active":"2023-11-26T07:17:32.130438+0000","last_became_peered":"2023-11-26T07:17:32.130438+0000","last_unstale":"2023-11-26T07:17:34.327442+0000","last_undegraded":"2023-11-26T07:17:34.327442+0000","last_fullsized":"2023-11-26T07:17:34.327442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:24.546723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328264+0000","last_change":"2023-11-26T07:17:33.136875+0000","last_active":"2023-11-26T07:17:34.328264+0000","last_peered":"2023-11-26T07:17:34.328264+0000","last_clean":"2023-11-26T07:17:34.328264+0000","last_became_active":"2023-11-26T07:17:33.136544+0000","last_became_peered":"2023-11-26T07:17:33.136544+0000","last_unstale":"2023-11-26T07:17:34.328264+0000","last_undegraded":"2023-11-26T07:17:34.328264+0000","last_fullsized":"2023-11-26T07:17:34.328264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327150+0000","last_change":"2023-11-26T07:17:32.127762+0000","last_active":"2023-11-26T07:17:34.327150+0000","last_peered":"2023-11-26T07:17:34.327150+0000","last_clean":"2023-11-26T07:17:34.327150+0000","last_became_active":"2023-11-26T07:17:32.127380+0000","last_became_peered":"2023-11-26T07:17:32.127380+0000","last_unstale":"2023-11-26T07:17:34.327150+0000","last_undegraded":"2023-11-26T07:17:34.327150+0000","last_fullsized":"2023-11-26T07:17:34.327150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128525+0000","last_change":"2023-11-26T07:17:33.147339+0000","last_active":"2023-11-26T07:17:34.128525+0000","last_peered":"2023-11-26T07:17:34.128525+0000","last_clean":"2023-11-26T07:17:34.128525+0000","last_became_active":"2023-11-26T07:17:33.147047+0000","last_became_peered":"2023-11-26T07:17:33.147047+0000","last_unstale":"2023-11-26T07:17:34.128525+0000","last_undegraded":"2023-11-26T07:17:34.128525+0000","last_fullsized":"2023-11-26T07:17:34.128525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:35.443597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128083+0000","last_change":"2023-11-26T07:17:32.133337+0000","last_active":"2023-11-26T07:17:34.128083+0000","last_peered":"2023-11-26T07:17:34.128083+0000","last_clean":"2023-11-26T07:17:34.128083+0000","last_became_active":"2023-11-26T07:17:32.132169+0000","last_became_peered":"2023-11-26T07:17:32.132169+0000","last_unstale":"2023-11-26T07:17:34.128083+0000","last_undegraded":"2023-11-26T07:17:34.128083+0000","last_fullsized":"2023-11-26T07:17:34.128083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:21.575382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326848+0000","last_change":"2023-11-26T07:17:33.150427+0000","last_active":"2023-11-26T07:17:34.326848+0000","last_peered":"2023-11-26T07:17:34.326848+0000","last_clean":"2023-11-26T07:17:34.326848+0000","last_became_active":"2023-11-26T07:17:33.150029+0000","last_became_peered":"2023-11-26T07:17:33.150029+0000","last_unstale":"2023-11-26T07:17:34.326848+0000","last_undegraded":"2023-11-26T07:17:34.326848+0000","last_fullsized":"2023-11-26T07:17:34.326848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:23:10.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328227+0000","last_change":"2023-11-26T07:17:32.133555+0000","last_active":"2023-11-26T07:17:34.328227+0000","last_peered":"2023-11-26T07:17:34.328227+0000","last_clean":"2023-11-26T07:17:34.328227+0000","last_became_active":"2023-11-26T07:17:32.132862+0000","last_became_peered":"2023-11-26T07:17:32.132862+0000","last_unstale":"2023-11-26T07:17:34.328227+0000","last_undegraded":"2023-11-26T07:17:34.328227+0000","last_fullsized":"2023-11-26T07:17:34.328227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:19.749525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328005+0000","last_change":"2023-11-26T07:17:33.144778+0000","last_active":"2023-11-26T07:17:34.328005+0000","last_peered":"2023-11-26T07:17:34.328005+0000","last_clean":"2023-11-26T07:17:34.328005+0000","last_became_active":"2023-11-26T07:17:33.144469+0000","last_became_peered":"2023-11-26T07:17:33.144469+0000","last_unstale":"2023-11-26T07:17:34.328005+0000","last_undegraded":"2023-11-26T07:17:34.328005+0000","last_fullsized":"2023-11-26T07:17:34.328005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:10.505099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327149+0000","last_change":"2023-11-26T07:17:32.140716+0000","last_active":"2023-11-26T07:17:34.327149+0000","last_peered":"2023-11-26T07:17:34.327149+0000","last_clean":"2023-11-26T07:17:34.327149+0000","last_became_active":"2023-11-26T07:17:32.139344+0000","last_became_peered":"2023-11-26T07:17:32.139344+0000","last_unstale":"2023-11-26T07:17:34.327149+0000","last_undegraded":"2023-11-26T07:17:34.327149+0000","last_fullsized":"2023-11-26T07:17:34.327149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326526+0000","last_change":"2023-11-26T07:17:33.152939+0000","last_active":"2023-11-26T07:17:34.326526+0000","last_peered":"2023-11-26T07:17:34.326526+0000","last_clean":"2023-11-26T07:17:34.326526+0000","last_became_active":"2023-11-26T07:17:33.152626+0000","last_became_peered":"2023-11-26T07:17:33.152626+0000","last_unstale":"2023-11-26T07:17:34.326526+0000","last_undegraded":"2023-11-26T07:17:34.326526+0000","last_fullsized":"2023-11-26T07:17:34.326526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:13.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127755+0000","last_change":"2023-11-26T07:17:32.127951+0000","last_active":"2023-11-26T07:17:34.127755+0000","last_peered":"2023-11-26T07:17:34.127755+0000","last_clean":"2023-11-26T07:17:34.127755+0000","last_became_active":"2023-11-26T07:17:32.127195+0000","last_became_peered":"2023-11-26T07:17:32.127195+0000","last_unstale":"2023-11-26T07:17:34.127755+0000","last_undegraded":"2023-11-26T07:17:34.127755+0000","last_fullsized":"2023-11-26T07:17:34.127755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.551435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127675+0000","last_change":"2023-11-26T07:17:33.132518+0000","last_active":"2023-11-26T07:17:34.127675+0000","last_peered":"2023-11-26T07:17:34.127675+0000","last_clean":"2023-11-26T07:17:34.127675+0000","last_became_active":"2023-11-26T07:17:33.131905+0000","last_became_peered":"2023-11-26T07:17:33.131905+0000","last_unstale":"2023-11-26T07:17:34.127675+0000","last_undegraded":"2023-11-26T07:17:34.127675+0000","last_fullsized":"2023-11-26T07:17:34.127675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326584+0000","last_change":"2023-11-26T07:17:32.129880+0000","last_active":"2023-11-26T07:17:34.326584+0000","last_peered":"2023-11-26T07:17:34.326584+0000","last_clean":"2023-11-26T07:17:34.326584+0000","last_became_active":"2023-11-26T07:17:32.129575+0000","last_became_peered":"2023-11-26T07:17:32.129575+0000","last_unstale":"2023-11-26T07:17:34.326584+0000","last_undegraded":"2023-11-26T07:17:34.326584+0000","last_fullsized":"2023-11-26T07:17:34.326584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:27.401076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326605+0000","last_change":"2023-11-26T07:17:33.152780+0000","last_active":"2023-11-26T07:17:34.326605+0000","last_peered":"2023-11-26T07:17:34.326605+0000","last_clean":"2023-11-26T07:17:34.326605+0000","last_became_active":"2023-11-26T07:17:33.152468+0000","last_became_peered":"2023-11-26T07:17:33.152468+0000","last_unstale":"2023-11-26T07:17:34.326605+0000","last_undegraded":"2023-11-26T07:17:34.326605+0000","last_fullsized":"2023-11-26T07:17:34.326605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:56:25.969462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128485+0000","last_change":"2023-11-26T07:17:32.132211+0000","last_active":"2023-11-26T07:17:34.128485+0000","last_peered":"2023-11-26T07:17:34.128485+0000","last_clean":"2023-11-26T07:17:34.128485+0000","last_became_active":"2023-11-26T07:17:32.131233+0000","last_became_peered":"2023-11-26T07:17:32.131233+0000","last_unstale":"2023-11-26T07:17:34.128485+0000","last_undegraded":"2023-11-26T07:17:34.128485+0000","last_fullsized":"2023-11-26T07:17:34.128485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:21.662285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326787+0000","last_change":"2023-11-26T07:17:33.154639+0000","last_active":"2023-11-26T07:17:34.326787+0000","last_peered":"2023-11-26T07:17:34.326787+0000","last_clean":"2023-11-26T07:17:34.326787+0000","last_became_active":"2023-11-26T07:17:33.153245+0000","last_became_peered":"2023-11-26T07:17:33.153245+0000","last_unstale":"2023-11-26T07:17:34.326787+0000","last_undegraded":"2023-11-26T07:17:34.326787+0000","last_fullsized":"2023-11-26T07:17:34.326787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:12.772210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326854+0000","last_change":"2023-11-26T07:17:32.138855+0000","last_active":"2023-11-26T07:17:34.326854+0000","last_peered":"2023-11-26T07:17:34.326854+0000","last_clean":"2023-11-26T07:17:34.326854+0000","last_became_active":"2023-11-26T07:17:32.138566+0000","last_became_peered":"2023-11-26T07:17:32.138566+0000","last_unstale":"2023-11-26T07:17:34.326854+0000","last_undegraded":"2023-11-26T07:17:34.326854+0000","last_fullsized":"2023-11-26T07:17:34.326854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:29.834563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327040+0000","last_change":"2023-11-26T07:17:33.152760+0000","last_active":"2023-11-26T07:17:34.327040+0000","last_peered":"2023-11-26T07:17:34.327040+0000","last_clean":"2023-11-26T07:17:34.327040+0000","last_became_active":"2023-11-26T07:17:33.152455+0000","last_became_peered":"2023-11-26T07:17:33.152455+0000","last_unstale":"2023-11-26T07:17:34.327040+0000","last_undegraded":"2023-11-26T07:17:34.327040+0000","last_fullsized":"2023-11-26T07:17:34.327040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:40.786432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327244+0000","last_change":"2023-11-26T07:17:32.132634+0000","last_active":"2023-11-26T07:17:34.327244+0000","last_peered":"2023-11-26T07:17:34.327244+0000","last_clean":"2023-11-26T07:17:34.327244+0000","last_became_active":"2023-11-26T07:17:32.132192+0000","last_became_peered":"2023-11-26T07:17:32.132192+0000","last_unstale":"2023-11-26T07:17:34.327244+0000","last_undegraded":"2023-11-26T07:17:34.327244+0000","last_fullsized":"2023-11-26T07:17:34.327244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:54:55.533333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327181+0000","last_change":"2023-11-26T07:17:33.147935+0000","last_active":"2023-11-26T07:17:34.327181+0000","last_peered":"2023-11-26T07:17:34.327181+0000","last_clean":"2023-11-26T07:17:34.327181+0000","last_became_active":"2023-11-26T07:17:33.147595+0000","last_became_peered":"2023-11-26T07:17:33.147595+0000","last_unstale":"2023-11-26T07:17:34.327181+0000","last_undegraded":"2023-11-26T07:17:34.327181+0000","last_fullsized":"2023-11-26T07:17:34.327181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:56:25.969462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326702+0000","last_change":"2023-11-26T07:17:32.128101+0000","last_active":"2023-11-26T07:17:34.326702+0000","last_peered":"2023-11-26T07:17:34.326702+0000","last_clean":"2023-11-26T07:17:34.326702+0000","last_became_active":"2023-11-26T07:17:32.127821+0000","last_became_peered":"2023-11-26T07:17:32.127821+0000","last_unstale":"2023-11-26T07:17:34.326702+0000","last_undegraded":"2023-11-26T07:17:34.326702+0000","last_fullsized":"2023-11-26T07:17:34.326702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:30:24.091629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326978+0000","last_change":"2023-11-26T07:17:33.153014+0000","last_active":"2023-11-26T07:17:34.326978+0000","last_peered":"2023-11-26T07:17:34.326978+0000","last_clean":"2023-11-26T07:17:34.326978+0000","last_became_active":"2023-11-26T07:17:33.152675+0000","last_became_peered":"2023-11-26T07:17:33.152675+0000","last_unstale":"2023-11-26T07:17:34.326978+0000","last_undegraded":"2023-11-26T07:17:34.326978+0000","last_fullsized":"2023-11-26T07:17:34.326978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:06.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-26T07:17:33.140703+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:17:33.140703+0000","last_peered":"2023-11-26T07:17:33.140703+0000","last_clean":"2023-11-26T07:17:33.140703+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:17:33.140703+0000","last_undegraded":"2023-11-26T07:17:33.140703+0000","last_fullsized":"2023-11-26T07:17:33.140703+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.325979+0000","last_change":"2023-11-26T07:17:32.127453+0000","last_active":"2023-11-26T07:17:34.325979+0000","last_peered":"2023-11-26T07:17:34.325979+0000","last_clean":"2023-11-26T07:17:34.325979+0000","last_became_active":"2023-11-26T07:17:32.127171+0000","last_became_peered":"2023-11-26T07:17:32.127171+0000","last_unstale":"2023-11-26T07:17:34.325979+0000","last_undegraded":"2023-11-26T07:17:34.325979+0000","last_fullsized":"2023-11-26T07:17:34.325979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:10.106650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130743+0000","last_change":"2023-11-26T07:17:33.132790+0000","last_active":"2023-11-26T07:17:34.130743+0000","last_peered":"2023-11-26T07:17:34.130743+0000","last_clean":"2023-11-26T07:17:34.130743+0000","last_became_active":"2023-11-26T07:17:33.132415+0000","last_became_peered":"2023-11-26T07:17:33.132415+0000","last_unstale":"2023-11-26T07:17:34.130743+0000","last_undegraded":"2023-11-26T07:17:34.130743+0000","last_fullsized":"2023-11-26T07:17:34.130743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326218+0000","last_change":"2023-11-26T07:17:32.134663+0000","last_active":"2023-11-26T07:17:34.326218+0000","last_peered":"2023-11-26T07:17:34.326218+0000","last_clean":"2023-11-26T07:17:34.326218+0000","last_became_active":"2023-11-26T07:17:32.134372+0000","last_became_peered":"2023-11-26T07:17:32.134372+0000","last_unstale":"2023-11-26T07:17:34.326218+0000","last_undegraded":"2023-11-26T07:17:34.326218+0000","last_fullsized":"2023-11-26T07:17:34.326218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:34.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327893+0000","last_change":"2023-11-26T07:17:33.131023+0000","last_active":"2023-11-26T07:17:34.327893+0000","last_peered":"2023-11-26T07:17:34.327893+0000","last_clean":"2023-11-26T07:17:34.327893+0000","last_became_active":"2023-11-26T07:17:33.130665+0000","last_became_peered":"2023-11-26T07:17:33.130665+0000","last_unstale":"2023-11-26T07:17:34.327893+0000","last_undegraded":"2023-11-26T07:17:34.327893+0000","last_fullsized":"2023-11-26T07:17:34.327893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:54:56.540164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327009+0000","last_change":"2023-11-26T07:17:32.141091+0000","last_active":"2023-11-26T07:17:34.327009+0000","last_peered":"2023-11-26T07:17:34.327009+0000","last_clean":"2023-11-26T07:17:34.327009+0000","last_became_active":"2023-11-26T07:17:32.139978+0000","last_became_peered":"2023-11-26T07:17:32.139978+0000","last_unstale":"2023-11-26T07:17:34.327009+0000","last_undegraded":"2023-11-26T07:17:34.327009+0000","last_fullsized":"2023-11-26T07:17:34.327009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:01.273952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326756+0000","last_change":"2023-11-26T07:17:33.137737+0000","last_active":"2023-11-26T07:17:34.326756+0000","last_peered":"2023-11-26T07:17:34.326756+0000","last_clean":"2023-11-26T07:17:34.326756+0000","last_became_active":"2023-11-26T07:17:33.137350+0000","last_became_peered":"2023-11-26T07:17:33.137350+0000","last_unstale":"2023-11-26T07:17:34.326756+0000","last_undegraded":"2023-11-26T07:17:34.326756+0000","last_fullsized":"2023-11-26T07:17:34.326756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327251+0000","last_change":"2023-11-26T07:17:32.134855+0000","last_active":"2023-11-26T07:17:34.327251+0000","last_peered":"2023-11-26T07:17:34.327251+0000","last_clean":"2023-11-26T07:17:34.327251+0000","last_became_active":"2023-11-26T07:17:32.134587+0000","last_became_peered":"2023-11-26T07:17:32.134587+0000","last_unstale":"2023-11-26T07:17:34.327251+0000","last_undegraded":"2023-11-26T07:17:34.327251+0000","last_fullsized":"2023-11-26T07:17:34.327251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128590+0000","last_change":"2023-11-26T07:17:33.122483+0000","last_active":"2023-11-26T07:17:34.128590+0000","last_peered":"2023-11-26T07:17:34.128590+0000","last_clean":"2023-11-26T07:17:34.128590+0000","last_became_active":"2023-11-26T07:17:33.122042+0000","last_became_peered":"2023-11-26T07:17:33.122042+0000","last_unstale":"2023-11-26T07:17:34.128590+0000","last_undegraded":"2023-11-26T07:17:34.128590+0000","last_fullsized":"2023-11-26T07:17:34.128590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:17.558267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327027+0000","last_change":"2023-11-26T07:17:32.124516+0000","last_active":"2023-11-26T07:17:34.327027+0000","last_peered":"2023-11-26T07:17:34.327027+0000","last_clean":"2023-11-26T07:17:34.327027+0000","last_became_active":"2023-11-26T07:17:32.123978+0000","last_became_peered":"2023-11-26T07:17:32.123978+0000","last_unstale":"2023-11-26T07:17:34.327027+0000","last_undegraded":"2023-11-26T07:17:34.327027+0000","last_fullsized":"2023-11-26T07:17:34.327027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:34.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326612+0000","last_change":"2023-11-26T07:17:33.139055+0000","last_active":"2023-11-26T07:17:34.326612+0000","last_peered":"2023-11-26T07:17:34.326612+0000","last_clean":"2023-11-26T07:17:34.326612+0000","last_became_active":"2023-11-26T07:17:33.138235+0000","last_became_peered":"2023-11-26T07:17:33.138235+0000","last_unstale":"2023-11-26T07:17:34.326612+0000","last_undegraded":"2023-11-26T07:17:34.326612+0000","last_fullsized":"2023-11-26T07:17:34.326612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327313+0000","last_change":"2023-11-26T07:17:32.135542+0000","last_active":"2023-11-26T07:17:34.327313+0000","last_peered":"2023-11-26T07:17:34.327313+0000","last_clean":"2023-11-26T07:17:34.327313+0000","last_became_active":"2023-11-26T07:17:32.135078+0000","last_became_peered":"2023-11-26T07:17:32.135078+0000","last_unstale":"2023-11-26T07:17:34.327313+0000","last_undegraded":"2023-11-26T07:17:34.327313+0000","last_fullsized":"2023-11-26T07:17:34.327313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:04.536431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327077+0000","last_change":"2023-11-26T07:17:33.149589+0000","last_active":"2023-11-26T07:17:34.327077+0000","last_peered":"2023-11-26T07:17:34.327077+0000","last_clean":"2023-11-26T07:17:34.327077+0000","last_became_active":"2023-11-26T07:17:33.149228+0000","last_became_peered":"2023-11-26T07:17:33.149228+0000","last_unstale":"2023-11-26T07:17:34.327077+0000","last_undegraded":"2023-11-26T07:17:34.327077+0000","last_fullsized":"2023-11-26T07:17:34.327077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:35.338911+0000","last_change":"2023-11-26T07:17:32.118350+0000","last_active":"2023-11-26T07:17:35.338911+0000","last_peered":"2023-11-26T07:17:35.338911+0000","last_clean":"2023-11-26T07:17:35.338911+0000","last_became_active":"2023-11-26T07:17:32.118040+0000","last_became_peered":"2023-11-26T07:17:32.118040+0000","last_unstale":"2023-11-26T07:17:35.338911+0000","last_undegraded":"2023-11-26T07:17:35.338911+0000","last_fullsized":"2023-11-26T07:17:35.338911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:21.575382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131017+0000","last_change":"2023-11-26T07:17:33.150796+0000","last_active":"2023-11-26T07:17:34.131017+0000","last_peered":"2023-11-26T07:17:34.131017+0000","last_clean":"2023-11-26T07:17:34.131017+0000","last_became_active":"2023-11-26T07:17:33.150543+0000","last_became_peered":"2023-11-26T07:17:33.150543+0000","last_unstale":"2023-11-26T07:17:34.131017+0000","last_undegraded":"2023-11-26T07:17:34.131017+0000","last_fullsized":"2023-11-26T07:17:34.131017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:59:35.145697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.129145+0000","last_change":"2023-11-26T07:17:32.134890+0000","last_active":"2023-11-26T07:17:34.129145+0000","last_peered":"2023-11-26T07:17:34.129145+0000","last_clean":"2023-11-26T07:17:34.129145+0000","last_became_active":"2023-11-26T07:17:32.134621+0000","last_became_peered":"2023-11-26T07:17:32.134621+0000","last_unstale":"2023-11-26T07:17:34.129145+0000","last_undegraded":"2023-11-26T07:17:34.129145+0000","last_fullsized":"2023-11-26T07:17:34.129145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:09.498267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328230+0000","last_change":"2023-11-26T07:17:33.135948+0000","last_active":"2023-11-26T07:17:34.328230+0000","last_peered":"2023-11-26T07:17:34.328230+0000","last_clean":"2023-11-26T07:17:34.328230+0000","last_became_active":"2023-11-26T07:17:33.135636+0000","last_became_peered":"2023-11-26T07:17:33.135636+0000","last_unstale":"2023-11-26T07:17:34.328230+0000","last_undegraded":"2023-11-26T07:17:34.328230+0000","last_fullsized":"2023-11-26T07:17:34.328230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326291+0000","last_change":"2023-11-26T07:17:32.134899+0000","last_active":"2023-11-26T07:17:34.326291+0000","last_peered":"2023-11-26T07:17:34.326291+0000","last_clean":"2023-11-26T07:17:34.326291+0000","last_became_active":"2023-11-26T07:17:32.134566+0000","last_became_peered":"2023-11-26T07:17:32.134566+0000","last_unstale":"2023-11-26T07:17:34.326291+0000","last_undegraded":"2023-11-26T07:17:34.326291+0000","last_fullsized":"2023-11-26T07:17:34.326291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:29.834563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127563+0000","last_change":"2023-11-26T07:17:33.123386+0000","last_active":"2023-11-26T07:17:34.127563+0000","last_peered":"2023-11-26T07:17:34.127563+0000","last_clean":"2023-11-26T07:17:34.127563+0000","last_became_active":"2023-11-26T07:17:33.120220+0000","last_became_peered":"2023-11-26T07:17:33.120220+0000","last_unstale":"2023-11-26T07:17:34.127563+0000","last_undegraded":"2023-11-26T07:17:34.127563+0000","last_fullsized":"2023-11-26T07:17:34.127563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:20.756357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326867+0000","last_change":"2023-11-26T07:17:32.138806+0000","last_active":"2023-11-26T07:17:34.326867+0000","last_peered":"2023-11-26T07:17:34.326867+0000","last_clean":"2023-11-26T07:17:34.326867+0000","last_became_active":"2023-11-26T07:17:32.138612+0000","last_became_peered":"2023-11-26T07:17:32.138612+0000","last_unstale":"2023-11-26T07:17:34.326867+0000","last_undegraded":"2023-11-26T07:17:34.326867+0000","last_fullsized":"2023-11-26T07:17:34.326867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130595+0000","last_change":"2023-11-26T07:17:33.139703+0000","last_active":"2023-11-26T07:17:34.130595+0000","last_peered":"2023-11-26T07:17:34.130595+0000","last_clean":"2023-11-26T07:17:34.130595+0000","last_became_active":"2023-11-26T07:17:33.139347+0000","last_became_peered":"2023-11-26T07:17:33.139347+0000","last_unstale":"2023-11-26T07:17:34.130595+0000","last_undegraded":"2023-11-26T07:17:34.130595+0000","last_fullsized":"2023-11-26T07:17:34.130595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327161+0000","last_change":"2023-11-26T07:17:32.141098+0000","last_active":"2023-11-26T07:17:34.327161+0000","last_peered":"2023-11-26T07:17:34.327161+0000","last_clean":"2023-11-26T07:17:34.327161+0000","last_became_active":"2023-11-26T07:17:32.137814+0000","last_became_peered":"2023-11-26T07:17:32.137814+0000","last_unstale":"2023-11-26T07:17:34.327161+0000","last_undegraded":"2023-11-26T07:17:34.327161+0000","last_fullsized":"2023-11-26T07:17:34.327161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:53:41.643510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130835+0000","last_change":"2023-11-26T07:17:33.124135+0000","last_active":"2023-11-26T07:17:34.130835+0000","last_peered":"2023-11-26T07:17:34.130835+0000","last_clean":"2023-11-26T07:17:34.130835+0000","last_became_active":"2023-11-26T07:17:33.123814+0000","last_became_peered":"2023-11-26T07:17:33.123814+0000","last_unstale":"2023-11-26T07:17:34.130835+0000","last_undegraded":"2023-11-26T07:17:34.130835+0000","last_fullsized":"2023-11-26T07:17:34.130835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326630+0000","last_change":"2023-11-26T07:17:32.140282+0000","last_active":"2023-11-26T07:17:34.326630+0000","last_peered":"2023-11-26T07:17:34.326630+0000","last_clean":"2023-11-26T07:17:34.326630+0000","last_became_active":"2023-11-26T07:17:32.139388+0000","last_became_peered":"2023-11-26T07:17:32.139388+0000","last_unstale":"2023-11-26T07:17:34.326630+0000","last_undegraded":"2023-11-26T07:17:34.326630+0000","last_fullsized":"2023-11-26T07:17:34.326630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:24.546723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127842+0000","last_change":"2023-11-26T07:17:33.140095+0000","last_active":"2023-11-26T07:17:34.127842+0000","last_peered":"2023-11-26T07:17:34.127842+0000","last_clean":"2023-11-26T07:17:34.127842+0000","last_became_active":"2023-11-26T07:17:33.139715+0000","last_became_peered":"2023-11-26T07:17:33.139715+0000","last_unstale":"2023-11-26T07:17:34.127842+0000","last_undegraded":"2023-11-26T07:17:34.127842+0000","last_fullsized":"2023-11-26T07:17:34.127842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326469+0000","last_change":"2023-11-26T07:17:33.132899+0000","last_active":"2023-11-26T07:17:34.326469+0000","last_peered":"2023-11-26T07:17:34.326469+0000","last_clean":"2023-11-26T07:17:34.326469+0000","last_became_active":"2023-11-26T07:17:33.132617+0000","last_became_peered":"2023-11-26T07:17:33.132617+0000","last_unstale":"2023-11-26T07:17:34.326469+0000","last_undegraded":"2023-11-26T07:17:34.326469+0000","last_fullsized":"2023-11-26T07:17:34.326469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326817+0000","last_change":"2023-11-26T07:17:32.138620+0000","last_active":"2023-11-26T07:17:34.326817+0000","last_peered":"2023-11-26T07:17:34.326817+0000","last_clean":"2023-11-26T07:17:34.326817+0000","last_became_active":"2023-11-26T07:17:32.138135+0000","last_became_peered":"2023-11-26T07:17:32.138135+0000","last_unstale":"2023-11-26T07:17:34.326817+0000","last_undegraded":"2023-11-26T07:17:34.326817+0000","last_fullsized":"2023-11-26T07:17:34.326817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327036+0000","last_change":"2023-11-26T07:17:33.136866+0000","last_active":"2023-11-26T07:17:34.327036+0000","last_peered":"2023-11-26T07:17:34.327036+0000","last_clean":"2023-11-26T07:17:34.327036+0000","last_became_active":"2023-11-26T07:17:33.136571+0000","last_became_peered":"2023-11-26T07:17:33.136571+0000","last_unstale":"2023-11-26T07:17:34.327036+0000","last_undegraded":"2023-11-26T07:17:34.327036+0000","last_fullsized":"2023-11-26T07:17:34.327036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:13.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327073+0000","last_change":"2023-11-26T07:17:32.135405+0000","last_active":"2023-11-26T07:17:34.327073+0000","last_peered":"2023-11-26T07:17:34.327073+0000","last_clean":"2023-11-26T07:17:34.327073+0000","last_became_active":"2023-11-26T07:17:32.134722+0000","last_became_peered":"2023-11-26T07:17:32.134722+0000","last_unstale":"2023-11-26T07:17:34.327073+0000","last_undegraded":"2023-11-26T07:17:34.327073+0000","last_fullsized":"2023-11-26T07:17:34.327073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328275+0000","last_change":"2023-11-26T07:17:33.138935+0000","last_active":"2023-11-26T07:17:34.328275+0000","last_peered":"2023-11-26T07:17:34.328275+0000","last_clean":"2023-11-26T07:17:34.328275+0000","last_became_active":"2023-11-26T07:17:33.138684+0000","last_became_peered":"2023-11-26T07:17:33.138684+0000","last_unstale":"2023-11-26T07:17:34.328275+0000","last_undegraded":"2023-11-26T07:17:34.328275+0000","last_fullsized":"2023-11-26T07:17:34.328275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131063+0000","last_change":"2023-11-26T07:17:32.130834+0000","last_active":"2023-11-26T07:17:34.131063+0000","last_peered":"2023-11-26T07:17:34.131063+0000","last_clean":"2023-11-26T07:17:34.131063+0000","last_became_active":"2023-11-26T07:17:32.130525+0000","last_became_peered":"2023-11-26T07:17:32.130525+0000","last_unstale":"2023-11-26T07:17:34.131063+0000","last_undegraded":"2023-11-26T07:17:34.131063+0000","last_fullsized":"2023-11-26T07:17:34.131063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:09.498267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131026+0000","last_change":"2023-11-26T07:17:33.146148+0000","last_active":"2023-11-26T07:17:34.131026+0000","last_peered":"2023-11-26T07:17:34.131026+0000","last_clean":"2023-11-26T07:17:34.131026+0000","last_became_active":"2023-11-26T07:17:33.145838+0000","last_became_peered":"2023-11-26T07:17:33.145838+0000","last_unstale":"2023-11-26T07:17:34.131026+0000","last_undegraded":"2023-11-26T07:17:34.131026+0000","last_fullsized":"2023-11-26T07:17:34.131026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328317+0000","last_change":"2023-11-26T07:17:32.135921+0000","last_active":"2023-11-26T07:17:34.328317+0000","last_peered":"2023-11-26T07:17:34.328317+0000","last_clean":"2023-11-26T07:17:34.328317+0000","last_became_active":"2023-11-26T07:17:32.135634+0000","last_became_peered":"2023-11-26T07:17:32.135634+0000","last_unstale":"2023-11-26T07:17:34.328317+0000","last_undegraded":"2023-11-26T07:17:34.328317+0000","last_fullsized":"2023-11-26T07:17:34.328317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.551435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.492}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1520000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.677}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:41.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:17:42.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674c10f640 1 Processor -- start 2023-11-26T07:17:42.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674c10f640 1 -- start start 2023-11-26T07:17:42.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674c10f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f674415f540 0x7f67440b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674c10f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f674406cd40 0x7f6744070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674c10f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67440b2920 con 0x7f67440b1f40 2023-11-26T07:17:42.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674c10f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67440b2c00 con 0x7f674406cd40 2023-11-26T07:17:42.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674c10f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67440b2ee0 con 0x7f674415f540 2023-11-26T07:17:42.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674a6ac640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f674406cd40 0x7f6744070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f67496aa640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f67440b1f40 0x7f674406c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46354/0 (socket says 172.21.15.31:46354) 2023-11-26T07:17:42.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674a6ac640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f674406cd40 0x7f6744070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59718/0 (socket says 172.21.15.31:59718) 2023-11-26T07:17:42.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f67496aa640 1 -- 172.21.15.31:0/861224630 learned_addr learned my addr 172.21.15.31:0/861224630 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:42.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f6749eab640 1 --2- 172.21.15.31:0/861224630 >> v2:172.21.15.97:3300/0 conn(0x7f674415f540 0x7f67440b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674a6ac640 1 -- 172.21.15.31:0/861224630 >> v2:172.21.15.97:3300/0 conn(0x7f674415f540 msgr2=0x7f67440b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:42.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.093+0000 7f674a6ac640 1 --2- 172.21.15.31:0/861224630 >> v2:172.21.15.97:3300/0 conn(0x7f674415f540 0x7f67440b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674a6ac640 1 -- 172.21.15.31:0/861224630 >> v1:172.21.15.31:6789/0 conn(0x7f67440b1f40 legacy=0x7f674406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674a6ac640 1 -- 172.21.15.31:0/861224630 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440b31c0 con 0x7f674406cd40 2023-11-26T07:17:42.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f6749eab640 1 --2- 172.21.15.31:0/861224630 >> v2:172.21.15.97:3300/0 conn(0x7f674415f540 0x7f67440b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:42.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674a6ac640 1 --2- 172.21.15.31:0/861224630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674406cd40 0x7f6744070880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f674406c230 tx=0x7f673c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1303e97b7f9c055b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f6748ea9640 1 -- 172.21.15.31:0/861224630 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0a59c0 con 0x7f674406cd40 2023-11-26T07:17:42.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f6748ea9640 1 -- 172.21.15.31:0/861224630 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c0a5b20 con 0x7f674406cd40 2023-11-26T07:17:42.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f6748ea9640 1 -- 172.21.15.31:0/861224630 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0ad440 con 0x7f674406cd40 2023-11-26T07:17:42.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674c10f640 1 -- 172.21.15.31:0/861224630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674406cd40 msgr2=0x7f6744070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674c10f640 1 --2- 172.21.15.31:0/861224630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674406cd40 0x7f6744070880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f674406c230 tx=0x7f673c01c8e0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674c10f640 1 -- 172.21.15.31:0/861224630 shutdown_connections 2023-11-26T07:17:42.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674c10f640 1 --2- 172.21.15.31:0/861224630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674406cd40 0x7f6744070880 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674c10f640 1 --2- 172.21.15.31:0/861224630 >> v2:172.21.15.97:3300/0 conn(0x7f674415f540 0x7f67440b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674c10f640 1 -- 172.21.15.31:0/861224630 >> 172.21.15.31:0/861224630 conn(0x7f67440b0c40 msgr2=0x7f67440b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674c10f640 1 -- 172.21.15.31:0/861224630 shutdown_connections 2023-11-26T07:17:42.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.097+0000 7f674c10f640 1 -- 172.21.15.31:0/861224630 wait complete. 2023-11-26T07:17:42.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674c10f640 1 Processor -- start 2023-11-26T07:17:42.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674c10f640 1 -- start start 2023-11-26T07:17:42.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674c10f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674406cd40 0x7f674415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674c10f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440b1f40 0x7f6744160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674c10f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674415f540 0x7f67441513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674c10f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674415a9a0 con 0x7f674415f540 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674c10f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744161340 con 0x7f67440b1f40 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674c10f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6744160d80 con 0x7f674406cd40 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674a6ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674415f540 0x7f67441513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674a6ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674415f540 0x7f67441513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59722/0 (socket says 172.21.15.31:59722) 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674a6ac640 1 -- 172.21.15.31:0/3649365963 learned_addr learned my addr 172.21.15.31:0/3649365963 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f6749eab640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674406cd40 0x7f674415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f67496aa640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440b1f40 0x7f6744160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674a6ac640 1 -- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674406cd40 msgr2=0x7f674415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674a6ac640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674406cd40 0x7f674415dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674a6ac640 1 -- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440b1f40 msgr2=0x7f6744160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674a6ac640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440b1f40 0x7f6744160380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674a6ac640 1 -- 172.21.15.31:0/3649365963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c009a70 con 0x7f674415f540 2023-11-26T07:17:42.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674a6ac640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674415f540 0x7f67441513e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f673c0b0040 tx=0x7f673c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f673a7fc640 1 -- 172.21.15.31:0/3649365963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0ade20 con 0x7f674415f540 2023-11-26T07:17:42.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f673a7fc640 1 -- 172.21.15.31:0/3649365963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c009450 con 0x7f674415f540 2023-11-26T07:17:42.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f673a7fc640 1 -- 172.21.15.31:0/3649365963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0b9690 con 0x7f674415f540 2023-11-26T07:17:42.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674c10f640 1 -- 172.21.15.31:0/3649365963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6744161060 con 0x7f674415f540 2023-11-26T07:17:42.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.101+0000 7f674c10f640 1 -- 172.21.15.31:0/3649365963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674415e000 con 0x7f674415f540 2023-11-26T07:17:42.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.105+0000 7f674c10f640 1 -- 172.21.15.31:0/3649365963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674415e1f0 con 0x7f674415f540 2023-11-26T07:17:42.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.105+0000 7f673a7fc640 1 -- 172.21.15.31:0/3649365963 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f673c0b97f0 con 0x7f674415f540 2023-11-26T07:17:42.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.105+0000 7f673a7fc640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6728066470 0x7f6728068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.105+0000 7f673a7fc640 1 -- 172.21.15.31:0/3649365963 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f673c11f6c0 con 0x7f674415f540 2023-11-26T07:17:42.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.105+0000 7f6749eab640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6728066470 0x7f6728068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.105+0000 7f6749eab640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6728066470 0x7f6728068930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f674000b930 tx=0x7f6740008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.109+0000 7f673a7fc640 1 -- 172.21.15.31:0/3649365963 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f674415e1f0 con 0x7f674415f540 2023-11-26T07:17:42.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.265+0000 7f674c10f640 1 -- 172.21.15.31:0/3649365963 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f674415e1f0 con 0x7f6728066470 2023-11-26T07:17:42.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f673a7fc640 1 -- 172.21.15.31:0/3649365963 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264561 (secure 0 0 0) 0x7f674415e1f0 con 0x7f6728066470 2023-11-26T07:17:42.278 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:42.280 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 -- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6728066470 msgr2=0x7f6728068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6728066470 0x7f6728068930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f674000b930 tx=0x7f6740008040 comp rx=0 tx=0).stop 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 -- 172.21.15.31:0/3649365963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674415f540 msgr2=0x7f67441513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674415f540 0x7f67441513e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f673c0b0040 tx=0x7f673c019680 comp rx=0 tx=0).stop 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 -- 172.21.15.31:0/3649365963 shutdown_connections 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6728066470 0x7f6728068930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674415f540 0x7f67441513e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67440b1f40 0x7f6744160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 --2- 172.21.15.31:0/3649365963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674406cd40 0x7f674415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 -- 172.21.15.31:0/3649365963 >> 172.21.15.31:0/3649365963 conn(0x7f67440b0c40 msgr2=0x7f67440af290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 -- 172.21.15.31:0/3649365963 shutdown_connections 2023-11-26T07:17:42.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.273+0000 7f674c10f640 1 -- 172.21.15.31:0/3649365963 wait complete. 2023-11-26T07:17:42.298 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-26T07:17:40.308219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219004,"kb_used_data":2820,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755716,"statfs":{"total":773094113280,"available":772869853184,"internally_reserved":0,"allocated":2887680,"data_stored":1751817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001755"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326879+0000","last_change":"2023-11-26T07:17:33.157517+0000","last_active":"2023-11-26T07:17:34.326879+0000","last_peered":"2023-11-26T07:17:34.326879+0000","last_clean":"2023-11-26T07:17:34.326879+0000","last_became_active":"2023-11-26T07:17:33.157236+0000","last_became_peered":"2023-11-26T07:17:33.157236+0000","last_unstale":"2023-11-26T07:17:34.326879+0000","last_undegraded":"2023-11-26T07:17:34.326879+0000","last_fullsized":"2023-11-26T07:17:34.326879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:00:50.214046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327264+0000","last_change":"2023-11-26T07:17:33.153106+0000","last_active":"2023-11-26T07:17:34.327264+0000","last_peered":"2023-11-26T07:17:34.327264+0000","last_clean":"2023-11-26T07:17:34.327264+0000","last_became_active":"2023-11-26T07:17:33.152758+0000","last_became_peered":"2023-11-26T07:17:33.152758+0000","last_unstale":"2023-11-26T07:17:34.327264+0000","last_undegraded":"2023-11-26T07:17:34.327264+0000","last_fullsized":"2023-11-26T07:17:34.327264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328087+0000","last_change":"2023-11-26T07:17:33.119394+0000","last_active":"2023-11-26T07:17:34.328087+0000","last_peered":"2023-11-26T07:17:34.328087+0000","last_clean":"2023-11-26T07:17:34.328087+0000","last_became_active":"2023-11-26T07:17:33.118798+0000","last_became_peered":"2023-11-26T07:17:33.118798+0000","last_unstale":"2023-11-26T07:17:34.328087+0000","last_undegraded":"2023-11-26T07:17:34.328087+0000","last_fullsized":"2023-11-26T07:17:34.328087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:11.113481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326896+0000","last_change":"2023-11-26T07:17:33.147341+0000","last_active":"2023-11-26T07:17:34.326896+0000","last_peered":"2023-11-26T07:17:34.326896+0000","last_clean":"2023-11-26T07:17:34.326896+0000","last_became_active":"2023-11-26T07:17:33.147066+0000","last_became_peered":"2023-11-26T07:17:33.147066+0000","last_unstale":"2023-11-26T07:17:34.326896+0000","last_undegraded":"2023-11-26T07:17:34.326896+0000","last_fullsized":"2023-11-26T07:17:34.326896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:59:35.145697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326372+0000","last_change":"2023-11-26T07:17:33.155667+0000","last_active":"2023-11-26T07:17:34.326372+0000","last_peered":"2023-11-26T07:17:34.326372+0000","last_clean":"2023-11-26T07:17:34.326372+0000","last_became_active":"2023-11-26T07:17:33.155374+0000","last_became_peered":"2023-11-26T07:17:33.155374+0000","last_unstale":"2023-11-26T07:17:34.326372+0000","last_undegraded":"2023-11-26T07:17:34.326372+0000","last_fullsized":"2023-11-26T07:17:34.326372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327032+0000","last_change":"2023-11-26T07:17:33.149613+0000","last_active":"2023-11-26T07:17:34.327032+0000","last_peered":"2023-11-26T07:17:34.327032+0000","last_clean":"2023-11-26T07:17:34.327032+0000","last_became_active":"2023-11-26T07:17:33.143662+0000","last_became_peered":"2023-11-26T07:17:33.143662+0000","last_unstale":"2023-11-26T07:17:34.327032+0000","last_undegraded":"2023-11-26T07:17:34.327032+0000","last_fullsized":"2023-11-26T07:17:34.327032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326468+0000","last_change":"2023-11-26T07:17:33.157519+0000","last_active":"2023-11-26T07:17:34.326468+0000","last_peered":"2023-11-26T07:17:34.326468+0000","last_clean":"2023-11-26T07:17:34.326468+0000","last_became_active":"2023-11-26T07:17:33.157220+0000","last_became_peered":"2023-11-26T07:17:33.157220+0000","last_unstale":"2023-11-26T07:17:34.326468+0000","last_undegraded":"2023-11-26T07:17:34.326468+0000","last_fullsized":"2023-11-26T07:17:34.326468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:15:29.617633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130474+0000","last_change":"2023-11-26T07:17:33.136788+0000","last_active":"2023-11-26T07:17:34.130474+0000","last_peered":"2023-11-26T07:17:34.130474+0000","last_clean":"2023-11-26T07:17:34.130474+0000","last_became_active":"2023-11-26T07:17:33.136185+0000","last_became_peered":"2023-11-26T07:17:33.136185+0000","last_unstale":"2023-11-26T07:17:34.130474+0000","last_undegraded":"2023-11-26T07:17:34.130474+0000","last_fullsized":"2023-11-26T07:17:34.130474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:11.113481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326962+0000","last_change":"2023-11-26T07:17:33.154122+0000","last_active":"2023-11-26T07:17:34.326962+0000","last_peered":"2023-11-26T07:17:34.326962+0000","last_clean":"2023-11-26T07:17:34.326962+0000","last_became_active":"2023-11-26T07:17:33.153841+0000","last_became_peered":"2023-11-26T07:17:33.153841+0000","last_unstale":"2023-11-26T07:17:34.326962+0000","last_undegraded":"2023-11-26T07:17:34.326962+0000","last_fullsized":"2023-11-26T07:17:34.326962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:05.773940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128871+0000","last_change":"2023-11-26T07:17:33.134966+0000","last_active":"2023-11-26T07:17:34.128871+0000","last_peered":"2023-11-26T07:17:34.128871+0000","last_clean":"2023-11-26T07:17:34.128871+0000","last_became_active":"2023-11-26T07:17:33.134672+0000","last_became_peered":"2023-11-26T07:17:33.134672+0000","last_unstale":"2023-11-26T07:17:34.128871+0000","last_undegraded":"2023-11-26T07:17:34.128871+0000","last_fullsized":"2023-11-26T07:17:34.128871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128543+0000","last_change":"2023-11-26T07:17:33.134943+0000","last_active":"2023-11-26T07:17:34.128543+0000","last_peered":"2023-11-26T07:17:34.128543+0000","last_clean":"2023-11-26T07:17:34.128543+0000","last_became_active":"2023-11-26T07:17:33.134570+0000","last_became_peered":"2023-11-26T07:17:33.134570+0000","last_unstale":"2023-11-26T07:17:34.128543+0000","last_undegraded":"2023-11-26T07:17:34.128543+0000","last_fullsized":"2023-11-26T07:17:34.128543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128407+0000","last_change":"2023-11-26T07:17:33.136068+0000","last_active":"2023-11-26T07:17:34.128407+0000","last_peered":"2023-11-26T07:17:34.128407+0000","last_clean":"2023-11-26T07:17:34.128407+0000","last_became_active":"2023-11-26T07:17:33.135783+0000","last_became_peered":"2023-11-26T07:17:33.135783+0000","last_unstale":"2023-11-26T07:17:34.128407+0000","last_undegraded":"2023-11-26T07:17:34.128407+0000","last_fullsized":"2023-11-26T07:17:34.128407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:30.841394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130800+0000","last_change":"2023-11-26T07:17:33.145916+0000","last_active":"2023-11-26T07:17:34.130800+0000","last_peered":"2023-11-26T07:17:34.130800+0000","last_clean":"2023-11-26T07:17:34.130800+0000","last_became_active":"2023-11-26T07:17:33.141115+0000","last_became_peered":"2023-11-26T07:17:33.141115+0000","last_unstale":"2023-11-26T07:17:34.130800+0000","last_undegraded":"2023-11-26T07:17:34.130800+0000","last_fullsized":"2023-11-26T07:17:34.130800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327928+0000","last_change":"2023-11-26T07:17:33.153356+0000","last_active":"2023-11-26T07:17:34.327928+0000","last_peered":"2023-11-26T07:17:34.327928+0000","last_clean":"2023-11-26T07:17:34.327928+0000","last_became_active":"2023-11-26T07:17:33.153016+0000","last_became_peered":"2023-11-26T07:17:33.153016+0000","last_unstale":"2023-11-26T07:17:34.327928+0000","last_undegraded":"2023-11-26T07:17:34.327928+0000","last_fullsized":"2023-11-26T07:17:34.327928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:27.264806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326905+0000","last_change":"2023-11-26T07:17:33.138021+0000","last_active":"2023-11-26T07:17:34.326905+0000","last_peered":"2023-11-26T07:17:34.326905+0000","last_clean":"2023-11-26T07:17:34.326905+0000","last_became_active":"2023-11-26T07:17:33.137694+0000","last_became_peered":"2023-11-26T07:17:33.137694+0000","last_unstale":"2023-11-26T07:17:34.326905+0000","last_undegraded":"2023-11-26T07:17:34.326905+0000","last_fullsized":"2023-11-26T07:17:34.326905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.325959+0000","last_change":"2023-11-26T07:17:33.125499+0000","last_active":"2023-11-26T07:17:34.325959+0000","last_peered":"2023-11-26T07:17:34.325959+0000","last_clean":"2023-11-26T07:17:34.325959+0000","last_became_active":"2023-11-26T07:17:33.125132+0000","last_became_peered":"2023-11-26T07:17:33.125132+0000","last_unstale":"2023-11-26T07:17:34.325959+0000","last_undegraded":"2023-11-26T07:17:34.325959+0000","last_fullsized":"2023-11-26T07:17:34.325959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326610+0000","last_change":"2023-11-26T07:17:33.147715+0000","last_active":"2023-11-26T07:17:34.326610+0000","last_peered":"2023-11-26T07:17:34.326610+0000","last_clean":"2023-11-26T07:17:34.326610+0000","last_became_active":"2023-11-26T07:17:33.146632+0000","last_became_peered":"2023-11-26T07:17:33.146632+0000","last_unstale":"2023-11-26T07:17:34.326610+0000","last_undegraded":"2023-11-26T07:17:34.326610+0000","last_fullsized":"2023-11-26T07:17:34.326610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:54.079736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326760+0000","last_change":"2023-11-26T07:17:33.141795+0000","last_active":"2023-11-26T07:17:34.326760+0000","last_peered":"2023-11-26T07:17:34.326760+0000","last_clean":"2023-11-26T07:17:34.326760+0000","last_became_active":"2023-11-26T07:17:33.141529+0000","last_became_peered":"2023-11-26T07:17:33.141529+0000","last_unstale":"2023-11-26T07:17:34.326760+0000","last_undegraded":"2023-11-26T07:17:34.326760+0000","last_fullsized":"2023-11-26T07:17:34.326760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:35.443597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327208+0000","last_change":"2023-11-26T07:17:33.156644+0000","last_active":"2023-11-26T07:17:34.327208+0000","last_peered":"2023-11-26T07:17:34.327208+0000","last_clean":"2023-11-26T07:17:34.327208+0000","last_became_active":"2023-11-26T07:17:33.156430+0000","last_became_peered":"2023-11-26T07:17:33.156430+0000","last_unstale":"2023-11-26T07:17:34.327208+0000","last_undegraded":"2023-11-26T07:17:34.327208+0000","last_fullsized":"2023-11-26T07:17:34.327208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:20:54.433587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127997+0000","last_change":"2023-11-26T07:17:33.133505+0000","last_active":"2023-11-26T07:17:34.127997+0000","last_peered":"2023-11-26T07:17:34.127997+0000","last_clean":"2023-11-26T07:17:34.127997+0000","last_became_active":"2023-11-26T07:17:33.133134+0000","last_became_peered":"2023-11-26T07:17:33.133134+0000","last_unstale":"2023-11-26T07:17:34.127997+0000","last_undegraded":"2023-11-26T07:17:34.127997+0000","last_fullsized":"2023-11-26T07:17:34.127997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328056+0000","last_change":"2023-11-26T07:17:33.144471+0000","last_active":"2023-11-26T07:17:34.328056+0000","last_peered":"2023-11-26T07:17:34.328056+0000","last_clean":"2023-11-26T07:17:34.328056+0000","last_became_active":"2023-11-26T07:17:33.144183+0000","last_became_peered":"2023-11-26T07:17:33.144183+0000","last_unstale":"2023-11-26T07:17:34.328056+0000","last_undegraded":"2023-11-26T07:17:34.328056+0000","last_fullsized":"2023-11-26T07:17:34.328056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:52.817990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327032+0000","last_change":"2023-11-26T07:17:33.149400+0000","last_active":"2023-11-26T07:17:34.327032+0000","last_peered":"2023-11-26T07:17:34.327032+0000","last_clean":"2023-11-26T07:17:34.327032+0000","last_became_active":"2023-11-26T07:17:33.144991+0000","last_became_peered":"2023-11-26T07:17:33.144991+0000","last_unstale":"2023-11-26T07:17:34.327032+0000","last_undegraded":"2023-11-26T07:17:34.327032+0000","last_fullsized":"2023-11-26T07:17:34.327032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:15:29.617633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130588+0000","last_change":"2023-11-26T07:17:33.125477+0000","last_active":"2023-11-26T07:17:34.130588+0000","last_peered":"2023-11-26T07:17:34.130588+0000","last_clean":"2023-11-26T07:17:34.130588+0000","last_became_active":"2023-11-26T07:17:33.125109+0000","last_became_peered":"2023-11-26T07:17:33.125109+0000","last_unstale":"2023-11-26T07:17:34.130588+0000","last_undegraded":"2023-11-26T07:17:34.130588+0000","last_fullsized":"2023-11-26T07:17:34.130588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326026+0000","last_change":"2023-11-26T07:17:33.138762+0000","last_active":"2023-11-26T07:17:34.326026+0000","last_peered":"2023-11-26T07:17:34.326026+0000","last_clean":"2023-11-26T07:17:34.326026+0000","last_became_active":"2023-11-26T07:17:33.138466+0000","last_became_peered":"2023-11-26T07:17:33.138466+0000","last_unstale":"2023-11-26T07:17:34.326026+0000","last_undegraded":"2023-11-26T07:17:34.326026+0000","last_fullsized":"2023-11-26T07:17:34.326026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130957+0000","last_change":"2023-11-26T07:17:33.128236+0000","last_active":"2023-11-26T07:17:34.130957+0000","last_peered":"2023-11-26T07:17:34.130957+0000","last_clean":"2023-11-26T07:17:34.130957+0000","last_became_active":"2023-11-26T07:17:33.127900+0000","last_became_peered":"2023-11-26T07:17:33.127900+0000","last_unstale":"2023-11-26T07:17:34.130957+0000","last_undegraded":"2023-11-26T07:17:34.130957+0000","last_fullsized":"2023-11-26T07:17:34.130957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128993+0000","last_change":"2023-11-26T07:17:33.144760+0000","last_active":"2023-11-26T07:17:34.128993+0000","last_peered":"2023-11-26T07:17:34.128993+0000","last_clean":"2023-11-26T07:17:34.128993+0000","last_became_active":"2023-11-26T07:17:33.144272+0000","last_became_peered":"2023-11-26T07:17:33.144272+0000","last_unstale":"2023-11-26T07:17:34.128993+0000","last_undegraded":"2023-11-26T07:17:34.128993+0000","last_fullsized":"2023-11-26T07:17:34.128993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327140+0000","last_change":"2023-11-26T07:17:33.151793+0000","last_active":"2023-11-26T07:17:34.327140+0000","last_peered":"2023-11-26T07:17:34.327140+0000","last_clean":"2023-11-26T07:17:34.327140+0000","last_became_active":"2023-11-26T07:17:33.151517+0000","last_became_peered":"2023-11-26T07:17:33.151517+0000","last_unstale":"2023-11-26T07:17:34.327140+0000","last_undegraded":"2023-11-26T07:17:34.327140+0000","last_fullsized":"2023-11-26T07:17:34.327140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131015+0000","last_change":"2023-11-26T07:17:33.128368+0000","last_active":"2023-11-26T07:17:34.131015+0000","last_peered":"2023-11-26T07:17:34.131015+0000","last_clean":"2023-11-26T07:17:34.131015+0000","last_became_active":"2023-11-26T07:17:33.128068+0000","last_became_peered":"2023-11-26T07:17:33.128068+0000","last_unstale":"2023-11-26T07:17:34.131015+0000","last_undegraded":"2023-11-26T07:17:34.131015+0000","last_fullsized":"2023-11-26T07:17:34.131015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:54.079736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328159+0000","last_change":"2023-11-26T07:17:33.124145+0000","last_active":"2023-11-26T07:17:34.328159+0000","last_peered":"2023-11-26T07:17:34.328159+0000","last_clean":"2023-11-26T07:17:34.328159+0000","last_became_active":"2023-11-26T07:17:33.123870+0000","last_became_peered":"2023-11-26T07:17:33.123870+0000","last_unstale":"2023-11-26T07:17:34.328159+0000","last_undegraded":"2023-11-26T07:17:34.328159+0000","last_fullsized":"2023-11-26T07:17:34.328159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327220+0000","last_change":"2023-11-26T07:17:33.138523+0000","last_active":"2023-11-26T07:17:34.327220+0000","last_peered":"2023-11-26T07:17:34.327220+0000","last_clean":"2023-11-26T07:17:34.327220+0000","last_became_active":"2023-11-26T07:17:33.138213+0000","last_became_peered":"2023-11-26T07:17:33.138213+0000","last_unstale":"2023-11-26T07:17:34.327220+0000","last_undegraded":"2023-11-26T07:17:34.327220+0000","last_fullsized":"2023-11-26T07:17:34.327220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:12.772210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326933+0000","last_change":"2023-11-26T07:17:33.128197+0000","last_active":"2023-11-26T07:17:34.326933+0000","last_peered":"2023-11-26T07:17:34.326933+0000","last_clean":"2023-11-26T07:17:34.326933+0000","last_became_active":"2023-11-26T07:17:33.127911+0000","last_became_peered":"2023-11-26T07:17:33.127911+0000","last_unstale":"2023-11-26T07:17:34.326933+0000","last_undegraded":"2023-11-26T07:17:34.326933+0000","last_fullsized":"2023-11-26T07:17:34.326933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:17.558267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127456+0000","last_change":"2023-11-26T07:17:33.140214+0000","last_active":"2023-11-26T07:17:34.127456+0000","last_peered":"2023-11-26T07:17:34.127456+0000","last_clean":"2023-11-26T07:17:34.127456+0000","last_became_active":"2023-11-26T07:17:33.139927+0000","last_became_peered":"2023-11-26T07:17:33.139927+0000","last_unstale":"2023-11-26T07:17:34.127456+0000","last_undegraded":"2023-11-26T07:17:34.127456+0000","last_fullsized":"2023-11-26T07:17:34.127456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128003+0000","last_change":"2023-11-26T07:17:33.151273+0000","last_active":"2023-11-26T07:17:34.128003+0000","last_peered":"2023-11-26T07:17:34.128003+0000","last_clean":"2023-11-26T07:17:34.128003+0000","last_became_active":"2023-11-26T07:17:33.150182+0000","last_became_peered":"2023-11-26T07:17:33.150182+0000","last_unstale":"2023-11-26T07:17:34.128003+0000","last_undegraded":"2023-11-26T07:17:34.128003+0000","last_fullsized":"2023-11-26T07:17:34.128003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:02.280784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131100+0000","last_change":"2023-11-26T07:17:32.135703+0000","last_active":"2023-11-26T07:17:34.131100+0000","last_peered":"2023-11-26T07:17:34.131100+0000","last_clean":"2023-11-26T07:17:34.131100+0000","last_became_active":"2023-11-26T07:17:32.135392+0000","last_became_peered":"2023-11-26T07:17:32.135392+0000","last_unstale":"2023-11-26T07:17:34.131100+0000","last_undegraded":"2023-11-26T07:17:34.131100+0000","last_fullsized":"2023-11-26T07:17:34.131100+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328230+0000","last_change":"2023-11-26T07:17:33.123539+0000","last_active":"2023-11-26T07:17:34.328230+0000","last_peered":"2023-11-26T07:17:34.328230+0000","last_clean":"2023-11-26T07:17:34.328230+0000","last_became_active":"2023-11-26T07:17:33.123240+0000","last_became_peered":"2023-11-26T07:17:33.123240+0000","last_unstale":"2023-11-26T07:17:34.328230+0000","last_undegraded":"2023-11-26T07:17:34.328230+0000","last_fullsized":"2023-11-26T07:17:34.328230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127853+0000","last_change":"2023-11-26T07:17:32.124304+0000","last_active":"2023-11-26T07:17:34.127853+0000","last_peered":"2023-11-26T07:17:34.127853+0000","last_clean":"2023-11-26T07:17:34.127853+0000","last_became_active":"2023-11-26T07:17:32.122374+0000","last_became_peered":"2023-11-26T07:17:32.122374+0000","last_unstale":"2023-11-26T07:17:34.127853+0000","last_undegraded":"2023-11-26T07:17:34.127853+0000","last_fullsized":"2023-11-26T07:17:34.127853+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:53:41.643510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130553+0000","last_change":"2023-11-26T07:17:32.132756+0000","last_active":"2023-11-26T07:17:34.130553+0000","last_peered":"2023-11-26T07:17:34.130553+0000","last_clean":"2023-11-26T07:17:34.130553+0000","last_became_active":"2023-11-26T07:17:32.132329+0000","last_became_peered":"2023-11-26T07:17:32.132329+0000","last_unstale":"2023-11-26T07:17:34.130553+0000","last_undegraded":"2023-11-26T07:17:34.130553+0000","last_fullsized":"2023-11-26T07:17:34.130553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130543+0000","last_change":"2023-11-26T07:17:33.124808+0000","last_active":"2023-11-26T07:17:34.130543+0000","last_peered":"2023-11-26T07:17:34.130543+0000","last_clean":"2023-11-26T07:17:34.130543+0000","last_became_active":"2023-11-26T07:17:33.124508+0000","last_became_peered":"2023-11-26T07:17:33.124508+0000","last_unstale":"2023-11-26T07:17:34.130543+0000","last_undegraded":"2023-11-26T07:17:34.130543+0000","last_fullsized":"2023-11-26T07:17:34.130543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326062+0000","last_change":"2023-11-26T07:17:32.136207+0000","last_active":"2023-11-26T07:17:34.326062+0000","last_peered":"2023-11-26T07:17:34.326062+0000","last_clean":"2023-11-26T07:17:34.326062+0000","last_became_active":"2023-11-26T07:17:32.135850+0000","last_became_peered":"2023-11-26T07:17:32.135850+0000","last_unstale":"2023-11-26T07:17:34.326062+0000","last_undegraded":"2023-11-26T07:17:34.326062+0000","last_fullsized":"2023-11-26T07:17:34.326062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:48.178664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326761+0000","last_change":"2023-11-26T07:17:33.153245+0000","last_active":"2023-11-26T07:17:34.326761+0000","last_peered":"2023-11-26T07:17:34.326761+0000","last_clean":"2023-11-26T07:17:34.326761+0000","last_became_active":"2023-11-26T07:17:33.151566+0000","last_became_peered":"2023-11-26T07:17:33.151566+0000","last_unstale":"2023-11-26T07:17:34.326761+0000","last_undegraded":"2023-11-26T07:17:34.326761+0000","last_fullsized":"2023-11-26T07:17:34.326761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.129164+0000","last_change":"2023-11-26T07:17:32.126580+0000","last_active":"2023-11-26T07:17:34.129164+0000","last_peered":"2023-11-26T07:17:34.129164+0000","last_clean":"2023-11-26T07:17:34.129164+0000","last_became_active":"2023-11-26T07:17:32.125693+0000","last_became_peered":"2023-11-26T07:17:32.125693+0000","last_unstale":"2023-11-26T07:17:34.129164+0000","last_undegraded":"2023-11-26T07:17:34.129164+0000","last_fullsized":"2023-11-26T07:17:34.129164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327410+0000","last_change":"2023-11-26T07:17:33.145310+0000","last_active":"2023-11-26T07:17:34.327410+0000","last_peered":"2023-11-26T07:17:34.327410+0000","last_clean":"2023-11-26T07:17:34.327410+0000","last_became_active":"2023-11-26T07:17:33.145010+0000","last_became_peered":"2023-11-26T07:17:33.145010+0000","last_unstale":"2023-11-26T07:17:34.327410+0000","last_undegraded":"2023-11-26T07:17:34.327410+0000","last_fullsized":"2023-11-26T07:17:34.327410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:23:10.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327442+0000","last_change":"2023-11-26T07:17:32.130729+0000","last_active":"2023-11-26T07:17:34.327442+0000","last_peered":"2023-11-26T07:17:34.327442+0000","last_clean":"2023-11-26T07:17:34.327442+0000","last_became_active":"2023-11-26T07:17:32.130438+0000","last_became_peered":"2023-11-26T07:17:32.130438+0000","last_unstale":"2023-11-26T07:17:34.327442+0000","last_undegraded":"2023-11-26T07:17:34.327442+0000","last_fullsized":"2023-11-26T07:17:34.327442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:24.546723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328264+0000","last_change":"2023-11-26T07:17:33.136875+0000","last_active":"2023-11-26T07:17:34.328264+0000","last_peered":"2023-11-26T07:17:34.328264+0000","last_clean":"2023-11-26T07:17:34.328264+0000","last_became_active":"2023-11-26T07:17:33.136544+0000","last_became_peered":"2023-11-26T07:17:33.136544+0000","last_unstale":"2023-11-26T07:17:34.328264+0000","last_undegraded":"2023-11-26T07:17:34.328264+0000","last_fullsized":"2023-11-26T07:17:34.328264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327150+0000","last_change":"2023-11-26T07:17:32.127762+0000","last_active":"2023-11-26T07:17:34.327150+0000","last_peered":"2023-11-26T07:17:34.327150+0000","last_clean":"2023-11-26T07:17:34.327150+0000","last_became_active":"2023-11-26T07:17:32.127380+0000","last_became_peered":"2023-11-26T07:17:32.127380+0000","last_unstale":"2023-11-26T07:17:34.327150+0000","last_undegraded":"2023-11-26T07:17:34.327150+0000","last_fullsized":"2023-11-26T07:17:34.327150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128525+0000","last_change":"2023-11-26T07:17:33.147339+0000","last_active":"2023-11-26T07:17:34.128525+0000","last_peered":"2023-11-26T07:17:34.128525+0000","last_clean":"2023-11-26T07:17:34.128525+0000","last_became_active":"2023-11-26T07:17:33.147047+0000","last_became_peered":"2023-11-26T07:17:33.147047+0000","last_unstale":"2023-11-26T07:17:34.128525+0000","last_undegraded":"2023-11-26T07:17:34.128525+0000","last_fullsized":"2023-11-26T07:17:34.128525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:35.443597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128083+0000","last_change":"2023-11-26T07:17:32.133337+0000","last_active":"2023-11-26T07:17:34.128083+0000","last_peered":"2023-11-26T07:17:34.128083+0000","last_clean":"2023-11-26T07:17:34.128083+0000","last_became_active":"2023-11-26T07:17:32.132169+0000","last_became_peered":"2023-11-26T07:17:32.132169+0000","last_unstale":"2023-11-26T07:17:34.128083+0000","last_undegraded":"2023-11-26T07:17:34.128083+0000","last_fullsized":"2023-11-26T07:17:34.128083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:21.575382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326848+0000","last_change":"2023-11-26T07:17:33.150427+0000","last_active":"2023-11-26T07:17:34.326848+0000","last_peered":"2023-11-26T07:17:34.326848+0000","last_clean":"2023-11-26T07:17:34.326848+0000","last_became_active":"2023-11-26T07:17:33.150029+0000","last_became_peered":"2023-11-26T07:17:33.150029+0000","last_unstale":"2023-11-26T07:17:34.326848+0000","last_undegraded":"2023-11-26T07:17:34.326848+0000","last_fullsized":"2023-11-26T07:17:34.326848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:23:10.346590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328227+0000","last_change":"2023-11-26T07:17:32.133555+0000","last_active":"2023-11-26T07:17:34.328227+0000","last_peered":"2023-11-26T07:17:34.328227+0000","last_clean":"2023-11-26T07:17:34.328227+0000","last_became_active":"2023-11-26T07:17:32.132862+0000","last_became_peered":"2023-11-26T07:17:32.132862+0000","last_unstale":"2023-11-26T07:17:34.328227+0000","last_undegraded":"2023-11-26T07:17:34.328227+0000","last_fullsized":"2023-11-26T07:17:34.328227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:19.749525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328005+0000","last_change":"2023-11-26T07:17:33.144778+0000","last_active":"2023-11-26T07:17:34.328005+0000","last_peered":"2023-11-26T07:17:34.328005+0000","last_clean":"2023-11-26T07:17:34.328005+0000","last_became_active":"2023-11-26T07:17:33.144469+0000","last_became_peered":"2023-11-26T07:17:33.144469+0000","last_unstale":"2023-11-26T07:17:34.328005+0000","last_undegraded":"2023-11-26T07:17:34.328005+0000","last_fullsized":"2023-11-26T07:17:34.328005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:10.505099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327149+0000","last_change":"2023-11-26T07:17:32.140716+0000","last_active":"2023-11-26T07:17:34.327149+0000","last_peered":"2023-11-26T07:17:34.327149+0000","last_clean":"2023-11-26T07:17:34.327149+0000","last_became_active":"2023-11-26T07:17:32.139344+0000","last_became_peered":"2023-11-26T07:17:32.139344+0000","last_unstale":"2023-11-26T07:17:34.327149+0000","last_undegraded":"2023-11-26T07:17:34.327149+0000","last_fullsized":"2023-11-26T07:17:34.327149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326526+0000","last_change":"2023-11-26T07:17:33.152939+0000","last_active":"2023-11-26T07:17:34.326526+0000","last_peered":"2023-11-26T07:17:34.326526+0000","last_clean":"2023-11-26T07:17:34.326526+0000","last_became_active":"2023-11-26T07:17:33.152626+0000","last_became_peered":"2023-11-26T07:17:33.152626+0000","last_unstale":"2023-11-26T07:17:34.326526+0000","last_undegraded":"2023-11-26T07:17:34.326526+0000","last_fullsized":"2023-11-26T07:17:34.326526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:13.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127755+0000","last_change":"2023-11-26T07:17:32.127951+0000","last_active":"2023-11-26T07:17:34.127755+0000","last_peered":"2023-11-26T07:17:34.127755+0000","last_clean":"2023-11-26T07:17:34.127755+0000","last_became_active":"2023-11-26T07:17:32.127195+0000","last_became_peered":"2023-11-26T07:17:32.127195+0000","last_unstale":"2023-11-26T07:17:34.127755+0000","last_undegraded":"2023-11-26T07:17:34.127755+0000","last_fullsized":"2023-11-26T07:17:34.127755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.551435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127675+0000","last_change":"2023-11-26T07:17:33.132518+0000","last_active":"2023-11-26T07:17:34.127675+0000","last_peered":"2023-11-26T07:17:34.127675+0000","last_clean":"2023-11-26T07:17:34.127675+0000","last_became_active":"2023-11-26T07:17:33.131905+0000","last_became_peered":"2023-11-26T07:17:33.131905+0000","last_unstale":"2023-11-26T07:17:34.127675+0000","last_undegraded":"2023-11-26T07:17:34.127675+0000","last_fullsized":"2023-11-26T07:17:34.127675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326584+0000","last_change":"2023-11-26T07:17:32.129880+0000","last_active":"2023-11-26T07:17:34.326584+0000","last_peered":"2023-11-26T07:17:34.326584+0000","last_clean":"2023-11-26T07:17:34.326584+0000","last_became_active":"2023-11-26T07:17:32.129575+0000","last_became_peered":"2023-11-26T07:17:32.129575+0000","last_unstale":"2023-11-26T07:17:34.326584+0000","last_undegraded":"2023-11-26T07:17:34.326584+0000","last_fullsized":"2023-11-26T07:17:34.326584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:27.401076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326605+0000","last_change":"2023-11-26T07:17:33.152780+0000","last_active":"2023-11-26T07:17:34.326605+0000","last_peered":"2023-11-26T07:17:34.326605+0000","last_clean":"2023-11-26T07:17:34.326605+0000","last_became_active":"2023-11-26T07:17:33.152468+0000","last_became_peered":"2023-11-26T07:17:33.152468+0000","last_unstale":"2023-11-26T07:17:34.326605+0000","last_undegraded":"2023-11-26T07:17:34.326605+0000","last_fullsized":"2023-11-26T07:17:34.326605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:56:25.969462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128485+0000","last_change":"2023-11-26T07:17:32.132211+0000","last_active":"2023-11-26T07:17:34.128485+0000","last_peered":"2023-11-26T07:17:34.128485+0000","last_clean":"2023-11-26T07:17:34.128485+0000","last_became_active":"2023-11-26T07:17:32.131233+0000","last_became_peered":"2023-11-26T07:17:32.131233+0000","last_unstale":"2023-11-26T07:17:34.128485+0000","last_undegraded":"2023-11-26T07:17:34.128485+0000","last_fullsized":"2023-11-26T07:17:34.128485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:21.662285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326787+0000","last_change":"2023-11-26T07:17:33.154639+0000","last_active":"2023-11-26T07:17:34.326787+0000","last_peered":"2023-11-26T07:17:34.326787+0000","last_clean":"2023-11-26T07:17:34.326787+0000","last_became_active":"2023-11-26T07:17:33.153245+0000","last_became_peered":"2023-11-26T07:17:33.153245+0000","last_unstale":"2023-11-26T07:17:34.326787+0000","last_undegraded":"2023-11-26T07:17:34.326787+0000","last_fullsized":"2023-11-26T07:17:34.326787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:17:12.772210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326854+0000","last_change":"2023-11-26T07:17:32.138855+0000","last_active":"2023-11-26T07:17:34.326854+0000","last_peered":"2023-11-26T07:17:34.326854+0000","last_clean":"2023-11-26T07:17:34.326854+0000","last_became_active":"2023-11-26T07:17:32.138566+0000","last_became_peered":"2023-11-26T07:17:32.138566+0000","last_unstale":"2023-11-26T07:17:34.326854+0000","last_undegraded":"2023-11-26T07:17:34.326854+0000","last_fullsized":"2023-11-26T07:17:34.326854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:29.834563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327040+0000","last_change":"2023-11-26T07:17:33.152760+0000","last_active":"2023-11-26T07:17:34.327040+0000","last_peered":"2023-11-26T07:17:34.327040+0000","last_clean":"2023-11-26T07:17:34.327040+0000","last_became_active":"2023-11-26T07:17:33.152455+0000","last_became_peered":"2023-11-26T07:17:33.152455+0000","last_unstale":"2023-11-26T07:17:34.327040+0000","last_undegraded":"2023-11-26T07:17:34.327040+0000","last_fullsized":"2023-11-26T07:17:34.327040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:40:40.786432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327244+0000","last_change":"2023-11-26T07:17:32.132634+0000","last_active":"2023-11-26T07:17:34.327244+0000","last_peered":"2023-11-26T07:17:34.327244+0000","last_clean":"2023-11-26T07:17:34.327244+0000","last_became_active":"2023-11-26T07:17:32.132192+0000","last_became_peered":"2023-11-26T07:17:32.132192+0000","last_unstale":"2023-11-26T07:17:34.327244+0000","last_undegraded":"2023-11-26T07:17:34.327244+0000","last_fullsized":"2023-11-26T07:17:34.327244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:54:55.533333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327181+0000","last_change":"2023-11-26T07:17:33.147935+0000","last_active":"2023-11-26T07:17:34.327181+0000","last_peered":"2023-11-26T07:17:34.327181+0000","last_clean":"2023-11-26T07:17:34.327181+0000","last_became_active":"2023-11-26T07:17:33.147595+0000","last_became_peered":"2023-11-26T07:17:33.147595+0000","last_unstale":"2023-11-26T07:17:34.327181+0000","last_undegraded":"2023-11-26T07:17:34.327181+0000","last_fullsized":"2023-11-26T07:17:34.327181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:56:25.969462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326702+0000","last_change":"2023-11-26T07:17:32.128101+0000","last_active":"2023-11-26T07:17:34.326702+0000","last_peered":"2023-11-26T07:17:34.326702+0000","last_clean":"2023-11-26T07:17:34.326702+0000","last_became_active":"2023-11-26T07:17:32.127821+0000","last_became_peered":"2023-11-26T07:17:32.127821+0000","last_unstale":"2023-11-26T07:17:34.326702+0000","last_undegraded":"2023-11-26T07:17:34.326702+0000","last_fullsized":"2023-11-26T07:17:34.326702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:30:24.091629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326978+0000","last_change":"2023-11-26T07:17:33.153014+0000","last_active":"2023-11-26T07:17:34.326978+0000","last_peered":"2023-11-26T07:17:34.326978+0000","last_clean":"2023-11-26T07:17:34.326978+0000","last_became_active":"2023-11-26T07:17:33.152675+0000","last_became_peered":"2023-11-26T07:17:33.152675+0000","last_unstale":"2023-11-26T07:17:34.326978+0000","last_undegraded":"2023-11-26T07:17:34.326978+0000","last_fullsized":"2023-11-26T07:17:34.326978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:06.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-26T07:17:33.140703+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:17:33.140703+0000","last_peered":"2023-11-26T07:17:33.140703+0000","last_clean":"2023-11-26T07:17:33.140703+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:17:33.140703+0000","last_undegraded":"2023-11-26T07:17:33.140703+0000","last_fullsized":"2023-11-26T07:17:33.140703+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.325979+0000","last_change":"2023-11-26T07:17:32.127453+0000","last_active":"2023-11-26T07:17:34.325979+0000","last_peered":"2023-11-26T07:17:34.325979+0000","last_clean":"2023-11-26T07:17:34.325979+0000","last_became_active":"2023-11-26T07:17:32.127171+0000","last_became_peered":"2023-11-26T07:17:32.127171+0000","last_unstale":"2023-11-26T07:17:34.325979+0000","last_undegraded":"2023-11-26T07:17:34.325979+0000","last_fullsized":"2023-11-26T07:17:34.325979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:21:10.106650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130743+0000","last_change":"2023-11-26T07:17:33.132790+0000","last_active":"2023-11-26T07:17:34.130743+0000","last_peered":"2023-11-26T07:17:34.130743+0000","last_clean":"2023-11-26T07:17:34.130743+0000","last_became_active":"2023-11-26T07:17:33.132415+0000","last_became_peered":"2023-11-26T07:17:33.132415+0000","last_unstale":"2023-11-26T07:17:34.130743+0000","last_undegraded":"2023-11-26T07:17:34.130743+0000","last_fullsized":"2023-11-26T07:17:34.130743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326218+0000","last_change":"2023-11-26T07:17:32.134663+0000","last_active":"2023-11-26T07:17:34.326218+0000","last_peered":"2023-11-26T07:17:34.326218+0000","last_clean":"2023-11-26T07:17:34.326218+0000","last_became_active":"2023-11-26T07:17:32.134372+0000","last_became_peered":"2023-11-26T07:17:32.134372+0000","last_unstale":"2023-11-26T07:17:34.326218+0000","last_undegraded":"2023-11-26T07:17:34.326218+0000","last_fullsized":"2023-11-26T07:17:34.326218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:34.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327893+0000","last_change":"2023-11-26T07:17:33.131023+0000","last_active":"2023-11-26T07:17:34.327893+0000","last_peered":"2023-11-26T07:17:34.327893+0000","last_clean":"2023-11-26T07:17:34.327893+0000","last_became_active":"2023-11-26T07:17:33.130665+0000","last_became_peered":"2023-11-26T07:17:33.130665+0000","last_unstale":"2023-11-26T07:17:34.327893+0000","last_undegraded":"2023-11-26T07:17:34.327893+0000","last_fullsized":"2023-11-26T07:17:34.327893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:54:56.540164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327009+0000","last_change":"2023-11-26T07:17:32.141091+0000","last_active":"2023-11-26T07:17:34.327009+0000","last_peered":"2023-11-26T07:17:34.327009+0000","last_clean":"2023-11-26T07:17:34.327009+0000","last_became_active":"2023-11-26T07:17:32.139978+0000","last_became_peered":"2023-11-26T07:17:32.139978+0000","last_unstale":"2023-11-26T07:17:34.327009+0000","last_undegraded":"2023-11-26T07:17:34.327009+0000","last_fullsized":"2023-11-26T07:17:34.327009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:21:01.273952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326756+0000","last_change":"2023-11-26T07:17:33.137737+0000","last_active":"2023-11-26T07:17:34.326756+0000","last_peered":"2023-11-26T07:17:34.326756+0000","last_clean":"2023-11-26T07:17:34.326756+0000","last_became_active":"2023-11-26T07:17:33.137350+0000","last_became_peered":"2023-11-26T07:17:33.137350+0000","last_unstale":"2023-11-26T07:17:34.326756+0000","last_undegraded":"2023-11-26T07:17:34.326756+0000","last_fullsized":"2023-11-26T07:17:34.326756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:14:27.710583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327251+0000","last_change":"2023-11-26T07:17:32.134855+0000","last_active":"2023-11-26T07:17:34.327251+0000","last_peered":"2023-11-26T07:17:34.327251+0000","last_clean":"2023-11-26T07:17:34.327251+0000","last_became_active":"2023-11-26T07:17:32.134587+0000","last_became_peered":"2023-11-26T07:17:32.134587+0000","last_unstale":"2023-11-26T07:17:34.327251+0000","last_undegraded":"2023-11-26T07:17:34.327251+0000","last_fullsized":"2023-11-26T07:17:34.327251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.128590+0000","last_change":"2023-11-26T07:17:33.122483+0000","last_active":"2023-11-26T07:17:34.128590+0000","last_peered":"2023-11-26T07:17:34.128590+0000","last_clean":"2023-11-26T07:17:34.128590+0000","last_became_active":"2023-11-26T07:17:33.122042+0000","last_became_peered":"2023-11-26T07:17:33.122042+0000","last_unstale":"2023-11-26T07:17:34.128590+0000","last_undegraded":"2023-11-26T07:17:34.128590+0000","last_fullsized":"2023-11-26T07:17:34.128590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:17.558267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327027+0000","last_change":"2023-11-26T07:17:32.124516+0000","last_active":"2023-11-26T07:17:34.327027+0000","last_peered":"2023-11-26T07:17:34.327027+0000","last_clean":"2023-11-26T07:17:34.327027+0000","last_became_active":"2023-11-26T07:17:32.123978+0000","last_became_peered":"2023-11-26T07:17:32.123978+0000","last_unstale":"2023-11-26T07:17:34.327027+0000","last_undegraded":"2023-11-26T07:17:34.327027+0000","last_fullsized":"2023-11-26T07:17:34.327027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:34.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326612+0000","last_change":"2023-11-26T07:17:33.139055+0000","last_active":"2023-11-26T07:17:34.326612+0000","last_peered":"2023-11-26T07:17:34.326612+0000","last_clean":"2023-11-26T07:17:34.326612+0000","last_became_active":"2023-11-26T07:17:33.138235+0000","last_became_peered":"2023-11-26T07:17:33.138235+0000","last_unstale":"2023-11-26T07:17:34.326612+0000","last_undegraded":"2023-11-26T07:17:34.326612+0000","last_fullsized":"2023-11-26T07:17:34.326612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.726340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327313+0000","last_change":"2023-11-26T07:17:32.135542+0000","last_active":"2023-11-26T07:17:34.327313+0000","last_peered":"2023-11-26T07:17:34.327313+0000","last_clean":"2023-11-26T07:17:34.327313+0000","last_became_active":"2023-11-26T07:17:32.135078+0000","last_became_peered":"2023-11-26T07:17:32.135078+0000","last_unstale":"2023-11-26T07:17:34.327313+0000","last_undegraded":"2023-11-26T07:17:34.327313+0000","last_fullsized":"2023-11-26T07:17:34.327313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:04.536431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327077+0000","last_change":"2023-11-26T07:17:33.149589+0000","last_active":"2023-11-26T07:17:34.327077+0000","last_peered":"2023-11-26T07:17:34.327077+0000","last_clean":"2023-11-26T07:17:34.327077+0000","last_became_active":"2023-11-26T07:17:33.149228+0000","last_became_peered":"2023-11-26T07:17:33.149228+0000","last_unstale":"2023-11-26T07:17:34.327077+0000","last_undegraded":"2023-11-26T07:17:34.327077+0000","last_fullsized":"2023-11-26T07:17:34.327077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:21:05.543262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:35.338911+0000","last_change":"2023-11-26T07:17:32.118350+0000","last_active":"2023-11-26T07:17:35.338911+0000","last_peered":"2023-11-26T07:17:35.338911+0000","last_clean":"2023-11-26T07:17:35.338911+0000","last_became_active":"2023-11-26T07:17:32.118040+0000","last_became_peered":"2023-11-26T07:17:32.118040+0000","last_unstale":"2023-11-26T07:17:35.338911+0000","last_undegraded":"2023-11-26T07:17:35.338911+0000","last_fullsized":"2023-11-26T07:17:35.338911+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:18:21.575382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131017+0000","last_change":"2023-11-26T07:17:33.150796+0000","last_active":"2023-11-26T07:17:34.131017+0000","last_peered":"2023-11-26T07:17:34.131017+0000","last_clean":"2023-11-26T07:17:34.131017+0000","last_became_active":"2023-11-26T07:17:33.150543+0000","last_became_peered":"2023-11-26T07:17:33.150543+0000","last_unstale":"2023-11-26T07:17:34.131017+0000","last_undegraded":"2023-11-26T07:17:34.131017+0000","last_fullsized":"2023-11-26T07:17:34.131017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:59:35.145697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.129145+0000","last_change":"2023-11-26T07:17:32.134890+0000","last_active":"2023-11-26T07:17:34.129145+0000","last_peered":"2023-11-26T07:17:34.129145+0000","last_clean":"2023-11-26T07:17:34.129145+0000","last_became_active":"2023-11-26T07:17:32.134621+0000","last_became_peered":"2023-11-26T07:17:32.134621+0000","last_unstale":"2023-11-26T07:17:34.129145+0000","last_undegraded":"2023-11-26T07:17:34.129145+0000","last_fullsized":"2023-11-26T07:17:34.129145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:09.498267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328230+0000","last_change":"2023-11-26T07:17:33.135948+0000","last_active":"2023-11-26T07:17:34.328230+0000","last_peered":"2023-11-26T07:17:34.328230+0000","last_clean":"2023-11-26T07:17:34.328230+0000","last_became_active":"2023-11-26T07:17:33.135636+0000","last_became_peered":"2023-11-26T07:17:33.135636+0000","last_unstale":"2023-11-26T07:17:34.328230+0000","last_undegraded":"2023-11-26T07:17:34.328230+0000","last_fullsized":"2023-11-26T07:17:34.328230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:49.185495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326291+0000","last_change":"2023-11-26T07:17:32.134899+0000","last_active":"2023-11-26T07:17:34.326291+0000","last_peered":"2023-11-26T07:17:34.326291+0000","last_clean":"2023-11-26T07:17:34.326291+0000","last_became_active":"2023-11-26T07:17:32.134566+0000","last_became_peered":"2023-11-26T07:17:32.134566+0000","last_unstale":"2023-11-26T07:17:34.326291+0000","last_undegraded":"2023-11-26T07:17:34.326291+0000","last_fullsized":"2023-11-26T07:17:34.326291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:06:29.834563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127563+0000","last_change":"2023-11-26T07:17:33.123386+0000","last_active":"2023-11-26T07:17:34.127563+0000","last_peered":"2023-11-26T07:17:34.127563+0000","last_clean":"2023-11-26T07:17:34.127563+0000","last_became_active":"2023-11-26T07:17:33.120220+0000","last_became_peered":"2023-11-26T07:17:33.120220+0000","last_unstale":"2023-11-26T07:17:34.127563+0000","last_undegraded":"2023-11-26T07:17:34.127563+0000","last_fullsized":"2023-11-26T07:17:34.127563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:08:20.756357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326867+0000","last_change":"2023-11-26T07:17:32.138806+0000","last_active":"2023-11-26T07:17:34.326867+0000","last_peered":"2023-11-26T07:17:34.326867+0000","last_clean":"2023-11-26T07:17:34.326867+0000","last_became_active":"2023-11-26T07:17:32.138612+0000","last_became_peered":"2023-11-26T07:17:32.138612+0000","last_unstale":"2023-11-26T07:17:34.326867+0000","last_undegraded":"2023-11-26T07:17:34.326867+0000","last_fullsized":"2023-11-26T07:17:34.326867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130595+0000","last_change":"2023-11-26T07:17:33.139703+0000","last_active":"2023-11-26T07:17:34.130595+0000","last_peered":"2023-11-26T07:17:34.130595+0000","last_clean":"2023-11-26T07:17:34.130595+0000","last_became_active":"2023-11-26T07:17:33.139347+0000","last_became_peered":"2023-11-26T07:17:33.139347+0000","last_unstale":"2023-11-26T07:17:34.130595+0000","last_undegraded":"2023-11-26T07:17:34.130595+0000","last_fullsized":"2023-11-26T07:17:34.130595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327161+0000","last_change":"2023-11-26T07:17:32.141098+0000","last_active":"2023-11-26T07:17:34.327161+0000","last_peered":"2023-11-26T07:17:34.327161+0000","last_clean":"2023-11-26T07:17:34.327161+0000","last_became_active":"2023-11-26T07:17:32.137814+0000","last_became_peered":"2023-11-26T07:17:32.137814+0000","last_unstale":"2023-11-26T07:17:34.327161+0000","last_undegraded":"2023-11-26T07:17:34.327161+0000","last_fullsized":"2023-11-26T07:17:34.327161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:53:41.643510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.130835+0000","last_change":"2023-11-26T07:17:33.124135+0000","last_active":"2023-11-26T07:17:34.130835+0000","last_peered":"2023-11-26T07:17:34.130835+0000","last_clean":"2023-11-26T07:17:34.130835+0000","last_became_active":"2023-11-26T07:17:33.123814+0000","last_became_peered":"2023-11-26T07:17:33.123814+0000","last_unstale":"2023-11-26T07:17:34.130835+0000","last_undegraded":"2023-11-26T07:17:34.130835+0000","last_fullsized":"2023-11-26T07:17:34.130835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326630+0000","last_change":"2023-11-26T07:17:32.140282+0000","last_active":"2023-11-26T07:17:34.326630+0000","last_peered":"2023-11-26T07:17:34.326630+0000","last_clean":"2023-11-26T07:17:34.326630+0000","last_became_active":"2023-11-26T07:17:32.139388+0000","last_became_peered":"2023-11-26T07:17:32.139388+0000","last_unstale":"2023-11-26T07:17:34.326630+0000","last_undegraded":"2023-11-26T07:17:34.326630+0000","last_fullsized":"2023-11-26T07:17:34.326630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:24.546723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.127842+0000","last_change":"2023-11-26T07:17:33.140095+0000","last_active":"2023-11-26T07:17:34.127842+0000","last_peered":"2023-11-26T07:17:34.127842+0000","last_clean":"2023-11-26T07:17:34.127842+0000","last_became_active":"2023-11-26T07:17:33.139715+0000","last_became_peered":"2023-11-26T07:17:33.139715+0000","last_unstale":"2023-11-26T07:17:34.127842+0000","last_undegraded":"2023-11-26T07:17:34.127842+0000","last_fullsized":"2023-11-26T07:17:34.127842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:42:25.553555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326469+0000","last_change":"2023-11-26T07:17:33.132899+0000","last_active":"2023-11-26T07:17:34.326469+0000","last_peered":"2023-11-26T07:17:34.326469+0000","last_clean":"2023-11-26T07:17:34.326469+0000","last_became_active":"2023-11-26T07:17:33.132617+0000","last_became_peered":"2023-11-26T07:17:33.132617+0000","last_unstale":"2023-11-26T07:17:34.326469+0000","last_undegraded":"2023-11-26T07:17:34.326469+0000","last_fullsized":"2023-11-26T07:17:34.326469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:41:17.618107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.326817+0000","last_change":"2023-11-26T07:17:32.138620+0000","last_active":"2023-11-26T07:17:34.326817+0000","last_peered":"2023-11-26T07:17:34.326817+0000","last_clean":"2023-11-26T07:17:34.326817+0000","last_became_active":"2023-11-26T07:17:32.138135+0000","last_became_peered":"2023-11-26T07:17:32.138135+0000","last_unstale":"2023-11-26T07:17:34.326817+0000","last_undegraded":"2023-11-26T07:17:34.326817+0000","last_fullsized":"2023-11-26T07:17:34.326817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:57:26.257975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327036+0000","last_change":"2023-11-26T07:17:33.136866+0000","last_active":"2023-11-26T07:17:34.327036+0000","last_peered":"2023-11-26T07:17:34.327036+0000","last_clean":"2023-11-26T07:17:34.327036+0000","last_became_active":"2023-11-26T07:17:33.136571+0000","last_became_peered":"2023-11-26T07:17:33.136571+0000","last_unstale":"2023-11-26T07:17:34.327036+0000","last_undegraded":"2023-11-26T07:17:34.327036+0000","last_fullsized":"2023-11-26T07:17:34.327036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:38:13.297939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.327073+0000","last_change":"2023-11-26T07:17:32.135405+0000","last_active":"2023-11-26T07:17:34.327073+0000","last_peered":"2023-11-26T07:17:34.327073+0000","last_clean":"2023-11-26T07:17:34.327073+0000","last_became_active":"2023-11-26T07:17:32.134722+0000","last_became_peered":"2023-11-26T07:17:32.134722+0000","last_unstale":"2023-11-26T07:17:34.327073+0000","last_undegraded":"2023-11-26T07:17:34.327073+0000","last_fullsized":"2023-11-26T07:17:34.327073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:59:51.811158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328275+0000","last_change":"2023-11-26T07:17:33.138935+0000","last_active":"2023-11-26T07:17:34.328275+0000","last_peered":"2023-11-26T07:17:34.328275+0000","last_clean":"2023-11-26T07:17:34.328275+0000","last_became_active":"2023-11-26T07:17:33.138684+0000","last_became_peered":"2023-11-26T07:17:33.138684+0000","last_unstale":"2023-11-26T07:17:34.328275+0000","last_undegraded":"2023-11-26T07:17:34.328275+0000","last_fullsized":"2023-11-26T07:17:34.328275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:06.333076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131063+0000","last_change":"2023-11-26T07:17:32.130834+0000","last_active":"2023-11-26T07:17:34.131063+0000","last_peered":"2023-11-26T07:17:34.131063+0000","last_clean":"2023-11-26T07:17:34.131063+0000","last_became_active":"2023-11-26T07:17:32.130525+0000","last_became_peered":"2023-11-26T07:17:32.130525+0000","last_unstale":"2023-11-26T07:17:34.131063+0000","last_undegraded":"2023-11-26T07:17:34.131063+0000","last_fullsized":"2023-11-26T07:17:34.131063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:23:09.498267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.131026+0000","last_change":"2023-11-26T07:17:33.146148+0000","last_active":"2023-11-26T07:17:34.131026+0000","last_peered":"2023-11-26T07:17:34.131026+0000","last_clean":"2023-11-26T07:17:34.131026+0000","last_became_active":"2023-11-26T07:17:33.145838+0000","last_became_peered":"2023-11-26T07:17:33.145838+0000","last_unstale":"2023-11-26T07:17:34.131026+0000","last_undegraded":"2023-11-26T07:17:34.131026+0000","last_fullsized":"2023-11-26T07:17:34.131026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:32.096730+0000","last_clean_scrub_stamp":"2023-11-26T07:17:32.096730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:34.436211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-26T07:17:34.328317+0000","last_change":"2023-11-26T07:17:32.135921+0000","last_active":"2023-11-26T07:17:34.328317+0000","last_peered":"2023-11-26T07:17:34.328317+0000","last_clean":"2023-11-26T07:17:34.328317+0000","last_became_active":"2023-11-26T07:17:32.135634+0000","last_became_peered":"2023-11-26T07:17:32.135634+0000","last_unstale":"2023-11-26T07:17:34.328317+0000","last_undegraded":"2023-11-26T07:17:34.328317+0000","last_fullsized":"2023-11-26T07:17:34.328317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:31.089898+0000","last_clean_scrub_stamp":"2023-11-26T07:17:31.089898+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.551435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.492}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1520000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.677}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.016}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":806912,"data_stored":660897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:17:42.302 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-26T07:17:42.302 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-26T07:17:42.302 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-26T07:17:42.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-26T07:17:42.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbc051d640 1 Processor -- start 2023-11-26T07:17:42.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbc051d640 1 -- start start 2023-11-26T07:17:42.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbc051d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbb80cead0 0x7ffbb80d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbc051d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80d1eb0 0x7ffbb806bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbc051d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb80d2300 con 0x7ffbb815d7b0 2023-11-26T07:17:42.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbc051d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80d17e0 con 0x7ffbb80cead0 2023-11-26T07:17:42.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbc051d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80cefd0 con 0x7ffbb80d1eb0 2023-11-26T07:17:42.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbbdab8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbb80cead0 0x7ffbb80d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbbdab8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbb80cead0 0x7ffbb80d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59730/0 (socket says 172.21.15.31:59730) 2023-11-26T07:17:42.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbbdab8640 1 -- 172.21.15.31:0/58926606 learned_addr learned my addr 172.21.15.31:0/58926606 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:42.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbbeaba640 1 --2- 172.21.15.31:0/58926606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80d1eb0 0x7ffbb806bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbbdab8640 1 -- 172.21.15.31:0/58926606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80d1eb0 msgr2=0x7ffbb806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:42.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbbdab8640 1 --2- 172.21.15.31:0/58926606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80d1eb0 0x7ffbb806bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbbdab8640 1 -- 172.21.15.31:0/58926606 >> v1:172.21.15.97:6789/0 conn(0x7ffbb815d7b0 legacy=0x7ffbb80d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.477+0000 7ffbbdab8640 1 -- 172.21.15.31:0/58926606 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb80cf2b0 con 0x7ffbb80cead0 2023-11-26T07:17:42.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbbdab8640 1 --2- 172.21.15.31:0/58926606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb80cead0 0x7ffbb80d1970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009870 tx=0x7ffba801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9183d199045e4208 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbbd2b7640 1 -- 172.21.15.31:0/58926606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba80ab5f0 con 0x7ffbb80cead0 2023-11-26T07:17:42.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbbd2b7640 1 -- 172.21.15.31:0/58926606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba80ab750 con 0x7ffbb80cead0 2023-11-26T07:17:42.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbbd2b7640 1 -- 172.21.15.31:0/58926606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba80abae0 con 0x7ffbb80cead0 2023-11-26T07:17:42.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbc051d640 1 -- 172.21.15.31:0/58926606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb80cead0 msgr2=0x7ffbb80d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbc051d640 1 --2- 172.21.15.31:0/58926606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb80cead0 0x7ffbb80d1970 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009870 tx=0x7ffba801c3f0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbc051d640 1 -- 172.21.15.31:0/58926606 shutdown_connections 2023-11-26T07:17:42.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbc051d640 1 --2- 172.21.15.31:0/58926606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80d1eb0 0x7ffbb806bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbc051d640 1 --2- 172.21.15.31:0/58926606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb80cead0 0x7ffbb80d1970 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbc051d640 1 -- 172.21.15.31:0/58926606 >> 172.21.15.31:0/58926606 conn(0x7ffbb80d33b0 msgr2=0x7ffbb80d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbc051d640 1 -- 172.21.15.31:0/58926606 shutdown_connections 2023-11-26T07:17:42.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.481+0000 7ffbc051d640 1 -- 172.21.15.31:0/58926606 wait complete. 2023-11-26T07:17:42.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbc051d640 1 Processor -- start 2023-11-26T07:17:42.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbc051d640 1 -- start start 2023-11-26T07:17:42.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbc051d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb80cead0 0x7ffbb8159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbc051d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80d1eb0 0x7ffbb80b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbc051d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb815d7b0 0x7ffbb81548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbc051d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb8157380 con 0x7ffbb815d7b0 2023-11-26T07:17:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbc051d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80c93e0 con 0x7ffbb80cead0 2023-11-26T07:17:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbc051d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80c96c0 con 0x7ffbb80d1eb0 2023-11-26T07:17:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbeaba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb815d7b0 0x7ffbb81548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbeaba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb815d7b0 0x7ffbb81548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59738/0 (socket says 172.21.15.31:59738) 2023-11-26T07:17:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbeaba640 1 -- 172.21.15.31:0/621955485 learned_addr learned my addr 172.21.15.31:0/621955485 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbdab8640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80d1eb0 0x7ffbb80b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbe2b9640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb80cead0 0x7ffbb8159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbeaba640 1 -- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80d1eb0 msgr2=0x7ffbb80b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbeaba640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80d1eb0 0x7ffbb80b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbeaba640 1 -- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb80cead0 msgr2=0x7ffbb8159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbeaba640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb80cead0 0x7ffbb8159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbeaba640 1 -- 172.21.15.31:0/621955485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80094d0 con 0x7ffbb815d7b0 2023-11-26T07:17:42.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbbeaba640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb815d7b0 0x7ffbb81548f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0007d90 tx=0x7ffbb000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbaeffd640 1 -- 172.21.15.31:0/621955485 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0014420 con 0x7ffbb815d7b0 2023-11-26T07:17:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbaeffd640 1 -- 172.21.15.31:0/621955485 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb00145a0 con 0x7ffbb815d7b0 2023-11-26T07:17:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbaeffd640 1 -- 172.21.15.31:0/621955485 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0014950 con 0x7ffbb815d7b0 2023-11-26T07:17:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbc051d640 1 -- 172.21.15.31:0/621955485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb80c99a0 con 0x7ffbb815d7b0 2023-11-26T07:17:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.485+0000 7ffbc051d640 1 -- 172.21.15.31:0/621955485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb815a0d0 con 0x7ffbb815d7b0 2023-11-26T07:17:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.489+0000 7ffbc051d640 1 -- 172.21.15.31:0/621955485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb80c9e50 con 0x7ffbb815d7b0 2023-11-26T07:17:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.489+0000 7ffbaeffd640 1 -- 172.21.15.31:0/621955485 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ffbb0014ab0 con 0x7ffbb815d7b0 2023-11-26T07:17:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.489+0000 7ffbaeffd640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffb9c066470 0x7ffb9c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.489+0000 7ffbaeffd640 1 -- 172.21.15.31:0/621955485 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbb0084680 con 0x7ffbb815d7b0 2023-11-26T07:17:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.489+0000 7ffbbe2b9640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffb9c066470 0x7ffb9c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.489+0000 7ffbbe2b9640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffb9c066470 0x7ffb9c068930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80ce080 tx=0x7ffbb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.493+0000 7ffbaeffd640 1 -- 172.21.15.31:0/621955485 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffbb0051ba0 con 0x7ffbb815d7b0 2023-11-26T07:17:42.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 -- 172.21.15.31:0/621955485 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffbb804e850 con 0x7ffbb815d7b0 2023-11-26T07:17:42.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbaeffd640 1 -- 172.21.15.31:0/621955485 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ffbb00551c0 con 0x7ffbb815d7b0 2023-11-26T07:17:42.688 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:42.688 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-26T07:17:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 -- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffb9c066470 msgr2=0x7ffb9c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffb9c066470 0x7ffb9c068930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80ce080 tx=0x7ffbb4009040 comp rx=0 tx=0).stop 2023-11-26T07:17:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 -- 172.21.15.31:0/621955485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb815d7b0 msgr2=0x7ffbb81548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb815d7b0 0x7ffbb81548f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0007d90 tx=0x7ffbb000e4b0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 -- 172.21.15.31:0/621955485 shutdown_connections 2023-11-26T07:17:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffb9c066470 0x7ffb9c068930 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb815d7b0 0x7ffbb81548f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80d1eb0 0x7ffbb80b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 --2- 172.21.15.31:0/621955485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb80cead0 0x7ffbb8159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 -- 172.21.15.31:0/621955485 >> 172.21.15.31:0/621955485 conn(0x7ffbb80d33b0 msgr2=0x7ffbb80d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 -- 172.21.15.31:0/621955485 shutdown_connections 2023-11-26T07:17:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.685+0000 7ffbc051d640 1 -- 172.21.15.31:0/621955485 wait complete. 2023-11-26T07:17:42.709 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-26T07:17:42.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:42.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f0585901640 1 Processor -- start 2023-11-26T07:17:42.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f0585901640 1 -- start start 2023-11-26T07:17:42.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f0585901640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05800d2080 0x7f0580168160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f0585901640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05800cefd0 con 0x7f05800cead0 2023-11-26T07:17:42.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f0585901640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05800cf2b0 con 0x7f058015d7b0 2023-11-26T07:17:42.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f0585901640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05800d17e0 con 0x7f05800d2080 2023-11-26T07:17:42.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f05767fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f05800cead0 0x7f05800d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46364/0 (socket says 172.21.15.31:46364) 2023-11-26T07:17:42.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f05767fc640 1 -- 172.21.15.31:0/749088438 learned_addr learned my addr 172.21.15.31:0/749088438 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:42.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057f7fe640 1 --2- 172.21.15.31:0/749088438 >> v2:172.21.15.97:3300/0 conn(0x7f05800d2080 0x7f0580168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282180826 0 0) 0x7f05800cefd0 con 0x7f05800cead0 2023-11-26T07:17:42.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f056c003530 con 0x7f05800cead0 2023-11-26T07:17:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3494548604 0 0) 0x7f056c003530 con 0x7f05800cead0 2023-11-26T07:17:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 >> v2:172.21.15.97:3300/0 conn(0x7f05800d2080 msgr2=0x7f0580168160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057e7fc640 1 --2- 172.21.15.31:0/749088438 >> v2:172.21.15.97:3300/0 conn(0x7f05800d2080 0x7f0580168160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 >> v1:172.21.15.97:6789/0 conn(0x7f058015d7b0 legacy=0x7f05800d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058006b3f0 con 0x7f05800cead0 2023-11-26T07:17:42.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f0568001a00 con 0x7f05800cead0 2023-11-26T07:17:42.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 >> v1:172.21.15.31:6789/0 conn(0x7f05800cead0 legacy=0x7f05800d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.849+0000 7f057e7fc640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f056c003980 0x7f056c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057e7fc640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056c0047d0 0x7f056c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057e7fc640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f056c0052b0 0x7f056c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800d17e0 con 0x7f056c0047d0 2023-11-26T07:17:42.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c00c180 con 0x7f056c003980 2023-11-26T07:17:42.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c00c340 con 0x7f056c0052b0 2023-11-26T07:17:42.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f05767fc640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056c0047d0 0x7f056c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057f7fe640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f056c0052b0 0x7f056c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057effd640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f056c003980 0x7f056c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f05767fc640 1 -- 172.21.15.31:0/749088438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f056c0052b0 msgr2=0x7f056c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:42.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f05767fc640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f056c0052b0 0x7f056c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f05767fc640 1 -- 172.21.15.31:0/749088438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f056c003980 msgr2=0x7f056c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:42.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f05767fc640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f056c003980 0x7f056c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f05767fc640 1 -- 172.21.15.31:0/749088438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c003530 con 0x7f056c0047d0 2023-11-26T07:17:42.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057f7fe640 1 -- 172.21.15.31:0/749088438 reap_dead start 2023-11-26T07:17:42.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f05767fc640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056c0047d0 0x7f056c004c50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05800cef00 tx=0x7f056801d210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ee55e92ededbac server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680072d0 con 0x7f056c0047d0 2023-11-26T07:17:42.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05680a8070 con 0x7f056c0047d0 2023-11-26T07:17:42.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.853+0000 7f057e7fc640 1 -- 172.21.15.31:0/749088438 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680076f0 con 0x7f056c0047d0 2023-11-26T07:17:42.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 -- 172.21.15.31:0/749088438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056c0047d0 msgr2=0x7f056c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056c0047d0 0x7f056c004c50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05800cef00 tx=0x7f056801d210 comp rx=0 tx=0).stop 2023-11-26T07:17:42.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 -- 172.21.15.31:0/749088438 shutdown_connections 2023-11-26T07:17:42.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 --2- 172.21.15.31:0/749088438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f056c0047d0 0x7f056c004c50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 -- 172.21.15.31:0/749088438 >> 172.21.15.31:0/749088438 conn(0x7f05800d33b0 msgr2=0x7f05800d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:42.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 -- 172.21.15.31:0/749088438 shutdown_connections 2023-11-26T07:17:42.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 -- 172.21.15.31:0/749088438 wait complete. 2023-11-26T07:17:42.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 Processor -- start 2023-11-26T07:17:42.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 -- start start 2023-11-26T07:17:42.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f056c0047d0 0x7f058006c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f058014b660 0x7f058014bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f058009f170 0x7f058009f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800a0ab0 con 0x7f058014b660 2023-11-26T07:17:42.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058014c020 con 0x7f056c0047d0 2023-11-26T07:17:42.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f0585901640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058009fea0 con 0x7f058009f170 2023-11-26T07:17:42.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f05767fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f058014b660 0x7f058014bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f057effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f056c0047d0 0x7f058006c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f05767fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f058014b660 0x7f058014bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59748/0 (socket says 172.21.15.31:59748) 2023-11-26T07:17:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f05767fc640 1 -- 172.21.15.31:0/602902180 learned_addr learned my addr 172.21.15.31:0/602902180 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.857+0000 7f057f7fe640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f058009f170 0x7f058009f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f05767fc640 1 -- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f058009f170 msgr2=0x7f058009f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f05767fc640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f058009f170 0x7f058009f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f05767fc640 1 -- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f056c0047d0 msgr2=0x7f058006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:42.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f05767fc640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f056c0047d0 0x7f058006c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:42.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f05767fc640 1 -- 172.21.15.31:0/602902180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800a0180 con 0x7f058014b660 2023-11-26T07:17:42.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f057effd640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f056c0047d0 0x7f058006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:42.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f05767fc640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f058014b660 0x7f058014bae0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0568001cc0 tx=0x7f056c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f057cff9640 1 -- 172.21.15.31:0/602902180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568001a00 con 0x7f058014b660 2023-11-26T07:17:42.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f057cff9640 1 -- 172.21.15.31:0/602902180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05680070d0 con 0x7f058014b660 2023-11-26T07:17:42.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f0585901640 1 -- 172.21.15.31:0/602902180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05800a0460 con 0x7f058014b660 2023-11-26T07:17:42.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f057cff9640 1 -- 172.21.15.31:0/602902180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568007460 con 0x7f058014b660 2023-11-26T07:17:42.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f0585901640 1 -- 172.21.15.31:0/602902180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058009fb30 con 0x7f058014b660 2023-11-26T07:17:42.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f0585901640 1 -- 172.21.15.31:0/602902180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05800a0c20 con 0x7f058014b660 2023-11-26T07:17:42.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f057cff9640 1 -- 172.21.15.31:0/602902180 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f05680aa030 con 0x7f058014b660 2023-11-26T07:17:42.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f057cff9640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05600664a0 0x7f0560068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:42.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.861+0000 7f057cff9640 1 -- 172.21.15.31:0/602902180 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f056811def0 con 0x7f058014b660 2023-11-26T07:17:42.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.865+0000 7f057effd640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05600664a0 0x7f0560068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:42.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.865+0000 7f057effd640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05600664a0 0x7f0560068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0570007fd0 tx=0x7f0570009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:42.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:42.865+0000 7f057cff9640 1 -- 172.21.15.31:0/602902180 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f05680eb380 con 0x7f058014b660 2023-11-26T07:17:43.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.025+0000 7f0585901640 1 -- 172.21.15.31:0/602902180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f058004e850 con 0x7f058014b660 2023-11-26T07:17:43.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.025+0000 7f057cff9640 1 -- 172.21.15.31:0/602902180 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f05680ee9a0 con 0x7f058014b660 2023-11-26T07:17:43.030 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:43.030 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:43.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 -- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05600664a0 msgr2=0x7f0560068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05600664a0 0x7f0560068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0570007fd0 tx=0x7f0570009040 comp rx=0 tx=0).stop 2023-11-26T07:17:43.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 -- 172.21.15.31:0/602902180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f058014b660 msgr2=0x7f058014bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f058014b660 0x7f058014bae0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0568001cc0 tx=0x7f056c005800 comp rx=0 tx=0).stop 2023-11-26T07:17:43.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 -- 172.21.15.31:0/602902180 shutdown_connections 2023-11-26T07:17:43.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05600664a0 0x7f0560068960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f058009f170 0x7f058009f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f058014b660 0x7f058014bae0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 --2- 172.21.15.31:0/602902180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f056c0047d0 0x7f058006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 -- 172.21.15.31:0/602902180 >> 172.21.15.31:0/602902180 conn(0x7f05800d33b0 msgr2=0x7f05800b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 -- 172.21.15.31:0/602902180 shutdown_connections 2023-11-26T07:17:43.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.029+0000 7f0585901640 1 -- 172.21.15.31:0/602902180 wait complete. 2023-11-26T07:17:43.051 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:43.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096f98d640 1 Processor -- start 2023-11-26T07:17:43.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096f98d640 1 -- start start 2023-11-26T07:17:43.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096f98d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0968146820 0x7f0968070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096f98d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f096815eb50 0x7f09680b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096f98d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0968147190 con 0x7f09680b4210 2023-11-26T07:17:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096f98d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09680b4be0 con 0x7f096815eb50 2023-11-26T07:17:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096f98d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0968146c00 con 0x7f0968146820 2023-11-26T07:17:43.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096df2a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f096815eb50 0x7f09680b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096df2a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f096815eb50 0x7f09680b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59754/0 (socket says 172.21.15.31:59754) 2023-11-26T07:17:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096df2a640 1 -- 172.21.15.31:0/743473644 learned_addr learned my addr 172.21.15.31:0/743473644 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096d729640 1 --2- 172.21.15.31:0/743473644 >> v2:172.21.15.97:3300/0 conn(0x7f0968146820 0x7f0968070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096df2a640 1 -- 172.21.15.31:0/743473644 >> v2:172.21.15.97:3300/0 conn(0x7f0968146820 msgr2=0x7f0968070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096df2a640 1 --2- 172.21.15.31:0/743473644 >> v2:172.21.15.97:3300/0 conn(0x7f0968146820 0x7f0968070a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096df2a640 1 -- 172.21.15.31:0/743473644 >> v1:172.21.15.31:6789/0 conn(0x7f09680b4210 legacy=0x7f0968161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.193+0000 7f096df2a640 1 -- 172.21.15.31:0/743473644 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096806d030 con 0x7f096815eb50 2023-11-26T07:17:43.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f096df2a640 1 --2- 172.21.15.31:0/743473644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096815eb50 0x7f09680b46a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f096806e450 tx=0x7f096001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b379b2fda7c83b10 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f095ffff640 1 -- 172.21.15.31:0/743473644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600a5780 con 0x7f096815eb50 2023-11-26T07:17:43.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f095ffff640 1 -- 172.21.15.31:0/743473644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09600a58e0 con 0x7f096815eb50 2023-11-26T07:17:43.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f095ffff640 1 -- 172.21.15.31:0/743473644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600a5c70 con 0x7f096815eb50 2023-11-26T07:17:43.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f096f98d640 1 -- 172.21.15.31:0/743473644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096815eb50 msgr2=0x7f09680b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f096f98d640 1 --2- 172.21.15.31:0/743473644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096815eb50 0x7f09680b46a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f096806e450 tx=0x7f096001c9a0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f096f98d640 1 -- 172.21.15.31:0/743473644 shutdown_connections 2023-11-26T07:17:43.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f096f98d640 1 --2- 172.21.15.31:0/743473644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f096815eb50 0x7f09680b46a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f096f98d640 1 --2- 172.21.15.31:0/743473644 >> v2:172.21.15.97:3300/0 conn(0x7f0968146820 0x7f0968070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f096f98d640 1 -- 172.21.15.31:0/743473644 >> 172.21.15.31:0/743473644 conn(0x7f09680b2f10 msgr2=0x7f09680b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f096f98d640 1 -- 172.21.15.31:0/743473644 shutdown_connections 2023-11-26T07:17:43.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.197+0000 7f096f98d640 1 -- 172.21.15.31:0/743473644 wait complete. 2023-11-26T07:17:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096f98d640 1 Processor -- start 2023-11-26T07:17:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096f98d640 1 -- start start 2023-11-26T07:17:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096f98d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680b4210 0x7f096813c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096f98d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968146820 0x7f096813db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096f98d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f096815eb50 0x7f096815f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096f98d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096815f9a0 con 0x7f09680b4210 2023-11-26T07:17:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096f98d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096813caa0 con 0x7f0968146820 2023-11-26T07:17:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096f98d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096813cd80 con 0x7f096815eb50 2023-11-26T07:17:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096d729640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680b4210 0x7f096813c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096d729640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680b4210 0x7f096813c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59758/0 (socket says 172.21.15.31:59758) 2023-11-26T07:17:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096d729640 1 -- 172.21.15.31:0/3784042737 learned_addr learned my addr 172.21.15.31:0/3784042737 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096df2a640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f096815eb50 0x7f096815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096cf28640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968146820 0x7f096813db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096df2a640 1 -- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968146820 msgr2=0x7f096813db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096df2a640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968146820 0x7f096813db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096df2a640 1 -- 172.21.15.31:0/3784042737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680b4210 msgr2=0x7f096813c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096df2a640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680b4210 0x7f096813c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096df2a640 1 -- 172.21.15.31:0/3784042737 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0960009b30 con 0x7f096815eb50 2023-11-26T07:17:43.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096d729640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680b4210 0x7f096813c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096df2a640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f096815eb50 0x7f096815f460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f096806b400 tx=0x7f0960019740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f095dffb640 1 -- 172.21.15.31:0/3784042737 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960009510 con 0x7f096815eb50 2023-11-26T07:17:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f096f98d640 1 -- 172.21.15.31:0/3784042737 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096813d060 con 0x7f096815eb50 2023-11-26T07:17:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.201+0000 7f095dffb640 1 -- 172.21.15.31:0/3784042737 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09600b8470 con 0x7f096815eb50 2023-11-26T07:17:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.205+0000 7f096f98d640 1 -- 172.21.15.31:0/3784042737 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096813d620 con 0x7f096815eb50 2023-11-26T07:17:43.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.205+0000 7f095dffb640 1 -- 172.21.15.31:0/3784042737 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600b87e0 con 0x7f096815eb50 2023-11-26T07:17:43.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.205+0000 7f096f98d640 1 -- 172.21.15.31:0/3784042737 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096813d810 con 0x7f096815eb50 2023-11-26T07:17:43.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.205+0000 7f095dffb640 1 -- 172.21.15.31:0/3784042737 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f09600b8940 con 0x7f096815eb50 2023-11-26T07:17:43.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.205+0000 7f095dffb640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f094c066470 0x7f094c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.205+0000 7f095dffb640 1 -- 172.21.15.31:0/3784042737 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f096011ee30 con 0x7f096815eb50 2023-11-26T07:17:43.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.205+0000 7f096d729640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f094c066470 0x7f094c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.205+0000 7f096d729640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f094c066470 0x7f094c068930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0964011310 tx=0x7f0964009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.209+0000 7f095dffb640 1 -- 172.21.15.31:0/3784042737 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f09600ec3f0 con 0x7f096815eb50 2023-11-26T07:17:43.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.369+0000 7f096f98d640 1 -- 172.21.15.31:0/3784042737 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f096804e850 con 0x7f096815eb50 2023-11-26T07:17:43.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.369+0000 7f095dffb640 1 -- 172.21.15.31:0/3784042737 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09600efa10 con 0x7f096815eb50 2023-11-26T07:17:43.375 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:17:43.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 -- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f094c066470 msgr2=0x7f094c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f094c066470 0x7f094c068930 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0964011310 tx=0x7f0964009190 comp rx=0 tx=0).stop 2023-11-26T07:17:43.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 -- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f096815eb50 msgr2=0x7f096815f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f096815eb50 0x7f096815f460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f096806b400 tx=0x7f0960019740 comp rx=0 tx=0).stop 2023-11-26T07:17:43.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 -- 172.21.15.31:0/3784042737 shutdown_connections 2023-11-26T07:17:43.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f094c066470 0x7f094c068930 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f096815eb50 0x7f096815f460 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968146820 0x7f096813db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 --2- 172.21.15.31:0/3784042737 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680b4210 0x7f096813c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 -- 172.21.15.31:0/3784042737 >> 172.21.15.31:0/3784042737 conn(0x7f09680b2f10 msgr2=0x7f0968070820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 -- 172.21.15.31:0/3784042737 shutdown_connections 2023-11-26T07:17:43.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.373+0000 7f096f98d640 1 -- 172.21.15.31:0/3784042737 wait complete. 2023-11-26T07:17:43.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:17:43.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e6e8e4640 1 Processor -- start 2023-11-26T07:17:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e6e8e4640 1 -- start start 2023-11-26T07:17:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e6e8e4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e6815d7c0 0x7f3e680d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e6e8e4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e680d04c0 0x7f3e680cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e6e8e4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e680d11a0 con 0x7f3e68161f20 2023-11-26T07:17:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e6e8e4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e680d0f90 con 0x7f3e6815d7c0 2023-11-26T07:17:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e6e8e4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e680d1480 con 0x7f3e680d04c0 2023-11-26T07:17:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e67fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e6815d7c0 0x7f3e680d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e67fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e6815d7c0 0x7f3e680d3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59774/0 (socket says 172.21.15.31:59774) 2023-11-26T07:17:43.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e677fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3e68161f20 0x7f3e680cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46382/0 (socket says 172.21.15.31:46382) 2023-11-26T07:17:43.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.537+0000 7f3e67fff640 1 -- 172.21.15.31:0/886611324 learned_addr learned my addr 172.21.15.31:0/886611324 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:43.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.541+0000 7f3e6ce81640 1 --2- 172.21.15.31:0/886611324 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e680d04c0 0x7f3e680cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.541+0000 7f3e67fff640 1 -- 172.21.15.31:0/886611324 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e680d04c0 msgr2=0x7f3e680cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:43.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.541+0000 7f3e67fff640 1 --2- 172.21.15.31:0/886611324 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e680d04c0 0x7f3e680cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.541+0000 7f3e67fff640 1 -- 172.21.15.31:0/886611324 >> v1:172.21.15.31:6789/0 conn(0x7f3e68161f20 legacy=0x7f3e680cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.541+0000 7f3e67fff640 1 -- 172.21.15.31:0/886611324 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e680d1760 con 0x7f3e6815d7c0 2023-11-26T07:17:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.541+0000 7f3e66ffd640 1 -- 172.21.15.31:0/886611324 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110663550 0 0) 0x7f3e680d11a0 con 0x7f3e68161f20 2023-11-26T07:17:43.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.541+0000 7f3e67fff640 1 --2- 172.21.15.31:0/886611324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 0x7f3e680d3d50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009870 tx=0x7f3e5801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee05ce8bc698b75 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.541+0000 7f3e66ffd640 1 -- 172.21.15.31:0/886611324 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e580aa670 con 0x7f3e6815d7c0 2023-11-26T07:17:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.541+0000 7f3e66ffd640 1 -- 172.21.15.31:0/886611324 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e580aa7d0 con 0x7f3e6815d7c0 2023-11-26T07:17:43.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.541+0000 7f3e66ffd640 1 -- 172.21.15.31:0/886611324 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e580aab60 con 0x7f3e6815d7c0 2023-11-26T07:17:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/886611324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 msgr2=0x7f3e680d3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 --2- 172.21.15.31:0/886611324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 0x7f3e680d3d50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009870 tx=0x7f3e5801c510 comp rx=0 tx=0).stop 2023-11-26T07:17:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/886611324 shutdown_connections 2023-11-26T07:17:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 --2- 172.21.15.31:0/886611324 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e680d04c0 0x7f3e680cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 --2- 172.21.15.31:0/886611324 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 0x7f3e680d3d50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/886611324 >> 172.21.15.31:0/886611324 conn(0x7f3e6808afc0 msgr2=0x7f3e680d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/886611324 shutdown_connections 2023-11-26T07:17:43.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/886611324 wait complete. 2023-11-26T07:17:43.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 Processor -- start 2023-11-26T07:17:43.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 -- start start 2023-11-26T07:17:43.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e680d04c0 0x7f3e680b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 0x7f3e681569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e680b8b40 0x7f3e68155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e680b02a0 con 0x7f3e6815d7c0 2023-11-26T07:17:43.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e680b7220 con 0x7f3e680b8b40 2023-11-26T07:17:43.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6e8e4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e680b7500 con 0x7f3e680d04c0 2023-11-26T07:17:43.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 0x7f3e681569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 0x7f3e681569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59782/0 (socket says 172.21.15.31:59782) 2023-11-26T07:17:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e677fe640 1 -- 172.21.15.31:0/1286671407 learned_addr learned my addr 172.21.15.31:0/1286671407 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e67fff640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e680d04c0 0x7f3e680b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e6ce81640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e680b8b40 0x7f3e68155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e677fe640 1 -- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e680d04c0 msgr2=0x7f3e680b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e677fe640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e680d04c0 0x7f3e680b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.545+0000 7f3e677fe640 1 -- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e680b8b40 msgr2=0x7f3e68155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e677fe640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e680b8b40 0x7f3e68155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e677fe640 1 -- 172.21.15.31:0/1286671407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e580094d0 con 0x7f3e6815d7c0 2023-11-26T07:17:43.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e677fe640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 0x7f3e681569a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00c410 tx=0x7f3e5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e64ff9640 1 -- 172.21.15.31:0/1286671407 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c001110 con 0x7f3e6815d7c0 2023-11-26T07:17:43.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e64ff9640 1 -- 172.21.15.31:0/1286671407 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c007500 con 0x7f3e6815d7c0 2023-11-26T07:17:43.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e64ff9640 1 -- 172.21.15.31:0/1286671407 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0059f0 con 0x7f3e6815d7c0 2023-11-26T07:17:43.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/1286671407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e680b77e0 con 0x7f3e6815d7c0 2023-11-26T07:17:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/1286671407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e680b15b0 con 0x7f3e6815d7c0 2023-11-26T07:17:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/1286671407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e680b7d00 con 0x7f3e6815d7c0 2023-11-26T07:17:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e64ff9640 1 -- 172.21.15.31:0/1286671407 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f3e5c005b50 con 0x7f3e6815d7c0 2023-11-26T07:17:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e64ff9640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e540664a0 0x7f3e54068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e64ff9640 1 -- 172.21.15.31:0/1286671407 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e5c0833b0 con 0x7f3e6815d7c0 2023-11-26T07:17:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.549+0000 7f3e67fff640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e540664a0 0x7f3e54068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.553+0000 7f3e67fff640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e540664a0 0x7f3e54068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e580ae000 tx=0x7f3e580b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.553+0000 7f3e64ff9640 1 -- 172.21.15.31:0/1286671407 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e680b7d00 con 0x7f3e6815d7c0 2023-11-26T07:17:43.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/1286671407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e680b7d00 con 0x7f3e6815d7c0 2023-11-26T07:17:43.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e64ff9640 1 -- 172.21.15.31:0/1286671407 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e5c017020 con 0x7f3e6815d7c0 2023-11-26T07:17:43.716 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:17:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e540664a0 msgr2=0x7f3e54068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e540664a0 0x7f3e54068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e580ae000 tx=0x7f3e580b5040 comp rx=0 tx=0).stop 2023-11-26T07:17:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/1286671407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 msgr2=0x7f3e681569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 0x7f3e681569a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00c410 tx=0x7f3e5c00c8d0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/1286671407 shutdown_connections 2023-11-26T07:17:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e540664a0 0x7f3e54068960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e680b8b40 0x7f3e68155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815d7c0 0x7f3e681569a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 --2- 172.21.15.31:0/1286671407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e680d04c0 0x7f3e680b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/1286671407 >> 172.21.15.31:0/1286671407 conn(0x7f3e6808afc0 msgr2=0x7f3e680b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/1286671407 shutdown_connections 2023-11-26T07:17:43.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.713+0000 7f3e6e8e4640 1 -- 172.21.15.31:0/1286671407 wait complete. 2023-11-26T07:17:43.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:17:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.877+0000 7f3973580640 1 Processor -- start 2023-11-26T07:17:43.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3973580640 1 -- start start 2023-11-26T07:17:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3973580640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f396c0cc460 0x7f396c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3973580640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f396c0b2710 con 0x7f396c15a710 2023-11-26T07:17:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3973580640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f396c0b29f0 con 0x7f396c0cdfb0 2023-11-26T07:17:43.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3973580640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f396c0cca80 con 0x7f396c0cc460 2023-11-26T07:17:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3970b1b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f396c15a710 0x7f396c0cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54178/0 (socket says 172.21.15.31:54178) 2023-11-26T07:17:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3970b1b640 1 -- 172.21.15.31:0/2161032822 learned_addr learned my addr 172.21.15.31:0/2161032822 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3971b1d640 1 --2- 172.21.15.31:0/2161032822 >> v2:172.21.15.97:3300/0 conn(0x7f396c0cc460 0x7f396c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3963fff640 1 -- 172.21.15.31:0/2161032822 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439138111 0 0) 0x7f396c0b2710 con 0x7f396c15a710 2023-11-26T07:17:43.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3963fff640 1 -- 172.21.15.31:0/2161032822 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3958003530 con 0x7f396c15a710 2023-11-26T07:17:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3971b1d640 1 -- 172.21.15.31:0/2161032822 >> v1:172.21.15.97:6789/0 conn(0x7f396c0cdfb0 legacy=0x7f396c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3971b1d640 1 -- 172.21.15.31:0/2161032822 >> v1:172.21.15.31:6789/0 conn(0x7f396c15a710 legacy=0x7f396c0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3963fff640 1 -- 172.21.15.31:0/2161032822 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 522081416 0 0) 0x7f3958003530 con 0x7f396c15a710 2023-11-26T07:17:43.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.881+0000 7f3971b1d640 1 -- 172.21.15.31:0/2161032822 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c0b2cd0 con 0x7f396c0cc460 2023-11-26T07:17:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3971b1d640 1 --2- 172.21.15.31:0/2161032822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f396c0cc460 0x7f396c0abf90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3964009d90 tx=0x7f396401c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1273a64803853d64 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3963fff640 1 -- 172.21.15.31:0/2161032822 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395c001680 con 0x7f396c0cc460 2023-11-26T07:17:43.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3963fff640 1 -- 172.21.15.31:0/2161032822 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39640a56a0 con 0x7f396c0cc460 2023-11-26T07:17:43.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3963fff640 1 -- 172.21.15.31:0/2161032822 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39640a5a30 con 0x7f396c0cc460 2023-11-26T07:17:43.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3973580640 1 -- 172.21.15.31:0/2161032822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f396c0cc460 msgr2=0x7f396c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3973580640 1 --2- 172.21.15.31:0/2161032822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f396c0cc460 0x7f396c0abf90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3964009d90 tx=0x7f396401c840 comp rx=0 tx=0).stop 2023-11-26T07:17:43.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3973580640 1 -- 172.21.15.31:0/2161032822 shutdown_connections 2023-11-26T07:17:43.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3973580640 1 --2- 172.21.15.31:0/2161032822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f396c0cc460 0x7f396c0abf90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3973580640 1 -- 172.21.15.31:0/2161032822 >> 172.21.15.31:0/2161032822 conn(0x7f396c0b0cb0 msgr2=0x7f396c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:43.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3973580640 1 -- 172.21.15.31:0/2161032822 shutdown_connections 2023-11-26T07:17:43.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3973580640 1 -- 172.21.15.31:0/2161032822 wait complete. 2023-11-26T07:17:43.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.885+0000 7f3973580640 1 Processor -- start 2023-11-26T07:17:43.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3973580640 1 -- start start 2023-11-26T07:17:43.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3973580640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f396c0cc460 0x7f396c14e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3973580640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f396c0cdfb0 0x7f396c168640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3973580640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f396c15a710 0x7f396c0af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3973580640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c156ec0 con 0x7f396c0cc460 2023-11-26T07:17:43.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3973580640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c070f30 con 0x7f396c0cdfb0 2023-11-26T07:17:43.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3973580640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396c169b30 con 0x7f396c15a710 2023-11-26T07:17:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f397131c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f396c0cc460 0x7f396c14e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f397131c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f396c0cc460 0x7f396c14e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50046/0 (socket says 172.21.15.31:50046) 2023-11-26T07:17:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f397131c640 1 -- 172.21.15.31:0/2009319529 learned_addr learned my addr 172.21.15.31:0/2009319529 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3970b1b640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f396c0cdfb0 0x7f396c168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3971b1d640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f396c15a710 0x7f396c0af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f397131c640 1 -- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f396c15a710 msgr2=0x7f396c0af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f397131c640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f396c15a710 0x7f396c0af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f397131c640 1 -- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f396c0cdfb0 msgr2=0x7f396c168640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f397131c640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f396c0cdfb0 0x7f396c168640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f397131c640 1 -- 172.21.15.31:0/2009319529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3964009a70 con 0x7f396c0cc460 2023-11-26T07:17:43.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3971b1d640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f396c15a710 0x7f396c0af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3970b1b640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f396c0cdfb0 0x7f396c168640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f397131c640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f396c0cc460 0x7f396c14e1c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f396800c500 tx=0x7f396800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3961ffb640 1 -- 172.21.15.31:0/2009319529 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968005220 con 0x7f396c0cc460 2023-11-26T07:17:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3961ffb640 1 -- 172.21.15.31:0/2009319529 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3968005380 con 0x7f396c0cc460 2023-11-26T07:17:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3961ffb640 1 -- 172.21.15.31:0/2009319529 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968007270 con 0x7f396c0cc460 2023-11-26T07:17:43.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3973580640 1 -- 172.21.15.31:0/2009319529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396c168fb0 con 0x7f396c0cc460 2023-11-26T07:17:43.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3973580640 1 -- 172.21.15.31:0/2009319529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c169570 con 0x7f396c0cc460 2023-11-26T07:17:43.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.889+0000 7f3973580640 1 -- 172.21.15.31:0/2009319529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c0671f0 con 0x7f396c0cc460 2023-11-26T07:17:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.893+0000 7f3961ffb640 1 -- 172.21.15.31:0/2009319529 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f39680073d0 con 0x7f396c0cc460 2023-11-26T07:17:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.893+0000 7f3961ffb640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3950066470 0x7f3950068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.893+0000 7f3961ffb640 1 -- 172.21.15.31:0/2009319529 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3968083800 con 0x7f396c0cc460 2023-11-26T07:17:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.893+0000 7f3970b1b640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3950066470 0x7f3950068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:43.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.893+0000 7f3970b1b640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3950066470 0x7f3950068930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f396c072a80 tx=0x7f395c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:43.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:43.897+0000 7f3961ffb640 1 -- 172.21.15.31:0/2009319529 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3968050bf0 con 0x7f396c0cc460 2023-11-26T07:17:44.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.053+0000 7f3973580640 1 -- 172.21.15.31:0/2009319529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f396c06b460 con 0x7f396c0cc460 2023-11-26T07:17:44.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.053+0000 7f3961ffb640 1 -- 172.21.15.31:0/2009319529 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3968054210 con 0x7f396c0cc460 2023-11-26T07:17:44.057 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:17:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.053+0000 7f3973580640 1 -- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3950066470 msgr2=0x7f3950068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.053+0000 7f3973580640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3950066470 0x7f3950068930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f396c072a80 tx=0x7f395c009040 comp rx=0 tx=0).stop 2023-11-26T07:17:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.053+0000 7f3973580640 1 -- 172.21.15.31:0/2009319529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f396c0cc460 msgr2=0x7f396c14e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.053+0000 7f3973580640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f396c0cc460 0x7f396c14e1c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f396800c500 tx=0x7f396800c9c0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.057+0000 7f3973580640 1 -- 172.21.15.31:0/2009319529 shutdown_connections 2023-11-26T07:17:44.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.057+0000 7f3973580640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3950066470 0x7f3950068930 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.057+0000 7f3973580640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f396c15a710 0x7f396c0af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.057+0000 7f3973580640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f396c0cdfb0 0x7f396c168640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.057+0000 7f3973580640 1 --2- 172.21.15.31:0/2009319529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f396c0cc460 0x7f396c14e1c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.057+0000 7f3973580640 1 -- 172.21.15.31:0/2009319529 >> 172.21.15.31:0/2009319529 conn(0x7f396c0b0cb0 msgr2=0x7f396c165870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.057+0000 7f3973580640 1 -- 172.21.15.31:0/2009319529 shutdown_connections 2023-11-26T07:17:44.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.057+0000 7f3973580640 1 -- 172.21.15.31:0/2009319529 wait complete. 2023-11-26T07:17:44.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.217+0000 7fd35086a640 1 Processor -- start 2023-11-26T07:17:44.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.217+0000 7fd35086a640 1 -- start start 2023-11-26T07:17:44.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.217+0000 7fd35086a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd34c0d3760 0x7fd34c0d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.217+0000 7fd35086a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34c0d15f0 con 0x7fd34c0cfa20 2023-11-26T07:17:44.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.217+0000 7fd35086a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34c0ce000 con 0x7fd34c15d880 2023-11-26T07:17:44.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.217+0000 7fd35086a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34c0d4110 con 0x7fd34c0d3760 2023-11-26T07:17:44.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd343fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd34c0d3760 0x7fd34c0d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd34b59e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd34c0cfa20 0x7fd34c0cfe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54186/0 (socket says 172.21.15.31:54186) 2023-11-26T07:17:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd34b59e640 1 -- 172.21.15.31:0/3097657197 learned_addr learned my addr 172.21.15.31:0/3097657197 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd34a59c640 1 -- 172.21.15.31:0/3097657197 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511269852 0 0) 0x7fd34c0d15f0 con 0x7fd34c0cfa20 2023-11-26T07:17:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd34a59c640 1 -- 172.21.15.31:0/3097657197 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd338003530 con 0x7fd34c0cfa20 2023-11-26T07:17:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd343fff640 1 -- 172.21.15.31:0/3097657197 >> v1:172.21.15.97:6789/0 conn(0x7fd34c15d880 legacy=0x7fd34c0b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd343fff640 1 -- 172.21.15.31:0/3097657197 >> v1:172.21.15.31:6789/0 conn(0x7fd34c0cfa20 legacy=0x7fd34c0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd343fff640 1 -- 172.21.15.31:0/3097657197 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c0d0630 con 0x7fd34c0d3760 2023-11-26T07:17:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd343fff640 1 --2- 172.21.15.31:0/3097657197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0d3760 0x7fd34c0d3bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd3340097c0 tx=0x7fd33401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94575d529604950d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd34a59c640 1 -- 172.21.15.31:0/3097657197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3340ab680 con 0x7fd34c0d3760 2023-11-26T07:17:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd34a59c640 1 -- 172.21.15.31:0/3097657197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3340ab7e0 con 0x7fd34c0d3760 2023-11-26T07:17:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.221+0000 7fd34a59c640 1 -- 172.21.15.31:0/3097657197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3340abb70 con 0x7fd34c0d3760 2023-11-26T07:17:44.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 -- 172.21.15.31:0/3097657197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0d3760 msgr2=0x7fd34c0d3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 --2- 172.21.15.31:0/3097657197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0d3760 0x7fd34c0d3bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd3340097c0 tx=0x7fd33401c480 comp rx=0 tx=0).stop 2023-11-26T07:17:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 -- 172.21.15.31:0/3097657197 shutdown_connections 2023-11-26T07:17:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 --2- 172.21.15.31:0/3097657197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c0d3760 0x7fd34c0d3bd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 -- 172.21.15.31:0/3097657197 >> 172.21.15.31:0/3097657197 conn(0x7fd34c06b9f0 msgr2=0x7fd34c06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 -- 172.21.15.31:0/3097657197 shutdown_connections 2023-11-26T07:17:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 -- 172.21.15.31:0/3097657197 wait complete. 2023-11-26T07:17:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 Processor -- start 2023-11-26T07:17:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 -- start start 2023-11-26T07:17:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd34c0cfa20 0x7fd34c1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c0d3760 0x7fd34c152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c15d880 0x7fd34c158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c157360 con 0x7fd34c15d880 2023-11-26T07:17:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c0b3720 con 0x7fd34c0d3760 2023-11-26T07:17:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd35086a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c0b3a00 con 0x7fd34c0cfa20 2023-11-26T07:17:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd34b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c15d880 0x7fd34c158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd34b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c15d880 0x7fd34c158ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50072/0 (socket says 172.21.15.31:50072) 2023-11-26T07:17:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd34b59e640 1 -- 172.21.15.31:0/117829601 learned_addr learned my addr 172.21.15.31:0/117829601 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd34ad9d640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd34c0cfa20 0x7fd34c1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.225+0000 7fd343fff640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c0d3760 0x7fd34c152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd34b59e640 1 -- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd34c0cfa20 msgr2=0x7fd34c1557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd34b59e640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd34c0cfa20 0x7fd34c1557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd34b59e640 1 -- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c0d3760 msgr2=0x7fd34c152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd34b59e640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c0d3760 0x7fd34c152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd34b59e640 1 -- 172.21.15.31:0/117829601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd334009420 con 0x7fd34c15d880 2023-11-26T07:17:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd34b59e640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c15d880 0x7fd34c158ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd34400cbb0 tx=0x7fd34400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd3437fe640 1 -- 172.21.15.31:0/117829601 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34400ad40 con 0x7fd34c15d880 2023-11-26T07:17:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd3437fe640 1 -- 172.21.15.31:0/117829601 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd344012400 con 0x7fd34c15d880 2023-11-26T07:17:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd3437fe640 1 -- 172.21.15.31:0/117829601 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34400aea0 con 0x7fd34c15d880 2023-11-26T07:17:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd35086a640 1 -- 172.21.15.31:0/117829601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34c0b3ce0 con 0x7fd34c15d880 2023-11-26T07:17:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd35086a640 1 -- 172.21.15.31:0/117829601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c155e10 con 0x7fd34c15d880 2023-11-26T07:17:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd35086a640 1 -- 172.21.15.31:0/117829601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34c0b3580 con 0x7fd34c15d880 2023-11-26T07:17:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd3437fe640 1 -- 172.21.15.31:0/117829601 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd3440127b0 con 0x7fd34c15d880 2023-11-26T07:17:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd3437fe640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd32c066470 0x7fd32c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.229+0000 7fd3437fe640 1 -- 172.21.15.31:0/117829601 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd344083d30 con 0x7fd34c15d880 2023-11-26T07:17:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.233+0000 7fd34ad9d640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd32c066470 0x7fd32c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.233+0000 7fd34ad9d640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd32c066470 0x7fd32c068930 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0b34b0 tx=0x7fd33c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.233+0000 7fd3437fe640 1 -- 172.21.15.31:0/117829601 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd34c0b3580 con 0x7fd34c15d880 2023-11-26T07:17:44.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 -- 172.21.15.31:0/117829601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd34c0b3f20 con 0x7fd34c15d880 2023-11-26T07:17:44.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd3437fe640 1 -- 172.21.15.31:0/117829601 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7fd34c0b3f20 con 0x7fd34c15d880 2023-11-26T07:17:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:44.432 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4099326271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4099326271},{"type":"v1","addr":"172.21.15.31:6835","nonce":4099326271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2404164953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2404164953},{"type":"v1","addr":"172.21.15.97:6836","nonce":2404164953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1312969281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1312969281},{"type":"v1","addr":"172.21.15.31:6834","nonce":1312969281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:32.510772+0000","modified":"2023-11-26T07:17:35.514610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/527056216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":527056216},{"type":"v1","addr":"172.21.15.97:6837","nonce":527056216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:17:44.433 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-11-26T07:17:44.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 -- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd32c066470 msgr2=0x7fd32c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd32c066470 0x7fd32c068930 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0b34b0 tx=0x7fd33c009040 comp rx=0 tx=0).stop 2023-11-26T07:17:44.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 -- 172.21.15.31:0/117829601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c15d880 msgr2=0x7fd34c158ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c15d880 0x7fd34c158ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd34400cbb0 tx=0x7fd34400a430 comp rx=0 tx=0).stop 2023-11-26T07:17:44.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 -- 172.21.15.31:0/117829601 shutdown_connections 2023-11-26T07:17:44.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd32c066470 0x7fd32c068930 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd34c15d880 0x7fd34c158ab0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd34c0d3760 0x7fd34c152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 --2- 172.21.15.31:0/117829601 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd34c0cfa20 0x7fd34c1557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 -- 172.21.15.31:0/117829601 >> 172.21.15.31:0/117829601 conn(0x7fd34c06b9f0 msgr2=0x7fd34c06bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 -- 172.21.15.31:0/117829601 shutdown_connections 2023-11-26T07:17:44.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.429+0000 7fd35086a640 1 -- 172.21.15.31:0/117829601 wait complete. 2023-11-26T07:17:44.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:44.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b4617640 1 Processor -- start 2023-11-26T07:17:44.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b4617640 1 -- start start 2023-11-26T07:17:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b4617640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f01ac144c80 0x7f01ac0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b4617640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ac070df0 0x7f01ac06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b4617640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ac145060 con 0x7f01ac0ac460 2023-11-26T07:17:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b4617640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ac071fd0 con 0x7f01ac144c80 2023-11-26T07:17:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b4617640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0ac8c0 con 0x7f01ac070df0 2023-11-26T07:17:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b1bb2640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f01ac0ac460 0x7f01ac0706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54194/0 (socket says 172.21.15.31:54194) 2023-11-26T07:17:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b1bb2640 1 -- 172.21.15.31:0/2133682804 learned_addr learned my addr 172.21.15.31:0/2133682804 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b2bb4640 1 --2- 172.21.15.31:0/2133682804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ac070df0 0x7f01ac06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.593+0000 7f01b23b3640 1 --2- 172.21.15.31:0/2133682804 >> v2:172.21.15.97:3300/0 conn(0x7f01ac144c80 0x7f01ac0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.597+0000 7f01b13b1640 1 -- 172.21.15.31:0/2133682804 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533712533 0 0) 0x7f01ac145060 con 0x7f01ac0ac460 2023-11-26T07:17:44.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.597+0000 7f01b13b1640 1 -- 172.21.15.31:0/2133682804 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0198003530 con 0x7f01ac0ac460 2023-11-26T07:17:44.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.597+0000 7f01b23b3640 1 -- 172.21.15.31:0/2133682804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ac070df0 msgr2=0x7f01ac06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:44.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.597+0000 7f01b23b3640 1 --2- 172.21.15.31:0/2133682804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ac070df0 0x7f01ac06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.597+0000 7f01b23b3640 1 -- 172.21.15.31:0/2133682804 >> v1:172.21.15.31:6789/0 conn(0x7f01ac0ac460 legacy=0x7f01ac0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.597+0000 7f01b23b3640 1 -- 172.21.15.31:0/2133682804 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac071450 con 0x7f01ac144c80 2023-11-26T07:17:44.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.597+0000 7f01b23b3640 1 --2- 172.21.15.31:0/2133682804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ac144c80 0x7f01ac0abf20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01a80097c0 tx=0x7f01a801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5366b8a7acbc36 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.597+0000 7f01b13b1640 1 -- 172.21.15.31:0/2133682804 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a80ab600 con 0x7f01ac144c80 2023-11-26T07:17:44.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.597+0000 7f01b13b1640 1 -- 172.21.15.31:0/2133682804 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a80a7050 con 0x7f01ac144c80 2023-11-26T07:17:44.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.597+0000 7f01b13b1640 1 -- 172.21.15.31:0/2133682804 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a80ab960 con 0x7f01ac144c80 2023-11-26T07:17:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 -- 172.21.15.31:0/2133682804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ac144c80 msgr2=0x7f01ac0abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 --2- 172.21.15.31:0/2133682804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ac144c80 0x7f01ac0abf20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01a80097c0 tx=0x7f01a801c480 comp rx=0 tx=0).stop 2023-11-26T07:17:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 -- 172.21.15.31:0/2133682804 shutdown_connections 2023-11-26T07:17:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 --2- 172.21.15.31:0/2133682804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ac070df0 0x7f01ac06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 --2- 172.21.15.31:0/2133682804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ac144c80 0x7f01ac0abf20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 -- 172.21.15.31:0/2133682804 >> 172.21.15.31:0/2133682804 conn(0x7f01ac0b0c40 msgr2=0x7f01ac0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 -- 172.21.15.31:0/2133682804 shutdown_connections 2023-11-26T07:17:44.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 -- 172.21.15.31:0/2133682804 wait complete. 2023-11-26T07:17:44.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 Processor -- start 2023-11-26T07:17:44.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 -- start start 2023-11-26T07:17:44.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ac070df0 0x7f01ac13ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ac0ac460 0x7f01ac14db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac144c80 0x7f01ac0aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac14e0a0 con 0x7f01ac144c80 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac0b1d30 con 0x7f01ac0ac460 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b4617640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac15b810 con 0x7f01ac070df0 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b2bb4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac144c80 0x7f01ac0aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b2bb4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac144c80 0x7f01ac0aefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50078/0 (socket says 172.21.15.31:50078) 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.601+0000 7f01b2bb4640 1 -- 172.21.15.31:0/2449186463 learned_addr learned my addr 172.21.15.31:0/2449186463 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b23b3640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ac070df0 0x7f01ac13ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b1bb2640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ac0ac460 0x7f01ac14db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b2bb4640 1 -- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ac070df0 msgr2=0x7f01ac13ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b2bb4640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ac070df0 0x7f01ac13ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b2bb4640 1 -- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ac0ac460 msgr2=0x7f01ac14db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b2bb4640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ac0ac460 0x7f01ac14db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b2bb4640 1 -- 172.21.15.31:0/2449186463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a8009420 con 0x7f01ac144c80 2023-11-26T07:17:44.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b2bb4640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac144c80 0x7f01ac0aefd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01a400efc0 tx=0x7f01a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01a2ffd640 1 -- 172.21.15.31:0/2449186463 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4014480 con 0x7f01ac144c80 2023-11-26T07:17:44.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01a2ffd640 1 -- 172.21.15.31:0/2449186463 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a400f040 con 0x7f01ac144c80 2023-11-26T07:17:44.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01a2ffd640 1 -- 172.21.15.31:0/2449186463 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4014820 con 0x7f01ac144c80 2023-11-26T07:17:44.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b4617640 1 -- 172.21.15.31:0/2449186463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac13f3f0 con 0x7f01ac144c80 2023-11-26T07:17:44.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b4617640 1 -- 172.21.15.31:0/2449186463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac09c4c0 con 0x7f01ac144c80 2023-11-26T07:17:44.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01b4617640 1 -- 172.21.15.31:0/2449186463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ac0b1ba0 con 0x7f01ac144c80 2023-11-26T07:17:44.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.605+0000 7f01a2ffd640 1 -- 172.21.15.31:0/2449186463 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f01a4014980 con 0x7f01ac144c80 2023-11-26T07:17:44.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.609+0000 7f01a2ffd640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0190066470 0x7f0190068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.609+0000 7f01a2ffd640 1 -- 172.21.15.31:0/2449186463 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01a4084a40 con 0x7f01ac144c80 2023-11-26T07:17:44.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.609+0000 7f01b23b3640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0190066470 0x7f0190068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.609+0000 7f01b23b3640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0190066470 0x7f0190068930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f01a80ad000 tx=0x7f01a80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.613+0000 7f01a2ffd640 1 -- 172.21.15.31:0/2449186463 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f01a4086e60 con 0x7f01ac144c80 2023-11-26T07:17:44.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.773+0000 7f01b4617640 1 -- 172.21.15.31:0/2449186463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01ac04e850 con 0x7f01ac144c80 2023-11-26T07:17:44.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.773+0000 7f01a2ffd640 1 -- 172.21.15.31:0/2449186463 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f01a4018090 con 0x7f01ac144c80 2023-11-26T07:17:44.779 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:44.779 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 -- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0190066470 msgr2=0x7f0190068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0190066470 0x7f0190068930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f01a80ad000 tx=0x7f01a80a6000 comp rx=0 tx=0).stop 2023-11-26T07:17:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 -- 172.21.15.31:0/2449186463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac144c80 msgr2=0x7f01ac0aefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac144c80 0x7f01ac0aefd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01a400efc0 tx=0x7f01a400c490 comp rx=0 tx=0).stop 2023-11-26T07:17:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 -- 172.21.15.31:0/2449186463 shutdown_connections 2023-11-26T07:17:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0190066470 0x7f0190068930 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ac144c80 0x7f01ac0aefd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ac0ac460 0x7f01ac14db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 --2- 172.21.15.31:0/2449186463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ac070df0 0x7f01ac13ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 -- 172.21.15.31:0/2449186463 >> 172.21.15.31:0/2449186463 conn(0x7f01ac0b0c40 msgr2=0x7f01ac15ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 -- 172.21.15.31:0/2449186463 shutdown_connections 2023-11-26T07:17:44.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.777+0000 7f01b4617640 1 -- 172.21.15.31:0/2449186463 wait complete. 2023-11-26T07:17:44.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:44.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:44.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.941+0000 7fcc73dc4640 1 Processor -- start 2023-11-26T07:17:44.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.941+0000 7fcc73dc4640 1 -- start start 2023-11-26T07:17:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.941+0000 7fcc73dc4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c0cdfb0 0x7fcc6c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc73dc4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6c0b2710 con 0x7fcc6c0cbd50 2023-11-26T07:17:44.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc73dc4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6c0b29f0 con 0x7fcc6c15a710 2023-11-26T07:17:44.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc73dc4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0cca80 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc72361640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcc6c0cbd50 0x7fcc6c0cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54200/0 (socket says 172.21.15.31:54200) 2023-11-26T07:17:44.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc72361640 1 -- 172.21.15.31:0/4183459788 learned_addr learned my addr 172.21.15.31:0/4183459788 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc71b60640 1 --2- 172.21.15.31:0/4183459788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c0cdfb0 0x7fcc6c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc70b5e640 1 -- 172.21.15.31:0/4183459788 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682131488 0 0) 0x7fcc6c0b2710 con 0x7fcc6c0cbd50 2023-11-26T07:17:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc70b5e640 1 -- 172.21.15.31:0/4183459788 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc58003530 con 0x7fcc6c0cbd50 2023-11-26T07:17:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc70b5e640 1 -- 172.21.15.31:0/4183459788 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477230176 0 0) 0x7fcc6c0b29f0 con 0x7fcc6c15a710 2023-11-26T07:17:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc70b5e640 1 -- 172.21.15.31:0/4183459788 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc6c0b2710 con 0x7fcc6c15a710 2023-11-26T07:17:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc71b60640 1 -- 172.21.15.31:0/4183459788 >> v1:172.21.15.97:6789/0 conn(0x7fcc6c15a710 legacy=0x7fcc6c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc71b60640 1 -- 172.21.15.31:0/4183459788 >> v1:172.21.15.31:6789/0 conn(0x7fcc6c0cbd50 legacy=0x7fcc6c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.945+0000 7fcc71b60640 1 -- 172.21.15.31:0/4183459788 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c0b2cd0 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc71b60640 1 --2- 172.21.15.31:0/4183459788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c0cdfb0 0x7fcc6c0abf90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc680097f0 tx=0x7fcc6801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f878a7eb90d1dd94 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc70b5e640 1 -- 172.21.15.31:0/4183459788 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc680aa610 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc70b5e640 1 -- 172.21.15.31:0/4183459788 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc680a6040 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc70b5e640 1 -- 172.21.15.31:0/4183459788 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc680aa970 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 -- 172.21.15.31:0/4183459788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c0cdfb0 msgr2=0x7fcc6c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 --2- 172.21.15.31:0/4183459788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c0cdfb0 0x7fcc6c0abf90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcc680097f0 tx=0x7fcc6801c490 comp rx=0 tx=0).stop 2023-11-26T07:17:44.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 -- 172.21.15.31:0/4183459788 shutdown_connections 2023-11-26T07:17:44.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 --2- 172.21.15.31:0/4183459788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c0cdfb0 0x7fcc6c0abf90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 -- 172.21.15.31:0/4183459788 >> 172.21.15.31:0/4183459788 conn(0x7fcc6c0b0cb0 msgr2=0x7fcc6c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:44.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 -- 172.21.15.31:0/4183459788 shutdown_connections 2023-11-26T07:17:44.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 -- 172.21.15.31:0/4183459788 wait complete. 2023-11-26T07:17:44.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 Processor -- start 2023-11-26T07:17:44.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 -- start start 2023-11-26T07:17:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0cbd50 0x7fcc6c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc6c0cdfb0 0x7fcc6c150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c15a710 0x7fcc6c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c157e40 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c1589a0 con 0x7fcc6c0cbd50 2023-11-26T07:17:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc73dc4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c158070 con 0x7fcc6c15a710 2023-11-26T07:17:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.949+0000 7fcc7135f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc6c0cdfb0 0x7fcc6c150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc7135f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc6c0cdfb0 0x7fcc6c150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50090/0 (socket says 172.21.15.31:50090) 2023-11-26T07:17:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc7135f640 1 -- 172.21.15.31:0/1433715001 learned_addr learned my addr 172.21.15.31:0/1433715001 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:44.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc71b60640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0cbd50 0x7fcc6c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc72361640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c15a710 0x7fcc6c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc7135f640 1 -- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c15a710 msgr2=0x7fcc6c154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc7135f640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c15a710 0x7fcc6c154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc7135f640 1 -- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0cbd50 msgr2=0x7fcc6c15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:44.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc7135f640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0cbd50 0x7fcc6c15b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:44.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc7135f640 1 -- 172.21.15.31:0/1433715001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68009420 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc7135f640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc6c0cdfb0 0x7fcc6c150700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c430 tx=0x7fcc5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc627fc640 1 -- 172.21.15.31:0/1433715001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c0052d0 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc627fc640 1 -- 172.21.15.31:0/1433715001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c005450 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc73dc4640 1 -- 172.21.15.31:0/1433715001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c158350 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc627fc640 1 -- 172.21.15.31:0/1433715001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c003840 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc73dc4640 1 -- 172.21.15.31:0/1433715001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c150c40 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc73dc4640 1 -- 172.21.15.31:0/1433715001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c0671f0 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.953+0000 7fcc627fc640 1 -- 172.21.15.31:0/1433715001 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fcc5c0039a0 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.957+0000 7fcc627fc640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcc500664a0 0x7fcc50068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:44.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.957+0000 7fcc627fc640 1 -- 172.21.15.31:0/1433715001 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc5c0834d0 con 0x7fcc6c0cdfb0 2023-11-26T07:17:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.957+0000 7fcc71b60640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcc500664a0 0x7fcc50068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:44.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.957+0000 7fcc71b60640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcc500664a0 0x7fcc50068960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc680ae000 tx=0x7fcc680a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:44.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:44.957+0000 7fcc627fc640 1 -- 172.21.15.31:0/1433715001 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcc5c0508b0 con 0x7fcc6c0cdfb0 2023-11-26T07:17:45.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.117+0000 7fcc73dc4640 1 -- 172.21.15.31:0/1433715001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc6c06b460 con 0x7fcc6c0cdfb0 2023-11-26T07:17:45.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.117+0000 7fcc627fc640 1 -- 172.21.15.31:0/1433715001 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc5c053ed0 con 0x7fcc6c0cdfb0 2023-11-26T07:17:45.122 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:17:45.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 -- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcc500664a0 msgr2=0x7fcc50068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcc500664a0 0x7fcc50068960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc680ae000 tx=0x7fcc680a7000 comp rx=0 tx=0).stop 2023-11-26T07:17:45.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 -- 172.21.15.31:0/1433715001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc6c0cdfb0 msgr2=0x7fcc6c150700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc6c0cdfb0 0x7fcc6c150700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c430 tx=0x7fcc5c00c8f0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 -- 172.21.15.31:0/1433715001 shutdown_connections 2023-11-26T07:17:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcc500664a0 0x7fcc50068960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcc6c15a710 0x7fcc6c154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc6c0cdfb0 0x7fcc6c150700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 --2- 172.21.15.31:0/1433715001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc6c0cbd50 0x7fcc6c15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 -- 172.21.15.31:0/1433715001 >> 172.21.15.31:0/1433715001 conn(0x7fcc6c0b0cb0 msgr2=0x7fcc6c0acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 -- 172.21.15.31:0/1433715001 shutdown_connections 2023-11-26T07:17:45.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.121+0000 7fcc73dc4640 1 -- 172.21.15.31:0/1433715001 wait complete. 2023-11-26T07:17:45.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:17:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.285+0000 7f6b140d6640 1 Processor -- start 2023-11-26T07:17:45.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.285+0000 7f6b140d6640 1 -- start start 2023-11-26T07:17:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.285+0000 7f6b140d6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c0b4210 0x7f6b0c0b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.285+0000 7f6b140d6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b0c15db40 0x7f6b0c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.285+0000 7f6b140d6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c147190 con 0x7f6b0c146820 2023-11-26T07:17:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.285+0000 7f6b140d6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b0c146c00 con 0x7f6b0c15db40 2023-11-26T07:17:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.285+0000 7f6b140d6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c06bf90 con 0x7f6b0c0b4210 2023-11-26T07:17:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b11671640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c0b4210 0x7f6b0c0b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b11e72640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6b0c146820 0x7f6b0c070a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:37102/0 (socket says 172.21.15.31:37102) 2023-11-26T07:17:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b11e72640 1 -- 172.21.15.31:0/1363513808 learned_addr learned my addr 172.21.15.31:0/1363513808 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b12673640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b0c15db40 0x7f6b0c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b11671640 1 -- 172.21.15.31:0/1363513808 >> v2:172.21.15.97:3300/0 conn(0x7f6b0c15db40 msgr2=0x7f6b0c15df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b11671640 1 --2- 172.21.15.31:0/1363513808 >> v2:172.21.15.97:3300/0 conn(0x7f6b0c15db40 0x7f6b0c15df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b11671640 1 -- 172.21.15.31:0/1363513808 >> v1:172.21.15.97:6789/0 conn(0x7f6b0c146820 legacy=0x7f6b0c070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b11671640 1 -- 172.21.15.31:0/1363513808 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c0b4bb0 con 0x7f6b0c0b4210 2023-11-26T07:17:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b11671640 1 --2- 172.21.15.31:0/1363513808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c0b4210 0x7f6b0c0b4670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6afc0097c0 tx=0x7f6afc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21943e6e910a03e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b10e70640 1 -- 172.21.15.31:0/1363513808 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0ab650 con 0x7f6b0c0b4210 2023-11-26T07:17:45.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b10e70640 1 -- 172.21.15.31:0/1363513808 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc0ab7b0 con 0x7f6b0c0b4210 2023-11-26T07:17:45.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.289+0000 7f6b10e70640 1 -- 172.21.15.31:0/1363513808 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0abb40 con 0x7f6b0c0b4210 2023-11-26T07:17:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 -- 172.21.15.31:0/1363513808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c0b4210 msgr2=0x7f6b0c0b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 --2- 172.21.15.31:0/1363513808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c0b4210 0x7f6b0c0b4670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6afc0097c0 tx=0x7f6afc01c450 comp rx=0 tx=0).stop 2023-11-26T07:17:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 -- 172.21.15.31:0/1363513808 shutdown_connections 2023-11-26T07:17:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 --2- 172.21.15.31:0/1363513808 >> v2:172.21.15.97:3300/0 conn(0x7f6b0c15db40 0x7f6b0c15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 --2- 172.21.15.31:0/1363513808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c0b4210 0x7f6b0c0b4670 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 -- 172.21.15.31:0/1363513808 >> 172.21.15.31:0/1363513808 conn(0x7f6b0c0b2f10 msgr2=0x7f6b0c0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 -- 172.21.15.31:0/1363513808 shutdown_connections 2023-11-26T07:17:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 -- 172.21.15.31:0/1363513808 wait complete. 2023-11-26T07:17:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 Processor -- start 2023-11-26T07:17:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 -- start start 2023-11-26T07:17:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c0b4210 0x7f6b0c0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c146820 0x7f6b0c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0c15db40 0x7f6b0c0cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c144a40 con 0x7f6b0c0b4210 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c1f8670 con 0x7f6b0c15db40 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b140d6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c1f7a60 con 0x7f6b0c146820 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b11e72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c0b4210 0x7f6b0c0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b11e72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c0b4210 0x7f6b0c0cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50098/0 (socket says 172.21.15.31:50098) 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b11e72640 1 -- 172.21.15.31:0/4031120686 learned_addr learned my addr 172.21.15.31:0/4031120686 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b11671640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c146820 0x7f6b0c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.293+0000 7f6b12673640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0c15db40 0x7f6b0c0cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b11e72640 1 -- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c146820 msgr2=0x7f6b0c15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b11e72640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c146820 0x7f6b0c15c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b11e72640 1 -- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0c15db40 msgr2=0x7f6b0c0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b11e72640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0c15db40 0x7f6b0c0cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b11e72640 1 -- 172.21.15.31:0/4031120686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc009420 con 0x7f6b0c0b4210 2023-11-26T07:17:45.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b11e72640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c0b4210 0x7f6b0c0cc5a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c530 tx=0x7f6b0800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b027fc640 1 -- 172.21.15.31:0/4031120686 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08005cf0 con 0x7f6b0c0b4210 2023-11-26T07:17:45.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b027fc640 1 -- 172.21.15.31:0/4031120686 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08005e50 con 0x7f6b0c0b4210 2023-11-26T07:17:45.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b027fc640 1 -- 172.21.15.31:0/4031120686 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08007270 con 0x7f6b0c0b4210 2023-11-26T07:17:45.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b140d6640 1 -- 172.21.15.31:0/4031120686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c06d030 con 0x7f6b0c0b4210 2023-11-26T07:17:45.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b140d6640 1 -- 172.21.15.31:0/4031120686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c1f8020 con 0x7f6b0c0b4210 2023-11-26T07:17:45.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b140d6640 1 -- 172.21.15.31:0/4031120686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0c067790 con 0x7f6b0c0b4210 2023-11-26T07:17:45.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b027fc640 1 -- 172.21.15.31:0/4031120686 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6b080073d0 con 0x7f6b0c0b4210 2023-11-26T07:17:45.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b027fc640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6af0066470 0x7f6af0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b027fc640 1 -- 172.21.15.31:0/4031120686 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b08083320 con 0x7f6b0c0b4210 2023-11-26T07:17:45.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.297+0000 7f6b11671640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6af0066470 0x7f6af0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.301+0000 7f6b11671640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6af0066470 0x7f6af0068930 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6afc0af000 tx=0x7f6afc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.301+0000 7f6b027fc640 1 -- 172.21.15.31:0/4031120686 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b080507b0 con 0x7f6b0c0b4210 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.461+0000 7f6b140d6640 1 -- 172.21.15.31:0/4031120686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b0c04e850 con 0x7f6b0c0b4210 2023-11-26T07:17:45.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.461+0000 7f6b027fc640 1 -- 172.21.15.31:0/4031120686 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b08053dd0 con 0x7f6b0c0b4210 2023-11-26T07:17:45.466 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:17:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 -- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6af0066470 msgr2=0x7f6af0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6af0066470 0x7f6af0068930 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6afc0af000 tx=0x7f6afc0a7000 comp rx=0 tx=0).stop 2023-11-26T07:17:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 -- 172.21.15.31:0/4031120686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c0b4210 msgr2=0x7f6b0c0cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c0b4210 0x7f6b0c0cc5a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800c530 tx=0x7f6b0800c9f0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 -- 172.21.15.31:0/4031120686 shutdown_connections 2023-11-26T07:17:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6af0066470 0x7f6af0068930 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b0c15db40 0x7f6b0c0cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b0c146820 0x7f6b0c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 --2- 172.21.15.31:0/4031120686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b0c0b4210 0x7f6b0c0cc5a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 -- 172.21.15.31:0/4031120686 >> 172.21.15.31:0/4031120686 conn(0x7f6b0c0b2f10 msgr2=0x7f6b0c140f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:45.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 -- 172.21.15.31:0/4031120686 shutdown_connections 2023-11-26T07:17:45.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.465+0000 7f6b140d6640 1 -- 172.21.15.31:0/4031120686 wait complete. 2023-11-26T07:17:45.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:17:45.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7918f1d640 1 Processor -- start 2023-11-26T07:17:45.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7918f1d640 1 -- start start 2023-11-26T07:17:45.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7918f1d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 0x7f79140abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7918f1d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79140b2710 con 0x7f791415a710 2023-11-26T07:17:45.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7918f1d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79140b29f0 con 0x7f79140cdfb0 2023-11-26T07:17:45.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7918f1d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79140cca80 con 0x7f79140cc460 2023-11-26T07:17:45.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f790bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f791415a710 0x7f79140cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54206/0 (socket says 172.21.15.31:54206) 2023-11-26T07:17:45.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f790bfff640 1 -- 172.21.15.31:0/1028303812 learned_addr learned my addr 172.21.15.31:0/1028303812 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:45.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7912d9d640 1 --2- 172.21.15.31:0/1028303812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 0x7f79140abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7911d9b640 1 -- 172.21.15.31:0/1028303812 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194075652 0 0) 0x7f79140b2710 con 0x7f791415a710 2023-11-26T07:17:45.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7911d9b640 1 -- 172.21.15.31:0/1028303812 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7900003530 con 0x7f791415a710 2023-11-26T07:17:45.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7912d9d640 1 -- 172.21.15.31:0/1028303812 >> v1:172.21.15.97:6789/0 conn(0x7f79140cdfb0 legacy=0x7f79140ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7911d9b640 1 -- 172.21.15.31:0/1028303812 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687553385 0 0) 0x7f79140b29f0 con 0x7f79140cdfb0 2023-11-26T07:17:45.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7912d9d640 1 -- 172.21.15.31:0/1028303812 >> v1:172.21.15.31:6789/0 conn(0x7f791415a710 legacy=0x7f79140cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.629+0000 7f7912d9d640 1 -- 172.21.15.31:0/1028303812 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79140b2cd0 con 0x7f79140cc460 2023-11-26T07:17:45.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7912d9d640 1 --2- 172.21.15.31:0/1028303812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 0x7f79140abf90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f790c009dc0 tx=0x7f790c01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83eeba524c5c2493 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7911d9b640 1 -- 172.21.15.31:0/1028303812 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c0a5a20 con 0x7f79140cc460 2023-11-26T07:17:45.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7911d9b640 1 -- 172.21.15.31:0/1028303812 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f790c0a5b80 con 0x7f79140cc460 2023-11-26T07:17:45.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7911d9b640 1 -- 172.21.15.31:0/1028303812 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790c0ad440 con 0x7f79140cc460 2023-11-26T07:17:45.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7918f1d640 1 -- 172.21.15.31:0/1028303812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 msgr2=0x7f79140abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7918f1d640 1 --2- 172.21.15.31:0/1028303812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 0x7f79140abf90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f790c009dc0 tx=0x7f790c01c940 comp rx=0 tx=0).stop 2023-11-26T07:17:45.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7918f1d640 1 -- 172.21.15.31:0/1028303812 shutdown_connections 2023-11-26T07:17:45.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7918f1d640 1 --2- 172.21.15.31:0/1028303812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 0x7f79140abf90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7918f1d640 1 -- 172.21.15.31:0/1028303812 >> 172.21.15.31:0/1028303812 conn(0x7f79140b0cb0 msgr2=0x7f79140b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:45.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7918f1d640 1 -- 172.21.15.31:0/1028303812 shutdown_connections 2023-11-26T07:17:45.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.633+0000 7f7918f1d640 1 -- 172.21.15.31:0/1028303812 wait complete. 2023-11-26T07:17:45.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7918f1d640 1 Processor -- start 2023-11-26T07:17:45.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7918f1d640 1 -- start start 2023-11-26T07:17:45.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7918f1d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 0x7f791415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7918f1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79140cdfb0 0x7f7914164e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7918f1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791415a710 0x7f79140af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7918f1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7914164720 con 0x7f79140cdfb0 2023-11-26T07:17:45.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7918f1d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7914158af0 con 0x7f791415a710 2023-11-26T07:17:45.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7918f1d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7914158d90 con 0x7f79140cc460 2023-11-26T07:17:45.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79140cdfb0 0x7f7914164e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7912d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791415a710 0x7f79140af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f791259c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 0x7f791415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79140cdfb0 0x7f7914164e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50110/0 (socket says 172.21.15.31:50110) 2023-11-26T07:17:45.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7912d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791415a710 0x7f79140af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:52710/0 (socket says 172.21.15.31:52710) 2023-11-26T07:17:45.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790bfff640 1 -- 172.21.15.31:0/768308848 learned_addr learned my addr 172.21.15.31:0/768308848 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790bfff640 1 -- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 msgr2=0x7f791415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790bfff640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 0x7f791415b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790bfff640 1 -- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791415a710 msgr2=0x7f79140af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790bfff640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791415a710 0x7f79140af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790bfff640 1 -- 172.21.15.31:0/768308848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c009a70 con 0x7f79140cdfb0 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7912d9d640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791415a710 0x7f79140af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790bfff640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79140cdfb0 0x7f7914164e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00c390 tx=0x7f78fc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790affd640 1 -- 172.21.15.31:0/768308848 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc005cf0 con 0x7f79140cdfb0 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790affd640 1 -- 172.21.15.31:0/768308848 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc005e70 con 0x7f79140cdfb0 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7918f1d640 1 -- 172.21.15.31:0/768308848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7914159070 con 0x7f79140cdfb0 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f790affd640 1 -- 172.21.15.31:0/768308848 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0072f0 con 0x7f79140cdfb0 2023-11-26T07:17:45.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.637+0000 7f7918f1d640 1 -- 172.21.15.31:0/768308848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f791415b9b0 con 0x7f79140cdfb0 2023-11-26T07:17:45.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.641+0000 7f7918f1d640 1 -- 172.21.15.31:0/768308848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79140671f0 con 0x7f79140cdfb0 2023-11-26T07:17:45.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.641+0000 7f790affd640 1 -- 172.21.15.31:0/768308848 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f78fc007450 con 0x7f79140cdfb0 2023-11-26T07:17:45.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.641+0000 7f790affd640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f4066470 0x7f78f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.641+0000 7f790affd640 1 -- 172.21.15.31:0/768308848 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78fc083450 con 0x7f79140cdfb0 2023-11-26T07:17:45.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.641+0000 7f791259c640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f4066470 0x7f78f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.641+0000 7f791259c640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f4066470 0x7f78f4068930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7904000f70 tx=0x7f7904009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.645+0000 7f790affd640 1 -- 172.21.15.31:0/768308848 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f78fc0509a0 con 0x7f79140cdfb0 2023-11-26T07:17:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.801+0000 7f7918f1d640 1 -- 172.21.15.31:0/768308848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f791404e850 con 0x7f79140cdfb0 2023-11-26T07:17:45.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.801+0000 7f790affd640 1 -- 172.21.15.31:0/768308848 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78fc017020 con 0x7f79140cdfb0 2023-11-26T07:17:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:17:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 -- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f4066470 msgr2=0x7f78f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f4066470 0x7f78f4068930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7904000f70 tx=0x7f7904009040 comp rx=0 tx=0).stop 2023-11-26T07:17:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 -- 172.21.15.31:0/768308848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79140cdfb0 msgr2=0x7f7914164e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79140cdfb0 0x7f7914164e90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00c390 tx=0x7f78fc00c850 comp rx=0 tx=0).stop 2023-11-26T07:17:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 -- 172.21.15.31:0/768308848 shutdown_connections 2023-11-26T07:17:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f4066470 0x7f78f4068930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f791415a710 0x7f79140af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79140cdfb0 0x7f7914164e90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 --2- 172.21.15.31:0/768308848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79140cc460 0x7f791415b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 -- 172.21.15.31:0/768308848 >> 172.21.15.31:0/768308848 conn(0x7f79140b0cb0 msgr2=0x7f79140ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 -- 172.21.15.31:0/768308848 shutdown_connections 2023-11-26T07:17:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.805+0000 7f7918f1d640 1 -- 172.21.15.31:0/768308848 wait complete. 2023-11-26T07:17:45.828 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:45.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb4b59e640 1 Processor -- start 2023-11-26T07:17:45.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb4b59e640 1 -- start start 2023-11-26T07:17:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb4b59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb4c06bc90 0x7feb4c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb4b59e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4c06d4f0 con 0x7feb4c0b2850 2023-11-26T07:17:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb4b59e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4c0d2070 con 0x7feb4c160980 2023-11-26T07:17:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb4b59e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4c06c150 con 0x7feb4c06bc90 2023-11-26T07:17:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb4ad9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7feb4c0b2850 0x7feb4c0b18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54214/0 (socket says 172.21.15.31:54214) 2023-11-26T07:17:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb49d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feb4c06bc90 0x7feb4c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb4ad9d640 1 -- 172.21.15.31:0/2614959103 learned_addr learned my addr 172.21.15.31:0/2614959103 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb4959a640 1 -- 172.21.15.31:0/2614959103 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280581181 0 0) 0x7feb4c06d4f0 con 0x7feb4c0b2850 2023-11-26T07:17:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb4959a640 1 -- 172.21.15.31:0/2614959103 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb34003530 con 0x7feb4c0b2850 2023-11-26T07:17:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb49d9b640 1 -- 172.21.15.31:0/2614959103 >> v1:172.21.15.97:6789/0 conn(0x7feb4c160980 legacy=0x7feb4c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb49d9b640 1 -- 172.21.15.31:0/2614959103 >> v1:172.21.15.31:6789/0 conn(0x7feb4c0b2850 legacy=0x7feb4c0b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.969+0000 7feb49d9b640 1 -- 172.21.15.31:0/2614959103 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c06c430 con 0x7feb4c06bc90 2023-11-26T07:17:45.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb49d9b640 1 --2- 172.21.15.31:0/2614959103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c06bc90 0x7feb4c0cdc00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb400097c0 tx=0x7feb4001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea49e6e00a851f35 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb4959a640 1 -- 172.21.15.31:0/2614959103 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400ab650 con 0x7feb4c06bc90 2023-11-26T07:17:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb4959a640 1 -- 172.21.15.31:0/2614959103 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb400ab7b0 con 0x7feb4c06bc90 2023-11-26T07:17:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb4959a640 1 -- 172.21.15.31:0/2614959103 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400abb40 con 0x7feb4c06bc90 2023-11-26T07:17:45.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb4b59e640 1 -- 172.21.15.31:0/2614959103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c06bc90 msgr2=0x7feb4c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb4b59e640 1 --2- 172.21.15.31:0/2614959103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c06bc90 0x7feb4c0cdc00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb400097c0 tx=0x7feb4001c450 comp rx=0 tx=0).stop 2023-11-26T07:17:45.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb4b59e640 1 -- 172.21.15.31:0/2614959103 shutdown_connections 2023-11-26T07:17:45.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb4b59e640 1 --2- 172.21.15.31:0/2614959103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c06bc90 0x7feb4c0cdc00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb4b59e640 1 -- 172.21.15.31:0/2614959103 >> 172.21.15.31:0/2614959103 conn(0x7feb4c09c5c0 msgr2=0x7feb4c0d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:45.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb4b59e640 1 -- 172.21.15.31:0/2614959103 shutdown_connections 2023-11-26T07:17:45.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.973+0000 7feb4b59e640 1 -- 172.21.15.31:0/2614959103 wait complete. 2023-11-26T07:17:45.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 Processor -- start 2023-11-26T07:17:45.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 -- start start 2023-11-26T07:17:45.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb4c06bc90 0x7feb4c159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c0b2850 0x7feb4c15c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feb4c160980 0x7feb4c0c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c7b20 con 0x7feb4c0b2850 2023-11-26T07:17:45.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c6f10 con 0x7feb4c06bc90 2023-11-26T07:17:45.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c71f0 con 0x7feb4c160980 2023-11-26T07:17:45.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb49d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c0b2850 0x7feb4c15c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb49d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c0b2850 0x7feb4c15c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50132/0 (socket says 172.21.15.31:50132) 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb49d9b640 1 -- 172.21.15.31:0/1401903857 learned_addr learned my addr 172.21.15.31:0/1401903857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4a59c640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb4c06bc90 0x7feb4c159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4ad9d640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feb4c160980 0x7feb4c0c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb49d9b640 1 -- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feb4c160980 msgr2=0x7feb4c0c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb49d9b640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feb4c160980 0x7feb4c0c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb49d9b640 1 -- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb4c06bc90 msgr2=0x7feb4c159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb49d9b640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb4c06bc90 0x7feb4c159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb49d9b640 1 -- 172.21.15.31:0/1401903857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb40009420 con 0x7feb4c0b2850 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4ad9d640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feb4c160980 0x7feb4c0c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:45.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4a59c640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb4c06bc90 0x7feb4c159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:45.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb49d9b640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c0b2850 0x7feb4c15c000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb400ad040 tx=0x7feb4001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb3affd640 1 -- 172.21.15.31:0/1401903857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400bf610 con 0x7feb4c0b2850 2023-11-26T07:17:45.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb3affd640 1 -- 172.21.15.31:0/1401903857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb400bf790 con 0x7feb4c0b2850 2023-11-26T07:17:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 -- 172.21.15.31:0/1401903857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c0c74d0 con 0x7feb4c0b2850 2023-11-26T07:17:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb3affd640 1 -- 172.21.15.31:0/1401903857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400bfb40 con 0x7feb4c0b2850 2023-11-26T07:17:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 -- 172.21.15.31:0/1401903857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c159f50 con 0x7feb4c0b2850 2023-11-26T07:17:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.977+0000 7feb4b59e640 1 -- 172.21.15.31:0/1401903857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4c0c7c90 con 0x7feb4c0b2850 2023-11-26T07:17:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.981+0000 7feb3affd640 1 -- 172.21.15.31:0/1401903857 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7feb400bfca0 con 0x7feb4c0b2850 2023-11-26T07:17:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.981+0000 7feb3affd640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feb2c066470 0x7feb2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.981+0000 7feb3affd640 1 -- 172.21.15.31:0/1401903857 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb4011ed40 con 0x7feb4c0b2850 2023-11-26T07:17:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.981+0000 7feb4a59c640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feb2c066470 0x7feb2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:45.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.981+0000 7feb4a59c640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feb2c066470 0x7feb2c068930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0c9e90 tx=0x7feb3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:45.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:45.985+0000 7feb3affd640 1 -- 172.21.15.31:0/1401903857 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feb400ec120 con 0x7feb4c0b2850 2023-11-26T07:17:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.173+0000 7feb4b59e640 1 -- 172.21.15.31:0/1401903857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb4c074a00 con 0x7feb4c0b2850 2023-11-26T07:17:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.173+0000 7feb3affd640 1 -- 172.21.15.31:0/1401903857 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7feb400ef740 con 0x7feb4c0b2850 2023-11-26T07:17:46.178 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:46.178 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4099326271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4099326271},{"type":"v1","addr":"172.21.15.31:6835","nonce":4099326271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2404164953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2404164953},{"type":"v1","addr":"172.21.15.97:6836","nonce":2404164953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1312969281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1312969281},{"type":"v1","addr":"172.21.15.31:6834","nonce":1312969281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:32.510772+0000","modified":"2023-11-26T07:17:35.514610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/527056216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":527056216},{"type":"v1","addr":"172.21.15.97:6837","nonce":527056216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:17:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-11-26T07:17:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 -- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feb2c066470 msgr2=0x7feb2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feb2c066470 0x7feb2c068930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0c9e90 tx=0x7feb3c009040 comp rx=0 tx=0).stop 2023-11-26T07:17:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 -- 172.21.15.31:0/1401903857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c0b2850 msgr2=0x7feb4c15c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c0b2850 0x7feb4c15c000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb400ad040 tx=0x7feb4001ac90 comp rx=0 tx=0).stop 2023-11-26T07:17:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 -- 172.21.15.31:0/1401903857 shutdown_connections 2023-11-26T07:17:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feb2c066470 0x7feb2c068930 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feb4c160980 0x7feb4c0c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb4c0b2850 0x7feb4c15c000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 --2- 172.21.15.31:0/1401903857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb4c06bc90 0x7feb4c159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 -- 172.21.15.31:0/1401903857 >> 172.21.15.31:0/1401903857 conn(0x7feb4c09c5c0 msgr2=0x7feb4c0d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 -- 172.21.15.31:0/1401903857 shutdown_connections 2023-11-26T07:17:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.177+0000 7feb4b59e640 1 -- 172.21.15.31:0/1401903857 wait complete. 2023-11-26T07:17:46.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:46.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.337+0000 7f9392aaa640 1 Processor -- start 2023-11-26T07:17:46.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.337+0000 7f9392aaa640 1 -- start start 2023-11-26T07:17:46.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f9392aaa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c0d2080 0x7f938c168160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f9392aaa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938c0cefd0 con 0x7f938c0cead0 2023-11-26T07:17:46.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f9392aaa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938c0cf2b0 con 0x7f938c15d7b0 2023-11-26T07:17:46.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f9392aaa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0d17e0 con 0x7f938c0d2080 2023-11-26T07:17:46.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f938bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f938c0cead0 0x7f938c0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54226/0 (socket says 172.21.15.31:54226) 2023-11-26T07:17:46.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f938bfff640 1 -- 172.21.15.31:0/1468211535 learned_addr learned my addr 172.21.15.31:0/1468211535 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:46.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f9391047640 1 --2- 172.21.15.31:0/1468211535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c0d2080 0x7f938c168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f938b7fe640 1 -- 172.21.15.31:0/1468211535 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741134914 0 0) 0x7f938c0cefd0 con 0x7f938c0cead0 2023-11-26T07:17:46.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f938b7fe640 1 -- 172.21.15.31:0/1468211535 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f937c003530 con 0x7f938c0cead0 2023-11-26T07:17:46.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f938b7fe640 1 -- 172.21.15.31:0/1468211535 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3316596339 0 0) 0x7f938c0cf2b0 con 0x7f938c15d7b0 2023-11-26T07:17:46.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f938b7fe640 1 -- 172.21.15.31:0/1468211535 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f938c0cefd0 con 0x7f938c15d7b0 2023-11-26T07:17:46.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f9391047640 1 -- 172.21.15.31:0/1468211535 >> v1:172.21.15.97:6789/0 conn(0x7f938c15d7b0 legacy=0x7f938c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f9391047640 1 -- 172.21.15.31:0/1468211535 >> v1:172.21.15.31:6789/0 conn(0x7f938c0cead0 legacy=0x7f938c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f938b7fe640 1 -- 172.21.15.31:0/1468211535 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3130076555 0 0) 0x7f937c003530 con 0x7f938c0cead0 2023-11-26T07:17:46.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.341+0000 7f9391047640 1 -- 172.21.15.31:0/1468211535 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c06b3f0 con 0x7f938c0d2080 2023-11-26T07:17:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f9391047640 1 --2- 172.21.15.31:0/1468211535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c0d2080 0x7f938c168160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9384009dc0 tx=0x7f938401c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ad5ef493b7539e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f938b7fe640 1 -- 172.21.15.31:0/1468211535 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378001680 con 0x7f938c0d2080 2023-11-26T07:17:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f938b7fe640 1 -- 172.21.15.31:0/1468211535 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93840a5580 con 0x7f938c0d2080 2023-11-26T07:17:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f938b7fe640 1 -- 172.21.15.31:0/1468211535 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93840a5910 con 0x7f938c0d2080 2023-11-26T07:17:46.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f9392aaa640 1 -- 172.21.15.31:0/1468211535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c0d2080 msgr2=0x7f938c168160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f9392aaa640 1 --2- 172.21.15.31:0/1468211535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c0d2080 0x7f938c168160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9384009dc0 tx=0x7f938401c740 comp rx=0 tx=0).stop 2023-11-26T07:17:46.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f9392aaa640 1 -- 172.21.15.31:0/1468211535 shutdown_connections 2023-11-26T07:17:46.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f9392aaa640 1 --2- 172.21.15.31:0/1468211535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c0d2080 0x7f938c168160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f9392aaa640 1 -- 172.21.15.31:0/1468211535 >> 172.21.15.31:0/1468211535 conn(0x7f938c0d33b0 msgr2=0x7f938c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:46.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f9392aaa640 1 -- 172.21.15.31:0/1468211535 shutdown_connections 2023-11-26T07:17:46.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f9392aaa640 1 -- 172.21.15.31:0/1468211535 wait complete. 2023-11-26T07:17:46.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f9392aaa640 1 Processor -- start 2023-11-26T07:17:46.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.345+0000 7f9392aaa640 1 -- start start 2023-11-26T07:17:46.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9392aaa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f938c0cead0 0x7f938c155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9392aaa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f938c0d2080 0x7f938c158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9392aaa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c15d7b0 0x7f938c0c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9392aaa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0b7320 con 0x7f938c0d2080 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9392aaa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c155ba0 con 0x7f938c0cead0 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9392aaa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1593d0 con 0x7f938c15d7b0 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f938bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f938c0d2080 0x7f938c158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f938bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f938c0d2080 0x7f938c158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50146/0 (socket says 172.21.15.31:50146) 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f938bfff640 1 -- 172.21.15.31:0/3050516364 learned_addr learned my addr 172.21.15.31:0/3050516364 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9390846640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f938c0cead0 0x7f938c155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9391047640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c15d7b0 0x7f938c0c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f938bfff640 1 -- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c15d7b0 msgr2=0x7f938c0c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f938bfff640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c15d7b0 0x7f938c0c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f938bfff640 1 -- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f938c0cead0 msgr2=0x7f938c155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f938bfff640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f938c0cead0 0x7f938c155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f938bfff640 1 -- 172.21.15.31:0/3050516364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9384009a70 con 0x7f938c0d2080 2023-11-26T07:17:46.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9391047640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c15d7b0 0x7f938c0c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:46.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9390846640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f938c0cead0 0x7f938c155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:46.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f938bfff640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f938c0d2080 0x7f938c158e90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9378005890 tx=0x7f937800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f93897fa640 1 -- 172.21.15.31:0/3050516364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378005ae0 con 0x7f938c0d2080 2023-11-26T07:17:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f93897fa640 1 -- 172.21.15.31:0/3050516364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9378005c40 con 0x7f938c0d2080 2023-11-26T07:17:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9392aaa640 1 -- 172.21.15.31:0/3050516364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c156340 con 0x7f938c0d2080 2023-11-26T07:17:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f93897fa640 1 -- 172.21.15.31:0/3050516364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378007070 con 0x7f938c0d2080 2023-11-26T07:17:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9392aaa640 1 -- 172.21.15.31:0/3050516364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c0b6c60 con 0x7f938c0d2080 2023-11-26T07:17:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f9392aaa640 1 -- 172.21.15.31:0/3050516364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938c0b7460 con 0x7f938c0d2080 2023-11-26T07:17:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.349+0000 7f93897fa640 1 -- 172.21.15.31:0/3050516364 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f93780071f0 con 0x7f938c0d2080 2023-11-26T07:17:46.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.353+0000 7f93897fa640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9370066470 0x7f9370068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.353+0000 7f93897fa640 1 -- 172.21.15.31:0/3050516364 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93780834a0 con 0x7f938c0d2080 2023-11-26T07:17:46.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.353+0000 7f9390846640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9370066470 0x7f9370068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.353+0000 7f9390846640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9370066470 0x7f9370068930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f93800011f0 tx=0x7f9380009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.357+0000 7f93897fa640 1 -- 172.21.15.31:0/3050516364 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f938c0b7460 con 0x7f938c0d2080 2023-11-26T07:17:46.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.513+0000 7f9392aaa640 1 -- 172.21.15.31:0/3050516364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f938c0b7460 con 0x7f938c0d2080 2023-11-26T07:17:46.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.513+0000 7f93897fa640 1 -- 172.21.15.31:0/3050516364 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f938c0b7460 con 0x7f938c0d2080 2023-11-26T07:17:46.519 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:46.519 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 -- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9370066470 msgr2=0x7f9370068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9370066470 0x7f9370068930 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f93800011f0 tx=0x7f9380009040 comp rx=0 tx=0).stop 2023-11-26T07:17:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 -- 172.21.15.31:0/3050516364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f938c0d2080 msgr2=0x7f938c158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f938c0d2080 0x7f938c158e90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9378005890 tx=0x7f937800cbc0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 -- 172.21.15.31:0/3050516364 shutdown_connections 2023-11-26T07:17:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9370066470 0x7f9370068930 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f938c15d7b0 0x7f938c0c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f938c0d2080 0x7f938c158e90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 --2- 172.21.15.31:0/3050516364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f938c0cead0 0x7f938c155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 -- 172.21.15.31:0/3050516364 >> 172.21.15.31:0/3050516364 conn(0x7f938c0d33b0 msgr2=0x7f938c0d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:46.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 -- 172.21.15.31:0/3050516364 shutdown_connections 2023-11-26T07:17:46.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.517+0000 7f9392aaa640 1 -- 172.21.15.31:0/3050516364 wait complete. 2023-11-26T07:17:46.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:17:46.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a6ed8d640 1 Processor -- start 2023-11-26T07:17:46.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a6ed8d640 1 -- start start 2023-11-26T07:17:46.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a6ed8d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68146c60 0x7f0a68070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a6ed8d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6815dba0 con 0x7f0a68147040 2023-11-26T07:17:46.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a6ed8d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a6806c860 con 0x7f0a6806c220 2023-11-26T07:17:46.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a6ed8d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6806d100 con 0x7f0a68146c60 2023-11-26T07:17:46.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a67fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0a68147040 0x7f0a6806bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54228/0 (socket says 172.21.15.31:54228) 2023-11-26T07:17:46.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a67fff640 1 -- 172.21.15.31:0/4023882246 learned_addr learned my addr 172.21.15.31:0/4023882246 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:46.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a6cb29640 1 --2- 172.21.15.31:0/4023882246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68146c60 0x7f0a68070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a677fe640 1 -- 172.21.15.31:0/4023882246 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189561404 0 0) 0x7f0a6815dba0 con 0x7f0a68147040 2023-11-26T07:17:46.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a677fe640 1 -- 172.21.15.31:0/4023882246 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a58003530 con 0x7f0a68147040 2023-11-26T07:17:46.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a677fe640 1 -- 172.21.15.31:0/4023882246 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262016453 0 0) 0x7f0a6806c860 con 0x7f0a6806c220 2023-11-26T07:17:46.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a677fe640 1 -- 172.21.15.31:0/4023882246 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6815dba0 con 0x7f0a6806c220 2023-11-26T07:17:46.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a6cb29640 1 -- 172.21.15.31:0/4023882246 >> v1:172.21.15.97:6789/0 conn(0x7f0a6806c220 legacy=0x7f0a6815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a6cb29640 1 -- 172.21.15.31:0/4023882246 >> v1:172.21.15.31:6789/0 conn(0x7f0a68147040 legacy=0x7f0a6806bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.681+0000 7f0a6cb29640 1 -- 172.21.15.31:0/4023882246 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6806cb40 con 0x7f0a68146c60 2023-11-26T07:17:46.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a6cb29640 1 --2- 172.21.15.31:0/4023882246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68146c60 0x7f0a68070020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0097f0 tx=0x7f0a5c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e05e9fcf4724d71e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a677fe640 1 -- 172.21.15.31:0/4023882246 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c0ab600 con 0x7f0a68146c60 2023-11-26T07:17:46.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a677fe640 1 -- 172.21.15.31:0/4023882246 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5c0a7050 con 0x7f0a68146c60 2023-11-26T07:17:46.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a677fe640 1 -- 172.21.15.31:0/4023882246 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c0ab980 con 0x7f0a68146c60 2023-11-26T07:17:46.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4023882246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68146c60 msgr2=0x7f0a68070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a6ed8d640 1 --2- 172.21.15.31:0/4023882246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68146c60 0x7f0a68070020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0097f0 tx=0x7f0a5c01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4023882246 shutdown_connections 2023-11-26T07:17:46.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a6ed8d640 1 --2- 172.21.15.31:0/4023882246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68146c60 0x7f0a68070020 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4023882246 >> 172.21.15.31:0/4023882246 conn(0x7f0a680b2ea0 msgr2=0x7f0a680b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:46.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4023882246 shutdown_connections 2023-11-26T07:17:46.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.685+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4023882246 wait complete. 2023-11-26T07:17:46.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6ed8d640 1 Processor -- start 2023-11-26T07:17:46.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6ed8d640 1 -- start start 2023-11-26T07:17:46.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6ed8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a6806c220 0x7f0a681f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6ed8d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a68146c60 0x7f0a6813e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6ed8d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68147040 0x7f0a68139c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6ed8d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6815bcc0 con 0x7f0a68146c60 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6ed8d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681fa250 con 0x7f0a6806c220 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6ed8d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681fa530 con 0x7f0a68147040 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a67fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a68146c60 0x7f0a6813e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a67fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a68146c60 0x7f0a6813e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50160/0 (socket says 172.21.15.31:50160) 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a67fff640 1 -- 172.21.15.31:0/4120723426 learned_addr learned my addr 172.21.15.31:0/4120723426 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6cb29640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a6806c220 0x7f0a681f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6d32a640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68147040 0x7f0a68139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a67fff640 1 -- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68147040 msgr2=0x7f0a68139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a67fff640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68147040 0x7f0a68139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a67fff640 1 -- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a6806c220 msgr2=0x7f0a681f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a67fff640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a6806c220 0x7f0a681f9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a67fff640 1 -- 172.21.15.31:0/4120723426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c009420 con 0x7f0a68146c60 2023-11-26T07:17:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6d32a640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68147040 0x7f0a68139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a67fff640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a68146c60 0x7f0a6813e230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400c410 tx=0x7f0a5400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a657fa640 1 -- 172.21.15.31:0/4120723426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54005cf0 con 0x7f0a68146c60 2023-11-26T07:17:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a657fa640 1 -- 172.21.15.31:0/4120723426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a54005e50 con 0x7f0a68146c60 2023-11-26T07:17:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4120723426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a681fa810 con 0x7f0a68146c60 2023-11-26T07:17:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a657fa640 1 -- 172.21.15.31:0/4120723426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a540072d0 con 0x7f0a68146c60 2023-11-26T07:17:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.689+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4120723426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a681f9f60 con 0x7f0a68146c60 2023-11-26T07:17:46.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.693+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4120723426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6804e850 con 0x7f0a68146c60 2023-11-26T07:17:46.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.693+0000 7f0a657fa640 1 -- 172.21.15.31:0/4120723426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0a54007430 con 0x7f0a68146c60 2023-11-26T07:17:46.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.693+0000 7f0a657fa640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0a4c066470 0x7f0a4c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:46.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.693+0000 7f0a657fa640 1 -- 172.21.15.31:0/4120723426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a540833d0 con 0x7f0a68146c60 2023-11-26T07:17:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.693+0000 7f0a6cb29640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0a4c066470 0x7f0a4c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.693+0000 7f0a6cb29640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0a4c066470 0x7f0a4c068930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0ad000 tx=0x7f0a5c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.697+0000 7f0a657fa640 1 -- 172.21.15.31:0/4120723426 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0a54050880 con 0x7f0a68146c60 2023-11-26T07:17:46.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.885+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4120723426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a680d1f60 con 0x7f0a68146c60 2023-11-26T07:17:46.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.885+0000 7f0a657fa640 1 -- 172.21.15.31:0/4120723426 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f0a54053ea0 con 0x7f0a68146c60 2023-11-26T07:17:46.890 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:46.890 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4099326271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4099326271},{"type":"v1","addr":"172.21.15.31:6835","nonce":4099326271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2404164953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2404164953},{"type":"v1","addr":"172.21.15.97:6836","nonce":2404164953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1312969281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1312969281},{"type":"v1","addr":"172.21.15.31:6834","nonce":1312969281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:32.510772+0000","modified":"2023-11-26T07:17:35.514610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/527056216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":527056216},{"type":"v1","addr":"172.21.15.97:6837","nonce":527056216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:17:46.890 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-11-26T07:17:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0a4c066470 msgr2=0x7f0a4c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0a4c066470 0x7f0a4c068930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0ad000 tx=0x7f0a5c0a6000 comp rx=0 tx=0).stop 2023-11-26T07:17:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4120723426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a68146c60 msgr2=0x7f0a6813e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a68146c60 0x7f0a6813e230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400c410 tx=0x7f0a5400c8d0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4120723426 shutdown_connections 2023-11-26T07:17:46.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0a4c066470 0x7f0a4c068930 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a68147040 0x7f0a68139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a68146c60 0x7f0a6813e230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 --2- 172.21.15.31:0/4120723426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a6806c220 0x7f0a681f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:46.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4120723426 >> 172.21.15.31:0/4120723426 conn(0x7f0a680b2ea0 msgr2=0x7f0a68144b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:46.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4120723426 shutdown_connections 2023-11-26T07:17:46.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:46.889+0000 7f0a6ed8d640 1 -- 172.21.15.31:0/4120723426 wait complete. 2023-11-26T07:17:47.313 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:17:32.510772+0000', 'modified': '2023-11-26T07:17:35.514610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6837/527056216', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6835', 'nonce': 527056216}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 527056216}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:17:47.313 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:17:47.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-26T07:17:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.453+0000 7fc70325a640 1 Processor -- start 2023-11-26T07:17:47.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.453+0000 7fc70325a640 1 -- start start 2023-11-26T07:17:47.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc70325a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6fc0b15f0 0x7fc6fc0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc70325a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6fc161290 con 0x7fc6fc0b2870 2023-11-26T07:17:47.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc70325a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6fc161550 con 0x7fc6fc0b07a0 2023-11-26T07:17:47.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc70325a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6fc0b0b80 con 0x7fc6fc0b15f0 2023-11-26T07:17:47.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc6fc0b2870 0x7fc6fc0b0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54232/0 (socket says 172.21.15.31:54232) 2023-11-26T07:17:47.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fbfff640 1 -- 172.21.15.31:0/3869959481 learned_addr learned my addr 172.21.15.31:0/3869959481 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc7017f7640 1 --2- 172.21.15.31:0/3869959481 >> v2:172.21.15.97:3300/0 conn(0x7fc6fc0b15f0 0x7fc6fc0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338176253 0 0) 0x7fc6fc161290 con 0x7fc6fc0b2870 2023-11-26T07:17:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6ec003530 con 0x7fc6fc0b2870 2023-11-26T07:17:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135621307 0 0) 0x7fc6fc161550 con 0x7fc6fc0b07a0 2023-11-26T07:17:47.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6fc161290 con 0x7fc6fc0b07a0 2023-11-26T07:17:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1863332095 0 0) 0x7fc6ec003530 con 0x7fc6fc0b2870 2023-11-26T07:17:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 >> v2:172.21.15.97:3300/0 conn(0x7fc6fc0b15f0 msgr2=0x7fc6fc0cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 --2- 172.21.15.31:0/3869959481 >> v2:172.21.15.97:3300/0 conn(0x7fc6fc0b15f0 0x7fc6fc0cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 >> v1:172.21.15.97:6789/0 conn(0x7fc6fc0b07a0 legacy=0x7fc6fc1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc0c65a0 con 0x7fc6fc0b2870 2023-11-26T07:17:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fc6e8001680 con 0x7fc6fc0b2870 2023-11-26T07:17:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc7017f7640 1 --2- 172.21.15.31:0/3869959481 >> v2:172.21.15.97:3300/0 conn(0x7fc6fc0b15f0 0x7fc6fc0cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 >> v1:172.21.15.31:6789/0 conn(0x7fc6fc0b2870 legacy=0x7fc6fc0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6ec003b60 0x7fc6ec003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6ec0047d0 0x7fc6ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6ec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec00c060 con 0x7fc6ec005190 2023-11-26T07:17:47.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec00c1a0 con 0x7fc6ec0047d0 2023-11-26T07:17:47.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec00c360 con 0x7fc6ec003b60 2023-11-26T07:17:47.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc7017f7640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6ec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6ec0047d0 0x7fc6ec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc700ff6640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6ec003b60 0x7fc6ec003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc7017f7640 1 -- 172.21.15.31:0/3869959481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6ec003b60 msgr2=0x7fc6ec003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc7017f7640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6ec003b60 0x7fc6ec003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc7017f7640 1 -- 172.21.15.31:0/3869959481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6ec0047d0 msgr2=0x7fc6ec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc7017f7640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6ec0047d0 0x7fc6ec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc7017f7640 1 -- 172.21.15.31:0/3869959481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec003530 con 0x7fc6ec005190 2023-11-26T07:17:47.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.457+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6ec0047d0 0x7fc6ec004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc7017f7640 1 -- 172.21.15.31:0/3869959481 reap_dead start 2023-11-26T07:17:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc7017f7640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6ec00b990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e320 tx=0x7fc6f401ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf8f987a7aad3d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40a5ca0 con 0x7fc6ec005190 2023-11-26T07:17:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f400f040 con 0x7fc6ec005190 2023-11-26T07:17:47.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc6fb7fe640 1 -- 172.21.15.31:0/3869959481 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40039a0 con 0x7fc6ec005190 2023-11-26T07:17:47.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc70325a640 1 -- 172.21.15.31:0/3869959481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 msgr2=0x7fc6ec00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc70325a640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6ec00b990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc6f400e320 tx=0x7fc6f401ce20 comp rx=0 tx=0).stop 2023-11-26T07:17:47.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc70325a640 1 -- 172.21.15.31:0/3869959481 shutdown_connections 2023-11-26T07:17:47.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc70325a640 1 --2- 172.21.15.31:0/3869959481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6ec00b990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc70325a640 1 -- 172.21.15.31:0/3869959481 >> 172.21.15.31:0/3869959481 conn(0x7fc6fc0b1d50 msgr2=0x7fc6fc0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc70325a640 1 -- 172.21.15.31:0/3869959481 shutdown_connections 2023-11-26T07:17:47.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.461+0000 7fc70325a640 1 -- 172.21.15.31:0/3869959481 wait complete. 2023-11-26T07:17:47.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc70325a640 1 Processor -- start 2023-11-26T07:17:47.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc70325a640 1 -- start start 2023-11-26T07:17:47.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc70325a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6fc15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc70325a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6fc15cc10 0x7fc6fc15d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc70325a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6fc0ad150 0x7fc6fc0ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc70325a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc15d5d0 con 0x7fc6ec005190 2023-11-26T07:17:47.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc70325a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc15ab00 con 0x7fc6fc15cc10 2023-11-26T07:17:47.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc70325a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc09b590 con 0x7fc6fc0ad150 2023-11-26T07:17:47.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc700ff6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6fc15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc700ff6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6fc15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50186/0 (socket says 172.21.15.31:50186) 2023-11-26T07:17:47.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc700ff6640 1 -- 172.21.15.31:0/4223403161 learned_addr learned my addr 172.21.15.31:0/4223403161 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:47.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc7017f7640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6fc0ad150 0x7fc6fc0ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6fc15cc10 0x7fc6fc15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc700ff6640 1 -- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6fc0ad150 msgr2=0x7fc6fc0ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc700ff6640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6fc0ad150 0x7fc6fc0ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc700ff6640 1 -- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6fc15cc10 msgr2=0x7fc6fc15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc700ff6640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6fc15cc10 0x7fc6fc15d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc700ff6640 1 -- 172.21.15.31:0/4223403161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc0ae750 con 0x7fc6ec005190 2023-11-26T07:17:47.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc700ff6640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6fc15a5c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000f7d0 tx=0x7fc6f000fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc6f97fa640 1 -- 172.21.15.31:0/4223403161 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0005110 con 0x7fc6ec005190 2023-11-26T07:17:47.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc6f97fa640 1 -- 172.21.15.31:0/4223403161 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f0005270 con 0x7fc6ec005190 2023-11-26T07:17:47.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc6f97fa640 1 -- 172.21.15.31:0/4223403161 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f00067e0 con 0x7fc6ec005190 2023-11-26T07:17:47.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc70325a640 1 -- 172.21.15.31:0/4223403161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc0adeb0 con 0x7fc6ec005190 2023-11-26T07:17:47.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.465+0000 7fc70325a640 1 -- 172.21.15.31:0/4223403161 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc0ae470 con 0x7fc6ec005190 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc6f97fa640 1 -- 172.21.15.31:0/4223403161 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc6f0006940 con 0x7fc6ec005190 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc6f97fa640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6e0066470 0x7fc6e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc6f97fa640 1 -- 172.21.15.31:0/4223403161 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6f0083350 con 0x7fc6ec005190 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6e0066470 0x7fc6e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7065ca000 1 Processor -- start 2023-11-26T07:17:47.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7065ca000 1 -- start start 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6e0066470 0x7fc6e0068930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc161970 tx=0x7fc6e8001120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 0x55d0a9639070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 0x55d0a95eede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a95ef700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7065ca000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a9459130 con 0x55d0a95ef320 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7065ca000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a94469a0 con 0x55d0a9638c90 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7065ca000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0a9443ed0 con 0x55d0a96395b0 2023-11-26T07:17:47.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7017f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a95ef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7017f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a95ef700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50200/0 (socket says 172.21.15.31:50200) 2023-11-26T07:17:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc7017f7640 1 -- 172.21.15.31:0/1922288446 learned_addr learned my addr 172.21.15.31:0/1922288446 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.469+0000 7fc700ff6640 1 --2- 172.21.15.31:0/1922288446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 0x55d0a9639070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/1922288446 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 0x55d0a95eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7017f7640 1 -- 172.21.15.31:0/1922288446 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 msgr2=0x55d0a95eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7017f7640 1 --2- 172.21.15.31:0/1922288446 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 0x55d0a95eede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7017f7640 1 -- 172.21.15.31:0/1922288446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 msgr2=0x55d0a9639070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7017f7640 1 --2- 172.21.15.31:0/1922288446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 0x55d0a9639070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7017f7640 1 -- 172.21.15.31:0/1922288446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0a9137890 con 0x55d0a95ef320 2023-11-26T07:17:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/1922288446 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 0x55d0a95eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7017f7640 1 --2- 172.21.15.31:0/1922288446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a95ef700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40095c0 tx=0x7fc6f401b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c457edf840340b93 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc6e67fc640 1 -- 172.21.15.31:0/1922288446 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f400e040 con 0x55d0a95ef320 2023-11-26T07:17:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc6e67fc640 1 -- 172.21.15.31:0/1922288446 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f40a6e70 con 0x55d0a95ef320 2023-11-26T07:17:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc6e67fc640 1 -- 172.21.15.31:0/1922288446 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4003ba0 con 0x55d0a95ef320 2023-11-26T07:17:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 -- 172.21.15.31:0/1922288446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 msgr2=0x55d0a95ef700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1922288446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a95ef700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40095c0 tx=0x7fc6f401b660 comp rx=0 tx=0).stop 2023-11-26T07:17:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 -- 172.21.15.31:0/1922288446 shutdown_connections 2023-11-26T07:17:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1922288446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a95ef700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1922288446 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 0x55d0a95eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1922288446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 0x55d0a9639070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 -- 172.21.15.31:0/1922288446 >> 172.21.15.31:0/1922288446 conn(0x55d0a95eb2d0 msgr2=0x55d0a95ed6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 -- 172.21.15.31:0/1922288446 shutdown_connections 2023-11-26T07:17:47.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 -- 172.21.15.31:0/1922288446 wait complete. 2023-11-26T07:17:47.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 -- start start 2023-11-26T07:17:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 Processor -- start 2023-11-26T07:17:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a95ef320 0x55d0a95f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a9638c90 0x55d0a95f4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a96395b0 0x55d0a9607910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a943f7c0 con 0x55d0a9638c90 2023-11-26T07:17:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a93f7bf0 con 0x55d0a96395b0 2023-11-26T07:17:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0a93fda90 con 0x55d0a95ef320 2023-11-26T07:17:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc7065ca000 10 client.0 _ll_register_callbacks cb 0x55d0a95da3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:17:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.473+0000 7fc6fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a9638c90 0x55d0a95f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc700ff6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a95ef320 0x55d0a95f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7017f7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a96395b0 0x55d0a9607910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc6fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a9638c90 0x55d0a95f4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50208/0 (socket says 172.21.15.31:50208) 2023-11-26T07:17:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc6fbfff640 1 -- 172.21.15.31:0/649361437 learned_addr learned my addr 172.21.15.31:0/649361437 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7017f7640 1 -- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a95ef320 msgr2=0x55d0a95f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7017f7640 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a95ef320 0x55d0a95f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7017f7640 1 -- 172.21.15.31:0/649361437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a9638c90 msgr2=0x55d0a95f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7017f7640 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a9638c90 0x55d0a95f4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7017f7640 1 -- 172.21.15.31:0/649361437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec003530 con 0x55d0a96395b0 2023-11-26T07:17:47.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a9638c90 0x55d0a95f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:47.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7017f7640 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a96395b0 0x55d0a9607910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6f401cc50 tx=0x7fc6ec005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d376e8e2f99f4d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc6db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-26T07:17:47.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc6db7fe640 1 -- 172.21.15.31:0/649361437 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40c4070 con 0x55d0a96395b0 2023-11-26T07:17:47.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc6db7fe640 1 -- 172.21.15.31:0/649361437 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6f4003f30 con 0x55d0a96395b0 2023-11-26T07:17:47.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7065ca000 1 -- 172.21.15.31:0/649361437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0a9401470 con 0x55d0a96395b0 2023-11-26T07:17:47.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc6db7fe640 1 -- 172.21.15.31:0/649361437 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6f400e050 con 0x55d0a96395b0 2023-11-26T07:17:47.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7065ca000 10 client.4633 fetch_fsmap learned FSMap version 8 2023-11-26T07:17:47.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7065ca000 1 -- 172.21.15.31:0/649361437 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55d0a9400190 con 0x55d0a96395b0 2023-11-26T07:17:47.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc6db7fe640 1 -- 172.21.15.31:0/649361437 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d0a9401470 con 0x55d0a96395b0 2023-11-26T07:17:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc6db7fe640 1 -- 172.21.15.31:0/649361437 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc6f40c7020 con 0x55d0a96395b0 2023-11-26T07:17:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7065ca000 10 client.4633 fetch_fsmap finished waiting for FSMap version 8 2023-11-26T07:17:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7065ca000 10 client.4633 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:17:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7065ca000 20 client.4633 populate_metadata read hostname 'smithi031' 2023-11-26T07:17:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7065ca000 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:6835/527056216,v1:172.21.15.97:6837/527056216] conn(0x55d0a9608320 0x55d0a964af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7065ca000 4 client.4633 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:17:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc7065ca000 1 -- 172.21.15.31:0/649361437 --> [v2:172.21.15.97:6835/527056216,v1:172.21.15.97:6837/527056216] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0a9404b50 con 0x55d0a9608320 2023-11-26T07:17:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc700ff6640 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:6835/527056216,v1:172.21.15.97:6837/527056216] conn(0x55d0a9608320 0x55d0a964af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc700ff6640 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:6835/527056216,v1:172.21.15.97:6837/527056216] conn(0x55d0a9608320 0x55d0a964af50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3e570dd09171050 server_cookie=7389dfb07353b947 in_seq=0 out_seq=0 2023-11-26T07:17:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.477+0000 7fc6db7fe640 10 client.4633 ms_handle_connect on v2:172.21.15.97:6835/527056216 2023-11-26T07:17:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc6db7fe640 1 -- 172.21.15.31:0/649361437 <== mds.0 v2:172.21.15.97:6835/527056216 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0a9404b50 con 0x55d0a9608320 2023-11-26T07:17:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc6db7fe640 10 client.4633 handle_command_reply: tid=0 2023-11-26T07:17:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 client.4633 shutdown 2023-11-26T07:17:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 -- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a96395b0 msgr2=0x55d0a9607910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a96395b0 0x55d0a9607910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6f401cc50 tx=0x7fc6ec005800 comp rx=0 tx=0).stop 2023-11-26T07:17:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 -- 172.21.15.31:0/649361437 shutdown_connections 2023-11-26T07:17:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:6835/527056216,v1:172.21.15.97:6837/527056216] conn(0x55d0a9608320 0x55d0a964af50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a96395b0 0x55d0a9607910 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a9638c90 0x55d0a95f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc6db7fe640 0 client.4633 ms_handle_reset on v2:172.21.15.97:6835/527056216 2023-11-26T07:17:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 --2- 172.21.15.31:0/649361437 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a95ef320 0x55d0a95f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 -- 172.21.15.31:0/649361437 >> 172.21.15.31:0/649361437 conn(0x55d0a95eb2d0 msgr2=0x55d0a95ed6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 -- 172.21.15.31:0/649361437 shutdown_connections 2023-11-26T07:17:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 1 -- 172.21.15.31:0/649361437 wait complete. 2023-11-26T07:17:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.481+0000 7fc7065ca000 20 client.4633 trim_cache size 0 max 16384 2023-11-26T07:17:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 Processor -- start 2023-11-26T07:17:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 -- start start 2023-11-26T07:17:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a9637c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a9638c90 0x55d0a9636b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a96395b0 0x55d0a9637070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a90f8940 con 0x55d0a95ef320 2023-11-26T07:17:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a95f4fb0 con 0x55d0a96395b0 2023-11-26T07:17:47.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0a943e650 con 0x55d0a9638c90 2023-11-26T07:17:47.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc700ff6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a9637c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc700ff6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a9637c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50216/0 (socket says 172.21.15.31:50216) 2023-11-26T07:17:47.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc700ff6640 1 -- 172.21.15.31:0/1055133754 learned_addr learned my addr 172.21.15.31:0/1055133754 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:47.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc700ff6640 1 -- 172.21.15.31:0/1055133754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a9638c90 msgr2=0x55d0a9636b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc700ff6640 1 --2- 172.21.15.31:0/1055133754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a9638c90 0x55d0a9636b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc700ff6640 1 -- 172.21.15.31:0/1055133754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a96395b0 msgr2=0x55d0a9637070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc700ff6640 1 --2- 172.21.15.31:0/1055133754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a96395b0 0x55d0a9637070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc700ff6640 1 -- 172.21.15.31:0/1055133754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0a93e6490 con 0x55d0a95ef320 2023-11-26T07:17:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc700ff6640 1 --2- 172.21.15.31:0/1055133754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a9637c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00817e0 tx=0x7fc6f00816b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c4a1d911aa3667 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc6e6ffd640 1 -- 172.21.15.31:0/1055133754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f00830c0 con 0x55d0a95ef320 2023-11-26T07:17:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc6e6ffd640 1 -- 172.21.15.31:0/1055133754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f0069070 con 0x55d0a95ef320 2023-11-26T07:17:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc6e6ffd640 1 -- 172.21.15.31:0/1055133754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0050bb0 con 0x55d0a95ef320 2023-11-26T07:17:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 -- 172.21.15.31:0/1055133754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 msgr2=0x55d0a9637c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1055133754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a9637c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00817e0 tx=0x7fc6f00816b0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 -- 172.21.15.31:0/1055133754 shutdown_connections 2023-11-26T07:17:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1055133754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a96395b0 0x55d0a9637070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1055133754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a9638c90 0x55d0a9636b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1055133754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a9637c90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 -- 172.21.15.31:0/1055133754 >> 172.21.15.31:0/1055133754 conn(0x55d0a95eb2d0 msgr2=0x55d0a9646280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 -- 172.21.15.31:0/1055133754 shutdown_connections 2023-11-26T07:17:47.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.497+0000 7fc7065ca000 1 -- 172.21.15.31:0/1055133754 wait complete. 2023-11-26T07:17:47.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 -- start start 2023-11-26T07:17:47.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 Processor -- start 2023-11-26T07:17:47.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a961bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 0x55d0a95f06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 0x55d0a95f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a95f6890 con 0x55d0a95ef320 2023-11-26T07:17:47.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0a93f8d60 con 0x55d0a9638c90 2023-11-26T07:17:47.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0a9481140 con 0x55d0a96395b0 2023-11-26T07:17:47.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 10 client.0 _ll_register_callbacks cb 0x55d0a948dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:17:47.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc700ff6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a961bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc700ff6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a961bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50230/0 (socket says 172.21.15.31:50230) 2023-11-26T07:17:47.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc700ff6640 1 -- 172.21.15.31:0/1275544529 learned_addr learned my addr 172.21.15.31:0/1275544529 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7017f7640 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 0x55d0a95f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 0x55d0a95f06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc700ff6640 1 -- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 msgr2=0x55d0a95f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc700ff6640 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 0x55d0a95f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc700ff6640 1 -- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 msgr2=0x55d0a95f06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc700ff6640 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 0x55d0a95f06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc700ff6640 1 -- 172.21.15.31:0/1275544529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc0ae470 con 0x55d0a95ef320 2023-11-26T07:17:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 0x55d0a95f06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc700ff6640 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a961bff0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000f7a0 tx=0x7fc6f0020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a107d0498e55b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:47.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc6db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc6db7fe640 1 -- 172.21.15.31:0/1275544529 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0054900 con 0x55d0a95ef320 2023-11-26T07:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc6db7fe640 1 -- 172.21.15.31:0/1275544529 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6f0071070 con 0x55d0a95ef320 2023-11-26T07:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 -- 172.21.15.31:0/1275544529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0a943e110 con 0x55d0a95ef320 2023-11-26T07:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc6db7fe640 1 -- 172.21.15.31:0/1275544529 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6f00819f0 con 0x55d0a95ef320 2023-11-26T07:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 10 client.4647 fetch_fsmap learned FSMap version 8 2023-11-26T07:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 -- 172.21.15.31:0/1275544529 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55d0a9137cc0 con 0x55d0a95ef320 2023-11-26T07:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc6db7fe640 1 -- 172.21.15.31:0/1275544529 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d0a943e110 con 0x55d0a95ef320 2023-11-26T07:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc6db7fe640 1 -- 172.21.15.31:0/1275544529 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc6f0012070 con 0x55d0a95ef320 2023-11-26T07:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 10 client.4647 fetch_fsmap finished waiting for FSMap version 8 2023-11-26T07:17:47.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 10 client.4647 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 20 client.4647 populate_metadata read hostname 'smithi031' 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:6835/527056216,v1:172.21.15.97:6837/527056216] conn(0x55d0a964b3d0 0x55d0a964b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 4 client.4647 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc7065ca000 1 -- 172.21.15.31:0/1275544529 --> [v2:172.21.15.97:6835/527056216,v1:172.21.15.97:6837/527056216] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d0a946d9a0 con 0x55d0a964b3d0 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.501+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:6835/527056216,v1:172.21.15.97:6837/527056216] conn(0x55d0a964b3d0 0x55d0a964b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc6fbfff640 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:6835/527056216,v1:172.21.15.97:6837/527056216] conn(0x55d0a964b3d0 0x55d0a964b7b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e2bf92bd9450fd server_cookie=326d15a3659ba2c9 in_seq=0 out_seq=0 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc6db7fe640 10 client.4647 ms_handle_connect on v2:172.21.15.97:6835/527056216 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc6db7fe640 1 -- 172.21.15.31:0/1275544529 <== mds.0 v2:172.21.15.97:6835/527056216 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55d0a946d9a0 con 0x55d0a964b3d0 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc6db7fe640 10 client.4647 handle_command_reply: tid=0 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 client.4647 shutdown 2023-11-26T07:17:47.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 -- 172.21.15.31:0/1275544529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 msgr2=0x55d0a961bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a961bff0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000f7a0 tx=0x7fc6f0020040 comp rx=0 tx=0).stop 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 -- 172.21.15.31:0/1275544529 shutdown_connections 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:6835/527056216,v1:172.21.15.97:6837/527056216] conn(0x55d0a964b3d0 0x55d0a964b7b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d0a96395b0 0x55d0a95f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0a9638c90 0x55d0a95f06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 --2- 172.21.15.31:0/1275544529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d0a95ef320 0x55d0a961bff0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 -- 172.21.15.31:0/1275544529 >> 172.21.15.31:0/1275544529 conn(0x55d0a95eb2d0 msgr2=0x55d0a95ed6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc6db7fe640 0 client.4647 ms_handle_reset on v2:172.21.15.97:6835/527056216 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 -- 172.21.15.31:0/1275544529 shutdown_connections 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 1 -- 172.21.15.31:0/1275544529 wait complete. 2023-11-26T07:17:47.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc7065ca000 20 client.4647 trim_cache size 0 max 16384 2023-11-26T07:17:47.514 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:17:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "4c2b4df1-71b1-4841-8807-af144d2610e7", 2023-11-26T07:17:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-11-26T07:17:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 4282, 2023-11-26T07:17:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-11-26T07:17:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-11-26T07:17:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-11-26T07:17:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 14.399031115, 2023-11-26T07:17:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 8, 2023-11-26T07:17:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 20, 2023-11-26T07:17:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 20, 2023-11-26T07:17:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 21.610294620000001 2023-11-26T07:17:47.515 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:17:47.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 -- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6e0066470 msgr2=0x7fc6e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6e0066470 0x7fc6e0068930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc161970 tx=0x7fc6e8001120 comp rx=0 tx=0).stop 2023-11-26T07:17:47.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 -- 172.21.15.31:0/4223403161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 msgr2=0x7fc6fc15a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:47.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6fc15a5c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000f7d0 tx=0x7fc6f000fbd0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 -- 172.21.15.31:0/4223403161 shutdown_connections 2023-11-26T07:17:47.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6e0066470 0x7fc6e0068930 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6fc0ad150 0x7fc6fc0ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6fc15cc10 0x7fc6fc15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 --2- 172.21.15.31:0/4223403161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6ec005190 0x7fc6fc15a5c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:47.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 -- 172.21.15.31:0/4223403161 >> 172.21.15.31:0/4223403161 conn(0x7fc6fc0b1d50 msgr2=0x7fc6fc15ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:47.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 -- 172.21.15.31:0/4223403161 shutdown_connections 2023-11-26T07:17:47.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:47.505+0000 7fc70325a640 1 -- 172.21.15.31:0/4223403161 wait complete. 2023-11-26T07:17:47.528 INFO:teuthology.run_tasks:Running task kclient... 2023-11-26T07:17:47.554 INFO:tasks.kclient:Mounting kernel clients... 2023-11-26T07:17:47.554 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-26T07:17:47.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:47.554 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:17:47.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:47.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-11-26T07:17:47.631 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-11-26T07:17:47.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:47.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:47.633 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:17:47.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:47.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-11-26T07:17:47.750 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-11-26T07:17:47.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:47.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:47.753 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:17:47.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:47.764 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-11-26T07:17:47.831 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2023-11-26T07:17:47.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:47.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:47.834 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:17:47.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:47.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-11-26T07:17:47.951 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2023-11-26T07:17:47.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:17:47.954 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-11-26T07:17:47.954 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:17:47.954 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:17:47.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:17:48.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe51aaaa640 1 Processor -- start 2023-11-26T07:17:48.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe51aaaa640 1 -- start start 2023-11-26T07:17:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe51aaaa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe5140cead0 0x7fe5140d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe51aaaa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5140d1eb0 0x7fe51406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe51aaaa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5140d22c0 con 0x7fe51415d7b0 2023-11-26T07:17:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe51aaaa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5140d17e0 con 0x7fe5140cead0 2023-11-26T07:17:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe51aaaa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5140cefd0 con 0x7fe5140d1eb0 2023-11-26T07:17:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe513fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe5140cead0 0x7fe5140d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe518846640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe51415d7b0 0x7fe5140d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54248/0 (socket says 172.21.15.31:54248) 2023-11-26T07:17:48.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe513fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe5140cead0 0x7fe5140d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50246/0 (socket says 172.21.15.31:50246) 2023-11-26T07:17:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe518846640 1 -- 172.21.15.31:0/2182192395 learned_addr learned my addr 172.21.15.31:0/2182192395 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe519047640 1 --2- 172.21.15.31:0/2182192395 >> v2:172.21.15.97:3300/0 conn(0x7fe5140d1eb0 0x7fe51406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe513fff640 1 -- 172.21.15.31:0/2182192395 >> v2:172.21.15.97:3300/0 conn(0x7fe5140d1eb0 msgr2=0x7fe51406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe513fff640 1 --2- 172.21.15.31:0/2182192395 >> v2:172.21.15.97:3300/0 conn(0x7fe5140d1eb0 0x7fe51406bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe513fff640 1 -- 172.21.15.31:0/2182192395 >> v1:172.21.15.31:6789/0 conn(0x7fe51415d7b0 legacy=0x7fe5140d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.093+0000 7fe513fff640 1 -- 172.21.15.31:0/2182192395 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140cf2b0 con 0x7fe5140cead0 2023-11-26T07:17:48.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe513fff640 1 --2- 172.21.15.31:0/2182192395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140cead0 0x7fe5140d1970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe51408c870 tx=0x7fe50001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119673ef12d123c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:48.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe5137fe640 1 -- 172.21.15.31:0/2182192395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5000ac640 con 0x7fe5140cead0 2023-11-26T07:17:48.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe5137fe640 1 -- 172.21.15.31:0/2182192395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5000ac7a0 con 0x7fe5140cead0 2023-11-26T07:17:48.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe5137fe640 1 -- 172.21.15.31:0/2182192395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5000acb30 con 0x7fe5140cead0 2023-11-26T07:17:48.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe51aaaa640 1 -- 172.21.15.31:0/2182192395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140cead0 msgr2=0x7fe5140d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe51aaaa640 1 --2- 172.21.15.31:0/2182192395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140cead0 0x7fe5140d1970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe51408c870 tx=0x7fe50001c310 comp rx=0 tx=0).stop 2023-11-26T07:17:48.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe51aaaa640 1 -- 172.21.15.31:0/2182192395 shutdown_connections 2023-11-26T07:17:48.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe51aaaa640 1 --2- 172.21.15.31:0/2182192395 >> v2:172.21.15.97:3300/0 conn(0x7fe5140d1eb0 0x7fe51406bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe51aaaa640 1 --2- 172.21.15.31:0/2182192395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140cead0 0x7fe5140d1970 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe51aaaa640 1 -- 172.21.15.31:0/2182192395 >> 172.21.15.31:0/2182192395 conn(0x7fe5140d33b0 msgr2=0x7fe5140d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:48.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe51aaaa640 1 -- 172.21.15.31:0/2182192395 shutdown_connections 2023-11-26T07:17:48.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.097+0000 7fe51aaaa640 1 -- 172.21.15.31:0/2182192395 wait complete. 2023-11-26T07:17:48.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe51aaaa640 1 Processor -- start 2023-11-26T07:17:48.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe51aaaa640 1 -- start start 2023-11-26T07:17:48.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe51aaaa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5140cead0 0x7fe514154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe51aaaa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140d1eb0 0x7fe5140b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe51aaaa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51415d7b0 0x7fe5140c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe51aaaa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140c8060 con 0x7fe5140d1eb0 2023-11-26T07:17:48.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe51aaaa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140c8340 con 0x7fe51415d7b0 2023-11-26T07:17:48.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe51aaaa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140c8620 con 0x7fe5140cead0 2023-11-26T07:17:48.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe513fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140d1eb0 0x7fe5140b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe513fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140d1eb0 0x7fe5140b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50252/0 (socket says 172.21.15.31:50252) 2023-11-26T07:17:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe513fff640 1 -- 172.21.15.31:0/3014782872 learned_addr learned my addr 172.21.15.31:0/3014782872 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe519047640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51415d7b0 0x7fe5140c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe518846640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5140cead0 0x7fe514154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe513fff640 1 -- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5140cead0 msgr2=0x7fe514154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe513fff640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5140cead0 0x7fe514154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe513fff640 1 -- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51415d7b0 msgr2=0x7fe5140c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe513fff640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51415d7b0 0x7fe5140c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe513fff640 1 -- 172.21.15.31:0/3014782872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe500009420 con 0x7fe5140d1eb0 2023-11-26T07:17:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe518846640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5140cead0 0x7fe514154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:48.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe513fff640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140d1eb0 0x7fe5140b5f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe5000af040 tx=0x7fe5000190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:48.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe5117fa640 1 -- 172.21.15.31:0/3014782872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5000b5610 con 0x7fe5140d1eb0 2023-11-26T07:17:48.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe5117fa640 1 -- 172.21.15.31:0/3014782872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5000b5790 con 0x7fe5140d1eb0 2023-11-26T07:17:48.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe51aaaa640 1 -- 172.21.15.31:0/3014782872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5140c8900 con 0x7fe5140d1eb0 2023-11-26T07:17:48.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe5117fa640 1 -- 172.21.15.31:0/3014782872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5000b5b40 con 0x7fe5140d1eb0 2023-11-26T07:17:48.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.101+0000 7fe51aaaa640 1 -- 172.21.15.31:0/3014782872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5141558d0 con 0x7fe5140d1eb0 2023-11-26T07:17:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.105+0000 7fe51aaaa640 1 -- 172.21.15.31:0/3014782872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51404e850 con 0x7fe5140d1eb0 2023-11-26T07:17:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.105+0000 7fe5117fa640 1 -- 172.21.15.31:0/3014782872 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe5000b3040 con 0x7fe5140d1eb0 2023-11-26T07:17:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.105+0000 7fe5117fa640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4f8066470 0x7fe4f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.105+0000 7fe5117fa640 1 -- 172.21.15.31:0/3014782872 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe50011e160 con 0x7fe5140d1eb0 2023-11-26T07:17:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.105+0000 7fe518846640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4f8066470 0x7fe4f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:48.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.105+0000 7fe518846640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4f8066470 0x7fe4f8068930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe508005890 tx=0x7fe508009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.109+0000 7fe5117fa640 1 -- 172.21.15.31:0/3014782872 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe5000eb680 con 0x7fe5140d1eb0 2023-11-26T07:17:48.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.305+0000 7fe51aaaa640 1 -- 172.21.15.31:0/3014782872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe5140b19f0 con 0x7fe5140d1eb0 2023-11-26T07:17:48.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.305+0000 7fe5117fa640 1 -- 172.21.15.31:0/3014782872 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe5000eeca0 con 0x7fe5140d1eb0 2023-11-26T07:17:48.311 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:17:48.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 -- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4f8066470 msgr2=0x7fe4f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4f8066470 0x7fe4f8068930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe508005890 tx=0x7fe508009040 comp rx=0 tx=0).stop 2023-11-26T07:17:48.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 -- 172.21.15.31:0/3014782872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140d1eb0 msgr2=0x7fe5140b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:48.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140d1eb0 0x7fe5140b5f70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe5000af040 tx=0x7fe5000190b0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 -- 172.21.15.31:0/3014782872 shutdown_connections 2023-11-26T07:17:48.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4f8066470 0x7fe4f8068930 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51415d7b0 0x7fe5140c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140d1eb0 0x7fe5140b5f70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 --2- 172.21.15.31:0/3014782872 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5140cead0 0x7fe514154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:48.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 -- 172.21.15.31:0/3014782872 >> 172.21.15.31:0/3014782872 conn(0x7fe5140d33b0 msgr2=0x7fe514159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:48.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 -- 172.21.15.31:0/3014782872 shutdown_connections 2023-11-26T07:17:48.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:48.309+0000 7fe51aaaa640 1 -- 172.21.15.31:0/3014782872 wait complete. 2023-11-26T07:17:48.333 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:17:48.334 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:17:48.334 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-11-26T07:17:48.334 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:17:48.334 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:17:48.334 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:17:48.334 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:17:48.334 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:17:48.334 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:17:48.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:48.335 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-11-26T07:17:48.345 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:17:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:17:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:17:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-11-26T07:17:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:48.346 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-11-26T07:17:48.346 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-11-26T07:17:48.347 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:17:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-11-26T07:17:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-26T07:17:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42371sec preferred_lft 42371sec 2023-11-26T07:17:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-11-26T07:17:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:48.347 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-11-26T07:17:48.348 INFO:teuthology.orchestra.run.smithi031.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:17:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 02:42:1b:e4:a9:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:17:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:48.348 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-26T07:17:48.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:48.349 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:17:48.349 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:17:48.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2023-11-26T07:17:48.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2023-11-26T07:17:48.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2023-11-26T07:17:48.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-26T07:17:48.349 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:17:48.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:48.458 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:17:48.478 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-11-26T07:17:48.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:48.480 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-11-26T07:17:48.551 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-11-26T07:17:48.551 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:17:48.552 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-26T07:17:48.552 INFO:teuthology.orchestra.run.smithi031.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:17:48.552 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-26T07:17:48.552 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:17:48.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:48.552 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:17:48.552 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:17:48.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-26T07:17:48.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-26T07:17:48.553 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-26T07:17:48.553 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:17:48.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:48.647 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:17:48.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:48.657 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-11-26T07:17:48.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:48.708 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:17:48.708 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:17:48.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:17:48.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-26T07:17:48.708 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:17:48.791 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-26T07:17:48.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:48.792 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:17:48.792 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:17:48.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-26T07:17:48.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-26T07:17:48.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-26T07:17:48.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-26T07:17:48.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-26T07:17:48.792 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:17:48.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:48.954 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:17:48.954 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:17:48.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-11-26T07:17:48.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2023-11-26T07:17:48.954 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:17:48.996 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:48.996 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:17:49.051 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:17:49.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:49.052 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:17:49.102 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:17:49.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:49.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:49.203 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:49.203 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:49.302 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:49.302 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:49.302 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:17:49.302 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:17:49.302 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:17:49.306 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:17:49.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:49.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:17:49.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:49.328 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:49.407 INFO:teuthology.orchestra.run.smithi031.stdout:client4659 2023-11-26T07:17:49.407 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:49.407 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:49.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:49.459 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:49.525 INFO:teuthology.orchestra.run.smithi031.stdout:client4659 2023-11-26T07:17:49.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:49.526 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:49.585 INFO:teuthology.orchestra.run.smithi031.stdout:client4659 2023-11-26T07:17:49.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:49.586 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4c2b4df1-71b1-4841-8807-af144d2610e7.client4659/status') 2023-11-26T07:17:49.656 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.4659 (3)192.168.0.1:0/3790622287 2023-11-26T07:17:49.657 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2023-11-26T07:17:49.657 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2023-11-26T07:17:49.657 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2023-11-26T07:17:49.657 INFO:teuthology.orchestra.run.smithi031.stderr:69 bytes copied, 5.4169e-05 s, 1.3 MB/s 2023-11-26T07:17:49.659 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-11-26T07:17:49.659 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:17:49.659 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-26T07:17:49.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:17:49.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20ee0ea640 1 Processor -- start 2023-11-26T07:17:49.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20ee0ea640 1 -- start start 2023-11-26T07:17:49.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20ee0ea640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20e815d7b0 0x7f20e80d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20ee0ea640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e80cefd0 con 0x7f20e80d2080 2023-11-26T07:17:49.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20ee0ea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e80cf2b0 con 0x7f20e80cead0 2023-11-26T07:17:49.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20ee0ea640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e806b3f0 con 0x7f20e815d7b0 2023-11-26T07:17:49.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20e77fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20e815d7b0 0x7f20e80d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20e7fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f20e80d2080 0x7f20e806bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54264/0 (socket says 172.21.15.31:54264) 2023-11-26T07:17:49.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20e7fff640 1 -- 172.21.15.31:0/2539541926 learned_addr learned my addr 172.21.15.31:0/2539541926 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20e77fe640 1 -- 172.21.15.31:0/2539541926 >> v1:172.21.15.97:6789/0 conn(0x7f20e80cead0 legacy=0x7f20e80d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20e67fc640 1 -- 172.21.15.31:0/2539541926 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814528666 0 0) 0x7f20e80cefd0 con 0x7f20e80d2080 2023-11-26T07:17:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20e77fe640 1 -- 172.21.15.31:0/2539541926 >> v1:172.21.15.31:6789/0 conn(0x7f20e80d2080 legacy=0x7f20e806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.841+0000 7f20e77fe640 1 -- 172.21.15.31:0/2539541926 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e806b6d0 con 0x7f20e815d7b0 2023-11-26T07:17:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20e77fe640 1 --2- 172.21.15.31:0/2539541926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e815d7b0 0x7f20e80d10d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f20d80097c0 tx=0x7f20d801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72c09db5bc9a990 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20e67fc640 1 -- 172.21.15.31:0/2539541926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8004230 con 0x7f20e815d7b0 2023-11-26T07:17:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20e67fc640 1 -- 172.21.15.31:0/2539541926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d80a7050 con 0x7f20e815d7b0 2023-11-26T07:17:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20e67fc640 1 -- 172.21.15.31:0/2539541926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d8003620 con 0x7f20e815d7b0 2023-11-26T07:17:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2539541926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e815d7b0 msgr2=0x7f20e80d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20ee0ea640 1 --2- 172.21.15.31:0/2539541926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e815d7b0 0x7f20e80d10d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f20d80097c0 tx=0x7f20d801c330 comp rx=0 tx=0).stop 2023-11-26T07:17:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2539541926 shutdown_connections 2023-11-26T07:17:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20ee0ea640 1 --2- 172.21.15.31:0/2539541926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e815d7b0 0x7f20e80d10d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2539541926 >> 172.21.15.31:0/2539541926 conn(0x7f20e80d33b0 msgr2=0x7f20e80d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2539541926 shutdown_connections 2023-11-26T07:17:49.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.845+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2539541926 wait complete. 2023-11-26T07:17:49.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20ee0ea640 1 Processor -- start 2023-11-26T07:17:49.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20ee0ea640 1 -- start start 2023-11-26T07:17:49.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20ee0ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20e80cead0 0x7f20e80c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20ee0ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e80d2080 0x7f20e80c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20ee0ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e815d7b0 0x7f20e81519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20ee0ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80b6c20 con 0x7f20e80d2080 2023-11-26T07:17:49.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20ee0ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80c9ce0 con 0x7f20e815d7b0 2023-11-26T07:17:49.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20ee0ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e80b6010 con 0x7f20e80cead0 2023-11-26T07:17:49.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e80d2080 0x7f20e80c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e80d2080 0x7f20e80c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50270/0 (socket says 172.21.15.31:50270) 2023-11-26T07:17:49.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e6ffd640 1 -- 172.21.15.31:0/2892574295 learned_addr learned my addr 172.21.15.31:0/2892574295 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:49.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e7fff640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e815d7b0 0x7f20e81519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e77fe640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20e80cead0 0x7f20e80c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e6ffd640 1 -- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20e80cead0 msgr2=0x7f20e80c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:49.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e6ffd640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20e80cead0 0x7f20e80c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e6ffd640 1 -- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e815d7b0 msgr2=0x7f20e81519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:49.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e6ffd640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e815d7b0 0x7f20e81519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:49.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e6ffd640 1 -- 172.21.15.31:0/2892574295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d8009420 con 0x7f20e80d2080 2023-11-26T07:17:49.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20e6ffd640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e80d2080 0x7f20e80c97a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c340 tx=0x7f20dc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:49.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20cffff640 1 -- 172.21.15.31:0/2892574295 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0052d0 con 0x7f20e80d2080 2023-11-26T07:17:49.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20cffff640 1 -- 172.21.15.31:0/2892574295 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc005430 con 0x7f20e80d2080 2023-11-26T07:17:49.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20cffff640 1 -- 172.21.15.31:0/2892574295 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc007260 con 0x7f20e80d2080 2023-11-26T07:17:49.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2892574295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e80b62f0 con 0x7f20e80d2080 2023-11-26T07:17:49.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.849+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2892574295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e80b68b0 con 0x7f20e80d2080 2023-11-26T07:17:49.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.853+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2892574295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e804e850 con 0x7f20e80d2080 2023-11-26T07:17:49.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.853+0000 7f20cffff640 1 -- 172.21.15.31:0/2892574295 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f20dc0073c0 con 0x7f20e80d2080 2023-11-26T07:17:49.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.853+0000 7f20cffff640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20d4066470 0x7f20d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:49.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.853+0000 7f20cffff640 1 -- 172.21.15.31:0/2892574295 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20dc083200 con 0x7f20e80d2080 2023-11-26T07:17:49.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.853+0000 7f20e77fe640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20d4066470 0x7f20d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:49.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.853+0000 7f20e77fe640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20d4066470 0x7f20d4068930 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f20d80ad000 tx=0x7f20d80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:49.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:49.857+0000 7f20cffff640 1 -- 172.21.15.31:0/2892574295 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f20dc050690 con 0x7f20e80d2080 2023-11-26T07:17:50.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2892574295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f20e80c6800 con 0x7f20e80d2080 2023-11-26T07:17:50.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20cffff640 1 -- 172.21.15.31:0/2892574295 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f20dc053cb0 con 0x7f20e80d2080 2023-11-26T07:17:50.056 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20d4066470 msgr2=0x7f20d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20d4066470 0x7f20d4068930 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f20d80ad000 tx=0x7f20d80a6000 comp rx=0 tx=0).stop 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2892574295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e80d2080 msgr2=0x7f20e80c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e80d2080 0x7f20e80c97a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00c340 tx=0x7f20dc00c800 comp rx=0 tx=0).stop 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2892574295 shutdown_connections 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20d4066470 0x7f20d4068930 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20e815d7b0 0x7f20e81519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20e80d2080 0x7f20e80c97a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 --2- 172.21.15.31:0/2892574295 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20e80cead0 0x7f20e80c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2892574295 >> 172.21.15.31:0/2892574295 conn(0x7f20e80d33b0 msgr2=0x7f20e80cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:50.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2892574295 shutdown_connections 2023-11-26T07:17:50.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:50.053+0000 7f20ee0ea640 1 -- 172.21.15.31:0/2892574295 wait complete. 2023-11-26T07:17:50.078 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:17:50.078 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:17:50.078 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-11-26T07:17:50.079 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-26T07:17:50.079 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-26T07:17:50.079 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:17:50.079 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:17:50.079 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:17:50.079 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-26T07:17:50.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.079 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:50.089 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42370sec preferred_lft 42370sec 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-11-26T07:17:50.090 INFO:teuthology.orchestra.run.smithi031.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 02:42:1b:e4:a9:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:17:50.091 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-11-26T07:17:50.092 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.092 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:17:50.158 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-11-26T07:17:50.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.160 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-11-26T07:17:50.216 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-11-26T07:17:50.217 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:17:50.217 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-26T07:17:50.217 INFO:teuthology.orchestra.run.smithi031.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:17:50.217 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-26T07:17:50.217 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:17:50.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.217 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:17:50.217 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:17:50.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-26T07:17:50.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-26T07:17:50.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-26T07:17:50.218 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:17:50.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.312 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:17:50.321 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:17:50.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.322 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-11-26T07:17:50.372 INFO:teuthology.orchestra.run.smithi031.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-26T07:17:50.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.373 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:17:50.373 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:17:50.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:17:50.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-26T07:17:50.373 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:17:50.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-26T07:17:50.524 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:17:50.524 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:17:50.524 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:17:50.524 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.525 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-11-26T07:17:50.525 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.525 INFO:teuthology.orchestra.run.smithi031.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:17:50.525 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0e:2a:ba:6c:d8:24 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:17:50.525 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:17:50.525 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.525 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c2a:baff:fe6c:d824/64 scope link 2023-11-26T07:17:50.525 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-26T07:17:50.588 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:17:50.588 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:17:50.589 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:17:50.589 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.589 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-11-26T07:17:50.589 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.589 INFO:teuthology.orchestra.run.smithi031.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:17:50.589 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0e:2a:ba:6c:d8:24 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:17:50.589 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:17:50.589 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.589 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c2a:baff:fe6c:d824/64 scope link 2023-11-26T07:17:50.589 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:50.590 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-26T07:17:50.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.591 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:17:50.591 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:17:50.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-26T07:17:50.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-26T07:17:50.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-26T07:17:50.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-26T07:17:50.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-26T07:17:50.591 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:17:50.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.728 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:17:50.728 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:17:50.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.1 up 2023-11-26T07:17:50.728 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.1 master ceph-brx 2023-11-26T07:17:50.728 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:17:50.769 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:50.769 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-26T07:17:50.819 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-26T07:17:50.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:50.820 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-26T07:17:50.871 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-11-26T07:17:50.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:50.966 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:50.966 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:50.979 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:50.979 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:50.979 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:17:50.980 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 2@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:17:50.980 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:17:50.983 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:17:50.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:50.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-26T07:17:51.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:51.004 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:51.062 INFO:teuthology.orchestra.run.smithi031.stdout:client4517 2023-11-26T07:17:51.063 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:17:51.063 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:51.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:51.111 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:51.167 INFO:teuthology.orchestra.run.smithi031.stdout:client4517 2023-11-26T07:17:51.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:51.167 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:51.222 INFO:teuthology.orchestra.run.smithi031.stdout:client4517 2023-11-26T07:17:51.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:51.223 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4c2b4df1-71b1-4841-8807-af144d2610e7.client4517/status') 2023-11-26T07:17:51.284 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.4517 (3)172.21.15.31:0/3728517982 2023-11-26T07:17:51.284 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2023-11-26T07:17:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2023-11-26T07:17:51.284 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2023-11-26T07:17:51.285 INFO:teuthology.orchestra.run.smithi031.stderr:70 bytes copied, 5.339e-05 s, 1.3 MB/s 2023-11-26T07:17:51.286 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-11-26T07:17:51.286 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:17:51.286 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:17:51.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:17:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d24832640 1 Processor -- start 2023-11-26T07:17:51.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d24832640 1 -- start start 2023-11-26T07:17:51.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d24832640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d2006e410 0x7f4d200715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d24832640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d20071af0 0x7f4d200b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d24832640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d200ad740 con 0x7f4d200b2ee0 2023-11-26T07:17:51.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d24832640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d200b3e20 con 0x7f4d2006e410 2023-11-26T07:17:51.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d24832640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2015dde0 con 0x7f4d20071af0 2023-11-26T07:17:51.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d1ed9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4d200b2ee0 0x7f4d20161030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54274/0 (socket says 172.21.15.31:54274) 2023-11-26T07:17:51.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d1ed9d640 1 -- 172.21.15.31:0/3316081209 learned_addr learned my addr 172.21.15.31:0/3316081209 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:51.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d1e59c640 1 --2- 172.21.15.31:0/3316081209 >> v2:172.21.15.97:3300/0 conn(0x7f4d2006e410 0x7f4d200715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d17fff640 1 --2- 172.21.15.31:0/3316081209 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d20071af0 0x7f4d200b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d1dd9b640 1 -- 172.21.15.31:0/3316081209 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699936076 0 0) 0x7f4d200ad740 con 0x7f4d200b2ee0 2023-11-26T07:17:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d1dd9b640 1 -- 172.21.15.31:0/3316081209 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d0c003530 con 0x7f4d200b2ee0 2023-11-26T07:17:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d1e59c640 1 -- 172.21.15.31:0/3316081209 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d20071af0 msgr2=0x7f4d200b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d1e59c640 1 --2- 172.21.15.31:0/3316081209 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d20071af0 0x7f4d200b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d1e59c640 1 -- 172.21.15.31:0/3316081209 >> v1:172.21.15.31:6789/0 conn(0x7f4d200b2ee0 legacy=0x7f4d20161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.469+0000 7f4d1e59c640 1 -- 172.21.15.31:0/3316081209 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d200b3580 con 0x7f4d2006e410 2023-11-26T07:17:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d1e59c640 1 --2- 172.21.15.31:0/3316081209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2006e410 0x7f4d200715b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d100097c0 tx=0x7f4d1001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99937085d73581cf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d1dd9b640 1 -- 172.21.15.31:0/3316081209 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d100ab600 con 0x7f4d2006e410 2023-11-26T07:17:51.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d1dd9b640 1 -- 172.21.15.31:0/3316081209 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d100a7050 con 0x7f4d2006e410 2023-11-26T07:17:51.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d1dd9b640 1 -- 172.21.15.31:0/3316081209 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d100ab980 con 0x7f4d2006e410 2023-11-26T07:17:51.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d24832640 1 -- 172.21.15.31:0/3316081209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2006e410 msgr2=0x7f4d200715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d24832640 1 --2- 172.21.15.31:0/3316081209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2006e410 0x7f4d200715b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d100097c0 tx=0x7f4d1001c480 comp rx=0 tx=0).stop 2023-11-26T07:17:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d24832640 1 -- 172.21.15.31:0/3316081209 shutdown_connections 2023-11-26T07:17:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d24832640 1 --2- 172.21.15.31:0/3316081209 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d20071af0 0x7f4d200b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d24832640 1 --2- 172.21.15.31:0/3316081209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2006e410 0x7f4d200715b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d24832640 1 -- 172.21.15.31:0/3316081209 >> 172.21.15.31:0/3316081209 conn(0x7f4d200b1e40 msgr2=0x7f4d20070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d24832640 1 -- 172.21.15.31:0/3316081209 shutdown_connections 2023-11-26T07:17:51.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.473+0000 7f4d24832640 1 -- 172.21.15.31:0/3316081209 wait complete. 2023-11-26T07:17:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d24832640 1 Processor -- start 2023-11-26T07:17:51.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d24832640 1 -- start start 2023-11-26T07:17:51.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d24832640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2006e410 0x7f4d200ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d24832640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d20071af0 0x7f4d2014cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d24832640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d200b2ee0 0x7f4d2013cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d24832640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2014c650 con 0x7f4d20071af0 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d24832640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2014dce0 con 0x7f4d2006e410 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d24832640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2014c090 con 0x7f4d200b2ee0 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d17fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d20071af0 0x7f4d2014cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d17fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d20071af0 0x7f4d2014cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50280/0 (socket says 172.21.15.31:50280) 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d17fff640 1 -- 172.21.15.31:0/181514676 learned_addr learned my addr 172.21.15.31:0/181514676 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d1e59c640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2006e410 0x7f4d200ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d1ed9d640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d200b2ee0 0x7f4d2013cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d17fff640 1 -- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d200b2ee0 msgr2=0x7f4d2013cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d17fff640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d200b2ee0 0x7f4d2013cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d17fff640 1 -- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2006e410 msgr2=0x7f4d200ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d17fff640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2006e410 0x7f4d200ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d17fff640 1 -- 172.21.15.31:0/181514676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d10009420 con 0x7f4d20071af0 2023-11-26T07:17:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d17fff640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d20071af0 0x7f4d2014cef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800e8a0 tx=0x7f4d0800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d16ffd640 1 -- 172.21.15.31:0/181514676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08009510 con 0x7f4d20071af0 2023-11-26T07:17:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d16ffd640 1 -- 172.21.15.31:0/181514676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0800c940 con 0x7f4d20071af0 2023-11-26T07:17:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d16ffd640 1 -- 172.21.15.31:0/181514676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800ccb0 con 0x7f4d20071af0 2023-11-26T07:17:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d24832640 1 -- 172.21.15.31:0/181514676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2014c370 con 0x7f4d20071af0 2023-11-26T07:17:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.477+0000 7f4d24832640 1 -- 172.21.15.31:0/181514676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2014bc80 con 0x7f4d20071af0 2023-11-26T07:17:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.481+0000 7f4d24832640 1 -- 172.21.15.31:0/181514676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2004e850 con 0x7f4d20071af0 2023-11-26T07:17:51.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.481+0000 7f4d16ffd640 1 -- 172.21.15.31:0/181514676 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4d08013470 con 0x7f4d20071af0 2023-11-26T07:17:51.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.481+0000 7f4d16ffd640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d00066470 0x7f4d00068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.481+0000 7f4d16ffd640 1 -- 172.21.15.31:0/181514676 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d08083f00 con 0x7f4d20071af0 2023-11-26T07:17:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.481+0000 7f4d1e59c640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d00066470 0x7f4d00068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:51.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.481+0000 7f4d1e59c640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d00066470 0x7f4d00068930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4d100ad000 tx=0x7f4d100a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:51.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.485+0000 7f4d16ffd640 1 -- 172.21.15.31:0/181514676 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4d08051420 con 0x7f4d20071af0 2023-11-26T07:17:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.673+0000 7f4d24832640 1 -- 172.21.15.31:0/181514676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d2014de20 con 0x7f4d20071af0 2023-11-26T07:17:51.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.673+0000 7f4d16ffd640 1 -- 172.21.15.31:0/181514676 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d08054a40 con 0x7f4d20071af0 2023-11-26T07:17:51.679 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:17:51.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 -- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d00066470 msgr2=0x7f4d00068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d00066470 0x7f4d00068930 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4d100ad000 tx=0x7f4d100a6000 comp rx=0 tx=0).stop 2023-11-26T07:17:51.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 -- 172.21.15.31:0/181514676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d20071af0 msgr2=0x7f4d2014cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:51.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d20071af0 0x7f4d2014cef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800e8a0 tx=0x7f4d0800eca0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 -- 172.21.15.31:0/181514676 shutdown_connections 2023-11-26T07:17:51.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d00066470 0x7f4d00068930 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d200b2ee0 0x7f4d2013cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d20071af0 0x7f4d2014cef0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 --2- 172.21.15.31:0/181514676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d2006e410 0x7f4d200ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:51.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 -- 172.21.15.31:0/181514676 >> 172.21.15.31:0/181514676 conn(0x7f4d200b1e40 msgr2=0x7f4d2006d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:51.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 -- 172.21.15.31:0/181514676 shutdown_connections 2023-11-26T07:17:51.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:51.677+0000 7f4d24832640 1 -- 172.21.15.31:0/181514676 wait complete. 2023-11-26T07:17:51.700 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:17:51.700 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:17:51.700 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-26T07:17:51.700 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:17:51.701 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:17:51.701 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:17:51.701 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:17:51.701 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:17:51.701 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:17:51.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:51.701 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-26T07:17:51.711 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:51.712 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42391sec preferred_lft 42391sec 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:61:bc:5b:3d brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:51.713 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:17:51.714 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:51.714 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-26T07:17:51.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:51.715 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:17:51.715 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:17:51.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2023-11-26T07:17:51.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2023-11-26T07:17:51.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2023-11-26T07:17:51.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-26T07:17:51.715 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:17:51.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:51.835 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:17:51.854 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-26T07:17:51.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:51.857 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-26T07:17:51.921 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-26T07:17:51.921 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:17:51.921 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:17:51.921 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:17:51.921 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:17:51.921 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:17:51.921 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:17:51.922 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:17:51.922 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:17:51.922 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:17:51.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:51.922 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:17:51.922 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:17:51.922 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:17:51.922 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:17:51.922 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:17:51.922 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:17:52.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.020 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:17:52.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.030 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2023-11-26T07:17:52.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.084 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:17:52.084 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:17:52.084 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:17:52.084 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-26T07:17:52.084 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:17:52.170 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-26T07:17:52.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.171 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:17:52.171 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:17:52.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-26T07:17:52.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-26T07:17:52.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-26T07:17:52.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-26T07:17:52.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-26T07:17:52.171 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:17:52.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.339 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:17:52.339 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:17:52.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-26T07:17:52.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2023-11-26T07:17:52.339 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:17:52.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:52.380 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:17:52.435 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:17:52.435 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:52.435 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:17:52.486 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:17:52.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:52.588 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:52.588 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:52.689 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:52.690 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:52.690 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:17:52.690 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 1@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:17:52.690 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:17:52.693 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:17:52.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.694 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:17:52.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.714 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:52.780 INFO:teuthology.orchestra.run.smithi097.stdout:client4677 2023-11-26T07:17:52.780 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:52.780 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:52.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.831 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:52.885 INFO:teuthology.orchestra.run.smithi097.stdout:client4677 2023-11-26T07:17:52.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.886 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:52.937 INFO:teuthology.orchestra.run.smithi097.stdout:client4677 2023-11-26T07:17:52.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:52.938 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4c2b4df1-71b1-4841-8807-af144d2610e7.client4677/status') 2023-11-26T07:17:52.999 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.4677 (3)172.21.15.97:0/3846431474 2023-11-26T07:17:52.999 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2023-11-26T07:17:52.999 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2023-11-26T07:17:53.000 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2023-11-26T07:17:53.000 INFO:teuthology.orchestra.run.smithi097.stderr:70 bytes copied, 5.6724e-05 s, 1.2 MB/s 2023-11-26T07:17:53.001 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-11-26T07:17:53.001 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-26T07:17:53.001 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-26T07:17:53.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:17:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.149+0000 7f7928632640 1 Processor -- start 2023-11-26T07:17:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7928632640 1 -- start start 2023-11-26T07:17:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7928632640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920144c80 0x7f79200abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7928632640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7920070df0 0x7f792006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7928632640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7920145060 con 0x7f79200ac460 2023-11-26T07:17:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7928632640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7920071250 con 0x7f7920070df0 2023-11-26T07:17:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7928632640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7920071fd0 con 0x7f7920144c80 2023-11-26T07:17:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7926bcf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7920070df0 0x7f792006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7925bcd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f79200ac460 0x7f79200706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54288/0 (socket says 172.21.15.31:54288) 2023-11-26T07:17:53.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7926bcf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7920070df0 0x7f792006c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50284/0 (socket says 172.21.15.31:50284) 2023-11-26T07:17:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7925bcd640 1 -- 172.21.15.31:0/244444193 learned_addr learned my addr 172.21.15.31:0/244444193 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f79263ce640 1 --2- 172.21.15.31:0/244444193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920144c80 0x7f79200abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7926bcf640 1 -- 172.21.15.31:0/244444193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920144c80 msgr2=0x7f79200abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7926bcf640 1 --2- 172.21.15.31:0/244444193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920144c80 0x7f79200abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7926bcf640 1 -- 172.21.15.31:0/244444193 >> v1:172.21.15.31:6789/0 conn(0x7f79200ac460 legacy=0x7f79200706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f7926bcf640 1 -- 172.21.15.31:0/244444193 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200ac8c0 con 0x7f7920070df0 2023-11-26T07:17:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f79253cc640 1 -- 172.21.15.31:0/244444193 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958096593 0 0) 0x7f7920145060 con 0x7f79200ac460 2023-11-26T07:17:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.153+0000 7f79263ce640 1 --2- 172.21.15.31:0/244444193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920144c80 0x7f79200abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f7926bcf640 1 --2- 172.21.15.31:0/244444193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920070df0 0x7f792006c910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7918009d90 tx=0x7f791801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5306eb6c8a7ab3a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f79253cc640 1 -- 172.21.15.31:0/244444193 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180a59e0 con 0x7f7920070df0 2023-11-26T07:17:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f79253cc640 1 -- 172.21.15.31:0/244444193 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79180a5b40 con 0x7f7920070df0 2023-11-26T07:17:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f79253cc640 1 -- 172.21.15.31:0/244444193 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180ad440 con 0x7f7920070df0 2023-11-26T07:17:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f7928632640 1 -- 172.21.15.31:0/244444193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920070df0 msgr2=0x7f792006c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f7928632640 1 --2- 172.21.15.31:0/244444193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920070df0 0x7f792006c910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7918009d90 tx=0x7f791801c920 comp rx=0 tx=0).stop 2023-11-26T07:17:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f7928632640 1 -- 172.21.15.31:0/244444193 shutdown_connections 2023-11-26T07:17:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f7928632640 1 --2- 172.21.15.31:0/244444193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920070df0 0x7f792006c910 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f7928632640 1 --2- 172.21.15.31:0/244444193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920144c80 0x7f79200abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f7928632640 1 -- 172.21.15.31:0/244444193 >> 172.21.15.31:0/244444193 conn(0x7f79200b0c40 msgr2=0x7f79200b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f7928632640 1 -- 172.21.15.31:0/244444193 shutdown_connections 2023-11-26T07:17:53.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.157+0000 7f7928632640 1 -- 172.21.15.31:0/244444193 wait complete. 2023-11-26T07:17:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 Processor -- start 2023-11-26T07:17:53.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 -- start start 2023-11-26T07:17:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920070df0 0x7f792012cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79200ac460 0x7f7920161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920144c80 0x7f792015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792012d450 con 0x7f7920144c80 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920140cf0 con 0x7f79200ac460 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7920140fd0 con 0x7f7920070df0 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7926bcf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920144c80 0x7f792015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7926bcf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920144c80 0x7f792015a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50298/0 (socket says 172.21.15.31:50298) 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7926bcf640 1 -- 172.21.15.31:0/2597282026 learned_addr learned my addr 172.21.15.31:0/2597282026 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f79263ce640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920070df0 0x7f792012cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7925bcd640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79200ac460 0x7f7920161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7926bcf640 1 -- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920070df0 msgr2=0x7f792012cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7926bcf640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920070df0 0x7f792012cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7926bcf640 1 -- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79200ac460 msgr2=0x7f7920161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7926bcf640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79200ac460 0x7f7920161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7926bcf640 1 -- 172.21.15.31:0/2597282026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7918009a70 con 0x7f7920144c80 2023-11-26T07:17:53.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7926bcf640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920144c80 0x7f792015a8f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f791801a550 tx=0x7f79180196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7916ffd640 1 -- 172.21.15.31:0/2597282026 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180a59e0 con 0x7f7920144c80 2023-11-26T07:17:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7916ffd640 1 -- 172.21.15.31:0/2597282026 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7918009450 con 0x7f7920144c80 2023-11-26T07:17:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7916ffd640 1 -- 172.21.15.31:0/2597282026 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918003ba0 con 0x7f7920144c80 2023-11-26T07:17:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 -- 172.21.15.31:0/2597282026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79201412b0 con 0x7f7920144c80 2023-11-26T07:17:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 -- 172.21.15.31:0/2597282026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7920141870 con 0x7f7920144c80 2023-11-26T07:17:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.161+0000 7f7928632640 1 -- 172.21.15.31:0/2597282026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79200b1ba0 con 0x7f7920144c80 2023-11-26T07:17:53.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.165+0000 7f7916ffd640 1 -- 172.21.15.31:0/2597282026 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f79180b4050 con 0x7f7920144c80 2023-11-26T07:17:53.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.165+0000 7f7916ffd640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7904066470 0x7f7904068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:53.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.165+0000 7f7916ffd640 1 -- 172.21.15.31:0/2597282026 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f791811efe0 con 0x7f7920144c80 2023-11-26T07:17:53.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.165+0000 7f79263ce640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7904066470 0x7f7904068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:53.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.165+0000 7f79263ce640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7904066470 0x7f7904068930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f791c0045f0 tx=0x7f791c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:53.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.169+0000 7f7916ffd640 1 -- 172.21.15.31:0/2597282026 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79180ec500 con 0x7f7920144c80 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 -- 172.21.15.31:0/2597282026 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7920141a60 con 0x7f7920144c80 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7916ffd640 1 -- 172.21.15.31:0/2597282026 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f79180b2070 con 0x7f7920144c80 2023-11-26T07:17:53.364 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:17:53.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 -- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7904066470 msgr2=0x7f7904068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7904066470 0x7f7904068930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f791c0045f0 tx=0x7f791c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 -- 172.21.15.31:0/2597282026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920144c80 msgr2=0x7f792015a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920144c80 0x7f792015a8f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f791801a550 tx=0x7f79180196c0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 -- 172.21.15.31:0/2597282026 shutdown_connections 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7904066470 0x7f7904068930 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7920144c80 0x7f792015a8f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79200ac460 0x7f7920161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 --2- 172.21.15.31:0/2597282026 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7920070df0 0x7f792012cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 -- 172.21.15.31:0/2597282026 >> 172.21.15.31:0/2597282026 conn(0x7f79200b0c40 msgr2=0x7f79200af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 -- 172.21.15.31:0/2597282026 shutdown_connections 2023-11-26T07:17:53.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:53.361+0000 7f7928632640 1 -- 172.21.15.31:0/2597282026 wait complete. 2023-11-26T07:17:53.385 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-26T07:17:53.385 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:17:53.385 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-26T07:17:53.385 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-26T07:17:53.386 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-26T07:17:53.386 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-26T07:17:53.386 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-26T07:17:53.386 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:17:53.386 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-26T07:17:53.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:53.386 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-26T07:17:53.395 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:17:53.396 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:17:53.396 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:17:53.396 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.396 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-26T07:17:53.396 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.396 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:53.396 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:53.396 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-11-26T07:17:53.396 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:53.396 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42389sec preferred_lft 42389sec 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:61:bc:5b:3d brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:17:53.397 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.398 INFO:teuthology.orchestra.run.smithi097.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:17:53.398 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:17:53.398 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:17:53.398 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.398 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-11-26T07:17:53.398 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.398 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:17:53.398 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:17:53.398 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-11-26T07:17:53.398 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:53.399 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:17:53.461 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-26T07:17:53.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:53.464 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-26T07:17:53.522 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-26T07:17:53.522 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:17:53.522 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:17:53.523 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:17:53.523 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:17:53.523 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:17:53.523 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:17:53.523 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:17:53.523 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:17:53.523 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:17:53.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:53.523 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:17:53.523 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:17:53.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:17:53.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:17:53.524 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:17:53.524 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:17:53.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:53.618 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:17:53.627 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:17:53.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:53.628 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2023-11-26T07:17:53.679 INFO:teuthology.orchestra.run.smithi097.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-26T07:17:53.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:53.680 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:17:53.680 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:17:53.680 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:17:53.680 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-26T07:17:53.680 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:17:53.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:53.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-26T07:17:53.827 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:17:53.827 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:17:53.827 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:17:53.827 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.827 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-26T07:17:53.827 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.827 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:17:53.828 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 16:e6:e0:08:6c:57 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:17:53.828 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:17:53.828 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.828 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::14e6:e0ff:fe08:6c57/64 scope link tentative 2023-11-26T07:17:53.828 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:53.829 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-26T07:17:53.891 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:17:53.891 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:17:53.891 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:17:53.891 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.891 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-26T07:17:53.891 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.891 INFO:teuthology.orchestra.run.smithi097.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:17:53.892 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 16:e6:e0:08:6c:57 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-26T07:17:53.892 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-26T07:17:53.892 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.892 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::14e6:e0ff:fe08:6c57/64 scope link tentative 2023-11-26T07:17:53.892 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:17:53.893 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-26T07:17:53.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:53.893 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:17:53.893 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:17:53.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-26T07:17:53.893 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-26T07:17:53.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-26T07:17:53.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-26T07:17:53.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-26T07:17:53.894 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:17:54.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:54.034 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:17:54.035 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:17:54.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.1 up 2023-11-26T07:17:54.035 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.1 master ceph-brx 2023-11-26T07:17:54.035 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:17:54.075 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:54.076 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-26T07:17:54.131 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-26T07:17:54.131 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:17:54.131 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-26T07:17:54.182 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-11-26T07:17:54.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:54.182 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:54.272 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:54.272 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:54.285 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:17:54.285 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:17:54.285 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:17:54.286 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 3@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:17:54.286 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:17:54.290 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:17:54.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:54.290 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-26T07:17:54.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:54.310 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:54.361 INFO:teuthology.orchestra.run.smithi097.stdout:client4657 2023-11-26T07:17:54.362 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:17:54.362 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:17:54.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:54.415 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:54.469 INFO:teuthology.orchestra.run.smithi097.stdout:client4657 2023-11-26T07:17:54.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:54.469 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:17:54.521 INFO:teuthology.orchestra.run.smithi097.stdout:client4657 2023-11-26T07:17:54.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:17:54.521 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/4c2b4df1-71b1-4841-8807-af144d2610e7.client4657/status') 2023-11-26T07:17:54.582 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.4657 (3)192.168.0.2:0/3380421552 2023-11-26T07:17:54.582 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2023-11-26T07:17:54.582 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2023-11-26T07:17:54.582 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2023-11-26T07:17:54.582 INFO:teuthology.orchestra.run.smithi097.stderr:69 bytes copied, 4.8444e-05 s, 1.4 MB/s 2023-11-26T07:17:54.584 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-26T07:17:54.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb08513f640 1 Processor -- start 2023-11-26T07:17:54.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb08513f640 1 -- start start 2023-11-26T07:17:54.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb08513f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb0800b2850 0x7fb0800cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb08513f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08006d4f0 con 0x7fb080160980 2023-11-26T07:17:54.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb08513f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0800d2070 con 0x7fb08006bc90 2023-11-26T07:17:54.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb08513f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08006c150 con 0x7fb0800b2850 2023-11-26T07:17:54.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb07ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb080160980 0x7fb08006b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49050/0 (socket says 172.21.15.31:49050) 2023-11-26T07:17:54.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb07ffff640 1 -- 172.21.15.31:0/657219763 learned_addr learned my addr 172.21.15.31:0/657219763 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:54.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb08493e640 1 --2- 172.21.15.31:0/657219763 >> v2:172.21.15.31:3300/0 conn(0x7fb0800b2850 0x7fb0800cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb07effd640 1 -- 172.21.15.31:0/657219763 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310030231 0 0) 0x7fb08006d4f0 con 0x7fb080160980 2023-11-26T07:17:54.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb07effd640 1 -- 172.21.15.31:0/657219763 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb068003530 con 0x7fb080160980 2023-11-26T07:17:54.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb08493e640 1 -- 172.21.15.31:0/657219763 >> v1:172.21.15.97:6789/0 conn(0x7fb08006bc90 legacy=0x7fb0800b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb08493e640 1 -- 172.21.15.31:0/657219763 >> v1:172.21.15.31:6789/0 conn(0x7fb080160980 legacy=0x7fb08006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.745+0000 7fb08493e640 1 -- 172.21.15.31:0/657219763 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08006c430 con 0x7fb0800b2850 2023-11-26T07:17:54.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb08493e640 1 --2- 172.21.15.31:0/657219763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0800b2850 0x7fb0800cdc00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb078009d90 tx=0x7fb07801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7eba06b3d373204 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb07effd640 1 -- 172.21.15.31:0/657219763 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780a59e0 con 0x7fb0800b2850 2023-11-26T07:17:54.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb07effd640 1 -- 172.21.15.31:0/657219763 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0780a5b40 con 0x7fb0800b2850 2023-11-26T07:17:54.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb07effd640 1 -- 172.21.15.31:0/657219763 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780ad440 con 0x7fb0800b2850 2023-11-26T07:17:54.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb08513f640 1 -- 172.21.15.31:0/657219763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0800b2850 msgr2=0x7fb0800cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb08513f640 1 --2- 172.21.15.31:0/657219763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0800b2850 0x7fb0800cdc00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb078009d90 tx=0x7fb07801c920 comp rx=0 tx=0).stop 2023-11-26T07:17:54.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb08513f640 1 -- 172.21.15.31:0/657219763 shutdown_connections 2023-11-26T07:17:54.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb08513f640 1 --2- 172.21.15.31:0/657219763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0800b2850 0x7fb0800cdc00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb08513f640 1 -- 172.21.15.31:0/657219763 >> 172.21.15.31:0/657219763 conn(0x7fb08009c5c0 msgr2=0x7fb0800d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb08513f640 1 -- 172.21.15.31:0/657219763 shutdown_connections 2023-11-26T07:17:54.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.749+0000 7fb08513f640 1 -- 172.21.15.31:0/657219763 wait complete. 2023-11-26T07:17:54.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 Processor -- start 2023-11-26T07:17:54.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 -- start start 2023-11-26T07:17:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08006bc90 0x7fb0800c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0800b2850 0x7fb0800b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb080160980 0x7fb0800b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0800c93b0 con 0x7fb08006bc90 2023-11-26T07:17:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0800b7a20 con 0x7fb0800b2850 2023-11-26T07:17:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0800b7d00 con 0x7fb080160980 2023-11-26T07:17:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08006bc90 0x7fb0800c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08006bc90 0x7fb0800c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40248/0 (socket says 172.21.15.31:40248) 2023-11-26T07:17:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07ffff640 1 -- 172.21.15.31:0/3252754892 learned_addr learned my addr 172.21.15.31:0/3252754892 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08493e640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb080160980 0x7fb0800b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07f7fe640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0800b2850 0x7fb0800b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07ffff640 1 -- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb080160980 msgr2=0x7fb0800b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07ffff640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb080160980 0x7fb0800b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07ffff640 1 -- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0800b2850 msgr2=0x7fb0800b70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07ffff640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0800b2850 0x7fb0800b70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07ffff640 1 -- 172.21.15.31:0/3252754892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078009a70 con 0x7fb08006bc90 2023-11-26T07:17:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07ffff640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08006bc90 0x7fb0800c8e70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb07000c420 tx=0x7fb07000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07cff9640 1 -- 172.21.15.31:0/3252754892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb070005cf0 con 0x7fb08006bc90 2023-11-26T07:17:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07cff9640 1 -- 172.21.15.31:0/3252754892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb070005e50 con 0x7fb08006bc90 2023-11-26T07:17:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb07cff9640 1 -- 172.21.15.31:0/3252754892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0700072d0 con 0x7fb08006bc90 2023-11-26T07:17:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 -- 172.21.15.31:0/3252754892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0800b7fe0 con 0x7fb08006bc90 2023-11-26T07:17:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 -- 172.21.15.31:0/3252754892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0800b85a0 con 0x7fb08006bc90 2023-11-26T07:17:54.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.753+0000 7fb08513f640 1 -- 172.21.15.31:0/3252754892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0800d0f90 con 0x7fb08006bc90 2023-11-26T07:17:54.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.757+0000 7fb07cff9640 1 -- 172.21.15.31:0/3252754892 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb070007430 con 0x7fb08006bc90 2023-11-26T07:17:54.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.757+0000 7fb07cff9640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb06c066470 0x7fb06c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:54.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.757+0000 7fb07cff9640 1 -- 172.21.15.31:0/3252754892 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0700834a0 con 0x7fb08006bc90 2023-11-26T07:17:54.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.757+0000 7fb07f7fe640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb06c066470 0x7fb06c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:54.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.757+0000 7fb07f7fe640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb06c066470 0x7fb06c068930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb0800b0710 tx=0x7fb074009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:54.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.761+0000 7fb07cff9640 1 -- 172.21.15.31:0/3252754892 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb070050930 con 0x7fb08006bc90 2023-11-26T07:17:54.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.921+0000 7fb08513f640 1 -- 172.21.15.31:0/3252754892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb080074a00 con 0x7fb08006bc90 2023-11-26T07:17:54.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.921+0000 7fb07cff9640 1 -- 172.21.15.31:0/3252754892 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fb070053f50 con 0x7fb08006bc90 2023-11-26T07:17:54.927 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:54.928 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 -- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb06c066470 msgr2=0x7fb06c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb06c066470 0x7fb06c068930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb0800b0710 tx=0x7fb074009040 comp rx=0 tx=0).stop 2023-11-26T07:17:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 -- 172.21.15.31:0/3252754892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08006bc90 msgr2=0x7fb0800c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08006bc90 0x7fb0800c8e70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb07000c420 tx=0x7fb07000c8e0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 -- 172.21.15.31:0/3252754892 shutdown_connections 2023-11-26T07:17:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb06c066470 0x7fb06c068930 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb080160980 0x7fb0800b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0800b2850 0x7fb0800b70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 --2- 172.21.15.31:0/3252754892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb08006bc90 0x7fb0800c8e70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 -- 172.21.15.31:0/3252754892 >> 172.21.15.31:0/3252754892 conn(0x7fb08009c5c0 msgr2=0x7fb080156ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 -- 172.21.15.31:0/3252754892 shutdown_connections 2023-11-26T07:17:54.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:54.925+0000 7fb08513f640 1 -- 172.21.15.31:0/3252754892 wait complete. 2023-11-26T07:17:54.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:54.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:55.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.089+0000 7f31e2d0e640 1 Processor -- start 2023-11-26T07:17:55.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.089+0000 7f31e2d0e640 1 -- start start 2023-11-26T07:17:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31e2d0e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f31dc06e3f0 0x7f31dc15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31e2d0e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31dc159630 con 0x7f31dc0ce5d0 2023-11-26T07:17:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31e2d0e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31dc15f6d0 con 0x7f31dc0cb9b0 2023-11-26T07:17:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31e2d0e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31dc06e860 con 0x7f31dc06e3f0 2023-11-26T07:17:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31e12ab640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f31dc0ce5d0 0x7f31dc15bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49056/0 (socket says 172.21.15.31:49056) 2023-11-26T07:17:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31e12ab640 1 -- 172.21.15.31:0/1531888170 learned_addr learned my addr 172.21.15.31:0/1531888170 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:55.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31dbfff640 1 --2- 172.21.15.31:0/1531888170 >> v2:172.21.15.97:3300/0 conn(0x7f31dc06e3f0 0x7f31dc15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31db7fe640 1 -- 172.21.15.31:0/1531888170 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701004320 0 0) 0x7f31dc159630 con 0x7f31dc0ce5d0 2023-11-26T07:17:55.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31db7fe640 1 -- 172.21.15.31:0/1531888170 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31cc003530 con 0x7f31dc0ce5d0 2023-11-26T07:17:55.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31dbfff640 1 -- 172.21.15.31:0/1531888170 >> v1:172.21.15.97:6789/0 conn(0x7f31dc0cb9b0 legacy=0x7f31dc0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31dbfff640 1 -- 172.21.15.31:0/1531888170 >> v1:172.21.15.31:6789/0 conn(0x7f31dc0ce5d0 legacy=0x7f31dc15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.093+0000 7f31dbfff640 1 -- 172.21.15.31:0/1531888170 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc0cd1d0 con 0x7f31dc06e3f0 2023-11-26T07:17:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31dbfff640 1 --2- 172.21.15.31:0/1531888170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31dc06e3f0 0x7f31dc15f190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31c80097c0 tx=0x7f31c801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=172c42bf58f3a347 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31db7fe640 1 -- 172.21.15.31:0/1531888170 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80ab600 con 0x7f31dc06e3f0 2023-11-26T07:17:55.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31db7fe640 1 -- 172.21.15.31:0/1531888170 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c80a7050 con 0x7f31dc06e3f0 2023-11-26T07:17:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31db7fe640 1 -- 172.21.15.31:0/1531888170 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c80ab980 con 0x7f31dc06e3f0 2023-11-26T07:17:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 -- 172.21.15.31:0/1531888170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31dc06e3f0 msgr2=0x7f31dc15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 --2- 172.21.15.31:0/1531888170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31dc06e3f0 0x7f31dc15f190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f31c80097c0 tx=0x7f31c801c480 comp rx=0 tx=0).stop 2023-11-26T07:17:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 -- 172.21.15.31:0/1531888170 shutdown_connections 2023-11-26T07:17:55.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 --2- 172.21.15.31:0/1531888170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31dc06e3f0 0x7f31dc15f190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 -- 172.21.15.31:0/1531888170 >> 172.21.15.31:0/1531888170 conn(0x7f31dc0b0cb0 msgr2=0x7f31dc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 -- 172.21.15.31:0/1531888170 shutdown_connections 2023-11-26T07:17:55.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 -- 172.21.15.31:0/1531888170 wait complete. 2023-11-26T07:17:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 Processor -- start 2023-11-26T07:17:55.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 -- start start 2023-11-26T07:17:55.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31dc06e3f0 0x7f31dc099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31dc0cb9b0 0x7f31dc14fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f31dc0ce5d0 0x7f31dc0acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0b00c0 con 0x7f31dc06e3f0 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0b05b0 con 0x7f31dc0cb9b0 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.097+0000 7f31e2d0e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc167180 con 0x7f31dc0ce5d0 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e0aaa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31dc06e3f0 0x7f31dc099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e0aaa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31dc06e3f0 0x7f31dc099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40254/0 (socket says 172.21.15.31:40254) 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e0aaa640 1 -- 172.21.15.31:0/2083859246 learned_addr learned my addr 172.21.15.31:0/2083859246 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e12ab640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f31dc0ce5d0 0x7f31dc0acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31dbfff640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31dc0cb9b0 0x7f31dc14fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e0aaa640 1 -- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f31dc0ce5d0 msgr2=0x7f31dc0acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e0aaa640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f31dc0ce5d0 0x7f31dc0acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e0aaa640 1 -- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31dc0cb9b0 msgr2=0x7f31dc14fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e0aaa640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31dc0cb9b0 0x7f31dc14fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e0aaa640 1 -- 172.21.15.31:0/2083859246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c8009420 con 0x7f31dc06e3f0 2023-11-26T07:17:55.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31dbfff640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31dc0cb9b0 0x7f31dc14fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:55.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e12ab640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f31dc0ce5d0 0x7f31dc0acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e0aaa640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31dc06e3f0 0x7f31dc099a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f31d000c500 tx=0x7f31d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31d97fa640 1 -- 172.21.15.31:0/2083859246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0005220 con 0x7f31dc06e3f0 2023-11-26T07:17:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31d97fa640 1 -- 172.21.15.31:0/2083859246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d0005380 con 0x7f31dc06e3f0 2023-11-26T07:17:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e2d0e640 1 -- 172.21.15.31:0/2083859246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31dc0b02d0 con 0x7f31dc06e3f0 2023-11-26T07:17:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31d97fa640 1 -- 172.21.15.31:0/2083859246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d0007270 con 0x7f31dc06e3f0 2023-11-26T07:17:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e2d0e640 1 -- 172.21.15.31:0/2083859246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31dc150a60 con 0x7f31dc06e3f0 2023-11-26T07:17:55.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31e2d0e640 1 -- 172.21.15.31:0/2083859246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31dc0671f0 con 0x7f31dc06e3f0 2023-11-26T07:17:55.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.101+0000 7f31d97fa640 1 -- 172.21.15.31:0/2083859246 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f31d00073d0 con 0x7f31dc06e3f0 2023-11-26T07:17:55.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.105+0000 7f31d97fa640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f31c0066470 0x7f31c0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.105+0000 7f31d97fa640 1 -- 172.21.15.31:0/2083859246 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31d00836d0 con 0x7f31dc06e3f0 2023-11-26T07:17:55.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.105+0000 7f31dbfff640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f31c0066470 0x7f31c0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.105+0000 7f31dbfff640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f31c0066470 0x7f31c0068930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f31c80ad000 tx=0x7f31c80a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.105+0000 7f31d97fa640 1 -- 172.21.15.31:0/2083859246 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f31d0050b70 con 0x7f31dc06e3f0 2023-11-26T07:17:55.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.269+0000 7f31e2d0e640 1 -- 172.21.15.31:0/2083859246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31dc06b460 con 0x7f31dc06e3f0 2023-11-26T07:17:55.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.269+0000 7f31d97fa640 1 -- 172.21.15.31:0/2083859246 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31d0054190 con 0x7f31dc06e3f0 2023-11-26T07:17:55.275 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:17:55.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 -- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f31c0066470 msgr2=0x7f31c0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f31c0066470 0x7f31c0068930 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f31c80ad000 tx=0x7f31c80a6000 comp rx=0 tx=0).stop 2023-11-26T07:17:55.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 -- 172.21.15.31:0/2083859246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31dc06e3f0 msgr2=0x7f31dc099a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31dc06e3f0 0x7f31dc099a30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f31d000c500 tx=0x7f31d000c9c0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 -- 172.21.15.31:0/2083859246 shutdown_connections 2023-11-26T07:17:55.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f31c0066470 0x7f31c0068930 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f31dc0ce5d0 0x7f31dc0acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31dc0cb9b0 0x7f31dc14fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 --2- 172.21.15.31:0/2083859246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31dc06e3f0 0x7f31dc099a30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 -- 172.21.15.31:0/2083859246 >> 172.21.15.31:0/2083859246 conn(0x7f31dc0b0cb0 msgr2=0x7f31dc15bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 -- 172.21.15.31:0/2083859246 shutdown_connections 2023-11-26T07:17:55.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.273+0000 7f31e2d0e640 1 -- 172.21.15.31:0/2083859246 wait complete. 2023-11-26T07:17:55.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:17:55.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.437+0000 7fae60f1c640 1 Processor -- start 2023-11-26T07:17:55.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.437+0000 7fae60f1c640 1 -- start start 2023-11-26T07:17:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae60f1c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 0x7fae5c0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae60f1c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae5c06afc0 0x7fae5c0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae60f1c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5c161290 con 0x7fae5c0b07a0 2023-11-26T07:17:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae60f1c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5c0b0b80 con 0x7fae5c06afc0 2023-11-26T07:17:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae60f1c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c15ebc0 con 0x7fae5c0b2870 2023-11-26T07:17:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae5a59c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fae5c0b07a0 0x7fae5c1619e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49072/0 (socket says 172.21.15.31:49072) 2023-11-26T07:17:55.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae5a59c640 1 -- 172.21.15.31:0/1015119407 learned_addr learned my addr 172.21.15.31:0/1015119407 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae53fff640 1 --2- 172.21.15.31:0/1015119407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 0x7fae5c0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae5ad9d640 1 --2- 172.21.15.31:0/1015119407 >> v2:172.21.15.97:3300/0 conn(0x7fae5c06afc0 0x7fae5c0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae59d9b640 1 -- 172.21.15.31:0/1015119407 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746846578 0 0) 0x7fae5c161290 con 0x7fae5c0b07a0 2023-11-26T07:17:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae59d9b640 1 -- 172.21.15.31:0/1015119407 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae48003530 con 0x7fae5c0b07a0 2023-11-26T07:17:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae5ad9d640 1 -- 172.21.15.31:0/1015119407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 msgr2=0x7fae5c0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae5ad9d640 1 --2- 172.21.15.31:0/1015119407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 0x7fae5c0b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae5ad9d640 1 -- 172.21.15.31:0/1015119407 >> v1:172.21.15.31:6789/0 conn(0x7fae5c0b07a0 legacy=0x7fae5c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae5ad9d640 1 -- 172.21.15.31:0/1015119407 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c161550 con 0x7fae5c06afc0 2023-11-26T07:17:55.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae53fff640 1 --2- 172.21.15.31:0/1015119407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 0x7fae5c0b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:17:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.441+0000 7fae5ad9d640 1 --2- 172.21.15.31:0/1015119407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae5c06afc0 0x7fae5c0b1420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae54009dc0 tx=0x7fae5401c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa1a07b67d7f32b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae59d9b640 1 -- 172.21.15.31:0/1015119407 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540a58e0 con 0x7fae5c06afc0 2023-11-26T07:17:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae59d9b640 1 -- 172.21.15.31:0/1015119407 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae540a5a40 con 0x7fae5c06afc0 2023-11-26T07:17:55.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae59d9b640 1 -- 172.21.15.31:0/1015119407 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540a5dd0 con 0x7fae5c06afc0 2023-11-26T07:17:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 -- 172.21.15.31:0/1015119407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae5c06afc0 msgr2=0x7fae5c0b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 --2- 172.21.15.31:0/1015119407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae5c06afc0 0x7fae5c0b1420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae54009dc0 tx=0x7fae5401c930 comp rx=0 tx=0).stop 2023-11-26T07:17:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 -- 172.21.15.31:0/1015119407 shutdown_connections 2023-11-26T07:17:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 --2- 172.21.15.31:0/1015119407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae5c06afc0 0x7fae5c0b1420 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 --2- 172.21.15.31:0/1015119407 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 0x7fae5c0b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 -- 172.21.15.31:0/1015119407 >> 172.21.15.31:0/1015119407 conn(0x7fae5c0b1d50 msgr2=0x7fae5c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 -- 172.21.15.31:0/1015119407 shutdown_connections 2023-11-26T07:17:55.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 -- 172.21.15.31:0/1015119407 wait complete. 2023-11-26T07:17:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 Processor -- start 2023-11-26T07:17:55.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 -- start start 2023-11-26T07:17:55.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae5c06afc0 0x7fae5c1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae5c0b07a0 0x7fae5c1532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 0x7fae5c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c1537f0 con 0x7fae5c06afc0 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c158ae0 con 0x7fae5c0b07a0 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.445+0000 7fae60f1c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c0ad700 con 0x7fae5c0b2870 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae5a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae5c06afc0 0x7fae5c1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae5a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae5c06afc0 0x7fae5c1578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40258/0 (socket says 172.21.15.31:40258) 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae5a59c640 1 -- 172.21.15.31:0/174924199 learned_addr learned my addr 172.21.15.31:0/174924199 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae53fff640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae5c0b07a0 0x7fae5c1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae5ad9d640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 0x7fae5c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae5a59c640 1 -- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 msgr2=0x7fae5c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae5a59c640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 0x7fae5c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae5a59c640 1 -- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae5c0b07a0 msgr2=0x7fae5c1532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae5a59c640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae5c0b07a0 0x7fae5c1532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae5a59c640 1 -- 172.21.15.31:0/174924199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae44009420 con 0x7fae5c06afc0 2023-11-26T07:17:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae5a59c640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae5c06afc0 0x7fae5c1578a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00c450 tx=0x7fae4c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae52ffd640 1 -- 172.21.15.31:0/174924199 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c005cf0 con 0x7fae5c06afc0 2023-11-26T07:17:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae52ffd640 1 -- 172.21.15.31:0/174924199 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c005e50 con 0x7fae5c06afc0 2023-11-26T07:17:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae52ffd640 1 -- 172.21.15.31:0/174924199 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c007290 con 0x7fae5c06afc0 2023-11-26T07:17:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae60f1c640 1 -- 172.21.15.31:0/174924199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae54009a70 con 0x7fae5c06afc0 2023-11-26T07:17:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae60f1c640 1 -- 172.21.15.31:0/174924199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c1581b0 con 0x7fae5c06afc0 2023-11-26T07:17:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae60f1c640 1 -- 172.21.15.31:0/174924199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c04e850 con 0x7fae5c06afc0 2023-11-26T07:17:55.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae52ffd640 1 -- 172.21.15.31:0/174924199 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fae4c0073f0 con 0x7fae5c06afc0 2023-11-26T07:17:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.449+0000 7fae52ffd640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae3c0664a0 0x7fae3c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.453+0000 7fae52ffd640 1 -- 172.21.15.31:0/174924199 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae4c0833f0 con 0x7fae5c06afc0 2023-11-26T07:17:55.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.453+0000 7fae53fff640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae3c0664a0 0x7fae3c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.453+0000 7fae53fff640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae3c0664a0 0x7fae3c068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae5c1f9630 tx=0x7fae44009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.453+0000 7fae52ffd640 1 -- 172.21.15.31:0/174924199 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae4c050880 con 0x7fae5c06afc0 2023-11-26T07:17:55.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 -- 172.21.15.31:0/174924199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae5c06f530 con 0x7fae5c06afc0 2023-11-26T07:17:55.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae52ffd640 1 -- 172.21.15.31:0/174924199 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae4c017020 con 0x7fae5c06afc0 2023-11-26T07:17:55.616 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:17:55.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 -- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae3c0664a0 msgr2=0x7fae3c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae3c0664a0 0x7fae3c068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fae5c1f9630 tx=0x7fae44009190 comp rx=0 tx=0).stop 2023-11-26T07:17:55.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 -- 172.21.15.31:0/174924199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae5c06afc0 msgr2=0x7fae5c1578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae5c06afc0 0x7fae5c1578a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00c450 tx=0x7fae4c00c910 comp rx=0 tx=0).stop 2023-11-26T07:17:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 -- 172.21.15.31:0/174924199 shutdown_connections 2023-11-26T07:17:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae3c0664a0 0x7fae3c068960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae5c0b2870 0x7fae5c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae5c0b07a0 0x7fae5c1532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 --2- 172.21.15.31:0/174924199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae5c06afc0 0x7fae5c1578a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 -- 172.21.15.31:0/174924199 >> 172.21.15.31:0/174924199 conn(0x7fae5c0b1d50 msgr2=0x7fae5c06a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 -- 172.21.15.31:0/174924199 shutdown_connections 2023-11-26T07:17:55.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.613+0000 7fae60f1c640 1 -- 172.21.15.31:0/174924199 wait complete. 2023-11-26T07:17:55.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:17:55.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc890f2a640 1 Processor -- start 2023-11-26T07:17:55.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc890f2a640 1 -- start start 2023-11-26T07:17:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc890f2a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc88c15d610 0x7fc88c0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc890f2a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc88c0caa60 0x7fc88c0cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc890f2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc88c06c520 0x7fc88c06c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc890f2a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c06ceb0 con 0x7fc88c15d610 2023-11-26T07:17:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc890f2a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c0caef0 con 0x7fc88c0caa60 2023-11-26T07:17:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc890f2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c06b970 con 0x7fc88c06c520 2023-11-26T07:17:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc88a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc88c15d610 0x7fc88c0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc88a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc88c15d610 0x7fc88c0ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40268/0 (socket says 172.21.15.31:40268) 2023-11-26T07:17:55.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc88a59c640 1 -- 172.21.15.31:0/2993476899 learned_addr learned my addr 172.21.15.31:0/2993476899 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc88ad9d640 1 --2- 172.21.15.31:0/2993476899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc88c06c520 0x7fc88c06c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc889d9b640 1 --2- 172.21.15.31:0/2993476899 >> v2:172.21.15.97:3300/0 conn(0x7fc88c0caa60 0x7fc88c0cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc88a59c640 1 -- 172.21.15.31:0/2993476899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc88c06c520 msgr2=0x7fc88c06c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc88a59c640 1 --2- 172.21.15.31:0/2993476899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc88c06c520 0x7fc88c06c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc88a59c640 1 -- 172.21.15.31:0/2993476899 >> v2:172.21.15.97:3300/0 conn(0x7fc88c0caa60 msgr2=0x7fc88c0cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc88a59c640 1 --2- 172.21.15.31:0/2993476899 >> v2:172.21.15.97:3300/0 conn(0x7fc88c0caa60 0x7fc88c0cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc88a59c640 1 -- 172.21.15.31:0/2993476899 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c06bc50 con 0x7fc88c15d610 2023-11-26T07:17:55.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.781+0000 7fc889d9b640 1 --2- 172.21.15.31:0/2993476899 >> v2:172.21.15.97:3300/0 conn(0x7fc88c0caa60 0x7fc88c0cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:55.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc88a59c640 1 --2- 172.21.15.31:0/2993476899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c15d610 0x7fc88c0ce4b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc88c06e210 tx=0x7fc87c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d01439ca7f858cb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc88959a640 1 -- 172.21.15.31:0/2993476899 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0aa5f0 con 0x7fc88c15d610 2023-11-26T07:17:55.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc88959a640 1 -- 172.21.15.31:0/2993476899 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c0aa750 con 0x7fc88c15d610 2023-11-26T07:17:55.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc88959a640 1 -- 172.21.15.31:0/2993476899 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0aaae0 con 0x7fc88c15d610 2023-11-26T07:17:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc890f2a640 1 -- 172.21.15.31:0/2993476899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c15d610 msgr2=0x7fc88c0ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc890f2a640 1 --2- 172.21.15.31:0/2993476899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c15d610 0x7fc88c0ce4b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc88c06e210 tx=0x7fc87c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc890f2a640 1 -- 172.21.15.31:0/2993476899 shutdown_connections 2023-11-26T07:17:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc890f2a640 1 --2- 172.21.15.31:0/2993476899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc88c06c520 0x7fc88c06c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc890f2a640 1 --2- 172.21.15.31:0/2993476899 >> v2:172.21.15.97:3300/0 conn(0x7fc88c0caa60 0x7fc88c0cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc890f2a640 1 --2- 172.21.15.31:0/2993476899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c15d610 0x7fc88c0ce4b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc890f2a640 1 -- 172.21.15.31:0/2993476899 >> 172.21.15.31:0/2993476899 conn(0x7fc88c0b2f10 msgr2=0x7fc88c0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc890f2a640 1 -- 172.21.15.31:0/2993476899 shutdown_connections 2023-11-26T07:17:55.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.785+0000 7fc890f2a640 1 -- 172.21.15.31:0/2993476899 wait complete. 2023-11-26T07:17:55.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc890f2a640 1 Processor -- start 2023-11-26T07:17:55.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc890f2a640 1 -- start start 2023-11-26T07:17:55.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc890f2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c06c520 0x7fc88c15db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc890f2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88c0caa60 0x7fc88c14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc890f2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc88c15d610 0x7fc88c14b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc890f2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0b4f90 con 0x7fc88c06c520 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc890f2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c1535c0 con 0x7fc88c0caa60 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc890f2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c1532e0 con 0x7fc88c15d610 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc88a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c06c520 0x7fc88c15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc88a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c06c520 0x7fc88c15db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40270/0 (socket says 172.21.15.31:40270) 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc88a59c640 1 -- 172.21.15.31:0/946235839 learned_addr learned my addr 172.21.15.31:0/946235839 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc889d9b640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88c0caa60 0x7fc88c14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc88ad9d640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc88c15d610 0x7fc88c14b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc88a59c640 1 -- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc88c15d610 msgr2=0x7fc88c14b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc88a59c640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc88c15d610 0x7fc88c14b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc88a59c640 1 -- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88c0caa60 msgr2=0x7fc88c14ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc88a59c640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88c0caa60 0x7fc88c14ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc88a59c640 1 -- 172.21.15.31:0/946235839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c0094d0 con 0x7fc88c06c520 2023-11-26T07:17:55.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc88a59c640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c06c520 0x7fc88c15db70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc87c01ab80 tx=0x7fc87c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc87affd640 1 -- 172.21.15.31:0/946235839 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0bd5f0 con 0x7fc88c06c520 2023-11-26T07:17:55.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc87affd640 1 -- 172.21.15.31:0/946235839 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c0bd770 con 0x7fc88c06c520 2023-11-26T07:17:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc890f2a640 1 -- 172.21.15.31:0/946235839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c0b48d0 con 0x7fc88c06c520 2023-11-26T07:17:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc87affd640 1 -- 172.21.15.31:0/946235839 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c0bdb20 con 0x7fc88c06c520 2023-11-26T07:17:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.789+0000 7fc890f2a640 1 -- 172.21.15.31:0/946235839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c0b6a00 con 0x7fc88c06c520 2023-11-26T07:17:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.793+0000 7fc890f2a640 1 -- 172.21.15.31:0/946235839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c04e850 con 0x7fc88c06c520 2023-11-26T07:17:55.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.793+0000 7fc87affd640 1 -- 172.21.15.31:0/946235839 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc87c0b33f0 con 0x7fc88c06c520 2023-11-26T07:17:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.793+0000 7fc87affd640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc86c066470 0x7fc86c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.793+0000 7fc87affd640 1 -- 172.21.15.31:0/946235839 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc87c0a7070 con 0x7fc88c06c520 2023-11-26T07:17:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.793+0000 7fc889d9b640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc86c066470 0x7fc86c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.793+0000 7fc889d9b640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc86c066470 0x7fc86c068930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc88c14a490 tx=0x7fc880008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.797+0000 7fc87affd640 1 -- 172.21.15.31:0/946235839 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc87c0ec400 con 0x7fc88c06c520 2023-11-26T07:17:55.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.953+0000 7fc890f2a640 1 -- 172.21.15.31:0/946235839 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc88c157370 con 0x7fc88c06c520 2023-11-26T07:17:55.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.953+0000 7fc87affd640 1 -- 172.21.15.31:0/946235839 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc87c0efa20 con 0x7fc88c06c520 2023-11-26T07:17:55.958 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:17:55.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 -- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc86c066470 msgr2=0x7fc86c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc86c066470 0x7fc86c068930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc88c14a490 tx=0x7fc880008040 comp rx=0 tx=0).stop 2023-11-26T07:17:55.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 -- 172.21.15.31:0/946235839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c06c520 msgr2=0x7fc88c15db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:55.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c06c520 0x7fc88c15db70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc87c01ab80 tx=0x7fc87c01ad10 comp rx=0 tx=0).stop 2023-11-26T07:17:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 -- 172.21.15.31:0/946235839 shutdown_connections 2023-11-26T07:17:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc86c066470 0x7fc86c068930 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc88c15d610 0x7fc88c14b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc88c0caa60 0x7fc88c14ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 --2- 172.21.15.31:0/946235839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc88c06c520 0x7fc88c15db70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 -- 172.21.15.31:0/946235839 >> 172.21.15.31:0/946235839 conn(0x7fc88c0b2f10 msgr2=0x7fc88c14a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 -- 172.21.15.31:0/946235839 shutdown_connections 2023-11-26T07:17:55.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:55.957+0000 7fc890f2a640 1 -- 172.21.15.31:0/946235839 wait complete. 2023-11-26T07:17:55.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:56.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.117+0000 7f1741654640 1 Processor -- start 2023-11-26T07:17:56.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.117+0000 7f1741654640 1 -- start start 2023-11-26T07:17:56.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f1741654640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f173c144c80 0x7f173c0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f1741654640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f173c070df0 0x7f173c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f1741654640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173c145060 con 0x7f173c0ac460 2023-11-26T07:17:56.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f1741654640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173c071fd0 con 0x7f173c144c80 2023-11-26T07:17:56.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f1741654640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0ac8c0 con 0x7f173c070df0 2023-11-26T07:17:56.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f1733fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f173c0ac460 0x7f173c0706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49080/0 (socket says 172.21.15.31:49080) 2023-11-26T07:17:56.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f1733fff640 1 -- 172.21.15.31:0/3777390550 learned_addr learned my addr 172.21.15.31:0/3777390550 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:56.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f173affd640 1 --2- 172.21.15.31:0/3777390550 >> v2:172.21.15.97:3300/0 conn(0x7f173c144c80 0x7f173c0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f173b7fe640 1 --2- 172.21.15.31:0/3777390550 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f173c070df0 0x7f173c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f173a7fc640 1 -- 172.21.15.31:0/3777390550 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119804001 0 0) 0x7f173c145060 con 0x7f173c0ac460 2023-11-26T07:17:56.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f173a7fc640 1 -- 172.21.15.31:0/3777390550 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1728003530 con 0x7f173c0ac460 2023-11-26T07:17:56.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f173affd640 1 -- 172.21.15.31:0/3777390550 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f173c070df0 msgr2=0x7f173c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f173affd640 1 --2- 172.21.15.31:0/3777390550 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f173c070df0 0x7f173c06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f173a7fc640 1 -- 172.21.15.31:0/3777390550 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3402740741 0 0) 0x7f1728003530 con 0x7f173c0ac460 2023-11-26T07:17:56.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f173affd640 1 -- 172.21.15.31:0/3777390550 >> v1:172.21.15.31:6789/0 conn(0x7f173c0ac460 legacy=0x7f173c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.121+0000 7f173affd640 1 -- 172.21.15.31:0/3777390550 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c071450 con 0x7f173c144c80 2023-11-26T07:17:56.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f173affd640 1 --2- 172.21.15.31:0/3777390550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c144c80 0x7f173c0abf20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f172c0097c0 tx=0x7f172c01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1079d22619c08389 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f173a7fc640 1 -- 172.21.15.31:0/3777390550 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724001680 con 0x7f173c144c80 2023-11-26T07:17:56.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f173a7fc640 1 -- 172.21.15.31:0/3777390550 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c0a7050 con 0x7f173c144c80 2023-11-26T07:17:56.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f173a7fc640 1 -- 172.21.15.31:0/3777390550 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c0ab620 con 0x7f173c144c80 2023-11-26T07:17:56.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f1741654640 1 -- 172.21.15.31:0/3777390550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c144c80 msgr2=0x7f173c0abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f1741654640 1 --2- 172.21.15.31:0/3777390550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c144c80 0x7f173c0abf20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f172c0097c0 tx=0x7f172c01c3a0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f1741654640 1 -- 172.21.15.31:0/3777390550 shutdown_connections 2023-11-26T07:17:56.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f1741654640 1 --2- 172.21.15.31:0/3777390550 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f173c070df0 0x7f173c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f1741654640 1 --2- 172.21.15.31:0/3777390550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c144c80 0x7f173c0abf20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f1741654640 1 -- 172.21.15.31:0/3777390550 >> 172.21.15.31:0/3777390550 conn(0x7f173c0b0c40 msgr2=0x7f173c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f1741654640 1 -- 172.21.15.31:0/3777390550 shutdown_connections 2023-11-26T07:17:56.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f1741654640 1 -- 172.21.15.31:0/3777390550 wait complete. 2023-11-26T07:17:56.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f1741654640 1 Processor -- start 2023-11-26T07:17:56.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.125+0000 7f1741654640 1 -- start start 2023-11-26T07:17:56.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f1741654640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c070df0 0x7f173c0b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f1741654640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0ac460 0x7f173c15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f1741654640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f173c144c80 0x7f173c15ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f1741654640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c15bc90 con 0x7f173c070df0 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f1741654640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c15b200 con 0x7f173c0ac460 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f1741654640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173c0b39f0 con 0x7f173c144c80 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f173affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c070df0 0x7f173c0b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f173affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c070df0 0x7f173c0b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40286/0 (socket says 172.21.15.31:40286) 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f173affd640 1 -- 172.21.15.31:0/547917432 learned_addr learned my addr 172.21.15.31:0/547917432 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f173b7fe640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f173c144c80 0x7f173c15ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f1733fff640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0ac460 0x7f173c15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f173affd640 1 -- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f173c144c80 msgr2=0x7f173c15ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f173affd640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f173c144c80 0x7f173c15ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f173affd640 1 -- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0ac460 msgr2=0x7f173c15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f173affd640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0ac460 0x7f173c15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f173affd640 1 -- 172.21.15.31:0/547917432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c009420 con 0x7f173c070df0 2023-11-26T07:17:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f173affd640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c070df0 0x7f173c0b3360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f172c01aa50 tx=0x7f172c019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f17337fe640 1 -- 172.21.15.31:0/547917432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c0b4410 con 0x7f173c070df0 2023-11-26T07:17:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f17337fe640 1 -- 172.21.15.31:0/547917432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c0b4590 con 0x7f173c070df0 2023-11-26T07:17:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f1741654640 1 -- 172.21.15.31:0/547917432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c0b3cd0 con 0x7f173c070df0 2023-11-26T07:17:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f17337fe640 1 -- 172.21.15.31:0/547917432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c0b4940 con 0x7f173c070df0 2023-11-26T07:17:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f1741654640 1 -- 172.21.15.31:0/547917432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c0b3fb0 con 0x7f173c070df0 2023-11-26T07:17:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.129+0000 7f1741654640 1 -- 172.21.15.31:0/547917432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c0b1ba0 con 0x7f173c070df0 2023-11-26T07:17:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.133+0000 7f17337fe640 1 -- 172.21.15.31:0/547917432 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f172c0b2040 con 0x7f173c070df0 2023-11-26T07:17:56.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.133+0000 7f17337fe640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f171c066470 0x7f171c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.133+0000 7f17337fe640 1 -- 172.21.15.31:0/547917432 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f172c11dd10 con 0x7f173c070df0 2023-11-26T07:17:56.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.133+0000 7f1733fff640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f171c066470 0x7f171c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.133+0000 7f1733fff640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f171c066470 0x7f171c068930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f173c15b5b0 tx=0x7f1724009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.137+0000 7f17337fe640 1 -- 172.21.15.31:0/547917432 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f172c0eb100 con 0x7f173c070df0 2023-11-26T07:17:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.297+0000 7f1741654640 1 -- 172.21.15.31:0/547917432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f173c04e850 con 0x7f173c070df0 2023-11-26T07:17:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f17337fe640 1 -- 172.21.15.31:0/547917432 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f172c0ee720 con 0x7f173c070df0 2023-11-26T07:17:56.304 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:56.304 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 -- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f171c066470 msgr2=0x7f171c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f171c066470 0x7f171c068930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f173c15b5b0 tx=0x7f1724009040 comp rx=0 tx=0).stop 2023-11-26T07:17:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 -- 172.21.15.31:0/547917432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c070df0 msgr2=0x7f173c0b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c070df0 0x7f173c0b3360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f172c01aa50 tx=0x7f172c019140 comp rx=0 tx=0).stop 2023-11-26T07:17:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 -- 172.21.15.31:0/547917432 shutdown_connections 2023-11-26T07:17:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f171c066470 0x7f171c068930 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f173c144c80 0x7f173c15ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f173c0ac460 0x7f173c15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 --2- 172.21.15.31:0/547917432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f173c070df0 0x7f173c0b3360 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 -- 172.21.15.31:0/547917432 >> 172.21.15.31:0/547917432 conn(0x7f173c0b0c40 msgr2=0x7f173c09ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 -- 172.21.15.31:0/547917432 shutdown_connections 2023-11-26T07:17:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.301+0000 7f1741654640 1 -- 172.21.15.31:0/547917432 wait complete. 2023-11-26T07:17:56.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:56.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:56.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e3135640 1 Processor -- start 2023-11-26T07:17:56.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e3135640 1 -- start start 2023-11-26T07:17:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e3135640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48dc0b46a0 0x7f48dc15e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e3135640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48dc147190 con 0x7f48dc146820 2023-11-26T07:17:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e3135640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48dc06d030 con 0x7f48dc0b4210 2023-11-26T07:17:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e3135640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc146c00 con 0x7f48dc0b46a0 2023-11-26T07:17:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e0ed1640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f48dc146820 0x7f48dc070a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49090/0 (socket says 172.21.15.31:49090) 2023-11-26T07:17:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e0ed1640 1 -- 172.21.15.31:0/4060485265 learned_addr learned my addr 172.21.15.31:0/4060485265 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e16d2640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48dc0b46a0 0x7f48dc15e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2517908323 0 0) 0x7f48dc147190 con 0x7f48dc146820 2023-11-26T07:17:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48cc003530 con 0x7f48dc146820 2023-11-26T07:17:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3768276028 0 0) 0x7f48cc003530 con 0x7f48dc146820 2023-11-26T07:17:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48dc0b46a0 msgr2=0x7f48dc15e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48dc0b46a0 0x7f48dc15e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 >> v1:172.21.15.97:6789/0 conn(0x7f48dc0b4210 legacy=0x7f48dc15db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc06bf90 con 0x7f48dc146820 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f48d0001680 con 0x7f48dc146820 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 >> v1:172.21.15.31:6789/0 conn(0x7f48dc146820 legacy=0x7f48dc070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48dc0b46a0 0x7f48cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48cc003f30 0x7f48cc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48cc004d10 0x7f48cc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc146c00 con 0x7f48cc003f30 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc00ba70 con 0x7f48cc004d10 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc00bc30 con 0x7f48dc0b46a0 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48dbfff640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48cc003f30 0x7f48cc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e0ed1640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48dc0b46a0 0x7f48cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48e16d2640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48cc004d10 0x7f48cc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48dbfff640 1 -- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48dc0b46a0 msgr2=0x7f48cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48dbfff640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48dc0b46a0 0x7f48cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48dbfff640 1 -- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48cc004d10 msgr2=0x7f48cc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48dbfff640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48cc004d10 0x7f48cc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.469+0000 7f48dbfff640 1 -- 172.21.15.31:0/4060485265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc003530 con 0x7f48cc003f30 2023-11-26T07:17:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48dbfff640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48cc003f30 0x7f48cc0047d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48c8001780 tx=0x7f48c801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d65101dd6b3bce server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80052d0 con 0x7f48cc003f30 2023-11-26T07:17:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c80a7050 con 0x7f48cc003f30 2023-11-26T07:17:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48db7fe640 1 -- 172.21.15.31:0/4060485265 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8003600 con 0x7f48cc003f30 2023-11-26T07:17:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48e3135640 1 -- 172.21.15.31:0/4060485265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48cc003f30 msgr2=0x7f48cc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48e3135640 1 --2- 172.21.15.31:0/4060485265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48cc003f30 0x7f48cc0047d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48c8001780 tx=0x7f48c801c400 comp rx=0 tx=0).stop 2023-11-26T07:17:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48e16d2640 1 -- 172.21.15.31:0/4060485265 reap_dead start 2023-11-26T07:17:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48e3135640 1 -- 172.21.15.31:0/4060485265 shutdown_connections 2023-11-26T07:17:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48e3135640 1 -- 172.21.15.31:0/4060485265 >> 172.21.15.31:0/4060485265 conn(0x7f48dc0b2f10 msgr2=0x7f48dc0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48e3135640 1 -- 172.21.15.31:0/4060485265 shutdown_connections 2023-11-26T07:17:56.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.473+0000 7f48e3135640 1 -- 172.21.15.31:0/4060485265 wait complete. 2023-11-26T07:17:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e3135640 1 Processor -- start 2023-11-26T07:17:56.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e3135640 1 -- start start 2023-11-26T07:17:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e3135640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48cc003f30 0x7f48dc13c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e3135640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc0ce370 0x7f48dc0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e3135640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48dc0ce7d0 0x7f48dc0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e3135640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0cd2d0 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e3135640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0cd5b0 con 0x7f48dc0ce370 2023-11-26T07:17:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e3135640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0cd890 con 0x7f48cc003f30 2023-11-26T07:17:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e16d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48dc0ce7d0 0x7f48dc0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e16d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48dc0ce7d0 0x7f48dc0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40296/0 (socket says 172.21.15.31:40296) 2023-11-26T07:17:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e16d2640 1 -- 172.21.15.31:0/1324678157 learned_addr learned my addr 172.21.15.31:0/1324678157 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:56.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e16d2640 1 -- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48cc003f30 msgr2=0x7f48dc13c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e16d2640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48cc003f30 0x7f48dc13c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e16d2640 1 -- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc0ce370 msgr2=0x7f48dc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e16d2640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc0ce370 0x7f48dc0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e16d2640 1 -- 172.21.15.31:0/1324678157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0cdb70 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e16d2640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48dc0ce7d0 0x7f48dc0ccca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f48dc147340 tx=0x7f48d400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48d97fa640 1 -- 172.21.15.31:0/1324678157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d400e040 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48d97fa640 1 -- 172.21.15.31:0/1324678157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d400f040 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e3135640 1 -- 172.21.15.31:0/1324678157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc0cde50 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48d97fa640 1 -- 172.21.15.31:0/1324678157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d4015430 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.477+0000 7f48e3135640 1 -- 172.21.15.31:0/1324678157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc13fce0 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.481+0000 7f48e3135640 1 -- 172.21.15.31:0/1324678157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48dc04e850 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.481+0000 7f48d97fa640 1 -- 172.21.15.31:0/1324678157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f48d4015590 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.481+0000 7f48d97fa640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f48c00664a0 0x7f48c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.481+0000 7f48d97fa640 1 -- 172.21.15.31:0/1324678157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48d40846d0 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.481+0000 7f48e0ed1640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f48c00664a0 0x7f48c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.481+0000 7f48e0ed1640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f48c00664a0 0x7f48c0068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f48d00011f0 tx=0x7f48d0001120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.485+0000 7f48d97fa640 1 -- 172.21.15.31:0/1324678157 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f48d4051ab0 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 -- 172.21.15.31:0/1324678157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48dc0d1f00 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48d97fa640 1 -- 172.21.15.31:0/1324678157 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48d40550d0 con 0x7f48dc0ce7d0 2023-11-26T07:17:56.648 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:17:56.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 -- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f48c00664a0 msgr2=0x7f48c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f48c00664a0 0x7f48c0068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f48d00011f0 tx=0x7f48d0001120 comp rx=0 tx=0).stop 2023-11-26T07:17:56.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 -- 172.21.15.31:0/1324678157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48dc0ce7d0 msgr2=0x7f48dc0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48dc0ce7d0 0x7f48dc0ccca0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f48dc147340 tx=0x7f48d400f580 comp rx=0 tx=0).stop 2023-11-26T07:17:56.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 -- 172.21.15.31:0/1324678157 shutdown_connections 2023-11-26T07:17:56.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f48c00664a0 0x7f48c0068960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48dc0ce7d0 0x7f48dc0ccca0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48dc0ce370 0x7f48dc0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 --2- 172.21.15.31:0/1324678157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48cc003f30 0x7f48dc13c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 -- 172.21.15.31:0/1324678157 >> 172.21.15.31:0/1324678157 conn(0x7f48dc0b2f10 msgr2=0x7f48dc070250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 -- 172.21.15.31:0/1324678157 shutdown_connections 2023-11-26T07:17:56.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.645+0000 7f48e3135640 1 -- 172.21.15.31:0/1324678157 wait complete. 2023-11-26T07:17:56.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:17:56.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.809+0000 7ff47b59e640 1 Processor -- start 2023-11-26T07:17:56.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.809+0000 7ff47b59e640 1 -- start start 2023-11-26T07:17:56.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47b59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff47c15b2d0 0x7ff47c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47b59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff47c0b5250 0x7ff47c0b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47b59e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47c15b6b0 con 0x7ff47c0d31c0 2023-11-26T07:17:56.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47b59e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47c15ebd0 con 0x7ff47c15b2d0 2023-11-26T07:17:56.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47b59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0b5c10 con 0x7ff47c0b5250 2023-11-26T07:17:56.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff47c15b2d0 0x7ff47c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff47c15b2d0 0x7ff47c0ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40304/0 (socket says 172.21.15.31:40304) 2023-11-26T07:17:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47a59c640 1 -- 172.21.15.31:0/361593108 learned_addr learned my addr 172.21.15.31:0/361593108 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff473fff640 1 --2- 172.21.15.31:0/361593108 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff47c0b5250 0x7ff47c0b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47a59c640 1 -- 172.21.15.31:0/361593108 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff47c0b5250 msgr2=0x7ff47c0b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:56.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47a59c640 1 --2- 172.21.15.31:0/361593108 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff47c0b5250 0x7ff47c0b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47a59c640 1 -- 172.21.15.31:0/361593108 >> v1:172.21.15.97:6789/0 conn(0x7ff47c0d31c0 legacy=0x7ff47c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.813+0000 7ff47a59c640 1 -- 172.21.15.31:0/361593108 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c0cda70 con 0x7ff47c15b2d0 2023-11-26T07:17:56.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47a59c640 1 --2- 172.21.15.31:0/361593108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c15b2d0 0x7ff47c0ce7f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009870 tx=0x7ff46c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf160ac7841990c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff479d9b640 1 -- 172.21.15.31:0/361593108 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0a7070 con 0x7ff47c15b2d0 2023-11-26T07:17:56.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff479d9b640 1 -- 172.21.15.31:0/361593108 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c0ab430 con 0x7ff47c15b2d0 2023-11-26T07:17:56.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff479d9b640 1 -- 172.21.15.31:0/361593108 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0ab7c0 con 0x7ff47c15b2d0 2023-11-26T07:17:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47b59e640 1 -- 172.21.15.31:0/361593108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c15b2d0 msgr2=0x7ff47c0ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47b59e640 1 --2- 172.21.15.31:0/361593108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c15b2d0 0x7ff47c0ce7f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009870 tx=0x7ff46c01c540 comp rx=0 tx=0).stop 2023-11-26T07:17:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47b59e640 1 -- 172.21.15.31:0/361593108 shutdown_connections 2023-11-26T07:17:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47b59e640 1 --2- 172.21.15.31:0/361593108 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff47c0b5250 0x7ff47c0b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47b59e640 1 --2- 172.21.15.31:0/361593108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c15b2d0 0x7ff47c0ce7f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47b59e640 1 -- 172.21.15.31:0/361593108 >> 172.21.15.31:0/361593108 conn(0x7ff47c0666a0 msgr2=0x7ff47c066ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47b59e640 1 -- 172.21.15.31:0/361593108 shutdown_connections 2023-11-26T07:17:56.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47b59e640 1 -- 172.21.15.31:0/361593108 wait complete. 2023-11-26T07:17:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47b59e640 1 Processor -- start 2023-11-26T07:17:56.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.817+0000 7ff47b59e640 1 -- start start 2023-11-26T07:17:56.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47b59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff47c0b5250 0x7ff47c0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c0d31c0 0x7ff47c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47b59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c15b2d0 0x7ff47c14fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47b59e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0c7710 con 0x7ff47c0d31c0 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47b59e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0c6de0 con 0x7ff47c15b2d0 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47b59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0c70c0 con 0x7ff47c0b5250 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff473fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c0d31c0 0x7ff47c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff473fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c0d31c0 0x7ff47c1526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40314/0 (socket says 172.21.15.31:40314) 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff473fff640 1 -- 172.21.15.31:0/2878082313 learned_addr learned my addr 172.21.15.31:0/2878082313 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47a59c640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff47c0b5250 0x7ff47c0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47ad9d640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c15b2d0 0x7ff47c14fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff473fff640 1 -- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff47c0b5250 msgr2=0x7ff47c0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff473fff640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff47c0b5250 0x7ff47c0cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff473fff640 1 -- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c15b2d0 msgr2=0x7ff47c14fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff473fff640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c15b2d0 0x7ff47c14fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff473fff640 1 -- 172.21.15.31:0/2878082313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c0094d0 con 0x7ff47c0d31c0 2023-11-26T07:17:56.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff473fff640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c0d31c0 0x7ff47c1526e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff46400d800 tx=0x7ff46400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff472ffd640 1 -- 172.21.15.31:0/2878082313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400b6c0 con 0x7ff47c0d31c0 2023-11-26T07:17:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff472ffd640 1 -- 172.21.15.31:0/2878082313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46400b860 con 0x7ff47c0d31c0 2023-11-26T07:17:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff472ffd640 1 -- 172.21.15.31:0/2878082313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400bc10 con 0x7ff47c0d31c0 2023-11-26T07:17:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47b59e640 1 -- 172.21.15.31:0/2878082313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c0c73a0 con 0x7ff47c0d31c0 2023-11-26T07:17:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47b59e640 1 -- 172.21.15.31:0/2878082313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c0ccfd0 con 0x7ff47c0d31c0 2023-11-26T07:17:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff47b59e640 1 -- 172.21.15.31:0/2878082313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47c0cccd0 con 0x7ff47c0d31c0 2023-11-26T07:17:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.821+0000 7ff472ffd640 1 -- 172.21.15.31:0/2878082313 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff46401e460 con 0x7ff47c0d31c0 2023-11-26T07:17:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.825+0000 7ff472ffd640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff45c0664a0 0x7ff45c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.825+0000 7ff472ffd640 1 -- 172.21.15.31:0/2878082313 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4640842c0 con 0x7ff47c0d31c0 2023-11-26T07:17:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.825+0000 7ff47a59c640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff45c0664a0 0x7ff45c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:56.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.825+0000 7ff47a59c640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff45c0664a0 0x7ff45c068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0af000 tx=0x7ff46c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.829+0000 7ff472ffd640 1 -- 172.21.15.31:0/2878082313 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff4640517e0 con 0x7ff47c0d31c0 2023-11-26T07:17:56.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.985+0000 7ff47b59e640 1 -- 172.21.15.31:0/2878082313 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff47c073d30 con 0x7ff47c0d31c0 2023-11-26T07:17:56.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.985+0000 7ff472ffd640 1 -- 172.21.15.31:0/2878082313 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff47c073d30 con 0x7ff47c0d31c0 2023-11-26T07:17:56.990 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:17:56.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 -- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff45c0664a0 msgr2=0x7ff45c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff45c0664a0 0x7ff45c068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0af000 tx=0x7ff46c008040 comp rx=0 tx=0).stop 2023-11-26T07:17:56.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 -- 172.21.15.31:0/2878082313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c0d31c0 msgr2=0x7ff47c1526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:56.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c0d31c0 0x7ff47c1526e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff46400d800 tx=0x7ff46400dcc0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 -- 172.21.15.31:0/2878082313 shutdown_connections 2023-11-26T07:17:56.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff45c0664a0 0x7ff45c068960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff47c15b2d0 0x7ff47c14fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff47c0d31c0 0x7ff47c1526e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 --2- 172.21.15.31:0/2878082313 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff47c0b5250 0x7ff47c0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:56.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 -- 172.21.15.31:0/2878082313 >> 172.21.15.31:0/2878082313 conn(0x7ff47c0666a0 msgr2=0x7ff47c0d3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:56.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 -- 172.21.15.31:0/2878082313 shutdown_connections 2023-11-26T07:17:56.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:56.989+0000 7ff47b59e640 1 -- 172.21.15.31:0/2878082313 wait complete. 2023-11-26T07:17:57.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:17:57.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736f59e640 1 Processor -- start 2023-11-26T07:17:57.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736f59e640 1 -- start start 2023-11-26T07:17:57.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736f59e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f737015d550 0x7f7370161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736f59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f73700d3a00 0x7f73700cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736f59e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7370161ff0 con 0x7f737015e7a0 2023-11-26T07:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736f59e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73700d1210 con 0x7f73700d3a00 2023-11-26T07:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736f59e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73700d4230 con 0x7f737015d550 2023-11-26T07:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736dd9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f73700d3a00 0x7f73700cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736dd9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f73700d3a00 0x7f73700cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40318/0 (socket says 172.21.15.31:40318) 2023-11-26T07:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736dd9b640 1 -- 172.21.15.31:0/3767101339 learned_addr learned my addr 172.21.15.31:0/3767101339 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736ed9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f737015e7a0 0x7f73700d1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49094/0 (socket says 172.21.15.31:49094) 2023-11-26T07:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736e59c640 1 --2- 172.21.15.31:0/3767101339 >> v2:172.21.15.97:3300/0 conn(0x7f737015d550 0x7f7370161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736dd9b640 1 -- 172.21.15.31:0/3767101339 >> v2:172.21.15.97:3300/0 conn(0x7f737015d550 msgr2=0x7f7370161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736dd9b640 1 --2- 172.21.15.31:0/3767101339 >> v2:172.21.15.97:3300/0 conn(0x7f737015d550 0x7f7370161ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736dd9b640 1 -- 172.21.15.31:0/3767101339 >> v1:172.21.15.31:6789/0 conn(0x7f737015e7a0 legacy=0x7f73700d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736dd9b640 1 -- 172.21.15.31:0/3767101339 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737015dd00 con 0x7f73700d3a00 2023-11-26T07:17:57.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.157+0000 7f736e59c640 1 --2- 172.21.15.31:0/3767101339 >> v2:172.21.15.97:3300/0 conn(0x7f737015d550 0x7f7370161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:57.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736dd9b640 1 --2- 172.21.15.31:0/3767101339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 0x7f73700cfa80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f737008cce0 tx=0x7f736401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb94c0bdf7cee3e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736d59a640 1 -- 172.21.15.31:0/3767101339 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73640a89f0 con 0x7f73700d3a00 2023-11-26T07:17:57.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736d59a640 1 -- 172.21.15.31:0/3767101339 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73640a8b50 con 0x7f73700d3a00 2023-11-26T07:17:57.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736d59a640 1 -- 172.21.15.31:0/3767101339 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73640b0440 con 0x7f73700d3a00 2023-11-26T07:17:57.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736f59e640 1 -- 172.21.15.31:0/3767101339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 msgr2=0x7f73700cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736f59e640 1 --2- 172.21.15.31:0/3767101339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 0x7f73700cfa80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f737008cce0 tx=0x7f736401ecf0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736f59e640 1 -- 172.21.15.31:0/3767101339 shutdown_connections 2023-11-26T07:17:57.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736f59e640 1 --2- 172.21.15.31:0/3767101339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 0x7f73700cfa80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736f59e640 1 --2- 172.21.15.31:0/3767101339 >> v2:172.21.15.97:3300/0 conn(0x7f737015d550 0x7f7370161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736f59e640 1 -- 172.21.15.31:0/3767101339 >> 172.21.15.31:0/3767101339 conn(0x7f737009c670 msgr2=0x7f73700d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736f59e640 1 -- 172.21.15.31:0/3767101339 shutdown_connections 2023-11-26T07:17:57.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.161+0000 7f736f59e640 1 -- 172.21.15.31:0/3767101339 wait complete. 2023-11-26T07:17:57.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 Processor -- start 2023-11-26T07:17:57.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 -- start start 2023-11-26T07:17:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 0x7f73701f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f737015d550 0x7f73701588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f737015e7a0 0x7f73700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73701f7480 con 0x7f73700d3a00 2023-11-26T07:17:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73701f6be0 con 0x7f737015d550 2023-11-26T07:17:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73701f6ec0 con 0x7f737015e7a0 2023-11-26T07:17:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 0x7f73701f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 0x7f73701f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40322/0 (socket says 172.21.15.31:40322) 2023-11-26T07:17:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736e59c640 1 -- 172.21.15.31:0/3714330945 learned_addr learned my addr 172.21.15.31:0/3714330945 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736dd9b640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f737015d550 0x7f73701588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736ed9d640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f737015e7a0 0x7f73700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736e59c640 1 -- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f737015e7a0 msgr2=0x7f73700b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736e59c640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f737015e7a0 0x7f73700b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736e59c640 1 -- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f737015d550 msgr2=0x7f73701588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736e59c640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f737015d550 0x7f73701588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736e59c640 1 -- 172.21.15.31:0/3714330945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73640094d0 con 0x7f73700d3a00 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736dd9b640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f737015d550 0x7f73701588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736e59c640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 0x7f73701f6570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f736000e810 tx=0x7f736000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f735effd640 1 -- 172.21.15.31:0/3714330945 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000c8d0 con 0x7f73700d3a00 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f735effd640 1 -- 172.21.15.31:0/3714330945 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736000ca30 con 0x7f73700d3a00 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f735effd640 1 -- 172.21.15.31:0/3714330945 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000cdc0 con 0x7f73700d3a00 2023-11-26T07:17:57.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 -- 172.21.15.31:0/3714330945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73701f71a0 con 0x7f73700d3a00 2023-11-26T07:17:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 -- 172.21.15.31:0/3714330945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737015a740 con 0x7f73700d3a00 2023-11-26T07:17:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.165+0000 7f736f59e640 1 -- 172.21.15.31:0/3714330945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7370074990 con 0x7f73700d3a00 2023-11-26T07:17:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.169+0000 7f735effd640 1 -- 172.21.15.31:0/3714330945 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7360013410 con 0x7f73700d3a00 2023-11-26T07:17:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.169+0000 7f735effd640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73500664a0 0x7f7350068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.169+0000 7f735effd640 1 -- 172.21.15.31:0/3714330945 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7360083c60 con 0x7f73700d3a00 2023-11-26T07:17:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.169+0000 7f736dd9b640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73500664a0 0x7f7350068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.169+0000 7f736dd9b640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73500664a0 0x7f7350068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f73640b2000 tx=0x7f7364008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.173+0000 7f735effd640 1 -- 172.21.15.31:0/3714330945 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f73600510d0 con 0x7f73700d3a00 2023-11-26T07:17:57.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.329+0000 7f736f59e640 1 -- 172.21.15.31:0/3714330945 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73700d0970 con 0x7f73700d3a00 2023-11-26T07:17:57.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.329+0000 7f735effd640 1 -- 172.21.15.31:0/3714330945 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7360009480 con 0x7f73700d3a00 2023-11-26T07:17:57.334 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:17:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 -- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73500664a0 msgr2=0x7f7350068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73500664a0 0x7f7350068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f73640b2000 tx=0x7f7364008040 comp rx=0 tx=0).stop 2023-11-26T07:17:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 -- 172.21.15.31:0/3714330945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 msgr2=0x7f73701f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 0x7f73701f6570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f736000e810 tx=0x7f736000ecd0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 -- 172.21.15.31:0/3714330945 shutdown_connections 2023-11-26T07:17:57.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f73500664a0 0x7f7350068960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f737015e7a0 0x7f73700b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f737015d550 0x7f73701588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 --2- 172.21.15.31:0/3714330945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73700d3a00 0x7f73701f6570 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 -- 172.21.15.31:0/3714330945 >> 172.21.15.31:0/3714330945 conn(0x7f737009c670 msgr2=0x7f73700afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 -- 172.21.15.31:0/3714330945 shutdown_connections 2023-11-26T07:17:57.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.333+0000 7f736f59e640 1 -- 172.21.15.31:0/3714330945 wait complete. 2023-11-26T07:17:57.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:57.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb82debb640 1 Processor -- start 2023-11-26T07:17:57.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb82debb640 1 -- start start 2023-11-26T07:17:57.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb82debb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8280ac630 0x7fb8280706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb82debb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb828070c20 0x7fb82806c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb82debb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb828145060 con 0x7fb828144c80 2023-11-26T07:17:57.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb82debb640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb828071fd0 con 0x7fb828070c20 2023-11-26T07:17:57.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb82debb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb828071450 con 0x7fb8280ac630 2023-11-26T07:17:57.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb827fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb828070c20 0x7fb82806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb827fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb828070c20 0x7fb82806c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40324/0 (socket says 172.21.15.31:40324) 2023-11-26T07:17:57.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb827fff640 1 -- 172.21.15.31:0/3034956669 learned_addr learned my addr 172.21.15.31:0/3034956669 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:57.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb826ffd640 1 --2- 172.21.15.31:0/3034956669 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8280ac630 0x7fb8280706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb827fff640 1 -- 172.21.15.31:0/3034956669 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8280ac630 msgr2=0x7fb8280706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:57.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb827fff640 1 --2- 172.21.15.31:0/3034956669 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8280ac630 0x7fb8280706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb827fff640 1 -- 172.21.15.31:0/3034956669 >> v1:172.21.15.97:6789/0 conn(0x7fb828144c80 legacy=0x7fb8280abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.497+0000 7fb827fff640 1 -- 172.21.15.31:0/3034956669 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb828071730 con 0x7fb828070c20 2023-11-26T07:17:57.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb827fff640 1 --2- 172.21.15.31:0/3034956669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb828070c20 0x7fb82806c910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb820009e50 tx=0x7fb82001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38e7aadaff8f39c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb8267fc640 1 -- 172.21.15.31:0/3034956669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8200a58a0 con 0x7fb828070c20 2023-11-26T07:17:57.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb8267fc640 1 -- 172.21.15.31:0/3034956669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8200a5a00 con 0x7fb828070c20 2023-11-26T07:17:57.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb8267fc640 1 -- 172.21.15.31:0/3034956669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8200a5d90 con 0x7fb828070c20 2023-11-26T07:17:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb82debb640 1 -- 172.21.15.31:0/3034956669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb828070c20 msgr2=0x7fb82806c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb82debb640 1 --2- 172.21.15.31:0/3034956669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb828070c20 0x7fb82806c910 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb820009e50 tx=0x7fb82001c9e0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb82debb640 1 -- 172.21.15.31:0/3034956669 shutdown_connections 2023-11-26T07:17:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb82debb640 1 --2- 172.21.15.31:0/3034956669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb828070c20 0x7fb82806c910 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb82debb640 1 --2- 172.21.15.31:0/3034956669 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8280ac630 0x7fb8280706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb82debb640 1 -- 172.21.15.31:0/3034956669 >> 172.21.15.31:0/3034956669 conn(0x7fb8280b0c40 msgr2=0x7fb8280b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb82debb640 1 -- 172.21.15.31:0/3034956669 shutdown_connections 2023-11-26T07:17:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.501+0000 7fb82debb640 1 -- 172.21.15.31:0/3034956669 wait complete. 2023-11-26T07:17:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 Processor -- start 2023-11-26T07:17:57.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 -- start start 2023-11-26T07:17:57.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb828070c20 0x7fb82809b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8280ac630 0x7fb82814dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb828144c80 0x7fb82814e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82813a190 con 0x7fb8280ac630 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82814bb70 con 0x7fb828070c20 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82815af60 con 0x7fb828144c80 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb826ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8280ac630 0x7fb82814dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb826ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8280ac630 0x7fb82814dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40334/0 (socket says 172.21.15.31:40334) 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb826ffd640 1 -- 172.21.15.31:0/1310969128 learned_addr learned my addr 172.21.15.31:0/1310969128 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb8277fe640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb828070c20 0x7fb82809b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb827fff640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb828144c80 0x7fb82814e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb826ffd640 1 -- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb828144c80 msgr2=0x7fb82814e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb826ffd640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb828144c80 0x7fb82814e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb826ffd640 1 -- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb828070c20 msgr2=0x7fb82809b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb826ffd640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb828070c20 0x7fb82809b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb826ffd640 1 -- 172.21.15.31:0/1310969128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820009b30 con 0x7fb8280ac630 2023-11-26T07:17:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb826ffd640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8280ac630 0x7fb82814dc00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00e930 tx=0x7fb81c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb80ffff640 1 -- 172.21.15.31:0/1310969128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c009690 con 0x7fb8280ac630 2023-11-26T07:17:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb80ffff640 1 -- 172.21.15.31:0/1310969128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c00c9a0 con 0x7fb8280ac630 2023-11-26T07:17:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 -- 172.21.15.31:0/1310969128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82809bb30 con 0x7fb8280ac630 2023-11-26T07:17:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb80ffff640 1 -- 172.21.15.31:0/1310969128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c00cd10 con 0x7fb8280ac630 2023-11-26T07:17:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 -- 172.21.15.31:0/1310969128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82809c0f0 con 0x7fb8280ac630 2023-11-26T07:17:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.505+0000 7fb82debb640 1 -- 172.21.15.31:0/1310969128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82804e850 con 0x7fb8280ac630 2023-11-26T07:17:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.509+0000 7fb80ffff640 1 -- 172.21.15.31:0/1310969128 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb81c013470 con 0x7fb8280ac630 2023-11-26T07:17:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.509+0000 7fb80ffff640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066470 0x7fb814068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.509+0000 7fb80ffff640 1 -- 172.21.15.31:0/1310969128 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb81c0840d0 con 0x7fb8280ac630 2023-11-26T07:17:57.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.509+0000 7fb8277fe640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066470 0x7fb814068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.509+0000 7fb8277fe640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066470 0x7fb814068930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb818007fd0 tx=0x7fb818009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.513+0000 7fb80ffff640 1 -- 172.21.15.31:0/1310969128 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb81c051620 con 0x7fb8280ac630 2023-11-26T07:17:57.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.669+0000 7fb82debb640 1 -- 172.21.15.31:0/1310969128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb828072650 con 0x7fb8280ac630 2023-11-26T07:17:57.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb80ffff640 1 -- 172.21.15.31:0/1310969128 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fb81c054c40 con 0x7fb8280ac630 2023-11-26T07:17:57.676 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:57.676 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:57.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 -- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066470 msgr2=0x7fb814068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066470 0x7fb814068930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb818007fd0 tx=0x7fb818009040 comp rx=0 tx=0).stop 2023-11-26T07:17:57.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 -- 172.21.15.31:0/1310969128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8280ac630 msgr2=0x7fb82814dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8280ac630 0x7fb82814dc00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb81c00e930 tx=0x7fb81c00edf0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 -- 172.21.15.31:0/1310969128 shutdown_connections 2023-11-26T07:17:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066470 0x7fb814068930 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb828144c80 0x7fb82814e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8280ac630 0x7fb82814dc00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 --2- 172.21.15.31:0/1310969128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb828070c20 0x7fb82809b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 -- 172.21.15.31:0/1310969128 >> 172.21.15.31:0/1310969128 conn(0x7fb8280b0c40 msgr2=0x7fb8280b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 -- 172.21.15.31:0/1310969128 shutdown_connections 2023-11-26T07:17:57.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.673+0000 7fb82debb640 1 -- 172.21.15.31:0/1310969128 wait complete. 2023-11-26T07:17:57.698 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:57.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:57.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb33aa32640 1 Processor -- start 2023-11-26T07:17:57.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb33aa32640 1 -- start start 2023-11-26T07:17:57.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb33aa32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb33415a710 0x7fb3340abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb33aa32640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3340cbd50 0x7fb3340cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb33aa32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3340b2710 con 0x7fb3340cdfb0 2023-11-26T07:17:57.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb33aa32640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3340cca80 con 0x7fb3340cbd50 2023-11-26T07:17:57.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb33aa32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3340b29f0 con 0x7fb33415a710 2023-11-26T07:17:57.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb338fcf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3340cbd50 0x7fb3340cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb338fcf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb3340cbd50 0x7fb3340cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40348/0 (socket says 172.21.15.31:40348) 2023-11-26T07:17:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb338fcf640 1 -- 172.21.15.31:0/3605369074 learned_addr learned my addr 172.21.15.31:0/3605369074 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb3337fe640 1 --2- 172.21.15.31:0/3605369074 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb33415a710 0x7fb3340abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb338fcf640 1 -- 172.21.15.31:0/3605369074 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb33415a710 msgr2=0x7fb3340abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb338fcf640 1 --2- 172.21.15.31:0/3605369074 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb33415a710 0x7fb3340abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.841+0000 7fb338fcf640 1 -- 172.21.15.31:0/3605369074 >> v1:172.21.15.97:6789/0 conn(0x7fb3340cdfb0 legacy=0x7fb3340ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb338fcf640 1 -- 172.21.15.31:0/3605369074 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340b2cd0 con 0x7fb3340cbd50 2023-11-26T07:17:57.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb338fcf640 1 --2- 172.21.15.31:0/3605369074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 0x7fb3340cc1e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009e50 tx=0x7fb32c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813b961c590c0832 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb332ffd640 1 -- 172.21.15.31:0/3605369074 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c0a58d0 con 0x7fb3340cbd50 2023-11-26T07:17:57.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb332ffd640 1 -- 172.21.15.31:0/3605369074 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c0a5a30 con 0x7fb3340cbd50 2023-11-26T07:17:57.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb332ffd640 1 -- 172.21.15.31:0/3605369074 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c0a5dc0 con 0x7fb3340cbd50 2023-11-26T07:17:57.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb33aa32640 1 -- 172.21.15.31:0/3605369074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 msgr2=0x7fb3340cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb33aa32640 1 --2- 172.21.15.31:0/3605369074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 0x7fb3340cc1e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009e50 tx=0x7fb32c01c9e0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb33aa32640 1 -- 172.21.15.31:0/3605369074 shutdown_connections 2023-11-26T07:17:57.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb33aa32640 1 --2- 172.21.15.31:0/3605369074 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 0x7fb3340cc1e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb33aa32640 1 --2- 172.21.15.31:0/3605369074 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb33415a710 0x7fb3340abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb33aa32640 1 -- 172.21.15.31:0/3605369074 >> 172.21.15.31:0/3605369074 conn(0x7fb3340b0cb0 msgr2=0x7fb3340b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:57.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb33aa32640 1 -- 172.21.15.31:0/3605369074 shutdown_connections 2023-11-26T07:17:57.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.845+0000 7fb33aa32640 1 -- 172.21.15.31:0/3605369074 wait complete. 2023-11-26T07:17:57.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb33aa32640 1 Processor -- start 2023-11-26T07:17:57.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb33aa32640 1 -- start start 2023-11-26T07:17:57.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb33aa32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 0x7fb334152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb33aa32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3340cdfb0 0x7fb3341557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb33aa32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb33415a710 0x7fb334155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb33aa32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3341553f0 con 0x7fb3340cbd50 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb33aa32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33409ac60 con 0x7fb3340cdfb0 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb33aa32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33415b750 con 0x7fb33415a710 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb333fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 0x7fb334152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb333fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 0x7fb334152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40354/0 (socket says 172.21.15.31:40354) 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb333fff640 1 -- 172.21.15.31:0/1652182232 learned_addr learned my addr 172.21.15.31:0/1652182232 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb3337fe640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3340cdfb0 0x7fb3341557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb338fcf640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb33415a710 0x7fb334155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb333fff640 1 -- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb33415a710 msgr2=0x7fb334155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb333fff640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb33415a710 0x7fb334155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb333fff640 1 -- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3340cdfb0 msgr2=0x7fb3341557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:57.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb333fff640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3340cdfb0 0x7fb3341557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:57.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb333fff640 1 -- 172.21.15.31:0/1652182232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c009b30 con 0x7fb3340cbd50 2023-11-26T07:17:57.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb333fff640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 0x7fb334152be0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c500 tx=0x7fb32400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb330ff9640 1 -- 172.21.15.31:0/1652182232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324005220 con 0x7fb3340cbd50 2023-11-26T07:17:57.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb330ff9640 1 -- 172.21.15.31:0/1652182232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb324005380 con 0x7fb3340cbd50 2023-11-26T07:17:57.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb33aa32640 1 -- 172.21.15.31:0/1652182232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33415ba30 con 0x7fb3340cbd50 2023-11-26T07:17:57.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.849+0000 7fb330ff9640 1 -- 172.21.15.31:0/1652182232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324007270 con 0x7fb3340cbd50 2023-11-26T07:17:57.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.853+0000 7fb33aa32640 1 -- 172.21.15.31:0/1652182232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33415bff0 con 0x7fb3340cbd50 2023-11-26T07:17:57.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.853+0000 7fb33aa32640 1 -- 172.21.15.31:0/1652182232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3340671f0 con 0x7fb3340cbd50 2023-11-26T07:17:57.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.853+0000 7fb330ff9640 1 -- 172.21.15.31:0/1652182232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb3240073d0 con 0x7fb3340cbd50 2023-11-26T07:17:57.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.853+0000 7fb330ff9640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb320066470 0x7fb320068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:57.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.853+0000 7fb330ff9640 1 -- 172.21.15.31:0/1652182232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb324083670 con 0x7fb3340cbd50 2023-11-26T07:17:57.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.853+0000 7fb3337fe640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb320066470 0x7fb320068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:57.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.853+0000 7fb3337fe640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb320066470 0x7fb320068930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb334156510 tx=0x7fb328008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:57.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:57.857+0000 7fb330ff9640 1 -- 172.21.15.31:0/1652182232 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb324050bb0 con 0x7fb3340cbd50 2023-11-26T07:17:58.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 -- 172.21.15.31:0/1652182232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb33406b460 con 0x7fb3340cbd50 2023-11-26T07:17:58.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb330ff9640 1 -- 172.21.15.31:0/1652182232 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3240541d0 con 0x7fb3340cbd50 2023-11-26T07:17:58.021 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:17:58.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 -- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb320066470 msgr2=0x7fb320068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb320066470 0x7fb320068930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb334156510 tx=0x7fb328008040 comp rx=0 tx=0).stop 2023-11-26T07:17:58.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 -- 172.21.15.31:0/1652182232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 msgr2=0x7fb334152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 0x7fb334152be0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c500 tx=0x7fb32400c9c0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 -- 172.21.15.31:0/1652182232 shutdown_connections 2023-11-26T07:17:58.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb320066470 0x7fb320068930 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb33415a710 0x7fb334155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3340cdfb0 0x7fb3341557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 --2- 172.21.15.31:0/1652182232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3340cbd50 0x7fb334152be0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 -- 172.21.15.31:0/1652182232 >> 172.21.15.31:0/1652182232 conn(0x7fb3340b0cb0 msgr2=0x7fb33415e220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 -- 172.21.15.31:0/1652182232 shutdown_connections 2023-11-26T07:17:58.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.017+0000 7fb33aa32640 1 -- 172.21.15.31:0/1652182232 wait complete. 2023-11-26T07:17:58.042 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:17:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc47aa640 1 Processor -- start 2023-11-26T07:17:58.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc47aa640 1 -- start start 2023-11-26T07:17:58.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc47aa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5dbc0ce4b0 0x7f5dbc0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc47aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc06c520 0x7f5dbc06c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc47aa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dbc06ceb0 con 0x7f5dbc15d610 2023-11-26T07:17:58.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc47aa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dbc0cbfe0 con 0x7f5dbc0ce4b0 2023-11-26T07:17:58.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc47aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc06b970 con 0x7f5dbc06c520 2023-11-26T07:17:58.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc1d45640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5dbc0ce4b0 0x7f5dbc0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc1d45640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5dbc0ce4b0 0x7f5dbc0caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40366/0 (socket says 172.21.15.31:40366) 2023-11-26T07:17:58.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc2546640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5dbc15d610 0x7f5dbc0cb8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49104/0 (socket says 172.21.15.31:49104) 2023-11-26T07:17:58.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc1d45640 1 -- 172.21.15.31:0/8044053 learned_addr learned my addr 172.21.15.31:0/8044053 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:58.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc2d47640 1 --2- 172.21.15.31:0/8044053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc06c520 0x7f5dbc06c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc1d45640 1 -- 172.21.15.31:0/8044053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc06c520 msgr2=0x7f5dbc06c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:58.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc1d45640 1 --2- 172.21.15.31:0/8044053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc06c520 0x7f5dbc06c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc1d45640 1 -- 172.21.15.31:0/8044053 >> v1:172.21.15.31:6789/0 conn(0x7f5dbc15d610 legacy=0x7f5dbc0cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc1d45640 1 -- 172.21.15.31:0/8044053 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc06bc50 con 0x7f5dbc0ce4b0 2023-11-26T07:17:58.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.185+0000 7f5dc1544640 1 -- 172.21.15.31:0/8044053 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477279893 0 0) 0x7f5db8001120 con 0x7f5dbc15d610 2023-11-26T07:17:58.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc1d45640 1 --2- 172.21.15.31:0/8044053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc0ce4b0 0x7f5dbc0caa60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009870 tx=0x7f5dac01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51d5bf1808c82fd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc1544640 1 -- 172.21.15.31:0/8044053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0ab600 con 0x7f5dbc0ce4b0 2023-11-26T07:17:58.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc1544640 1 -- 172.21.15.31:0/8044053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac0a7050 con 0x7f5dbc0ce4b0 2023-11-26T07:17:58.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc1544640 1 -- 172.21.15.31:0/8044053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0ab980 con 0x7f5dbc0ce4b0 2023-11-26T07:17:58.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc47aa640 1 -- 172.21.15.31:0/8044053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc0ce4b0 msgr2=0x7f5dbc0caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc47aa640 1 --2- 172.21.15.31:0/8044053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc0ce4b0 0x7f5dbc0caa60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009870 tx=0x7f5dac01c540 comp rx=0 tx=0).stop 2023-11-26T07:17:58.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc47aa640 1 -- 172.21.15.31:0/8044053 shutdown_connections 2023-11-26T07:17:58.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc47aa640 1 --2- 172.21.15.31:0/8044053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc06c520 0x7f5dbc06c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc47aa640 1 --2- 172.21.15.31:0/8044053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc0ce4b0 0x7f5dbc0caa60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc47aa640 1 -- 172.21.15.31:0/8044053 >> 172.21.15.31:0/8044053 conn(0x7f5dbc0b2f10 msgr2=0x7f5dbc0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc47aa640 1 -- 172.21.15.31:0/8044053 shutdown_connections 2023-11-26T07:17:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.189+0000 7f5dc47aa640 1 -- 172.21.15.31:0/8044053 wait complete. 2023-11-26T07:17:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 Processor -- start 2023-11-26T07:17:58.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 -- start start 2023-11-26T07:17:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dbc06c520 0x7f5dbc13fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc0ce4b0 0x7f5dbc0b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc15d610 0x7f5dbc15c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc14f400 con 0x7f5dbc15d610 2023-11-26T07:17:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0b6af0 con 0x7f5dbc06c520 2023-11-26T07:17:58.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc14e0b0 con 0x7f5dbc0ce4b0 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2d47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc15d610 0x7f5dbc15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2d47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc15d610 0x7f5dbc15c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40372/0 (socket says 172.21.15.31:40372) 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2d47640 1 -- 172.21.15.31:0/3656661503 learned_addr learned my addr 172.21.15.31:0/3656661503 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc1d45640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc0ce4b0 0x7f5dbc0b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2546640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dbc06c520 0x7f5dbc13fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2d47640 1 -- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc0ce4b0 msgr2=0x7f5dbc0b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2d47640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc0ce4b0 0x7f5dbc0b65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2d47640 1 -- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dbc06c520 msgr2=0x7f5dbc13fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2d47640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dbc06c520 0x7f5dbc13fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2d47640 1 -- 172.21.15.31:0/3656661503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dac0094d0 con 0x7f5dbc15d610 2023-11-26T07:17:58.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc1d45640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc0ce4b0 0x7f5dbc0b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:58.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2546640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dbc06c520 0x7f5dbc13fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:58.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc2d47640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc15d610 0x7f5dbc15c080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5db400efc0 tx=0x7f5db400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5db2ffd640 1 -- 172.21.15.31:0/3656661503 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4009ba0 con 0x7f5dbc15d610 2023-11-26T07:17:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5db2ffd640 1 -- 172.21.15.31:0/3656661503 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db400f040 con 0x7f5dbc15d610 2023-11-26T07:17:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 -- 172.21.15.31:0/3656661503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc14e390 con 0x7f5dbc15d610 2023-11-26T07:17:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5db2ffd640 1 -- 172.21.15.31:0/3656661503 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db40146c0 con 0x7f5dbc15d610 2023-11-26T07:17:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 -- 172.21.15.31:0/3656661503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc14e950 con 0x7f5dbc15d610 2023-11-26T07:17:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.193+0000 7f5dc47aa640 1 -- 172.21.15.31:0/3656661503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dbc04e850 con 0x7f5dbc15d610 2023-11-26T07:17:58.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.197+0000 7f5db2ffd640 1 -- 172.21.15.31:0/3656661503 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5db4014820 con 0x7f5dbc15d610 2023-11-26T07:17:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.197+0000 7f5db2ffd640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5da0066470 0x7f5da0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.197+0000 7f5db2ffd640 1 -- 172.21.15.31:0/3656661503 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5db4084a60 con 0x7f5dbc15d610 2023-11-26T07:17:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.197+0000 7f5dc2546640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5da0066470 0x7f5da0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.197+0000 7f5dc2546640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5da0066470 0x7f5da0068930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000f70 tx=0x7f5db8007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.201+0000 7f5db2ffd640 1 -- 172.21.15.31:0/3656661503 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5db4051e40 con 0x7f5dbc15d610 2023-11-26T07:17:58.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 -- 172.21.15.31:0/3656661503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5dbc0cf0f0 con 0x7f5dbc15d610 2023-11-26T07:17:58.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5db2ffd640 1 -- 172.21.15.31:0/3656661503 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5db4018020 con 0x7f5dbc15d610 2023-11-26T07:17:58.365 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:17:58.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 -- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5da0066470 msgr2=0x7f5da0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5da0066470 0x7f5da0068930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000f70 tx=0x7f5db8007a60 comp rx=0 tx=0).stop 2023-11-26T07:17:58.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 -- 172.21.15.31:0/3656661503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc15d610 msgr2=0x7f5dbc15c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc15d610 0x7f5dbc15c080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5db400efc0 tx=0x7f5db400c490 comp rx=0 tx=0).stop 2023-11-26T07:17:58.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 -- 172.21.15.31:0/3656661503 shutdown_connections 2023-11-26T07:17:58.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5da0066470 0x7f5da0068930 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5dbc15d610 0x7f5dbc15c080 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5dbc0ce4b0 0x7f5dbc0b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 --2- 172.21.15.31:0/3656661503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dbc06c520 0x7f5dbc13fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 -- 172.21.15.31:0/3656661503 >> 172.21.15.31:0/3656661503 conn(0x7f5dbc0b2f10 msgr2=0x7f5dbc0cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 -- 172.21.15.31:0/3656661503 shutdown_connections 2023-11-26T07:17:58.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.361+0000 7f5dc47aa640 1 -- 172.21.15.31:0/3656661503 wait complete. 2023-11-26T07:17:58.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:17:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.525+0000 7fb25e1c1640 1 Processor -- start 2023-11-26T07:17:58.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.525+0000 7fb25e1c1640 1 -- start start 2023-11-26T07:17:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb25e1c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258159f00 0x7fb2580ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb25e1c1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb258072950 con 0x7fb2580ae680 2023-11-26T07:17:58.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb25e1c1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb25806e2b0 con 0x7fb258070890 2023-11-26T07:17:58.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb25e1c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25806c6f0 con 0x7fb258159f00 2023-11-26T07:17:58.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb256ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb2580ae680 0x7fb2580cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49114/0 (socket says 172.21.15.31:49114) 2023-11-26T07:17:58.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb256ffd640 1 -- 172.21.15.31:0/2841006694 learned_addr learned my addr 172.21.15.31:0/2841006694 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:58.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb2577fe640 1 --2- 172.21.15.31:0/2841006694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258159f00 0x7fb2580ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb2567fc640 1 -- 172.21.15.31:0/2841006694 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260970103 0 0) 0x7fb258072950 con 0x7fb2580ae680 2023-11-26T07:17:58.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb2567fc640 1 -- 172.21.15.31:0/2841006694 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb240003530 con 0x7fb2580ae680 2023-11-26T07:17:58.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb2577fe640 1 -- 172.21.15.31:0/2841006694 >> v1:172.21.15.97:6789/0 conn(0x7fb258070890 legacy=0x7fb258070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb2567fc640 1 -- 172.21.15.31:0/2841006694 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126335128 0 0) 0x7fb25806e2b0 con 0x7fb258070890 2023-11-26T07:17:58.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb2577fe640 1 -- 172.21.15.31:0/2841006694 >> v1:172.21.15.31:6789/0 conn(0x7fb2580ae680 legacy=0x7fb2580cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.529+0000 7fb2577fe640 1 -- 172.21.15.31:0/2841006694 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25806e550 con 0x7fb258159f00 2023-11-26T07:17:58.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb2577fe640 1 --2- 172.21.15.31:0/2841006694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258159f00 0x7fb2580ccea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2480097f0 tx=0x7fb24801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33569fc0aaca671a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb2567fc640 1 -- 172.21.15.31:0/2841006694 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2480ab610 con 0x7fb258159f00 2023-11-26T07:17:58.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb2567fc640 1 -- 172.21.15.31:0/2841006694 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2480ab770 con 0x7fb258159f00 2023-11-26T07:17:58.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb2567fc640 1 -- 172.21.15.31:0/2841006694 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2480abb00 con 0x7fb258159f00 2023-11-26T07:17:58.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb25e1c1640 1 -- 172.21.15.31:0/2841006694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258159f00 msgr2=0x7fb2580ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb25e1c1640 1 --2- 172.21.15.31:0/2841006694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258159f00 0x7fb2580ccea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2480097f0 tx=0x7fb24801c490 comp rx=0 tx=0).stop 2023-11-26T07:17:58.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb25e1c1640 1 -- 172.21.15.31:0/2841006694 shutdown_connections 2023-11-26T07:17:58.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb25e1c1640 1 --2- 172.21.15.31:0/2841006694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258159f00 0x7fb2580ccea0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb25e1c1640 1 -- 172.21.15.31:0/2841006694 >> 172.21.15.31:0/2841006694 conn(0x7fb2580b0cb0 msgr2=0x7fb2580b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb25e1c1640 1 -- 172.21.15.31:0/2841006694 shutdown_connections 2023-11-26T07:17:58.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb25e1c1640 1 -- 172.21.15.31:0/2841006694 wait complete. 2023-11-26T07:17:58.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb25e1c1640 1 Processor -- start 2023-11-26T07:17:58.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.533+0000 7fb25e1c1640 1 -- start start 2023-11-26T07:17:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb25e1c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258070890 0x7fb258099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb25e1c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2580ae680 0x7fb25809c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb25e1c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb25809c900 0x7fb258140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb25e1c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25813f780 con 0x7fb25809c900 2023-11-26T07:17:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb25e1c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25809cdb0 con 0x7fb2580ae680 2023-11-26T07:17:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb25e1c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2581405f0 con 0x7fb258070890 2023-11-26T07:17:58.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb257fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb25809c900 0x7fb258140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb257fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb25809c900 0x7fb258140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40382/0 (socket says 172.21.15.31:40382) 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb257fff640 1 -- 172.21.15.31:0/1082734681 learned_addr learned my addr 172.21.15.31:0/1082734681 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb2577fe640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258070890 0x7fb258099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb256ffd640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2580ae680 0x7fb25809c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb257fff640 1 -- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258070890 msgr2=0x7fb258099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb257fff640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258070890 0x7fb258099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb257fff640 1 -- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2580ae680 msgr2=0x7fb25809c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb257fff640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2580ae680 0x7fb25809c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb257fff640 1 -- 172.21.15.31:0/1082734681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb248009420 con 0x7fb25809c900 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb256ffd640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2580ae680 0x7fb25809c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb257fff640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb25809c900 0x7fb258140a80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb250001570 tx=0x7fb25000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb23ffff640 1 -- 172.21.15.31:0/1082734681 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250003db0 con 0x7fb25809c900 2023-11-26T07:17:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb23ffff640 1 -- 172.21.15.31:0/1082734681 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb250005070 con 0x7fb25809c900 2023-11-26T07:17:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb25e1c1640 1 -- 172.21.15.31:0/1082734681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25813f9e0 con 0x7fb25809c900 2023-11-26T07:17:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb23ffff640 1 -- 172.21.15.31:0/1082734681 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250005420 con 0x7fb25809c900 2023-11-26T07:17:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb25e1c1640 1 -- 172.21.15.31:0/1082734681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25813ffa0 con 0x7fb25809c900 2023-11-26T07:17:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.537+0000 7fb25e1c1640 1 -- 172.21.15.31:0/1082734681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2580671f0 con 0x7fb25809c900 2023-11-26T07:17:58.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.541+0000 7fb23ffff640 1 -- 172.21.15.31:0/1082734681 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb250005580 con 0x7fb25809c900 2023-11-26T07:17:58.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.541+0000 7fb23ffff640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb244066470 0x7fb244068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.541+0000 7fb23ffff640 1 -- 172.21.15.31:0/1082734681 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb250083790 con 0x7fb25809c900 2023-11-26T07:17:58.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.541+0000 7fb2577fe640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb244066470 0x7fb244068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.541+0000 7fb2577fe640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb244066470 0x7fb244068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb2480af000 tx=0x7fb2480b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.545+0000 7fb23ffff640 1 -- 172.21.15.31:0/1082734681 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb250050ba0 con 0x7fb25809c900 2023-11-26T07:17:58.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 -- 172.21.15.31:0/1082734681 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb25806b460 con 0x7fb25809c900 2023-11-26T07:17:58.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb23ffff640 1 -- 172.21.15.31:0/1082734681 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2500170a0 con 0x7fb25809c900 2023-11-26T07:17:58.704 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:17:58.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 -- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb244066470 msgr2=0x7fb244068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb244066470 0x7fb244068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb2480af000 tx=0x7fb2480b6040 comp rx=0 tx=0).stop 2023-11-26T07:17:58.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 -- 172.21.15.31:0/1082734681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb25809c900 msgr2=0x7fb258140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb25809c900 0x7fb258140a80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb250001570 tx=0x7fb25000cd20 comp rx=0 tx=0).stop 2023-11-26T07:17:58.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 -- 172.21.15.31:0/1082734681 shutdown_connections 2023-11-26T07:17:58.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb244066470 0x7fb244068930 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb25809c900 0x7fb258140a80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2580ae680 0x7fb25809c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 --2- 172.21.15.31:0/1082734681 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb258070890 0x7fb258099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 -- 172.21.15.31:0/1082734681 >> 172.21.15.31:0/1082734681 conn(0x7fb2580b0cb0 msgr2=0x7fb2580ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 -- 172.21.15.31:0/1082734681 shutdown_connections 2023-11-26T07:17:58.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.701+0000 7fb25e1c1640 1 -- 172.21.15.31:0/1082734681 wait complete. 2023-11-26T07:17:58.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:17:58.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac730f640 1 Processor -- start 2023-11-26T07:17:58.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac730f640 1 -- start start 2023-11-26T07:17:58.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac730f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faac00ae3d0 0x7faac006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac730f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac00ae7f0 con 0x7faac0143ae0 2023-11-26T07:17:58.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac730f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac006e980 con 0x7faac006dfd0 2023-11-26T07:17:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac730f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac0070670 con 0x7faac00ae3d0 2023-11-26T07:17:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac50ab640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faac0143ae0 0x7faac00b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49118/0 (socket says 172.21.15.31:49118) 2023-11-26T07:17:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac50ab640 1 -- 172.21.15.31:0/1496732081 learned_addr learned my addr 172.21.15.31:0/1496732081 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac58ac640 1 --2- 172.21.15.31:0/1496732081 >> v2:172.21.15.31:3300/0 conn(0x7faac00ae3d0 0x7faac006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faab7fff640 1 -- 172.21.15.31:0/1496732081 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575411291 0 0) 0x7faac00ae7f0 con 0x7faac0143ae0 2023-11-26T07:17:58.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faab7fff640 1 -- 172.21.15.31:0/1496732081 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaac003530 con 0x7faac0143ae0 2023-11-26T07:17:58.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac58ac640 1 -- 172.21.15.31:0/1496732081 >> v1:172.21.15.97:6789/0 conn(0x7faac006dfd0 legacy=0x7faac00adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac58ac640 1 -- 172.21.15.31:0/1496732081 >> v1:172.21.15.31:6789/0 conn(0x7faac0143ae0 legacy=0x7faac00b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.865+0000 7faac58ac640 1 -- 172.21.15.31:0/1496732081 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0070950 con 0x7faac00ae3d0 2023-11-26T07:17:58.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faac58ac640 1 --2- 172.21.15.31:0/1496732081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac00ae3d0 0x7faac006e440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faab8009d90 tx=0x7faab801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d3c82764a468ef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faab7fff640 1 -- 172.21.15.31:0/1496732081 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab80a59e0 con 0x7faac00ae3d0 2023-11-26T07:17:58.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faab7fff640 1 -- 172.21.15.31:0/1496732081 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab80a5b40 con 0x7faac00ae3d0 2023-11-26T07:17:58.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faab7fff640 1 -- 172.21.15.31:0/1496732081 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab80ad440 con 0x7faac00ae3d0 2023-11-26T07:17:58.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faac730f640 1 -- 172.21.15.31:0/1496732081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac00ae3d0 msgr2=0x7faac006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faac730f640 1 --2- 172.21.15.31:0/1496732081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac00ae3d0 0x7faac006e440 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faab8009d90 tx=0x7faab801c920 comp rx=0 tx=0).stop 2023-11-26T07:17:58.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faac730f640 1 -- 172.21.15.31:0/1496732081 shutdown_connections 2023-11-26T07:17:58.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faac730f640 1 --2- 172.21.15.31:0/1496732081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac00ae3d0 0x7faac006e440 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faac730f640 1 -- 172.21.15.31:0/1496732081 >> 172.21.15.31:0/1496732081 conn(0x7faac00b0c40 msgr2=0x7faac00b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:58.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faac730f640 1 -- 172.21.15.31:0/1496732081 shutdown_connections 2023-11-26T07:17:58.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.869+0000 7faac730f640 1 -- 172.21.15.31:0/1496732081 wait complete. 2023-11-26T07:17:58.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac730f640 1 Processor -- start 2023-11-26T07:17:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac730f640 1 -- start start 2023-11-26T07:17:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac730f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faac006dfd0 0x7faac015d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac730f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac00ae3d0 0x7faac014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac730f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac0143ae0 0x7faac01f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac730f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac015da30 con 0x7faac0143ae0 2023-11-26T07:17:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac730f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac014cde0 con 0x7faac00ae3d0 2023-11-26T07:17:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac730f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac01f7730 con 0x7faac006dfd0 2023-11-26T07:17:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac58ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac0143ae0 0x7faac01f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac58ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac0143ae0 0x7faac01f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40408/0 (socket says 172.21.15.31:40408) 2023-11-26T07:17:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac58ac640 1 -- 172.21.15.31:0/1701745341 learned_addr learned my addr 172.21.15.31:0/1701745341 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac48aa640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac00ae3d0 0x7faac014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac50ab640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faac006dfd0 0x7faac015d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac58ac640 1 -- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faac006dfd0 msgr2=0x7faac015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac58ac640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faac006dfd0 0x7faac015d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac58ac640 1 -- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac00ae3d0 msgr2=0x7faac014c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac58ac640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac00ae3d0 0x7faac014c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac58ac640 1 -- 172.21.15.31:0/1701745341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab8009a70 con 0x7faac0143ae0 2023-11-26T07:17:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac58ac640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac0143ae0 0x7faac01f6290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faab80b0040 tx=0x7faab80196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faab5ffb640 1 -- 172.21.15.31:0/1701745341 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab80add90 con 0x7faac0143ae0 2023-11-26T07:17:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faab5ffb640 1 -- 172.21.15.31:0/1701745341 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab8009450 con 0x7faac0143ae0 2023-11-26T07:17:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faab5ffb640 1 -- 172.21.15.31:0/1701745341 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab80b9690 con 0x7faac0143ae0 2023-11-26T07:17:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac730f640 1 -- 172.21.15.31:0/1701745341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac01f67d0 con 0x7faac0143ae0 2023-11-26T07:17:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.873+0000 7faac730f640 1 -- 172.21.15.31:0/1701745341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac01f6d10 con 0x7faac0143ae0 2023-11-26T07:17:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.877+0000 7faac730f640 1 -- 172.21.15.31:0/1701745341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac01f78e0 con 0x7faac0143ae0 2023-11-26T07:17:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.877+0000 7faab5ffb640 1 -- 172.21.15.31:0/1701745341 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faab80b97f0 con 0x7faac0143ae0 2023-11-26T07:17:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.877+0000 7faab5ffb640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faaa4066470 0x7faaa4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.877+0000 7faab5ffb640 1 -- 172.21.15.31:0/1701745341 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faab811f580 con 0x7faac0143ae0 2023-11-26T07:17:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.877+0000 7faac50ab640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faaa4066470 0x7faaa4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:58.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.877+0000 7faac50ab640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faaa4066470 0x7faaa4068930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faabc0011f0 tx=0x7faabc007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:58.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:58.881+0000 7faab5ffb640 1 -- 172.21.15.31:0/1701745341 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faac01f78e0 con 0x7faac0143ae0 2023-11-26T07:17:59.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.037+0000 7faac730f640 1 -- 172.21.15.31:0/1701745341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faac01f78e0 con 0x7faac0143ae0 2023-11-26T07:17:59.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.037+0000 7faab5ffb640 1 -- 172.21.15.31:0/1701745341 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7faac01f78e0 con 0x7faac0143ae0 2023-11-26T07:17:59.043 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:17:59.043 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:17:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 -- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faaa4066470 msgr2=0x7faaa4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faaa4066470 0x7faaa4068930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faabc0011f0 tx=0x7faabc007d40 comp rx=0 tx=0).stop 2023-11-26T07:17:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 -- 172.21.15.31:0/1701745341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac0143ae0 msgr2=0x7faac01f6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac0143ae0 0x7faac01f6290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faab80b0040 tx=0x7faab80196c0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 -- 172.21.15.31:0/1701745341 shutdown_connections 2023-11-26T07:17:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faaa4066470 0x7faaa4068930 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faac0143ae0 0x7faac01f6290 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faac00ae3d0 0x7faac014c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 --2- 172.21.15.31:0/1701745341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faac006dfd0 0x7faac015d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 -- 172.21.15.31:0/1701745341 >> 172.21.15.31:0/1701745341 conn(0x7faac00b0c40 msgr2=0x7faac00b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 -- 172.21.15.31:0/1701745341 shutdown_connections 2023-11-26T07:17:59.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.041+0000 7faac730f640 1 -- 172.21.15.31:0/1701745341 wait complete. 2023-11-26T07:17:59.064 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:17:59.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:17:59.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.205+0000 7f48107ec640 1 Processor -- start 2023-11-26T07:17:59.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.205+0000 7f48107ec640 1 -- start start 2023-11-26T07:17:59.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.205+0000 7f48107ec640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48080cb9b0 0x7f480806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.205+0000 7f48107ec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f48080cbd90 0x7f480815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.205+0000 7f48107ec640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4808159630 con 0x7f48080ce5d0 2023-11-26T07:17:59.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.205+0000 7f48107ec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480815bba0 con 0x7f48080cbd90 2023-11-26T07:17:59.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.205+0000 7f48107ec640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080cdd50 con 0x7f48080cb9b0 2023-11-26T07:17:59.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.205+0000 7f480dd87640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f48080cbd90 0x7f480815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480ed89640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f48080ce5d0 0x7f480815bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49134/0 (socket says 172.21.15.31:49134) 2023-11-26T07:17:59.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480ed89640 1 -- 172.21.15.31:0/3456347537 learned_addr learned my addr 172.21.15.31:0/3456347537 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:59.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480e588640 1 --2- 172.21.15.31:0/3456347537 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48080cb9b0 0x7f480806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480dd87640 1 -- 172.21.15.31:0/3456347537 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48080cb9b0 msgr2=0x7f480806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:17:59.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480dd87640 1 --2- 172.21.15.31:0/3456347537 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48080cb9b0 0x7f480806e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480dd87640 1 -- 172.21.15.31:0/3456347537 >> v1:172.21.15.31:6789/0 conn(0x7f48080ce5d0 legacy=0x7f480815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480d586640 1 -- 172.21.15.31:0/3456347537 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887291338 0 0) 0x7f4808159630 con 0x7f48080ce5d0 2023-11-26T07:17:59.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480dd87640 1 -- 172.21.15.31:0/3456347537 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480815f6d0 con 0x7f48080cbd90 2023-11-26T07:17:59.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480dd87640 1 --2- 172.21.15.31:0/3456347537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 0x7f480815f190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47f80097c0 tx=0x7f47f801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e32618b827bfc0b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480d586640 1 -- 172.21.15.31:0/3456347537 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80ab650 con 0x7f48080cbd90 2023-11-26T07:17:59.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480d586640 1 -- 172.21.15.31:0/3456347537 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f80ab7b0 con 0x7f48080cbd90 2023-11-26T07:17:59.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.209+0000 7f480d586640 1 -- 172.21.15.31:0/3456347537 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80abb40 con 0x7f48080cbd90 2023-11-26T07:17:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 -- 172.21.15.31:0/3456347537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 msgr2=0x7f480815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 --2- 172.21.15.31:0/3456347537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 0x7f480815f190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f47f80097c0 tx=0x7f47f801c450 comp rx=0 tx=0).stop 2023-11-26T07:17:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 -- 172.21.15.31:0/3456347537 shutdown_connections 2023-11-26T07:17:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 --2- 172.21.15.31:0/3456347537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 0x7f480815f190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 --2- 172.21.15.31:0/3456347537 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48080cb9b0 0x7f480806e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 -- 172.21.15.31:0/3456347537 >> 172.21.15.31:0/3456347537 conn(0x7f48080b0cb0 msgr2=0x7f48080b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 -- 172.21.15.31:0/3456347537 shutdown_connections 2023-11-26T07:17:59.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 -- 172.21.15.31:0/3456347537 wait complete. 2023-11-26T07:17:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 Processor -- start 2023-11-26T07:17:59.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 -- start start 2023-11-26T07:17:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080cb9b0 0x7f4808159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 0x7f480815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48080ce5d0 0x7f48080afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480815a480 con 0x7f48080cbd90 2023-11-26T07:17:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48081594a0 con 0x7f48080cb9b0 2023-11-26T07:17:59.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f48107ec640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480815b2c0 con 0x7f48080ce5d0 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f480dd87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 0x7f480815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f480dd87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 0x7f480815aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40426/0 (socket says 172.21.15.31:40426) 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f480dd87640 1 -- 172.21.15.31:0/3213006979 learned_addr learned my addr 172.21.15.31:0/3213006979 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f480e588640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080cb9b0 0x7f4808159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f480ed89640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48080ce5d0 0x7f48080afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f480dd87640 1 -- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48080ce5d0 msgr2=0x7f48080afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f480dd87640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48080ce5d0 0x7f48080afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f480dd87640 1 -- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080cb9b0 msgr2=0x7f4808159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f480dd87640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080cb9b0 0x7f4808159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.213+0000 7f480dd87640 1 -- 172.21.15.31:0/3213006979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f8009420 con 0x7f48080cbd90 2023-11-26T07:17:59.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f480dd87640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 0x7f480815aa70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f47f80ad040 tx=0x7f47f801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f47feffd640 1 -- 172.21.15.31:0/3213006979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80bf610 con 0x7f48080cbd90 2023-11-26T07:17:59.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f47feffd640 1 -- 172.21.15.31:0/3213006979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f80bf790 con 0x7f48080cbd90 2023-11-26T07:17:59.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f47feffd640 1 -- 172.21.15.31:0/3213006979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80bfb40 con 0x7f48080cbd90 2023-11-26T07:17:59.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f48107ec640 1 -- 172.21.15.31:0/3213006979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4808159740 con 0x7f48080cbd90 2023-11-26T07:17:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f48107ec640 1 -- 172.21.15.31:0/3213006979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480815b5a0 con 0x7f48080cbd90 2023-11-26T07:17:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f48107ec640 1 -- 172.21.15.31:0/3213006979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48080671f0 con 0x7f48080cbd90 2023-11-26T07:17:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f47feffd640 1 -- 172.21.15.31:0/3213006979 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f47f80b5400 con 0x7f48080cbd90 2023-11-26T07:17:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f47feffd640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f47ec066470 0x7f47ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f47feffd640 1 -- 172.21.15.31:0/3213006979 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47f811eb00 con 0x7f48080cbd90 2023-11-26T07:17:59.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.217+0000 7f480e588640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f47ec066470 0x7f47ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.221+0000 7f480e588640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f47ec066470 0x7f47ec068930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f480815aa00 tx=0x7f4804009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.221+0000 7f47feffd640 1 -- 172.21.15.31:0/3213006979 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f47f8120e60 con 0x7f48080cbd90 2023-11-26T07:17:59.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.381+0000 7f48107ec640 1 -- 172.21.15.31:0/3213006979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4808159c60 con 0x7f48080cbd90 2023-11-26T07:17:59.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.381+0000 7f47feffd640 1 -- 172.21.15.31:0/3213006979 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4808159c60 con 0x7f48080cbd90 2023-11-26T07:17:59.387 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:17:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 -- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f47ec066470 msgr2=0x7f47ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f47ec066470 0x7f47ec068930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f480815aa00 tx=0x7f4804009190 comp rx=0 tx=0).stop 2023-11-26T07:17:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 -- 172.21.15.31:0/3213006979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 msgr2=0x7f480815aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 0x7f480815aa70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f47f80ad040 tx=0x7f47f801ac90 comp rx=0 tx=0).stop 2023-11-26T07:17:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 -- 172.21.15.31:0/3213006979 shutdown_connections 2023-11-26T07:17:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f47ec066470 0x7f47ec068930 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48080ce5d0 0x7f48080afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48080cbd90 0x7f480815aa70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 --2- 172.21.15.31:0/3213006979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48080cb9b0 0x7f4808159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 -- 172.21.15.31:0/3213006979 >> 172.21.15.31:0/3213006979 conn(0x7f48080b0cb0 msgr2=0x7f4808071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 -- 172.21.15.31:0/3213006979 shutdown_connections 2023-11-26T07:17:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.385+0000 7f48107ec640 1 -- 172.21.15.31:0/3213006979 wait complete. 2023-11-26T07:17:59.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:17:59.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.549+0000 7fa8782fe640 1 Processor -- start 2023-11-26T07:17:59.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.549+0000 7fa8782fe640 1 -- start start 2023-11-26T07:17:59.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.549+0000 7fa8782fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa8700b5830 0x7fa87015e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.549+0000 7fa8782fe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8700af630 con 0x7fa8700ad8e0 2023-11-26T07:17:59.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.549+0000 7fa8782fe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8700acb10 con 0x7fa8700acd70 2023-11-26T07:17:59.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.549+0000 7fa8782fe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700ae5b0 con 0x7fa8700b5830 2023-11-26T07:17:59.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.549+0000 7fa87609a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa8700b5830 0x7fa87015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.549+0000 7fa87689b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa8700ad8e0 0x7fa87006d190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49150/0 (socket says 172.21.15.31:49150) 2023-11-26T07:17:59.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.549+0000 7fa87689b640 1 -- 172.21.15.31:0/1539905948 learned_addr learned my addr 172.21.15.31:0/1539905948 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:59.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa87609a640 1 -- 172.21.15.31:0/1539905948 >> v1:172.21.15.97:6789/0 conn(0x7fa8700acd70 legacy=0x7fa8700ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa87609a640 1 -- 172.21.15.31:0/1539905948 >> v1:172.21.15.31:6789/0 conn(0x7fa8700ad8e0 legacy=0x7fa87006d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa87609a640 1 -- 172.21.15.31:0/1539905948 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700ae870 con 0x7fa8700b5830 2023-11-26T07:17:59.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa87609a640 1 --2- 172.21.15.31:0/1539905948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 0x7fa87015e690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0097c0 tx=0x7fa86c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b1e427db30972 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa875098640 1 -- 172.21.15.31:0/1539905948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0ab680 con 0x7fa8700b5830 2023-11-26T07:17:59.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa875098640 1 -- 172.21.15.31:0/1539905948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c0ab7e0 con 0x7fa8700b5830 2023-11-26T07:17:59.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa875098640 1 -- 172.21.15.31:0/1539905948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0abb70 con 0x7fa8700b5830 2023-11-26T07:17:59.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa8782fe640 1 -- 172.21.15.31:0/1539905948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 msgr2=0x7fa87015e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa8782fe640 1 --2- 172.21.15.31:0/1539905948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 0x7fa87015e690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0097c0 tx=0x7fa86c01c480 comp rx=0 tx=0).stop 2023-11-26T07:17:59.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa8782fe640 1 -- 172.21.15.31:0/1539905948 shutdown_connections 2023-11-26T07:17:59.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa8782fe640 1 --2- 172.21.15.31:0/1539905948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 0x7fa87015e690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.553+0000 7fa8782fe640 1 -- 172.21.15.31:0/1539905948 >> 172.21.15.31:0/1539905948 conn(0x7fa870071d80 msgr2=0x7fa87006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 -- 172.21.15.31:0/1539905948 shutdown_connections 2023-11-26T07:17:59.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 -- 172.21.15.31:0/1539905948 wait complete. 2023-11-26T07:17:59.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 Processor -- start 2023-11-26T07:17:59.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 -- start start 2023-11-26T07:17:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8700acd70 0x7fa8701543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8700ad8e0 0x7fa87014c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 0x7fa87015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa870153db0 con 0x7fa8700b5830 2023-11-26T07:17:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87014cb90 con 0x7fa8700acd70 2023-11-26T07:17:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8701568e0 con 0x7fa8700ad8e0 2023-11-26T07:17:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa87689b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 0x7fa87015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa87689b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 0x7fa87015aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40450/0 (socket says 172.21.15.31:40450) 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa87689b640 1 -- 172.21.15.31:0/1405545766 learned_addr learned my addr 172.21.15.31:0/1405545766 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa87609a640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8700acd70 0x7fa8701543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa875899640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8700ad8e0 0x7fa87014c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa87689b640 1 -- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8700ad8e0 msgr2=0x7fa87014c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa87689b640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8700ad8e0 0x7fa87014c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa87689b640 1 -- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8700acd70 msgr2=0x7fa8701543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa87689b640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8700acd70 0x7fa8701543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa87689b640 1 -- 172.21.15.31:0/1405545766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c009420 con 0x7fa8700b5830 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa875899640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8700ad8e0 0x7fa87014c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa87689b640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 0x7fa87015aae0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa86800cbf0 tx=0x7fa86800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa866ffd640 1 -- 172.21.15.31:0/1405545766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86800ad40 con 0x7fa8700b5830 2023-11-26T07:17:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa866ffd640 1 -- 172.21.15.31:0/1405545766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa868012400 con 0x7fa8700b5830 2023-11-26T07:17:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa866ffd640 1 -- 172.21.15.31:0/1405545766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86800aea0 con 0x7fa8700b5830 2023-11-26T07:17:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.557+0000 7fa8782fe640 1 -- 172.21.15.31:0/1405545766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa870156bc0 con 0x7fa8700b5830 2023-11-26T07:17:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.561+0000 7fa8782fe640 1 -- 172.21.15.31:0/1405545766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa870157180 con 0x7fa8700b5830 2023-11-26T07:17:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.561+0000 7fa8782fe640 1 -- 172.21.15.31:0/1405545766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8701573c0 con 0x7fa8700b5830 2023-11-26T07:17:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.561+0000 7fa866ffd640 1 -- 172.21.15.31:0/1405545766 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa868007070 con 0x7fa8700b5830 2023-11-26T07:17:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.561+0000 7fa866ffd640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa854066470 0x7fa854068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.561+0000 7fa866ffd640 1 -- 172.21.15.31:0/1405545766 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa868083b20 con 0x7fa8700b5830 2023-11-26T07:17:59.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.561+0000 7fa87609a640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa854066470 0x7fa854068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.565+0000 7fa87609a640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa854066470 0x7fa854068930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0af000 tx=0x7fa86c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.565+0000 7fa866ffd640 1 -- 172.21.15.31:0/1405545766 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa8701573c0 con 0x7fa8700b5830 2023-11-26T07:17:59.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.721+0000 7fa8782fe640 1 -- 172.21.15.31:0/1405545766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8701573c0 con 0x7fa8700b5830 2023-11-26T07:17:59.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.721+0000 7fa866ffd640 1 -- 172.21.15.31:0/1405545766 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa8701573c0 con 0x7fa8700b5830 2023-11-26T07:17:59.726 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:17:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 -- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa854066470 msgr2=0x7fa854068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa854066470 0x7fa854068930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0af000 tx=0x7fa86c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:17:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 -- 172.21.15.31:0/1405545766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 msgr2=0x7fa87015aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 0x7fa87015aae0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa86800cbf0 tx=0x7fa86800a430 comp rx=0 tx=0).stop 2023-11-26T07:17:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 -- 172.21.15.31:0/1405545766 shutdown_connections 2023-11-26T07:17:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa854066470 0x7fa854068930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8700b5830 0x7fa87015aae0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8700ad8e0 0x7fa87014c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 --2- 172.21.15.31:0/1405545766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8700acd70 0x7fa8701543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 -- 172.21.15.31:0/1405545766 >> 172.21.15.31:0/1405545766 conn(0x7fa870071d80 msgr2=0x7fa870153210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 -- 172.21.15.31:0/1405545766 shutdown_connections 2023-11-26T07:17:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.725+0000 7fa8782fe640 1 -- 172.21.15.31:0/1405545766 wait complete. 2023-11-26T07:17:59.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:17:59.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f75247c1640 1 Processor -- start 2023-11-26T07:17:59.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f75247c1640 1 -- start start 2023-11-26T07:17:59.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f75247c1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f751c0ce5d0 0x7f751c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f75247c1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751c159630 con 0x7f751c0cb9b0 2023-11-26T07:17:59.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f75247c1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751c0cd1d0 con 0x7f751c06e3f0 2023-11-26T07:17:59.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f75247c1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751c06e860 con 0x7f751c0ce5d0 2023-11-26T07:17:59.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f752255d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f751c0cb9b0 0x7f751c0cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49164/0 (socket says 172.21.15.31:49164) 2023-11-26T07:17:59.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f752255d640 1 -- 172.21.15.31:0/1862816569 learned_addr learned my addr 172.21.15.31:0/1862816569 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:17:59.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f7522d5e640 1 --2- 172.21.15.31:0/1862816569 >> v2:172.21.15.31:3300/0 conn(0x7f751c0ce5d0 0x7f751c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f7522d5e640 1 -- 172.21.15.31:0/1862816569 >> v1:172.21.15.97:6789/0 conn(0x7f751c06e3f0 legacy=0x7f751c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f7522d5e640 1 -- 172.21.15.31:0/1862816569 >> v1:172.21.15.31:6789/0 conn(0x7f751c0cb9b0 legacy=0x7f751c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.889+0000 7f7522d5e640 1 -- 172.21.15.31:0/1862816569 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c0cd4b0 con 0x7f751c0ce5d0 2023-11-26T07:17:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f7522d5e640 1 --2- 172.21.15.31:0/1862816569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0ce5d0 0x7f751c15bfd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7514009d90 tx=0x7f751401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61570ec72fbbc69d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f752155b640 1 -- 172.21.15.31:0/1862816569 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75140a57b0 con 0x7f751c0ce5d0 2023-11-26T07:17:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f752155b640 1 -- 172.21.15.31:0/1862816569 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75140a5910 con 0x7f751c0ce5d0 2023-11-26T07:17:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f752155b640 1 -- 172.21.15.31:0/1862816569 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75140a5ca0 con 0x7f751c0ce5d0 2023-11-26T07:17:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f75247c1640 1 -- 172.21.15.31:0/1862816569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0ce5d0 msgr2=0x7f751c15bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f75247c1640 1 --2- 172.21.15.31:0/1862816569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0ce5d0 0x7f751c15bfd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7514009d90 tx=0x7f751401c910 comp rx=0 tx=0).stop 2023-11-26T07:17:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f75247c1640 1 -- 172.21.15.31:0/1862816569 shutdown_connections 2023-11-26T07:17:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f75247c1640 1 --2- 172.21.15.31:0/1862816569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0ce5d0 0x7f751c15bfd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f75247c1640 1 -- 172.21.15.31:0/1862816569 >> 172.21.15.31:0/1862816569 conn(0x7f751c0b0cb0 msgr2=0x7f751c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:17:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f75247c1640 1 -- 172.21.15.31:0/1862816569 shutdown_connections 2023-11-26T07:17:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.893+0000 7f75247c1640 1 -- 172.21.15.31:0/1862816569 wait complete. 2023-11-26T07:17:59.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f75247c1640 1 Processor -- start 2023-11-26T07:17:59.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f75247c1640 1 -- start start 2023-11-26T07:17:59.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f75247c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c06e3f0 0x7f751c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f75247c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0cb9b0 0x7f751c1fa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f75247c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0ce5d0 0x7f751c150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f75247c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c1fad80 con 0x7f751c0cb9b0 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f75247c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c0ad1f0 con 0x7f751c06e3f0 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f75247c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751c0ac5e0 con 0x7f751c0ce5d0 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7521d5c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0cb9b0 0x7f751c1fa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7521d5c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0cb9b0 0x7f751c1fa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40454/0 (socket says 172.21.15.31:40454) 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7521d5c640 1 -- 172.21.15.31:0/1165386512 learned_addr learned my addr 172.21.15.31:0/1165386512 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f752255d640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c06e3f0 0x7f751c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7522d5e640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0ce5d0 0x7f751c150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7521d5c640 1 -- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0ce5d0 msgr2=0x7f751c150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7521d5c640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0ce5d0 0x7f751c150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7521d5c640 1 -- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c06e3f0 msgr2=0x7f751c0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:17:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7521d5c640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c06e3f0 0x7f751c0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:17:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7521d5c640 1 -- 172.21.15.31:0/1165386512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7514009a70 con 0x7f751c0cb9b0 2023-11-26T07:17:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f752255d640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c06e3f0 0x7f751c0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7522d5e640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0ce5d0 0x7f751c150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:17:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7521d5c640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0cb9b0 0x7f751c1fa840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f750c00c550 tx=0x7f750c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7512ffd640 1 -- 172.21.15.31:0/1165386512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c012830 con 0x7f751c0cb9b0 2023-11-26T07:17:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7512ffd640 1 -- 172.21.15.31:0/1165386512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750c012990 con 0x7f751c0cb9b0 2023-11-26T07:17:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f7512ffd640 1 -- 172.21.15.31:0/1165386512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c012d40 con 0x7f751c0cb9b0 2023-11-26T07:17:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f75247c1640 1 -- 172.21.15.31:0/1165386512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751c0ac8c0 con 0x7f751c0cb9b0 2023-11-26T07:17:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.897+0000 7f75247c1640 1 -- 172.21.15.31:0/1165386512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751c0ace80 con 0x7f751c0cb9b0 2023-11-26T07:17:59.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.901+0000 7f75247c1640 1 -- 172.21.15.31:0/1165386512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751c0671f0 con 0x7f751c0cb9b0 2023-11-26T07:17:59.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.901+0000 7f7512ffd640 1 -- 172.21.15.31:0/1165386512 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f750c01b400 con 0x7f751c0cb9b0 2023-11-26T07:17:59.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.901+0000 7f7512ffd640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066470 0x7f7500068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:17:59.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.901+0000 7f7512ffd640 1 -- 172.21.15.31:0/1165386512 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f750c0840b0 con 0x7f751c0cb9b0 2023-11-26T07:17:59.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.901+0000 7f752255d640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066470 0x7f7500068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:17:59.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.901+0000 7f752255d640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066470 0x7f7500068930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7518007fd0 tx=0x7f7518009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:17:59.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:17:59.905+0000 7f7512ffd640 1 -- 172.21.15.31:0/1165386512 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f750c0515e0 con 0x7f751c0cb9b0 2023-11-26T07:18:00.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:00.043 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:00.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.065+0000 7f75247c1640 1 -- 172.21.15.31:0/1165386512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f751c06b460 con 0x7f751c0cb9b0 2023-11-26T07:18:00.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.065+0000 7f7512ffd640 1 -- 172.21.15.31:0/1165386512 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f750c0160a0 con 0x7f751c0cb9b0 2023-11-26T07:18:00.070 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:18:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 -- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066470 msgr2=0x7f7500068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066470 0x7f7500068930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7518007fd0 tx=0x7f7518009040 comp rx=0 tx=0).stop 2023-11-26T07:18:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 -- 172.21.15.31:0/1165386512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0cb9b0 msgr2=0x7f751c1fa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0cb9b0 0x7f751c1fa840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f750c00c550 tx=0x7f750c00ca10 comp rx=0 tx=0).stop 2023-11-26T07:18:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 -- 172.21.15.31:0/1165386512 shutdown_connections 2023-11-26T07:18:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066470 0x7f7500068930 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0ce5d0 0x7f751c150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0cb9b0 0x7f751c1fa840 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 --2- 172.21.15.31:0/1165386512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c06e3f0 0x7f751c0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 -- 172.21.15.31:0/1165386512 >> 172.21.15.31:0/1165386512 conn(0x7f751c0b0cb0 msgr2=0x7f751c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 -- 172.21.15.31:0/1165386512 shutdown_connections 2023-11-26T07:18:00.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.069+0000 7f75247c1640 1 -- 172.21.15.31:0/1165386512 wait complete. 2023-11-26T07:18:00.160 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2023-11-26T07:18:00.160 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:18:00.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:00.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.341+0000 7fcb9d935640 1 Processor -- start 2023-11-26T07:18:00.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.341+0000 7fcb9d935640 1 -- start start 2023-11-26T07:18:00.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.341+0000 7fcb9d935640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb98143ae0 0x7fcb980b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.341+0000 7fcb9d935640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcb980ae3d0 0x7fcb9806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.341+0000 7fcb9d935640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb980ae7f0 con 0x7fcb9806dfd0 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.341+0000 7fcb9d935640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9806e980 con 0x7fcb98143ae0 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.341+0000 7fcb9d935640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb980b2480 con 0x7fcb980ae3d0 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb96ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb98143ae0 0x7fcb980b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb96ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcb98143ae0 0x7fcb980b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40462/0 (socket says 172.21.15.31:40462) 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb96ffd640 1 -- 172.21.15.31:0/702479547 learned_addr learned my addr 172.21.15.31:0/702479547 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb977fe640 1 --2- 172.21.15.31:0/702479547 >> v2:172.21.15.97:3300/0 conn(0x7fcb980ae3d0 0x7fcb9806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb96ffd640 1 -- 172.21.15.31:0/702479547 >> v2:172.21.15.97:3300/0 conn(0x7fcb980ae3d0 msgr2=0x7fcb9806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb96ffd640 1 --2- 172.21.15.31:0/702479547 >> v2:172.21.15.97:3300/0 conn(0x7fcb980ae3d0 0x7fcb9806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb96ffd640 1 -- 172.21.15.31:0/702479547 >> v1:172.21.15.31:6789/0 conn(0x7fcb9806dfd0 legacy=0x7fcb980adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb96ffd640 1 -- 172.21.15.31:0/702479547 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb98070670 con 0x7fcb98143ae0 2023-11-26T07:18:00.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb967fc640 1 -- 172.21.15.31:0/702479547 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138656463 0 0) 0x7fcb980ae7f0 con 0x7fcb9806dfd0 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb96ffd640 1 --2- 172.21.15.31:0/702479547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 0x7fcb980b1f40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb98072ad0 tx=0x7fcb8801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb339a962508d248 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb967fc640 1 -- 172.21.15.31:0/702479547 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880aa430 con 0x7fcb98143ae0 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb967fc640 1 -- 172.21.15.31:0/702479547 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb880aa590 con 0x7fcb98143ae0 2023-11-26T07:18:00.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.345+0000 7fcb967fc640 1 -- 172.21.15.31:0/702479547 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880aa920 con 0x7fcb98143ae0 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 -- 172.21.15.31:0/702479547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 msgr2=0x7fcb980b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 --2- 172.21.15.31:0/702479547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 0x7fcb980b1f40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb98072ad0 tx=0x7fcb8801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 -- 172.21.15.31:0/702479547 shutdown_connections 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 --2- 172.21.15.31:0/702479547 >> v2:172.21.15.97:3300/0 conn(0x7fcb980ae3d0 0x7fcb9806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 --2- 172.21.15.31:0/702479547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 0x7fcb980b1f40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 -- 172.21.15.31:0/702479547 >> 172.21.15.31:0/702479547 conn(0x7fcb980b0c40 msgr2=0x7fcb980b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 -- 172.21.15.31:0/702479547 shutdown_connections 2023-11-26T07:18:00.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 -- 172.21.15.31:0/702479547 wait complete. 2023-11-26T07:18:00.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 Processor -- start 2023-11-26T07:18:00.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 -- start start 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9806dfd0 0x7fcb98159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb980ae3d0 0x7fcb981f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 0x7fcb9815d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9809a900 con 0x7fcb98143ae0 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9809abe0 con 0x7fcb9806dfd0 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb9d935640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9809aec0 con 0x7fcb980ae3d0 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb977fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 0x7fcb9815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb977fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 0x7fcb9815d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40464/0 (socket says 172.21.15.31:40464) 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb977fe640 1 -- 172.21.15.31:0/1960059982 learned_addr learned my addr 172.21.15.31:0/1960059982 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb96ffd640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9806dfd0 0x7fcb98159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb8ffff640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb980ae3d0 0x7fcb981f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb977fe640 1 -- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb980ae3d0 msgr2=0x7fcb981f6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.349+0000 7fcb977fe640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb980ae3d0 0x7fcb981f6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb977fe640 1 -- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9806dfd0 msgr2=0x7fcb98159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb977fe640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9806dfd0 0x7fcb98159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb977fe640 1 -- 172.21.15.31:0/1960059982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb880094d0 con 0x7fcb98143ae0 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb977fe640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 0x7fcb9815d130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000efc0 tx=0x7fcb9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb94ff9640 1 -- 172.21.15.31:0/1960059982 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90014480 con 0x7fcb98143ae0 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb94ff9640 1 -- 172.21.15.31:0/1960059982 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9000f040 con 0x7fcb98143ae0 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb94ff9640 1 -- 172.21.15.31:0/1960059982 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90014820 con 0x7fcb98143ae0 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb9d935640 1 -- 172.21.15.31:0/1960059982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9809b1a0 con 0x7fcb98143ae0 2023-11-26T07:18:00.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb9d935640 1 -- 172.21.15.31:0/1960059982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb981f6690 con 0x7fcb98143ae0 2023-11-26T07:18:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb9d935640 1 -- 172.21.15.31:0/1960059982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9809b6c0 con 0x7fcb98143ae0 2023-11-26T07:18:00.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb94ff9640 1 -- 172.21.15.31:0/1960059982 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fcb90014980 con 0x7fcb98143ae0 2023-11-26T07:18:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb94ff9640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcb78066470 0x7fcb78068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb94ff9640 1 -- 172.21.15.31:0/1960059982 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb90084950 con 0x7fcb98143ae0 2023-11-26T07:18:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.353+0000 7fcb96ffd640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcb78066470 0x7fcb78068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.357+0000 7fcb96ffd640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcb78066470 0x7fcb78068930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb880ae000 tx=0x7fcb880b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.357+0000 7fcb94ff9640 1 -- 172.21.15.31:0/1960059982 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcb9809b6c0 con 0x7fcb98143ae0 2023-11-26T07:18:00.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.517+0000 7fcb9d935640 1 -- 172.21.15.31:0/1960059982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb9809b6c0 con 0x7fcb98143ae0 2023-11-26T07:18:00.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.517+0000 7fcb94ff9640 1 -- 172.21.15.31:0/1960059982 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fcb9809b6c0 con 0x7fcb98143ae0 2023-11-26T07:18:00.523 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:00.523 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:00.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 -- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcb78066470 msgr2=0x7fcb78068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcb78066470 0x7fcb78068930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb880ae000 tx=0x7fcb880b5040 comp rx=0 tx=0).stop 2023-11-26T07:18:00.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 -- 172.21.15.31:0/1960059982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 msgr2=0x7fcb9815d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 0x7fcb9815d130 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000efc0 tx=0x7fcb9000c490 comp rx=0 tx=0).stop 2023-11-26T07:18:00.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 -- 172.21.15.31:0/1960059982 shutdown_connections 2023-11-26T07:18:00.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcb78066470 0x7fcb78068930 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb98143ae0 0x7fcb9815d130 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcb980ae3d0 0x7fcb981f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 --2- 172.21.15.31:0/1960059982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb9806dfd0 0x7fcb98159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 -- 172.21.15.31:0/1960059982 >> 172.21.15.31:0/1960059982 conn(0x7fcb980b0c40 msgr2=0x7fcb98141b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 -- 172.21.15.31:0/1960059982 shutdown_connections 2023-11-26T07:18:00.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.521+0000 7fcb9d935640 1 -- 172.21.15.31:0/1960059982 wait complete. 2023-11-26T07:18:00.544 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:00.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:00.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.685+0000 7fe4eb92a640 1 Processor -- start 2023-11-26T07:18:00.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.685+0000 7fe4eb92a640 1 -- start start 2023-11-26T07:18:00.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4eb92a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe4e4144d50 0x7fe4e40b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4eb92a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe4e40b5160 0x7fe4e415eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4eb92a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4e415f420 0x7fe4e4070040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4eb92a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e415f8b0 con 0x7fe4e4144d50 2023-11-26T07:18:00.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4eb92a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e415fcb0 con 0x7fe4e40b5160 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4eb92a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40b5790 con 0x7fe4e415f420 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e96c6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe4e4144d50 0x7fe4e40b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e96c6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe4e4144d50 0x7fe4e40b4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40472/0 (socket says 172.21.15.31:40472) 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e96c6640 1 -- 172.21.15.31:0/17997690 learned_addr learned my addr 172.21.15.31:0/17997690 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e9ec7640 1 --2- 172.21.15.31:0/17997690 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4e415f420 0x7fe4e4070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e8ec5640 1 --2- 172.21.15.31:0/17997690 >> v2:172.21.15.97:3300/0 conn(0x7fe4e40b5160 0x7fe4e415eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e96c6640 1 -- 172.21.15.31:0/17997690 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4e415f420 msgr2=0x7fe4e4070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e96c6640 1 --2- 172.21.15.31:0/17997690 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4e415f420 0x7fe4e4070040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e96c6640 1 -- 172.21.15.31:0/17997690 >> v2:172.21.15.97:3300/0 conn(0x7fe4e40b5160 msgr2=0x7fe4e415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e96c6640 1 --2- 172.21.15.31:0/17997690 >> v2:172.21.15.97:3300/0 conn(0x7fe4e40b5160 0x7fe4e415eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e96c6640 1 -- 172.21.15.31:0/17997690 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e40b5a70 con 0x7fe4e4144d50 2023-11-26T07:18:00.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.689+0000 7fe4e96c6640 1 --2- 172.21.15.31:0/17997690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4144d50 0x7fe4e40b4c20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4067190 tx=0x7fe4e001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1e7e4d5973f58a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4dbfff640 1 -- 172.21.15.31:0/17997690 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00ab600 con 0x7fe4e4144d50 2023-11-26T07:18:00.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4dbfff640 1 -- 172.21.15.31:0/17997690 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e00a7050 con 0x7fe4e4144d50 2023-11-26T07:18:00.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4dbfff640 1 -- 172.21.15.31:0/17997690 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00ab980 con 0x7fe4e4144d50 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 -- 172.21.15.31:0/17997690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4144d50 msgr2=0x7fe4e40b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 --2- 172.21.15.31:0/17997690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4144d50 0x7fe4e40b4c20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4067190 tx=0x7fe4e001c240 comp rx=0 tx=0).stop 2023-11-26T07:18:00.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 -- 172.21.15.31:0/17997690 shutdown_connections 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 --2- 172.21.15.31:0/17997690 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4e415f420 0x7fe4e4070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 --2- 172.21.15.31:0/17997690 >> v2:172.21.15.97:3300/0 conn(0x7fe4e40b5160 0x7fe4e415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 --2- 172.21.15.31:0/17997690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e4144d50 0x7fe4e40b4c20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 -- 172.21.15.31:0/17997690 >> 172.21.15.31:0/17997690 conn(0x7fe4e40b2f10 msgr2=0x7fe4e40b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 -- 172.21.15.31:0/17997690 shutdown_connections 2023-11-26T07:18:00.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 -- 172.21.15.31:0/17997690 wait complete. 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 Processor -- start 2023-11-26T07:18:00.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 -- start start 2023-11-26T07:18:00.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4e40b5160 0x7fe4e413a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4e4144d50 0x7fe4e414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e415f420 0x7fe4e414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e415d3b0 con 0x7fe4e415f420 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e4156820 con 0x7fe4e4144d50 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.693+0000 7fe4eb92a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e415ca80 con 0x7fe4e40b5160 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e9ec7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e415f420 0x7fe4e414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e9ec7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e415f420 0x7fe4e414e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40484/0 (socket says 172.21.15.31:40484) 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e9ec7640 1 -- 172.21.15.31:0/3903232767 learned_addr learned my addr 172.21.15.31:0/3903232767 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e96c6640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4e40b5160 0x7fe4e413a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e8ec5640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4e4144d50 0x7fe4e414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e9ec7640 1 -- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4e40b5160 msgr2=0x7fe4e413a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e9ec7640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4e40b5160 0x7fe4e413a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e9ec7640 1 -- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4e4144d50 msgr2=0x7fe4e414e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e9ec7640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4e4144d50 0x7fe4e414e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e9ec7640 1 -- 172.21.15.31:0/3903232767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e00094d0 con 0x7fe4e415f420 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4e9ec7640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e415f420 0x7fe4e414e9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00efc0 tx=0x7fe4dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4d9ffb640 1 -- 172.21.15.31:0/3903232767 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc009ba0 con 0x7fe4e415f420 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4d9ffb640 1 -- 172.21.15.31:0/3903232767 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc00f040 con 0x7fe4e415f420 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4eb92a640 1 -- 172.21.15.31:0/3903232767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e415cd60 con 0x7fe4e415f420 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4d9ffb640 1 -- 172.21.15.31:0/3903232767 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc0146c0 con 0x7fe4e415f420 2023-11-26T07:18:00.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4eb92a640 1 -- 172.21.15.31:0/3903232767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e413ab10 con 0x7fe4e415f420 2023-11-26T07:18:00.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4eb92a640 1 -- 172.21.15.31:0/3903232767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e415d560 con 0x7fe4e415f420 2023-11-26T07:18:00.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.697+0000 7fe4d9ffb640 1 -- 172.21.15.31:0/3903232767 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe4dc014820 con 0x7fe4e415f420 2023-11-26T07:18:00.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.701+0000 7fe4d9ffb640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4c80664a0 0x7fe4c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:00.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.701+0000 7fe4d9ffb640 1 -- 172.21.15.31:0/3903232767 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4dc084a60 con 0x7fe4e415f420 2023-11-26T07:18:00.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.701+0000 7fe4e96c6640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4c80664a0 0x7fe4c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:00.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.701+0000 7fe4e96c6640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4c80664a0 0x7fe4c8068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00ad000 tx=0x7fe4e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:00.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.701+0000 7fe4d9ffb640 1 -- 172.21.15.31:0/3903232767 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe4e415d560 con 0x7fe4e415f420 2023-11-26T07:18:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 -- 172.21.15.31:0/3903232767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4e404e850 con 0x7fe4e415f420 2023-11-26T07:18:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4d9ffb640 1 -- 172.21.15.31:0/3903232767 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4dc0180a0 con 0x7fe4e415f420 2023-11-26T07:18:00.865 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 -- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4c80664a0 msgr2=0x7fe4c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4c80664a0 0x7fe4c8068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00ad000 tx=0x7fe4e0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 -- 172.21.15.31:0/3903232767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e415f420 msgr2=0x7fe4e414e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e415f420 0x7fe4e414e9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00efc0 tx=0x7fe4dc00c490 comp rx=0 tx=0).stop 2023-11-26T07:18:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 -- 172.21.15.31:0/3903232767 shutdown_connections 2023-11-26T07:18:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe4c80664a0 0x7fe4c8068960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4e415f420 0x7fe4e414e9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4e4144d50 0x7fe4e414e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 --2- 172.21.15.31:0/3903232767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe4e40b5160 0x7fe4e413a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 -- 172.21.15.31:0/3903232767 >> 172.21.15.31:0/3903232767 conn(0x7fe4e40b2f10 msgr2=0x7fe4e40b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.861+0000 7fe4eb92a640 1 -- 172.21.15.31:0/3903232767 shutdown_connections 2023-11-26T07:18:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:00.865+0000 7fe4eb92a640 1 -- 172.21.15.31:0/3903232767 wait complete. 2023-11-26T07:18:00.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:01.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa176afe640 1 Processor -- start 2023-11-26T07:18:01.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa176afe640 1 -- start start 2023-11-26T07:18:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa176afe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa1700cb9b0 0x7fa17006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa176afe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa170159630 con 0x7fa1700cbd90 2023-11-26T07:18:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa176afe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1700cd1d0 con 0x7fa1700ce5d0 2023-11-26T07:18:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa176afe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17015bba0 con 0x7fa1700cb9b0 2023-11-26T07:18:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa17489a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa1700cb9b0 0x7fa17006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa16ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa1700cbd90 0x7fa17015f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49182/0 (socket says 172.21.15.31:49182) 2023-11-26T07:18:01.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa16ffff640 1 -- 172.21.15.31:0/3058221397 learned_addr learned my addr 172.21.15.31:0/3058221397 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa17489a640 1 -- 172.21.15.31:0/3058221397 >> v1:172.21.15.97:6789/0 conn(0x7fa1700ce5d0 legacy=0x7fa17015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa17489a640 1 -- 172.21.15.31:0/3058221397 >> v1:172.21.15.31:6789/0 conn(0x7fa1700cbd90 legacy=0x7fa17015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa17489a640 1 -- 172.21.15.31:0/3058221397 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1700cd4b0 con 0x7fa1700cb9b0 2023-11-26T07:18:01.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.029+0000 7fa16f7fe640 1 -- 172.21.15.31:0/3058221397 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259187010 0 0) 0x7fa170159630 con 0x7fa1700cbd90 2023-11-26T07:18:01.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa17489a640 1 --2- 172.21.15.31:0/3058221397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 0x7fa17006e3f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa1640097c0 tx=0x7fa16401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc5d1da6b93d49a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:01.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa16f7fe640 1 -- 172.21.15.31:0/3058221397 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1640ab600 con 0x7fa1700cb9b0 2023-11-26T07:18:01.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa16f7fe640 1 -- 172.21.15.31:0/3058221397 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1640a7050 con 0x7fa1700cb9b0 2023-11-26T07:18:01.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa16f7fe640 1 -- 172.21.15.31:0/3058221397 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1640ab980 con 0x7fa1700cb9b0 2023-11-26T07:18:01.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa176afe640 1 -- 172.21.15.31:0/3058221397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 msgr2=0x7fa17006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa176afe640 1 --2- 172.21.15.31:0/3058221397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 0x7fa17006e3f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa1640097c0 tx=0x7fa16401c480 comp rx=0 tx=0).stop 2023-11-26T07:18:01.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa176afe640 1 -- 172.21.15.31:0/3058221397 shutdown_connections 2023-11-26T07:18:01.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa176afe640 1 --2- 172.21.15.31:0/3058221397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 0x7fa17006e3f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa176afe640 1 -- 172.21.15.31:0/3058221397 >> 172.21.15.31:0/3058221397 conn(0x7fa1700b0cb0 msgr2=0x7fa1700b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa176afe640 1 -- 172.21.15.31:0/3058221397 shutdown_connections 2023-11-26T07:18:01.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.033+0000 7fa176afe640 1 -- 172.21.15.31:0/3058221397 wait complete. 2023-11-26T07:18:01.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 Processor -- start 2023-11-26T07:18:01.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 -- start start 2023-11-26T07:18:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 0x7fa170099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1700cbd90 0x7fa1701660c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1700ce5d0 0x7fa17009c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17009b400 con 0x7fa1700cb9b0 2023-11-26T07:18:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa170099f70 con 0x7fa1700cbd90 2023-11-26T07:18:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17009b610 con 0x7fa1700ce5d0 2023-11-26T07:18:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17489a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 0x7fa170099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17489a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 0x7fa170099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40498/0 (socket says 172.21.15.31:40498) 2023-11-26T07:18:01.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17489a640 1 -- 172.21.15.31:0/1481441147 learned_addr learned my addr 172.21.15.31:0/1481441147 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:01.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa16ffff640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1700cbd90 0x7fa1701660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17489a640 1 -- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1700ce5d0 msgr2=0x7fa17009c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17509b640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1700ce5d0 0x7fa17009c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17489a640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1700ce5d0 0x7fa17009c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17489a640 1 -- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1700cbd90 msgr2=0x7fa1701660c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17489a640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1700cbd90 0x7fa1701660c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17489a640 1 -- 172.21.15.31:0/1481441147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa164009420 con 0x7fa1700cb9b0 2023-11-26T07:18:01.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa16ffff640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1700cbd90 0x7fa1701660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:01.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17509b640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1700ce5d0 0x7fa17009c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:01.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa17489a640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 0x7fa170099a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa16401a0b0 tx=0x7fa16401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:01.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa16d7fa640 1 -- 172.21.15.31:0/1481441147 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164004020 con 0x7fa1700cb9b0 2023-11-26T07:18:01.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa16d7fa640 1 -- 172.21.15.31:0/1481441147 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa164004180 con 0x7fa1700cb9b0 2023-11-26T07:18:01.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa16d7fa640 1 -- 172.21.15.31:0/1481441147 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1640044f0 con 0x7fa1700cb9b0 2023-11-26T07:18:01.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 -- 172.21.15.31:0/1481441147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17009b8f0 con 0x7fa1700cb9b0 2023-11-26T07:18:01.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 -- 172.21.15.31:0/1481441147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17009beb0 con 0x7fa1700cb9b0 2023-11-26T07:18:01.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.037+0000 7fa176afe640 1 -- 172.21.15.31:0/1481441147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1700671f0 con 0x7fa1700cb9b0 2023-11-26T07:18:01.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.041+0000 7fa16d7fa640 1 -- 172.21.15.31:0/1481441147 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa164003620 con 0x7fa1700cb9b0 2023-11-26T07:18:01.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.041+0000 7fa16d7fa640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa154066470 0x7fa154068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.041+0000 7fa16d7fa640 1 -- 172.21.15.31:0/1481441147 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa16411e4b0 con 0x7fa1700cb9b0 2023-11-26T07:18:01.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.041+0000 7fa16ffff640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa154066470 0x7fa154068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.041+0000 7fa16ffff640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa154066470 0x7fa154068930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa17015b7d0 tx=0x7fa15c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:01.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.045+0000 7fa16d7fa640 1 -- 172.21.15.31:0/1481441147 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa1640eb890 con 0x7fa1700cb9b0 2023-11-26T07:18:01.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.205+0000 7fa176afe640 1 -- 172.21.15.31:0/1481441147 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa17006b460 con 0x7fa1700cb9b0 2023-11-26T07:18:01.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.205+0000 7fa16d7fa640 1 -- 172.21.15.31:0/1481441147 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1640eeeb0 con 0x7fa1700cb9b0 2023-11-26T07:18:01.210 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:18:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 -- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa154066470 msgr2=0x7fa154068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa154066470 0x7fa154068930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa17015b7d0 tx=0x7fa15c009040 comp rx=0 tx=0).stop 2023-11-26T07:18:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 -- 172.21.15.31:0/1481441147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 msgr2=0x7fa170099a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 0x7fa170099a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa16401a0b0 tx=0x7fa16401acc0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 -- 172.21.15.31:0/1481441147 shutdown_connections 2023-11-26T07:18:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa154066470 0x7fa154068930 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa1700ce5d0 0x7fa17009c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1700cbd90 0x7fa1701660c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 --2- 172.21.15.31:0/1481441147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1700cb9b0 0x7fa170099a30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 -- 172.21.15.31:0/1481441147 >> 172.21.15.31:0/1481441147 conn(0x7fa1700b0cb0 msgr2=0x7fa17006e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 -- 172.21.15.31:0/1481441147 shutdown_connections 2023-11-26T07:18:01.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.209+0000 7fa176afe640 1 -- 172.21.15.31:0/1481441147 wait complete. 2023-11-26T07:18:01.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:01.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.373+0000 7fb11752a640 1 Processor -- start 2023-11-26T07:18:01.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.373+0000 7fb11752a640 1 -- start start 2023-11-26T07:18:01.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb11752a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb110147060 0x7fb1100b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb11752a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1100ad000 0x7fb11006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb11752a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1100b2cb0 con 0x7fb110070a50 2023-11-26T07:18:01.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb11752a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1100b2f90 con 0x7fb110147060 2023-11-26T07:18:01.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb11752a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1100b3270 con 0x7fb1100ad000 2023-11-26T07:18:01.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb1152c6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb110147060 0x7fb1100b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb1152c6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb110147060 0x7fb1100b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40510/0 (socket says 172.21.15.31:40510) 2023-11-26T07:18:01.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb1152c6640 1 -- 172.21.15.31:0/2291572052 learned_addr learned my addr 172.21.15.31:0/2291572052 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:01.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb114ac5640 1 --2- 172.21.15.31:0/2291572052 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1100ad000 0x7fb11006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb1152c6640 1 -- 172.21.15.31:0/2291572052 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1100ad000 msgr2=0x7fb11006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:01.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb1152c6640 1 --2- 172.21.15.31:0/2291572052 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1100ad000 0x7fb11006cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb1152c6640 1 -- 172.21.15.31:0/2291572052 >> v1:172.21.15.97:6789/0 conn(0x7fb110070a50 legacy=0x7fb1100b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:01.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb1152c6640 1 -- 172.21.15.31:0/2291572052 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1100b3550 con 0x7fb110147060 2023-11-26T07:18:01.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.377+0000 7fb114ac5640 1 --2- 172.21.15.31:0/2291572052 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1100ad000 0x7fb11006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:01.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb1152c6640 1 --2- 172.21.15.31:0/2291572052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb110147060 0x7fb1100b2460 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009870 tx=0x7fb10c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b11e1a859e147 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb107fff640 1 -- 172.21.15.31:0/2291572052 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0ab600 con 0x7fb110147060 2023-11-26T07:18:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb107fff640 1 -- 172.21.15.31:0/2291572052 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c0a7050 con 0x7fb110147060 2023-11-26T07:18:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb107fff640 1 -- 172.21.15.31:0/2291572052 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0ab980 con 0x7fb110147060 2023-11-26T07:18:01.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 -- 172.21.15.31:0/2291572052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb110147060 msgr2=0x7fb1100b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 --2- 172.21.15.31:0/2291572052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb110147060 0x7fb1100b2460 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009870 tx=0x7fb10c01c500 comp rx=0 tx=0).stop 2023-11-26T07:18:01.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 -- 172.21.15.31:0/2291572052 shutdown_connections 2023-11-26T07:18:01.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 --2- 172.21.15.31:0/2291572052 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1100ad000 0x7fb11006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 --2- 172.21.15.31:0/2291572052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb110147060 0x7fb1100b2460 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 -- 172.21.15.31:0/2291572052 >> 172.21.15.31:0/2291572052 conn(0x7fb1100cf3b0 msgr2=0x7fb1100cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 -- 172.21.15.31:0/2291572052 shutdown_connections 2023-11-26T07:18:01.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 -- 172.21.15.31:0/2291572052 wait complete. 2023-11-26T07:18:01.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 Processor -- start 2023-11-26T07:18:01.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 -- start start 2023-11-26T07:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb110070a50 0x7fb1100b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1100ad000 0x7fb1100af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb110147060 0x7fb11015ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11013fc00 con 0x7fb1100ad000 2023-11-26T07:18:01.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb110160090 con 0x7fb110070a50 2023-11-26T07:18:01.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.381+0000 7fb11752a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11013f770 con 0x7fb110147060 2023-11-26T07:18:01.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb114ac5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1100ad000 0x7fb1100af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb114ac5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1100ad000 0x7fb1100af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40522/0 (socket says 172.21.15.31:40522) 2023-11-26T07:18:01.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb114ac5640 1 -- 172.21.15.31:0/2948355241 learned_addr learned my addr 172.21.15.31:0/2948355241 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:01.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb1152c6640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb110070a50 0x7fb1100b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb115ac7640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb110147060 0x7fb11015ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb114ac5640 1 -- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb110147060 msgr2=0x7fb11015ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb114ac5640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb110147060 0x7fb11015ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb114ac5640 1 -- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb110070a50 msgr2=0x7fb1100b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb114ac5640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb110070a50 0x7fb1100b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb114ac5640 1 -- 172.21.15.31:0/2948355241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0094d0 con 0x7fb1100ad000 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb1152c6640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb110070a50 0x7fb1100b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb114ac5640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1100ad000 0x7fb1100af7c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb10000c7f0 tx=0x7fb10000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb105ffb640 1 -- 172.21.15.31:0/2948355241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10000cea0 con 0x7fb1100ad000 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb105ffb640 1 -- 172.21.15.31:0/2948355241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb100007a30 con 0x7fb1100ad000 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb105ffb640 1 -- 172.21.15.31:0/2948355241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100007de0 con 0x7fb1100ad000 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb11752a640 1 -- 172.21.15.31:0/2948355241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb110160a00 con 0x7fb1100ad000 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb11752a640 1 -- 172.21.15.31:0/2948355241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb110160720 con 0x7fb1100ad000 2023-11-26T07:18:01.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb11752a640 1 -- 172.21.15.31:0/2948355241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11004e850 con 0x7fb1100ad000 2023-11-26T07:18:01.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.385+0000 7fb105ffb640 1 -- 172.21.15.31:0/2948355241 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb1000113f0 con 0x7fb1100ad000 2023-11-26T07:18:01.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.389+0000 7fb105ffb640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0f4066470 0x7fb0f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.389+0000 7fb105ffb640 1 -- 172.21.15.31:0/2948355241 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb100083fb0 con 0x7fb1100ad000 2023-11-26T07:18:01.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.389+0000 7fb1152c6640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0f4066470 0x7fb0f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.389+0000 7fb1152c6640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0f4066470 0x7fb0f4068930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0ad000 tx=0x7fb10c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:01.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.389+0000 7fb105ffb640 1 -- 172.21.15.31:0/2948355241 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb100051440 con 0x7fb1100ad000 2023-11-26T07:18:01.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.549+0000 7fb11752a640 1 -- 172.21.15.31:0/2948355241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb11013f8b0 con 0x7fb1100ad000 2023-11-26T07:18:01.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.549+0000 7fb105ffb640 1 -- 172.21.15.31:0/2948355241 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb100054a60 con 0x7fb1100ad000 2023-11-26T07:18:01.555 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:18:01.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 -- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0f4066470 msgr2=0x7fb0f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0f4066470 0x7fb0f4068930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0ad000 tx=0x7fb10c0a6000 comp rx=0 tx=0).stop 2023-11-26T07:18:01.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 -- 172.21.15.31:0/2948355241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1100ad000 msgr2=0x7fb1100af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1100ad000 0x7fb1100af7c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb10000c7f0 tx=0x7fb10000ccb0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 -- 172.21.15.31:0/2948355241 shutdown_connections 2023-11-26T07:18:01.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0f4066470 0x7fb0f4068930 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb110147060 0x7fb11015ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1100ad000 0x7fb1100af7c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 --2- 172.21.15.31:0/2948355241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb110070a50 0x7fb1100b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 -- 172.21.15.31:0/2948355241 >> 172.21.15.31:0/2948355241 conn(0x7fb1100cf3b0 msgr2=0x7fb11006d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 -- 172.21.15.31:0/2948355241 shutdown_connections 2023-11-26T07:18:01.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.553+0000 7fb11752a640 1 -- 172.21.15.31:0/2948355241 wait complete. 2023-11-26T07:18:01.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:18:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.713+0000 7f700b372640 1 Processor -- start 2023-11-26T07:18:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700b372640 1 -- start start 2023-11-26T07:18:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700b372640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040d0220 0x7f7004168bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700b372640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70040d06b0 con 0x7f700415d530 2023-11-26T07:18:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700b372640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70040d12e0 con 0x7f70040cfdb0 2023-11-26T07:18:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700b372640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70040d15c0 con 0x7f70040d0220 2023-11-26T07:18:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700910e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f700415d530 0x7f70040cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49196/0 (socket says 172.21.15.31:49196) 2023-11-26T07:18:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700910e640 1 -- 172.21.15.31:0/271982734 learned_addr learned my addr 172.21.15.31:0/271982734 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700990f640 1 --2- 172.21.15.31:0/271982734 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040d0220 0x7f7004168bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f6ffbfff640 1 -- 172.21.15.31:0/271982734 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107710648 0 0) 0x7f70040d06b0 con 0x7f700415d530 2023-11-26T07:18:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f6ffbfff640 1 -- 172.21.15.31:0/271982734 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff0003530 con 0x7f700415d530 2023-11-26T07:18:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700990f640 1 -- 172.21.15.31:0/271982734 >> v1:172.21.15.97:6789/0 conn(0x7f70040cfdb0 legacy=0x7f70041654c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f6ffbfff640 1 -- 172.21.15.31:0/271982734 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992904685 0 0) 0x7f70040d12e0 con 0x7f70040cfdb0 2023-11-26T07:18:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700990f640 1 -- 172.21.15.31:0/271982734 >> v1:172.21.15.31:6789/0 conn(0x7f700415d530 legacy=0x7f70040cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.717+0000 7f700990f640 1 -- 172.21.15.31:0/271982734 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70040d18a0 con 0x7f70040d0220 2023-11-26T07:18:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f700990f640 1 --2- 172.21.15.31:0/271982734 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040d0220 0x7f7004168bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009df0 tx=0x7f6ffc01efe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca70af282e509b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f6ffbfff640 1 -- 172.21.15.31:0/271982734 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000001680 con 0x7f70040d0220 2023-11-26T07:18:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f6ffbfff640 1 -- 172.21.15.31:0/271982734 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc0a8bb0 con 0x7f70040d0220 2023-11-26T07:18:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f6ffbfff640 1 -- 172.21.15.31:0/271982734 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc0b0440 con 0x7f70040d0220 2023-11-26T07:18:01.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f700b372640 1 -- 172.21.15.31:0/271982734 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040d0220 msgr2=0x7f7004168bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f700b372640 1 --2- 172.21.15.31:0/271982734 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040d0220 0x7f7004168bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009df0 tx=0x7f6ffc01efe0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f700b372640 1 -- 172.21.15.31:0/271982734 shutdown_connections 2023-11-26T07:18:01.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f700b372640 1 --2- 172.21.15.31:0/271982734 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040d0220 0x7f7004168bf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f700b372640 1 -- 172.21.15.31:0/271982734 >> 172.21.15.31:0/271982734 conn(0x7f700409c6d0 msgr2=0x7f70040d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:01.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f700b372640 1 -- 172.21.15.31:0/271982734 shutdown_connections 2023-11-26T07:18:01.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.721+0000 7f700b372640 1 -- 172.21.15.31:0/271982734 wait complete. 2023-11-26T07:18:01.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 Processor -- start 2023-11-26T07:18:01.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 -- start start 2023-11-26T07:18:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040cfdb0 0x7f700406d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70040d0220 0x7f70041e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f700415d530 0x7f70040b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700406d550 con 0x7f70040d0220 2023-11-26T07:18:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7004147ba0 con 0x7f700415d530 2023-11-26T07:18:01.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7004147e80 con 0x7f70040cfdb0 2023-11-26T07:18:01.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700890d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70040d0220 0x7f70041e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700890d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70040d0220 0x7f70041e76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40534/0 (socket says 172.21.15.31:40534) 2023-11-26T07:18:01.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700890d640 1 -- 172.21.15.31:0/1765068547 learned_addr learned my addr 172.21.15.31:0/1765068547 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:01.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700910e640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040cfdb0 0x7f700406d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700990f640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f700415d530 0x7f70040b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700890d640 1 -- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040cfdb0 msgr2=0x7f700406d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700890d640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040cfdb0 0x7f700406d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700890d640 1 -- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f700415d530 msgr2=0x7f70040b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700890d640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f700415d530 0x7f70040b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700890d640 1 -- 172.21.15.31:0/1765068547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc009a70 con 0x7f70040d0220 2023-11-26T07:18:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700890d640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70040d0220 0x7f70041e76c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400c4d0 tx=0x7f6ff400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f6ff9ffb640 1 -- 172.21.15.31:0/1765068547 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4005480 con 0x7f70040d0220 2023-11-26T07:18:01.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f6ff9ffb640 1 -- 172.21.15.31:0/1765068547 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff4003600 con 0x7f70040d0220 2023-11-26T07:18:01.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f6ff9ffb640 1 -- 172.21.15.31:0/1765068547 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff40039b0 con 0x7f70040d0220 2023-11-26T07:18:01.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 -- 172.21.15.31:0/1765068547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7004148160 con 0x7f70040d0220 2023-11-26T07:18:01.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 -- 172.21.15.31:0/1765068547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7004148720 con 0x7f70040d0220 2023-11-26T07:18:01.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.725+0000 7f700b372640 1 -- 172.21.15.31:0/1765068547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7004067a40 con 0x7f70040d0220 2023-11-26T07:18:01.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.729+0000 7f6ff9ffb640 1 -- 172.21.15.31:0/1765068547 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6ff4003b10 con 0x7f70040d0220 2023-11-26T07:18:01.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.729+0000 7f6ff9ffb640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fe8066470 0x7f6fe8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:01.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.729+0000 7f6ff9ffb640 1 -- 172.21.15.31:0/1765068547 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ff40834a0 con 0x7f70040d0220 2023-11-26T07:18:01.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.729+0000 7f700910e640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fe8066470 0x7f6fe8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:01.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.729+0000 7f700910e640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fe8066470 0x7f6fe8068930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7000001c40 tx=0x7f7000001bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:01.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.733+0000 7f6ff9ffb640 1 -- 172.21.15.31:0/1765068547 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ff40509f0 con 0x7f70040d0220 2023-11-26T07:18:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:01.945+0000 7f700b372640 1 -- 172.21.15.31:0/1765068547 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f700404e850 con 0x7f70040d0220 2023-11-26T07:18:02.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.240+0000 7f6ff9ffb640 1 -- 172.21.15.31:0/1765068547 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6ff40170e0 con 0x7f70040d0220 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 -- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fe8066470 msgr2=0x7f6fe8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fe8066470 0x7f6fe8068930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7000001c40 tx=0x7f7000001bd0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 -- 172.21.15.31:0/1765068547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70040d0220 msgr2=0x7f70041e76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70040d0220 0x7f70041e76c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400c4d0 tx=0x7f6ff400c990 comp rx=0 tx=0).stop 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 -- 172.21.15.31:0/1765068547 shutdown_connections 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6fe8066470 0x7f6fe8068930 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f700415d530 0x7f70040b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70040d0220 0x7f70041e76c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 --2- 172.21.15.31:0/1765068547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70040cfdb0 0x7f700406d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 -- 172.21.15.31:0/1765068547 >> 172.21.15.31:0/1765068547 conn(0x7f700409c6d0 msgr2=0x7f70040b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 -- 172.21.15.31:0/1765068547 shutdown_connections 2023-11-26T07:18:02.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.244+0000 7f700b372640 1 -- 172.21.15.31:0/1765068547 wait complete. 2023-11-26T07:18:02.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:18:02.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.408+0000 7fdd04ac8640 1 Processor -- start 2023-11-26T07:18:02.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.408+0000 7fdd04ac8640 1 -- start start 2023-11-26T07:18:02.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdd04ac8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd0015d2a0 0x7fdd000d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdd04ac8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd000d2c20 0x7fdd000b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdd04ac8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd000b5180 0x7fdd0006b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdd04ac8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd000ce8c0 con 0x7fdd000b5180 2023-11-26T07:18:02.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdd04ac8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd000b6250 con 0x7fdd0015d2a0 2023-11-26T07:18:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdd04ac8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00161cf0 con 0x7fdd000d2c20 2023-11-26T07:18:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcfffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd000b5180 0x7fdd0006b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcfffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd000b5180 0x7fdd0006b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40536/0 (socket says 172.21.15.31:40536) 2023-11-26T07:18:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcfffff640 1 -- 172.21.15.31:0/1615357417 learned_addr learned my addr 172.21.15.31:0/1615357417 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcff7fe640 1 --2- 172.21.15.31:0/1615357417 >> v2:172.21.15.97:3300/0 conn(0x7fdd0015d2a0 0x7fdd000d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcfeffd640 1 --2- 172.21.15.31:0/1615357417 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd000d2c20 0x7fdd000b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcfffff640 1 -- 172.21.15.31:0/1615357417 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd000d2c20 msgr2=0x7fdd000b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcfffff640 1 --2- 172.21.15.31:0/1615357417 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd000d2c20 0x7fdd000b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcfffff640 1 -- 172.21.15.31:0/1615357417 >> v2:172.21.15.97:3300/0 conn(0x7fdd0015d2a0 msgr2=0x7fdd000d26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:02.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcfffff640 1 --2- 172.21.15.31:0/1615357417 >> v2:172.21.15.97:3300/0 conn(0x7fdd0015d2a0 0x7fdd000d26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcfffff640 1 -- 172.21.15.31:0/1615357417 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd000b5590 con 0x7fdd000b5180 2023-11-26T07:18:02.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.412+0000 7fdcff7fe640 1 --2- 172.21.15.31:0/1615357417 >> v2:172.21.15.97:3300/0 conn(0x7fdd0015d2a0 0x7fdd000d26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:02.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdcfffff640 1 --2- 172.21.15.31:0/1615357417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 0x7fdd0006b650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdd000cdb60 tx=0x7fdcf801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32244ca591b78f12 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdcfe7fc640 1 -- 172.21.15.31:0/1615357417 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80a5560 con 0x7fdd000b5180 2023-11-26T07:18:02.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdcfe7fc640 1 -- 172.21.15.31:0/1615357417 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf80a56c0 con 0x7fdd000b5180 2023-11-26T07:18:02.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdcfe7fc640 1 -- 172.21.15.31:0/1615357417 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80a5a50 con 0x7fdd000b5180 2023-11-26T07:18:02.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 -- 172.21.15.31:0/1615357417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 msgr2=0x7fdd0006b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 --2- 172.21.15.31:0/1615357417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 0x7fdd0006b650 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdd000cdb60 tx=0x7fdcf801c6e0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 -- 172.21.15.31:0/1615357417 shutdown_connections 2023-11-26T07:18:02.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 --2- 172.21.15.31:0/1615357417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 0x7fdd0006b650 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 --2- 172.21.15.31:0/1615357417 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd000d2c20 0x7fdd000b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 --2- 172.21.15.31:0/1615357417 >> v2:172.21.15.97:3300/0 conn(0x7fdd0015d2a0 0x7fdd000d26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 -- 172.21.15.31:0/1615357417 >> 172.21.15.31:0/1615357417 conn(0x7fdd000668a0 msgr2=0x7fdd000cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 -- 172.21.15.31:0/1615357417 shutdown_connections 2023-11-26T07:18:02.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 -- 172.21.15.31:0/1615357417 wait complete. 2023-11-26T07:18:02.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 Processor -- start 2023-11-26T07:18:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.416+0000 7fdd04ac8640 1 -- start start 2023-11-26T07:18:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdd04ac8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 0x7fdd000b8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdd04ac8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd000d2c20 0x7fdd00158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdd04ac8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd0015d2a0 0x7fdd000cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdd04ac8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0015be10 con 0x7fdd000b5180 2023-11-26T07:18:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdd04ac8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd000b8b60 con 0x7fdd0015d2a0 2023-11-26T07:18:02.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdd04ac8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd000cb1b0 con 0x7fdd000d2c20 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcff7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 0x7fdd000b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcff7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 0x7fdd000b8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40544/0 (socket says 172.21.15.31:40544) 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcff7fe640 1 -- 172.21.15.31:0/4244218870 learned_addr learned my addr 172.21.15.31:0/4244218870 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcfffff640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd0015d2a0 0x7fdd000cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcfeffd640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd000d2c20 0x7fdd00158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcff7fe640 1 -- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd000d2c20 msgr2=0x7fdd00158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcff7fe640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd000d2c20 0x7fdd00158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcff7fe640 1 -- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd0015d2a0 msgr2=0x7fdd000cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcff7fe640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd0015d2a0 0x7fdd000cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcff7fe640 1 -- 172.21.15.31:0/4244218870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf8009b30 con 0x7fdd000b5180 2023-11-26T07:18:02.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdcff7fe640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 0x7fdd000b8620 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000d800 tx=0x7fdcf000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdce7fff640 1 -- 172.21.15.31:0/4244218870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000b6c0 con 0x7fdd000b5180 2023-11-26T07:18:02.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdce7fff640 1 -- 172.21.15.31:0/4244218870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf000b820 con 0x7fdd000b5180 2023-11-26T07:18:02.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdce7fff640 1 -- 172.21.15.31:0/4244218870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000bbd0 con 0x7fdd000b5180 2023-11-26T07:18:02.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdd04ac8640 1 -- 172.21.15.31:0/4244218870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd000caed0 con 0x7fdd000b5180 2023-11-26T07:18:02.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdd04ac8640 1 -- 172.21.15.31:0/4244218870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0015ba30 con 0x7fdd000b5180 2023-11-26T07:18:02.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdd04ac8640 1 -- 172.21.15.31:0/4244218870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd00074990 con 0x7fdd000b5180 2023-11-26T07:18:02.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.420+0000 7fdce7fff640 1 -- 172.21.15.31:0/4244218870 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fdcf000bd30 con 0x7fdd000b5180 2023-11-26T07:18:02.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.424+0000 7fdce7fff640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcec066470 0x7fdcec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:02.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.424+0000 7fdce7fff640 1 -- 172.21.15.31:0/4244218870 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcf0084250 con 0x7fdd000b5180 2023-11-26T07:18:02.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.424+0000 7fdcfeffd640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcec066470 0x7fdcec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:02.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.424+0000 7fdcfeffd640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcec066470 0x7fdcec068930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdd000b4190 tx=0x7fdcf4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:02.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.428+0000 7fdce7fff640 1 -- 172.21.15.31:0/4244218870 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdcf0051760 con 0x7fdd000b5180 2023-11-26T07:18:02.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.584+0000 7fdd04ac8640 1 -- 172.21.15.31:0/4244218870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdd000c9770 con 0x7fdd000b5180 2023-11-26T07:18:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.588+0000 7fdce7fff640 1 -- 172.21.15.31:0/4244218870 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdcf0054d80 con 0x7fdd000b5180 2023-11-26T07:18:02.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 -- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcec066470 msgr2=0x7fdcec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcec066470 0x7fdcec068930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdd000b4190 tx=0x7fdcf4008040 comp rx=0 tx=0).stop 2023-11-26T07:18:02.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 -- 172.21.15.31:0/4244218870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 msgr2=0x7fdd000b8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:02.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 0x7fdd000b8620 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000d800 tx=0x7fdcf000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 -- 172.21.15.31:0/4244218870 shutdown_connections 2023-11-26T07:18:02.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcec066470 0x7fdcec068930 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd0015d2a0 0x7fdd000cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd000d2c20 0x7fdd00158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 --2- 172.21.15.31:0/4244218870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd000b5180 0x7fdd000b8620 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:02.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 -- 172.21.15.31:0/4244218870 >> 172.21.15.31:0/4244218870 conn(0x7fdd000668a0 msgr2=0x7fdd000b7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:02.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 -- 172.21.15.31:0/4244218870 shutdown_connections 2023-11-26T07:18:02.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:02.592+0000 7fdd04ac8640 1 -- 172.21.15.31:0/4244218870 wait complete. 2023-11-26T07:18:02.615 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:18:02.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:18:02.682 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:18:02.726 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:18:02.726 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:02.726 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:02.735 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:02.736 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:02.787 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:02.788 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:02.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.020+0000 7f6f13079640 1 Processor -- start 2023-11-26T07:18:03.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f13079640 1 -- start start 2023-11-26T07:18:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f13079640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f0c144c80 0x7f6f0c0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f13079640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f0c070df0 0x7f6f0c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f13079640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0c145060 con 0x7f6f0c0ac460 2023-11-26T07:18:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f13079640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0c071fd0 con 0x7f6f0c144c80 2023-11-26T07:18:03.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f13079640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0ac8c0 con 0x7f6f0c070df0 2023-11-26T07:18:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f10e15640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f0c144c80 0x7f6f0c0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f10e15640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6f0c144c80 0x7f6f0c0abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40546/0 (socket says 172.21.15.31:40546) 2023-11-26T07:18:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f10e15640 1 -- 172.21.15.31:0/4196035900 learned_addr learned my addr 172.21.15.31:0/4196035900 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f11616640 1 --2- 172.21.15.31:0/4196035900 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f0c070df0 0x7f6f0c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f10e15640 1 -- 172.21.15.31:0/4196035900 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f0c070df0 msgr2=0x7f6f0c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f10e15640 1 --2- 172.21.15.31:0/4196035900 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f0c070df0 0x7f6f0c06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f10e15640 1 -- 172.21.15.31:0/4196035900 >> v1:172.21.15.97:6789/0 conn(0x7f6f0c0ac460 legacy=0x7f6f0c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.024+0000 7f6f10e15640 1 -- 172.21.15.31:0/4196035900 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c071450 con 0x7f6f0c144c80 2023-11-26T07:18:03.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f10e15640 1 --2- 172.21.15.31:0/4196035900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c144c80 0x7f6f0c0abf20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009870 tx=0x7f6f0001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f5122f292962c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f0b7fe640 1 -- 172.21.15.31:0/4196035900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f000a7070 con 0x7f6f0c144c80 2023-11-26T07:18:03.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f0b7fe640 1 -- 172.21.15.31:0/4196035900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f000ab430 con 0x7f6f0c144c80 2023-11-26T07:18:03.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f0b7fe640 1 -- 172.21.15.31:0/4196035900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f000ab7c0 con 0x7f6f0c144c80 2023-11-26T07:18:03.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f13079640 1 -- 172.21.15.31:0/4196035900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c144c80 msgr2=0x7f6f0c0abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f13079640 1 --2- 172.21.15.31:0/4196035900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c144c80 0x7f6f0c0abf20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009870 tx=0x7f6f0001c540 comp rx=0 tx=0).stop 2023-11-26T07:18:03.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f13079640 1 -- 172.21.15.31:0/4196035900 shutdown_connections 2023-11-26T07:18:03.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f13079640 1 --2- 172.21.15.31:0/4196035900 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f0c070df0 0x7f6f0c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f13079640 1 --2- 172.21.15.31:0/4196035900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c144c80 0x7f6f0c0abf20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f13079640 1 -- 172.21.15.31:0/4196035900 >> 172.21.15.31:0/4196035900 conn(0x7f6f0c0b0c40 msgr2=0x7f6f0c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f13079640 1 -- 172.21.15.31:0/4196035900 shutdown_connections 2023-11-26T07:18:03.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f13079640 1 -- 172.21.15.31:0/4196035900 wait complete. 2023-11-26T07:18:03.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f13079640 1 Processor -- start 2023-11-26T07:18:03.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.028+0000 7f6f13079640 1 -- start start 2023-11-26T07:18:03.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f13079640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c070df0 0x7f6f0c1f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f13079640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f0c0ac460 0x7f6f0c13de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f13079640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f0c144c80 0x7f6f0c0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f13079640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c1575c0 con 0x7f6f0c070df0 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f13079640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c099af0 con 0x7f6f0c0ac460 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f13079640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c1f9750 con 0x7f6f0c144c80 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f10e15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c070df0 0x7f6f0c1f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f10e15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c070df0 0x7f6f0c1f9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40562/0 (socket says 172.21.15.31:40562) 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f10e15640 1 -- 172.21.15.31:0/3447512282 learned_addr learned my addr 172.21.15.31:0/3447512282 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f10e15640 1 -- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f0c144c80 msgr2=0x7f6f0c0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f10e15640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f0c144c80 0x7f6f0c0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f10e15640 1 -- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f0c0ac460 msgr2=0x7f6f0c13de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f10e15640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f0c0ac460 0x7f6f0c13de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f10e15640 1 -- 172.21.15.31:0/3447512282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f000094d0 con 0x7f6f0c070df0 2023-11-26T07:18:03.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f10e15640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c070df0 0x7f6f0c1f9170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f000ad040 tx=0x7f6f000192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f097fa640 1 -- 172.21.15.31:0/3447512282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f000bc070 con 0x7f6f0c070df0 2023-11-26T07:18:03.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f097fa640 1 -- 172.21.15.31:0/3447512282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f000b4420 con 0x7f6f0c070df0 2023-11-26T07:18:03.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f097fa640 1 -- 172.21.15.31:0/3447512282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f000b47d0 con 0x7f6f0c070df0 2023-11-26T07:18:03.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f13079640 1 -- 172.21.15.31:0/3447512282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c1f9a30 con 0x7f6f0c070df0 2023-11-26T07:18:03.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f13079640 1 -- 172.21.15.31:0/3447512282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c09c8b0 con 0x7f6f0c070df0 2023-11-26T07:18:03.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.032+0000 7f6f13079640 1 -- 172.21.15.31:0/3447512282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c0b1ba0 con 0x7f6f0c070df0 2023-11-26T07:18:03.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.036+0000 7f6f097fa640 1 -- 172.21.15.31:0/3447512282 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6f000b4930 con 0x7f6f0c070df0 2023-11-26T07:18:03.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.036+0000 7f6f097fa640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ef0066470 0x7f6ef0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.036+0000 7f6f097fa640 1 -- 172.21.15.31:0/3447512282 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f0011d850 con 0x7f6f0c070df0 2023-11-26T07:18:03.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.036+0000 7f6f0bfff640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ef0066470 0x7f6ef0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.036+0000 7f6f0bfff640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ef0066470 0x7f6ef0068930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8000f70 tx=0x7f6ef80079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.040+0000 7f6f097fa640 1 -- 172.21.15.31:0/3447512282 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6f000eacc0 con 0x7f6f0c070df0 2023-11-26T07:18:03.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.196+0000 7f6f13079640 1 -- 172.21.15.31:0/3447512282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f0c04e850 con 0x7f6f0c070df0 2023-11-26T07:18:03.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.196+0000 7f6f097fa640 1 -- 172.21.15.31:0/3447512282 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f6f000ee2e0 con 0x7f6f0c070df0 2023-11-26T07:18:03.203 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:03.203 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:03.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 -- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ef0066470 msgr2=0x7f6ef0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ef0066470 0x7f6ef0068930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8000f70 tx=0x7f6ef80079c0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 -- 172.21.15.31:0/3447512282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c070df0 msgr2=0x7f6f0c1f9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c070df0 0x7f6f0c1f9170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f000ad040 tx=0x7f6f000192e0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 -- 172.21.15.31:0/3447512282 shutdown_connections 2023-11-26T07:18:03.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ef0066470 0x7f6ef0068930 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f0c144c80 0x7f6f0c0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f0c0ac460 0x7f6f0c13de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 --2- 172.21.15.31:0/3447512282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f0c070df0 0x7f6f0c1f9170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 -- 172.21.15.31:0/3447512282 >> 172.21.15.31:0/3447512282 conn(0x7f6f0c0b0c40 msgr2=0x7f6f0c164950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 -- 172.21.15.31:0/3447512282 shutdown_connections 2023-11-26T07:18:03.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.200+0000 7f6f13079640 1 -- 172.21.15.31:0/3447512282 wait complete. 2023-11-26T07:18:03.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:03.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.364+0000 7fbadc4aa640 1 Processor -- start 2023-11-26T07:18:03.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.364+0000 7fbadc4aa640 1 -- start start 2023-11-26T07:18:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbadc4aa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbad406e3f0 0x7fbad415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbadc4aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbad40ce5d0 0x7fbad415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbadc4aa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad4159630 con 0x7fbad40cb9b0 2023-11-26T07:18:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbadc4aa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad406e860 con 0x7fbad406e3f0 2023-11-26T07:18:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbadc4aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad415bba0 con 0x7fbad40ce5d0 2023-11-26T07:18:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbada246640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbad40cb9b0 0x7fbad40cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49202/0 (socket says 172.21.15.31:49202) 2023-11-26T07:18:03.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbada246640 1 -- 172.21.15.31:0/4194091914 learned_addr learned my addr 172.21.15.31:0/4194091914 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698901891 0 0) 0x7fbad4159630 con 0x7fbad40cb9b0 2023-11-26T07:18:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9a45640 1 --2- 172.21.15.31:0/4194091914 >> v2:172.21.15.97:3300/0 conn(0x7fbad406e3f0 0x7fbad415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac0003530 con 0x7fbad40cb9b0 2023-11-26T07:18:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbadaa47640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbad40ce5d0 0x7fbad415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2534378587 0 0) 0x7fbac0003530 con 0x7fbad40cb9b0 2023-11-26T07:18:03.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbad40ce5d0 msgr2=0x7fbad415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbad40ce5d0 0x7fbad415bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 >> v2:172.21.15.97:3300/0 conn(0x7fbad406e3f0 msgr2=0x7fbad415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 --2- 172.21.15.31:0/4194091914 >> v2:172.21.15.97:3300/0 conn(0x7fbad406e3f0 0x7fbad415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad415f6d0 con 0x7fbad40cb9b0 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fbad0001680 con 0x7fbad40cb9b0 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbadaa47640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbad40ce5d0 0x7fbad415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 >> v1:172.21.15.31:6789/0 conn(0x7fbad40cb9b0 legacy=0x7fbad40cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbad40ce5d0 0x7fbac0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0003f10 0x7fbac00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbac0004d10 0x7fbac0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad406e860 con 0x7fbac0003f10 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac000b990 con 0x7fbac0004d10 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac000bb50 con 0x7fbad40ce5d0 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9a45640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0003f10 0x7fbac00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbada246640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbad40ce5d0 0x7fbac0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbadaa47640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbac0004d10 0x7fbac0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9a45640 1 -- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbad40ce5d0 msgr2=0x7fbac0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9a45640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbad40ce5d0 0x7fbac0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9a45640 1 -- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbac0004d10 msgr2=0x7fbac0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9a45640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbac0004d10 0x7fbac0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbad9a45640 1 -- 172.21.15.31:0/4194091914 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac0003530 con 0x7fbac0003f10 2023-11-26T07:18:03.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.368+0000 7fbadaa47640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbac0004d10 0x7fbac0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:03.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbad9a45640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0003f10 0x7fbac00047d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009ae0 tx=0x7fbac401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5769561dab859eef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac40a5680 con 0x7fbac0003f10 2023-11-26T07:18:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac40a57e0 con 0x7fbac0003f10 2023-11-26T07:18:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbad9244640 1 -- 172.21.15.31:0/4194091914 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac40a5b90 con 0x7fbac0003f10 2023-11-26T07:18:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbadc4aa640 1 -- 172.21.15.31:0/4194091914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0003f10 msgr2=0x7fbac00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbadc4aa640 1 --2- 172.21.15.31:0/4194091914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac0003f10 0x7fbac00047d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009ae0 tx=0x7fbac401ca90 comp rx=0 tx=0).stop 2023-11-26T07:18:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbadaa47640 1 -- 172.21.15.31:0/4194091914 reap_dead start 2023-11-26T07:18:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbadc4aa640 1 -- 172.21.15.31:0/4194091914 shutdown_connections 2023-11-26T07:18:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbadc4aa640 1 -- 172.21.15.31:0/4194091914 >> 172.21.15.31:0/4194091914 conn(0x7fbad40b0cb0 msgr2=0x7fbad40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbadc4aa640 1 -- 172.21.15.31:0/4194091914 shutdown_connections 2023-11-26T07:18:03.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.372+0000 7fbadc4aa640 1 -- 172.21.15.31:0/4194091914 wait complete. 2023-11-26T07:18:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadc4aa640 1 Processor -- start 2023-11-26T07:18:03.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadc4aa640 1 -- start start 2023-11-26T07:18:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadc4aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbac0003f10 0x7fbad40ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadc4aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad40b3120 0x7fbad40c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadc4aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad40c9a70 0x7fbad4158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadc4aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad40b4160 con 0x7fbad40c9a70 2023-11-26T07:18:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadc4aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad41e51d0 con 0x7fbad40b3120 2023-11-26T07:18:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadc4aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad40b35e0 con 0x7fbac0003f10 2023-11-26T07:18:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadaa47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad40c9a70 0x7fbad4158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadaa47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad40c9a70 0x7fbad4158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40572/0 (socket says 172.21.15.31:40572) 2023-11-26T07:18:03.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadaa47640 1 -- 172.21.15.31:0/1416209118 learned_addr learned my addr 172.21.15.31:0/1416209118 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbada246640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbac0003f10 0x7fbad40ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbad9a45640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad40b3120 0x7fbad40c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadaa47640 1 -- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbac0003f10 msgr2=0x7fbad40ad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadaa47640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbac0003f10 0x7fbad40ad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadaa47640 1 -- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad40b3120 msgr2=0x7fbad40c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadaa47640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad40b3120 0x7fbad40c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadaa47640 1 -- 172.21.15.31:0/1416209118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad40b38c0 con 0x7fbad40c9a70 2023-11-26T07:18:03.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbada246640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbac0003f10 0x7fbad40ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadaa47640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad40c9a70 0x7fbad4158720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbad4071440 tx=0x7fbacc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbacaffd640 1 -- 172.21.15.31:0/1416209118 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc007c70 con 0x7fbad40c9a70 2023-11-26T07:18:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbacaffd640 1 -- 172.21.15.31:0/1416209118 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc00f050 con 0x7fbad40c9a70 2023-11-26T07:18:03.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadc4aa640 1 -- 172.21.15.31:0/1416209118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad40b3ba0 con 0x7fbad40c9a70 2023-11-26T07:18:03.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbacaffd640 1 -- 172.21.15.31:0/1416209118 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc014410 con 0x7fbad40c9a70 2023-11-26T07:18:03.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.376+0000 7fbadc4aa640 1 -- 172.21.15.31:0/1416209118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad40ad8b0 con 0x7fbad40c9a70 2023-11-26T07:18:03.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.380+0000 7fbadc4aa640 1 -- 172.21.15.31:0/1416209118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad40671f0 con 0x7fbad40c9a70 2023-11-26T07:18:03.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.380+0000 7fbacaffd640 1 -- 172.21.15.31:0/1416209118 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fbacc014590 con 0x7fbad40c9a70 2023-11-26T07:18:03.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.380+0000 7fbacaffd640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbab8066470 0x7fbab8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.380+0000 7fbacaffd640 1 -- 172.21.15.31:0/1416209118 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbacc0842b0 con 0x7fbad40c9a70 2023-11-26T07:18:03.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.380+0000 7fbada246640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbab8066470 0x7fbab8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.380+0000 7fbada246640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbab8066470 0x7fbab8068930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbad00078f0 tx=0x7fbad0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.384+0000 7fbacaffd640 1 -- 172.21.15.31:0/1416209118 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbacc051690 con 0x7fbad40c9a70 2023-11-26T07:18:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 -- 172.21.15.31:0/1416209118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbad406b460 con 0x7fbad40c9a70 2023-11-26T07:18:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbacaffd640 1 -- 172.21.15.31:0/1416209118 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbacc054cb0 con 0x7fbad40c9a70 2023-11-26T07:18:03.548 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:03.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 -- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbab8066470 msgr2=0x7fbab8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbab8066470 0x7fbab8068930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbad00078f0 tx=0x7fbad0009040 comp rx=0 tx=0).stop 2023-11-26T07:18:03.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 -- 172.21.15.31:0/1416209118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad40c9a70 msgr2=0x7fbad4158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad40c9a70 0x7fbad4158720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbad4071440 tx=0x7fbacc004770 comp rx=0 tx=0).stop 2023-11-26T07:18:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 -- 172.21.15.31:0/1416209118 shutdown_connections 2023-11-26T07:18:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbab8066470 0x7fbab8068930 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad40c9a70 0x7fbad4158720 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad40b3120 0x7fbad40c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 --2- 172.21.15.31:0/1416209118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbac0003f10 0x7fbad40ad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 -- 172.21.15.31:0/1416209118 >> 172.21.15.31:0/1416209118 conn(0x7fbad40b0cb0 msgr2=0x7fbad40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 -- 172.21.15.31:0/1416209118 shutdown_connections 2023-11-26T07:18:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.544+0000 7fbadc4aa640 1 -- 172.21.15.31:0/1416209118 wait complete. 2023-11-26T07:18:03.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.708+0000 7f4fc5479640 1 Processor -- start 2023-11-26T07:18:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.708+0000 7f4fc5479640 1 -- start start 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fc5479640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc00b4210 0x7f4fc00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fc5479640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fc015db40 0x7f4fc015df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fc5479640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc0147190 con 0x7f4fc0146820 2023-11-26T07:18:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fc5479640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc0146c00 con 0x7f4fc015db40 2023-11-26T07:18:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fc5479640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc006bf90 con 0x7f4fc00b4210 2023-11-26T07:18:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fbf7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fc015db40 0x7f4fc015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fbf7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4fc015db40 0x7f4fc015df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45274/0 (socket says 172.21.15.31:45274) 2023-11-26T07:18:03.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fbf7fe640 1 -- 172.21.15.31:0/1323700989 learned_addr learned my addr 172.21.15.31:0/1323700989 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:03.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fb7fff640 1 --2- 172.21.15.31:0/1323700989 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc00b4210 0x7f4fc00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fbf7fe640 1 -- 172.21.15.31:0/1323700989 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc00b4210 msgr2=0x7f4fc00b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:03.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fbf7fe640 1 --2- 172.21.15.31:0/1323700989 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc00b4210 0x7f4fc00b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fbf7fe640 1 -- 172.21.15.31:0/1323700989 >> v1:172.21.15.97:6789/0 conn(0x7f4fc0146820 legacy=0x7f4fc0070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:03.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.712+0000 7f4fbf7fe640 1 -- 172.21.15.31:0/1323700989 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00b4bb0 con 0x7f4fc015db40 2023-11-26T07:18:03.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fbf7fe640 1 --2- 172.21.15.31:0/1323700989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc015db40 0x7f4fc015df90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009e50 tx=0x7f4fb801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900da1841d999b76 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fbe7fc640 1 -- 172.21.15.31:0/1323700989 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80a5820 con 0x7f4fc015db40 2023-11-26T07:18:03.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fbe7fc640 1 -- 172.21.15.31:0/1323700989 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb80a5980 con 0x7f4fc015db40 2023-11-26T07:18:03.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fbe7fc640 1 -- 172.21.15.31:0/1323700989 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80a5d10 con 0x7f4fc015db40 2023-11-26T07:18:03.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fc5479640 1 -- 172.21.15.31:0/1323700989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc015db40 msgr2=0x7f4fc015df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fc5479640 1 --2- 172.21.15.31:0/1323700989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc015db40 0x7f4fc015df90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4fb8009e50 tx=0x7f4fb801c9a0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fc5479640 1 -- 172.21.15.31:0/1323700989 shutdown_connections 2023-11-26T07:18:03.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fc5479640 1 --2- 172.21.15.31:0/1323700989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc015db40 0x7f4fc015df90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fc5479640 1 --2- 172.21.15.31:0/1323700989 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc00b4210 0x7f4fc00b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fc5479640 1 -- 172.21.15.31:0/1323700989 >> 172.21.15.31:0/1323700989 conn(0x7f4fc00b2f10 msgr2=0x7f4fc00b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fc5479640 1 -- 172.21.15.31:0/1323700989 shutdown_connections 2023-11-26T07:18:03.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fc5479640 1 -- 172.21.15.31:0/1323700989 wait complete. 2023-11-26T07:18:03.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fc5479640 1 Processor -- start 2023-11-26T07:18:03.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.716+0000 7f4fc5479640 1 -- start start 2023-11-26T07:18:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fc5479640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc00b4210 0x7f4fc013ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fc5479640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc0146820 0x7f4fc015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fc5479640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc015db40 0x7f4fc00b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fc5479640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015cbf0 con 0x7f4fc00b4210 2023-11-26T07:18:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fc5479640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc013f1c0 con 0x7f4fc0146820 2023-11-26T07:18:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fc5479640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015bdd0 con 0x7f4fc015db40 2023-11-26T07:18:03.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc00b4210 0x7f4fc013ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc00b4210 0x7f4fc013ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45284/0 (socket says 172.21.15.31:45284) 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbeffd640 1 -- 172.21.15.31:0/2086431960 learned_addr learned my addr 172.21.15.31:0/2086431960 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fb7fff640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc0146820 0x7f4fc015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbf7fe640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc015db40 0x7f4fc00b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbeffd640 1 -- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc015db40 msgr2=0x7f4fc00b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbeffd640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc015db40 0x7f4fc00b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbeffd640 1 -- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc0146820 msgr2=0x7f4fc015b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbeffd640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc0146820 0x7f4fc015b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbeffd640 1 -- 172.21.15.31:0/2086431960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8009b30 con 0x7f4fc00b4210 2023-11-26T07:18:03.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbf7fe640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc015db40 0x7f4fc00b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:03.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fbeffd640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc00b4210 0x7f4fc013ec80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c450 tx=0x7f4fb000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fb77fe640 1 -- 172.21.15.31:0/2086431960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0012830 con 0x7f4fc00b4210 2023-11-26T07:18:03.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fb77fe640 1 -- 172.21.15.31:0/2086431960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb00129b0 con 0x7f4fc00b4210 2023-11-26T07:18:03.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fb77fe640 1 -- 172.21.15.31:0/2086431960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0012d60 con 0x7f4fc00b4210 2023-11-26T07:18:03.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fc5479640 1 -- 172.21.15.31:0/2086431960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc006d030 con 0x7f4fc00b4210 2023-11-26T07:18:03.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fc5479640 1 -- 172.21.15.31:0/2086431960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc015c350 con 0x7f4fc00b4210 2023-11-26T07:18:03.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fc5479640 1 -- 172.21.15.31:0/2086431960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc004e850 con 0x7f4fc00b4210 2023-11-26T07:18:03.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.720+0000 7f4fb77fe640 1 -- 172.21.15.31:0/2086431960 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4fb001c460 con 0x7f4fc00b4210 2023-11-26T07:18:03.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.724+0000 7f4fb77fe640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4fa0066470 0x7f4fa0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:03.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.724+0000 7f4fb77fe640 1 -- 172.21.15.31:0/2086431960 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fb0083c80 con 0x7f4fc00b4210 2023-11-26T07:18:03.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.724+0000 7f4fb7fff640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4fa0066470 0x7f4fa0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:03.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.724+0000 7f4fb7fff640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4fa0066470 0x7f4fa0068930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4fc006f1e0 tx=0x7f4fa8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:03.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.724+0000 7f4fb77fe640 1 -- 172.21.15.31:0/2086431960 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4fb00510f0 con 0x7f4fc00b4210 2023-11-26T07:18:03.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.880+0000 7f4fc5479640 1 -- 172.21.15.31:0/2086431960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4fc00d1f00 con 0x7f4fc00b4210 2023-11-26T07:18:03.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.880+0000 7f4fb77fe640 1 -- 172.21.15.31:0/2086431960 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4fb0054710 con 0x7f4fc00b4210 2023-11-26T07:18:03.887 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:18:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 -- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4fa0066470 msgr2=0x7f4fa0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4fa0066470 0x7f4fa0068930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4fc006f1e0 tx=0x7f4fa8008040 comp rx=0 tx=0).stop 2023-11-26T07:18:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 -- 172.21.15.31:0/2086431960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc00b4210 msgr2=0x7f4fc013ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc00b4210 0x7f4fc013ec80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c450 tx=0x7f4fb000c910 comp rx=0 tx=0).stop 2023-11-26T07:18:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 -- 172.21.15.31:0/2086431960 shutdown_connections 2023-11-26T07:18:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4fa0066470 0x7f4fa0068930 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4fc015db40 0x7f4fc00b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fc0146820 0x7f4fc015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 --2- 172.21.15.31:0/2086431960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fc00b4210 0x7f4fc013ec80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 -- 172.21.15.31:0/2086431960 >> 172.21.15.31:0/2086431960 conn(0x7f4fc00b2f10 msgr2=0x7f4fc0143e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 -- 172.21.15.31:0/2086431960 shutdown_connections 2023-11-26T07:18:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:03.884+0000 7f4fc5479640 1 -- 172.21.15.31:0/2086431960 wait complete. 2023-11-26T07:18:03.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:04.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.048+0000 7f323f90e640 1 Processor -- start 2023-11-26T07:18:04.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.048+0000 7f323f90e640 1 -- start start 2023-11-26T07:18:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f323f90e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3238144e60 0x7f32380cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f323f90e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32380d05e0 con 0x7f3238070ca0 2023-11-26T07:18:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f323f90e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3238145240 con 0x7f32380707a0 2023-11-26T07:18:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f323f90e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380cf9d0 con 0x7f3238144e60 2023-11-26T07:18:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f323cea9640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3238070ca0 0x7f3238070090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51930/0 (socket says 172.21.15.31:51930) 2023-11-26T07:18:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f323cea9640 1 -- 172.21.15.31:0/993719353 learned_addr learned my addr 172.21.15.31:0/993719353 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f323d6aa640 1 --2- 172.21.15.31:0/993719353 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3238144e60 0x7f32380cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f322ffff640 1 -- 172.21.15.31:0/993719353 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882107884 0 0) 0x7f32380d05e0 con 0x7f3238070ca0 2023-11-26T07:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f322ffff640 1 -- 172.21.15.31:0/993719353 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3224003530 con 0x7f3238070ca0 2023-11-26T07:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f322ffff640 1 -- 172.21.15.31:0/993719353 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171853008 0 0) 0x7f3238145240 con 0x7f32380707a0 2023-11-26T07:18:04.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f322ffff640 1 -- 172.21.15.31:0/993719353 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32380d05e0 con 0x7f32380707a0 2023-11-26T07:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f323d6aa640 1 -- 172.21.15.31:0/993719353 >> v1:172.21.15.97:6789/0 conn(0x7f32380707a0 legacy=0x7f32380b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f323d6aa640 1 -- 172.21.15.31:0/993719353 >> v1:172.21.15.31:6789/0 conn(0x7f3238070ca0 legacy=0x7f3238070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.052+0000 7f323d6aa640 1 -- 172.21.15.31:0/993719353 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380cfcb0 con 0x7f3238144e60 2023-11-26T07:18:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f323d6aa640 1 --2- 172.21.15.31:0/993719353 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3238144e60 0x7f32380cf060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f32340097f0 tx=0x7f323401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b88c231c0e7c4495 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f322ffff640 1 -- 172.21.15.31:0/993719353 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340aa5f0 con 0x7f3238144e60 2023-11-26T07:18:04.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f322ffff640 1 -- 172.21.15.31:0/993719353 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32340aa750 con 0x7f3238144e60 2023-11-26T07:18:04.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f322ffff640 1 -- 172.21.15.31:0/993719353 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340aaac0 con 0x7f3238144e60 2023-11-26T07:18:04.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f323f90e640 1 -- 172.21.15.31:0/993719353 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3238144e60 msgr2=0x7f32380cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f323f90e640 1 --2- 172.21.15.31:0/993719353 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3238144e60 0x7f32380cf060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f32340097f0 tx=0x7f323401c490 comp rx=0 tx=0).stop 2023-11-26T07:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f323f90e640 1 -- 172.21.15.31:0/993719353 shutdown_connections 2023-11-26T07:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f323f90e640 1 --2- 172.21.15.31:0/993719353 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3238144e60 0x7f32380cf060 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f323f90e640 1 -- 172.21.15.31:0/993719353 >> 172.21.15.31:0/993719353 conn(0x7f32380b2f10 msgr2=0x7f32380b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f323f90e640 1 -- 172.21.15.31:0/993719353 shutdown_connections 2023-11-26T07:18:04.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f323f90e640 1 -- 172.21.15.31:0/993719353 wait complete. 2023-11-26T07:18:04.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f323f90e640 1 Processor -- start 2023-11-26T07:18:04.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.056+0000 7f323f90e640 1 -- start start 2023-11-26T07:18:04.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323f90e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32380707a0 0x7f323806c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323f90e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3238070ca0 0x7f32380ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323f90e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3238144e60 0x7f323815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323f90e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3238143510 con 0x7f3238144e60 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323f90e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323806cc70 con 0x7f3238070ca0 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323f90e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3238142ec0 con 0x7f32380707a0 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323d6aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32380707a0 0x7f323806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323deab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3238144e60 0x7f323815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323d6aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32380707a0 0x7f323806c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:54894/0 (socket says 172.21.15.31:54894) 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323d6aa640 1 -- 172.21.15.31:0/3400210540 learned_addr learned my addr 172.21.15.31:0/3400210540 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323cea9640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3238070ca0 0x7f32380ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323d6aa640 1 -- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3238070ca0 msgr2=0x7f32380ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323d6aa640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3238070ca0 0x7f32380ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323d6aa640 1 -- 172.21.15.31:0/3400210540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3238144e60 msgr2=0x7f323815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323d6aa640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3238144e60 0x7f323815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323d6aa640 1 -- 172.21.15.31:0/3400210540 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3234009420 con 0x7f32380707a0 2023-11-26T07:18:04.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323deab640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3238144e60 0x7f323815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:04.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323d6aa640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32380707a0 0x7f323806c300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3234000c10 tx=0x7f323401acd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f322dffb640 1 -- 172.21.15.31:0/3400210540 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340be610 con 0x7f32380707a0 2023-11-26T07:18:04.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f322dffb640 1 -- 172.21.15.31:0/3400210540 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32340be770 con 0x7f32380707a0 2023-11-26T07:18:04.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f322dffb640 1 -- 172.21.15.31:0/3400210540 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340beae0 con 0x7f32380707a0 2023-11-26T07:18:04.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323f90e640 1 -- 172.21.15.31:0/3400210540 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32381431a0 con 0x7f32380707a0 2023-11-26T07:18:04.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323f90e640 1 -- 172.21.15.31:0/3400210540 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323806c840 con 0x7f32380707a0 2023-11-26T07:18:04.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.060+0000 7f323f90e640 1 -- 172.21.15.31:0/3400210540 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323806ca30 con 0x7f32380707a0 2023-11-26T07:18:04.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.064+0000 7f322dffb640 1 -- 172.21.15.31:0/3400210540 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f32340c7420 con 0x7f32380707a0 2023-11-26T07:18:04.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.064+0000 7f322dffb640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f321c0664a0 0x7f321c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.064+0000 7f322dffb640 1 -- 172.21.15.31:0/3400210540 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32340a7070 con 0x7f32380707a0 2023-11-26T07:18:04.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.064+0000 7f323cea9640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f321c0664a0 0x7f321c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.064+0000 7f323cea9640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f321c0664a0 0x7f321c068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f32380d0f90 tx=0x7f3228007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.068+0000 7f322dffb640 1 -- 172.21.15.31:0/3400210540 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f32340ebf10 con 0x7f32380707a0 2023-11-26T07:18:04.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.224+0000 7f323f90e640 1 -- 172.21.15.31:0/3400210540 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f323806ca30 con 0x7f32380707a0 2023-11-26T07:18:04.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f322dffb640 1 -- 172.21.15.31:0/3400210540 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f323806ca30 con 0x7f32380707a0 2023-11-26T07:18:04.232 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:18:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 -- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f321c0664a0 msgr2=0x7f321c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f321c0664a0 0x7f321c068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f32380d0f90 tx=0x7f3228007d40 comp rx=0 tx=0).stop 2023-11-26T07:18:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 -- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32380707a0 msgr2=0x7f323806c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32380707a0 0x7f323806c300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3234000c10 tx=0x7f323401acd0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 -- 172.21.15.31:0/3400210540 shutdown_connections 2023-11-26T07:18:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f321c0664a0 0x7f321c068960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3238144e60 0x7f323815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3238070ca0 0x7f32380ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 --2- 172.21.15.31:0/3400210540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32380707a0 0x7f323806c300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 -- 172.21.15.31:0/3400210540 >> 172.21.15.31:0/3400210540 conn(0x7f32380b2f10 msgr2=0x7f32380d1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 -- 172.21.15.31:0/3400210540 shutdown_connections 2023-11-26T07:18:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.228+0000 7f323f90e640 1 -- 172.21.15.31:0/3400210540 wait complete. 2023-11-26T07:18:04.253 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:04.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.392+0000 7fa702580640 1 Processor -- start 2023-11-26T07:18:04.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.392+0000 7fa702580640 1 -- start start 2023-11-26T07:18:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.392+0000 7fa702580640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0cb9b0 0x7fa6fc06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.392+0000 7fa702580640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6fc0ce5d0 0x7fa6fc15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.392+0000 7fa702580640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc159630 con 0x7fa6fc0cbd90 2023-11-26T07:18:04.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.392+0000 7fa702580640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc15bba0 con 0x7fa6fc0ce5d0 2023-11-26T07:18:04.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.392+0000 7fa702580640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0cdd50 con 0x7fa6fc0cb9b0 2023-11-26T07:18:04.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6fb7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa6fc0cbd90 0x7fa6fc15f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51944/0 (socket says 172.21.15.31:51944) 2023-11-26T07:18:04.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6fb7fe640 1 -- 172.21.15.31:0/4147873478 learned_addr learned my addr 172.21.15.31:0/4147873478 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:04.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6fbfff640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0cb9b0 0x7fa6fc06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa700b1d640 1 --2- 172.21.15.31:0/4147873478 >> v2:172.21.15.97:3300/0 conn(0x7fa6fc0ce5d0 0x7fa6fc15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395160025 0 0) 0x7fa6fc159630 con 0x7fa6fc0cbd90 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e4003530 con 0x7fa6fc0cbd90 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 332874178 0 0) 0x7fa6e4003530 con 0x7fa6fc0cbd90 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0cb9b0 msgr2=0x7fa6fc06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0cb9b0 0x7fa6fc06e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 >> v2:172.21.15.97:3300/0 conn(0x7fa6fc0ce5d0 msgr2=0x7fa6fc15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 --2- 172.21.15.31:0/4147873478 >> v2:172.21.15.97:3300/0 conn(0x7fa6fc0ce5d0 0x7fa6fc15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0cd1d0 con 0x7fa6fc0cbd90 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fa6f0001680 con 0x7fa6fc0cbd90 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 >> v1:172.21.15.31:6789/0 conn(0x7fa6fc0cbd90 legacy=0x7fa6fc15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6e4003980 0x7fa6e4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0cb9b0 0x7fa6e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e4004d10 0x7fa6e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0cdd50 con 0x7fa6e4004d10 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e400b990 con 0x7fa6e4003980 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e400bad0 con 0x7fa6fc0cb9b0 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa700b1d640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e4004d10 0x7fa6e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6fbfff640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6e4003980 0x7fa6e4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa6fb7fe640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0cb9b0 0x7fa6e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa700b1d640 1 -- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0cb9b0 msgr2=0x7fa6e40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa700b1d640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0cb9b0 0x7fa6e40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa700b1d640 1 -- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6e4003980 msgr2=0x7fa6e4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa700b1d640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6e4003980 0x7fa6e4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.396+0000 7fa700b1d640 1 -- 172.21.15.31:0/4147873478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4003530 con 0x7fa6e4004d10 2023-11-26T07:18:04.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa700b1d640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e4004d10 0x7fa6e4005190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009fc0 tx=0x7fa6f401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfd2a1b83bda02e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40a5cb0 con 0x7fa6e4004d10 2023-11-26T07:18:04.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f400f040 con 0x7fa6e4004d10 2023-11-26T07:18:04.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa6faffd640 1 -- 172.21.15.31:0/4147873478 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40ad680 con 0x7fa6e4004d10 2023-11-26T07:18:04.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa702580640 1 -- 172.21.15.31:0/4147873478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e4004d10 msgr2=0x7fa6e4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa702580640 1 --2- 172.21.15.31:0/4147873478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6e4004d10 0x7fa6e4005190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009fc0 tx=0x7fa6f401cca0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa700b1d640 1 -- 172.21.15.31:0/4147873478 reap_dead start 2023-11-26T07:18:04.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa702580640 1 -- 172.21.15.31:0/4147873478 shutdown_connections 2023-11-26T07:18:04.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa702580640 1 -- 172.21.15.31:0/4147873478 >> 172.21.15.31:0/4147873478 conn(0x7fa6fc0b0cb0 msgr2=0x7fa6fc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa702580640 1 -- 172.21.15.31:0/4147873478 shutdown_connections 2023-11-26T07:18:04.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa702580640 1 -- 172.21.15.31:0/4147873478 wait complete. 2023-11-26T07:18:04.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa702580640 1 Processor -- start 2023-11-26T07:18:04.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.400+0000 7fa702580640 1 -- start start 2023-11-26T07:18:04.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa702580640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6e4004d10 0x7fa6fc1475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa702580640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0c40e0 0x7fa6fc0b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa702580640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6fc149d40 0x7fa6fc14a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa702580640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0ad800 con 0x7fa6fc149d40 2023-11-26T07:18:04.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa702580640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0ac9a0 con 0x7fa6e4004d10 2023-11-26T07:18:04.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa702580640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc147b20 con 0x7fa6fc0c40e0 2023-11-26T07:18:04.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa700b1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6fc149d40 0x7fa6fc14a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa700b1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6fc149d40 0x7fa6fc14a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45304/0 (socket says 172.21.15.31:45304) 2023-11-26T07:18:04.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa700b1d640 1 -- 172.21.15.31:0/3224676694 learned_addr learned my addr 172.21.15.31:0/3224676694 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:04.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa6fb7fe640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0c40e0 0x7fa6fc0b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa6fbfff640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6e4004d10 0x7fa6fc1475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa700b1d640 1 -- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0c40e0 msgr2=0x7fa6fc0b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa700b1d640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0c40e0 0x7fa6fc0b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa700b1d640 1 -- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6e4004d10 msgr2=0x7fa6fc1475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa700b1d640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6e4004d10 0x7fa6fc1475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa700b1d640 1 -- 172.21.15.31:0/3224676694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0b0a10 con 0x7fa6fc149d40 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa6fbfff640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6e4004d10 0x7fa6fc1475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:04.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa700b1d640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6fc149d40 0x7fa6fc14a1d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40b0040 tx=0x7fa6f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa6f8ff9640 1 -- 172.21.15.31:0/3224676694 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40ad440 con 0x7fa6fc149d40 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa6f8ff9640 1 -- 172.21.15.31:0/3224676694 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40aeab0 con 0x7fa6fc149d40 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa6f8ff9640 1 -- 172.21.15.31:0/3224676694 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40aee60 con 0x7fa6fc149d40 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa702580640 1 -- 172.21.15.31:0/3224676694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc0acbf0 con 0x7fa6fc149d40 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa702580640 1 -- 172.21.15.31:0/3224676694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc0ad1b0 con 0x7fa6fc149d40 2023-11-26T07:18:04.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.404+0000 7fa702580640 1 -- 172.21.15.31:0/3224676694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc0671f0 con 0x7fa6fc149d40 2023-11-26T07:18:04.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.408+0000 7fa6f8ff9640 1 -- 172.21.15.31:0/3224676694 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa6f40b4040 con 0x7fa6fc149d40 2023-11-26T07:18:04.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.408+0000 7fa6f8ff9640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa6e80664a0 0x7fa6e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.408+0000 7fa6f8ff9640 1 -- 172.21.15.31:0/3224676694 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6f411db90 con 0x7fa6fc149d40 2023-11-26T07:18:04.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.408+0000 7fa6fbfff640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa6e80664a0 0x7fa6e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.408+0000 7fa6fbfff640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa6e80664a0 0x7fa6e8068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0093f0 tx=0x7fa6ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.412+0000 7fa6f8ff9640 1 -- 172.21.15.31:0/3224676694 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6f40eb020 con 0x7fa6fc149d40 2023-11-26T07:18:04.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 -- 172.21.15.31:0/3224676694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa6fc06b460 con 0x7fa6fc149d40 2023-11-26T07:18:04.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa6f8ff9640 1 -- 172.21.15.31:0/3224676694 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa6f40be080 con 0x7fa6fc149d40 2023-11-26T07:18:04.585 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:18:04.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 -- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa6e80664a0 msgr2=0x7fa6e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa6e80664a0 0x7fa6e8068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0093f0 tx=0x7fa6ec008040 comp rx=0 tx=0).stop 2023-11-26T07:18:04.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 -- 172.21.15.31:0/3224676694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6fc149d40 msgr2=0x7fa6fc14a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6fc149d40 0x7fa6fc14a1d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40b0040 tx=0x7fa6f4009750 comp rx=0 tx=0).stop 2023-11-26T07:18:04.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 -- 172.21.15.31:0/3224676694 shutdown_connections 2023-11-26T07:18:04.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa6e80664a0 0x7fa6e8068960 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6fc149d40 0x7fa6fc14a1d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6fc0c40e0 0x7fa6fc0b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 --2- 172.21.15.31:0/3224676694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6e4004d10 0x7fa6fc1475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 -- 172.21.15.31:0/3224676694 >> 172.21.15.31:0/3224676694 conn(0x7fa6fc0b0cb0 msgr2=0x7fa6fc0ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 -- 172.21.15.31:0/3224676694 shutdown_connections 2023-11-26T07:18:04.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.580+0000 7fa702580640 1 -- 172.21.15.31:0/3224676694 wait complete. 2023-11-26T07:18:04.607 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:18:04.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:04.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-26T07:18:04.642 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:18:04.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:04.642 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:18:04.729 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:04.729 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:04.739 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:04.739 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:04.791 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:04.792 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:04.844 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:04.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.980+0000 7f8ac9be4640 1 Processor -- start 2023-11-26T07:18:04.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.980+0000 7f8ac9be4640 1 -- start start 2023-11-26T07:18:04.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac9be4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 0x7f8ac40d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac9be4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8ac40cfa20 0x7f8ac40cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac9be4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac40d15f0 con 0x7f8ac415d880 2023-11-26T07:18:04.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac9be4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac40d40e0 con 0x7f8ac40cfa20 2023-11-26T07:18:04.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac9be4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40d03d0 con 0x7f8ac40d3760 2023-11-26T07:18:04.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac93e3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8ac40cfa20 0x7f8ac40cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac8be2640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8ac415d880 0x7f8ac40b6730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51956/0 (socket says 172.21.15.31:51956) 2023-11-26T07:18:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac8be2640 1 -- 172.21.15.31:0/3356294995 learned_addr learned my addr 172.21.15.31:0/3356294995 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:04.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac3fff640 1 --2- 172.21.15.31:0/3356294995 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 0x7f8ac40d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac93e3640 1 -- 172.21.15.31:0/3356294995 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 msgr2=0x7f8ac40d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac93e3640 1 --2- 172.21.15.31:0/3356294995 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 0x7f8ac40d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac93e3640 1 -- 172.21.15.31:0/3356294995 >> v1:172.21.15.31:6789/0 conn(0x7f8ac415d880 legacy=0x7f8ac40b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac93e3640 1 -- 172.21.15.31:0/3356294995 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac40ce000 con 0x7f8ac40cfa20 2023-11-26T07:18:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac37fe640 1 -- 172.21.15.31:0/3356294995 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341711507 0 0) 0x7f8ac40d15f0 con 0x7f8ac415d880 2023-11-26T07:18:04.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.984+0000 7f8ac3fff640 1 --2- 172.21.15.31:0/3356294995 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 0x7f8ac40d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:04.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac93e3640 1 --2- 172.21.15.31:0/3356294995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 0x7f8ac40cfe90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d90 tx=0x7f8abc01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5762c79c4eb428f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac37fe640 1 -- 172.21.15.31:0/3356294995 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0a5a10 con 0x7f8ac40cfa20 2023-11-26T07:18:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac37fe640 1 -- 172.21.15.31:0/3356294995 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc0a5b70 con 0x7f8ac40cfa20 2023-11-26T07:18:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac37fe640 1 -- 172.21.15.31:0/3356294995 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0ad440 con 0x7f8ac40cfa20 2023-11-26T07:18:04.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac9be4640 1 -- 172.21.15.31:0/3356294995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 msgr2=0x7f8ac40cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac9be4640 1 --2- 172.21.15.31:0/3356294995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 0x7f8ac40cfe90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d90 tx=0x7f8abc01c950 comp rx=0 tx=0).stop 2023-11-26T07:18:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac9be4640 1 -- 172.21.15.31:0/3356294995 shutdown_connections 2023-11-26T07:18:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac9be4640 1 --2- 172.21.15.31:0/3356294995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 0x7f8ac40cfe90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac9be4640 1 --2- 172.21.15.31:0/3356294995 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 0x7f8ac40d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac9be4640 1 -- 172.21.15.31:0/3356294995 >> 172.21.15.31:0/3356294995 conn(0x7f8ac406b9f0 msgr2=0x7f8ac406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac9be4640 1 -- 172.21.15.31:0/3356294995 shutdown_connections 2023-11-26T07:18:04.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac9be4640 1 -- 172.21.15.31:0/3356294995 wait complete. 2023-11-26T07:18:04.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac9be4640 1 Processor -- start 2023-11-26T07:18:04.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.988+0000 7f8ac9be4640 1 -- start start 2023-11-26T07:18:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac9be4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 0x7f8ac4156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac9be4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 0x7f8ac415b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac9be4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac415d880 0x7f8ac415be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac9be4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac415c3c0 con 0x7f8ac40cfa20 2023-11-26T07:18:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac9be4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40cb750 con 0x7f8ac415d880 2023-11-26T07:18:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac9be4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac415c600 con 0x7f8ac40d3760 2023-11-26T07:18:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac8be2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 0x7f8ac4156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac8be2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 0x7f8ac4156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45326/0 (socket says 172.21.15.31:45326) 2023-11-26T07:18:04.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac8be2640 1 -- 172.21.15.31:0/1695041446 learned_addr learned my addr 172.21.15.31:0/1695041446 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac93e3640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac415d880 0x7f8ac415be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac3fff640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 0x7f8ac415b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac8be2640 1 -- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 msgr2=0x7f8ac415b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac8be2640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 0x7f8ac415b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac8be2640 1 -- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac415d880 msgr2=0x7f8ac415be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac8be2640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac415d880 0x7f8ac415be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac8be2640 1 -- 172.21.15.31:0/1695041446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc009a70 con 0x7f8ac40cfa20 2023-11-26T07:18:04.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac93e3640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac415d880 0x7f8ac415be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac8be2640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 0x7f8ac4156930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c480 tx=0x7f8ab800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac17fa640 1 -- 172.21.15.31:0/1695041446 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80074e0 con 0x7f8ac40cfa20 2023-11-26T07:18:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac17fa640 1 -- 172.21.15.31:0/1695041446 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab8007660 con 0x7f8ac40cfa20 2023-11-26T07:18:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac9be4640 1 -- 172.21.15.31:0/1695041446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac40cae60 con 0x7f8ac40cfa20 2023-11-26T07:18:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac17fa640 1 -- 172.21.15.31:0/1695041446 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8004a30 con 0x7f8ac40cfa20 2023-11-26T07:18:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac9be4640 1 -- 172.21.15.31:0/1695041446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac40cb100 con 0x7f8ac40cfa20 2023-11-26T07:18:04.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac9be4640 1 -- 172.21.15.31:0/1695041446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac4074a00 con 0x7f8ac40cfa20 2023-11-26T07:18:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.992+0000 7f8ac17fa640 1 -- 172.21.15.31:0/1695041446 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8ab8004b90 con 0x7f8ac40cfa20 2023-11-26T07:18:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.996+0000 7f8ac17fa640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8aa8066470 0x7f8aa8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.996+0000 7f8ac17fa640 1 -- 172.21.15.31:0/1695041446 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ab80833c0 con 0x7f8ac40cfa20 2023-11-26T07:18:05.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.996+0000 7f8ac3fff640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8aa8066470 0x7f8aa8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:04.996+0000 7f8ac3fff640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8aa8066470 0x7f8aa8068930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8ac415d2a0 tx=0x7f8ab0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.000+0000 7f8ac17fa640 1 -- 172.21.15.31:0/1695041446 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8ab80542c0 con 0x7f8ac40cfa20 2023-11-26T07:18:05.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.164+0000 7f8ac9be4640 1 -- 172.21.15.31:0/1695041446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ac40d11a0 con 0x7f8ac40cfa20 2023-11-26T07:18:05.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.164+0000 7f8ac17fa640 1 -- 172.21.15.31:0/1695041446 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f8ab8053e50 con 0x7f8ac40cfa20 2023-11-26T07:18:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 -- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8aa8066470 msgr2=0x7f8aa8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8aa8066470 0x7f8aa8068930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8ac415d2a0 tx=0x7f8ab0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 -- 172.21.15.31:0/1695041446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 msgr2=0x7f8ac4156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 0x7f8ac4156930 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c480 tx=0x7f8ab800c940 comp rx=0 tx=0).stop 2023-11-26T07:18:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 -- 172.21.15.31:0/1695041446 shutdown_connections 2023-11-26T07:18:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8aa8066470 0x7f8aa8068930 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8ac415d880 0x7f8ac415be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8ac40d3760 0x7f8ac415b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 --2- 172.21.15.31:0/1695041446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac40cfa20 0x7f8ac4156930 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 -- 172.21.15.31:0/1695041446 >> 172.21.15.31:0/1695041446 conn(0x7f8ac406b9f0 msgr2=0x7f8ac406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 -- 172.21.15.31:0/1695041446 shutdown_connections 2023-11-26T07:18:05.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.168+0000 7f8ac9be4640 1 -- 172.21.15.31:0/1695041446 wait complete. 2023-11-26T07:18:05.191 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:05.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:05.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf98032640 1 Processor -- start 2023-11-26T07:18:05.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf98032640 1 -- start start 2023-11-26T07:18:05.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf98032640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b1fb0 0x7faf90165d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf98032640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf900b2590 con 0x7faf900b3c60 2023-11-26T07:18:05.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf98032640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf900b2870 con 0x7faf900b4040 2023-11-26T07:18:05.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf98032640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf900b2b50 con 0x7faf900b1fb0 2023-11-26T07:18:05.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf95dce640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faf900b3c60 0x7faf900ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51966/0 (socket says 172.21.15.31:51966) 2023-11-26T07:18:05.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf95dce640 1 -- 172.21.15.31:0/950369009 learned_addr learned my addr 172.21.15.31:0/950369009 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:05.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf965cf640 1 --2- 172.21.15.31:0/950369009 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b1fb0 0x7faf90165d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf94dcc640 1 -- 172.21.15.31:0/950369009 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260049019 0 0) 0x7faf900b2590 con 0x7faf900b3c60 2023-11-26T07:18:05.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf94dcc640 1 -- 172.21.15.31:0/950369009 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf7c003530 con 0x7faf900b3c60 2023-11-26T07:18:05.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf94dcc640 1 -- 172.21.15.31:0/950369009 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749821497 0 0) 0x7faf900b2870 con 0x7faf900b4040 2023-11-26T07:18:05.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf94dcc640 1 -- 172.21.15.31:0/950369009 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf900b2590 con 0x7faf900b4040 2023-11-26T07:18:05.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf965cf640 1 -- 172.21.15.31:0/950369009 >> v1:172.21.15.97:6789/0 conn(0x7faf900b4040 legacy=0x7faf90162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf965cf640 1 -- 172.21.15.31:0/950369009 >> v1:172.21.15.31:6789/0 conn(0x7faf900b3c60 legacy=0x7faf900ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.336+0000 7faf965cf640 1 -- 172.21.15.31:0/950369009 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf900b2e30 con 0x7faf900b1fb0 2023-11-26T07:18:05.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf965cf640 1 --2- 172.21.15.31:0/950369009 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b1fb0 0x7faf90165d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf88009dc0 tx=0x7faf8801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41d4581d30799981 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf94dcc640 1 -- 172.21.15.31:0/950369009 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf880a5910 con 0x7faf900b1fb0 2023-11-26T07:18:05.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf94dcc640 1 -- 172.21.15.31:0/950369009 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf880a5a70 con 0x7faf900b1fb0 2023-11-26T07:18:05.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf94dcc640 1 -- 172.21.15.31:0/950369009 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf880a5e00 con 0x7faf900b1fb0 2023-11-26T07:18:05.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf98032640 1 -- 172.21.15.31:0/950369009 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b1fb0 msgr2=0x7faf90165d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf98032640 1 --2- 172.21.15.31:0/950369009 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b1fb0 0x7faf90165d90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faf88009dc0 tx=0x7faf8801c940 comp rx=0 tx=0).stop 2023-11-26T07:18:05.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf98032640 1 -- 172.21.15.31:0/950369009 shutdown_connections 2023-11-26T07:18:05.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf98032640 1 --2- 172.21.15.31:0/950369009 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b1fb0 0x7faf90165d90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf98032640 1 -- 172.21.15.31:0/950369009 >> 172.21.15.31:0/950369009 conn(0x7faf900b0cb0 msgr2=0x7faf900b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf98032640 1 -- 172.21.15.31:0/950369009 shutdown_connections 2023-11-26T07:18:05.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.340+0000 7faf98032640 1 -- 172.21.15.31:0/950369009 wait complete. 2023-11-26T07:18:05.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf98032640 1 Processor -- start 2023-11-26T07:18:05.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf98032640 1 -- start start 2023-11-26T07:18:05.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf98032640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf900b1fb0 0x7faf900ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf98032640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b3c60 0x7faf9015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf98032640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9015b5b0 0x7faf90145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf98032640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9015a420 con 0x7faf900b1fb0 2023-11-26T07:18:05.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf98032640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf90159590 con 0x7faf9015b5b0 2023-11-26T07:18:05.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf98032640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf90164cb0 con 0x7faf900b3c60 2023-11-26T07:18:05.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf95dce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf900b1fb0 0x7faf900ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf95dce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf900b1fb0 0x7faf900ad240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45336/0 (socket says 172.21.15.31:45336) 2023-11-26T07:18:05.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf95dce640 1 -- 172.21.15.31:0/819343429 learned_addr learned my addr 172.21.15.31:0/819343429 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:05.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf955cd640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b3c60 0x7faf9015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf965cf640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9015b5b0 0x7faf90145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf95dce640 1 -- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b3c60 msgr2=0x7faf9015b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf95dce640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b3c60 0x7faf9015b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf95dce640 1 -- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9015b5b0 msgr2=0x7faf90145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf95dce640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9015b5b0 0x7faf90145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf95dce640 1 -- 172.21.15.31:0/819343429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf88009a70 con 0x7faf900b1fb0 2023-11-26T07:18:05.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf965cf640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9015b5b0 0x7faf90145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:05.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf955cd640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b3c60 0x7faf9015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:05.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf95dce640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf900b1fb0 0x7faf900ad240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c4f0 tx=0x7faf8c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf867fc640 1 -- 172.21.15.31:0/819343429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c005cf0 con 0x7faf900b1fb0 2023-11-26T07:18:05.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf867fc640 1 -- 172.21.15.31:0/819343429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c005e50 con 0x7faf900b1fb0 2023-11-26T07:18:05.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf867fc640 1 -- 172.21.15.31:0/819343429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c007270 con 0x7faf900b1fb0 2023-11-26T07:18:05.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf98032640 1 -- 172.21.15.31:0/819343429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf901597a0 con 0x7faf900b1fb0 2023-11-26T07:18:05.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.344+0000 7faf98032640 1 -- 172.21.15.31:0/819343429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf90159d60 con 0x7faf900b1fb0 2023-11-26T07:18:05.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.348+0000 7faf98032640 1 -- 172.21.15.31:0/819343429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf900671f0 con 0x7faf900b1fb0 2023-11-26T07:18:05.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.348+0000 7faf867fc640 1 -- 172.21.15.31:0/819343429 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7faf8c0073d0 con 0x7faf900b1fb0 2023-11-26T07:18:05.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.348+0000 7faf867fc640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf74066470 0x7faf74068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.348+0000 7faf867fc640 1 -- 172.21.15.31:0/819343429 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf8c083410 con 0x7faf900b1fb0 2023-11-26T07:18:05.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.348+0000 7faf955cd640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf74066470 0x7faf74068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.348+0000 7faf955cd640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf74066470 0x7faf74068930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf9015ae10 tx=0x7faf80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.352+0000 7faf867fc640 1 -- 172.21.15.31:0/819343429 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faf8c0508a0 con 0x7faf900b1fb0 2023-11-26T07:18:05.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.508+0000 7faf98032640 1 -- 172.21.15.31:0/819343429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf9006b460 con 0x7faf900b1fb0 2023-11-26T07:18:05.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.508+0000 7faf867fc640 1 -- 172.21.15.31:0/819343429 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf8c053ec0 con 0x7faf900b1fb0 2023-11-26T07:18:05.515 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 -- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf74066470 msgr2=0x7faf74068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf74066470 0x7faf74068930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf9015ae10 tx=0x7faf80009040 comp rx=0 tx=0).stop 2023-11-26T07:18:05.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 -- 172.21.15.31:0/819343429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf900b1fb0 msgr2=0x7faf900ad240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf900b1fb0 0x7faf900ad240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00c4f0 tx=0x7faf8c00c9b0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 -- 172.21.15.31:0/819343429 shutdown_connections 2023-11-26T07:18:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf74066470 0x7faf74068930 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf9015b5b0 0x7faf90145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf900b3c60 0x7faf9015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 --2- 172.21.15.31:0/819343429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf900b1fb0 0x7faf900ad240 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 -- 172.21.15.31:0/819343429 >> 172.21.15.31:0/819343429 conn(0x7faf900b0cb0 msgr2=0x7faf900b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 -- 172.21.15.31:0/819343429 shutdown_connections 2023-11-26T07:18:05.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.512+0000 7faf98032640 1 -- 172.21.15.31:0/819343429 wait complete. 2023-11-26T07:18:05.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:05.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.676+0000 7f8fadc32640 1 Processor -- start 2023-11-26T07:18:05.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.676+0000 7f8fadc32640 1 -- start start 2023-11-26T07:18:05.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fadc32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fa80b3c60 0x7f8fa80b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fadc32640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8fa80b4040 0x7f8fa80ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fadc32640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8fa8162190 0x7f8fa8162600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fadc32640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa80b2590 con 0x7f8fa80b4040 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fadc32640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa80b2870 con 0x7f8fa8162190 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fadc32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa80b2b50 con 0x7f8fa80b3c60 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa6ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8fa80b4040 0x7f8fa80ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa6ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8fa80b4040 0x7f8fa80ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45338/0 (socket says 172.21.15.31:45338) 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa6ffd640 1 -- 172.21.15.31:0/58991194 learned_addr learned my addr 172.21.15.31:0/58991194 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa77fe640 1 --2- 172.21.15.31:0/58991194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fa80b3c60 0x7f8fa80b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa7fff640 1 --2- 172.21.15.31:0/58991194 >> v2:172.21.15.97:3300/0 conn(0x7f8fa8162190 0x7f8fa8162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa6ffd640 1 -- 172.21.15.31:0/58991194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fa80b3c60 msgr2=0x7f8fa80b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:05.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa6ffd640 1 --2- 172.21.15.31:0/58991194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fa80b3c60 0x7f8fa80b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa6ffd640 1 -- 172.21.15.31:0/58991194 >> v2:172.21.15.97:3300/0 conn(0x7f8fa8162190 msgr2=0x7f8fa8162600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:05.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa6ffd640 1 --2- 172.21.15.31:0/58991194 >> v2:172.21.15.97:3300/0 conn(0x7f8fa8162190 0x7f8fa8162600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa6ffd640 1 -- 172.21.15.31:0/58991194 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa80b2e30 con 0x7f8fa80b4040 2023-11-26T07:18:05.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa7fff640 1 --2- 172.21.15.31:0/58991194 >> v2:172.21.15.97:3300/0 conn(0x7f8fa8162190 0x7f8fa8162600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:05.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.680+0000 7f8fa6ffd640 1 --2- 172.21.15.31:0/58991194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa80b4040 0x7f8fa80ce5e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8fa806c0c0 tx=0x7f8f9c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84fdd0ba536da50 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fa67fc640 1 -- 172.21.15.31:0/58991194 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0ab620 con 0x7f8fa80b4040 2023-11-26T07:18:05.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fa67fc640 1 -- 172.21.15.31:0/58991194 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c0ab780 con 0x7f8fa80b4040 2023-11-26T07:18:05.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fa67fc640 1 -- 172.21.15.31:0/58991194 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0abb10 con 0x7f8fa80b4040 2023-11-26T07:18:05.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 -- 172.21.15.31:0/58991194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa80b4040 msgr2=0x7f8fa80ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 --2- 172.21.15.31:0/58991194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa80b4040 0x7f8fa80ce5e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8fa806c0c0 tx=0x7f8f9c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 -- 172.21.15.31:0/58991194 shutdown_connections 2023-11-26T07:18:05.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 --2- 172.21.15.31:0/58991194 >> v2:172.21.15.97:3300/0 conn(0x7f8fa8162190 0x7f8fa8162600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 --2- 172.21.15.31:0/58991194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa80b4040 0x7f8fa80ce5e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 --2- 172.21.15.31:0/58991194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fa80b3c60 0x7f8fa80b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 -- 172.21.15.31:0/58991194 >> 172.21.15.31:0/58991194 conn(0x7f8fa80b0cb0 msgr2=0x7f8fa80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 -- 172.21.15.31:0/58991194 shutdown_connections 2023-11-26T07:18:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 -- 172.21.15.31:0/58991194 wait complete. 2023-11-26T07:18:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 Processor -- start 2023-11-26T07:18:05.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 -- start start 2023-11-26T07:18:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fa80b3c60 0x7f8fa815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8162190 0x7f8fa8164d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa815d7a0 0x7f8fa8162dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa8165260 con 0x7f8fa815d7a0 2023-11-26T07:18:05.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa81460b0 con 0x7f8fa8162190 2023-11-26T07:18:05.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.684+0000 7f8fadc32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa8147080 con 0x7f8fa80b3c60 2023-11-26T07:18:05.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa815d7a0 0x7f8fa8162dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa77fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fa80b3c60 0x7f8fa815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa815d7a0 0x7f8fa8162dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45354/0 (socket says 172.21.15.31:45354) 2023-11-26T07:18:05.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa7fff640 1 -- 172.21.15.31:0/3561306900 learned_addr learned my addr 172.21.15.31:0/3561306900 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:05.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa6ffd640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8162190 0x7f8fa8164d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa7fff640 1 -- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fa80b3c60 msgr2=0x7f8fa815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa7fff640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fa80b3c60 0x7f8fa815cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa7fff640 1 -- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8162190 msgr2=0x7f8fa8164d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa7fff640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8162190 0x7f8fa8164d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa7fff640 1 -- 172.21.15.31:0/3561306900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c0094d0 con 0x7f8fa815d7a0 2023-11-26T07:18:05.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fa7fff640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa815d7a0 0x7f8fa8162dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0007e20 tx=0x7f8fa000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8f8ffff640 1 -- 172.21.15.31:0/3561306900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0003ba0 con 0x7f8fa815d7a0 2023-11-26T07:18:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8f8ffff640 1 -- 172.21.15.31:0/3561306900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa000f040 con 0x7f8fa815d7a0 2023-11-26T07:18:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fadc32640 1 -- 172.21.15.31:0/3561306900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa81462c0 con 0x7f8fa815d7a0 2023-11-26T07:18:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8f8ffff640 1 -- 172.21.15.31:0/3561306900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000de80 con 0x7f8fa815d7a0 2023-11-26T07:18:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fadc32640 1 -- 172.21.15.31:0/3561306900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa8146880 con 0x7f8fa815d7a0 2023-11-26T07:18:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8fadc32640 1 -- 172.21.15.31:0/3561306900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa80671f0 con 0x7f8fa815d7a0 2023-11-26T07:18:05.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8f8ffff640 1 -- 172.21.15.31:0/3561306900 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8fa0004030 con 0x7f8fa815d7a0 2023-11-26T07:18:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8f8ffff640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8f94066470 0x7f8f94068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.688+0000 7f8f8ffff640 1 -- 172.21.15.31:0/3561306900 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fa00840d0 con 0x7f8fa815d7a0 2023-11-26T07:18:05.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.692+0000 7f8fa77fe640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8f94066470 0x7f8f94068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:05.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.692+0000 7f8fa77fe640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8f94066470 0x7f8f94068930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fa815c720 tx=0x7f8f98009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:05.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.692+0000 7f8f8ffff640 1 -- 172.21.15.31:0/3561306900 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8fa00515f0 con 0x7f8fa815d7a0 2023-11-26T07:18:05.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 -- 172.21.15.31:0/3561306900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fa806b460 con 0x7f8fa815d7a0 2023-11-26T07:18:05.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8f8ffff640 1 -- 172.21.15.31:0/3561306900 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8fa0054c10 con 0x7f8fa815d7a0 2023-11-26T07:18:05.860 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:18:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 -- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8f94066470 msgr2=0x7f8f94068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8f94066470 0x7f8f94068930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fa815c720 tx=0x7f8f98009190 comp rx=0 tx=0).stop 2023-11-26T07:18:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 -- 172.21.15.31:0/3561306900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa815d7a0 msgr2=0x7f8fa8162dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa815d7a0 0x7f8fa8162dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0007e20 tx=0x7f8fa000d4a0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 -- 172.21.15.31:0/3561306900 shutdown_connections 2023-11-26T07:18:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8f94066470 0x7f8f94068930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fa815d7a0 0x7f8fa8162dd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa8162190 0x7f8fa8164d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 --2- 172.21.15.31:0/3561306900 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fa80b3c60 0x7f8fa815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 -- 172.21.15.31:0/3561306900 >> 172.21.15.31:0/3561306900 conn(0x7f8fa80b0cb0 msgr2=0x7f8fa80b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 -- 172.21.15.31:0/3561306900 shutdown_connections 2023-11-26T07:18:05.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:05.856+0000 7f8fadc32640 1 -- 172.21.15.31:0/3561306900 wait complete. 2023-11-26T07:18:05.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.020+0000 7f12e2638640 1 Processor -- start 2023-11-26T07:18:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.020+0000 7f12e2638640 1 -- start start 2023-11-26T07:18:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e2638640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f12dc15d2a0 0x7f12dc0d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e2638640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 0x7f12dc0b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e2638640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12dc0b6250 con 0x7f12dc0b5180 2023-11-26T07:18:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e2638640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc0ce8c0 con 0x7f12dc15d2a0 2023-11-26T07:18:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e2638640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc161cf0 con 0x7f12dc0d2c20 2023-11-26T07:18:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e1e37640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f12dc0b5180 0x7f12dc06b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51974/0 (socket says 172.21.15.31:51974) 2023-11-26T07:18:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e1e37640 1 -- 172.21.15.31:0/1021425720 learned_addr learned my addr 172.21.15.31:0/1021425720 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e0e35640 1 --2- 172.21.15.31:0/1021425720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 0x7f12dc0b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e1636640 1 --2- 172.21.15.31:0/1021425720 >> v2:172.21.15.97:3300/0 conn(0x7f12dc15d2a0 0x7f12dc0d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12d3fff640 1 -- 172.21.15.31:0/1021425720 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957055067 0 0) 0x7f12dc0b6250 con 0x7f12dc0b5180 2023-11-26T07:18:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12d3fff640 1 -- 172.21.15.31:0/1021425720 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c8003530 con 0x7f12dc0b5180 2023-11-26T07:18:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e1636640 1 -- 172.21.15.31:0/1021425720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 msgr2=0x7f12dc0b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e1636640 1 --2- 172.21.15.31:0/1021425720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 0x7f12dc0b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e1636640 1 -- 172.21.15.31:0/1021425720 >> v1:172.21.15.31:6789/0 conn(0x7f12dc0b5180 legacy=0x7f12dc06b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e1636640 1 -- 172.21.15.31:0/1021425720 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc0b5590 con 0x7f12dc15d2a0 2023-11-26T07:18:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.024+0000 7f12e0e35640 1 --2- 172.21.15.31:0/1021425720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 0x7f12dc0b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:06.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e1636640 1 --2- 172.21.15.31:0/1021425720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12dc15d2a0 0x7f12dc0d26e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12d80097c0 tx=0x7f12d801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cc63043bbfe256b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12d3fff640 1 -- 172.21.15.31:0/1021425720 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80ab600 con 0x7f12dc15d2a0 2023-11-26T07:18:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12d3fff640 1 -- 172.21.15.31:0/1021425720 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d80a7050 con 0x7f12dc15d2a0 2023-11-26T07:18:06.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12d3fff640 1 -- 172.21.15.31:0/1021425720 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80ab980 con 0x7f12dc15d2a0 2023-11-26T07:18:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 -- 172.21.15.31:0/1021425720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12dc15d2a0 msgr2=0x7f12dc0d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 --2- 172.21.15.31:0/1021425720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12dc15d2a0 0x7f12dc0d26e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f12d80097c0 tx=0x7f12d801c450 comp rx=0 tx=0).stop 2023-11-26T07:18:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 -- 172.21.15.31:0/1021425720 shutdown_connections 2023-11-26T07:18:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 --2- 172.21.15.31:0/1021425720 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 0x7f12dc0b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 --2- 172.21.15.31:0/1021425720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12dc15d2a0 0x7f12dc0d26e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 -- 172.21.15.31:0/1021425720 >> 172.21.15.31:0/1021425720 conn(0x7f12dc0668a0 msgr2=0x7f12dc0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 -- 172.21.15.31:0/1021425720 shutdown_connections 2023-11-26T07:18:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 -- 172.21.15.31:0/1021425720 wait complete. 2023-11-26T07:18:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 Processor -- start 2023-11-26T07:18:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 -- start start 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12dc0b5180 0x7f12dc0b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.028+0000 7f12e2638640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 0x7f12dc156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e2638640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12dc15d2a0 0x7f12dc14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e2638640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc155f30 con 0x7f12dc0b5180 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e2638640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0b4eb0 con 0x7f12dc15d2a0 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e2638640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc1552b0 con 0x7f12dc0d2c20 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e1636640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12dc0b5180 0x7f12dc0b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e1636640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12dc0b5180 0x7f12dc0b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45370/0 (socket says 172.21.15.31:45370) 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e1636640 1 -- 172.21.15.31:0/936355718 learned_addr learned my addr 172.21.15.31:0/936355718 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e0e35640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 0x7f12dc156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e1e37640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12dc15d2a0 0x7f12dc14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e1636640 1 -- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 msgr2=0x7f12dc156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e1636640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 0x7f12dc156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e1636640 1 -- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12dc15d2a0 msgr2=0x7f12dc14da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e1636640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12dc15d2a0 0x7f12dc14da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e1636640 1 -- 172.21.15.31:0/936355718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc009420 con 0x7f12dc0b5180 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e0e35640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 0x7f12dc156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e1636640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12dc0b5180 0x7f12dc0b2bd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f12d80ae040 tx=0x7f12d80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12d1ffb640 1 -- 172.21.15.31:0/936355718 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80b4430 con 0x7f12dc0b5180 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12d1ffb640 1 -- 172.21.15.31:0/936355718 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d80b45b0 con 0x7f12dc0b5180 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e2638640 1 -- 172.21.15.31:0/936355718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d8009420 con 0x7f12dc0b5180 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12d1ffb640 1 -- 172.21.15.31:0/936355718 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80b4960 con 0x7f12dc0b5180 2023-11-26T07:18:06.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e2638640 1 -- 172.21.15.31:0/936355718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc155870 con 0x7f12dc0b5180 2023-11-26T07:18:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12e2638640 1 -- 172.21.15.31:0/936355718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc074990 con 0x7f12dc0b5180 2023-11-26T07:18:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.032+0000 7f12d1ffb640 1 -- 172.21.15.31:0/936355718 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f12d80b2040 con 0x7f12dc0b5180 2023-11-26T07:18:06.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.036+0000 7f12d1ffb640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f12c00664c0 0x7f12c0068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.036+0000 7f12d1ffb640 1 -- 172.21.15.31:0/936355718 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12d811dcc0 con 0x7f12dc0b5180 2023-11-26T07:18:06.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.036+0000 7f12e0e35640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f12c00664c0 0x7f12c0068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.036+0000 7f12e0e35640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f12c00664c0 0x7f12c0068980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0cd8a0 tx=0x7f12cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.040+0000 7f12d1ffb640 1 -- 172.21.15.31:0/936355718 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f12d8124080 con 0x7f12dc0b5180 2023-11-26T07:18:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 -- 172.21.15.31:0/936355718 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12dc0b5f60 con 0x7f12dc0b5180 2023-11-26T07:18:06.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12d1ffb640 1 -- 172.21.15.31:0/936355718 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12d80eb120 con 0x7f12dc0b5180 2023-11-26T07:18:06.201 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:18:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 -- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f12c00664c0 msgr2=0x7f12c0068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f12c00664c0 0x7f12c0068980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0cd8a0 tx=0x7f12cc008040 comp rx=0 tx=0).stop 2023-11-26T07:18:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 -- 172.21.15.31:0/936355718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12dc0b5180 msgr2=0x7f12dc0b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12dc0b5180 0x7f12dc0b2bd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f12d80ae040 tx=0x7f12d80191f0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 -- 172.21.15.31:0/936355718 shutdown_connections 2023-11-26T07:18:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f12c00664c0 0x7f12c0068980 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12dc15d2a0 0x7f12dc14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12dc0d2c20 0x7f12dc156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 --2- 172.21.15.31:0/936355718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12dc0b5180 0x7f12dc0b2bd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 -- 172.21.15.31:0/936355718 >> 172.21.15.31:0/936355718 conn(0x7f12dc0668a0 msgr2=0x7f12dc0b6410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 -- 172.21.15.31:0/936355718 shutdown_connections 2023-11-26T07:18:06.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.196+0000 7f12e2638640 1 -- 172.21.15.31:0/936355718 wait complete. 2023-11-26T07:18:06.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b6353e640 1 Processor -- start 2023-11-26T07:18:06.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b6353e640 1 -- start start 2023-11-26T07:18:06.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b6353e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c0ced30 0x7f0b5c0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b6353e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0b5c0cf1b0 0x7f0b5c0d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b6353e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5c0cdfb0 con 0x7f0b5c15d8f0 2023-11-26T07:18:06.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b6353e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5c0d4420 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b6353e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0d4140 con 0x7f0b5c0ced30 2023-11-26T07:18:06.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b612da640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0b5c15d8f0 0x7f0b5c15e780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51978/0 (socket says 172.21.15.31:51978) 2023-11-26T07:18:06.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b612da640 1 -- 172.21.15.31:0/2742311629 learned_addr learned my addr 172.21.15.31:0/2742311629 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b60ad9640 1 --2- 172.21.15.31:0/2742311629 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c0ced30 0x7f0b5c0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b61adb640 1 --2- 172.21.15.31:0/2742311629 >> v2:172.21.15.97:3300/0 conn(0x7f0b5c0cf1b0 0x7f0b5c0d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b53fff640 1 -- 172.21.15.31:0/2742311629 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745993396 0 0) 0x7f0b5c0cdfb0 con 0x7f0b5c15d8f0 2023-11-26T07:18:06.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b53fff640 1 -- 172.21.15.31:0/2742311629 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b48003530 con 0x7f0b5c15d8f0 2023-11-26T07:18:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b53fff640 1 -- 172.21.15.31:0/2742311629 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046676652 0 0) 0x7f0b48003530 con 0x7f0b5c15d8f0 2023-11-26T07:18:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b61adb640 1 -- 172.21.15.31:0/2742311629 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c0ced30 msgr2=0x7f0b5c0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b61adb640 1 --2- 172.21.15.31:0/2742311629 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c0ced30 0x7f0b5c0cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b61adb640 1 -- 172.21.15.31:0/2742311629 >> v1:172.21.15.31:6789/0 conn(0x7f0b5c15d8f0 legacy=0x7f0b5c15e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b61adb640 1 -- 172.21.15.31:0/2742311629 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c0ce250 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.364+0000 7f0b60ad9640 1 --2- 172.21.15.31:0/2742311629 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c0ced30 0x7f0b5c0cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b61adb640 1 --2- 172.21.15.31:0/2742311629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0cf1b0 0x7f0b5c0d0340 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009d90 tx=0x7f0b5401c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceda677d8e1a8772 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b53fff640 1 -- 172.21.15.31:0/2742311629 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58001680 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b53fff640 1 -- 172.21.15.31:0/2742311629 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b540a5640 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b53fff640 1 -- 172.21.15.31:0/2742311629 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b540a59d0 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b6353e640 1 -- 172.21.15.31:0/2742311629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0cf1b0 msgr2=0x7f0b5c0d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b6353e640 1 --2- 172.21.15.31:0/2742311629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0cf1b0 0x7f0b5c0d0340 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009d90 tx=0x7f0b5401c840 comp rx=0 tx=0).stop 2023-11-26T07:18:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b6353e640 1 -- 172.21.15.31:0/2742311629 shutdown_connections 2023-11-26T07:18:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b6353e640 1 --2- 172.21.15.31:0/2742311629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0cf1b0 0x7f0b5c0d0340 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b6353e640 1 --2- 172.21.15.31:0/2742311629 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c0ced30 0x7f0b5c0cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b6353e640 1 -- 172.21.15.31:0/2742311629 >> 172.21.15.31:0/2742311629 conn(0x7f0b5c09c750 msgr2=0x7f0b5c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b6353e640 1 -- 172.21.15.31:0/2742311629 shutdown_connections 2023-11-26T07:18:06.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.368+0000 7f0b6353e640 1 -- 172.21.15.31:0/2742311629 wait complete. 2023-11-26T07:18:06.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b6353e640 1 Processor -- start 2023-11-26T07:18:06.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b6353e640 1 -- start start 2023-11-26T07:18:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b6353e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0ced30 0x7f0b5c0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b6353e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c0cf1b0 0x7f0b5c160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b6353e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c15d8f0 0x7f0b5c160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b6353e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0ca6c0 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b6353e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c06d1d0 con 0x7f0b5c0ced30 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b6353e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0cd540 con 0x7f0b5c15d8f0 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b60ad9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c0cf1b0 0x7f0b5c160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b60ad9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c0cf1b0 0x7f0b5c160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45380/0 (socket says 172.21.15.31:45380) 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b60ad9640 1 -- 172.21.15.31:0/3961161516 learned_addr learned my addr 172.21.15.31:0/3961161516 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b61adb640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c15d8f0 0x7f0b5c160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b612da640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0ced30 0x7f0b5c0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b60ad9640 1 -- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c15d8f0 msgr2=0x7f0b5c160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b60ad9640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c15d8f0 0x7f0b5c160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b60ad9640 1 -- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0ced30 msgr2=0x7f0b5c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b60ad9640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0ced30 0x7f0b5c0ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b60ad9640 1 -- 172.21.15.31:0/3961161516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c009420 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b60ad9640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c0cf1b0 0x7f0b5c160930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00e930 tx=0x7f0b4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b51ffb640 1 -- 172.21.15.31:0/3961161516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c00ca00 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b51ffb640 1 -- 172.21.15.31:0/3961161516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c00cb60 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b51ffb640 1 -- 172.21.15.31:0/3961161516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c013470 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.372+0000 7f0b6353e640 1 -- 172.21.15.31:0/3961161516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b54009a70 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.376+0000 7f0b6353e640 1 -- 172.21.15.31:0/3961161516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c06cef0 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.376+0000 7f0b6353e640 1 -- 172.21.15.31:0/3961161516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5c04e850 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.376+0000 7f0b51ffb640 1 -- 172.21.15.31:0/3961161516 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0b4c0135d0 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.376+0000 7f0b51ffb640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b40066470 0x7f0b40068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.376+0000 7f0b51ffb640 1 -- 172.21.15.31:0/3961161516 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b4c084090 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.376+0000 7f0b612da640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b40066470 0x7f0b40068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.380+0000 7f0b612da640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b40066470 0x7f0b40068930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0b580011f0 tx=0x7f0b58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.380+0000 7f0b51ffb640 1 -- 172.21.15.31:0/3961161516 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0b4c051500 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.552+0000 7f0b6353e640 1 -- 172.21.15.31:0/3961161516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0b5c0cf9d0 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.552+0000 7f0b51ffb640 1 -- 172.21.15.31:0/3961161516 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0b4c054b20 con 0x7f0b5c0cf1b0 2023-11-26T07:18:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:18:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 -- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b40066470 msgr2=0x7f0b40068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b40066470 0x7f0b40068930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0b580011f0 tx=0x7f0b58009040 comp rx=0 tx=0).stop 2023-11-26T07:18:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 -- 172.21.15.31:0/3961161516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c0cf1b0 msgr2=0x7f0b5c160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c0cf1b0 0x7f0b5c160930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00e930 tx=0x7f0b4c00edf0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 -- 172.21.15.31:0/3961161516 shutdown_connections 2023-11-26T07:18:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b40066470 0x7f0b40068930 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b5c15d8f0 0x7f0b5c160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b5c0cf1b0 0x7f0b5c160930 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 --2- 172.21.15.31:0/3961161516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b5c0ced30 0x7f0b5c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 -- 172.21.15.31:0/3961161516 >> 172.21.15.31:0/3961161516 conn(0x7f0b5c09c750 msgr2=0x7f0b5c0b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 -- 172.21.15.31:0/3961161516 shutdown_connections 2023-11-26T07:18:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.556+0000 7f0b6353e640 1 -- 172.21.15.31:0/3961161516 wait complete. 2023-11-26T07:18:06.579 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:18:06.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:06.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-26T07:18:06.618 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:18:06.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:06.619 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:18:06.677 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:06.678 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:06.687 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:06.687 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:06.740 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:06.740 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:06.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:06.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.972+0000 7fe2d0d80640 1 Processor -- start 2023-11-26T07:18:06.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.972+0000 7fe2d0d80640 1 -- start start 2023-11-26T07:18:06.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2d0d80640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc144c80 0x7fe2cc0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2d0d80640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2cc0ac460 0x7fe2cc0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2d0d80640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2cc145060 con 0x7fe2cc070c20 2023-11-26T07:18:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2d0d80640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2cc071fd0 con 0x7fe2cc0ac460 2023-11-26T07:18:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2d0d80640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0ac8c0 con 0x7fe2cc144c80 2023-11-26T07:18:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2c3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2cc0ac460 0x7fe2cc0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2c3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe2cc0ac460 0x7fe2cc0706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45394/0 (socket says 172.21.15.31:45394) 2023-11-26T07:18:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2cad9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe2cc070c20 0x7fe2cc15eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51980/0 (socket says 172.21.15.31:51980) 2023-11-26T07:18:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2c3fff640 1 -- 172.21.15.31:0/3178233856 learned_addr learned my addr 172.21.15.31:0/3178233856 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2ca59c640 1 --2- 172.21.15.31:0/3178233856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc144c80 0x7fe2cc0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2c3fff640 1 -- 172.21.15.31:0/3178233856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc144c80 msgr2=0x7fe2cc0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2c3fff640 1 --2- 172.21.15.31:0/3178233856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc144c80 0x7fe2cc0abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2c3fff640 1 -- 172.21.15.31:0/3178233856 >> v1:172.21.15.31:6789/0 conn(0x7fe2cc070c20 legacy=0x7fe2cc15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2c3fff640 1 -- 172.21.15.31:0/3178233856 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc071450 con 0x7fe2cc0ac460 2023-11-26T07:18:06.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.976+0000 7fe2ca59c640 1 --2- 172.21.15.31:0/3178233856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc144c80 0x7fe2cc0abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:06.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2c3fff640 1 --2- 172.21.15.31:0/3178233856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc0ac460 0x7fe2cc0706e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009870 tx=0x7fe2b401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e304d81519fcf012 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2c9d9b640 1 -- 172.21.15.31:0/3178233856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40aa670 con 0x7fe2cc0ac460 2023-11-26T07:18:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2c9d9b640 1 -- 172.21.15.31:0/3178233856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b40aa7d0 con 0x7fe2cc0ac460 2023-11-26T07:18:06.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2c9d9b640 1 -- 172.21.15.31:0/3178233856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b40aab60 con 0x7fe2cc0ac460 2023-11-26T07:18:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3178233856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc0ac460 msgr2=0x7fe2cc0706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2d0d80640 1 --2- 172.21.15.31:0/3178233856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc0ac460 0x7fe2cc0706e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009870 tx=0x7fe2b401c510 comp rx=0 tx=0).stop 2023-11-26T07:18:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3178233856 shutdown_connections 2023-11-26T07:18:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2d0d80640 1 --2- 172.21.15.31:0/3178233856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc0ac460 0x7fe2cc0706e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2d0d80640 1 --2- 172.21.15.31:0/3178233856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc144c80 0x7fe2cc0abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3178233856 >> 172.21.15.31:0/3178233856 conn(0x7fe2cc0b0c40 msgr2=0x7fe2cc0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3178233856 shutdown_connections 2023-11-26T07:18:06.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3178233856 wait complete. 2023-11-26T07:18:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2d0d80640 1 Processor -- start 2023-11-26T07:18:06.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.980+0000 7fe2d0d80640 1 -- start start 2023-11-26T07:18:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2d0d80640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc070c20 0x7fe2cc15a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2d0d80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2cc0ac460 0x7fe2cc159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2d0d80640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc144c80 0x7fe2cc1653c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2d0d80640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0b36b0 con 0x7fe2cc144c80 2023-11-26T07:18:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2d0d80640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc164fa0 con 0x7fe2cc0ac460 2023-11-26T07:18:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2d0d80640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc1393d0 con 0x7fe2cc070c20 2023-11-26T07:18:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2cad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc144c80 0x7fe2cc1653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2cad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc144c80 0x7fe2cc1653c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45398/0 (socket says 172.21.15.31:45398) 2023-11-26T07:18:06.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2cad9d640 1 -- 172.21.15.31:0/3236745066 learned_addr learned my addr 172.21.15.31:0/3236745066 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:06.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2ca59c640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc070c20 0x7fe2cc15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2c3fff640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2cc0ac460 0x7fe2cc159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2cad9d640 1 -- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc070c20 msgr2=0x7fe2cc15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2cad9d640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc070c20 0x7fe2cc15a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2cad9d640 1 -- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2cc0ac460 msgr2=0x7fe2cc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2cad9d640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2cc0ac460 0x7fe2cc159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2cad9d640 1 -- 172.21.15.31:0/3236745066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b40094d0 con 0x7fe2cc144c80 2023-11-26T07:18:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2cad9d640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc144c80 0x7fe2cc1653c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c7a0 tx=0x7fe2c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2c2ffd640 1 -- 172.21.15.31:0/3236745066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c400a640 con 0x7fe2cc144c80 2023-11-26T07:18:06.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2c2ffd640 1 -- 172.21.15.31:0/3236745066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c400a7a0 con 0x7fe2cc144c80 2023-11-26T07:18:06.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2c2ffd640 1 -- 172.21.15.31:0/3236745066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c400ab10 con 0x7fe2cc144c80 2023-11-26T07:18:06.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3236745066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2cc0b30f0 con 0x7fe2cc144c80 2023-11-26T07:18:06.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3236745066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2cc15af80 con 0x7fe2cc144c80 2023-11-26T07:18:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3236745066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2cc0b1ba0 con 0x7fe2cc144c80 2023-11-26T07:18:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.984+0000 7fe2c2ffd640 1 -- 172.21.15.31:0/3236745066 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fe2c400ac70 con 0x7fe2cc144c80 2023-11-26T07:18:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.988+0000 7fe2c2ffd640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2ac066470 0x7fe2ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.988+0000 7fe2c2ffd640 1 -- 172.21.15.31:0/3236745066 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2c4083630 con 0x7fe2cc144c80 2023-11-26T07:18:06.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.988+0000 7fe2ca59c640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2ac066470 0x7fe2ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:06.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.988+0000 7fe2ca59c640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2ac066470 0x7fe2ac068930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00ba10 tx=0x7fe2bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:06.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:06.992+0000 7fe2c2ffd640 1 -- 172.21.15.31:0/3236745066 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe2c4050ac0 con 0x7fe2cc144c80 2023-11-26T07:18:07.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.148+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3236745066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2cc164d00 con 0x7fe2cc144c80 2023-11-26T07:18:07.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2c2ffd640 1 -- 172.21.15.31:0/3236745066 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7fe2cc164d00 con 0x7fe2cc144c80 2023-11-26T07:18:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:07.157 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2ac066470 msgr2=0x7fe2ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2ac066470 0x7fe2ac068930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc00ba10 tx=0x7fe2bc008040 comp rx=0 tx=0).stop 2023-11-26T07:18:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3236745066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc144c80 msgr2=0x7fe2cc1653c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc144c80 0x7fe2cc1653c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c7a0 tx=0x7fe2c400cc60 comp rx=0 tx=0).stop 2023-11-26T07:18:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3236745066 shutdown_connections 2023-11-26T07:18:07.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe2ac066470 0x7fe2ac068930 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe2cc144c80 0x7fe2cc1653c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe2cc0ac460 0x7fe2cc159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 --2- 172.21.15.31:0/3236745066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe2cc070c20 0x7fe2cc15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3236745066 >> 172.21.15.31:0/3236745066 conn(0x7fe2cc0b0c40 msgr2=0x7fe2cc0add30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3236745066 shutdown_connections 2023-11-26T07:18:07.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.152+0000 7fe2d0d80640 1 -- 172.21.15.31:0/3236745066 wait complete. 2023-11-26T07:18:07.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:07.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.324+0000 7fb40f535640 1 Processor -- start 2023-11-26T07:18:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.324+0000 7fb40f535640 1 -- start start 2023-11-26T07:18:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.324+0000 7fb40f535640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb408146820 0x7fb408070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.324+0000 7fb40f535640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4080b4210 0x7fb4080b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.324+0000 7fb40f535640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb408147190 con 0x7fb408161180 2023-11-26T07:18:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.324+0000 7fb40f535640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4080b4be0 con 0x7fb408146820 2023-11-26T07:18:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.324+0000 7fb40f535640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb408146c00 con 0x7fb4080b4210 2023-11-26T07:18:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40d2d1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb408146820 0x7fb408070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40d2d1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb408146820 0x7fb408070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45406/0 (socket says 172.21.15.31:45406) 2023-11-26T07:18:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40d2d1640 1 -- 172.21.15.31:0/4239679000 learned_addr learned my addr 172.21.15.31:0/4239679000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40dad2640 1 --1- 172.21.15.31:0/4239679000 >> v1:172.21.15.31:6789/0 conn(0x7fb408161180 0x7fb408161590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51990/0 (socket says 172.21.15.31:51990) 2023-11-26T07:18:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40d2d1640 1 -- 172.21.15.31:0/4239679000 >> v2:172.21.15.97:3300/0 conn(0x7fb4080b4210 msgr2=0x7fb4080b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40cad0640 1 --2- 172.21.15.31:0/4239679000 >> v2:172.21.15.97:3300/0 conn(0x7fb4080b4210 0x7fb4080b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40d2d1640 1 --2- 172.21.15.31:0/4239679000 >> v2:172.21.15.97:3300/0 conn(0x7fb4080b4210 0x7fb4080b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb3fffff640 1 -- 172.21.15.31:0/4239679000 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156142905 0 0) 0x7fb408147190 con 0x7fb408161180 2023-11-26T07:18:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40d2d1640 1 -- 172.21.15.31:0/4239679000 >> v1:172.21.15.31:6789/0 conn(0x7fb408161180 legacy=0x7fb408161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40d2d1640 1 -- 172.21.15.31:0/4239679000 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40806d030 con 0x7fb408146820 2023-11-26T07:18:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40cad0640 1 --2- 172.21.15.31:0/4239679000 >> v2:172.21.15.97:3300/0 conn(0x7fb4080b4210 0x7fb4080b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:07.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb40d2d1640 1 --2- 172.21.15.31:0/4239679000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb408146820 0x7fb408070a50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb4080b5ad0 tx=0x7fb40401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28886e3c07d93e38 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb3fffff640 1 -- 172.21.15.31:0/4239679000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040aa430 con 0x7fb408146820 2023-11-26T07:18:07.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb3fffff640 1 -- 172.21.15.31:0/4239679000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4040aa590 con 0x7fb408146820 2023-11-26T07:18:07.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.328+0000 7fb3fffff640 1 -- 172.21.15.31:0/4239679000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040aa920 con 0x7fb408146820 2023-11-26T07:18:07.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 -- 172.21.15.31:0/4239679000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb408146820 msgr2=0x7fb408070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 --2- 172.21.15.31:0/4239679000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb408146820 0x7fb408070a50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb4080b5ad0 tx=0x7fb40401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 -- 172.21.15.31:0/4239679000 shutdown_connections 2023-11-26T07:18:07.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 --2- 172.21.15.31:0/4239679000 >> v2:172.21.15.97:3300/0 conn(0x7fb4080b4210 0x7fb4080b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 --2- 172.21.15.31:0/4239679000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb408146820 0x7fb408070a50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 -- 172.21.15.31:0/4239679000 >> 172.21.15.31:0/4239679000 conn(0x7fb4080b2f10 msgr2=0x7fb4080b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 -- 172.21.15.31:0/4239679000 shutdown_connections 2023-11-26T07:18:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 -- 172.21.15.31:0/4239679000 wait complete. 2023-11-26T07:18:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 Processor -- start 2023-11-26T07:18:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 -- start start 2023-11-26T07:18:07.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4080b4210 0x7fb40813d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb408146820 0x7fb4081620b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb408161180 0x7fb4081625f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb408162b30 con 0x7fb4080b4210 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40815cb70 con 0x7fb408161180 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40f535640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40815ce50 con 0x7fb408146820 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40d2d1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4080b4210 0x7fb40813d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40d2d1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4080b4210 0x7fb40813d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45408/0 (socket says 172.21.15.31:45408) 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40d2d1640 1 -- 172.21.15.31:0/997369684 learned_addr learned my addr 172.21.15.31:0/997369684 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40dad2640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb408161180 0x7fb4081625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40d2d1640 1 -- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb408146820 msgr2=0x7fb4081620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40cad0640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb408146820 0x7fb4081620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40d2d1640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb408146820 0x7fb4081620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40d2d1640 1 -- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb408161180 msgr2=0x7fb4081625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40d2d1640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb408161180 0x7fb4081625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40d2d1640 1 -- 172.21.15.31:0/997369684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4040094d0 con 0x7fb4080b4210 2023-11-26T07:18:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40dad2640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb408161180 0x7fb4081625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.332+0000 7fb40cad0640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb408146820 0x7fb4081620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb40d2d1640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4080b4210 0x7fb40813d9a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb4040190e0 tx=0x7fb40401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb3fdffb640 1 -- 172.21.15.31:0/997369684 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040b38c0 con 0x7fb4080b4210 2023-11-26T07:18:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb3fdffb640 1 -- 172.21.15.31:0/997369684 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4040b3a20 con 0x7fb4080b4210 2023-11-26T07:18:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb40f535640 1 -- 172.21.15.31:0/997369684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40815d130 con 0x7fb4080b4210 2023-11-26T07:18:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb3fdffb640 1 -- 172.21.15.31:0/997369684 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040b3dd0 con 0x7fb4080b4210 2023-11-26T07:18:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb40f535640 1 -- 172.21.15.31:0/997369684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4081f6360 con 0x7fb4080b4210 2023-11-26T07:18:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb40f535640 1 -- 172.21.15.31:0/997369684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4081f6550 con 0x7fb4080b4210 2023-11-26T07:18:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb3fdffb640 1 -- 172.21.15.31:0/997369684 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb4040b2380 con 0x7fb4080b4210 2023-11-26T07:18:07.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb3fdffb640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb3ec0664a0 0x7fb3ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb3fdffb640 1 -- 172.21.15.31:0/997369684 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb40411e490 con 0x7fb4080b4210 2023-11-26T07:18:07.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.336+0000 7fb40cad0640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb3ec0664a0 0x7fb3ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.340+0000 7fb40cad0640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb3ec0664a0 0x7fb3ec068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4080b1ca0 tx=0x7fb3f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.340+0000 7fb3fdffb640 1 -- 172.21.15.31:0/997369684 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4040eba50 con 0x7fb4080b4210 2023-11-26T07:18:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.508+0000 7fb40f535640 1 -- 172.21.15.31:0/997369684 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb40804e850 con 0x7fb4080b4210 2023-11-26T07:18:07.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.508+0000 7fb3fdffb640 1 -- 172.21.15.31:0/997369684 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4040ef070 con 0x7fb4080b4210 2023-11-26T07:18:07.515 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 -- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb3ec0664a0 msgr2=0x7fb3ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb3ec0664a0 0x7fb3ec068960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4080b1ca0 tx=0x7fb3f8008040 comp rx=0 tx=0).stop 2023-11-26T07:18:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 -- 172.21.15.31:0/997369684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4080b4210 msgr2=0x7fb40813d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4080b4210 0x7fb40813d9a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb4040190e0 tx=0x7fb40401ad10 comp rx=0 tx=0).stop 2023-11-26T07:18:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 -- 172.21.15.31:0/997369684 shutdown_connections 2023-11-26T07:18:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb3ec0664a0 0x7fb3ec068960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb408161180 0x7fb4081625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb408146820 0x7fb4081620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 --2- 172.21.15.31:0/997369684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4080b4210 0x7fb40813d9a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 -- 172.21.15.31:0/997369684 >> 172.21.15.31:0/997369684 conn(0x7fb4080b2f10 msgr2=0x7fb40813f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 -- 172.21.15.31:0/997369684 shutdown_connections 2023-11-26T07:18:07.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.512+0000 7fb40f535640 1 -- 172.21.15.31:0/997369684 wait complete. 2023-11-26T07:18:07.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:07.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d59bdf640 1 Processor -- start 2023-11-26T07:18:07.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d59bdf640 1 -- start start 2023-11-26T07:18:07.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d59bdf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d5415a710 0x7f8d540abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d59bdf640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540b2710 con 0x7f8d540cdfb0 2023-11-26T07:18:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d59bdf640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540b29f0 con 0x7f8d540cbd50 2023-11-26T07:18:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d59bdf640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540cca80 con 0x7f8d5415a710 2023-11-26T07:18:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d537fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8d540cdfb0 0x7f8d540ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52006/0 (socket says 172.21.15.31:52006) 2023-11-26T07:18:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d537fe640 1 -- 172.21.15.31:0/3403888159 learned_addr learned my addr 172.21.15.31:0/3403888159 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d52ffd640 1 --2- 172.21.15.31:0/3403888159 >> v2:172.21.15.97:3300/0 conn(0x7f8d5415a710 0x7f8d540abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d527fc640 1 -- 172.21.15.31:0/3403888159 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273360775 0 0) 0x7f8d540b2710 con 0x7f8d540cdfb0 2023-11-26T07:18:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.676+0000 7f8d527fc640 1 -- 172.21.15.31:0/3403888159 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d3c003530 con 0x7f8d540cdfb0 2023-11-26T07:18:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d52ffd640 1 -- 172.21.15.31:0/3403888159 >> v1:172.21.15.97:6789/0 conn(0x7f8d540cbd50 legacy=0x7f8d540cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d52ffd640 1 -- 172.21.15.31:0/3403888159 >> v1:172.21.15.31:6789/0 conn(0x7f8d540cdfb0 legacy=0x7f8d540ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d52ffd640 1 -- 172.21.15.31:0/3403888159 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540b2cd0 con 0x7f8d5415a710 2023-11-26T07:18:07.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d52ffd640 1 --2- 172.21.15.31:0/3403888159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5415a710 0x7f8d540abf90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d480097c0 tx=0x7f8d4801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1635cbfe72f08b80 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d527fc640 1 -- 172.21.15.31:0/3403888159 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d480ab670 con 0x7f8d5415a710 2023-11-26T07:18:07.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d527fc640 1 -- 172.21.15.31:0/3403888159 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d480ab7d0 con 0x7f8d5415a710 2023-11-26T07:18:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d527fc640 1 -- 172.21.15.31:0/3403888159 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d480abb60 con 0x7f8d5415a710 2023-11-26T07:18:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d59bdf640 1 -- 172.21.15.31:0/3403888159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5415a710 msgr2=0x7f8d540abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d59bdf640 1 --2- 172.21.15.31:0/3403888159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5415a710 0x7f8d540abf90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d480097c0 tx=0x7f8d4801c450 comp rx=0 tx=0).stop 2023-11-26T07:18:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d59bdf640 1 -- 172.21.15.31:0/3403888159 shutdown_connections 2023-11-26T07:18:07.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d59bdf640 1 --2- 172.21.15.31:0/3403888159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d5415a710 0x7f8d540abf90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d59bdf640 1 -- 172.21.15.31:0/3403888159 >> 172.21.15.31:0/3403888159 conn(0x7f8d540b0cb0 msgr2=0x7f8d540b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.680+0000 7f8d59bdf640 1 -- 172.21.15.31:0/3403888159 shutdown_connections 2023-11-26T07:18:07.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d59bdf640 1 -- 172.21.15.31:0/3403888159 wait complete. 2023-11-26T07:18:07.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d59bdf640 1 Processor -- start 2023-11-26T07:18:07.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d59bdf640 1 -- start start 2023-11-26T07:18:07.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d59bdf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d540cbd50 0x7f8d5414bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d59bdf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d540cdfb0 0x7f8d541645f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d59bdf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d5415a710 0x7f8d54164b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d59bdf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5409b270 con 0x7f8d540cdfb0 2023-11-26T07:18:07.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d59bdf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5415c780 con 0x7f8d540cbd50 2023-11-26T07:18:07.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d59bdf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5414c0b0 con 0x7f8d5415a710 2023-11-26T07:18:07.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d52ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d540cdfb0 0x7f8d541645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d52ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d540cdfb0 0x7f8d541645f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45422/0 (socket says 172.21.15.31:45422) 2023-11-26T07:18:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d52ffd640 1 -- 172.21.15.31:0/4056618657 learned_addr learned my addr 172.21.15.31:0/4056618657 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d537fe640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d540cbd50 0x7f8d5414bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d53fff640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d5415a710 0x7f8d54164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d52ffd640 1 -- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d5415a710 msgr2=0x7f8d54164b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d52ffd640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d5415a710 0x7f8d54164b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d52ffd640 1 -- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d540cbd50 msgr2=0x7f8d5414bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d52ffd640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d540cbd50 0x7f8d5414bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d52ffd640 1 -- 172.21.15.31:0/4056618657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48009420 con 0x7f8d540cdfb0 2023-11-26T07:18:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.684+0000 7f8d52ffd640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d540cdfb0 0x7f8d541645f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8d4801a080 tx=0x7f8d4801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.688+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4056618657 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d480b5810 con 0x7f8d540cdfb0 2023-11-26T07:18:07.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.688+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4056618657 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d480b5990 con 0x7f8d540cdfb0 2023-11-26T07:18:07.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.688+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4056618657 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d480b5d40 con 0x7f8d540cdfb0 2023-11-26T07:18:07.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.688+0000 7f8d59bdf640 1 -- 172.21.15.31:0/4056618657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5409a660 con 0x7f8d540cdfb0 2023-11-26T07:18:07.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.688+0000 7f8d59bdf640 1 -- 172.21.15.31:0/4056618657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5409ac20 con 0x7f8d540cdfb0 2023-11-26T07:18:07.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.688+0000 7f8d59bdf640 1 -- 172.21.15.31:0/4056618657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d540671f0 con 0x7f8d540cdfb0 2023-11-26T07:18:07.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.688+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4056618657 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8d480b4390 con 0x7f8d540cdfb0 2023-11-26T07:18:07.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.688+0000 7f8d3bfff640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d40066470 0x7f8d40068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:07.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.688+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4056618657 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d4811e910 con 0x7f8d540cdfb0 2023-11-26T07:18:07.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.688+0000 7f8d537fe640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d40066470 0x7f8d40068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:07.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.692+0000 7f8d537fe640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d40066470 0x7f8d40068930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d440011f0 tx=0x7f8d44007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.692+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4056618657 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8d480ebe40 con 0x7f8d540cdfb0 2023-11-26T07:18:07.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.852+0000 7f8d59bdf640 1 -- 172.21.15.31:0/4056618657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d5406b460 con 0x7f8d540cdfb0 2023-11-26T07:18:07.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d3bfff640 1 -- 172.21.15.31:0/4056618657 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d480ef460 con 0x7f8d540cdfb0 2023-11-26T07:18:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:18:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 -- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d40066470 msgr2=0x7f8d40068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d40066470 0x7f8d40068930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8d440011f0 tx=0x7f8d44007d40 comp rx=0 tx=0).stop 2023-11-26T07:18:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 -- 172.21.15.31:0/4056618657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d540cdfb0 msgr2=0x7f8d541645f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d540cdfb0 0x7f8d541645f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8d4801a080 tx=0x7f8d4801ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 -- 172.21.15.31:0/4056618657 shutdown_connections 2023-11-26T07:18:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d40066470 0x7f8d40068930 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8d5415a710 0x7f8d54164b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d540cdfb0 0x7f8d541645f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 --2- 172.21.15.31:0/4056618657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d540cbd50 0x7f8d5414bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 -- 172.21.15.31:0/4056618657 >> 172.21.15.31:0/4056618657 conn(0x7f8d540b0cb0 msgr2=0x7f8d540b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 -- 172.21.15.31:0/4056618657 shutdown_connections 2023-11-26T07:18:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:07.856+0000 7f8d59bdf640 1 -- 172.21.15.31:0/4056618657 wait complete. 2023-11-26T07:18:07.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:08.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51fd7aa640 1 Processor -- start 2023-11-26T07:18:08.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51fd7aa640 1 -- start start 2023-11-26T07:18:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51fd7aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51f815a710 0x7f51f80abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51fd7aa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f51f80cbd50 0x7f51f80cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51fd7aa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f80b2710 con 0x7f51f80cdfb0 2023-11-26T07:18:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51fd7aa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f80cca80 con 0x7f51f80cbd50 2023-11-26T07:18:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51fd7aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f80b29f0 con 0x7f51f815a710 2023-11-26T07:18:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51f6ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f51f80cdfb0 0x7f51f80ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52022/0 (socket says 172.21.15.31:52022) 2023-11-26T07:18:08.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51f6ffd640 1 -- 172.21.15.31:0/2328886963 learned_addr learned my addr 172.21.15.31:0/2328886963 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51effff640 1 --2- 172.21.15.31:0/2328886963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51f815a710 0x7f51f80abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51f77fe640 1 --2- 172.21.15.31:0/2328886963 >> v2:172.21.15.97:3300/0 conn(0x7f51f80cbd50 0x7f51f80cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51f67fc640 1 -- 172.21.15.31:0/2328886963 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503871444 0 0) 0x7f51f80b2710 con 0x7f51f80cdfb0 2023-11-26T07:18:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51f67fc640 1 -- 172.21.15.31:0/2328886963 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e4005740 con 0x7f51f80cdfb0 2023-11-26T07:18:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51f77fe640 1 -- 172.21.15.31:0/2328886963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51f815a710 msgr2=0x7f51f80abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51f77fe640 1 --2- 172.21.15.31:0/2328886963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51f815a710 0x7f51f80abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51f77fe640 1 -- 172.21.15.31:0/2328886963 >> v1:172.21.15.31:6789/0 conn(0x7f51f80cdfb0 legacy=0x7f51f80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.024+0000 7f51f77fe640 1 -- 172.21.15.31:0/2328886963 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f80b2cd0 con 0x7f51f80cbd50 2023-11-26T07:18:08.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51f77fe640 1 --2- 172.21.15.31:0/2328886963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51f80cbd50 0x7f51f80cc1e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009d90 tx=0x7f51f001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c1b6a99e793c2b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51f67fc640 1 -- 172.21.15.31:0/2328886963 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00a5a00 con 0x7f51f80cbd50 2023-11-26T07:18:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51f67fc640 1 -- 172.21.15.31:0/2328886963 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f00a5b60 con 0x7f51f80cbd50 2023-11-26T07:18:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51f67fc640 1 -- 172.21.15.31:0/2328886963 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00ad440 con 0x7f51f80cbd50 2023-11-26T07:18:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51fd7aa640 1 -- 172.21.15.31:0/2328886963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51f80cbd50 msgr2=0x7f51f80cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51fd7aa640 1 --2- 172.21.15.31:0/2328886963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51f80cbd50 0x7f51f80cc1e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009d90 tx=0x7f51f001c920 comp rx=0 tx=0).stop 2023-11-26T07:18:08.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51fd7aa640 1 -- 172.21.15.31:0/2328886963 shutdown_connections 2023-11-26T07:18:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51fd7aa640 1 --2- 172.21.15.31:0/2328886963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51f80cbd50 0x7f51f80cc1e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51fd7aa640 1 --2- 172.21.15.31:0/2328886963 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51f815a710 0x7f51f80abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51fd7aa640 1 -- 172.21.15.31:0/2328886963 >> 172.21.15.31:0/2328886963 conn(0x7f51f80b0cb0 msgr2=0x7f51f80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51fd7aa640 1 -- 172.21.15.31:0/2328886963 shutdown_connections 2023-11-26T07:18:08.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.028+0000 7f51fd7aa640 1 -- 172.21.15.31:0/2328886963 wait complete. 2023-11-26T07:18:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 Processor -- start 2023-11-26T07:18:08.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 -- start start 2023-11-26T07:18:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51f80cbd50 0x7f51f81f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51f80cdfb0 0x7f51f8152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f815a710 0x7f51f80ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f81f5c70 con 0x7f51f815a710 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8152f20 con 0x7f51f80cbd50 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8153200 con 0x7f51f80cdfb0 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51f77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f815a710 0x7f51f80ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51f77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f815a710 0x7f51f80ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45436/0 (socket says 172.21.15.31:45436) 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51f77fe640 1 -- 172.21.15.31:0/1598136229 learned_addr learned my addr 172.21.15.31:0/1598136229 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51f6ffd640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51f80cbd50 0x7f51f81f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51effff640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51f80cdfb0 0x7f51f8152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51f77fe640 1 -- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51f80cdfb0 msgr2=0x7f51f8152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51f77fe640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51f80cdfb0 0x7f51f8152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51f77fe640 1 -- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51f80cbd50 msgr2=0x7f51f81f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51f77fe640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51f80cbd50 0x7f51f81f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51f77fe640 1 -- 172.21.15.31:0/1598136229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0009a70 con 0x7f51f815a710 2023-11-26T07:18:08.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51f77fe640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f815a710 0x7f51f80ac510 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f51f001a550 tx=0x7f51f00196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51ef7fe640 1 -- 172.21.15.31:0/1598136229 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00aed30 con 0x7f51f815a710 2023-11-26T07:18:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51ef7fe640 1 -- 172.21.15.31:0/1598136229 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f00aee90 con 0x7f51f815a710 2023-11-26T07:18:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51ef7fe640 1 -- 172.21.15.31:0/1598136229 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00b7610 con 0x7f51f815a710 2023-11-26T07:18:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 -- 172.21.15.31:0/1598136229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f8154ac0 con 0x7f51f815a710 2023-11-26T07:18:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 -- 172.21.15.31:0/1598136229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f81534e0 con 0x7f51f815a710 2023-11-26T07:18:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.032+0000 7f51fd7aa640 1 -- 172.21.15.31:0/1598136229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f80671f0 con 0x7f51f815a710 2023-11-26T07:18:08.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.036+0000 7f51ef7fe640 1 -- 172.21.15.31:0/1598136229 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f51f00b4050 con 0x7f51f815a710 2023-11-26T07:18:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.036+0000 7f51ef7fe640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f51d8066470 0x7f51d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.036+0000 7f51ef7fe640 1 -- 172.21.15.31:0/1598136229 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51f011eea0 con 0x7f51f815a710 2023-11-26T07:18:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.036+0000 7f51f6ffd640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f51d8066470 0x7f51d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.036+0000 7f51f6ffd640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f51d8066470 0x7f51d8068930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51e8007fd0 tx=0x7f51e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.040+0000 7f51ef7fe640 1 -- 172.21.15.31:0/1598136229 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f51f00ec450 con 0x7f51f815a710 2023-11-26T07:18:08.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.196+0000 7f51fd7aa640 1 -- 172.21.15.31:0/1598136229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51f8154fe0 con 0x7f51f815a710 2023-11-26T07:18:08.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.196+0000 7f51ef7fe640 1 -- 172.21.15.31:0/1598136229 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51f8154fe0 con 0x7f51f815a710 2023-11-26T07:18:08.202 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:18:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 -- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f51d8066470 msgr2=0x7f51d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f51d8066470 0x7f51d8068930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f51e8007fd0 tx=0x7f51e8009040 comp rx=0 tx=0).stop 2023-11-26T07:18:08.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 -- 172.21.15.31:0/1598136229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f815a710 msgr2=0x7f51f80ac510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f815a710 0x7f51f80ac510 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f51f001a550 tx=0x7f51f00196c0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 -- 172.21.15.31:0/1598136229 shutdown_connections 2023-11-26T07:18:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f51d8066470 0x7f51d8068930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51f815a710 0x7f51f80ac510 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51f80cdfb0 0x7f51f8152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 --2- 172.21.15.31:0/1598136229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51f80cbd50 0x7f51f81f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 -- 172.21.15.31:0/1598136229 >> 172.21.15.31:0/1598136229 conn(0x7f51f80b0cb0 msgr2=0x7f51f80b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 -- 172.21.15.31:0/1598136229 shutdown_connections 2023-11-26T07:18:08.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.200+0000 7f51fd7aa640 1 -- 172.21.15.31:0/1598136229 wait complete. 2023-11-26T07:18:08.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:08.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.364+0000 7fb773b2a640 1 Processor -- start 2023-11-26T07:18:08.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.364+0000 7fb773b2a640 1 -- start start 2023-11-26T07:18:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.364+0000 7fb773b2a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb76c0cdfb0 0x7fb76c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.364+0000 7fb773b2a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76c0b2710 con 0x7fb76c15a710 2023-11-26T07:18:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.364+0000 7fb773b2a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76c0b29f0 con 0x7fb76c0cbd50 2023-11-26T07:18:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.364+0000 7fb773b2a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76c0cca80 con 0x7fb76c0cdfb0 2023-11-26T07:18:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.364+0000 7fb7718c6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb76c0cdfb0 0x7fb76c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.364+0000 7fb7710c5640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb76c15a710 0x7fb76c0ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52026/0 (socket says 172.21.15.31:52026) 2023-11-26T07:18:08.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.364+0000 7fb7710c5640 1 -- 172.21.15.31:0/1584394663 learned_addr learned my addr 172.21.15.31:0/1584394663 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:08.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.368+0000 7fb7718c6640 1 -- 172.21.15.31:0/1584394663 >> v1:172.21.15.97:6789/0 conn(0x7fb76c0cbd50 legacy=0x7fb76c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.368+0000 7fb7718c6640 1 -- 172.21.15.31:0/1584394663 >> v1:172.21.15.31:6789/0 conn(0x7fb76c15a710 legacy=0x7fb76c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.368+0000 7fb7718c6640 1 -- 172.21.15.31:0/1584394663 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c0b2cd0 con 0x7fb76c0cdfb0 2023-11-26T07:18:08.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.368+0000 7fb7708c4640 1 -- 172.21.15.31:0/1584394663 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302463917 0 0) 0x7fb76c0b2710 con 0x7fb76c15a710 2023-11-26T07:18:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.368+0000 7fb7718c6640 1 --2- 172.21.15.31:0/1584394663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cdfb0 0x7fb76c0abf90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb7680097c0 tx=0x7fb76801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72061f3617bf64d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.368+0000 7fb7708c4640 1 -- 172.21.15.31:0/1584394663 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680ab670 con 0x7fb76c0cdfb0 2023-11-26T07:18:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.368+0000 7fb7708c4640 1 -- 172.21.15.31:0/1584394663 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7680ab7d0 con 0x7fb76c0cdfb0 2023-11-26T07:18:08.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.368+0000 7fb7708c4640 1 -- 172.21.15.31:0/1584394663 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680abb60 con 0x7fb76c0cdfb0 2023-11-26T07:18:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 -- 172.21.15.31:0/1584394663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cdfb0 msgr2=0x7fb76c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 --2- 172.21.15.31:0/1584394663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cdfb0 0x7fb76c0abf90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb7680097c0 tx=0x7fb76801c450 comp rx=0 tx=0).stop 2023-11-26T07:18:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 -- 172.21.15.31:0/1584394663 shutdown_connections 2023-11-26T07:18:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 --2- 172.21.15.31:0/1584394663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cdfb0 0x7fb76c0abf90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 -- 172.21.15.31:0/1584394663 >> 172.21.15.31:0/1584394663 conn(0x7fb76c0b0cb0 msgr2=0x7fb76c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 -- 172.21.15.31:0/1584394663 shutdown_connections 2023-11-26T07:18:08.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 -- 172.21.15.31:0/1584394663 wait complete. 2023-11-26T07:18:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 Processor -- start 2023-11-26T07:18:08.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 -- start start 2023-11-26T07:18:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cbd50 0x7fb76c0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb76c0cdfb0 0x7fb76c1636c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb76c15a710 0x7fb76c14c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c1640a0 con 0x7fb76c0cbd50 2023-11-26T07:18:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c164380 con 0x7fb76c15a710 2023-11-26T07:18:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb773b2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c164660 con 0x7fb76c0cdfb0 2023-11-26T07:18:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb7718c6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cbd50 0x7fb76c0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb7718c6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cbd50 0x7fb76c0b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45450/0 (socket says 172.21.15.31:45450) 2023-11-26T07:18:08.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb7718c6640 1 -- 172.21.15.31:0/615413326 learned_addr learned my addr 172.21.15.31:0/615413326 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:08.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb7720c7640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb76c15a710 0x7fb76c14c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb7710c5640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb76c0cdfb0 0x7fb76c1636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb7718c6640 1 -- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb76c0cdfb0 msgr2=0x7fb76c1636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb7718c6640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb76c0cdfb0 0x7fb76c1636c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb7718c6640 1 -- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb76c15a710 msgr2=0x7fb76c14c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb7718c6640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb76c15a710 0x7fb76c14c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.372+0000 7fb7718c6640 1 -- 172.21.15.31:0/615413326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb768009420 con 0x7fb76c0cbd50 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.376+0000 7fb7718c6640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cbd50 0x7fb76c0b0130 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb7680ad040 tx=0x7fb76801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.376+0000 7fb7627fc640 1 -- 172.21.15.31:0/615413326 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768004220 con 0x7fb76c0cbd50 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.376+0000 7fb7627fc640 1 -- 172.21.15.31:0/615413326 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7680b9070 con 0x7fb76c0cbd50 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.376+0000 7fb7627fc640 1 -- 172.21.15.31:0/615413326 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768003620 con 0x7fb76c0cbd50 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.376+0000 7fb773b2a640 1 -- 172.21.15.31:0/615413326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76c164940 con 0x7fb76c0cbd50 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.376+0000 7fb773b2a640 1 -- 172.21.15.31:0/615413326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76c0b0670 con 0x7fb76c0cbd50 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.376+0000 7fb773b2a640 1 -- 172.21.15.31:0/615413326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76c0671f0 con 0x7fb76c0cbd50 2023-11-26T07:18:08.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.376+0000 7fb7627fc640 1 -- 172.21.15.31:0/615413326 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb768003780 con 0x7fb76c0cbd50 2023-11-26T07:18:08.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.376+0000 7fb7627fc640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7500664a0 0x7fb750068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.380+0000 7fb7627fc640 1 -- 172.21.15.31:0/615413326 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb76811e8c0 con 0x7fb76c0cbd50 2023-11-26T07:18:08.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.380+0000 7fb7710c5640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7500664a0 0x7fb750068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.380+0000 7fb7710c5640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7500664a0 0x7fb750068960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb76c1f9760 tx=0x7fb75c007240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.380+0000 7fb7627fc640 1 -- 172.21.15.31:0/615413326 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb7680ebde0 con 0x7fb76c0cbd50 2023-11-26T07:18:08.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.552+0000 7fb773b2a640 1 -- 172.21.15.31:0/615413326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb76c1647a0 con 0x7fb76c0cbd50 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.552+0000 7fb7627fc640 1 -- 172.21.15.31:0/615413326 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb76c1647a0 con 0x7fb76c0cbd50 2023-11-26T07:18:08.558 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 -- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7500664a0 msgr2=0x7fb750068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7500664a0 0x7fb750068960 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb76c1f9760 tx=0x7fb75c007240 comp rx=0 tx=0).stop 2023-11-26T07:18:08.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 -- 172.21.15.31:0/615413326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cbd50 msgr2=0x7fb76c0b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cbd50 0x7fb76c0b0130 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb7680ad040 tx=0x7fb76801ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 -- 172.21.15.31:0/615413326 shutdown_connections 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7500664a0 0x7fb750068960 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb76c15a710 0x7fb76c14c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb76c0cdfb0 0x7fb76c1636c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 --2- 172.21.15.31:0/615413326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb76c0cbd50 0x7fb76c0b0130 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 -- 172.21.15.31:0/615413326 >> 172.21.15.31:0/615413326 conn(0x7fb76c0b0cb0 msgr2=0x7fb76c15ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 -- 172.21.15.31:0/615413326 shutdown_connections 2023-11-26T07:18:08.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.556+0000 7fb773b2a640 1 -- 172.21.15.31:0/615413326 wait complete. 2023-11-26T07:18:08.579 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-26T07:18:08.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:08.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-26T07:18:08.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:18:08.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:08.604 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-26T07:18:08.656 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:08.656 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:08.666 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:08.666 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:08.716 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:08.716 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:08.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:08.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f0906e91640 1 Processor -- start 2023-11-26T07:18:08.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f0906e91640 1 -- start start 2023-11-26T07:18:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f0906e91640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f090015a760 0x7f09000715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f0906e91640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015ac10 0x7f0900071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f0906e91640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090015b750 con 0x7f090013d4c0 2023-11-26T07:18:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f0906e91640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090015b190 con 0x7f090015a760 2023-11-26T07:18:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f0906e91640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090015b470 con 0x7f090015ac10 2023-11-26T07:18:08.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f08fffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f090015a760 0x7f09000715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f0904c2d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f090013d4c0 0x7f090015e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52030/0 (socket says 172.21.15.31:52030) 2023-11-26T07:18:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f08fffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f090015a760 0x7f09000715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45454/0 (socket says 172.21.15.31:45454) 2023-11-26T07:18:08.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.904+0000 7f0904c2d640 1 -- 172.21.15.31:0/659892206 learned_addr learned my addr 172.21.15.31:0/659892206 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f090542e640 1 --2- 172.21.15.31:0/659892206 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015ac10 0x7f0900071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f08fffff640 1 -- 172.21.15.31:0/659892206 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015ac10 msgr2=0x7f0900071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f08fffff640 1 --2- 172.21.15.31:0/659892206 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015ac10 0x7f0900071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f08fffff640 1 -- 172.21.15.31:0/659892206 >> v1:172.21.15.31:6789/0 conn(0x7f090013d4c0 legacy=0x7f090015e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f08fffff640 1 -- 172.21.15.31:0/659892206 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090015de80 con 0x7f090015a760 2023-11-26T07:18:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f090542e640 1 --2- 172.21.15.31:0/659892206 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015ac10 0x7f0900071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:08.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f08ff7fe640 1 -- 172.21.15.31:0/659892206 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264197366 0 0) 0x7f090015b750 con 0x7f090013d4c0 2023-11-26T07:18:08.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f08fffff640 1 --2- 172.21.15.31:0/659892206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090015a760 0x7f09000715b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0097c0 tx=0x7f08ec01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b76d511bf9c614 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f08ff7fe640 1 -- 172.21.15.31:0/659892206 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0ad680 con 0x7f090015a760 2023-11-26T07:18:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f08ff7fe640 1 -- 172.21.15.31:0/659892206 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec0ad7e0 con 0x7f090015a760 2023-11-26T07:18:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f08ff7fe640 1 -- 172.21.15.31:0/659892206 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0adb70 con 0x7f090015a760 2023-11-26T07:18:08.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f0906e91640 1 -- 172.21.15.31:0/659892206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090015a760 msgr2=0x7f09000715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f0906e91640 1 --2- 172.21.15.31:0/659892206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090015a760 0x7f09000715b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0097c0 tx=0x7f08ec01c360 comp rx=0 tx=0).stop 2023-11-26T07:18:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f0906e91640 1 -- 172.21.15.31:0/659892206 shutdown_connections 2023-11-26T07:18:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f0906e91640 1 --2- 172.21.15.31:0/659892206 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015ac10 0x7f0900071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f0906e91640 1 --2- 172.21.15.31:0/659892206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090015a760 0x7f09000715b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f0906e91640 1 -- 172.21.15.31:0/659892206 >> 172.21.15.31:0/659892206 conn(0x7f09000b1e40 msgr2=0x7f0900070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f0906e91640 1 -- 172.21.15.31:0/659892206 shutdown_connections 2023-11-26T07:18:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.908+0000 7f0906e91640 1 -- 172.21.15.31:0/659892206 wait complete. 2023-11-26T07:18:08.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0906e91640 1 Processor -- start 2023-11-26T07:18:08.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0906e91640 1 -- start start 2023-11-26T07:18:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0906e91640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090013d4c0 0x7f0900130090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0906e91640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015a760 0x7f090006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0906e91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f090015ac10 0x7f0900156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0906e91640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090015d900 con 0x7f090013d4c0 2023-11-26T07:18:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0906e91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09001305d0 con 0x7f090015ac10 2023-11-26T07:18:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0906e91640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090015cb60 con 0x7f090015a760 2023-11-26T07:18:08.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0904c2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090013d4c0 0x7f0900130090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0904c2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090013d4c0 0x7f0900130090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45470/0 (socket says 172.21.15.31:45470) 2023-11-26T07:18:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0904c2d640 1 -- 172.21.15.31:0/2899418029 learned_addr learned my addr 172.21.15.31:0/2899418029 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f08fffff640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015a760 0x7f090006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f090542e640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f090015ac10 0x7f0900156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0904c2d640 1 -- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015a760 msgr2=0x7f090006e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0904c2d640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015a760 0x7f090006e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0904c2d640 1 -- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f090015ac10 msgr2=0x7f0900156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0904c2d640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f090015ac10 0x7f0900156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:08.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0904c2d640 1 -- 172.21.15.31:0/2899418029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec009420 con 0x7f090013d4c0 2023-11-26T07:18:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f090542e640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f090015ac10 0x7f0900156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f08fffff640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015a760 0x7f090006e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f0904c2d640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090013d4c0 0x7f0900130090 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f08f40011f0 tx=0x7f08f400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f08fd7fa640 1 -- 172.21.15.31:0/2899418029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f40074e0 con 0x7f090013d4c0 2023-11-26T07:18:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f08fd7fa640 1 -- 172.21.15.31:0/2899418029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f4007640 con 0x7f090013d4c0 2023-11-26T07:18:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.912+0000 7f08fd7fa640 1 -- 172.21.15.31:0/2899418029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4004a10 con 0x7f090013d4c0 2023-11-26T07:18:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.916+0000 7f0906e91640 1 -- 172.21.15.31:0/2899418029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090015ce40 con 0x7f090013d4c0 2023-11-26T07:18:08.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.916+0000 7f0906e91640 1 -- 172.21.15.31:0/2899418029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090015d400 con 0x7f090013d4c0 2023-11-26T07:18:08.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.916+0000 7f0906e91640 1 -- 172.21.15.31:0/2899418029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090004e850 con 0x7f090013d4c0 2023-11-26T07:18:08.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.916+0000 7f08fd7fa640 1 -- 172.21.15.31:0/2899418029 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f08f4004b70 con 0x7f090013d4c0 2023-11-26T07:18:08.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.916+0000 7f08fd7fa640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08e4066470 0x7f08e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:08.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.916+0000 7f08fd7fa640 1 -- 172.21.15.31:0/2899418029 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08f4083170 con 0x7f090013d4c0 2023-11-26T07:18:08.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.916+0000 7f08fffff640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08e4066470 0x7f08e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:08.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.920+0000 7f08fffff640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08e4066470 0x7f08e4068930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0af000 tx=0x7f08ec0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:08.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:08.920+0000 7f08fd7fa640 1 -- 172.21.15.31:0/2899418029 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f08f40505e0 con 0x7f090013d4c0 2023-11-26T07:18:09.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.076+0000 7f0906e91640 1 -- 172.21.15.31:0/2899418029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0900066ae0 con 0x7f090013d4c0 2023-11-26T07:18:09.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f08fd7fa640 1 -- 172.21.15.31:0/2899418029 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f0900066ae0 con 0x7f090013d4c0 2023-11-26T07:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:09.085 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:09.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 -- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08e4066470 msgr2=0x7f08e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08e4066470 0x7f08e4068930 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0af000 tx=0x7f08ec0a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:09.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 -- 172.21.15.31:0/2899418029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090013d4c0 msgr2=0x7f0900130090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090013d4c0 0x7f0900130090 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f08f40011f0 tx=0x7f08f400c9b0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 -- 172.21.15.31:0/2899418029 shutdown_connections 2023-11-26T07:18:09.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08e4066470 0x7f08e4068930 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f090015ac10 0x7f0900156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f090015a760 0x7f090006e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 --2- 172.21.15.31:0/2899418029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f090013d4c0 0x7f0900130090 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 -- 172.21.15.31:0/2899418029 >> 172.21.15.31:0/2899418029 conn(0x7f09000b1e40 msgr2=0x7f09001585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:09.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 -- 172.21.15.31:0/2899418029 shutdown_connections 2023-11-26T07:18:09.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.080+0000 7f0906e91640 1 -- 172.21.15.31:0/2899418029 wait complete. 2023-11-26T07:18:09.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:09.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:09.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f691458d640 1 Processor -- start 2023-11-26T07:18:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f691458d640 1 -- start start 2023-11-26T07:18:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f691458d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f690c15d8f0 0x7f690c15e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f691458d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c0ced30 0x7f690c0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f691458d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c15ecc0 con 0x7f690c0cf1b0 2023-11-26T07:18:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f691458d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0d4420 con 0x7f690c15d8f0 2023-11-26T07:18:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f691458d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0cdfb0 con 0x7f690c0ced30 2023-11-26T07:18:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f6912329640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f690c15d8f0 0x7f690c15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f6912b2a640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f690c0cf1b0 0x7f690c0d1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52042/0 (socket says 172.21.15.31:52042) 2023-11-26T07:18:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f6912b2a640 1 -- 172.21.15.31:0/845176976 learned_addr learned my addr 172.21.15.31:0/845176976 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f6912329640 1 -- 172.21.15.31:0/845176976 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c0ced30 msgr2=0x7f690c0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f6911b28640 1 --2- 172.21.15.31:0/845176976 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c0ced30 0x7f690c0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f6912329640 1 --2- 172.21.15.31:0/845176976 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c0ced30 0x7f690c0cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f6912329640 1 -- 172.21.15.31:0/845176976 >> v1:172.21.15.31:6789/0 conn(0x7f690c0cf1b0 legacy=0x7f690c0d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f6911327640 1 -- 172.21.15.31:0/845176976 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273572573 0 0) 0x7f690c15ecc0 con 0x7f690c0cf1b0 2023-11-26T07:18:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f6912329640 1 -- 172.21.15.31:0/845176976 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0d4140 con 0x7f690c15d8f0 2023-11-26T07:18:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.248+0000 7f6911b28640 1 --2- 172.21.15.31:0/845176976 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c0ced30 0x7f690c0cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f6912329640 1 --2- 172.21.15.31:0/845176976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c15d8f0 0x7f690c15e780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f69080097c0 tx=0x7f690801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8069225feafd1d5d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f6911327640 1 -- 172.21.15.31:0/845176976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080ab650 con 0x7f690c15d8f0 2023-11-26T07:18:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f6911327640 1 -- 172.21.15.31:0/845176976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69080ab7b0 con 0x7f690c15d8f0 2023-11-26T07:18:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f6911327640 1 -- 172.21.15.31:0/845176976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080abb40 con 0x7f690c15d8f0 2023-11-26T07:18:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f691458d640 1 -- 172.21.15.31:0/845176976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c15d8f0 msgr2=0x7f690c15e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f691458d640 1 --2- 172.21.15.31:0/845176976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c15d8f0 0x7f690c15e780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f69080097c0 tx=0x7f690801c450 comp rx=0 tx=0).stop 2023-11-26T07:18:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f691458d640 1 -- 172.21.15.31:0/845176976 shutdown_connections 2023-11-26T07:18:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f691458d640 1 --2- 172.21.15.31:0/845176976 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c0ced30 0x7f690c0cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f691458d640 1 --2- 172.21.15.31:0/845176976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c15d8f0 0x7f690c15e780 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f691458d640 1 -- 172.21.15.31:0/845176976 >> 172.21.15.31:0/845176976 conn(0x7f690c09c750 msgr2=0x7f690c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f691458d640 1 -- 172.21.15.31:0/845176976 shutdown_connections 2023-11-26T07:18:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f691458d640 1 -- 172.21.15.31:0/845176976 wait complete. 2023-11-26T07:18:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f691458d640 1 Processor -- start 2023-11-26T07:18:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.252+0000 7f691458d640 1 -- start start 2023-11-26T07:18:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f691458d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c0ced30 0x7f690c154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f691458d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f690c0cf1b0 0x7f690c0b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f691458d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c15d8f0 0x7f690c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f691458d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c155740 con 0x7f690c0ced30 2023-11-26T07:18:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f691458d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0b8f50 con 0x7f690c0cf1b0 2023-11-26T07:18:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f691458d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c155950 con 0x7f690c15d8f0 2023-11-26T07:18:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6912329640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c0ced30 0x7f690c154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6912329640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c0ced30 0x7f690c154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45494/0 (socket says 172.21.15.31:45494) 2023-11-26T07:18:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6912329640 1 -- 172.21.15.31:0/3936307714 learned_addr learned my addr 172.21.15.31:0/3936307714 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6911b28640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f690c0cf1b0 0x7f690c0b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6912b2a640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c15d8f0 0x7f690c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6912329640 1 -- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c15d8f0 msgr2=0x7f690c0ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6912329640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c15d8f0 0x7f690c0ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6912329640 1 -- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f690c0cf1b0 msgr2=0x7f690c0b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6912329640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f690c0cf1b0 0x7f690c0b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6912329640 1 -- 172.21.15.31:0/3936307714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6908009420 con 0x7f690c0ced30 2023-11-26T07:18:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6912329640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c0ced30 0x7f690c154db0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f69080ad040 tx=0x7f690801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6902ffd640 1 -- 172.21.15.31:0/3936307714 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080b5810 con 0x7f690c0ced30 2023-11-26T07:18:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6902ffd640 1 -- 172.21.15.31:0/3936307714 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69080b5990 con 0x7f690c0ced30 2023-11-26T07:18:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f691458d640 1 -- 172.21.15.31:0/3936307714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c155c30 con 0x7f690c0ced30 2023-11-26T07:18:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f6902ffd640 1 -- 172.21.15.31:0/3936307714 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080b5d40 con 0x7f690c0ced30 2023-11-26T07:18:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f691458d640 1 -- 172.21.15.31:0/3936307714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c1561f0 con 0x7f690c0ced30 2023-11-26T07:18:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.256+0000 7f691458d640 1 -- 172.21.15.31:0/3936307714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c04e850 con 0x7f690c0ced30 2023-11-26T07:18:09.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.260+0000 7f6902ffd640 1 -- 172.21.15.31:0/3936307714 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f69080bf410 con 0x7f690c0ced30 2023-11-26T07:18:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.260+0000 7f6902ffd640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f68f0066470 0x7f68f0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.260+0000 7f6902ffd640 1 -- 172.21.15.31:0/3936307714 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f690811ebe0 con 0x7f690c0ced30 2023-11-26T07:18:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.260+0000 7f6911b28640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f68f0066470 0x7f68f0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.260+0000 7f6911b28640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f68f0066470 0x7f68f0068930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f68fc006fd0 tx=0x7f68fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:09.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.264+0000 7f6902ffd640 1 -- 172.21.15.31:0/3936307714 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f69080ec0e0 con 0x7f690c0ced30 2023-11-26T07:18:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 -- 172.21.15.31:0/3936307714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f690c0cf9d0 con 0x7f690c0ced30 2023-11-26T07:18:09.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f6902ffd640 1 -- 172.21.15.31:0/3936307714 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69080ef700 con 0x7f690c0ced30 2023-11-26T07:18:09.431 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 -- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f68f0066470 msgr2=0x7f68f0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f68f0066470 0x7f68f0068930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f68fc006fd0 tx=0x7f68fc008040 comp rx=0 tx=0).stop 2023-11-26T07:18:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 -- 172.21.15.31:0/3936307714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c0ced30 msgr2=0x7f690c154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c0ced30 0x7f690c154db0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f69080ad040 tx=0x7f690801ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 -- 172.21.15.31:0/3936307714 shutdown_connections 2023-11-26T07:18:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f68f0066470 0x7f68f0068930 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f690c15d8f0 0x7f690c0ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f690c0cf1b0 0x7f690c0b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 --2- 172.21.15.31:0/3936307714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f690c0ced30 0x7f690c154db0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 -- 172.21.15.31:0/3936307714 >> 172.21.15.31:0/3936307714 conn(0x7f690c09c750 msgr2=0x7f690c15dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.424+0000 7f691458d640 1 -- 172.21.15.31:0/3936307714 shutdown_connections 2023-11-26T07:18:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.428+0000 7f691458d640 1 -- 172.21.15.31:0/3936307714 wait complete. 2023-11-26T07:18:09.451 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:09.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f363205a640 1 Processor -- start 2023-11-26T07:18:09.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f363205a640 1 -- start start 2023-11-26T07:18:09.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f363205a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f362c15a710 0x7f362c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f363205a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0b2710 con 0x7f362c0cbd50 2023-11-26T07:18:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f363205a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0b29f0 con 0x7f362c0cdfb0 2023-11-26T07:18:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f363205a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c0cca80 con 0x7f362c15a710 2023-11-26T07:18:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f362bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f362c0cbd50 0x7f362c0cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52048/0 (socket says 172.21.15.31:52048) 2023-11-26T07:18:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f362bfff640 1 -- 172.21.15.31:0/134132711 learned_addr learned my addr 172.21.15.31:0/134132711 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f362affd640 1 --2- 172.21.15.31:0/134132711 >> v2:172.21.15.97:3300/0 conn(0x7f362c15a710 0x7f362c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f362a7fc640 1 -- 172.21.15.31:0/134132711 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529345257 0 0) 0x7f362c0b2710 con 0x7f362c0cbd50 2023-11-26T07:18:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f362a7fc640 1 -- 172.21.15.31:0/134132711 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3614005740 con 0x7f362c0cbd50 2023-11-26T07:18:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f362affd640 1 -- 172.21.15.31:0/134132711 >> v1:172.21.15.97:6789/0 conn(0x7f362c0cdfb0 legacy=0x7f362c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f362affd640 1 -- 172.21.15.31:0/134132711 >> v1:172.21.15.31:6789/0 conn(0x7f362c0cbd50 legacy=0x7f362c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.592+0000 7f362affd640 1 -- 172.21.15.31:0/134132711 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0b2cd0 con 0x7f362c15a710 2023-11-26T07:18:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f362affd640 1 --2- 172.21.15.31:0/134132711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f362c15a710 0x7f362c0abf90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f36200097c0 tx=0x7f362001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc361d5418b48d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f362a7fc640 1 -- 172.21.15.31:0/134132711 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36200ab670 con 0x7f362c15a710 2023-11-26T07:18:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f362a7fc640 1 -- 172.21.15.31:0/134132711 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36200ab7d0 con 0x7f362c15a710 2023-11-26T07:18:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f362a7fc640 1 -- 172.21.15.31:0/134132711 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36200abb40 con 0x7f362c15a710 2023-11-26T07:18:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f363205a640 1 -- 172.21.15.31:0/134132711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f362c15a710 msgr2=0x7f362c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f363205a640 1 --2- 172.21.15.31:0/134132711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f362c15a710 0x7f362c0abf90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f36200097c0 tx=0x7f362001c450 comp rx=0 tx=0).stop 2023-11-26T07:18:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f363205a640 1 -- 172.21.15.31:0/134132711 shutdown_connections 2023-11-26T07:18:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f363205a640 1 --2- 172.21.15.31:0/134132711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f362c15a710 0x7f362c0abf90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f363205a640 1 -- 172.21.15.31:0/134132711 >> 172.21.15.31:0/134132711 conn(0x7f362c0b0cb0 msgr2=0x7f362c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f363205a640 1 -- 172.21.15.31:0/134132711 shutdown_connections 2023-11-26T07:18:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.596+0000 7f363205a640 1 -- 172.21.15.31:0/134132711 wait complete. 2023-11-26T07:18:09.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f363205a640 1 Processor -- start 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f363205a640 1 -- start start 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f363205a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f362c0cbd50 0x7f362c155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f363205a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cdfb0 0x7f362c15aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f363205a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f362c15a710 0x7f362c15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f363205a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c156530 con 0x7f362c0cdfb0 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f363205a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c15b9e0 con 0x7f362c0cbd50 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f363205a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1581b0 con 0x7f362c15a710 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cdfb0 0x7f362c15aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cdfb0 0x7f362c15aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45496/0 (socket says 172.21.15.31:45496) 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362affd640 1 -- 172.21.15.31:0/3867269371 learned_addr learned my addr 172.21.15.31:0/3867269371 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362b7fe640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f362c0cbd50 0x7f362c155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362bfff640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f362c15a710 0x7f362c15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362affd640 1 -- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f362c15a710 msgr2=0x7f362c15cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362affd640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f362c15a710 0x7f362c15cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362affd640 1 -- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f362c0cbd50 msgr2=0x7f362c155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362affd640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f362c0cbd50 0x7f362c155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362affd640 1 -- 172.21.15.31:0/3867269371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620009420 con 0x7f362c0cdfb0 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362bfff640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f362c15a710 0x7f362c15cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f362affd640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cdfb0 0x7f362c15aaf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f362001a080 tx=0x7f362001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f3613fff640 1 -- 172.21.15.31:0/3867269371 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620004220 con 0x7f362c0cdfb0 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f3613fff640 1 -- 172.21.15.31:0/3867269371 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36200b9070 con 0x7f362c0cdfb0 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f3613fff640 1 -- 172.21.15.31:0/3867269371 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620006e20 con 0x7f362c0cdfb0 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f363205a640 1 -- 172.21.15.31:0/3867269371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c158490 con 0x7f362c0cdfb0 2023-11-26T07:18:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.600+0000 7f363205a640 1 -- 172.21.15.31:0/3867269371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c158a50 con 0x7f362c0cdfb0 2023-11-26T07:18:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.604+0000 7f363205a640 1 -- 172.21.15.31:0/3867269371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c0671f0 con 0x7f362c0cdfb0 2023-11-26T07:18:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.604+0000 7f3613fff640 1 -- 172.21.15.31:0/3867269371 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f36200b4610 con 0x7f362c0cdfb0 2023-11-26T07:18:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.604+0000 7f3613fff640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3618066470 0x7f3618068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.604+0000 7f3613fff640 1 -- 172.21.15.31:0/3867269371 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36200bb070 con 0x7f362c0cdfb0 2023-11-26T07:18:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.604+0000 7f362b7fe640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3618066470 0x7f3618068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.604+0000 7f362b7fe640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3618066470 0x7f3618068930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f361c001650 tx=0x7f361c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:09.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.608+0000 7f3613fff640 1 -- 172.21.15.31:0/3867269371 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f36200ec250 con 0x7f362c0cdfb0 2023-11-26T07:18:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 -- 172.21.15.31:0/3867269371 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f362c1582f0 con 0x7f362c0cdfb0 2023-11-26T07:18:09.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f3613fff640 1 -- 172.21.15.31:0/3867269371 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f362c1582f0 con 0x7f362c0cdfb0 2023-11-26T07:18:09.773 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:18:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 -- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3618066470 msgr2=0x7f3618068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3618066470 0x7f3618068930 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f361c001650 tx=0x7f361c009040 comp rx=0 tx=0).stop 2023-11-26T07:18:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 -- 172.21.15.31:0/3867269371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cdfb0 msgr2=0x7f362c15aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cdfb0 0x7f362c15aaf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f362001a080 tx=0x7f362001ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 -- 172.21.15.31:0/3867269371 shutdown_connections 2023-11-26T07:18:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3618066470 0x7f3618068930 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f362c15a710 0x7f362c15cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f362c0cdfb0 0x7f362c15aaf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 --2- 172.21.15.31:0/3867269371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f362c0cbd50 0x7f362c155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 -- 172.21.15.31:0/3867269371 >> 172.21.15.31:0/3867269371 conn(0x7f362c0b0cb0 msgr2=0x7f362c15e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:09.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 -- 172.21.15.31:0/3867269371 shutdown_connections 2023-11-26T07:18:09.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.768+0000 7f363205a640 1 -- 172.21.15.31:0/3867269371 wait complete. 2023-11-26T07:18:09.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:09.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f475989c640 1 Processor -- start 2023-11-26T07:18:09.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f475989c640 1 -- start start 2023-11-26T07:18:09.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f475989c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4754161f20 0x7f47540cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f475989c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47540d0740 con 0x7f475415d7c0 2023-11-26T07:18:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f475989c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47540d11a0 con 0x7f47540d02f0 2023-11-26T07:18:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f475989c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47540d0f90 con 0x7f4754161f20 2023-11-26T07:18:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f4752ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f475415d7c0 0x7f47540d3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52052/0 (socket says 172.21.15.31:52052) 2023-11-26T07:18:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f4752ffd640 1 -- 172.21.15.31:0/1518156758 learned_addr learned my addr 172.21.15.31:0/1518156758 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f47527fc640 1 --2- 172.21.15.31:0/1518156758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4754161f20 0x7f47540cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f4751ffb640 1 -- 172.21.15.31:0/1518156758 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482282239 0 0) 0x7f4744001120 con 0x7f475415d7c0 2023-11-26T07:18:09.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f4751ffb640 1 -- 172.21.15.31:0/1518156758 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f473c003530 con 0x7f475415d7c0 2023-11-26T07:18:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f4751ffb640 1 -- 172.21.15.31:0/1518156758 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406730163 0 0) 0x7f47540d11a0 con 0x7f47540d02f0 2023-11-26T07:18:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f47527fc640 1 -- 172.21.15.31:0/1518156758 >> v1:172.21.15.97:6789/0 conn(0x7f47540d02f0 legacy=0x7f47540cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f47527fc640 1 -- 172.21.15.31:0/1518156758 >> v1:172.21.15.31:6789/0 conn(0x7f475415d7c0 legacy=0x7f47540d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.936+0000 7f47527fc640 1 -- 172.21.15.31:0/1518156758 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540d1480 con 0x7f4754161f20 2023-11-26T07:18:09.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f47527fc640 1 --2- 172.21.15.31:0/1518156758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4754161f20 0x7f47540cfdb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f47480097d0 tx=0x7f474801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a6fafce2b67477 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f4751ffb640 1 -- 172.21.15.31:0/1518156758 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47480ac640 con 0x7f4754161f20 2023-11-26T07:18:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f4751ffb640 1 -- 172.21.15.31:0/1518156758 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47480ac7a0 con 0x7f4754161f20 2023-11-26T07:18:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f4751ffb640 1 -- 172.21.15.31:0/1518156758 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47480acb30 con 0x7f4754161f20 2023-11-26T07:18:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f475989c640 1 -- 172.21.15.31:0/1518156758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4754161f20 msgr2=0x7f47540cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f475989c640 1 --2- 172.21.15.31:0/1518156758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4754161f20 0x7f47540cfdb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f47480097d0 tx=0x7f474801c3e0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f475989c640 1 -- 172.21.15.31:0/1518156758 shutdown_connections 2023-11-26T07:18:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f475989c640 1 --2- 172.21.15.31:0/1518156758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4754161f20 0x7f47540cfdb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f475989c640 1 -- 172.21.15.31:0/1518156758 >> 172.21.15.31:0/1518156758 conn(0x7f475408afc0 msgr2=0x7f47540d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f475989c640 1 -- 172.21.15.31:0/1518156758 shutdown_connections 2023-11-26T07:18:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.940+0000 7f475989c640 1 -- 172.21.15.31:0/1518156758 wait complete. 2023-11-26T07:18:09.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f475989c640 1 Processor -- start 2023-11-26T07:18:09.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f475989c640 1 -- start start 2023-11-26T07:18:09.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f475989c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47540d02f0 0x7f475415de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f475989c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f475415d7c0 0x7f4754154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f475989c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4754154b40 0x7f475415f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f475989c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47540b8ec0 con 0x7f47540d02f0 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f475989c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4754154ff0 con 0x7f4754154b40 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f475989c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4754161040 con 0x7f475415d7c0 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f4752ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47540d02f0 0x7f475415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f4752ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47540d02f0 0x7f475415de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45506/0 (socket says 172.21.15.31:45506) 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f4752ffd640 1 -- 172.21.15.31:0/3731306872 learned_addr learned my addr 172.21.15.31:0/3731306872 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f47537fe640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4754154b40 0x7f475415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f47527fc640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f475415d7c0 0x7f4754154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f4752ffd640 1 -- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f475415d7c0 msgr2=0x7f4754154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f4752ffd640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f475415d7c0 0x7f4754154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f4752ffd640 1 -- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4754154b40 msgr2=0x7f475415f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f4752ffd640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4754154b40 0x7f475415f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f4752ffd640 1 -- 172.21.15.31:0/3731306872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47480093b0 con 0x7f47540d02f0 2023-11-26T07:18:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f47527fc640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f475415d7c0 0x7f4754154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f4752ffd640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47540d02f0 0x7f475415de00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f474400c700 tx=0x7f474400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f473bfff640 1 -- 172.21.15.31:0/3731306872 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744005ac0 con 0x7f47540d02f0 2023-11-26T07:18:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f473bfff640 1 -- 172.21.15.31:0/3731306872 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4744005c60 con 0x7f47540d02f0 2023-11-26T07:18:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f473bfff640 1 -- 172.21.15.31:0/3731306872 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744001d60 con 0x7f47540d02f0 2023-11-26T07:18:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f475989c640 1 -- 172.21.15.31:0/3731306872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475415ffa0 con 0x7f47540d02f0 2023-11-26T07:18:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.944+0000 7f475989c640 1 -- 172.21.15.31:0/3731306872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4754160560 con 0x7f47540d02f0 2023-11-26T07:18:09.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.948+0000 7f475989c640 1 -- 172.21.15.31:0/3731306872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475404e850 con 0x7f47540d02f0 2023-11-26T07:18:09.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.948+0000 7f473bfff640 1 -- 172.21.15.31:0/3731306872 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4744007070 con 0x7f47540d02f0 2023-11-26T07:18:09.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.948+0000 7f473bfff640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4740066470 0x7f4740068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:09.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.948+0000 7f473bfff640 1 -- 172.21.15.31:0/3731306872 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47440838d0 con 0x7f47540d02f0 2023-11-26T07:18:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.948+0000 7f47527fc640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4740066470 0x7f4740068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.952+0000 7f47527fc640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4740066470 0x7f4740068930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f47480ae000 tx=0x7f4748008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:09.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:09.952+0000 7f473bfff640 1 -- 172.21.15.31:0/3731306872 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4744050ee0 con 0x7f47540d02f0 2023-11-26T07:18:10.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 -- 172.21.15.31:0/3731306872 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47540d0740 con 0x7f47540d02f0 2023-11-26T07:18:10.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f473bfff640 1 -- 172.21.15.31:0/3731306872 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47540d0740 con 0x7f47540d02f0 2023-11-26T07:18:10.113 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:18:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 -- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4740066470 msgr2=0x7f4740068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4740066470 0x7f4740068930 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f47480ae000 tx=0x7f4748008040 comp rx=0 tx=0).stop 2023-11-26T07:18:10.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 -- 172.21.15.31:0/3731306872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47540d02f0 msgr2=0x7f475415de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47540d02f0 0x7f475415de00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f474400c700 tx=0x7f474400cbc0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 -- 172.21.15.31:0/3731306872 shutdown_connections 2023-11-26T07:18:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4740066470 0x7f4740068930 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4754154b40 0x7f475415f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f475415d7c0 0x7f4754154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 --2- 172.21.15.31:0/3731306872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47540d02f0 0x7f475415de00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 -- 172.21.15.31:0/3731306872 >> 172.21.15.31:0/3731306872 conn(0x7f475408afc0 msgr2=0x7f47540b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 -- 172.21.15.31:0/3731306872 shutdown_connections 2023-11-26T07:18:10.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.108+0000 7f475989c640 1 -- 172.21.15.31:0/3731306872 wait complete. 2023-11-26T07:18:10.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:10.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.272+0000 7fc711fc4640 1 Processor -- start 2023-11-26T07:18:10.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.272+0000 7fc711fc4640 1 -- start start 2023-11-26T07:18:10.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc711fc4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc70c15b4b0 0x7fc70c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc711fc4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc70c0b1820 0x7fc70c0ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc711fc4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70c0b0130 con 0x7fc70c06a5b0 2023-11-26T07:18:10.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc711fc4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70c0b0b00 con 0x7fc70c0b1820 2023-11-26T07:18:10.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc711fc4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70c0afb00 con 0x7fc70c15b4b0 2023-11-26T07:18:10.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc70bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc70c0b1820 0x7fc70c0ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc70bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc70c0b1820 0x7fc70c0ad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45516/0 (socket says 172.21.15.31:45516) 2023-11-26T07:18:10.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc70bfff640 1 -- 172.21.15.31:0/2343075991 learned_addr learned my addr 172.21.15.31:0/2343075991 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:10.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc70b7fe640 1 --2- 172.21.15.31:0/2343075991 >> v2:172.21.15.97:3300/0 conn(0x7fc70c15b4b0 0x7fc70c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc70bfff640 1 -- 172.21.15.31:0/2343075991 >> v2:172.21.15.97:3300/0 conn(0x7fc70c15b4b0 msgr2=0x7fc70c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:10.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc70bfff640 1 --2- 172.21.15.31:0/2343075991 >> v2:172.21.15.97:3300/0 conn(0x7fc70c15b4b0 0x7fc70c0b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc70bfff640 1 -- 172.21.15.31:0/2343075991 >> v1:172.21.15.97:6789/0 conn(0x7fc70c06a5b0 legacy=0x7fc70c0b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.276+0000 7fc70bfff640 1 -- 172.21.15.31:0/2343075991 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c06af70 con 0x7fc70c0b1820 2023-11-26T07:18:10.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc70bfff640 1 --2- 172.21.15.31:0/2343075991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c0b1820 0x7fc70c0ad030 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0cdca0 tx=0x7fc70401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878cefc0cd6f25ba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:10.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc70a7fc640 1 -- 172.21.15.31:0/2343075991 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7040a56b0 con 0x7fc70c0b1820 2023-11-26T07:18:10.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc70a7fc640 1 -- 172.21.15.31:0/2343075991 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7040a5810 con 0x7fc70c0b1820 2023-11-26T07:18:10.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc70a7fc640 1 -- 172.21.15.31:0/2343075991 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7040a5ba0 con 0x7fc70c0b1820 2023-11-26T07:18:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc711fc4640 1 -- 172.21.15.31:0/2343075991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c0b1820 msgr2=0x7fc70c0ad030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc711fc4640 1 --2- 172.21.15.31:0/2343075991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c0b1820 0x7fc70c0ad030 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0cdca0 tx=0x7fc70401c9a0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc711fc4640 1 -- 172.21.15.31:0/2343075991 shutdown_connections 2023-11-26T07:18:10.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc711fc4640 1 --2- 172.21.15.31:0/2343075991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c0b1820 0x7fc70c0ad030 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc711fc4640 1 --2- 172.21.15.31:0/2343075991 >> v2:172.21.15.97:3300/0 conn(0x7fc70c15b4b0 0x7fc70c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc711fc4640 1 -- 172.21.15.31:0/2343075991 >> 172.21.15.31:0/2343075991 conn(0x7fc70c0b1d50 msgr2=0x7fc70c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:10.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc711fc4640 1 -- 172.21.15.31:0/2343075991 shutdown_connections 2023-11-26T07:18:10.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc711fc4640 1 -- 172.21.15.31:0/2343075991 wait complete. 2023-11-26T07:18:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc711fc4640 1 Processor -- start 2023-11-26T07:18:10.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.280+0000 7fc711fc4640 1 -- start start 2023-11-26T07:18:10.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc711fc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c06a5b0 0x7fc70c148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc711fc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc70c0b1820 0x7fc70c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc711fc4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc70c15b4b0 0x7fc70c154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc711fc4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c152e80 con 0x7fc70c06a5b0 2023-11-26T07:18:10.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc711fc4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c153ce0 con 0x7fc70c0b1820 2023-11-26T07:18:10.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc711fc4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc70c1530d0 con 0x7fc70c15b4b0 2023-11-26T07:18:10.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c06a5b0 0x7fc70c148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c06a5b0 0x7fc70c148080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45518/0 (socket says 172.21.15.31:45518) 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70b7fe640 1 -- 172.21.15.31:0/2238708784 learned_addr learned my addr 172.21.15.31:0/2238708784 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70bfff640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc70c15b4b0 0x7fc70c154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70affd640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc70c0b1820 0x7fc70c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70b7fe640 1 -- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc70c15b4b0 msgr2=0x7fc70c154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70b7fe640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc70c15b4b0 0x7fc70c154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70b7fe640 1 -- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc70c0b1820 msgr2=0x7fc70c154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70b7fe640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc70c0b1820 0x7fc70c154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70b7fe640 1 -- 172.21.15.31:0/2238708784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc704009b30 con 0x7fc70c06a5b0 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc70b7fe640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c06a5b0 0x7fc70c148080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00e930 tx=0x7fc6fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc6f3fff640 1 -- 172.21.15.31:0/2238708784 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc009690 con 0x7fc70c06a5b0 2023-11-26T07:18:10.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc6f3fff640 1 -- 172.21.15.31:0/2238708784 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc00ca00 con 0x7fc70c06a5b0 2023-11-26T07:18:10.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc6f3fff640 1 -- 172.21.15.31:0/2238708784 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00cd70 con 0x7fc70c06a5b0 2023-11-26T07:18:10.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc711fc4640 1 -- 172.21.15.31:0/2238708784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70c1533b0 con 0x7fc70c06a5b0 2023-11-26T07:18:10.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc711fc4640 1 -- 172.21.15.31:0/2238708784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70c153970 con 0x7fc70c06a5b0 2023-11-26T07:18:10.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc711fc4640 1 -- 172.21.15.31:0/2238708784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc70c04e850 con 0x7fc70c06a5b0 2023-11-26T07:18:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.284+0000 7fc6f3fff640 1 -- 172.21.15.31:0/2238708784 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc6fc013470 con 0x7fc70c06a5b0 2023-11-26T07:18:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.288+0000 7fc6f3fff640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6f80664a0 0x7fc6f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.288+0000 7fc6f3fff640 1 -- 172.21.15.31:0/2238708784 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6fc084080 con 0x7fc70c06a5b0 2023-11-26T07:18:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.288+0000 7fc70affd640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6f80664a0 0x7fc6f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.288+0000 7fc70affd640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6f80664a0 0x7fc6f8068960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc70c151590 tx=0x7fc700009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:10.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.292+0000 7fc6f3fff640 1 -- 172.21.15.31:0/2238708784 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc6fc0b5280 con 0x7fc70c06a5b0 2023-11-26T07:18:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 -- 172.21.15.31:0/2238708784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc70c165e40 con 0x7fc70c06a5b0 2023-11-26T07:18:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc6f3fff640 1 -- 172.21.15.31:0/2238708784 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc6fc0515e0 con 0x7fc70c06a5b0 2023-11-26T07:18:10.468 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 -- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6f80664a0 msgr2=0x7fc6f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6f80664a0 0x7fc6f8068960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc70c151590 tx=0x7fc700009040 comp rx=0 tx=0).stop 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 -- 172.21.15.31:0/2238708784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c06a5b0 msgr2=0x7fc70c148080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c06a5b0 0x7fc70c148080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00e930 tx=0x7fc6fc00edf0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 -- 172.21.15.31:0/2238708784 shutdown_connections 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc6f80664a0 0x7fc6f8068960 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc70c15b4b0 0x7fc70c154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc70c0b1820 0x7fc70c154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 --2- 172.21.15.31:0/2238708784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc70c06a5b0 0x7fc70c148080 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 -- 172.21.15.31:0/2238708784 >> 172.21.15.31:0/2238708784 conn(0x7fc70c0b1d50 msgr2=0x7fc70c0cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:10.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 -- 172.21.15.31:0/2238708784 shutdown_connections 2023-11-26T07:18:10.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.464+0000 7fc711fc4640 1 -- 172.21.15.31:0/2238708784 wait complete. 2023-11-26T07:18:10.490 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-26T07:18:10.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:10.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-26T07:18:10.517 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:18:10.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:10.518 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-26T07:18:10.572 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:18:10.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.708+0000 7f9cbb05a640 1 Processor -- start 2023-11-26T07:18:10.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.708+0000 7f9cbb05a640 1 -- start start 2023-11-26T07:18:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cbb05a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9cb40716a0 0x7f9cb4071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cbb05a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb4071b30 con 0x7f9cb4072470 2023-11-26T07:18:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cbb05a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb4070920 con 0x7f9cb415d3e0 2023-11-26T07:18:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cbb05a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb40728e0 con 0x7f9cb40716a0 2023-11-26T07:18:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb95f7640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9cb4072470 0x7f9cb40b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52064/0 (socket says 172.21.15.31:52064) 2023-11-26T07:18:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb95f7640 1 -- 172.21.15.31:0/1464161425 learned_addr learned my addr 172.21.15.31:0/1464161425 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb3fff640 1 --2- 172.21.15.31:0/1464161425 >> v2:172.21.15.97:3300/0 conn(0x7f9cb40716a0 0x7f9cb4071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb37fe640 1 -- 172.21.15.31:0/1464161425 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443054270 0 0) 0x7f9cb4071b30 con 0x7f9cb4072470 2023-11-26T07:18:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb37fe640 1 -- 172.21.15.31:0/1464161425 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ca4003530 con 0x7f9cb4072470 2023-11-26T07:18:10.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb37fe640 1 -- 172.21.15.31:0/1464161425 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561515558 0 0) 0x7f9cb4070920 con 0x7f9cb415d3e0 2023-11-26T07:18:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb37fe640 1 -- 172.21.15.31:0/1464161425 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cb4071b30 con 0x7f9cb415d3e0 2023-11-26T07:18:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb37fe640 1 -- 172.21.15.31:0/1464161425 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2749661661 0 0) 0x7f9ca4003530 con 0x7f9cb4072470 2023-11-26T07:18:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb3fff640 1 -- 172.21.15.31:0/1464161425 >> v1:172.21.15.97:6789/0 conn(0x7f9cb415d3e0 legacy=0x7f9cb406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb3fff640 1 -- 172.21.15.31:0/1464161425 >> v1:172.21.15.31:6789/0 conn(0x7f9cb4072470 legacy=0x7f9cb40b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.712+0000 7f9cb3fff640 1 -- 172.21.15.31:0/1464161425 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb415d7c0 con 0x7f9cb40716a0 2023-11-26T07:18:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cb3fff640 1 --2- 172.21.15.31:0/1464161425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cb40716a0 0x7f9cb4071f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00097c0 tx=0x7f9ca001c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f477d809fee2fa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cb37fe640 1 -- 172.21.15.31:0/1464161425 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac001a00 con 0x7f9cb40716a0 2023-11-26T07:18:10.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cb37fe640 1 -- 172.21.15.31:0/1464161425 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca00a7050 con 0x7f9cb40716a0 2023-11-26T07:18:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cb37fe640 1 -- 172.21.15.31:0/1464161425 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00ab620 con 0x7f9cb40716a0 2023-11-26T07:18:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1464161425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cb40716a0 msgr2=0x7f9cb4071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cbb05a640 1 --2- 172.21.15.31:0/1464161425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cb40716a0 0x7f9cb4071f30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00097c0 tx=0x7f9ca001c370 comp rx=0 tx=0).stop 2023-11-26T07:18:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1464161425 shutdown_connections 2023-11-26T07:18:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cbb05a640 1 --2- 172.21.15.31:0/1464161425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cb40716a0 0x7f9cb4071f30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1464161425 >> 172.21.15.31:0/1464161425 conn(0x7f9cb40b0c40 msgr2=0x7f9cb40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1464161425 shutdown_connections 2023-11-26T07:18:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1464161425 wait complete. 2023-11-26T07:18:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cbb05a640 1 Processor -- start 2023-11-26T07:18:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.716+0000 7f9cbb05a640 1 -- start start 2023-11-26T07:18:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cbb05a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cb40716a0 0x7f9cb4156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cbb05a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cb4072470 0x7f9cb40b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cbb05a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9cb415d3e0 0x7f9cb414e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cbb05a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb4150520 con 0x7f9cb4072470 2023-11-26T07:18:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cbb05a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb415c9e0 con 0x7f9cb40716a0 2023-11-26T07:18:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cbb05a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb414ff60 con 0x7f9cb415d3e0 2023-11-26T07:18:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cb4072470 0x7f9cb40b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cb4072470 0x7f9cb40b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45524/0 (socket says 172.21.15.31:45524) 2023-11-26T07:18:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb3fff640 1 -- 172.21.15.31:0/1560384727 learned_addr learned my addr 172.21.15.31:0/1560384727 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb8df6640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cb40716a0 0x7f9cb4156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb95f7640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9cb415d3e0 0x7f9cb414e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb3fff640 1 -- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9cb415d3e0 msgr2=0x7f9cb414e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb3fff640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9cb415d3e0 0x7f9cb414e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb3fff640 1 -- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cb40716a0 msgr2=0x7f9cb4156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb3fff640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cb40716a0 0x7f9cb4156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb3fff640 1 -- 172.21.15.31:0/1560384727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca0009420 con 0x7f9cb4072470 2023-11-26T07:18:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb3fff640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cb4072470 0x7f9cb40b0040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00ae040 tx=0x7f9ca0019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:10.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb17fa640 1 -- 172.21.15.31:0/1560384727 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00abe70 con 0x7f9cb4072470 2023-11-26T07:18:10.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb17fa640 1 -- 172.21.15.31:0/1560384727 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca0004020 con 0x7f9cb4072470 2023-11-26T07:18:10.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cb17fa640 1 -- 172.21.15.31:0/1560384727 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0004390 con 0x7f9cb4072470 2023-11-26T07:18:10.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1560384727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb4150240 con 0x7f9cb4072470 2023-11-26T07:18:10.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1560384727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb41570d0 con 0x7f9cb4072470 2023-11-26T07:18:10.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.720+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1560384727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb415c4d0 con 0x7f9cb4072470 2023-11-26T07:18:10.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.724+0000 7f9cb17fa640 1 -- 172.21.15.31:0/1560384727 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9ca00b2040 con 0x7f9cb4072470 2023-11-26T07:18:10.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.724+0000 7f9cb17fa640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c98066470 0x7f9c98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:10.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.724+0000 7f9cb17fa640 1 -- 172.21.15.31:0/1560384727 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ca011cd80 con 0x7f9cb4072470 2023-11-26T07:18:10.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.724+0000 7f9cb8df6640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c98066470 0x7f9c98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:10.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.724+0000 7f9cb8df6640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c98066470 0x7f9c98068930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80011f0 tx=0x7f9ca8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:10.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.728+0000 7f9cb17fa640 1 -- 172.21.15.31:0/1560384727 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9cb415c4d0 con 0x7f9cb4072470 2023-11-26T07:18:10.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.916+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1560384727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cb415c4d0 con 0x7f9cb4072470 2023-11-26T07:18:10.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.916+0000 7f9cb17fa640 1 -- 172.21.15.31:0/1560384727 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f9cb415c4d0 con 0x7f9cb4072470 2023-11-26T07:18:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:10.923 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4099326271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4099326271},{"type":"v1","addr":"172.21.15.31:6835","nonce":4099326271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2404164953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2404164953},{"type":"v1","addr":"172.21.15.97:6836","nonce":2404164953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1312969281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1312969281},{"type":"v1","addr":"172.21.15.31:6834","nonce":1312969281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:32.510772+0000","modified":"2023-11-26T07:17:35.514610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/527056216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":527056216},{"type":"v1","addr":"172.21.15.97:6837","nonce":527056216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:10.923 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-11-26T07:18:10.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c98066470 msgr2=0x7f9c98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c98066470 0x7f9c98068930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80011f0 tx=0x7f9ca8009040 comp rx=0 tx=0).stop 2023-11-26T07:18:10.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1560384727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cb4072470 msgr2=0x7f9cb40b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:10.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cb4072470 0x7f9cb40b0040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00ae040 tx=0x7f9ca0019110 comp rx=0 tx=0).stop 2023-11-26T07:18:10.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1560384727 shutdown_connections 2023-11-26T07:18:10.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9c98066470 0x7f9c98068930 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9cb415d3e0 0x7f9cb414e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9cb4072470 0x7f9cb40b0040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 --2- 172.21.15.31:0/1560384727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9cb40716a0 0x7f9cb4156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:10.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1560384727 >> 172.21.15.31:0/1560384727 conn(0x7f9cb40b0c40 msgr2=0x7f9cb40b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:10.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1560384727 shutdown_connections 2023-11-26T07:18:10.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:10.920+0000 7f9cbb05a640 1 -- 172.21.15.31:0/1560384727 wait complete. 2023-11-26T07:18:10.944 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:11.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.080+0000 7f483b50f640 1 Processor -- start 2023-11-26T07:18:11.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f483b50f640 1 -- start start 2023-11-26T07:18:11.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f483b50f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4834144c80 0x7f48340abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f483b50f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48340ac460 0x7f48340706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f483b50f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4834145060 con 0x7f4834070c20 2023-11-26T07:18:11.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f483b50f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4834071fd0 con 0x7f4834144c80 2023-11-26T07:18:11.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f483b50f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48340ac8c0 con 0x7f48340ac460 2023-11-26T07:18:11.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f48392ab640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4834144c80 0x7f48340abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f4839aac640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4834070c20 0x7f483415eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52070/0 (socket says 172.21.15.31:52070) 2023-11-26T07:18:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f4839aac640 1 -- 172.21.15.31:0/3710769578 learned_addr learned my addr 172.21.15.31:0/3710769578 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f4838aaa640 1 --2- 172.21.15.31:0/3710769578 >> v2:172.21.15.97:3300/0 conn(0x7f48340ac460 0x7f48340706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f482bfff640 1 -- 172.21.15.31:0/3710769578 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705478725 0 0) 0x7f4834145060 con 0x7f4834070c20 2023-11-26T07:18:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f482bfff640 1 -- 172.21.15.31:0/3710769578 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4820003530 con 0x7f4834070c20 2023-11-26T07:18:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f48392ab640 1 -- 172.21.15.31:0/3710769578 >> v2:172.21.15.97:3300/0 conn(0x7f48340ac460 msgr2=0x7f48340706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f48392ab640 1 --2- 172.21.15.31:0/3710769578 >> v2:172.21.15.97:3300/0 conn(0x7f48340ac460 0x7f48340706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f48392ab640 1 -- 172.21.15.31:0/3710769578 >> v1:172.21.15.31:6789/0 conn(0x7f4834070c20 legacy=0x7f483415eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.084+0000 7f48392ab640 1 -- 172.21.15.31:0/3710769578 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4834071450 con 0x7f4834144c80 2023-11-26T07:18:11.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f48392ab640 1 --2- 172.21.15.31:0/3710769578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834144c80 0x7f48340abf20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4834072d10 tx=0x7f483001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca88177c2f8a365 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:11.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f482bfff640 1 -- 172.21.15.31:0/3710769578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300a7070 con 0x7f4834144c80 2023-11-26T07:18:11.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f482bfff640 1 -- 172.21.15.31:0/3710769578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48300ab430 con 0x7f4834144c80 2023-11-26T07:18:11.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f482bfff640 1 -- 172.21.15.31:0/3710769578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300ab7c0 con 0x7f4834144c80 2023-11-26T07:18:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f483b50f640 1 -- 172.21.15.31:0/3710769578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834144c80 msgr2=0x7f48340abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f483b50f640 1 --2- 172.21.15.31:0/3710769578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834144c80 0x7f48340abf20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4834072d10 tx=0x7f483001c430 comp rx=0 tx=0).stop 2023-11-26T07:18:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f483b50f640 1 -- 172.21.15.31:0/3710769578 shutdown_connections 2023-11-26T07:18:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f483b50f640 1 --2- 172.21.15.31:0/3710769578 >> v2:172.21.15.97:3300/0 conn(0x7f48340ac460 0x7f48340706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f483b50f640 1 --2- 172.21.15.31:0/3710769578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834144c80 0x7f48340abf20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f483b50f640 1 -- 172.21.15.31:0/3710769578 >> 172.21.15.31:0/3710769578 conn(0x7f48340b0c40 msgr2=0x7f48340b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f483b50f640 1 -- 172.21.15.31:0/3710769578 shutdown_connections 2023-11-26T07:18:11.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f483b50f640 1 -- 172.21.15.31:0/3710769578 wait complete. 2023-11-26T07:18:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f483b50f640 1 Processor -- start 2023-11-26T07:18:11.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.088+0000 7f483b50f640 1 -- start start 2023-11-26T07:18:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f483b50f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834070c20 0x7f483413bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f483b50f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48340ac460 0x7f483409a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f483b50f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834144c80 0x7f483409be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f483b50f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483413c0c0 con 0x7f4834070c20 2023-11-26T07:18:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f483b50f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48340b1dd0 con 0x7f48340ac460 2023-11-26T07:18:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f483b50f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48340b20b0 con 0x7f4834144c80 2023-11-26T07:18:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f48392ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834070c20 0x7f483413bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f48392ab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834070c20 0x7f483413bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45548/0 (socket says 172.21.15.31:45548) 2023-11-26T07:18:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f48392ab640 1 -- 172.21.15.31:0/1316648087 learned_addr learned my addr 172.21.15.31:0/1316648087 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f4838aaa640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48340ac460 0x7f483409a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f4839aac640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834144c80 0x7f483409be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f48392ab640 1 -- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834144c80 msgr2=0x7f483409be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f48392ab640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834144c80 0x7f483409be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f48392ab640 1 -- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48340ac460 msgr2=0x7f483409a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f48392ab640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48340ac460 0x7f483409a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f48392ab640 1 -- 172.21.15.31:0/1316648087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830009420 con 0x7f4834070c20 2023-11-26T07:18:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f4838aaa640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48340ac460 0x7f483409a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:11.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f48392ab640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834070c20 0x7f483413bb80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f48300ad040 tx=0x7f483001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f4829ffb640 1 -- 172.21.15.31:0/1316648087 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300bf610 con 0x7f4834070c20 2023-11-26T07:18:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f4829ffb640 1 -- 172.21.15.31:0/1316648087 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48300bf790 con 0x7f4834070c20 2023-11-26T07:18:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f4829ffb640 1 -- 172.21.15.31:0/1316648087 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300bfb40 con 0x7f4834070c20 2023-11-26T07:18:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f483b50f640 1 -- 172.21.15.31:0/1316648087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48340b2390 con 0x7f4834070c20 2023-11-26T07:18:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f483b50f640 1 -- 172.21.15.31:0/1316648087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48340b2950 con 0x7f4834070c20 2023-11-26T07:18:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.092+0000 7f483b50f640 1 -- 172.21.15.31:0/1316648087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48340b1ba0 con 0x7f4834070c20 2023-11-26T07:18:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.096+0000 7f4829ffb640 1 -- 172.21.15.31:0/1316648087 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f48300b5400 con 0x7f4834070c20 2023-11-26T07:18:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.096+0000 7f4829ffb640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4818066470 0x7f4818068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.096+0000 7f4829ffb640 1 -- 172.21.15.31:0/1316648087 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f483011e500 con 0x7f4834070c20 2023-11-26T07:18:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.096+0000 7f4838aaa640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4818066470 0x7f4818068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.096+0000 7f4838aaa640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4818066470 0x7f4818068930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f48341583a0 tx=0x7f4824008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:11.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.100+0000 7f4829ffb640 1 -- 172.21.15.31:0/1316648087 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f48300ebab0 con 0x7f4834070c20 2023-11-26T07:18:11.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 -- 172.21.15.31:0/1316648087 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48340b2b40 con 0x7f4834070c20 2023-11-26T07:18:11.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f4829ffb640 1 -- 172.21.15.31:0/1316648087 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f48300ef0d0 con 0x7f4834070c20 2023-11-26T07:18:11.265 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:11.265 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 -- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4818066470 msgr2=0x7f4818068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4818066470 0x7f4818068930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f48341583a0 tx=0x7f4824008040 comp rx=0 tx=0).stop 2023-11-26T07:18:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 -- 172.21.15.31:0/1316648087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834070c20 msgr2=0x7f483413bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834070c20 0x7f483413bb80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f48300ad040 tx=0x7f483001ac70 comp rx=0 tx=0).stop 2023-11-26T07:18:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 -- 172.21.15.31:0/1316648087 shutdown_connections 2023-11-26T07:18:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4818066470 0x7f4818068930 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4834144c80 0x7f483409be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48340ac460 0x7f483409a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 --2- 172.21.15.31:0/1316648087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834070c20 0x7f483413bb80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 -- 172.21.15.31:0/1316648087 >> 172.21.15.31:0/1316648087 conn(0x7f48340b0c40 msgr2=0x7f4834164da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:11.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 -- 172.21.15.31:0/1316648087 shutdown_connections 2023-11-26T07:18:11.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.260+0000 7f483b50f640 1 -- 172.21.15.31:0/1316648087 wait complete. 2023-11-26T07:18:11.288 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:17:30.635783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:17:31.500091+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:11.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:11.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.428+0000 7f7a08da1640 1 Processor -- start 2023-11-26T07:18:11.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.428+0000 7f7a08da1640 1 -- start start 2023-11-26T07:18:11.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.428+0000 7f7a08da1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a040cdfb0 0x7f7a040abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.428+0000 7f7a08da1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7a040cbd50 0x7f7a040cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.428+0000 7f7a08da1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a040b2710 con 0x7f7a0415a710 2023-11-26T07:18:11.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.428+0000 7f7a08da1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a040cca80 con 0x7f7a040cbd50 2023-11-26T07:18:11.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.428+0000 7f7a08da1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a040b29f0 con 0x7f7a040cdfb0 2023-11-26T07:18:11.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a02d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7a040cbd50 0x7f7a040cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a02d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7a040cbd50 0x7f7a040cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45554/0 (socket says 172.21.15.31:45554) 2023-11-26T07:18:11.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a02d9d640 1 -- 172.21.15.31:0/3232504829 learned_addr learned my addr 172.21.15.31:0/3232504829 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:11.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a0259c640 1 --2- 172.21.15.31:0/3232504829 >> v2:172.21.15.97:3300/0 conn(0x7f7a040cdfb0 0x7f7a040abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a02d9d640 1 -- 172.21.15.31:0/3232504829 >> v2:172.21.15.97:3300/0 conn(0x7f7a040cdfb0 msgr2=0x7f7a040abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:11.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a02d9d640 1 --2- 172.21.15.31:0/3232504829 >> v2:172.21.15.97:3300/0 conn(0x7f7a040cdfb0 0x7f7a040abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a02d9d640 1 -- 172.21.15.31:0/3232504829 >> v1:172.21.15.97:6789/0 conn(0x7f7a0415a710 legacy=0x7f7a040ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:11.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a02d9d640 1 -- 172.21.15.31:0/3232504829 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a040b2cd0 con 0x7f7a040cbd50 2023-11-26T07:18:11.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a02d9d640 1 --2- 172.21.15.31:0/3232504829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cbd50 0x7f7a040cc1e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a0406c270 tx=0x7f79fc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa11c89642d5baf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:11.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a01d9b640 1 -- 172.21.15.31:0/3232504829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0a5880 con 0x7f7a040cbd50 2023-11-26T07:18:11.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a01d9b640 1 -- 172.21.15.31:0/3232504829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc0a59e0 con 0x7f7a040cbd50 2023-11-26T07:18:11.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.432+0000 7f7a01d9b640 1 -- 172.21.15.31:0/3232504829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0a5d70 con 0x7f7a040cbd50 2023-11-26T07:18:11.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 -- 172.21.15.31:0/3232504829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cbd50 msgr2=0x7f7a040cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 --2- 172.21.15.31:0/3232504829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cbd50 0x7f7a040cc1e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a0406c270 tx=0x7f79fc01c9a0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 -- 172.21.15.31:0/3232504829 shutdown_connections 2023-11-26T07:18:11.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 --2- 172.21.15.31:0/3232504829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cbd50 0x7f7a040cc1e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 --2- 172.21.15.31:0/3232504829 >> v2:172.21.15.97:3300/0 conn(0x7f7a040cdfb0 0x7f7a040abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 -- 172.21.15.31:0/3232504829 >> 172.21.15.31:0/3232504829 conn(0x7f7a040b0cb0 msgr2=0x7f7a040b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 -- 172.21.15.31:0/3232504829 shutdown_connections 2023-11-26T07:18:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 -- 172.21.15.31:0/3232504829 wait complete. 2023-11-26T07:18:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 Processor -- start 2023-11-26T07:18:11.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 -- start start 2023-11-26T07:18:11.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a040cbd50 0x7f7a0415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cdfb0 0x7f7a04153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a0415a710 0x7f7a04153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0415ca80 con 0x7f7a040cdfb0 2023-11-26T07:18:11.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0415b610 con 0x7f7a0415a710 2023-11-26T07:18:11.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a08da1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0415cd00 con 0x7f7a040cbd50 2023-11-26T07:18:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cdfb0 0x7f7a04153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cdfb0 0x7f7a04153a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45568/0 (socket says 172.21.15.31:45568) 2023-11-26T07:18:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f79fbfff640 1 -- 172.21.15.31:0/3321577002 learned_addr learned my addr 172.21.15.31:0/3321577002 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a0259c640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a040cbd50 0x7f7a0415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.436+0000 7f7a02d9d640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a0415a710 0x7f7a04153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79fbfff640 1 -- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a040cbd50 msgr2=0x7f7a0415b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79fbfff640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a040cbd50 0x7f7a0415b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79fbfff640 1 -- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a0415a710 msgr2=0x7f7a04153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79fbfff640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a0415a710 0x7f7a04153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79fbfff640 1 -- 172.21.15.31:0/3321577002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc009b30 con 0x7f7a040cdfb0 2023-11-26T07:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79fbfff640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cdfb0 0x7f7a04153a30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c390 tx=0x7f79ec00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79faffd640 1 -- 172.21.15.31:0/3321577002 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0052d0 con 0x7f7a040cdfb0 2023-11-26T07:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79faffd640 1 -- 172.21.15.31:0/3321577002 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec005430 con 0x7f7a040cdfb0 2023-11-26T07:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79faffd640 1 -- 172.21.15.31:0/3321577002 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec003820 con 0x7f7a040cdfb0 2023-11-26T07:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f7a08da1640 1 -- 172.21.15.31:0/3321577002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0415cfe0 con 0x7f7a040cdfb0 2023-11-26T07:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f7a08da1640 1 -- 172.21.15.31:0/3321577002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0415d5a0 con 0x7f7a040cdfb0 2023-11-26T07:18:11.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f7a08da1640 1 -- 172.21.15.31:0/3321577002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a040671f0 con 0x7f7a040cdfb0 2023-11-26T07:18:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79faffd640 1 -- 172.21.15.31:0/3321577002 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f79ec003980 con 0x7f7a040cdfb0 2023-11-26T07:18:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79faffd640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79e4066470 0x7f79e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f79faffd640 1 -- 172.21.15.31:0/3321577002 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79ec0833e0 con 0x7f7a040cdfb0 2023-11-26T07:18:11.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.440+0000 7f7a0259c640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79e4066470 0x7f79e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.444+0000 7f7a0259c640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79e4066470 0x7f79e4068930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7a04156500 tx=0x7f79f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:11.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.444+0000 7f79faffd640 1 -- 172.21.15.31:0/3321577002 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79ec050870 con 0x7f7a040cdfb0 2023-11-26T07:18:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 -- 172.21.15.31:0/3321577002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a0406b460 con 0x7f7a040cdfb0 2023-11-26T07:18:11.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f79faffd640 1 -- 172.21.15.31:0/3321577002 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79ec053e90 con 0x7f7a040cdfb0 2023-11-26T07:18:11.610 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:11.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 -- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79e4066470 msgr2=0x7f79e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79e4066470 0x7f79e4068930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7a04156500 tx=0x7f79f4008040 comp rx=0 tx=0).stop 2023-11-26T07:18:11.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 -- 172.21.15.31:0/3321577002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cdfb0 msgr2=0x7f7a04153a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cdfb0 0x7f7a04153a30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c390 tx=0x7f79ec00c850 comp rx=0 tx=0).stop 2023-11-26T07:18:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 -- 172.21.15.31:0/3321577002 shutdown_connections 2023-11-26T07:18:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79e4066470 0x7f79e4068930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a0415a710 0x7f7a04153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a040cdfb0 0x7f7a04153a30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 --2- 172.21.15.31:0/3321577002 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a040cbd50 0x7f7a0415b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 -- 172.21.15.31:0/3321577002 >> 172.21.15.31:0/3321577002 conn(0x7f7a040b0cb0 msgr2=0x7f7a0406cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 -- 172.21.15.31:0/3321577002 shutdown_connections 2023-11-26T07:18:11.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.604+0000 7f7a08da1640 1 -- 172.21.15.31:0/3321577002 wait complete. 2023-11-26T07:18:11.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:11.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f580d757640 1 Processor -- start 2023-11-26T07:18:11.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f580d757640 1 -- start start 2023-11-26T07:18:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f580d757640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5808161f20 0x7f58080cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f580d757640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58080d0740 con 0x7f58080d02f0 2023-11-26T07:18:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f580d757640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58080d11a0 con 0x7f580815d7c0 2023-11-26T07:18:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f580d757640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58080d0f90 con 0x7f5808161f20 2023-11-26T07:18:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f57fffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5808161f20 0x7f58080cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f57fffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5808161f20 0x7f58080cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45576/0 (socket says 172.21.15.31:45576) 2023-11-26T07:18:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f57fffff640 1 -- 172.21.15.31:0/2191830029 learned_addr learned my addr 172.21.15.31:0/2191830029 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f58077fe640 1 --1- 172.21.15.31:0/2191830029 >> v1:172.21.15.31:6789/0 conn(0x7f58080d02f0 0x7f58080cdf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52086/0 (socket says 172.21.15.31:52086) 2023-11-26T07:18:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f58067fc640 1 -- 172.21.15.31:0/2191830029 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418057825 0 0) 0x7f58000014a0 con 0x7f58080d02f0 2023-11-26T07:18:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f58067fc640 1 -- 172.21.15.31:0/2191830029 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57f4003530 con 0x7f58080d02f0 2023-11-26T07:18:11.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f57fffff640 1 -- 172.21.15.31:0/2191830029 >> v1:172.21.15.97:6789/0 conn(0x7f580815d7c0 legacy=0x7f58080d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f58067fc640 1 -- 172.21.15.31:0/2191830029 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260581808 0 0) 0x7f58080d11a0 con 0x7f580815d7c0 2023-11-26T07:18:11.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f57fffff640 1 -- 172.21.15.31:0/2191830029 >> v1:172.21.15.31:6789/0 conn(0x7f58080d02f0 legacy=0x7f58080cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.772+0000 7f57fffff640 1 -- 172.21.15.31:0/2191830029 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080d1480 con 0x7f5808161f20 2023-11-26T07:18:11.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f57fffff640 1 --2- 172.21.15.31:0/2191830029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808161f20 0x7f58080cfdb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57f0006fd0 tx=0x7f57f001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315aaf16abcfd5ed server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:11.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f58067fc640 1 -- 172.21.15.31:0/2191830029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00ab660 con 0x7f5808161f20 2023-11-26T07:18:11.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f58067fc640 1 -- 172.21.15.31:0/2191830029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f00ab7c0 con 0x7f5808161f20 2023-11-26T07:18:11.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f58067fc640 1 -- 172.21.15.31:0/2191830029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00abb50 con 0x7f5808161f20 2023-11-26T07:18:11.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f580d757640 1 -- 172.21.15.31:0/2191830029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808161f20 msgr2=0x7f58080cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f580d757640 1 --2- 172.21.15.31:0/2191830029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808161f20 0x7f58080cfdb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57f0006fd0 tx=0x7f57f001c460 comp rx=0 tx=0).stop 2023-11-26T07:18:11.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f580d757640 1 -- 172.21.15.31:0/2191830029 shutdown_connections 2023-11-26T07:18:11.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f580d757640 1 --2- 172.21.15.31:0/2191830029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5808161f20 0x7f58080cfdb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f580d757640 1 -- 172.21.15.31:0/2191830029 >> 172.21.15.31:0/2191830029 conn(0x7f580808afc0 msgr2=0x7f58080d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:11.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f580d757640 1 -- 172.21.15.31:0/2191830029 shutdown_connections 2023-11-26T07:18:11.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.776+0000 7f580d757640 1 -- 172.21.15.31:0/2191830029 wait complete. 2023-11-26T07:18:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f580d757640 1 Processor -- start 2023-11-26T07:18:11.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f580d757640 1 -- start start 2023-11-26T07:18:11.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f580d757640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58080d02f0 0x7f5808153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f580d757640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f580815d7c0 0x7f58080b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f580d757640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58080ccc60 0x7f58080ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f580d757640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080b37d0 con 0x7f58080ccc60 2023-11-26T07:18:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f580d757640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58080b2b50 con 0x7f58080d02f0 2023-11-26T07:18:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f580d757640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58080b2e30 con 0x7f580815d7c0 2023-11-26T07:18:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f58077fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58080ccc60 0x7f58080ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f58077fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58080ccc60 0x7f58080ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45588/0 (socket says 172.21.15.31:45588) 2023-11-26T07:18:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f58077fe640 1 -- 172.21.15.31:0/2083181053 learned_addr learned my addr 172.21.15.31:0/2083181053 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f5806ffd640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58080d02f0 0x7f5808153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f57fffff640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f580815d7c0 0x7f58080b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f58077fe640 1 -- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f580815d7c0 msgr2=0x7f58080b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f58077fe640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f580815d7c0 0x7f58080b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f58077fe640 1 -- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58080d02f0 msgr2=0x7f5808153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f58077fe640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58080d02f0 0x7f5808153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f58077fe640 1 -- 172.21.15.31:0/2083181053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0009480 con 0x7f58080ccc60 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f58077fe640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58080ccc60 0x7f58080ca490 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f580000c7c0 tx=0x7f580000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f57ff7fe640 1 -- 172.21.15.31:0/2083181053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580000a640 con 0x7f58080ccc60 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f57ff7fe640 1 -- 172.21.15.31:0/2083181053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580000a7a0 con 0x7f58080ccc60 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f580d757640 1 -- 172.21.15.31:0/2083181053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58080b3110 con 0x7f58080ccc60 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f57ff7fe640 1 -- 172.21.15.31:0/2083181053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580000abb0 con 0x7f58080ccc60 2023-11-26T07:18:11.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.780+0000 7f580d757640 1 -- 172.21.15.31:0/2083181053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58080b3bf0 con 0x7f58080ccc60 2023-11-26T07:18:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.784+0000 7f580d757640 1 -- 172.21.15.31:0/2083181053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58080b3910 con 0x7f58080ccc60 2023-11-26T07:18:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.784+0000 7f57ff7fe640 1 -- 172.21.15.31:0/2083181053 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f580000ad10 con 0x7f58080ccc60 2023-11-26T07:18:11.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.784+0000 7f57ff7fe640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f57e8066470 0x7f57e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.784+0000 7f57ff7fe640 1 -- 172.21.15.31:0/2083181053 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5800083550 con 0x7f58080ccc60 2023-11-26T07:18:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.784+0000 7f5806ffd640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f57e8066470 0x7f57e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:11.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.784+0000 7f5806ffd640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f57e8066470 0x7f57e8068930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57f8000f70 tx=0x7f57f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:11.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.788+0000 7f57ff7fe640 1 -- 172.21.15.31:0/2083181053 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f58080b3910 con 0x7f58080ccc60 2023-11-26T07:18:11.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.948+0000 7f580d757640 1 -- 172.21.15.31:0/2083181053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58080d0740 con 0x7f58080ccc60 2023-11-26T07:18:11.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.948+0000 7f57ff7fe640 1 -- 172.21.15.31:0/2083181053 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58080d0740 con 0x7f58080ccc60 2023-11-26T07:18:11.955 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:18:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 -- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f57e8066470 msgr2=0x7f57e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f57e8066470 0x7f57e8068930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57f8000f70 tx=0x7f57f8009040 comp rx=0 tx=0).stop 2023-11-26T07:18:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 -- 172.21.15.31:0/2083181053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58080ccc60 msgr2=0x7f58080ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58080ccc60 0x7f58080ca490 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f580000c7c0 tx=0x7f580000cc80 comp rx=0 tx=0).stop 2023-11-26T07:18:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 -- 172.21.15.31:0/2083181053 shutdown_connections 2023-11-26T07:18:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f57e8066470 0x7f57e8068930 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58080ccc60 0x7f58080ca490 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f580815d7c0 0x7f58080b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 --2- 172.21.15.31:0/2083181053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58080d02f0 0x7f5808153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 -- 172.21.15.31:0/2083181053 >> 172.21.15.31:0/2083181053 conn(0x7f580808afc0 msgr2=0x7f58080cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 -- 172.21.15.31:0/2083181053 shutdown_connections 2023-11-26T07:18:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:11.952+0000 7f580d757640 1 -- 172.21.15.31:0/2083181053 wait complete. 2023-11-26T07:18:11.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:12.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.116+0000 7fef7e632640 1 Processor -- start 2023-11-26T07:18:12.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.116+0000 7fef7e632640 1 -- start start 2023-11-26T07:18:12.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef7e632640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fef7815d7c0 0x7fef780d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef7e632640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef780d04c0 0x7fef780cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef7e632640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef780d11a0 con 0x7fef78161f20 2023-11-26T07:18:12.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef7e632640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef780d0f90 con 0x7fef7815d7c0 2023-11-26T07:18:12.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef7e632640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef780d1480 con 0x7fef780d04c0 2023-11-26T07:18:12.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef777fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fef78161f20 0x7fef780cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52098/0 (socket says 172.21.15.31:52098) 2023-11-26T07:18:12.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef777fe640 1 -- 172.21.15.31:0/1799355427 learned_addr learned my addr 172.21.15.31:0/1799355427 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:12.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef7cbcf640 1 --2- 172.21.15.31:0/1799355427 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef780d04c0 0x7fef780cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef77fff640 1 --2- 172.21.15.31:0/1799355427 >> v2:172.21.15.97:3300/0 conn(0x7fef7815d7c0 0x7fef780d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef76ffd640 1 -- 172.21.15.31:0/1799355427 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902271900 0 0) 0x7fef780d11a0 con 0x7fef78161f20 2023-11-26T07:18:12.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef76ffd640 1 -- 172.21.15.31:0/1799355427 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef60003530 con 0x7fef78161f20 2023-11-26T07:18:12.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef77fff640 1 -- 172.21.15.31:0/1799355427 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef780d04c0 msgr2=0x7fef780cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:12.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef77fff640 1 --2- 172.21.15.31:0/1799355427 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef780d04c0 0x7fef780cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef77fff640 1 -- 172.21.15.31:0/1799355427 >> v1:172.21.15.31:6789/0 conn(0x7fef78161f20 legacy=0x7fef780cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.120+0000 7fef77fff640 1 -- 172.21.15.31:0/1799355427 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780d1760 con 0x7fef7815d7c0 2023-11-26T07:18:12.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef77fff640 1 --2- 172.21.15.31:0/1799355427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7815d7c0 0x7fef780d3d50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fef680097c0 tx=0x7fef6801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92e5423ec72dc56f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef76ffd640 1 -- 172.21.15.31:0/1799355427 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680ab650 con 0x7fef7815d7c0 2023-11-26T07:18:12.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef76ffd640 1 -- 172.21.15.31:0/1799355427 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef680ab7b0 con 0x7fef7815d7c0 2023-11-26T07:18:12.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef76ffd640 1 -- 172.21.15.31:0/1799355427 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680abb40 con 0x7fef7815d7c0 2023-11-26T07:18:12.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef7e632640 1 -- 172.21.15.31:0/1799355427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7815d7c0 msgr2=0x7fef780d3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef7e632640 1 --2- 172.21.15.31:0/1799355427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7815d7c0 0x7fef780d3d50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fef680097c0 tx=0x7fef6801c450 comp rx=0 tx=0).stop 2023-11-26T07:18:12.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef7e632640 1 -- 172.21.15.31:0/1799355427 shutdown_connections 2023-11-26T07:18:12.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef7e632640 1 --2- 172.21.15.31:0/1799355427 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef780d04c0 0x7fef780cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef7e632640 1 --2- 172.21.15.31:0/1799355427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7815d7c0 0x7fef780d3d50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef7e632640 1 -- 172.21.15.31:0/1799355427 >> 172.21.15.31:0/1799355427 conn(0x7fef7808afc0 msgr2=0x7fef780d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:12.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef7e632640 1 -- 172.21.15.31:0/1799355427 shutdown_connections 2023-11-26T07:18:12.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef7e632640 1 -- 172.21.15.31:0/1799355427 wait complete. 2023-11-26T07:18:12.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef7e632640 1 Processor -- start 2023-11-26T07:18:12.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.124+0000 7fef7e632640 1 -- start start 2023-11-26T07:18:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef7e632640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780d04c0 0x7fef780ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef7e632640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7815d7c0 0x7fef7815fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef7e632640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef78160320 0x7fef780b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef7e632640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef781607b0 con 0x7fef780d04c0 2023-11-26T07:18:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef7e632640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78155490 con 0x7fef7815d7c0 2023-11-26T07:18:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef7e632640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7815adc0 con 0x7fef78160320 2023-11-26T07:18:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef77fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780d04c0 0x7fef780ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef77fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780d04c0 0x7fef780ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45594/0 (socket says 172.21.15.31:45594) 2023-11-26T07:18:12.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef77fff640 1 -- 172.21.15.31:0/506835053 learned_addr learned my addr 172.21.15.31:0/506835053 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:12.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef77fff640 1 -- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef78160320 msgr2=0x7fef780b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef77fff640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef78160320 0x7fef780b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef77fff640 1 -- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7815d7c0 msgr2=0x7fef7815fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef77fff640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7815d7c0 0x7fef7815fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef77fff640 1 -- 172.21.15.31:0/506835053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef68009420 con 0x7fef780d04c0 2023-11-26T07:18:12.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef77fff640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780d04c0 0x7fef780ccb60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fef680ad040 tx=0x7fef6801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef74ff9640 1 -- 172.21.15.31:0/506835053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680bf610 con 0x7fef780d04c0 2023-11-26T07:18:12.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef74ff9640 1 -- 172.21.15.31:0/506835053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef680bf7b0 con 0x7fef780d04c0 2023-11-26T07:18:12.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef74ff9640 1 -- 172.21.15.31:0/506835053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680bfb60 con 0x7fef780d04c0 2023-11-26T07:18:12.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef7e632640 1 -- 172.21.15.31:0/506835053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7815aa90 con 0x7fef780d04c0 2023-11-26T07:18:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef7e632640 1 -- 172.21.15.31:0/506835053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef780b78b0 con 0x7fef780d04c0 2023-11-26T07:18:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.128+0000 7fef7e632640 1 -- 172.21.15.31:0/506835053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7804e850 con 0x7fef780d04c0 2023-11-26T07:18:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.132+0000 7fef74ff9640 1 -- 172.21.15.31:0/506835053 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fef680bfcc0 con 0x7fef780d04c0 2023-11-26T07:18:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.132+0000 7fef74ff9640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fef64066470 0x7fef64068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.132+0000 7fef74ff9640 1 -- 172.21.15.31:0/506835053 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef6811e7f0 con 0x7fef780d04c0 2023-11-26T07:18:12.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.132+0000 7fef777fe640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fef64066470 0x7fef64068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.132+0000 7fef777fe640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fef64066470 0x7fef64068930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fef6c001370 tx=0x7fef6c0077d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.136+0000 7fef74ff9640 1 -- 172.21.15.31:0/506835053 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fef680ebc80 con 0x7fef780d04c0 2023-11-26T07:18:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 -- 172.21.15.31:0/506835053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fef780b9340 con 0x7fef780d04c0 2023-11-26T07:18:12.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef74ff9640 1 -- 172.21.15.31:0/506835053 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fef680ef2a0 con 0x7fef780d04c0 2023-11-26T07:18:12.301 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:18:12.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 -- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fef64066470 msgr2=0x7fef64068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fef64066470 0x7fef64068930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fef6c001370 tx=0x7fef6c0077d0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 -- 172.21.15.31:0/506835053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780d04c0 msgr2=0x7fef780ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780d04c0 0x7fef780ccb60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fef680ad040 tx=0x7fef6801ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:12.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 -- 172.21.15.31:0/506835053 shutdown_connections 2023-11-26T07:18:12.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fef64066470 0x7fef64068930 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef78160320 0x7fef780b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7815d7c0 0x7fef7815fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 --2- 172.21.15.31:0/506835053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef780d04c0 0x7fef780ccb60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 -- 172.21.15.31:0/506835053 >> 172.21.15.31:0/506835053 conn(0x7fef7808afc0 msgr2=0x7fef7815df80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:12.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 -- 172.21.15.31:0/506835053 shutdown_connections 2023-11-26T07:18:12.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.296+0000 7fef7e632640 1 -- 172.21.15.31:0/506835053 wait complete. 2023-11-26T07:18:12.323 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:12.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.460+0000 7f16b2017640 1 Processor -- start 2023-11-26T07:18:12.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.460+0000 7f16b2017640 1 -- start start 2023-11-26T07:18:12.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16b2017640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0ae3d0 0x7f16ac06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16b2017640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac0ae7f0 con 0x7f16ac143ae0 2023-11-26T07:18:12.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16b2017640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac06e980 con 0x7f16ac06dfd0 2023-11-26T07:18:12.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16b2017640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac070670 con 0x7f16ac0ae3d0 2023-11-26T07:18:12.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16ab7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f16ac143ae0 0x7f16ac0b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52102/0 (socket says 172.21.15.31:52102) 2023-11-26T07:18:12.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16ab7fe640 1 -- 172.21.15.31:0/752762421 learned_addr learned my addr 172.21.15.31:0/752762421 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:12.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16abfff640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0ae3d0 0x7f16ac06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249568431 0 0) 0x7f16ac0ae7f0 con 0x7f16ac143ae0 2023-11-26T07:18:12.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1694003530 con 0x7f16ac143ae0 2023-11-26T07:18:12.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1440001678 0 0) 0x7f1694003530 con 0x7f16ac143ae0 2023-11-26T07:18:12.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0ae3d0 msgr2=0x7f16ac06e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0ae3d0 0x7f16ac06e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 >> v1:172.21.15.97:6789/0 conn(0x7f16ac06dfd0 legacy=0x7f16ac0adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac070950 con 0x7f16ac143ae0 2023-11-26T07:18:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f169c001680 con 0x7f16ac143ae0 2023-11-26T07:18:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 >> v1:172.21.15.31:6789/0 conn(0x7f16ac143ae0 legacy=0x7f16ac0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16940039b0 0x7f1694003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1694004360 0x7f1694004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0ae3d0 0x7f1694005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac070670 con 0x7f16940039b0 2023-11-26T07:18:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169400dbd0 con 0x7f1694004360 2023-11-26T07:18:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169400dd90 con 0x7f16ac0ae3d0 2023-11-26T07:18:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16ab7fe640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16940039b0 0x7f1694003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16aaffd640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1694004360 0x7f1694004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16abfff640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0ae3d0 0x7f1694005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16ab7fe640 1 -- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0ae3d0 msgr2=0x7f1694005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:12.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16ab7fe640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0ae3d0 0x7f1694005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16ab7fe640 1 -- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1694004360 msgr2=0x7f1694004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:12.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16ab7fe640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1694004360 0x7f1694004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.464+0000 7f16ab7fe640 1 -- 172.21.15.31:0/752762421 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1694003530 con 0x7f16940039b0 2023-11-26T07:18:12.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16ab7fe640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16940039b0 0x7f1694003d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c190 tx=0x7f169c01cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88be918a0b9207b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c005ae0 con 0x7f16940039b0 2023-11-26T07:18:12.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c005c60 con 0x7f16940039b0 2023-11-26T07:18:12.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16aa7fc640 1 -- 172.21.15.31:0/752762421 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0a57a0 con 0x7f16940039b0 2023-11-26T07:18:12.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16b2017640 1 -- 172.21.15.31:0/752762421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16940039b0 msgr2=0x7f1694003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16b2017640 1 --2- 172.21.15.31:0/752762421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16940039b0 0x7f1694003d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c190 tx=0x7f169c01cbb0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16abfff640 1 -- 172.21.15.31:0/752762421 reap_dead start 2023-11-26T07:18:12.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16b2017640 1 -- 172.21.15.31:0/752762421 shutdown_connections 2023-11-26T07:18:12.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16b2017640 1 -- 172.21.15.31:0/752762421 >> 172.21.15.31:0/752762421 conn(0x7f16ac0b0c40 msgr2=0x7f16ac0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:12.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16b2017640 1 -- 172.21.15.31:0/752762421 shutdown_connections 2023-11-26T07:18:12.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16b2017640 1 -- 172.21.15.31:0/752762421 wait complete. 2023-11-26T07:18:12.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16b2017640 1 Processor -- start 2023-11-26T07:18:12.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.468+0000 7f16b2017640 1 -- start start 2023-11-26T07:18:12.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16b2017640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16940039b0 0x7f16ac09abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16b2017640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac13de90 0x7f16ac1f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16b2017640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac13e310 0x7f16ac1f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16b2017640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac09b110 con 0x7f16ac13e310 2023-11-26T07:18:12.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16b2017640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac13e800 con 0x7f16ac13de90 2023-11-26T07:18:12.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16b2017640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac13eae0 con 0x7f16940039b0 2023-11-26T07:18:12.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac13e310 0x7f16ac1f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16abfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac13e310 0x7f16ac1f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45602/0 (socket says 172.21.15.31:45602) 2023-11-26T07:18:12.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16abfff640 1 -- 172.21.15.31:0/787656194 learned_addr learned my addr 172.21.15.31:0/787656194 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:12.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16aaffd640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac13de90 0x7f16ac1f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16ab7fe640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16940039b0 0x7f16ac09abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16abfff640 1 -- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16940039b0 msgr2=0x7f16ac09abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16abfff640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16940039b0 0x7f16ac09abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16abfff640 1 -- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac13de90 msgr2=0x7f16ac1f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16abfff640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac13de90 0x7f16ac1f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16abfff640 1 -- 172.21.15.31:0/787656194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac13edc0 con 0x7f16ac13e310 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16aaffd640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac13de90 0x7f16ac1f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16abfff640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac13e310 0x7f16ac1f62e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f16ac15dcd0 tx=0x7f16a400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f1693fff640 1 -- 172.21.15.31:0/787656194 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a400e040 con 0x7f16ac13e310 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f1693fff640 1 -- 172.21.15.31:0/787656194 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a400f040 con 0x7f16ac13e310 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16b2017640 1 -- 172.21.15.31:0/787656194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac13f0a0 con 0x7f16ac13e310 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f1693fff640 1 -- 172.21.15.31:0/787656194 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4015670 con 0x7f16ac13e310 2023-11-26T07:18:12.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16b2017640 1 -- 172.21.15.31:0/787656194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac1f6820 con 0x7f16ac13e310 2023-11-26T07:18:12.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.472+0000 7f16b2017640 1 -- 172.21.15.31:0/787656194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ac04e850 con 0x7f16ac13e310 2023-11-26T07:18:12.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.476+0000 7f1693fff640 1 -- 172.21.15.31:0/787656194 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f16a40157d0 con 0x7f16ac13e310 2023-11-26T07:18:12.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.476+0000 7f1693fff640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16980664a0 0x7f1698068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.476+0000 7f1693fff640 1 -- 172.21.15.31:0/787656194 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16a4084660 con 0x7f16ac13e310 2023-11-26T07:18:12.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.476+0000 7f16ab7fe640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16980664a0 0x7f1698068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.476+0000 7f16ab7fe640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16980664a0 0x7f1698068960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f169c0ab000 tx=0x7f169c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.480+0000 7f1693fff640 1 -- 172.21.15.31:0/787656194 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f16a4051a40 con 0x7f16ac13e310 2023-11-26T07:18:12.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.668+0000 7f16b2017640 1 -- 172.21.15.31:0/787656194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16ac13f550 con 0x7f16ac13e310 2023-11-26T07:18:12.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.668+0000 7f1693fff640 1 -- 172.21.15.31:0/787656194 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f16a4055060 con 0x7f16ac13e310 2023-11-26T07:18:12.675 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:12.675 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4099326271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4099326271},{"type":"v1","addr":"172.21.15.31:6835","nonce":4099326271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2404164953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2404164953},{"type":"v1","addr":"172.21.15.97:6836","nonce":2404164953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1312969281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1312969281},{"type":"v1","addr":"172.21.15.31:6834","nonce":1312969281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:32.510772+0000","modified":"2023-11-26T07:17:35.514610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/527056216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":527056216},{"type":"v1","addr":"172.21.15.97:6837","nonce":527056216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:12.676 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-11-26T07:18:12.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 -- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16980664a0 msgr2=0x7f1698068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16980664a0 0x7f1698068960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f169c0ab000 tx=0x7f169c0a7070 comp rx=0 tx=0).stop 2023-11-26T07:18:12.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 -- 172.21.15.31:0/787656194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac13e310 msgr2=0x7f16ac1f62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac13e310 0x7f16ac1f62e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f16ac15dcd0 tx=0x7f16a400f580 comp rx=0 tx=0).stop 2023-11-26T07:18:12.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 -- 172.21.15.31:0/787656194 shutdown_connections 2023-11-26T07:18:12.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16980664a0 0x7f1698068960 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac13e310 0x7f16ac1f62e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac13de90 0x7f16ac1f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 --2- 172.21.15.31:0/787656194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16940039b0 0x7f16ac09abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 -- 172.21.15.31:0/787656194 >> 172.21.15.31:0/787656194 conn(0x7f16ac0b0c40 msgr2=0x7f16ac06d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:12.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 -- 172.21.15.31:0/787656194 shutdown_connections 2023-11-26T07:18:12.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.672+0000 7f16b2017640 1 -- 172.21.15.31:0/787656194 wait complete. 2023-11-26T07:18:12.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c4a72a640 1 Processor -- start 2023-11-26T07:18:12.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c4a72a640 1 -- start start 2023-11-26T07:18:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c4a72a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c4415d7b0 0x7f6c440d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c4a72a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c440cead0 0x7f6c440d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c4a72a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c440d1eb0 0x7f6c440cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c4a72a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440d2300 con 0x7f6c4415d7b0 2023-11-26T07:18:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c4a72a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440cefd0 con 0x7f6c440cead0 2023-11-26T07:18:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c4a72a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440cf2b0 con 0x7f6c440d1eb0 2023-11-26T07:18:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c43fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c4415d7b0 0x7f6c440d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c43fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c4415d7b0 0x7f6c440d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45612/0 (socket says 172.21.15.31:45612) 2023-11-26T07:18:12.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c43fff640 1 -- 172.21.15.31:0/1420465373 learned_addr learned my addr 172.21.15.31:0/1420465373 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:12.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c437fe640 1 --2- 172.21.15.31:0/1420465373 >> v2:172.21.15.97:3300/0 conn(0x7f6c440cead0 0x7f6c440d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c48cc7640 1 --2- 172.21.15.31:0/1420465373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c440d1eb0 0x7f6c440cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c43fff640 1 -- 172.21.15.31:0/1420465373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c440d1eb0 msgr2=0x7f6c440cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:12.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c43fff640 1 --2- 172.21.15.31:0/1420465373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c440d1eb0 0x7f6c440cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c43fff640 1 -- 172.21.15.31:0/1420465373 >> v2:172.21.15.97:3300/0 conn(0x7f6c440cead0 msgr2=0x7f6c440d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:12.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c43fff640 1 --2- 172.21.15.31:0/1420465373 >> v2:172.21.15.97:3300/0 conn(0x7f6c440cead0 0x7f6c440d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c43fff640 1 -- 172.21.15.31:0/1420465373 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4406b3f0 con 0x7f6c4415d7b0 2023-11-26T07:18:12.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c48cc7640 1 --2- 172.21.15.31:0/1420465373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c440d1eb0 0x7f6c440cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:12.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.836+0000 7f6c437fe640 1 --2- 172.21.15.31:0/1420465373 >> v2:172.21.15.97:3300/0 conn(0x7f6c440cead0 0x7f6c440d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:12.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c43fff640 1 --2- 172.21.15.31:0/1420465373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4415d7b0 0x7f6c440d10d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6c4408ba60 tx=0x7f6c3401c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52574c0e794f7bb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c42ffd640 1 -- 172.21.15.31:0/1420465373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c340ab600 con 0x7f6c4415d7b0 2023-11-26T07:18:12.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c42ffd640 1 -- 172.21.15.31:0/1420465373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c340a7050 con 0x7f6c4415d7b0 2023-11-26T07:18:12.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c42ffd640 1 -- 172.21.15.31:0/1420465373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c340ab980 con 0x7f6c4415d7b0 2023-11-26T07:18:12.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c4a72a640 1 -- 172.21.15.31:0/1420465373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4415d7b0 msgr2=0x7f6c440d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c4a72a640 1 --2- 172.21.15.31:0/1420465373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4415d7b0 0x7f6c440d10d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6c4408ba60 tx=0x7f6c3401c0f0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c4a72a640 1 -- 172.21.15.31:0/1420465373 shutdown_connections 2023-11-26T07:18:12.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c4a72a640 1 --2- 172.21.15.31:0/1420465373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c440d1eb0 0x7f6c440cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c4a72a640 1 --2- 172.21.15.31:0/1420465373 >> v2:172.21.15.97:3300/0 conn(0x7f6c440cead0 0x7f6c440d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c4a72a640 1 --2- 172.21.15.31:0/1420465373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c4415d7b0 0x7f6c440d10d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c4a72a640 1 -- 172.21.15.31:0/1420465373 >> 172.21.15.31:0/1420465373 conn(0x7f6c440d33b0 msgr2=0x7f6c440d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:12.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c4a72a640 1 -- 172.21.15.31:0/1420465373 shutdown_connections 2023-11-26T07:18:12.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.840+0000 7f6c4a72a640 1 -- 172.21.15.31:0/1420465373 wait complete. 2023-11-26T07:18:12.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c4a72a640 1 Processor -- start 2023-11-26T07:18:12.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c4a72a640 1 -- start start 2023-11-26T07:18:12.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c4a72a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c440cead0 0x7f6c44156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c4a72a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440d1eb0 0x7f6c4415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c4a72a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c4415d7b0 0x7f6c4415bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c4a72a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440c8c40 con 0x7f6c440d1eb0 2023-11-26T07:18:12.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c4a72a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440c8f20 con 0x7f6c440cead0 2023-11-26T07:18:12.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c4a72a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440c9200 con 0x7f6c4415d7b0 2023-11-26T07:18:12.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440d1eb0 0x7f6c4415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440d1eb0 0x7f6c4415b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45624/0 (socket says 172.21.15.31:45624) 2023-11-26T07:18:12.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c437fe640 1 -- 172.21.15.31:0/2741262517 learned_addr learned my addr 172.21.15.31:0/2741262517 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:12.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c48cc7640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c4415d7b0 0x7f6c4415bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c43fff640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c440cead0 0x7f6c44156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c437fe640 1 -- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c4415d7b0 msgr2=0x7f6c4415bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c437fe640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c4415d7b0 0x7f6c4415bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c437fe640 1 -- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c440cead0 msgr2=0x7f6c44156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:12.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c437fe640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c440cead0 0x7f6c44156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:12.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c437fe640 1 -- 172.21.15.31:0/2741262517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340094d0 con 0x7f6c440d1eb0 2023-11-26T07:18:12.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c437fe640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440d1eb0 0x7f6c4415b9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800c7f0 tx=0x7f6c3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c40ff9640 1 -- 172.21.15.31:0/2741262517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800cea0 con 0x7f6c440d1eb0 2023-11-26T07:18:12.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.844+0000 7f6c40ff9640 1 -- 172.21.15.31:0/2741262517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c38007a30 con 0x7f6c440d1eb0 2023-11-26T07:18:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.848+0000 7f6c4a72a640 1 -- 172.21.15.31:0/2741262517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c440c94e0 con 0x7f6c440d1eb0 2023-11-26T07:18:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.848+0000 7f6c40ff9640 1 -- 172.21.15.31:0/2741262517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38007dc0 con 0x7f6c440d1eb0 2023-11-26T07:18:12.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.848+0000 7f6c4a72a640 1 -- 172.21.15.31:0/2741262517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c44157020 con 0x7f6c440d1eb0 2023-11-26T07:18:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.848+0000 7f6c4a72a640 1 -- 172.21.15.31:0/2741262517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4404e850 con 0x7f6c440d1eb0 2023-11-26T07:18:12.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.848+0000 7f6c40ff9640 1 -- 172.21.15.31:0/2741262517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6c380113f0 con 0x7f6c440d1eb0 2023-11-26T07:18:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.848+0000 7f6c40ff9640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 0x7f6c30068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.848+0000 7f6c40ff9640 1 -- 172.21.15.31:0/2741262517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c38083fc0 con 0x7f6c440d1eb0 2023-11-26T07:18:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.848+0000 7f6c43fff640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 0x7f6c30068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:12.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.852+0000 7f6c43fff640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 0x7f6c30068930 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c340ad000 tx=0x7f6c34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:12.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:12.852+0000 7f6c40ff9640 1 -- 172.21.15.31:0/2741262517 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c380514e0 con 0x7f6c440d1eb0 2023-11-26T07:18:13.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.012+0000 7f6c4a72a640 1 -- 172.21.15.31:0/2741262517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c440d1890 con 0x7f6c440d1eb0 2023-11-26T07:18:13.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.012+0000 7f6c40ff9640 1 -- 172.21.15.31:0/2741262517 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16421 (secure 0 0 0) 0x7f6c38054b00 con 0x7f6c440d1eb0 2023-11-26T07:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:13.019 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:17:34.117596+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:13.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 -- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 msgr2=0x7f6c30068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 0x7f6c30068930 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c340ad000 tx=0x7f6c34008040 comp rx=0 tx=0).stop 2023-11-26T07:18:13.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 -- 172.21.15.31:0/2741262517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440d1eb0 msgr2=0x7f6c4415b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440d1eb0 0x7f6c4415b9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800c7f0 tx=0x7f6c3800ccb0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 -- 172.21.15.31:0/2741262517 shutdown_connections 2023-11-26T07:18:13.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c30066470 0x7f6c30068930 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c4415d7b0 0x7f6c4415bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440d1eb0 0x7f6c4415b9c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 --2- 172.21.15.31:0/2741262517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c440cead0 0x7f6c44156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 -- 172.21.15.31:0/2741262517 >> 172.21.15.31:0/2741262517 conn(0x7f6c440d33b0 msgr2=0x7f6c4406c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:13.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 -- 172.21.15.31:0/2741262517 shutdown_connections 2023-11-26T07:18:13.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.016+0000 7f6c4a72a640 1 -- 172.21.15.31:0/2741262517 wait complete. 2023-11-26T07:18:13.040 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:18:13.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:13.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.180+0000 7f8dfde5a640 1 Processor -- start 2023-11-26T07:18:13.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.180+0000 7f8dfde5a640 1 -- start start 2023-11-26T07:18:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8dfde5a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8df815f330 0x7f8df806c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8dfde5a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df815dba0 con 0x7f8df8070730 2023-11-26T07:18:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8dfde5a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df806c860 con 0x7f8df8146c60 2023-11-26T07:18:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8dfde5a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df806d100 con 0x7f8df815f330 2023-11-26T07:18:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df7fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8df815f330 0x7f8df806c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df6ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8df8070730 0x7f8df806bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52114/0 (socket says 172.21.15.31:52114) 2023-11-26T07:18:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df6ffd640 1 -- 172.21.15.31:0/3036500302 learned_addr learned my addr 172.21.15.31:0/3036500302 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df7fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8df815f330 0x7f8df806c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45636/0 (socket says 172.21.15.31:45636) 2023-11-26T07:18:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df7fff640 1 -- 172.21.15.31:0/3036500302 >> v1:172.21.15.97:6789/0 conn(0x7f8df8146c60 legacy=0x7f8df8070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df7fff640 1 -- 172.21.15.31:0/3036500302 >> v1:172.21.15.31:6789/0 conn(0x7f8df8070730 legacy=0x7f8df806bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df7fff640 1 -- 172.21.15.31:0/3036500302 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df806cb40 con 0x7f8df815f330 2023-11-26T07:18:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df7fff640 1 --2- 172.21.15.31:0/3036500302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df815f330 0x7f8df806c220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009d90 tx=0x7f8df001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4924e92d07febfa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df67fc640 1 -- 172.21.15.31:0/3036500302 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00a59a0 con 0x7f8df815f330 2023-11-26T07:18:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df67fc640 1 -- 172.21.15.31:0/3036500302 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df00a5b00 con 0x7f8df815f330 2023-11-26T07:18:13.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.184+0000 7f8df67fc640 1 -- 172.21.15.31:0/3036500302 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00a5e90 con 0x7f8df815f330 2023-11-26T07:18:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3036500302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df815f330 msgr2=0x7f8df806c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 --2- 172.21.15.31:0/3036500302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df815f330 0x7f8df806c220 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009d90 tx=0x7f8df001c8e0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3036500302 shutdown_connections 2023-11-26T07:18:13.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 --2- 172.21.15.31:0/3036500302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df815f330 0x7f8df806c220 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3036500302 >> 172.21.15.31:0/3036500302 conn(0x7f8df80b2ea0 msgr2=0x7f8df80b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3036500302 shutdown_connections 2023-11-26T07:18:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3036500302 wait complete. 2023-11-26T07:18:13.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 Processor -- start 2023-11-26T07:18:13.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 -- start start 2023-11-26T07:18:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df8070730 0x7f8df813a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df8146c60 0x7f8df815c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8df815f330 0x7f8df8145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8154c80 con 0x7f8df8070730 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8154f60 con 0x7f8df8146c60 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8dfde5a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8140ce0 con 0x7f8df815f330 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8df77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df8070730 0x7f8df813a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8df77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df8070730 0x7f8df813a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45640/0 (socket says 172.21.15.31:45640) 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.188+0000 7f8df77fe640 1 -- 172.21.15.31:0/3182511664 learned_addr learned my addr 172.21.15.31:0/3182511664 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8df6ffd640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df8146c60 0x7f8df815c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8df7fff640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8df815f330 0x7f8df8145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8df77fe640 1 -- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8df815f330 msgr2=0x7f8df8145020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8df77fe640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8df815f330 0x7f8df8145020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8df77fe640 1 -- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df8146c60 msgr2=0x7f8df815c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8df77fe640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df8146c60 0x7f8df815c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8df77fe640 1 -- 172.21.15.31:0/3182511664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df0009a70 con 0x7f8df8070730 2023-11-26T07:18:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8df77fe640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df8070730 0x7f8df813a540 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8de800c450 tx=0x7f8de800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8ddffff640 1 -- 172.21.15.31:0/3182511664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8012830 con 0x7f8df8070730 2023-11-26T07:18:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8ddffff640 1 -- 172.21.15.31:0/3182511664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de80129b0 con 0x7f8df8070730 2023-11-26T07:18:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8ddffff640 1 -- 172.21.15.31:0/3182511664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8012d60 con 0x7f8df8070730 2023-11-26T07:18:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3182511664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df8140fc0 con 0x7f8df8070730 2023-11-26T07:18:13.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3182511664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df813aa80 con 0x7f8df8070730 2023-11-26T07:18:13.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3182511664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df804e850 con 0x7f8df8070730 2023-11-26T07:18:13.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.192+0000 7f8ddffff640 1 -- 172.21.15.31:0/3182511664 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8de801c460 con 0x7f8df8070730 2023-11-26T07:18:13.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.196+0000 7f8ddffff640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8de4066470 0x7f8de4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.196+0000 7f8ddffff640 1 -- 172.21.15.31:0/3182511664 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8de8083b40 con 0x7f8df8070730 2023-11-26T07:18:13.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.196+0000 7f8df6ffd640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8de4066470 0x7f8de4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.196+0000 7f8df6ffd640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8de4066470 0x7f8de4068930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8df8155ab0 tx=0x7f8dec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:13.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.200+0000 7f8ddffff640 1 -- 172.21.15.31:0/3182511664 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8de8051070 con 0x7f8df8070730 2023-11-26T07:18:13.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.392+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3182511664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8df80d1f60 con 0x7f8df8070730 2023-11-26T07:18:13.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8ddffff640 1 -- 172.21.15.31:0/3182511664 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f8de8054690 con 0x7f8df8070730 2023-11-26T07:18:13.400 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:13.400 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4099326271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4099326271},{"type":"v1","addr":"172.21.15.31:6835","nonce":4099326271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2404164953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2404164953},{"type":"v1","addr":"172.21.15.97:6836","nonce":2404164953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1312969281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1312969281},{"type":"v1","addr":"172.21.15.31:6834","nonce":1312969281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:32.510772+0000","modified":"2023-11-26T07:17:35.514610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/527056216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":527056216},{"type":"v1","addr":"172.21.15.97:6837","nonce":527056216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:13.401 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-11-26T07:18:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8de4066470 msgr2=0x7f8de4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8de4066470 0x7f8de4068930 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8df8155ab0 tx=0x7f8dec009040 comp rx=0 tx=0).stop 2023-11-26T07:18:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3182511664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df8070730 msgr2=0x7f8df813a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df8070730 0x7f8df813a540 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8de800c450 tx=0x7f8de800c910 comp rx=0 tx=0).stop 2023-11-26T07:18:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3182511664 shutdown_connections 2023-11-26T07:18:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8de4066470 0x7f8de4068930 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8df815f330 0x7f8df8145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8df8146c60 0x7f8df815c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 --2- 172.21.15.31:0/3182511664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8df8070730 0x7f8df813a540 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3182511664 >> 172.21.15.31:0/3182511664 conn(0x7f8df80b2ea0 msgr2=0x7f8df80b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3182511664 shutdown_connections 2023-11-26T07:18:13.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.396+0000 7f8dfde5a640 1 -- 172.21.15.31:0/3182511664 wait complete. 2023-11-26T07:18:13.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:13.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff663580640 1 Processor -- start 2023-11-26T07:18:13.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff663580640 1 -- start start 2023-11-26T07:18:13.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff663580640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff65c06dfd0 0x7ff65c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff663580640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c0adcc0 0x7ff65c0ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff663580640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65c06e980 con 0x7ff65c143ae0 2023-11-26T07:18:13.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff663580640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65c070670 con 0x7ff65c06dfd0 2023-11-26T07:18:13.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff663580640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c070950 con 0x7ff65c0adcc0 2023-11-26T07:18:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff660b1b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff65c06dfd0 0x7ff65c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff660b1b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff65c06dfd0 0x7ff65c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45654/0 (socket says 172.21.15.31:45654) 2023-11-26T07:18:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff660b1b640 1 -- 172.21.15.31:0/3037527929 learned_addr learned my addr 172.21.15.31:0/3037527929 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff661b1d640 1 --2- 172.21.15.31:0/3037527929 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c0adcc0 0x7ff65c0ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff660b1b640 1 -- 172.21.15.31:0/3037527929 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c0adcc0 msgr2=0x7ff65c0ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff660b1b640 1 --2- 172.21.15.31:0/3037527929 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c0adcc0 0x7ff65c0ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff660b1b640 1 -- 172.21.15.31:0/3037527929 >> v1:172.21.15.97:6789/0 conn(0x7ff65c143ae0 legacy=0x7ff65c0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:13.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff660b1b640 1 -- 172.21.15.31:0/3037527929 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c070c30 con 0x7ff65c06dfd0 2023-11-26T07:18:13.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.560+0000 7ff661b1d640 1 --2- 172.21.15.31:0/3037527929 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c0adcc0 0x7ff65c0ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:13.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff660b1b640 1 --2- 172.21.15.31:0/3037527929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c06dfd0 0x7ff65c06e440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009870 tx=0x7ff64c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a88ff86269360f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:13.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff653fff640 1 -- 172.21.15.31:0/3037527929 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c0aa5f0 con 0x7ff65c06dfd0 2023-11-26T07:18:13.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff653fff640 1 -- 172.21.15.31:0/3037527929 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64c0aa750 con 0x7ff65c06dfd0 2023-11-26T07:18:13.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff653fff640 1 -- 172.21.15.31:0/3037527929 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c0aaae0 con 0x7ff65c06dfd0 2023-11-26T07:18:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff663580640 1 -- 172.21.15.31:0/3037527929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c06dfd0 msgr2=0x7ff65c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff663580640 1 --2- 172.21.15.31:0/3037527929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c06dfd0 0x7ff65c06e440 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009870 tx=0x7ff64c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff663580640 1 -- 172.21.15.31:0/3037527929 shutdown_connections 2023-11-26T07:18:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff663580640 1 --2- 172.21.15.31:0/3037527929 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c0adcc0 0x7ff65c0ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff663580640 1 --2- 172.21.15.31:0/3037527929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c06dfd0 0x7ff65c06e440 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff663580640 1 -- 172.21.15.31:0/3037527929 >> 172.21.15.31:0/3037527929 conn(0x7ff65c0b0c40 msgr2=0x7ff65c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:13.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff663580640 1 -- 172.21.15.31:0/3037527929 shutdown_connections 2023-11-26T07:18:13.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.564+0000 7ff663580640 1 -- 172.21.15.31:0/3037527929 wait complete. 2023-11-26T07:18:13.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 Processor -- start 2023-11-26T07:18:13.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 -- start start 2023-11-26T07:18:13.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c06dfd0 0x7ff65c158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c0adcc0 0x7ff65c14a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c143ae0 0x7ff65c15b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c159f20 con 0x7ff65c0adcc0 2023-11-26T07:18:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c14aec0 con 0x7ff65c143ae0 2023-11-26T07:18:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c1592a0 con 0x7ff65c06dfd0 2023-11-26T07:18:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff660b1b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c0adcc0 0x7ff65c14a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff660b1b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c0adcc0 0x7ff65c14a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45656/0 (socket says 172.21.15.31:45656) 2023-11-26T07:18:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff660b1b640 1 -- 172.21.15.31:0/1937955484 learned_addr learned my addr 172.21.15.31:0/1937955484 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff66131c640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c06dfd0 0x7ff65c158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff661b1d640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c143ae0 0x7ff65c15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff660b1b640 1 -- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c06dfd0 msgr2=0x7ff65c158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff660b1b640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c06dfd0 0x7ff65c158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff660b1b640 1 -- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c143ae0 msgr2=0x7ff65c15b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff660b1b640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c143ae0 0x7ff65c15b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff660b1b640 1 -- 172.21.15.31:0/1937955484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c0094d0 con 0x7ff65c0adcc0 2023-11-26T07:18:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff660b1b640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c0adcc0 0x7ff65c14a980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0ac040 tx=0x7ff64c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:13.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff651ffb640 1 -- 172.21.15.31:0/1937955484 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c0b3800 con 0x7ff65c0adcc0 2023-11-26T07:18:13.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff651ffb640 1 -- 172.21.15.31:0/1937955484 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64c0b3960 con 0x7ff65c0adcc0 2023-11-26T07:18:13.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 -- 172.21.15.31:0/1937955484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65c159580 con 0x7ff65c0adcc0 2023-11-26T07:18:13.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff651ffb640 1 -- 172.21.15.31:0/1937955484 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c0b3d10 con 0x7ff65c0adcc0 2023-11-26T07:18:13.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 -- 172.21.15.31:0/1937955484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c159b40 con 0x7ff65c0adcc0 2023-11-26T07:18:13.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.568+0000 7ff663580640 1 -- 172.21.15.31:0/1937955484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65c1395e0 con 0x7ff65c0adcc0 2023-11-26T07:18:13.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.572+0000 7ff651ffb640 1 -- 172.21.15.31:0/1937955484 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff64c0c6440 con 0x7ff65c0adcc0 2023-11-26T07:18:13.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.572+0000 7ff651ffb640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff640066470 0x7ff640068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.572+0000 7ff651ffb640 1 -- 172.21.15.31:0/1937955484 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5804+0+0 (secure 0 0 0) 0x7ff64c0a7070 con 0x7ff65c0adcc0 2023-11-26T07:18:13.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.572+0000 7ff66131c640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff640066470 0x7ff640068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.572+0000 7ff66131c640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff640066470 0x7ff640068930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff65800b810 tx=0x7ff658008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:13.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.576+0000 7ff651ffb640 1 -- 172.21.15.31:0/1937955484 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff64c0ebfa0 con 0x7ff65c0adcc0 2023-11-26T07:18:13.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 -- 172.21.15.31:0/1937955484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff65c04e850 con 0x7ff65c0adcc0 2023-11-26T07:18:13.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff651ffb640 1 -- 172.21.15.31:0/1937955484 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7ff64c0ef5c0 con 0x7ff65c0adcc0 2023-11-26T07:18:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:13.774 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4099326271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4099326271},{"type":"v1","addr":"172.21.15.31:6835","nonce":4099326271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2404164953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2404164953},{"type":"v1","addr":"172.21.15.97:6836","nonce":2404164953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1312969281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1312969281},{"type":"v1","addr":"172.21.15.31:6834","nonce":1312969281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:17:32.510772+0000","modified":"2023-11-26T07:17:35.514610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6837/527056216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":527056216},{"type":"v1","addr":"172.21.15.97:6837","nonce":527056216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-26T07:18:13.774 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 8 2023-11-26T07:18:13.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 -- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff640066470 msgr2=0x7ff640068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff640066470 0x7ff640068930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff65800b810 tx=0x7ff658008040 comp rx=0 tx=0).stop 2023-11-26T07:18:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 -- 172.21.15.31:0/1937955484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c0adcc0 msgr2=0x7ff65c14a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c0adcc0 0x7ff65c14a980 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff64c0ac040 tx=0x7ff64c01ad10 comp rx=0 tx=0).stop 2023-11-26T07:18:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 -- 172.21.15.31:0/1937955484 shutdown_connections 2023-11-26T07:18:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff640066470 0x7ff640068930 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65c143ae0 0x7ff65c15b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff65c0adcc0 0x7ff65c14a980 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 --2- 172.21.15.31:0/1937955484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65c06dfd0 0x7ff65c158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.768+0000 7ff663580640 1 -- 172.21.15.31:0/1937955484 >> 172.21.15.31:0/1937955484 conn(0x7ff65c0b0c40 msgr2=0x7ff65c0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:13.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.772+0000 7ff663580640 1 -- 172.21.15.31:0/1937955484 shutdown_connections 2023-11-26T07:18:13.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.772+0000 7ff663580640 1 -- 172.21.15.31:0/1937955484 wait complete. 2023-11-26T07:18:13.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:13.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.932+0000 7f2978ec9640 1 Processor -- start 2023-11-26T07:18:13.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.932+0000 7f2978ec9640 1 -- start start 2023-11-26T07:18:13.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2978ec9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f297406dfd0 0x7f297406e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2978ec9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f29740adcc0 0x7f29740ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2978ec9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297406e950 con 0x7f2974143ae0 2023-11-26T07:18:13.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2978ec9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2974070670 con 0x7f29740adcc0 2023-11-26T07:18:13.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2978ec9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2974070950 con 0x7f297406dfd0 2023-11-26T07:18:13.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297259c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2974143ae0 0x7f29740b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47464/0 (socket says 172.21.15.31:47464) 2023-11-26T07:18:13.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297259c640 1 -- 172.21.15.31:0/3585023556 learned_addr learned my addr 172.21.15.31:0/3585023556 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:13.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2971d9b640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f297406dfd0 0x7f297406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308540226 0 0) 0x7f297406e950 con 0x7f2974143ae0 2023-11-26T07:18:13.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2972d9d640 1 --2- 172.21.15.31:0/3585023556 >> v2:172.21.15.97:3300/0 conn(0x7f29740adcc0 0x7f29740ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f295c003530 con 0x7f2974143ae0 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 993895039 0 0) 0x7f295c003530 con 0x7f2974143ae0 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f297406dfd0 msgr2=0x7f297406e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f297406dfd0 0x7f297406e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 >> v2:172.21.15.97:3300/0 conn(0x7f29740adcc0 msgr2=0x7f29740ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 --2- 172.21.15.31:0/3585023556 >> v2:172.21.15.97:3300/0 conn(0x7f29740adcc0 0x7f29740ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2974070c30 con 0x7f2974143ae0 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f2964001680 con 0x7f2974143ae0 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 >> v1:172.21.15.31:6789/0 conn(0x7f2974143ae0 legacy=0x7f29740b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f297406dfd0 0x7f295c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f295c003f00 0x7f295c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 0x7f295c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2974070670 con 0x7f295c004d10 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2974070950 con 0x7f295c003f00 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c00b990 con 0x7f297406dfd0 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2972d9d640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 0x7f295c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f297259c640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f297406dfd0 0x7f295c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2971d9b640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f295c003f00 0x7f295c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2972d9d640 1 -- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f297406dfd0 msgr2=0x7f295c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2972d9d640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f297406dfd0 0x7f295c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2972d9d640 1 -- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f295c003f00 msgr2=0x7f295c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:13.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2972d9d640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f295c003f00 0x7f295c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.936+0000 7f2972d9d640 1 -- 172.21.15.31:0/3585023556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c003530 con 0x7f295c004d10 2023-11-26T07:18:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f2972d9d640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 0x7f295c005190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f296c009fc0 tx=0x7f296c01ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e2ab8f31531fce server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0af440 con 0x7f295c004d10 2023-11-26T07:18:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c00f040 con 0x7f295c004d10 2023-11-26T07:18:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f297159a640 1 -- 172.21.15.31:0/3585023556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0af7e0 con 0x7f295c004d10 2023-11-26T07:18:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f2978ec9640 1 -- 172.21.15.31:0/3585023556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 msgr2=0x7f295c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f2978ec9640 1 --2- 172.21.15.31:0/3585023556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 0x7f295c005190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f296c009fc0 tx=0x7f296c01ee50 comp rx=0 tx=0).stop 2023-11-26T07:18:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f2972d9d640 1 -- 172.21.15.31:0/3585023556 reap_dead start 2023-11-26T07:18:13.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f2978ec9640 1 -- 172.21.15.31:0/3585023556 shutdown_connections 2023-11-26T07:18:13.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f2978ec9640 1 -- 172.21.15.31:0/3585023556 >> 172.21.15.31:0/3585023556 conn(0x7f29740b0c40 msgr2=0x7f29740b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:13.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f2978ec9640 1 -- 172.21.15.31:0/3585023556 shutdown_connections 2023-11-26T07:18:13.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.940+0000 7f2978ec9640 1 -- 172.21.15.31:0/3585023556 wait complete. 2023-11-26T07:18:13.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 Processor -- start 2023-11-26T07:18:13.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 -- start start 2023-11-26T07:18:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 0x7f2974099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297409aa70 0x7f297409c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29741f6650 0x7f29741ec8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297409b450 con 0x7f295c004d10 2023-11-26T07:18:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297409b730 con 0x7f297409aa70 2023-11-26T07:18:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297409ba10 con 0x7f29741f6650 2023-11-26T07:18:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f297259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 0x7f2974099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f297259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 0x7f2974099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48142/0 (socket says 172.21.15.31:48142) 2023-11-26T07:18:13.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f297259c640 1 -- 172.21.15.31:0/3809816206 learned_addr learned my addr 172.21.15.31:0/3809816206 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2971d9b640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297409aa70 0x7f297409c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2972d9d640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29741f6650 0x7f29741ec8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f297259c640 1 -- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29741f6650 msgr2=0x7f29741ec8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f297259c640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29741f6650 0x7f29741ec8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f297259c640 1 -- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297409aa70 msgr2=0x7f297409c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f297259c640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297409aa70 0x7f297409c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f297259c640 1 -- 172.21.15.31:0/3809816206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297409bcf0 con 0x7f295c004d10 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f297259c640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 0x7f2974099a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2964007340 tx=0x7f2964007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2962ffd640 1 -- 172.21.15.31:0/3809816206 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296400f9c0 con 0x7f295c004d10 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2962ffd640 1 -- 172.21.15.31:0/3809816206 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296400fb40 con 0x7f295c004d10 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 -- 172.21.15.31:0/3809816206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297409bfd0 con 0x7f295c004d10 2023-11-26T07:18:13.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2962ffd640 1 -- 172.21.15.31:0/3809816206 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29640133f0 con 0x7f295c004d10 2023-11-26T07:18:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 -- 172.21.15.31:0/3809816206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297409aef0 con 0x7f295c004d10 2023-11-26T07:18:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.944+0000 7f2978ec9640 1 -- 172.21.15.31:0/3809816206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297404e850 con 0x7f295c004d10 2023-11-26T07:18:13.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.948+0000 7f2962ffd640 1 -- 172.21.15.31:0/3809816206 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2964013550 con 0x7f295c004d10 2023-11-26T07:18:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.948+0000 7f2962ffd640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2954066470 0x7f2954068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.948+0000 7f2962ffd640 1 -- 172.21.15.31:0/3809816206 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f2964083fd0 con 0x7f295c004d10 2023-11-26T07:18:13.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.948+0000 7f2971d9b640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2954066470 0x7f2954068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:13.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.948+0000 7f2971d9b640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2954066470 0x7f2954068930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f29740b3fd0 tx=0x7f2968009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:13.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:13.952+0000 7f2962ffd640 1 -- 172.21.15.31:0/3809816206 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f29640513f0 con 0x7f295c004d10 2023-11-26T07:18:14.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.112+0000 7f2978ec9640 1 -- 172.21.15.31:0/3809816206 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29740b1520 con 0x7f295c004d10 2023-11-26T07:18:14.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.112+0000 7f2962ffd640 1 -- 172.21.15.31:0/3809816206 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f2964054a10 con 0x7f295c004d10 2023-11-26T07:18:14.118 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:13.288842+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:17:30.635783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-26T07:17:31.500091+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1},{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:14.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 -- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2954066470 msgr2=0x7f2954068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2954066470 0x7f2954068930 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f29740b3fd0 tx=0x7f2968009190 comp rx=0 tx=0).stop 2023-11-26T07:18:14.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 -- 172.21.15.31:0/3809816206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 msgr2=0x7f2974099a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 0x7f2974099a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2964007340 tx=0x7f2964007740 comp rx=0 tx=0).stop 2023-11-26T07:18:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 -- 172.21.15.31:0/3809816206 shutdown_connections 2023-11-26T07:18:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2954066470 0x7f2954068930 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29741f6650 0x7f29741ec8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f297409aa70 0x7f297409c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 --2- 172.21.15.31:0/3809816206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f295c004d10 0x7f2974099a30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 -- 172.21.15.31:0/3809816206 >> 172.21.15.31:0/3809816206 conn(0x7f29740b0c40 msgr2=0x7f29741597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 -- 172.21.15.31:0/3809816206 shutdown_connections 2023-11-26T07:18:14.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.116+0000 7f2978ec9640 1 -- 172.21.15.31:0/3809816206 wait complete. 2023-11-26T07:18:14.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:18:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f7017c32640 1 Processor -- start 2023-11-26T07:18:14.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f7017c32640 1 -- start start 2023-11-26T07:18:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f7017c32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 0x7f70100abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f7017c32640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f70100cbd50 0x7f70100cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f7017c32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70100b2710 con 0x7f701015a710 2023-11-26T07:18:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f7017c32640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70100cca80 con 0x7f70100cbd50 2023-11-26T07:18:14.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f7017c32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70100b29f0 con 0x7f70100cdfb0 2023-11-26T07:18:14.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f70159ce640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 0x7f70100abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f70159ce640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 0x7f70100abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:36572/0 (socket says 172.21.15.31:36572) 2023-11-26T07:18:14.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f70159ce640 1 -- 172.21.15.31:0/1116635404 learned_addr learned my addr 172.21.15.31:0/1116635404 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:14.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f70151cd640 1 --1- 172.21.15.31:0/1116635404 >> v1:172.21.15.97:6789/0 conn(0x7f701015a710 0x7f70100ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:52796/0 (socket says 172.21.15.31:52796) 2023-11-26T07:18:14.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f70161cf640 1 --2- 172.21.15.31:0/1116635404 >> v2:172.21.15.97:3300/0 conn(0x7f70100cbd50 0x7f70100cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f70161cf640 1 -- 172.21.15.31:0/1116635404 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 msgr2=0x7f70100abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:14.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f70161cf640 1 --2- 172.21.15.31:0/1116635404 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 0x7f70100abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f70161cf640 1 -- 172.21.15.31:0/1116635404 >> v1:172.21.15.97:6789/0 conn(0x7f701015a710 legacy=0x7f70100ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.280+0000 7f70161cf640 1 -- 172.21.15.31:0/1116635404 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70100b2cd0 con 0x7f70100cbd50 2023-11-26T07:18:14.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f70161cf640 1 --2- 172.21.15.31:0/1116635404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70100cbd50 0x7f70100cc1e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f700800bd90 tx=0x7f700801e7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=887a0eec68edad66 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:14.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f70149cc640 1 -- 172.21.15.31:0/1116635404 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080a77d0 con 0x7f70100cbd50 2023-11-26T07:18:14.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f70149cc640 1 -- 172.21.15.31:0/1116635404 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70080a7930 con 0x7f70100cbd50 2023-11-26T07:18:14.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f70149cc640 1 -- 172.21.15.31:0/1116635404 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080a7cc0 con 0x7f70100cbd50 2023-11-26T07:18:14.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f7017c32640 1 -- 172.21.15.31:0/1116635404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70100cbd50 msgr2=0x7f70100cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f7017c32640 1 --2- 172.21.15.31:0/1116635404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70100cbd50 0x7f70100cc1e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f700800bd90 tx=0x7f700801e7a0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f7017c32640 1 -- 172.21.15.31:0/1116635404 shutdown_connections 2023-11-26T07:18:14.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f7017c32640 1 --2- 172.21.15.31:0/1116635404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70100cbd50 0x7f70100cc1e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f7017c32640 1 --2- 172.21.15.31:0/1116635404 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 0x7f70100abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f7017c32640 1 -- 172.21.15.31:0/1116635404 >> 172.21.15.31:0/1116635404 conn(0x7f70100b0cb0 msgr2=0x7f70100b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:14.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f7017c32640 1 -- 172.21.15.31:0/1116635404 shutdown_connections 2023-11-26T07:18:14.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.284+0000 7f7017c32640 1 -- 172.21.15.31:0/1116635404 wait complete. 2023-11-26T07:18:14.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f7017c32640 1 Processor -- start 2023-11-26T07:18:14.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f7017c32640 1 -- start start 2023-11-26T07:18:14.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f7017c32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70100cbd50 0x7f70101483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f7017c32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 0x7f70100cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f7017c32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701015a710 0x7f7010148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f7017c32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70100ca0a0 con 0x7f70100cbd50 2023-11-26T07:18:14.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f7017c32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70100ca320 con 0x7f701015a710 2023-11-26T07:18:14.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f7017c32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70101f3de0 con 0x7f70100cdfb0 2023-11-26T07:18:14.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70159ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70100cbd50 0x7f70101483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70159ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70100cbd50 0x7f70101483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48152/0 (socket says 172.21.15.31:48152) 2023-11-26T07:18:14.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70159ce640 1 -- 172.21.15.31:0/1508801513 learned_addr learned my addr 172.21.15.31:0/1508801513 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:14.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70161cf640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701015a710 0x7f7010148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70151cd640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 0x7f70100cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70159ce640 1 -- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 msgr2=0x7f70100cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70159ce640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 0x7f70100cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70159ce640 1 -- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701015a710 msgr2=0x7f7010148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70159ce640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701015a710 0x7f7010148a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70159ce640 1 -- 172.21.15.31:0/1508801513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700800ba70 con 0x7f70100cbd50 2023-11-26T07:18:14.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70159ce640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70100cbd50 0x7f70101483c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f700c00e890 tx=0x7f700c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:14.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70067fc640 1 -- 172.21.15.31:0/1508801513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c009620 con 0x7f70100cbd50 2023-11-26T07:18:14.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70067fc640 1 -- 172.21.15.31:0/1508801513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c00c940 con 0x7f70100cbd50 2023-11-26T07:18:14.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f7017c32640 1 -- 172.21.15.31:0/1508801513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70100ca5e0 con 0x7f70100cbd50 2023-11-26T07:18:14.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f70067fc640 1 -- 172.21.15.31:0/1508801513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c00ccf0 con 0x7f70100cbd50 2023-11-26T07:18:14.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.288+0000 7f7017c32640 1 -- 172.21.15.31:0/1508801513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70100cb320 con 0x7f70100cbd50 2023-11-26T07:18:14.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.292+0000 7f7017c32640 1 -- 172.21.15.31:0/1508801513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70100671f0 con 0x7f70100cbd50 2023-11-26T07:18:14.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.292+0000 7f70067fc640 1 -- 172.21.15.31:0/1508801513 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f700c013470 con 0x7f70100cbd50 2023-11-26T07:18:14.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.292+0000 7f70067fc640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff4066470 0x7f6ff4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.292+0000 7f70067fc640 1 -- 172.21.15.31:0/1508801513 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5804+0+0 (secure 0 0 0) 0x7f700c083fe0 con 0x7f70100cbd50 2023-11-26T07:18:14.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.292+0000 7f70151cd640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff4066470 0x7f6ff4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.292+0000 7f70151cd640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff4066470 0x7f6ff4068930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f70100b1c90 tx=0x7f700000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:14.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.296+0000 7f70067fc640 1 -- 172.21.15.31:0/1508801513 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f700c0514c0 con 0x7f70100cbd50 2023-11-26T07:18:14.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.492+0000 7f7017c32640 1 -- 172.21.15.31:0/1508801513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f701006b460 con 0x7f70100cbd50 2023-11-26T07:18:14.502 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-11-26T07:18:14.496+0000 7fe86499d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:18:14.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.496+0000 7f70067fc640 1 -- 172.21.15.31:0/1508801513 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f700c054ae0 con 0x7f70100cbd50 2023-11-26T07:18:14.503 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:18:14.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 -- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff4066470 msgr2=0x7f6ff4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff4066470 0x7f6ff4068930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f70100b1c90 tx=0x7f700000b040 comp rx=0 tx=0).stop 2023-11-26T07:18:14.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 -- 172.21.15.31:0/1508801513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70100cbd50 msgr2=0x7f70101483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70100cbd50 0x7f70101483c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f700c00e890 tx=0x7f700c00ed50 comp rx=0 tx=0).stop 2023-11-26T07:18:14.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 -- 172.21.15.31:0/1508801513 shutdown_connections 2023-11-26T07:18:14.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff4066470 0x7f6ff4068930 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f701015a710 0x7f7010148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70100cdfb0 0x7f70100cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 --2- 172.21.15.31:0/1508801513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70100cbd50 0x7f70101483c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 -- 172.21.15.31:0/1508801513 >> 172.21.15.31:0/1508801513 conn(0x7f70100b0cb0 msgr2=0x7f70100ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:14.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 -- 172.21.15.31:0/1508801513 shutdown_connections 2023-11-26T07:18:14.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.500+0000 7f7017c32640 1 -- 172.21.15.31:0/1508801513 wait complete. 2023-11-26T07:18:14.517 INFO:tasks.ceph.mds.b.smithi097.stderr: -6689> 2023-11-26T07:17:36.900+0000 7f4eee0d8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:18:14.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:18:14.568 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2023-11-26T07:18:14.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f07c5a640 1 Processor -- start 2023-11-26T07:18:14.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f07c5a640 1 -- start start 2023-11-26T07:18:14.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f07c5a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f0006e3f0 0x7f9f0015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f07c5a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f00159630 con 0x7f9f000ce5d0 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f07c5a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0015f6d0 con 0x7f9f000cb9b0 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f07c5a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0006e860 con 0x7f9f0006e3f0 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f061f7640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9f000ce5d0 0x7f9f0015bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47474/0 (socket says 172.21.15.31:47474) 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f061f7640 1 -- 172.21.15.31:0/1908014917 learned_addr learned my addr 172.21.15.31:0/1908014917 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f051f5640 1 --2- 172.21.15.31:0/1908014917 >> v2:172.21.15.97:3300/0 conn(0x7f9f0006e3f0 0x7f9f0015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f049f4640 1 -- 172.21.15.31:0/1908014917 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199805758 0 0) 0x7f9f00159630 con 0x7f9f000ce5d0 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.668+0000 7f9f049f4640 1 -- 172.21.15.31:0/1908014917 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eec003530 con 0x7f9f000ce5d0 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f049f4640 1 -- 172.21.15.31:0/1908014917 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707516294 0 0) 0x7f9f0015f6d0 con 0x7f9f000cb9b0 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f049f4640 1 -- 172.21.15.31:0/1908014917 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f00159630 con 0x7f9f000cb9b0 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f051f5640 1 -- 172.21.15.31:0/1908014917 >> v1:172.21.15.97:6789/0 conn(0x7f9f000cb9b0 legacy=0x7f9f000cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f051f5640 1 -- 172.21.15.31:0/1908014917 >> v1:172.21.15.31:6789/0 conn(0x7f9f000ce5d0 legacy=0x7f9f0015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f051f5640 1 -- 172.21.15.31:0/1908014917 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f000cd1d0 con 0x7f9f0006e3f0 2023-11-26T07:18:14.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f051f5640 1 --2- 172.21.15.31:0/1908014917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f0006e3f0 0x7f9f0015f190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00097c0 tx=0x7f9ef001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff832af37ae8dd1d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:14.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f049f4640 1 -- 172.21.15.31:0/1908014917 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00ab600 con 0x7f9f0006e3f0 2023-11-26T07:18:14.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f049f4640 1 -- 172.21.15.31:0/1908014917 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef00a7050 con 0x7f9f0006e3f0 2023-11-26T07:18:14.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f049f4640 1 -- 172.21.15.31:0/1908014917 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00ab980 con 0x7f9f0006e3f0 2023-11-26T07:18:14.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1908014917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f0006e3f0 msgr2=0x7f9f0015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.672+0000 7f9f07c5a640 1 --2- 172.21.15.31:0/1908014917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f0006e3f0 0x7f9f0015f190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00097c0 tx=0x7f9ef001c480 comp rx=0 tx=0).stop 2023-11-26T07:18:14.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1908014917 shutdown_connections 2023-11-26T07:18:14.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 --2- 172.21.15.31:0/1908014917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f0006e3f0 0x7f9f0015f190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1908014917 >> 172.21.15.31:0/1908014917 conn(0x7f9f000b0cb0 msgr2=0x7f9f000b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:14.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1908014917 shutdown_connections 2023-11-26T07:18:14.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1908014917 wait complete. 2023-11-26T07:18:14.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 Processor -- start 2023-11-26T07:18:14.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 -- start start 2023-11-26T07:18:14.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0006e3f0 0x7f9f00153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f000cb9b0 0x7f9f00150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f000ce5d0 0x7f9f000ac830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f00151e00 con 0x7f9f0006e3f0 2023-11-26T07:18:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f001545d0 con 0x7f9f000cb9b0 2023-11-26T07:18:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f07c5a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f00154010 con 0x7f9f000ce5d0 2023-11-26T07:18:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f059f6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0006e3f0 0x7f9f00153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f059f6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0006e3f0 0x7f9f00153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48166/0 (socket says 172.21.15.31:48166) 2023-11-26T07:18:14.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f059f6640 1 -- 172.21.15.31:0/1291719882 learned_addr learned my addr 172.21.15.31:0/1291719882 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:14.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f061f7640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f000ce5d0 0x7f9f000ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f051f5640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f000cb9b0 0x7f9f00150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f059f6640 1 -- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f000ce5d0 msgr2=0x7f9f000ac830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f059f6640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f000ce5d0 0x7f9f000ac830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f059f6640 1 -- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f000cb9b0 msgr2=0x7f9f00150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f059f6640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f000cb9b0 0x7f9f00150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f059f6640 1 -- 172.21.15.31:0/1291719882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef0009420 con 0x7f9f0006e3f0 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f051f5640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f000cb9b0 0x7f9f00150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f061f7640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f000ce5d0 0x7f9f000ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.676+0000 7f9f059f6640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0006e3f0 0x7f9f00153960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00c490 tx=0x7f9efc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.680+0000 7f9ef67fc640 1 -- 172.21.15.31:0/1291719882 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0052d0 con 0x7f9f0006e3f0 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.680+0000 7f9ef67fc640 1 -- 172.21.15.31:0/1291719882 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc005430 con 0x7f9f0006e3f0 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.680+0000 7f9ef67fc640 1 -- 172.21.15.31:0/1291719882 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc012870 con 0x7f9f0006e3f0 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.680+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1291719882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f001542f0 con 0x7f9f0006e3f0 2023-11-26T07:18:14.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.680+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1291719882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f000ac310 con 0x7f9f0006e3f0 2023-11-26T07:18:14.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.680+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1291719882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f000671f0 con 0x7f9f0006e3f0 2023-11-26T07:18:14.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.680+0000 7f9ef67fc640 1 -- 172.21.15.31:0/1291719882 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f9efc0129d0 con 0x7f9f0006e3f0 2023-11-26T07:18:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.680+0000 7f9ef67fc640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9ee4066470 0x7f9ee4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.680+0000 7f9ef67fc640 1 -- 172.21.15.31:0/1291719882 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f9efc083bf0 con 0x7f9f0006e3f0 2023-11-26T07:18:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.680+0000 7f9f051f5640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9ee4066470 0x7f9ee4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:14.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.684+0000 7f9f051f5640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9ee4066470 0x7f9ee4068930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00ad000 tx=0x7f9ef0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:14.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.684+0000 7f9ef67fc640 1 -- 172.21.15.31:0/1291719882 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9efc050f10 con 0x7f9f0006e3f0 2023-11-26T07:18:14.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:14.876+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1291719882 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9f0006b460 con 0x7f9f0006e3f0 2023-11-26T07:18:15.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.500+0000 7f9ef67fc640 1 -- 172.21.15.31:0/1291719882 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9efc054530 con 0x7f9f0006e3f0 2023-11-26T07:18:15.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.500+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9ee4066470 msgr2=0x7f9ee4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:15.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9ee4066470 0x7f9ee4068930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00ad000 tx=0x7f9ef0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:15.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1291719882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0006e3f0 msgr2=0x7f9f00153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:15.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0006e3f0 0x7f9f00153960 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00c490 tx=0x7f9efc00c950 comp rx=0 tx=0).stop 2023-11-26T07:18:15.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1291719882 shutdown_connections 2023-11-26T07:18:15.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9ee4066470 0x7f9ee4068930 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:15.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f000ce5d0 0x7f9f000ac830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:15.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f000cb9b0 0x7f9f00150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:15.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 --2- 172.21.15.31:0/1291719882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f0006e3f0 0x7f9f00153960 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:15.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1291719882 >> 172.21.15.31:0/1291719882 conn(0x7f9f000b0cb0 msgr2=0x7f9f0015f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:15.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1291719882 shutdown_connections 2023-11-26T07:18:15.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.504+0000 7f9f07c5a640 1 -- 172.21.15.31:0/1291719882 wait complete. 2023-11-26T07:18:15.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:18:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.676+0000 7f7bc87f8640 1 Processor -- start 2023-11-26T07:18:15.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.676+0000 7f7bc87f8640 1 -- start start 2023-11-26T07:18:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc87f8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bc00cb9b0 0x7f7bc006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc87f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bc00ce5d0 0x7f7bc015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc87f8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc0159630 con 0x7f7bc00cbd90 2023-11-26T07:18:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc87f8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc015bba0 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc87f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc00cdd50 con 0x7f7bc00ce5d0 2023-11-26T07:18:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc6594640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bc00cb9b0 0x7f7bc006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc6594640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bc00cb9b0 0x7f7bc006e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48180/0 (socket says 172.21.15.31:48180) 2023-11-26T07:18:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc6594640 1 -- 172.21.15.31:0/197138092 learned_addr learned my addr 172.21.15.31:0/197138092 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:15.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc6d95640 1 --2- 172.21.15.31:0/197138092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bc00ce5d0 0x7f7bc015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc6594640 1 -- 172.21.15.31:0/197138092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bc00ce5d0 msgr2=0x7f7bc015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc6594640 1 --2- 172.21.15.31:0/197138092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bc00ce5d0 0x7f7bc015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc6594640 1 -- 172.21.15.31:0/197138092 >> v1:172.21.15.97:6789/0 conn(0x7f7bc00cbd90 legacy=0x7f7bc015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:15.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.680+0000 7f7bc6594640 1 -- 172.21.15.31:0/197138092 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc00cd1d0 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc6594640 1 --2- 172.21.15.31:0/197138092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 0x7f7bc006e3f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009870 tx=0x7f7bbc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adf58138ea3da8a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:15.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc5592640 1 -- 172.21.15.31:0/197138092 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0aa650 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc5592640 1 -- 172.21.15.31:0/197138092 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc0aa7b0 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc5592640 1 -- 172.21.15.31:0/197138092 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0aab40 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc87f8640 1 -- 172.21.15.31:0/197138092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 msgr2=0x7f7bc006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:15.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc87f8640 1 --2- 172.21.15.31:0/197138092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 0x7f7bc006e3f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009870 tx=0x7f7bbc01c510 comp rx=0 tx=0).stop 2023-11-26T07:18:15.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc87f8640 1 -- 172.21.15.31:0/197138092 shutdown_connections 2023-11-26T07:18:15.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc87f8640 1 --2- 172.21.15.31:0/197138092 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bc00ce5d0 0x7f7bc015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:15.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc87f8640 1 --2- 172.21.15.31:0/197138092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 0x7f7bc006e3f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:15.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc87f8640 1 -- 172.21.15.31:0/197138092 >> 172.21.15.31:0/197138092 conn(0x7f7bc00b0cb0 msgr2=0x7f7bc00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:15.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc87f8640 1 -- 172.21.15.31:0/197138092 shutdown_connections 2023-11-26T07:18:15.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc87f8640 1 -- 172.21.15.31:0/197138092 wait complete. 2023-11-26T07:18:15.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc87f8640 1 Processor -- start 2023-11-26T07:18:15.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.684+0000 7f7bc87f8640 1 -- start start 2023-11-26T07:18:15.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc87f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 0x7f7bc01f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:15.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc87f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bc00cbd90 0x7f7bc014b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc87f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc00ce5d0 0x7f7bc015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc87f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc00b2fa0 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc87f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc009a6e0 con 0x7f7bc00ce5d0 2023-11-26T07:18:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc87f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc009b1d0 con 0x7f7bc00cbd90 2023-11-26T07:18:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc6594640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 0x7f7bc01f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc6594640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 0x7f7bc01f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48190/0 (socket says 172.21.15.31:48190) 2023-11-26T07:18:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc6594640 1 -- 172.21.15.31:0/4174922088 learned_addr learned my addr 172.21.15.31:0/4174922088 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc5d93640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bc00cbd90 0x7f7bc014b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:15.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc6d95640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc00ce5d0 0x7f7bc015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:15.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc6594640 1 -- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bc00cbd90 msgr2=0x7f7bc014b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:15.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc6594640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bc00cbd90 0x7f7bc014b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:15.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc6594640 1 -- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc00ce5d0 msgr2=0x7f7bc015dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:15.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc6594640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc00ce5d0 0x7f7bc015dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:15.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc6594640 1 -- 172.21.15.31:0/4174922088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc0094d0 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc6594640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 0x7f7bc01f6720 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0ac040 tx=0x7f7bbc01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:15.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bb6ffd640 1 -- 172.21.15.31:0/4174922088 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0b3800 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bb6ffd640 1 -- 172.21.15.31:0/4174922088 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc0b3980 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc87f8640 1 -- 172.21.15.31:0/4174922088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc01f6c60 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bb6ffd640 1 -- 172.21.15.31:0/4174922088 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0b3d30 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc87f8640 1 -- 172.21.15.31:0/4174922088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc009ac10 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bc87f8640 1 -- 172.21.15.31:0/4174922088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc00671f0 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.688+0000 7f7bb6ffd640 1 -- 172.21.15.31:0/4174922088 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7bbc0bd3f0 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.692+0000 7f7bb6ffd640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066470 0x7f7ba4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:15.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.692+0000 7f7bb6ffd640 1 -- 172.21.15.31:0/4174922088 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f7bbc11ea90 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.692+0000 7f7bc5d93640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066470 0x7f7ba4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:15.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.692+0000 7f7bc5d93640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066470 0x7f7ba4068930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0099f10 tx=0x7f7bb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:15.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.696+0000 7f7bb6ffd640 1 -- 172.21.15.31:0/4174922088 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7bbc0ebf90 con 0x7f7bc00cb9b0 2023-11-26T07:18:15.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:15.852+0000 7f7bc87f8640 1 -- 172.21.15.31:0/4174922088 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7bc006b460 con 0x7f7bc00cb9b0 2023-11-26T07:18:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.504+0000 7f7bb6ffd640 1 -- 172.21.15.31:0/4174922088 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7bbc0c3020 con 0x7f7bc00cb9b0 2023-11-26T07:18:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:18:16.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.504+0000 7f7bc87f8640 1 -- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066470 msgr2=0x7f7ba4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.504+0000 7f7bc87f8640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066470 0x7f7ba4068930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0099f10 tx=0x7f7bb0009040 comp rx=0 tx=0).stop 2023-11-26T07:18:16.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.504+0000 7f7bc87f8640 1 -- 172.21.15.31:0/4174922088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 msgr2=0x7f7bc01f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.504+0000 7f7bc87f8640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 0x7f7bc01f6720 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0ac040 tx=0x7f7bbc01ad50 comp rx=0 tx=0).stop 2023-11-26T07:18:16.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.508+0000 7f7bc87f8640 1 -- 172.21.15.31:0/4174922088 shutdown_connections 2023-11-26T07:18:16.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.508+0000 7f7bc87f8640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066470 0x7f7ba4068930 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.508+0000 7f7bc87f8640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bc00ce5d0 0x7f7bc015dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.508+0000 7f7bc87f8640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bc00cbd90 0x7f7bc014b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.508+0000 7f7bc87f8640 1 --2- 172.21.15.31:0/4174922088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bc00cb9b0 0x7f7bc01f6720 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.508+0000 7f7bc87f8640 1 -- 172.21.15.31:0/4174922088 >> 172.21.15.31:0/4174922088 conn(0x7f7bc00b0cb0 msgr2=0x7f7bc00cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:16.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.508+0000 7f7bc87f8640 1 -- 172.21.15.31:0/4174922088 shutdown_connections 2023-11-26T07:18:16.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.508+0000 7f7bc87f8640 1 -- 172.21.15.31:0/4174922088 wait complete. 2023-11-26T07:18:16.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:18:16.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.684+0000 7f324df2a640 1 Processor -- start 2023-11-26T07:18:16.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.684+0000 7f324df2a640 1 -- start start 2023-11-26T07:18:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f324df2a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3248166580 0x7f3248168970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f324df2a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32480af630 con 0x7f324815e690 2023-11-26T07:18:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f324df2a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32480acb10 con 0x7f32480b5830 2023-11-26T07:18:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f324df2a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32480adb90 con 0x7f3248166580 2023-11-26T07:18:16.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f3246ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f324815e690 0x7f32480ad480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47490/0 (socket says 172.21.15.31:47490) 2023-11-26T07:18:16.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f3246ffd640 1 -- 172.21.15.31:0/4089695239 learned_addr learned my addr 172.21.15.31:0/4089695239 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:16.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f3247fff640 1 --2- 172.21.15.31:0/4089695239 >> v2:172.21.15.97:3300/0 conn(0x7f3248166580 0x7f3248168970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132449987 0 0) 0x7f323c001120 con 0x7f324815e690 2023-11-26T07:18:16.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3230003530 con 0x7f324815e690 2023-11-26T07:18:16.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1164990346 0 0) 0x7f3230003530 con 0x7f324815e690 2023-11-26T07:18:16.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 >> v2:172.21.15.97:3300/0 conn(0x7f3248166580 msgr2=0x7f3248168970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:16.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 --2- 172.21.15.31:0/4089695239 >> v2:172.21.15.97:3300/0 conn(0x7f3248166580 0x7f3248168970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 >> v1:172.21.15.97:6789/0 conn(0x7f32480b5830 legacy=0x7f32480acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324815eaf0 con 0x7f324815e690 2023-11-26T07:18:16.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f323c001810 con 0x7f324815e690 2023-11-26T07:18:16.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 >> v1:172.21.15.31:6789/0 conn(0x7f324815e690 legacy=0x7f32480ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32300039d0 0x7f3230003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32300047d0 0x7f3230004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3230005190 0x7f323000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32480adb90 con 0x7f3230005190 2023-11-26T07:18:16.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323000c090 con 0x7f32300039d0 2023-11-26T07:18:16.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323000c250 con 0x7f32300047d0 2023-11-26T07:18:16.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f3247fff640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3230005190 0x7f323000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f32477fe640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32300039d0 0x7f3230003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f3246ffd640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32300047d0 0x7f3230004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f3247fff640 1 -- 172.21.15.31:0/4089695239 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32300047d0 msgr2=0x7f3230004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:16.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f3247fff640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32300047d0 0x7f3230004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f3247fff640 1 -- 172.21.15.31:0/4089695239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32300039d0 msgr2=0x7f3230003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:16.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f3247fff640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32300039d0 0x7f3230003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.688+0000 7f3247fff640 1 -- 172.21.15.31:0/4089695239 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3230003530 con 0x7f3230005190 2023-11-26T07:18:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f3247fff640 1 -- 172.21.15.31:0/4089695239 reap_dead start 2023-11-26T07:18:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f3247fff640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3230005190 0x7f323000b990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3240009fc0 tx=0x7f324001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bf5421695e8bc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32400a5cb0 con 0x7f3230005190 2023-11-26T07:18:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324000f040 con 0x7f3230005190 2023-11-26T07:18:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f32467fc640 1 -- 172.21.15.31:0/4089695239 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240003be0 con 0x7f3230005190 2023-11-26T07:18:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f324df2a640 1 -- 172.21.15.31:0/4089695239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3230005190 msgr2=0x7f323000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f324df2a640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3230005190 0x7f323000b990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3240009fc0 tx=0x7f324001cca0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f324df2a640 1 -- 172.21.15.31:0/4089695239 shutdown_connections 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f324df2a640 1 --2- 172.21.15.31:0/4089695239 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3230005190 0x7f323000b990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f324df2a640 1 -- 172.21.15.31:0/4089695239 >> 172.21.15.31:0/4089695239 conn(0x7f3248071d80 msgr2=0x7f324806a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f324df2a640 1 -- 172.21.15.31:0/4089695239 shutdown_connections 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.692+0000 7f324df2a640 1 -- 172.21.15.31:0/4089695239 wait complete. 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f324df2a640 1 Processor -- start 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f324df2a640 1 -- start start 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f324df2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230005190 0x7f3248154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f324df2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248155ef0 0x7f324814b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f324df2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32480b0170 0x7f32480b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f324df2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32480b0b30 con 0x7f3248155ef0 2023-11-26T07:18:16.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f324df2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3248156740 con 0x7f3230005190 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f324df2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3248156a20 con 0x7f32480b0170 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f3246ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248155ef0 0x7f324814b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f3246ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248155ef0 0x7f324814b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48196/0 (socket says 172.21.15.31:48196) 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f3246ffd640 1 -- 172.21.15.31:0/1501619012 learned_addr learned my addr 172.21.15.31:0/1501619012 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f3247fff640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32480b0170 0x7f32480b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f32477fe640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230005190 0x7f3248154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f3246ffd640 1 -- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32480b0170 msgr2=0x7f32480b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f3246ffd640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32480b0170 0x7f32480b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f3246ffd640 1 -- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230005190 msgr2=0x7f3248154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f3246ffd640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230005190 0x7f3248154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f3246ffd640 1 -- 172.21.15.31:0/1501619012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3248156d00 con 0x7f3248155ef0 2023-11-26T07:18:16.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f32477fe640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230005190 0x7f3248154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:16.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f3246ffd640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248155ef0 0x7f324814b590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f323c001940 tx=0x7f323c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:16.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f322ffff640 1 -- 172.21.15.31:0/1501619012 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c007e20 con 0x7f3248155ef0 2023-11-26T07:18:16.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f322ffff640 1 -- 172.21.15.31:0/1501619012 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c00f940 con 0x7f3248155ef0 2023-11-26T07:18:16.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f322ffff640 1 -- 172.21.15.31:0/1501619012 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c00fcf0 con 0x7f3248155ef0 2023-11-26T07:18:16.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f324df2a640 1 -- 172.21.15.31:0/1501619012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3248156fe0 con 0x7f3248155ef0 2023-11-26T07:18:16.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.696+0000 7f324df2a640 1 -- 172.21.15.31:0/1501619012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32481577e0 con 0x7f3248155ef0 2023-11-26T07:18:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.700+0000 7f324df2a640 1 -- 172.21.15.31:0/1501619012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324804e850 con 0x7f3248155ef0 2023-11-26T07:18:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.700+0000 7f322ffff640 1 -- 172.21.15.31:0/1501619012 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f323c01d460 con 0x7f3248155ef0 2023-11-26T07:18:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.700+0000 7f322ffff640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32340664f0 0x7f32340689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.700+0000 7f322ffff640 1 -- 172.21.15.31:0/1501619012 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5452+0+0 (secure 0 0 0) 0x7f323c083c80 con 0x7f3248155ef0 2023-11-26T07:18:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.700+0000 7f32477fe640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32340664f0 0x7f32340689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:16.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.700+0000 7f32477fe640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32340664f0 0x7f32340689b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3238000c10 tx=0x7f3238009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:16.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.704+0000 7f322ffff640 1 -- 172.21.15.31:0/1501619012 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f323c051270 con 0x7f3248155ef0 2023-11-26T07:18:16.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:16.860+0000 7f324df2a640 1 -- 172.21.15.31:0/1501619012 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f32480af630 con 0x7f3248155ef0 2023-11-26T07:18:17.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.504+0000 7f322ffff640 1 -- 172.21.15.31:0/1501619012 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f32480af630 con 0x7f3248155ef0 2023-11-26T07:18:17.511 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-11-26T07:18:17.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 -- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32340664f0 msgr2=0x7f32340689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32340664f0 0x7f32340689b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3238000c10 tx=0x7f3238009040 comp rx=0 tx=0).stop 2023-11-26T07:18:17.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 -- 172.21.15.31:0/1501619012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248155ef0 msgr2=0x7f324814b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248155ef0 0x7f324814b590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f323c001940 tx=0x7f323c007720 comp rx=0 tx=0).stop 2023-11-26T07:18:17.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 -- 172.21.15.31:0/1501619012 shutdown_connections 2023-11-26T07:18:17.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32340664f0 0x7f32340689b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32480b0170 0x7f32480b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3248155ef0 0x7f324814b590 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 --2- 172.21.15.31:0/1501619012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3230005190 0x7f3248154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 -- 172.21.15.31:0/1501619012 >> 172.21.15.31:0/1501619012 conn(0x7f3248071d80 msgr2=0x7f324806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 -- 172.21.15.31:0/1501619012 shutdown_connections 2023-11-26T07:18:17.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.508+0000 7f324df2a640 1 -- 172.21.15.31:0/1501619012 wait complete. 2023-11-26T07:18:17.540 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:18:17.540 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:18:17.540 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:18:17.541 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:17.541 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:18:17.541 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:18:17.542 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:18:17.542 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:17.542 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:18:17.542 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:18:17.542 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:18:17.542 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:17.543 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:18:17.543 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:18:17.543 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:18:17.543 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:17.555 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:18:17.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:18:17.557 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:18:17.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:18:17.565 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:18:17.565 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:18:17.566 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:18:17.566 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:18:17.602 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:18:17.602 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:17.604 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:18:17.604 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:17.614 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:18:17.614 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:17.616 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:18:17.616 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:17.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:18:17.704 INFO:tasks.ceph.mds.d.smithi097.stdout:starting mds.d at 2023-11-26T07:18:17.705 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2023-11-26T07:18:17.706 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-11-26T07:18:17.717 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-11-26T07:18:17.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a65a2f640 1 Processor -- start 2023-11-26T07:18:17.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a65a2f640 1 -- start start 2023-11-26T07:18:17.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a65a2f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a6015d2a0 0x7f1a600d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a65a2f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 0x7f1a600d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a65a2f640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a600b6250 con 0x7f1a600d2c20 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a65a2f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a600ce8c0 con 0x7f1a6015d2a0 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a65a2f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a60161cf0 con 0x7f1a600b5350 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a64a2d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a6015d2a0 0x7f1a600d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a5ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1a600d2c20 0x7f1a600b4c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47510/0 (socket says 172.21.15.31:47510) 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a5ffff640 1 -- 172.21.15.31:0/2244645293 learned_addr learned my addr 172.21.15.31:0/2244645293 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a6522e640 1 --2- 172.21.15.31:0/2244645293 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 0x7f1a600d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a64a2d640 1 -- 172.21.15.31:0/2244645293 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 msgr2=0x7f1a600d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a64a2d640 1 --2- 172.21.15.31:0/2244645293 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 0x7f1a600d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a64a2d640 1 -- 172.21.15.31:0/2244645293 >> v1:172.21.15.31:6789/0 conn(0x7f1a600d2c20 legacy=0x7f1a600b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a64a2d640 1 -- 172.21.15.31:0/2244645293 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a60161170 con 0x7f1a6015d2a0 2023-11-26T07:18:17.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.784+0000 7f1a5f7fe640 1 -- 172.21.15.31:0/2244645293 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546530301 0 0) 0x7f1a600b6250 con 0x7f1a600d2c20 2023-11-26T07:18:17.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a64a2d640 1 --2- 172.21.15.31:0/2244645293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 0x7f1a600d26e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1a540097c0 tx=0x7f1a5401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77b0c6b20efb509 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a5f7fe640 1 -- 172.21.15.31:0/2244645293 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a540ab650 con 0x7f1a6015d2a0 2023-11-26T07:18:17.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a5f7fe640 1 -- 172.21.15.31:0/2244645293 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a540ab7b0 con 0x7f1a6015d2a0 2023-11-26T07:18:17.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a5f7fe640 1 -- 172.21.15.31:0/2244645293 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a540abb40 con 0x7f1a6015d2a0 2023-11-26T07:18:17.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a65a2f640 1 -- 172.21.15.31:0/2244645293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 msgr2=0x7f1a600d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a65a2f640 1 --2- 172.21.15.31:0/2244645293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 0x7f1a600d26e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1a540097c0 tx=0x7f1a5401c450 comp rx=0 tx=0).stop 2023-11-26T07:18:17.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a65a2f640 1 -- 172.21.15.31:0/2244645293 shutdown_connections 2023-11-26T07:18:17.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a65a2f640 1 --2- 172.21.15.31:0/2244645293 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 0x7f1a600d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a65a2f640 1 --2- 172.21.15.31:0/2244645293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 0x7f1a600d26e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a65a2f640 1 -- 172.21.15.31:0/2244645293 >> 172.21.15.31:0/2244645293 conn(0x7f1a600668a0 msgr2=0x7f1a600cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:17.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a65a2f640 1 -- 172.21.15.31:0/2244645293 shutdown_connections 2023-11-26T07:18:17.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.788+0000 7f1a65a2f640 1 -- 172.21.15.31:0/2244645293 wait complete. 2023-11-26T07:18:17.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a65a2f640 1 Processor -- start 2023-11-26T07:18:17.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a65a2f640 1 -- start start 2023-11-26T07:18:17.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a65a2f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 0x7f1a6014d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a65a2f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a600d2c20 0x7f1a600b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a65a2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 0x7f1a600b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a65a2f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600cc370 con 0x7f1a6015d2a0 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a65a2f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600cb2d0 con 0x7f1a600d2c20 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a65a2f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600cb5b0 con 0x7f1a600b5350 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a6522e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 0x7f1a600b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a6522e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 0x7f1a600b85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48244/0 (socket says 172.21.15.31:48244) 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a6522e640 1 -- 172.21.15.31:0/3310544712 learned_addr learned my addr 172.21.15.31:0/3310544712 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a64a2d640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 0x7f1a6014d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a5ffff640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a600d2c20 0x7f1a600b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a6522e640 1 -- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 msgr2=0x7f1a6014d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a6522e640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 0x7f1a6014d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a6522e640 1 -- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a600d2c20 msgr2=0x7f1a600b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:17.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a6522e640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a600d2c20 0x7f1a600b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:17.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a6522e640 1 -- 172.21.15.31:0/3310544712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a54009420 con 0x7f1a6015d2a0 2023-11-26T07:18:17.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a64a2d640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 0x7f1a6014d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:17.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a6522e640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 0x7f1a600b85e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800efc0 tx=0x7f1a5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a5d7fa640 1 -- 172.21.15.31:0/3310544712 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58014480 con 0x7f1a6015d2a0 2023-11-26T07:18:17.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a5d7fa640 1 -- 172.21.15.31:0/3310544712 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5800f040 con 0x7f1a6015d2a0 2023-11-26T07:18:17.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a5d7fa640 1 -- 172.21.15.31:0/3310544712 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58014820 con 0x7f1a6015d2a0 2023-11-26T07:18:17.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a65a2f640 1 -- 172.21.15.31:0/3310544712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a600cb890 con 0x7f1a6015d2a0 2023-11-26T07:18:17.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.792+0000 7f1a65a2f640 1 -- 172.21.15.31:0/3310544712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a600cbe50 con 0x7f1a6015d2a0 2023-11-26T07:18:17.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.796+0000 7f1a65a2f640 1 -- 172.21.15.31:0/3310544712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a60074990 con 0x7f1a6015d2a0 2023-11-26T07:18:17.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.796+0000 7f1a5d7fa640 1 -- 172.21.15.31:0/3310544712 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1a58014980 con 0x7f1a6015d2a0 2023-11-26T07:18:17.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.796+0000 7f1a5d7fa640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1a44066470 0x7f1a44068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:17.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.796+0000 7f1a5d7fa640 1 -- 172.21.15.31:0/3310544712 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1a58084890 con 0x7f1a6015d2a0 2023-11-26T07:18:17.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.796+0000 7f1a64a2d640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1a44066470 0x7f1a44068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:17.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.796+0000 7f1a64a2d640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1a44066470 0x7f1a44068930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1a540af000 tx=0x7f1a540a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:17.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.800+0000 7f1a5d7fa640 1 -- 172.21.15.31:0/3310544712 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1a580520a0 con 0x7f1a6015d2a0 2023-11-26T07:18:17.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:17.968+0000 7f1a65a2f640 1 -- 172.21.15.31:0/3310544712 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1a600c8e80 con 0x7f1a6015d2a0 2023-11-26T07:18:18.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a5d7fa640 1 -- 172.21.15.31:0/3310544712 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1a58018080 con 0x7f1a6015d2a0 2023-11-26T07:18:18.540 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-11-26T07:18:18.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 -- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1a44066470 msgr2=0x7f1a44068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1a44066470 0x7f1a44068930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1a540af000 tx=0x7f1a540a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:18.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 -- 172.21.15.31:0/3310544712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 msgr2=0x7f1a600b85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 0x7f1a600b85e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800efc0 tx=0x7f1a5800c490 comp rx=0 tx=0).stop 2023-11-26T07:18:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 -- 172.21.15.31:0/3310544712 shutdown_connections 2023-11-26T07:18:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1a44066470 0x7f1a44068930 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a6015d2a0 0x7f1a600b85e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a600d2c20 0x7f1a600b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 --2- 172.21.15.31:0/3310544712 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a600b5350 0x7f1a6014d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 -- 172.21.15.31:0/3310544712 >> 172.21.15.31:0/3310544712 conn(0x7f1a600668a0 msgr2=0x7f1a600b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 -- 172.21.15.31:0/3310544712 shutdown_connections 2023-11-26T07:18:18.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.536+0000 7f1a65a2f640 1 -- 172.21.15.31:0/3310544712 wait complete. 2023-11-26T07:18:18.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:18:18.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.704+0000 7ff2390fe640 1 Processor -- start 2023-11-26T07:18:18.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.704+0000 7ff2390fe640 1 -- start start 2023-11-26T07:18:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff2390fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff234144c80 0x7ff2340abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff2390fe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff234145060 con 0x7ff2340ac460 2023-11-26T07:18:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff2390fe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff234071fd0 con 0x7ff234070df0 2023-11-26T07:18:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff2390fe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2340ac8c0 con 0x7ff234144c80 2023-11-26T07:18:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff232d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff234144c80 0x7ff2340abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff22bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff2340ac460 0x7ff2340706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47518/0 (socket says 172.21.15.31:47518) 2023-11-26T07:18:18.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff22bfff640 1 -- 172.21.15.31:0/3974291909 learned_addr learned my addr 172.21.15.31:0/3974291909 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:18.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff232d9d640 1 -- 172.21.15.31:0/3974291909 >> v1:172.21.15.97:6789/0 conn(0x7ff234070df0 legacy=0x7ff23406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff232d9d640 1 -- 172.21.15.31:0/3974291909 >> v1:172.21.15.31:6789/0 conn(0x7ff2340ac460 legacy=0x7ff2340706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff232d9d640 1 -- 172.21.15.31:0/3974291909 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff234071450 con 0x7ff234144c80 2023-11-26T07:18:18.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.708+0000 7ff232d9d640 1 --2- 172.21.15.31:0/3974291909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 0x7ff2340abf20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff2240097c0 tx=0x7ff22401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ac3b228e0d220d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff23259c640 1 -- 172.21.15.31:0/3974291909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2240ab600 con 0x7ff234144c80 2023-11-26T07:18:18.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff23259c640 1 -- 172.21.15.31:0/3974291909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2240a7050 con 0x7ff234144c80 2023-11-26T07:18:18.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff23259c640 1 -- 172.21.15.31:0/3974291909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2240ab980 con 0x7ff234144c80 2023-11-26T07:18:18.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 -- 172.21.15.31:0/3974291909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 msgr2=0x7ff2340abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 --2- 172.21.15.31:0/3974291909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 0x7ff2340abf20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff2240097c0 tx=0x7ff22401c480 comp rx=0 tx=0).stop 2023-11-26T07:18:18.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 -- 172.21.15.31:0/3974291909 shutdown_connections 2023-11-26T07:18:18.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 --2- 172.21.15.31:0/3974291909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 0x7ff2340abf20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 -- 172.21.15.31:0/3974291909 >> 172.21.15.31:0/3974291909 conn(0x7ff2340b0c40 msgr2=0x7ff2340b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 -- 172.21.15.31:0/3974291909 shutdown_connections 2023-11-26T07:18:18.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 -- 172.21.15.31:0/3974291909 wait complete. 2023-11-26T07:18:18.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 Processor -- start 2023-11-26T07:18:18.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 -- start start 2023-11-26T07:18:18.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff234070df0 0x7ff234164de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2340ac460 0x7ff23415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 0x7ff23409a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23415ba40 con 0x7ff234144c80 2023-11-26T07:18:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341653c0 con 0x7ff234070df0 2023-11-26T07:18:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.712+0000 7ff2390fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341656a0 con 0x7ff2340ac460 2023-11-26T07:18:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff23359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 0x7ff23409a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff23359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 0x7ff23409a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48254/0 (socket says 172.21.15.31:48254) 2023-11-26T07:18:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff23359e640 1 -- 172.21.15.31:0/3509545303 learned_addr learned my addr 172.21.15.31:0/3509545303 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff232d9d640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff234070df0 0x7ff234164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff22bfff640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2340ac460 0x7ff23415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff23359e640 1 -- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2340ac460 msgr2=0x7ff23415ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff23359e640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2340ac460 0x7ff23415ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff23359e640 1 -- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff234070df0 msgr2=0x7ff234164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff23359e640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff234070df0 0x7ff234164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff23359e640 1 -- 172.21.15.31:0/3509545303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224009420 con 0x7ff234144c80 2023-11-26T07:18:18.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff23359e640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 0x7ff23409a810 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c730 tx=0x7ff22c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff22b7fe640 1 -- 172.21.15.31:0/3509545303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c003b80 con 0x7ff234144c80 2023-11-26T07:18:18.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff22b7fe640 1 -- 172.21.15.31:0/3509545303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c003ce0 con 0x7ff234144c80 2023-11-26T07:18:18.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff22b7fe640 1 -- 172.21.15.31:0/3509545303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c00aa30 con 0x7ff234144c80 2023-11-26T07:18:18.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff2390fe640 1 -- 172.21.15.31:0/3509545303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff234165980 con 0x7ff234144c80 2023-11-26T07:18:18.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff2390fe640 1 -- 172.21.15.31:0/3509545303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff234165c60 con 0x7ff234144c80 2023-11-26T07:18:18.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff2390fe640 1 -- 172.21.15.31:0/3509545303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2340b1ba0 con 0x7ff234144c80 2023-11-26T07:18:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff22b7fe640 1 -- 172.21.15.31:0/3509545303 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7ff22c00ab90 con 0x7ff234144c80 2023-11-26T07:18:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff22b7fe640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff214066470 0x7ff214068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.716+0000 7ff22b7fe640 1 -- 172.21.15.31:0/3509545303 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff22c083720 con 0x7ff234144c80 2023-11-26T07:18:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.720+0000 7ff232d9d640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff214066470 0x7ff214068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.720+0000 7ff232d9d640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff214066470 0x7ff214068930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff2240ad000 tx=0x7ff224008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:18.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.724+0000 7ff22b7fe640 1 -- 172.21.15.31:0/3509545303 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff22c051030 con 0x7ff234144c80 2023-11-26T07:18:18.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.932+0000 7ff2390fe640 1 -- 172.21.15.31:0/3509545303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff23404e850 con 0x7ff234144c80 2023-11-26T07:18:18.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff22b7fe640 1 -- 172.21.15.31:0/3509545303 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7ff22c054650 con 0x7ff234144c80 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlRJmWGBAAuUAS68YFt9PaqDV8RFi+6w==", 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:18:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlIdH1JBAAmPlAQXq78ht4pJf6I7Cd5w==", 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlfuXoHxAAf7vKZr+LVZaN6HR3qiks5w==", 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlWmgkLBAAojAzX06O7kbwmj154qUGog==", 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-11-26T07:18:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDU8GJlc/bqEBAAueOQvSeK6MW+WOT746yQzA==", 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDX8GJlEaW6FxAA8q4rEKSJtIl/Hr2uCNSMUg==", 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDa8GJldV90HxAAQRgcLf8ABSU4Sfb3A1Xa4g==", 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDd8GJl2erPJhAAoC71eDwCOmskb+VCLYQyUw==", 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDj8GJllq08LxAAGuz9+Ou/ybWxdkN3PSMaEQ==", 2023-11-26T07:18:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDm8GJlww4eNxAAjRaBq45nrNwJPlopyOhXNg==", 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDq8GJl2jxiAhAAhu2OULy84YRJYxTmShoVBg==", 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDt8GJld8HNCBAA3deA4HKTqRggbVc73Y8efg==", 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA==", 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlEBAQOhAAx5TsMN17L2fNm+j3yqUIOQ==", 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:18:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl/CN+NhAAeyry8gIOtxwLxCQb0t1eKw==", 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:18:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDR8GJlA9e7BBAATe3WYA79PgfLc5UHgKGkpw==", 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDP8GJlfT+4NhAACA0t23bQtWvdFLsB9X+X8w==", 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-mds", 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDz8GJlFHMyDRAAx3os0uXmG2JhVbV8f/1ctg==", 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-mds" 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-mgr", 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDz8GJl3n4yDRAAm0TdpHFsYkIdWt0LqVnaaQ==", 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-osd", 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDz8GJlMocyDRAAVVBq/khidBuPFQSW6lFqPA==", 2023-11-26T07:18:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-osd" 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rbd", 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDz8GJlYo8yDRAAuEKlvKs7GJ1j6M+6TDtMUg==", 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-26T07:18:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDz8GJltJcyDRAARrMUeOvXipQouEmnm3lJaA==", 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.bootstrap-rgw", 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDz8GJlO6AyDRAARR3zhOgUxQ7JykO8YGSF4w==", 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlnh/pExAA6qyPM7So2BLbxbGbquLZZA==", 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlYhb/DxAAx1G3Ve23IdZ7cX9nPb2WhA==", 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:18:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:18:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:18:18.958 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:18:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 -- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff214066470 msgr2=0x7ff214068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff214066470 0x7ff214068930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff2240ad000 tx=0x7ff224008040 comp rx=0 tx=0).stop 2023-11-26T07:18:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 -- 172.21.15.31:0/3509545303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 msgr2=0x7ff23409a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 0x7ff23409a810 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c730 tx=0x7ff22c00cbf0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 -- 172.21.15.31:0/3509545303 shutdown_connections 2023-11-26T07:18:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff214066470 0x7ff214068930 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff234144c80 0x7ff23409a810 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff2340ac460 0x7ff23415ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 --2- 172.21.15.31:0/3509545303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff234070df0 0x7ff234164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:18.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 -- 172.21.15.31:0/3509545303 >> 172.21.15.31:0/3509545303 conn(0x7ff2340b0c40 msgr2=0x7ff2340add30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:18.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 -- 172.21.15.31:0/3509545303 shutdown_connections 2023-11-26T07:18:18.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:18.936+0000 7ff2390fe640 1 -- 172.21.15.31:0/3509545303 wait complete. 2023-11-26T07:18:18.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-26T07:18:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea2aea640 1 Processor -- start 2023-11-26T07:18:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea2aea640 1 -- start start 2023-11-26T07:18:19.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea2aea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1e9c0cb9b0 0x7f1e9c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea2aea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e9c0ce5d0 0x7f1e9c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea2aea640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c159630 con 0x7f1e9c0cbd90 2023-11-26T07:18:19.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea2aea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9c15bba0 con 0x7f1e9c0cb9b0 2023-11-26T07:18:19.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea2aea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0cdd50 con 0x7f1e9c0ce5d0 2023-11-26T07:18:19.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1e9c0cbd90 0x7f1e9c15f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47530/0 (socket says 172.21.15.31:47530) 2023-11-26T07:18:19.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9bfff640 1 -- 172.21.15.31:0/4121168393 learned_addr learned my addr 172.21.15.31:0/4121168393 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:19.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea0886640 1 --2- 172.21.15.31:0/4121168393 >> v2:172.21.15.97:3300/0 conn(0x7f1e9c0cb9b0 0x7f1e9c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea1087640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e9c0ce5d0 0x7f1e9c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504610025 0 0) 0x7f1e9c159630 con 0x7f1e9c0cbd90 2023-11-26T07:18:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e8c003530 con 0x7f1e9c0cbd90 2023-11-26T07:18:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1689583258 0 0) 0x7f1e8c003530 con 0x7f1e9c0cbd90 2023-11-26T07:18:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e9c0ce5d0 msgr2=0x7f1e9c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e9c0ce5d0 0x7f1e9c15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 >> v2:172.21.15.97:3300/0 conn(0x7f1e9c0cb9b0 msgr2=0x7f1e9c06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 --2- 172.21.15.31:0/4121168393 >> v2:172.21.15.97:3300/0 conn(0x7f1e9c0cb9b0 0x7f1e9c06e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c0cd1d0 con 0x7f1e9c0cbd90 2023-11-26T07:18:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f1e88001680 con 0x7f1e9c0cbd90 2023-11-26T07:18:19.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 >> v1:172.21.15.31:6789/0 conn(0x7f1e9c0cbd90 legacy=0x7f1e9c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e8c003980 0x7f1e8c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8c0047d0 0x7f1e8c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e9c0ce5d0 0x7f1e8c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0cdd50 con 0x7f1e8c003980 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c00b990 con 0x7f1e8c0047d0 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c00bad0 con 0x7f1e9c0ce5d0 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea0886640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e8c003980 0x7f1e8c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea1087640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e9c0ce5d0 0x7f1e8c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1e9bfff640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8c0047d0 0x7f1e8c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea0886640 1 -- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e9c0ce5d0 msgr2=0x7f1e8c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea0886640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e9c0ce5d0 0x7f1e8c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea0886640 1 -- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8c0047d0 msgr2=0x7f1e8c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:19.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea0886640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e8c0047d0 0x7f1e8c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.100+0000 7f1ea0886640 1 -- 172.21.15.31:0/4121168393 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c003530 con 0x7f1e8c003980 2023-11-26T07:18:19.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1ea0886640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e8c003980 0x7f1e8c003da0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009ae0 tx=0x7f1e9001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe07080a8729893d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e900a5550 con 0x7f1e8c003980 2023-11-26T07:18:19.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e900a56b0 con 0x7f1e8c003980 2023-11-26T07:18:19.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1e9b7fe640 1 -- 172.21.15.31:0/4121168393 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e900a5a60 con 0x7f1e8c003980 2023-11-26T07:18:19.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1ea2aea640 1 -- 172.21.15.31:0/4121168393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e8c003980 msgr2=0x7f1e8c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1ea2aea640 1 --2- 172.21.15.31:0/4121168393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e8c003980 0x7f1e8c003da0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009ae0 tx=0x7f1e9001ca00 comp rx=0 tx=0).stop 2023-11-26T07:18:19.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1ea1087640 1 -- 172.21.15.31:0/4121168393 reap_dead start 2023-11-26T07:18:19.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1ea2aea640 1 -- 172.21.15.31:0/4121168393 shutdown_connections 2023-11-26T07:18:19.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1ea2aea640 1 -- 172.21.15.31:0/4121168393 >> 172.21.15.31:0/4121168393 conn(0x7f1e9c0b0cb0 msgr2=0x7f1e9c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1ea2aea640 1 -- 172.21.15.31:0/4121168393 shutdown_connections 2023-11-26T07:18:19.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.104+0000 7f1ea2aea640 1 -- 172.21.15.31:0/4121168393 wait complete. 2023-11-26T07:18:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea2aea640 1 Processor -- start 2023-11-26T07:18:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea2aea640 1 -- start start 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea2aea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e8c003980 0x7f1e9c0b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea2aea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9c1f5f10 0x7f1e9c1f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea2aea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e9c0abf90 0x7f1e9c0ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea2aea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c1f6a00 con 0x7f1e9c0abf90 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea2aea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c1f6ce0 con 0x7f1e9c1f5f10 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea2aea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0c9600 con 0x7f1e8c003980 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea1087640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e9c0abf90 0x7f1e9c0ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea1087640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e9c0abf90 0x7f1e9c0ac410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48280/0 (socket says 172.21.15.31:48280) 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea1087640 1 -- 172.21.15.31:0/1411863865 learned_addr learned my addr 172.21.15.31:0/1411863865 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1e9bfff640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9c1f5f10 0x7f1e9c1f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea0886640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e8c003980 0x7f1e9c0b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea1087640 1 -- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e8c003980 msgr2=0x7f1e9c0b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea1087640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e8c003980 0x7f1e9c0b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea1087640 1 -- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9c1f5f10 msgr2=0x7f1e9c1f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea1087640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9c1f5f10 0x7f1e9c1f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea1087640 1 -- 172.21.15.31:0/1411863865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c0c98e0 con 0x7f1e9c0abf90 2023-11-26T07:18:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea0886640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e8c003980 0x7f1e9c0b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea1087640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e9c0abf90 0x7f1e9c0ac410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c070450 tx=0x7f1e9400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1e997fa640 1 -- 172.21.15.31:0/1411863865 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400e040 con 0x7f1e9c0abf90 2023-11-26T07:18:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1e997fa640 1 -- 172.21.15.31:0/1411863865 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9400f040 con 0x7f1e9c0abf90 2023-11-26T07:18:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1e997fa640 1 -- 172.21.15.31:0/1411863865 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94015670 con 0x7f1e9c0abf90 2023-11-26T07:18:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea2aea640 1 -- 172.21.15.31:0/1411863865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c0cacd0 con 0x7f1e9c0abf90 2023-11-26T07:18:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.108+0000 7f1ea2aea640 1 -- 172.21.15.31:0/1411863865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c0c9e00 con 0x7f1e9c0abf90 2023-11-26T07:18:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.112+0000 7f1ea2aea640 1 -- 172.21.15.31:0/1411863865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c0671f0 con 0x7f1e9c0abf90 2023-11-26T07:18:19.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.112+0000 7f1e997fa640 1 -- 172.21.15.31:0/1411863865 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1e940157d0 con 0x7f1e9c0abf90 2023-11-26T07:18:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.112+0000 7f1e997fa640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1e800664a0 0x7f1e80068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.112+0000 7f1e997fa640 1 -- 172.21.15.31:0/1411863865 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1e94084540 con 0x7f1e9c0abf90 2023-11-26T07:18:19.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.112+0000 7f1ea0886640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1e800664a0 0x7f1e80068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.112+0000 7f1ea0886640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1e800664a0 0x7f1e80068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1e900af000 tx=0x7f1e900046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.116+0000 7f1e997fa640 1 -- 172.21.15.31:0/1411863865 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1e94051d00 con 0x7f1e9c0abf90 2023-11-26T07:18:19.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.324+0000 7f1ea2aea640 1 -- 172.21.15.31:0/1411863865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f1e9c1f6ea0 con 0x7f1e9c0abf90 2023-11-26T07:18:19.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.328+0000 7f1e997fa640 1 -- 172.21.15.31:0/1411863865 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1e9c1f6ea0 con 0x7f1e9c0abf90 2023-11-26T07:18:19.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 -- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1e800664a0 msgr2=0x7f1e80068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1e800664a0 0x7f1e80068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1e900af000 tx=0x7f1e900046d0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 -- 172.21.15.31:0/1411863865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e9c0abf90 msgr2=0x7f1e9c0ac410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e9c0abf90 0x7f1e9c0ac410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c070450 tx=0x7f1e9400f580 comp rx=0 tx=0).stop 2023-11-26T07:18:19.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 -- 172.21.15.31:0/1411863865 shutdown_connections 2023-11-26T07:18:19.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1e800664a0 0x7f1e80068960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e9c0abf90 0x7f1e9c0ac410 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e9c1f5f10 0x7f1e9c1f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 --2- 172.21.15.31:0/1411863865 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1e8c003980 0x7f1e9c0b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 -- 172.21.15.31:0/1411863865 >> 172.21.15.31:0/1411863865 conn(0x7f1e9c0b0cb0 msgr2=0x7f1e9c071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 -- 172.21.15.31:0/1411863865 shutdown_connections 2023-11-26T07:18:19.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.332+0000 7f1ea2aea640 1 -- 172.21.15.31:0/1411863865 wait complete. 2023-11-26T07:18:19.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-26T07:18:19.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57fefe640 1 Processor -- start 2023-11-26T07:18:19.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57fefe640 1 -- start start 2023-11-26T07:18:19.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57fefe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5780ae680 0x7fa5780cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57fefe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa578072950 con 0x7fa578070890 2023-11-26T07:18:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57fefe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57806e2b0 con 0x7fa578159f00 2023-11-26T07:18:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57fefe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57806c6f0 con 0x7fa5780ae680 2023-11-26T07:18:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57e49b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa578070890 0x7fa578070cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47544/0 (socket says 172.21.15.31:47544) 2023-11-26T07:18:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57e49b640 1 -- 172.21.15.31:0/3813368284 learned_addr learned my addr 172.21.15.31:0/3813368284 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:19.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57d499640 1 --2- 172.21.15.31:0/3813368284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5780ae680 0x7fa5780cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57cc98640 1 -- 172.21.15.31:0/3813368284 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874970320 0 0) 0x7fa578072950 con 0x7fa578070890 2023-11-26T07:18:19.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57cc98640 1 -- 172.21.15.31:0/3813368284 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa564004640 con 0x7fa578070890 2023-11-26T07:18:19.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57cc98640 1 -- 172.21.15.31:0/3813368284 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309391977 0 0) 0x7fa57806e2b0 con 0x7fa578159f00 2023-11-26T07:18:19.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57cc98640 1 -- 172.21.15.31:0/3813368284 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa578072950 con 0x7fa578159f00 2023-11-26T07:18:19.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57d499640 1 -- 172.21.15.31:0/3813368284 >> v1:172.21.15.97:6789/0 conn(0x7fa578159f00 legacy=0x7fa5780ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57d499640 1 -- 172.21.15.31:0/3813368284 >> v1:172.21.15.31:6789/0 conn(0x7fa578070890 legacy=0x7fa578070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57cc98640 1 -- 172.21.15.31:0/3813368284 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 121865122 0 0) 0x7fa564004640 con 0x7fa578070890 2023-11-26T07:18:19.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.496+0000 7fa57d499640 1 -- 172.21.15.31:0/3813368284 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57806e550 con 0x7fa5780ae680 2023-11-26T07:18:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57d499640 1 --2- 172.21.15.31:0/3813368284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5780ae680 0x7fa5780cd5b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5680097f0 tx=0x7fa56801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d3dd3de7771e6f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57cc98640 1 -- 172.21.15.31:0/3813368284 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa570001a00 con 0x7fa5780ae680 2023-11-26T07:18:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57cc98640 1 -- 172.21.15.31:0/3813368284 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5680a7050 con 0x7fa5780ae680 2023-11-26T07:18:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57cc98640 1 -- 172.21.15.31:0/3813368284 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680ab600 con 0x7fa5780ae680 2023-11-26T07:18:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57fefe640 1 -- 172.21.15.31:0/3813368284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5780ae680 msgr2=0x7fa5780cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57fefe640 1 --2- 172.21.15.31:0/3813368284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5780ae680 0x7fa5780cd5b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5680097f0 tx=0x7fa56801c3b0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57fefe640 1 -- 172.21.15.31:0/3813368284 shutdown_connections 2023-11-26T07:18:19.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57fefe640 1 --2- 172.21.15.31:0/3813368284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5780ae680 0x7fa5780cd5b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57fefe640 1 -- 172.21.15.31:0/3813368284 >> 172.21.15.31:0/3813368284 conn(0x7fa5780b0cb0 msgr2=0x7fa5780b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57fefe640 1 -- 172.21.15.31:0/3813368284 shutdown_connections 2023-11-26T07:18:19.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.500+0000 7fa57fefe640 1 -- 172.21.15.31:0/3813368284 wait complete. 2023-11-26T07:18:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57fefe640 1 Processor -- start 2023-11-26T07:18:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57fefe640 1 -- start start 2023-11-26T07:18:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57fefe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa578070890 0x7fa5781fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57fefe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ae680 0x7fa57814b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57fefe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5780ad920 0x7fa578152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57fefe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57814c280 con 0x7fa5780ae680 2023-11-26T07:18:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57fefe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57814bd10 con 0x7fa5780ad920 2023-11-26T07:18:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57fefe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa578159230 con 0x7fa578070890 2023-11-26T07:18:19.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57d499640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ae680 0x7fa57814b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57d499640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ae680 0x7fa57814b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48282/0 (socket says 172.21.15.31:48282) 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57d499640 1 -- 172.21.15.31:0/1502388784 learned_addr learned my addr 172.21.15.31:0/1502388784 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57e49b640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5780ad920 0x7fa578152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57dc9a640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa578070890 0x7fa5781fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57d499640 1 -- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa578070890 msgr2=0x7fa5781fa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57d499640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa578070890 0x7fa5781fa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57d499640 1 -- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5780ad920 msgr2=0x7fa578152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57d499640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5780ad920 0x7fa578152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57d499640 1 -- 172.21.15.31:0/1502388784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa568009420 con 0x7fa5780ae680 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57dc9a640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa578070890 0x7fa5781fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57e49b640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5780ad920 0x7fa578152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:19.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57d499640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ae680 0x7fa57814b7d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa568000c10 tx=0x7fa568019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa56e7fc640 1 -- 172.21.15.31:0/1502388784 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680b9070 con 0x7fa5780ae680 2023-11-26T07:18:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa56e7fc640 1 -- 172.21.15.31:0/1502388784 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5680038b0 con 0x7fa5780ae680 2023-11-26T07:18:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa56e7fc640 1 -- 172.21.15.31:0/1502388784 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568003c20 con 0x7fa5780ae680 2023-11-26T07:18:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57fefe640 1 -- 172.21.15.31:0/1502388784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa578159510 con 0x7fa5780ae680 2023-11-26T07:18:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.504+0000 7fa57fefe640 1 -- 172.21.15.31:0/1502388784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5781f98f0 con 0x7fa5780ae680 2023-11-26T07:18:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.508+0000 7fa57fefe640 1 -- 172.21.15.31:0/1502388784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5780671f0 con 0x7fa5780ae680 2023-11-26T07:18:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.508+0000 7fa56e7fc640 1 -- 172.21.15.31:0/1502388784 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa5680b2040 con 0x7fa5780ae680 2023-11-26T07:18:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.508+0000 7fa56e7fc640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa55c066470 0x7fa55c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.508+0000 7fa56e7fc640 1 -- 172.21.15.31:0/1502388784 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa56811d2a0 con 0x7fa5780ae680 2023-11-26T07:18:19.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.508+0000 7fa57dc9a640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa55c066470 0x7fa55c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.508+0000 7fa57dc9a640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa55c066470 0x7fa55c068930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa5740011f0 tx=0x7fa574009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.512+0000 7fa56e7fc640 1 -- 172.21.15.31:0/1502388784 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa5680ea9d0 con 0x7fa5780ae680 2023-11-26T07:18:19.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.736+0000 7fa57fefe640 1 -- 172.21.15.31:0/1502388784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fa57806b460 con 0x7fa5780ae680 2023-11-26T07:18:19.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa56e7fc640 1 -- 172.21.15.31:0/1502388784 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa5680edff0 con 0x7fa5780ae680 2023-11-26T07:18:19.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 -- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa55c066470 msgr2=0x7fa55c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa55c066470 0x7fa55c068930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa5740011f0 tx=0x7fa574009040 comp rx=0 tx=0).stop 2023-11-26T07:18:19.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 -- 172.21.15.31:0/1502388784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ae680 msgr2=0x7fa57814b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ae680 0x7fa57814b7d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa568000c10 tx=0x7fa568019150 comp rx=0 tx=0).stop 2023-11-26T07:18:19.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 -- 172.21.15.31:0/1502388784 shutdown_connections 2023-11-26T07:18:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa55c066470 0x7fa55c068930 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5780ad920 0x7fa578152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5780ae680 0x7fa57814b7d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 --2- 172.21.15.31:0/1502388784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa578070890 0x7fa5781fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 -- 172.21.15.31:0/1502388784 >> 172.21.15.31:0/1502388784 conn(0x7fa5780b0cb0 msgr2=0x7fa5780ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 -- 172.21.15.31:0/1502388784 shutdown_connections 2023-11-26T07:18:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.740+0000 7fa57fefe640 1 -- 172.21.15.31:0/1502388784 wait complete. 2023-11-26T07:18:19.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-26T07:18:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.904+0000 7f1936df8640 1 Processor -- start 2023-11-26T07:18:19.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.904+0000 7f1936df8640 1 -- start start 2023-11-26T07:18:19.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.904+0000 7f1936df8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f19300b4210 0x7f19300b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.904+0000 7f1936df8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1930147190 con 0x7f193015db40 2023-11-26T07:18:19.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.904+0000 7f1936df8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193006d030 con 0x7f1930146820 2023-11-26T07:18:19.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.904+0000 7f1936df8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19300b4be0 con 0x7f19300b4210 2023-11-26T07:18:19.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.904+0000 7f1935395640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f193015db40 0x7f193015df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47552/0 (socket says 172.21.15.31:47552) 2023-11-26T07:18:19.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.904+0000 7f1935395640 1 -- 172.21.15.31:0/3730115736 learned_addr learned my addr 172.21.15.31:0/3730115736 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:19.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192ffff640 1 --2- 172.21.15.31:0/3730115736 >> v2:172.21.15.97:3300/0 conn(0x7f19300b4210 0x7f19300b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192f7fe640 1 -- 172.21.15.31:0/3730115736 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664895910 0 0) 0x7f1930147190 con 0x7f193015db40 2023-11-26T07:18:19.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192f7fe640 1 -- 172.21.15.31:0/3730115736 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1920003530 con 0x7f193015db40 2023-11-26T07:18:19.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192f7fe640 1 -- 172.21.15.31:0/3730115736 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952171454 0 0) 0x7f193006d030 con 0x7f1930146820 2023-11-26T07:18:19.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192f7fe640 1 -- 172.21.15.31:0/3730115736 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1930147190 con 0x7f1930146820 2023-11-26T07:18:19.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192ffff640 1 -- 172.21.15.31:0/3730115736 >> v1:172.21.15.97:6789/0 conn(0x7f1930146820 legacy=0x7f1930070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192ffff640 1 -- 172.21.15.31:0/3730115736 >> v1:172.21.15.31:6789/0 conn(0x7f193015db40 legacy=0x7f193015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192ffff640 1 -- 172.21.15.31:0/3730115736 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1930146c00 con 0x7f19300b4210 2023-11-26T07:18:19.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192ffff640 1 --2- 172.21.15.31:0/3730115736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19300b4210 0x7f19300b46a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f191c0097c0 tx=0x7f191c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a1fb01f6449b61d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192f7fe640 1 -- 172.21.15.31:0/3730115736 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c0ab650 con 0x7f19300b4210 2023-11-26T07:18:19.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192f7fe640 1 -- 172.21.15.31:0/3730115736 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c0ab7b0 con 0x7f19300b4210 2023-11-26T07:18:19.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.908+0000 7f192f7fe640 1 -- 172.21.15.31:0/3730115736 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c0abb40 con 0x7f19300b4210 2023-11-26T07:18:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 -- 172.21.15.31:0/3730115736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19300b4210 msgr2=0x7f19300b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 --2- 172.21.15.31:0/3730115736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19300b4210 0x7f19300b46a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f191c0097c0 tx=0x7f191c01c450 comp rx=0 tx=0).stop 2023-11-26T07:18:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 -- 172.21.15.31:0/3730115736 shutdown_connections 2023-11-26T07:18:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 --2- 172.21.15.31:0/3730115736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19300b4210 0x7f19300b46a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 -- 172.21.15.31:0/3730115736 >> 172.21.15.31:0/3730115736 conn(0x7f19300b2f10 msgr2=0x7f19300b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 -- 172.21.15.31:0/3730115736 shutdown_connections 2023-11-26T07:18:19.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 -- 172.21.15.31:0/3730115736 wait complete. 2023-11-26T07:18:19.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 Processor -- start 2023-11-26T07:18:19.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 -- start start 2023-11-26T07:18:19.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19300b4210 0x7f193015a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1930146820 0x7f193013f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f193015db40 0x7f1930141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193013dce0 con 0x7f1930146820 2023-11-26T07:18:19.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193013dfc0 con 0x7f193015db40 2023-11-26T07:18:19.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1936df8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193013e2a0 con 0x7f19300b4210 2023-11-26T07:18:19.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f192ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1930146820 0x7f193013f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f192ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1930146820 0x7f193013f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48290/0 (socket says 172.21.15.31:48290) 2023-11-26T07:18:19.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f192ffff640 1 -- 172.21.15.31:0/2702023058 learned_addr learned my addr 172.21.15.31:0/2702023058 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1934b94640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19300b4210 0x7f193015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1935395640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f193015db40 0x7f1930141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f192ffff640 1 -- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19300b4210 msgr2=0x7f193015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f192ffff640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19300b4210 0x7f193015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f192ffff640 1 -- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f193015db40 msgr2=0x7f1930141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f192ffff640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f193015db40 0x7f1930141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f192ffff640 1 -- 172.21.15.31:0/2702023058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c009420 con 0x7f1930146820 2023-11-26T07:18:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.912+0000 7f1935395640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f193015db40 0x7f1930141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f192ffff640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1930146820 0x7f193013f8c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f191c0ad040 tx=0x7f191c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f192d7fa640 1 -- 172.21.15.31:0/2702023058 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c0bf610 con 0x7f1930146820 2023-11-26T07:18:19.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f192d7fa640 1 -- 172.21.15.31:0/2702023058 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c0bf7b0 con 0x7f1930146820 2023-11-26T07:18:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f192d7fa640 1 -- 172.21.15.31:0/2702023058 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c0bfb60 con 0x7f1930146820 2023-11-26T07:18:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f1936df8640 1 -- 172.21.15.31:0/2702023058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193013e580 con 0x7f1930146820 2023-11-26T07:18:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f1936df8640 1 -- 172.21.15.31:0/2702023058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193013d9d0 con 0x7f1930146820 2023-11-26T07:18:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f1936df8640 1 -- 172.21.15.31:0/2702023058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193006bb10 con 0x7f1930146820 2023-11-26T07:18:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f192d7fa640 1 -- 172.21.15.31:0/2702023058 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f191c0b5400 con 0x7f1930146820 2023-11-26T07:18:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f192d7fa640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1914066470 0x7f1914068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f192d7fa640 1 -- 172.21.15.31:0/2702023058 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f191c11e4e0 con 0x7f1930146820 2023-11-26T07:18:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.916+0000 7f1934b94640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1914066470 0x7f1914068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:19.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.920+0000 7f1934b94640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1914066470 0x7f1914068930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19240011f0 tx=0x7f1924009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:19.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:19.920+0000 7f192d7fa640 1 -- 172.21.15.31:0/2702023058 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f191c0ebcc0 con 0x7f1930146820 2023-11-26T07:18:20.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.140+0000 7f1936df8640 1 -- 172.21.15.31:0/2702023058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f19300b4070 con 0x7f1930146820 2023-11-26T07:18:20.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.144+0000 7f192d7fa640 1 -- 172.21.15.31:0/2702023058 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f19300b4070 con 0x7f1930146820 2023-11-26T07:18:20.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.144+0000 7f1936df8640 1 -- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1914066470 msgr2=0x7f1914068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.144+0000 7f1936df8640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1914066470 0x7f1914068930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19240011f0 tx=0x7f1924009040 comp rx=0 tx=0).stop 2023-11-26T07:18:20.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.144+0000 7f1936df8640 1 -- 172.21.15.31:0/2702023058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1930146820 msgr2=0x7f193013f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.144+0000 7f1936df8640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1930146820 0x7f193013f8c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f191c0ad040 tx=0x7f191c01ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:20.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.148+0000 7f1936df8640 1 -- 172.21.15.31:0/2702023058 shutdown_connections 2023-11-26T07:18:20.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.148+0000 7f1936df8640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1914066470 0x7f1914068930 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.148+0000 7f1936df8640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f193015db40 0x7f1930141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.148+0000 7f1936df8640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1930146820 0x7f193013f8c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.148+0000 7f1936df8640 1 --2- 172.21.15.31:0/2702023058 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f19300b4210 0x7f193015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.148+0000 7f1936df8640 1 -- 172.21.15.31:0/2702023058 >> 172.21.15.31:0/2702023058 conn(0x7f19300b2f10 msgr2=0x7f193015df50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.148+0000 7f1936df8640 1 -- 172.21.15.31:0/2702023058 shutdown_connections 2023-11-26T07:18:20.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.148+0000 7f1936df8640 1 -- 172.21.15.31:0/2702023058 wait complete. 2023-11-26T07:18:20.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-26T07:18:20.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.308+0000 7fb2f272a640 1 Processor -- start 2023-11-26T07:18:20.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.308+0000 7fb2f272a640 1 -- start start 2023-11-26T07:18:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2f272a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 0x7fb2ec0cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2f272a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ec072950 con 0x7fb2ec0cd5b0 2023-11-26T07:18:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2f272a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ec06e2b0 con 0x7fb2ec0ae680 2023-11-26T07:18:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2f272a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec06c6f0 con 0x7fb2ec159f00 2023-11-26T07:18:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2f0cc7640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb2ec0cd5b0 0x7fb2ec164b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47566/0 (socket says 172.21.15.31:47566) 2023-11-26T07:18:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2f0cc7640 1 -- 172.21.15.31:0/697987515 learned_addr learned my addr 172.21.15.31:0/697987515 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:20.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2ebfff640 1 --2- 172.21.15.31:0/697987515 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 0x7fb2ec0cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2eaffd640 1 -- 172.21.15.31:0/697987515 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860253042 0 0) 0x7fb2ec072950 con 0x7fb2ec0cd5b0 2023-11-26T07:18:20.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2eaffd640 1 -- 172.21.15.31:0/697987515 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d4005740 con 0x7fb2ec0cd5b0 2023-11-26T07:18:20.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2eaffd640 1 -- 172.21.15.31:0/697987515 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355831578 0 0) 0x7fb2ec06e2b0 con 0x7fb2ec0ae680 2023-11-26T07:18:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2eaffd640 1 -- 172.21.15.31:0/697987515 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2ec072950 con 0x7fb2ec0ae680 2023-11-26T07:18:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2ebfff640 1 -- 172.21.15.31:0/697987515 >> v1:172.21.15.97:6789/0 conn(0x7fb2ec0ae680 legacy=0x7fb2ec0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2ebfff640 1 -- 172.21.15.31:0/697987515 >> v1:172.21.15.31:6789/0 conn(0x7fb2ec0cd5b0 legacy=0x7fb2ec164b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2eaffd640 1 -- 172.21.15.31:0/697987515 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2557207310 0 0) 0x7fb2d4005740 con 0x7fb2ec0cd5b0 2023-11-26T07:18:20.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.312+0000 7fb2ebfff640 1 -- 172.21.15.31:0/697987515 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec06e550 con 0x7fb2ec159f00 2023-11-26T07:18:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2ebfff640 1 --2- 172.21.15.31:0/697987515 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 0x7fb2ec0cba20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0097f0 tx=0x7fb2dc01c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff5b56e38e1d621 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2eaffd640 1 -- 172.21.15.31:0/697987515 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4001a00 con 0x7fb2ec159f00 2023-11-26T07:18:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2eaffd640 1 -- 172.21.15.31:0/697987515 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc0a7050 con 0x7fb2ec159f00 2023-11-26T07:18:20.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2eaffd640 1 -- 172.21.15.31:0/697987515 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0ab600 con 0x7fb2ec159f00 2023-11-26T07:18:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2f272a640 1 -- 172.21.15.31:0/697987515 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 msgr2=0x7fb2ec0cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2f272a640 1 --2- 172.21.15.31:0/697987515 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 0x7fb2ec0cba20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0097f0 tx=0x7fb2dc01c3b0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2f272a640 1 -- 172.21.15.31:0/697987515 shutdown_connections 2023-11-26T07:18:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2f272a640 1 --2- 172.21.15.31:0/697987515 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 0x7fb2ec0cba20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2f272a640 1 -- 172.21.15.31:0/697987515 >> 172.21.15.31:0/697987515 conn(0x7fb2ec0b0cb0 msgr2=0x7fb2ec0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2f272a640 1 -- 172.21.15.31:0/697987515 shutdown_connections 2023-11-26T07:18:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.316+0000 7fb2f272a640 1 -- 172.21.15.31:0/697987515 wait complete. 2023-11-26T07:18:20.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 Processor -- start 2023-11-26T07:18:20.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 -- start start 2023-11-26T07:18:20.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec0ae680 0x7fb2ec099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec0cd5b0 0x7fb2ec09b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 0x7fb2ec14f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec15b9a0 con 0x7fb2ec0ae680 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec09c450 con 0x7fb2ec0cd5b0 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec09c730 con 0x7fb2ec159f00 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec0ae680 0x7fb2ec099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec0ae680 0x7fb2ec099ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48292/0 (socket says 172.21.15.31:48292) 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2ebfff640 1 -- 172.21.15.31:0/1392589338 learned_addr learned my addr 172.21.15.31:0/1392589338 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2eb7fe640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec0cd5b0 0x7fb2ec09b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f0cc7640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 0x7fb2ec14f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2ebfff640 1 -- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 msgr2=0x7fb2ec14f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2ebfff640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 0x7fb2ec14f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2ebfff640 1 -- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec0cd5b0 msgr2=0x7fb2ec09b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2ebfff640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec0cd5b0 0x7fb2ec09b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2ebfff640 1 -- 172.21.15.31:0/1392589338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc009420 con 0x7fb2ec0ae680 2023-11-26T07:18:20.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2ebfff640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec0ae680 0x7fb2ec099ec0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0097f0 tx=0x7fb2dc019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2e8ff9640 1 -- 172.21.15.31:0/1392589338 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc006e20 con 0x7fb2ec0ae680 2023-11-26T07:18:20.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2e8ff9640 1 -- 172.21.15.31:0/1392589338 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc004020 con 0x7fb2ec0ae680 2023-11-26T07:18:20.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2e8ff9640 1 -- 172.21.15.31:0/1392589338 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc004390 con 0x7fb2ec0ae680 2023-11-26T07:18:20.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 -- 172.21.15.31:0/1392589338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec15b630 con 0x7fb2ec0ae680 2023-11-26T07:18:20.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 -- 172.21.15.31:0/1392589338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec09a5d0 con 0x7fb2ec0ae680 2023-11-26T07:18:20.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.320+0000 7fb2f272a640 1 -- 172.21.15.31:0/1392589338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec0671f0 con 0x7fb2ec0ae680 2023-11-26T07:18:20.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.324+0000 7fb2e8ff9640 1 -- 172.21.15.31:0/1392589338 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb2dc0b2040 con 0x7fb2ec0ae680 2023-11-26T07:18:20.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.324+0000 7fb2e8ff9640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2d8066470 0x7fb2d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.324+0000 7fb2e8ff9640 1 -- 172.21.15.31:0/1392589338 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb2dc11d640 con 0x7fb2ec0ae680 2023-11-26T07:18:20.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.324+0000 7fb2eb7fe640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2d8066470 0x7fb2d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.324+0000 7fb2eb7fe640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2d8066470 0x7fb2d8068930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec150870 tx=0x7fb2e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.328+0000 7fb2e8ff9640 1 -- 172.21.15.31:0/1392589338 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb2dc0ead70 con 0x7fb2ec0ae680 2023-11-26T07:18:20.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.540+0000 7fb2f272a640 1 -- 172.21.15.31:0/1392589338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fb2ec06b460 con 0x7fb2ec0ae680 2023-11-26T07:18:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.548+0000 7fb2e8ff9640 1 -- 172.21.15.31:0/1392589338 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb2dc0b00a0 con 0x7fb2ec0ae680 2023-11-26T07:18:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 -- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2d8066470 msgr2=0x7fb2d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2d8066470 0x7fb2d8068930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec150870 tx=0x7fb2e0009040 comp rx=0 tx=0).stop 2023-11-26T07:18:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 -- 172.21.15.31:0/1392589338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec0ae680 msgr2=0x7fb2ec099ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec0ae680 0x7fb2ec099ec0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0097f0 tx=0x7fb2dc019150 comp rx=0 tx=0).stop 2023-11-26T07:18:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 -- 172.21.15.31:0/1392589338 shutdown_connections 2023-11-26T07:18:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2d8066470 0x7fb2d8068930 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2ec159f00 0x7fb2ec14f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec0cd5b0 0x7fb2ec09b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 --2- 172.21.15.31:0/1392589338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2ec0ae680 0x7fb2ec099ec0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 -- 172.21.15.31:0/1392589338 >> 172.21.15.31:0/1392589338 conn(0x7fb2ec0b0cb0 msgr2=0x7fb2ec0abf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 -- 172.21.15.31:0/1392589338 shutdown_connections 2023-11-26T07:18:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.552+0000 7fb2f272a640 1 -- 172.21.15.31:0/1392589338 wait complete. 2023-11-26T07:18:20.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-26T07:18:20.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.712+0000 7f1b2c6ea640 1 Processor -- start 2023-11-26T07:18:20.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.712+0000 7f1b2c6ea640 1 -- start start 2023-11-26T07:18:20.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2c6ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b2415a710 0x7f1b240abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2c6ea640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1b240cbd50 0x7f1b240cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2c6ea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b240b2710 con 0x7f1b240cdfb0 2023-11-26T07:18:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2c6ea640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b240cca80 con 0x7f1b240cbd50 2023-11-26T07:18:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2c6ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240b29f0 con 0x7f1b2415a710 2023-11-26T07:18:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2ac87640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1b240cbd50 0x7f1b240cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2ac87640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1b240cbd50 0x7f1b240cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48300/0 (socket says 172.21.15.31:48300) 2023-11-26T07:18:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2ac87640 1 -- 172.21.15.31:0/1775966116 learned_addr learned my addr 172.21.15.31:0/1775966116 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b29c85640 1 --2- 172.21.15.31:0/1775966116 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b2415a710 0x7f1b240abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2ac87640 1 -- 172.21.15.31:0/1775966116 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b2415a710 msgr2=0x7f1b240abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2ac87640 1 --2- 172.21.15.31:0/1775966116 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b2415a710 0x7f1b240abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2ac87640 1 -- 172.21.15.31:0/1775966116 >> v1:172.21.15.97:6789/0 conn(0x7f1b240cdfb0 legacy=0x7f1b240ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.716+0000 7f1b2ac87640 1 -- 172.21.15.31:0/1775966116 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b240b2cd0 con 0x7f1b240cbd50 2023-11-26T07:18:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2ac87640 1 --2- 172.21.15.31:0/1775966116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b240cbd50 0x7f1b240cc1e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009e50 tx=0x7f1b1c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30370683983de144 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b29484640 1 -- 172.21.15.31:0/1775966116 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0a58d0 con 0x7f1b240cbd50 2023-11-26T07:18:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b29484640 1 -- 172.21.15.31:0/1775966116 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1c0a5a30 con 0x7f1b240cbd50 2023-11-26T07:18:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b29484640 1 -- 172.21.15.31:0/1775966116 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0a5dc0 con 0x7f1b240cbd50 2023-11-26T07:18:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/1775966116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b240cbd50 msgr2=0x7f1b240cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2c6ea640 1 --2- 172.21.15.31:0/1775966116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b240cbd50 0x7f1b240cc1e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009e50 tx=0x7f1b1c01c9e0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/1775966116 shutdown_connections 2023-11-26T07:18:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2c6ea640 1 --2- 172.21.15.31:0/1775966116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b240cbd50 0x7f1b240cc1e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2c6ea640 1 --2- 172.21.15.31:0/1775966116 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b2415a710 0x7f1b240abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/1775966116 >> 172.21.15.31:0/1775966116 conn(0x7f1b240b0cb0 msgr2=0x7f1b240b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/1775966116 shutdown_connections 2023-11-26T07:18:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/1775966116 wait complete. 2023-11-26T07:18:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2c6ea640 1 Processor -- start 2023-11-26T07:18:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.720+0000 7f1b2c6ea640 1 -- start start 2023-11-26T07:18:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2c6ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b240cbd50 0x7f1b24163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2c6ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b240cdfb0 0x7f1b241556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2c6ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b2415a710 0x7f1b24157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2c6ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2415bcc0 con 0x7f1b2415a710 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2c6ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2415cb20 con 0x7f1b240cbd50 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2c6ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2415bf10 con 0x7f1b240cdfb0 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2ac87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b2415a710 0x7f1b24157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2ac87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b2415a710 0x7f1b24157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48316/0 (socket says 172.21.15.31:48316) 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2ac87640 1 -- 172.21.15.31:0/3670408714 learned_addr learned my addr 172.21.15.31:0/3670408714 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2a486640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b240cbd50 0x7f1b24163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b29c85640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b240cdfb0 0x7f1b241556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2ac87640 1 -- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b240cdfb0 msgr2=0x7f1b241556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2ac87640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b240cdfb0 0x7f1b241556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2ac87640 1 -- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b240cbd50 msgr2=0x7f1b24163fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2ac87640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b240cbd50 0x7f1b24163fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2ac87640 1 -- 172.21.15.31:0/3670408714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c009b30 con 0x7f1b2415a710 2023-11-26T07:18:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2ac87640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b2415a710 0x7f1b24157280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c01a610 tx=0x7f1b1c019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b1affd640 1 -- 172.21.15.31:0/3670408714 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0b8670 con 0x7f1b2415a710 2023-11-26T07:18:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b1affd640 1 -- 172.21.15.31:0/3670408714 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1c0b87d0 con 0x7f1b2415a710 2023-11-26T07:18:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b1affd640 1 -- 172.21.15.31:0/3670408714 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0b8b80 con 0x7f1b2415a710 2023-11-26T07:18:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/3670408714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2415c1f0 con 0x7f1b2415a710 2023-11-26T07:18:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/3670408714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2415c7b0 con 0x7f1b2415a710 2023-11-26T07:18:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/3670408714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b240671f0 con 0x7f1b2415a710 2023-11-26T07:18:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.724+0000 7f1b1affd640 1 -- 172.21.15.31:0/3670408714 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f1b1c0b8ce0 con 0x7f1b2415a710 2023-11-26T07:18:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.728+0000 7f1b1affd640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b080664a0 0x7f1b08068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.728+0000 7f1b1affd640 1 -- 172.21.15.31:0/3670408714 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1b1c11efe0 con 0x7f1b2415a710 2023-11-26T07:18:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.728+0000 7f1b2a486640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b080664a0 0x7f1b08068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.728+0000 7f1b2a486640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b080664a0 0x7f1b08068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b20007fd0 tx=0x7f1b20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:20.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.732+0000 7f1b1affd640 1 -- 172.21.15.31:0/3670408714 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1b1c0eca00 con 0x7f1b2415a710 2023-11-26T07:18:20.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.948+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/3670408714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f1b2406b460 con 0x7f1b2415a710 2023-11-26T07:18:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b1affd640 1 -- 172.21.15.31:0/3670408714 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1b1c0f0020 con 0x7f1b2415a710 2023-11-26T07:18:20.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b080664a0 msgr2=0x7f1b08068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b080664a0 0x7f1b08068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b20007fd0 tx=0x7f1b20009040 comp rx=0 tx=0).stop 2023-11-26T07:18:20.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/3670408714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b2415a710 msgr2=0x7f1b24157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:20.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b2415a710 0x7f1b24157280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c01a610 tx=0x7f1b1c019780 comp rx=0 tx=0).stop 2023-11-26T07:18:20.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/3670408714 shutdown_connections 2023-11-26T07:18:20.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b080664a0 0x7f1b08068960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b2415a710 0x7f1b24157280 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b240cdfb0 0x7f1b241556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 --2- 172.21.15.31:0/3670408714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b240cbd50 0x7f1b24163fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/3670408714 >> 172.21.15.31:0/3670408714 conn(0x7f1b240b0cb0 msgr2=0x7f1b240b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/3670408714 shutdown_connections 2023-11-26T07:18:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:20.952+0000 7f1b2c6ea640 1 -- 172.21.15.31:0/3670408714 wait complete. 2023-11-26T07:18:20.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-26T07:18:21.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d7e32640 1 Processor -- start 2023-11-26T07:18:21.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d7e32640 1 -- start start 2023-11-26T07:18:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d7e32640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45d00ced30 0x7f45d00cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d7e32640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45d00cf1e0 0x7f45d00d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d7e32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d00cdfb0 con 0x7f45d015d8f0 2023-11-26T07:18:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d7e32640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00d4420 con 0x7f45d00ced30 2023-11-26T07:18:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d7e32640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00d4140 con 0x7f45d00cf1e0 2023-11-26T07:18:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d53cd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45d00ced30 0x7f45d00cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d53cd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45d00ced30 0x7f45d00cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48332/0 (socket says 172.21.15.31:48332) 2023-11-26T07:18:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d53cd640 1 -- 172.21.15.31:0/1825346458 learned_addr learned my addr 172.21.15.31:0/1825346458 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d63cf640 1 --2- 172.21.15.31:0/1825346458 >> v2:172.21.15.97:3300/0 conn(0x7f45d00cf1e0 0x7f45d00d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d53cd640 1 -- 172.21.15.31:0/1825346458 >> v2:172.21.15.97:3300/0 conn(0x7f45d00cf1e0 msgr2=0x7f45d00d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d53cd640 1 --2- 172.21.15.31:0/1825346458 >> v2:172.21.15.97:3300/0 conn(0x7f45d00cf1e0 0x7f45d00d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d53cd640 1 -- 172.21.15.31:0/1825346458 >> v1:172.21.15.97:6789/0 conn(0x7f45d015d8f0 legacy=0x7f45d015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d53cd640 1 -- 172.21.15.31:0/1825346458 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00ce250 con 0x7f45d00ced30 2023-11-26T07:18:21.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.116+0000 7f45d63cf640 1 --2- 172.21.15.31:0/1825346458 >> v2:172.21.15.97:3300/0 conn(0x7f45d00cf1e0 0x7f45d00d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d53cd640 1 --2- 172.21.15.31:0/1825346458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d00ced30 0x7f45d00cfe00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f45d00b21f0 tx=0x7f45c001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1c89af70b80c79 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d4bcc640 1 -- 172.21.15.31:0/1825346458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00aa5f0 con 0x7f45d00ced30 2023-11-26T07:18:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d4bcc640 1 -- 172.21.15.31:0/1825346458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c00aa750 con 0x7f45d00ced30 2023-11-26T07:18:21.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d4bcc640 1 -- 172.21.15.31:0/1825346458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c00aaae0 con 0x7f45d00ced30 2023-11-26T07:18:21.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d7e32640 1 -- 172.21.15.31:0/1825346458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d00ced30 msgr2=0x7f45d00cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d7e32640 1 --2- 172.21.15.31:0/1825346458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d00ced30 0x7f45d00cfe00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f45d00b21f0 tx=0x7f45c001c490 comp rx=0 tx=0).stop 2023-11-26T07:18:21.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d7e32640 1 -- 172.21.15.31:0/1825346458 shutdown_connections 2023-11-26T07:18:21.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d7e32640 1 --2- 172.21.15.31:0/1825346458 >> v2:172.21.15.97:3300/0 conn(0x7f45d00cf1e0 0x7f45d00d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d7e32640 1 --2- 172.21.15.31:0/1825346458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d00ced30 0x7f45d00cfe00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d7e32640 1 -- 172.21.15.31:0/1825346458 >> 172.21.15.31:0/1825346458 conn(0x7f45d009c750 msgr2=0x7f45d00d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d7e32640 1 -- 172.21.15.31:0/1825346458 shutdown_connections 2023-11-26T07:18:21.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.120+0000 7f45d7e32640 1 -- 172.21.15.31:0/1825346458 wait complete. 2023-11-26T07:18:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d7e32640 1 Processor -- start 2023-11-26T07:18:21.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d7e32640 1 -- start start 2023-11-26T07:18:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d7e32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45d00ced30 0x7f45d0150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d7e32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d00cf1e0 0x7f45d0150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d7e32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d015d8f0 0x7f45d01511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d7e32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d015cb60 con 0x7f45d015d8f0 2023-11-26T07:18:21.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d7e32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d015bfe0 con 0x7f45d00cf1e0 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d7e32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d015c2c0 con 0x7f45d00ced30 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d63cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d015d8f0 0x7f45d01511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d63cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d015d8f0 0x7f45d01511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48336/0 (socket says 172.21.15.31:48336) 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d63cf640 1 -- 172.21.15.31:0/2010427329 learned_addr learned my addr 172.21.15.31:0/2010427329 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d53cd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d00cf1e0 0x7f45d0150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d5bce640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45d00ced30 0x7f45d0150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d63cf640 1 -- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45d00ced30 msgr2=0x7f45d0150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d63cf640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45d00ced30 0x7f45d0150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d63cf640 1 -- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d00cf1e0 msgr2=0x7f45d0150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d63cf640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d00cf1e0 0x7f45d0150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d63cf640 1 -- 172.21.15.31:0/2010427329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c00094d0 con 0x7f45d015d8f0 2023-11-26T07:18:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d63cf640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d015d8f0 0x7f45d01511a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f45c800cea0 tx=0x7f45c800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45c67fc640 1 -- 172.21.15.31:0/2010427329 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8003ba0 con 0x7f45d015d8f0 2023-11-26T07:18:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45c67fc640 1 -- 172.21.15.31:0/2010427329 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c8003d20 con 0x7f45d015d8f0 2023-11-26T07:18:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d7e32640 1 -- 172.21.15.31:0/2010427329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d015c5a0 con 0x7f45d015d8f0 2023-11-26T07:18:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45c67fc640 1 -- 172.21.15.31:0/2010427329 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c800d880 con 0x7f45d015d8f0 2023-11-26T07:18:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.124+0000 7f45d7e32640 1 -- 172.21.15.31:0/2010427329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d0152440 con 0x7f45d015d8f0 2023-11-26T07:18:21.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.128+0000 7f45d7e32640 1 -- 172.21.15.31:0/2010427329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d004e850 con 0x7f45d015d8f0 2023-11-26T07:18:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.128+0000 7f45c67fc640 1 -- 172.21.15.31:0/2010427329 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f45c800d9e0 con 0x7f45d015d8f0 2023-11-26T07:18:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.128+0000 7f45c67fc640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45b4066470 0x7f45b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.128+0000 7f45c67fc640 1 -- 172.21.15.31:0/2010427329 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f45c8084450 con 0x7f45d015d8f0 2023-11-26T07:18:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.128+0000 7f45d5bce640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45b4066470 0x7f45b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.132+0000 7f45d5bce640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45b4066470 0x7f45b4068930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f45cc007fd0 tx=0x7f45cc007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.132+0000 7f45c67fc640 1 -- 172.21.15.31:0/2010427329 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f45c8051c30 con 0x7f45d015d8f0 2023-11-26T07:18:21.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.344+0000 7f45d7e32640 1 -- 172.21.15.31:0/2010427329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f45d00cf9d0 con 0x7f45d015d8f0 2023-11-26T07:18:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45c67fc640 1 -- 172.21.15.31:0/2010427329 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f45c8055250 con 0x7f45d015d8f0 2023-11-26T07:18:21.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 -- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45b4066470 msgr2=0x7f45b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45b4066470 0x7f45b4068930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f45cc007fd0 tx=0x7f45cc007e40 comp rx=0 tx=0).stop 2023-11-26T07:18:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 -- 172.21.15.31:0/2010427329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d015d8f0 msgr2=0x7f45d01511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d015d8f0 0x7f45d01511a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f45c800cea0 tx=0x7f45c800cf90 comp rx=0 tx=0).stop 2023-11-26T07:18:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 -- 172.21.15.31:0/2010427329 shutdown_connections 2023-11-26T07:18:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45b4066470 0x7f45b4068930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45d015d8f0 0x7f45d01511a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45d00cf1e0 0x7f45d0150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 --2- 172.21.15.31:0/2010427329 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45d00ced30 0x7f45d0150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 -- 172.21.15.31:0/2010427329 >> 172.21.15.31:0/2010427329 conn(0x7f45d009c750 msgr2=0x7f45d006c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 -- 172.21.15.31:0/2010427329 shutdown_connections 2023-11-26T07:18:21.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.348+0000 7f45d7e32640 1 -- 172.21.15.31:0/2010427329 wait complete. 2023-11-26T07:18:21.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:21.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.512+0000 7f2c4cd57640 1 Processor -- start 2023-11-26T07:18:21.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.512+0000 7f2c4cd57640 1 -- start start 2023-11-26T07:18:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.512+0000 7f2c4cd57640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2c480ce5c0 0x7f2c480cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.512+0000 7f2c4cd57640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c4806a5b0 0x7f2c4806aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.512+0000 7f2c4cd57640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c480b0a70 con 0x7f2c4815d8b0 2023-11-26T07:18:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.512+0000 7f2c4cd57640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c480cddb0 con 0x7f2c4806a5b0 2023-11-26T07:18:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.512+0000 7f2c4cd57640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c480b4100 con 0x7f2c480ce5c0 2023-11-26T07:18:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.512+0000 7f2c4659c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2c4815d8b0 0x7f2c480b4be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47568/0 (socket says 172.21.15.31:47568) 2023-11-26T07:18:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.512+0000 7f2c4659c640 1 -- 172.21.15.31:0/3674712210 learned_addr learned my addr 172.21.15.31:0/3674712210 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.512+0000 7f2c46d9d640 1 --2- 172.21.15.31:0/3674712210 >> v2:172.21.15.31:3300/0 conn(0x7f2c4806a5b0 0x7f2c4806aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.516+0000 7f2c3ffff640 1 --2- 172.21.15.31:0/3674712210 >> v2:172.21.15.97:3300/0 conn(0x7f2c480ce5c0 0x7f2c480cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.516+0000 7f2c46d9d640 1 -- 172.21.15.31:0/3674712210 >> v2:172.21.15.97:3300/0 conn(0x7f2c480ce5c0 msgr2=0x7f2c480cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.516+0000 7f2c46d9d640 1 --2- 172.21.15.31:0/3674712210 >> v2:172.21.15.97:3300/0 conn(0x7f2c480ce5c0 0x7f2c480cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.516+0000 7f2c46d9d640 1 -- 172.21.15.31:0/3674712210 >> v1:172.21.15.31:6789/0 conn(0x7f2c4815d8b0 legacy=0x7f2c480b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.516+0000 7f2c45d9b640 1 -- 172.21.15.31:0/3674712210 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129515694 0 0) 0x7f2c480b0a70 con 0x7f2c4815d8b0 2023-11-26T07:18:21.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.516+0000 7f2c46d9d640 1 -- 172.21.15.31:0/3674712210 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c480b43e0 con 0x7f2c4806a5b0 2023-11-26T07:18:21.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.516+0000 7f2c46d9d640 1 --2- 172.21.15.31:0/3674712210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4806a5b0 0x7f2c4806aa20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2c480b1780 tx=0x7f2c4001f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312429a935c63b8e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.516+0000 7f2c45d9b640 1 -- 172.21.15.31:0/3674712210 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c400b0440 con 0x7f2c4806a5b0 2023-11-26T07:18:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.516+0000 7f2c45d9b640 1 -- 172.21.15.31:0/3674712210 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c400b05a0 con 0x7f2c4806a5b0 2023-11-26T07:18:21.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.516+0000 7f2c45d9b640 1 -- 172.21.15.31:0/3674712210 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c400b0930 con 0x7f2c4806a5b0 2023-11-26T07:18:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 -- 172.21.15.31:0/3674712210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4806a5b0 msgr2=0x7f2c4806aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 --2- 172.21.15.31:0/3674712210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4806a5b0 0x7f2c4806aa20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2c480b1780 tx=0x7f2c4001f0f0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 -- 172.21.15.31:0/3674712210 shutdown_connections 2023-11-26T07:18:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 --2- 172.21.15.31:0/3674712210 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4806a5b0 0x7f2c4806aa20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 --2- 172.21.15.31:0/3674712210 >> v2:172.21.15.97:3300/0 conn(0x7f2c480ce5c0 0x7f2c480cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 -- 172.21.15.31:0/3674712210 >> 172.21.15.31:0/3674712210 conn(0x7f2c480b5570 msgr2=0x7f2c480b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 -- 172.21.15.31:0/3674712210 shutdown_connections 2023-11-26T07:18:21.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 -- 172.21.15.31:0/3674712210 wait complete. 2023-11-26T07:18:21.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 Processor -- start 2023-11-26T07:18:21.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 -- start start 2023-11-26T07:18:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c4806a5b0 0x7f2c480b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2c480ce5c0 0x7f2c480b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4815d8b0 0x7f2c481f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481f4d40 con 0x7f2c4815d8b0 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481f4070 con 0x7f2c4806a5b0 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4cd57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481f43a0 con 0x7f2c480ce5c0 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c46d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4815d8b0 0x7f2c481f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c46d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4815d8b0 0x7f2c481f3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48348/0 (socket says 172.21.15.31:48348) 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c46d9d640 1 -- 172.21.15.31:0/690174291 learned_addr learned my addr 172.21.15.31:0/690174291 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4659c640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c4806a5b0 0x7f2c480b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c3ffff640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2c480ce5c0 0x7f2c480b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c46d9d640 1 -- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2c480ce5c0 msgr2=0x7f2c480b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c46d9d640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2c480ce5c0 0x7f2c480b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c46d9d640 1 -- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c4806a5b0 msgr2=0x7f2c480b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c46d9d640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c4806a5b0 0x7f2c480b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c46d9d640 1 -- 172.21.15.31:0/690174291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c40009a70 con 0x7f2c4815d8b0 2023-11-26T07:18:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.520+0000 7f2c4659c640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c4806a5b0 0x7f2c480b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:21.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c46d9d640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4815d8b0 0x7f2c481f3310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2c4001bd00 tx=0x7f2c4001be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c3effd640 1 -- 172.21.15.31:0/690174291 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c400b0440 con 0x7f2c4815d8b0 2023-11-26T07:18:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c3effd640 1 -- 172.21.15.31:0/690174291 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c400bc860 con 0x7f2c4815d8b0 2023-11-26T07:18:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c4cd57640 1 -- 172.21.15.31:0/690174291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c481f4680 con 0x7f2c4815d8b0 2023-11-26T07:18:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c3effd640 1 -- 172.21.15.31:0/690174291 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c400bcc10 con 0x7f2c4815d8b0 2023-11-26T07:18:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c4cd57640 1 -- 172.21.15.31:0/690174291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4814de00 con 0x7f2c4815d8b0 2023-11-26T07:18:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c4cd57640 1 -- 172.21.15.31:0/690174291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c481f4e80 con 0x7f2c4815d8b0 2023-11-26T07:18:21.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c3effd640 1 -- 172.21.15.31:0/690174291 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2c400ba400 con 0x7f2c4815d8b0 2023-11-26T07:18:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c3effd640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c280664a0 0x7f2c28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c3effd640 1 -- 172.21.15.31:0/690174291 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2c40122830 con 0x7f2c4815d8b0 2023-11-26T07:18:21.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.524+0000 7f2c4659c640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c280664a0 0x7f2c28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.528+0000 7f2c4659c640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c280664a0 0x7f2c28068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2c38000f70 tx=0x7f2c38007cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.528+0000 7f2c3effd640 1 -- 172.21.15.31:0/690174291 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c481f4e80 con 0x7f2c4815d8b0 2023-11-26T07:18:21.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.688+0000 7f2c4cd57640 1 -- 172.21.15.31:0/690174291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c481f4e80 con 0x7f2c4815d8b0 2023-11-26T07:18:21.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.688+0000 7f2c3effd640 1 -- 172.21.15.31:0/690174291 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12773 (secure 0 0 0) 0x7f2c481f4e80 con 0x7f2c4815d8b0 2023-11-26T07:18:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:21.695 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":29,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:18.552263+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 -- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c280664a0 msgr2=0x7f2c28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c280664a0 0x7f2c28068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2c38000f70 tx=0x7f2c38007cd0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 -- 172.21.15.31:0/690174291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4815d8b0 msgr2=0x7f2c481f3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4815d8b0 0x7f2c481f3310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2c4001bd00 tx=0x7f2c4001be90 comp rx=0 tx=0).stop 2023-11-26T07:18:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 -- 172.21.15.31:0/690174291 shutdown_connections 2023-11-26T07:18:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c280664a0 0x7f2c28068960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c4815d8b0 0x7f2c481f3310 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2c480ce5c0 0x7f2c480b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 --2- 172.21.15.31:0/690174291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c4806a5b0 0x7f2c480b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 -- 172.21.15.31:0/690174291 >> 172.21.15.31:0/690174291 conn(0x7f2c480b5570 msgr2=0x7f2c4815ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 -- 172.21.15.31:0/690174291 shutdown_connections 2023-11-26T07:18:21.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.692+0000 7f2c4cd57640 1 -- 172.21.15.31:0/690174291 wait complete. 2023-11-26T07:18:21.716 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:21.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:21.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.856+0000 7f82a66c8640 1 Processor -- start 2023-11-26T07:18:21.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.856+0000 7f82a66c8640 1 -- start start 2023-11-26T07:18:21.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f82a66c8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f82a006c6b0 0x7f82a00b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f82a66c8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a015ef10 con 0x7f82a0143e50 2023-11-26T07:18:21.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f82a66c8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a015f1f0 con 0x7f82a006c250 2023-11-26T07:18:21.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f82a66c8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00cecf0 con 0x7f82a006c6b0 2023-11-26T07:18:21.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f82a56c6640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f82a0143e50 0x7f82a01444c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47574/0 (socket says 172.21.15.31:47574) 2023-11-26T07:18:21.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f82a56c6640 1 -- 172.21.15.31:0/3390462557 learned_addr learned my addr 172.21.15.31:0/3390462557 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:21.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f82a5ec7640 1 --2- 172.21.15.31:0/3390462557 >> v2:172.21.15.97:3300/0 conn(0x7f82a006c6b0 0x7f82a00b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f8297fff640 1 -- 172.21.15.31:0/3390462557 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647130042 0 0) 0x7f82a015ef10 con 0x7f82a0143e50 2023-11-26T07:18:21.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f8297fff640 1 -- 172.21.15.31:0/3390462557 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f828c003530 con 0x7f82a0143e50 2023-11-26T07:18:21.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f8297fff640 1 -- 172.21.15.31:0/3390462557 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833571063 0 0) 0x7f82a015f1f0 con 0x7f82a006c250 2023-11-26T07:18:21.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f8297fff640 1 -- 172.21.15.31:0/3390462557 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82a015ef10 con 0x7f82a006c250 2023-11-26T07:18:21.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f8297fff640 1 -- 172.21.15.31:0/3390462557 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14164165 0 0) 0x7f828c003530 con 0x7f82a0143e50 2023-11-26T07:18:21.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f82a5ec7640 1 -- 172.21.15.31:0/3390462557 >> v1:172.21.15.97:6789/0 conn(0x7f82a006c250 legacy=0x7f82a00b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f82a5ec7640 1 -- 172.21.15.31:0/3390462557 >> v1:172.21.15.31:6789/0 conn(0x7f82a0143e50 legacy=0x7f82a01444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.860+0000 7f82a5ec7640 1 -- 172.21.15.31:0/3390462557 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a015f4d0 con 0x7f82a006c6b0 2023-11-26T07:18:21.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f82a5ec7640 1 --2- 172.21.15.31:0/3390462557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a006c6b0 0x7f82a00b3a50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8298009d90 tx=0x7f829801c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dff9bc438995732 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f8297fff640 1 -- 172.21.15.31:0/3390462557 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c001680 con 0x7f82a006c6b0 2023-11-26T07:18:21.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f8297fff640 1 -- 172.21.15.31:0/3390462557 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82980a5610 con 0x7f82a006c6b0 2023-11-26T07:18:21.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f8297fff640 1 -- 172.21.15.31:0/3390462557 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82980a59a0 con 0x7f82a006c6b0 2023-11-26T07:18:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f82a66c8640 1 -- 172.21.15.31:0/3390462557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a006c6b0 msgr2=0x7f82a00b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f82a66c8640 1 --2- 172.21.15.31:0/3390462557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a006c6b0 0x7f82a00b3a50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8298009d90 tx=0x7f829801c7b0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f82a66c8640 1 -- 172.21.15.31:0/3390462557 shutdown_connections 2023-11-26T07:18:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f82a66c8640 1 --2- 172.21.15.31:0/3390462557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a006c6b0 0x7f82a00b3a50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f82a66c8640 1 -- 172.21.15.31:0/3390462557 >> 172.21.15.31:0/3390462557 conn(0x7f82a006df30 msgr2=0x7f82a006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f82a66c8640 1 -- 172.21.15.31:0/3390462557 shutdown_connections 2023-11-26T07:18:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f82a66c8640 1 -- 172.21.15.31:0/3390462557 wait complete. 2023-11-26T07:18:21.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f82a66c8640 1 Processor -- start 2023-11-26T07:18:21.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.864+0000 7f82a66c8640 1 -- start start 2023-11-26T07:18:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a66c8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a006c250 0x7f82a013d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a66c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a006c6b0 0x7f82a015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a66c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a0143e50 0x7f82a014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a66c8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a014e6c0 con 0x7f82a0143e50 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a66c8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a015d550 con 0x7f82a006c6b0 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a66c8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a015d830 con 0x7f82a006c250 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a5ec7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a0143e50 0x7f82a014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a5ec7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a0143e50 0x7f82a014b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48358/0 (socket says 172.21.15.31:48358) 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a5ec7640 1 -- 172.21.15.31:0/3613132771 learned_addr learned my addr 172.21.15.31:0/3613132771 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a4ec5640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a006c6b0 0x7f82a015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a56c6640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a006c250 0x7f82a013d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a5ec7640 1 -- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a006c250 msgr2=0x7f82a013d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a5ec7640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a006c250 0x7f82a013d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a5ec7640 1 -- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a006c6b0 msgr2=0x7f82a015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:21.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a5ec7640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a006c6b0 0x7f82a015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a5ec7640 1 -- 172.21.15.31:0/3613132771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8298009a70 con 0x7f82a0143e50 2023-11-26T07:18:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a5ec7640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a0143e50 0x7f82a014b8b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f82980193c0 tx=0x7f8298019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f8295ffb640 1 -- 172.21.15.31:0/3613132771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298007000 con 0x7f82a0143e50 2023-11-26T07:18:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f8295ffb640 1 -- 172.21.15.31:0/3613132771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82980ad650 con 0x7f82a0143e50 2023-11-26T07:18:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a66c8640 1 -- 172.21.15.31:0/3613132771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a015db10 con 0x7f82a0143e50 2023-11-26T07:18:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f8295ffb640 1 -- 172.21.15.31:0/3613132771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82980ada00 con 0x7f82a0143e50 2023-11-26T07:18:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a66c8640 1 -- 172.21.15.31:0/3613132771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a013d670 con 0x7f82a0143e50 2023-11-26T07:18:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f82a66c8640 1 -- 172.21.15.31:0/3613132771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a0074990 con 0x7f82a0143e50 2023-11-26T07:18:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.868+0000 7f8295ffb640 1 -- 172.21.15.31:0/3613132771 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f82980adb60 con 0x7f82a0143e50 2023-11-26T07:18:21.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.872+0000 7f8295ffb640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8284066470 0x7f8284068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:21.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.872+0000 7f8295ffb640 1 -- 172.21.15.31:0/3613132771 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f829811eda0 con 0x7f82a0143e50 2023-11-26T07:18:21.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.872+0000 7f82a56c6640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8284066470 0x7f8284068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:21.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.872+0000 7f82a56c6640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8284066470 0x7f8284068930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f829c0011f0 tx=0x7f829c000fa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:21.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:21.876+0000 7f8295ffb640 1 -- 172.21.15.31:0/3613132771 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82980ec6b0 con 0x7f82a0143e50 2023-11-26T07:18:22.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.032+0000 7f82a66c8640 1 -- 172.21.15.31:0/3613132771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82a015ec50 con 0x7f82a0143e50 2023-11-26T07:18:22.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.032+0000 7f8295ffb640 1 -- 172.21.15.31:0/3613132771 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82980c60a0 con 0x7f82a0143e50 2023-11-26T07:18:22.039 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 -- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8284066470 msgr2=0x7f8284068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8284066470 0x7f8284068930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f829c0011f0 tx=0x7f829c000fa0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 -- 172.21.15.31:0/3613132771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a0143e50 msgr2=0x7f82a014b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a0143e50 0x7f82a014b8b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f82980193c0 tx=0x7f8298019550 comp rx=0 tx=0).stop 2023-11-26T07:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 -- 172.21.15.31:0/3613132771 shutdown_connections 2023-11-26T07:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8284066470 0x7f8284068930 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a0143e50 0x7f82a014b8b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a006c6b0 0x7f82a015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 --2- 172.21.15.31:0/3613132771 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a006c250 0x7f82a013d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 -- 172.21.15.31:0/3613132771 >> 172.21.15.31:0/3613132771 conn(0x7f82a006df30 msgr2=0x7f82a006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 -- 172.21.15.31:0/3613132771 shutdown_connections 2023-11-26T07:18:22.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.036+0000 7f82a66c8640 1 -- 172.21.15.31:0/3613132771 wait complete. 2023-11-26T07:18:22.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:18:22.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.196+0000 7f4e03df8640 1 Processor -- start 2023-11-26T07:18:22.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.196+0000 7f4e03df8640 1 -- start start 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.196+0000 7f4e03df8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4dfc071cc0 0x7f4dfc0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.196+0000 7f4e03df8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4dfc15a9c0 0x7f4dfc0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.196+0000 7f4e03df8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dfc157940 con 0x7f4dfc157560 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.196+0000 7f4e03df8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dfc0b4ca0 con 0x7f4dfc071cc0 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.196+0000 7f4e03df8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc0b4e90 con 0x7f4dfc15a9c0 2023-11-26T07:18:22.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e01b94640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4dfc157560 0x7f4dfc0715b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47590/0 (socket says 172.21.15.31:47590) 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e01b94640 1 -- 172.21.15.31:0/1601636311 learned_addr learned my addr 172.21.15.31:0/1601636311 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e01393640 1 --2- 172.21.15.31:0/1601636311 >> v2:172.21.15.97:3300/0 conn(0x7f4dfc071cc0 0x7f4dfc0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e02395640 1 --2- 172.21.15.31:0/1601636311 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4dfc15a9c0 0x7f4dfc0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e00b92640 1 -- 172.21.15.31:0/1601636311 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838596329 0 0) 0x7f4dfc157940 con 0x7f4dfc157560 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e00b92640 1 -- 172.21.15.31:0/1601636311 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4de8003530 con 0x7f4dfc157560 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e02395640 1 -- 172.21.15.31:0/1601636311 >> v2:172.21.15.97:3300/0 conn(0x7f4dfc071cc0 msgr2=0x7f4dfc0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e02395640 1 --2- 172.21.15.31:0/1601636311 >> v2:172.21.15.97:3300/0 conn(0x7f4dfc071cc0 0x7f4dfc0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e02395640 1 -- 172.21.15.31:0/1601636311 >> v1:172.21.15.31:6789/0 conn(0x7f4dfc157560 legacy=0x7f4dfc0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e02395640 1 -- 172.21.15.31:0/1601636311 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc15e390 con 0x7f4dfc15a9c0 2023-11-26T07:18:22.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e01393640 1 --2- 172.21.15.31:0/1601636311 >> v2:172.21.15.97:3300/0 conn(0x7f4dfc071cc0 0x7f4dfc0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.200+0000 7f4e02395640 1 --2- 172.21.15.31:0/1601636311 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4dfc15a9c0 0x7f4dfc0b29a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009d90 tx=0x7f4df401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33b7191aa4c68981 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e00b92640 1 -- 172.21.15.31:0/1601636311 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df40a58d0 con 0x7f4dfc15a9c0 2023-11-26T07:18:22.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e00b92640 1 -- 172.21.15.31:0/1601636311 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df40a5a30 con 0x7f4dfc15a9c0 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e00b92640 1 -- 172.21.15.31:0/1601636311 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df40a5dc0 con 0x7f4dfc15a9c0 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 -- 172.21.15.31:0/1601636311 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4dfc15a9c0 msgr2=0x7f4dfc0b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 --2- 172.21.15.31:0/1601636311 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4dfc15a9c0 0x7f4dfc0b29a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009d90 tx=0x7f4df401c890 comp rx=0 tx=0).stop 2023-11-26T07:18:22.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 -- 172.21.15.31:0/1601636311 shutdown_connections 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 --2- 172.21.15.31:0/1601636311 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4dfc15a9c0 0x7f4dfc0b29a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 --2- 172.21.15.31:0/1601636311 >> v2:172.21.15.97:3300/0 conn(0x7f4dfc071cc0 0x7f4dfc0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 -- 172.21.15.31:0/1601636311 >> 172.21.15.31:0/1601636311 conn(0x7f4dfc0b1e40 msgr2=0x7f4dfc070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 -- 172.21.15.31:0/1601636311 shutdown_connections 2023-11-26T07:18:22.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 -- 172.21.15.31:0/1601636311 wait complete. 2023-11-26T07:18:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 Processor -- start 2023-11-26T07:18:22.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 -- start start 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc071cc0 0x7f4dfc1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dfc157560 0x7f4dfc155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4dfc15a9c0 0x7f4dfc14da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc148a80 con 0x7f4dfc071cc0 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc149870 con 0x7f4dfc157560 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e03df8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dfc148c60 con 0x7f4dfc15a9c0 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e01b94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc071cc0 0x7f4dfc1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e01b94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc071cc0 0x7f4dfc1553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48372/0 (socket says 172.21.15.31:48372) 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.204+0000 7f4e01b94640 1 -- 172.21.15.31:0/598691626 learned_addr learned my addr 172.21.15.31:0/598691626 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4e01b94640 1 -- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4dfc15a9c0 msgr2=0x7f4dfc14da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4e01b94640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4dfc15a9c0 0x7f4dfc14da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4e01b94640 1 -- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dfc157560 msgr2=0x7f4dfc155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4e01b94640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dfc157560 0x7f4dfc155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4e01b94640 1 -- 172.21.15.31:0/598691626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec009420 con 0x7f4dfc071cc0 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4e01b94640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc071cc0 0x7f4dfc1553c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4df800c4d0 tx=0x7f4df800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4df27fc640 1 -- 172.21.15.31:0/598691626 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8005cf0 con 0x7f4dfc071cc0 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4df27fc640 1 -- 172.21.15.31:0/598691626 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df8005e50 con 0x7f4dfc071cc0 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4df27fc640 1 -- 172.21.15.31:0/598691626 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8007270 con 0x7f4dfc071cc0 2023-11-26T07:18:22.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4e03df8640 1 -- 172.21.15.31:0/598691626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df4009a70 con 0x7f4dfc071cc0 2023-11-26T07:18:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4e03df8640 1 -- 172.21.15.31:0/598691626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dfc149220 con 0x7f4dfc071cc0 2023-11-26T07:18:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4e03df8640 1 -- 172.21.15.31:0/598691626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dfc04e850 con 0x7f4dfc071cc0 2023-11-26T07:18:22.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4df27fc640 1 -- 172.21.15.31:0/598691626 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4df80073d0 con 0x7f4dfc071cc0 2023-11-26T07:18:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4df27fc640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4de0066470 0x7f4de0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.208+0000 7f4df27fc640 1 -- 172.21.15.31:0/598691626 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4df8083120 con 0x7f4dfc071cc0 2023-11-26T07:18:22.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.212+0000 7f4e01393640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4de0066470 0x7f4de0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.212+0000 7f4e01393640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4de0066470 0x7f4de0068930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4dec006e90 tx=0x7f4dec006de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.216+0000 7f4df27fc640 1 -- 172.21.15.31:0/598691626 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4df8050900 con 0x7f4dfc071cc0 2023-11-26T07:18:22.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.372+0000 7f4e03df8640 1 -- 172.21.15.31:0/598691626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4dfc154920 con 0x7f4dfc071cc0 2023-11-26T07:18:22.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.372+0000 7f4df27fc640 1 -- 172.21.15.31:0/598691626 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4df8053f20 con 0x7f4dfc071cc0 2023-11-26T07:18:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:22.379 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:18:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-11-26T07:18:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-11-26T07:18:22.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:22.379 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:18:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 -- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4de0066470 msgr2=0x7f4de0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4de0066470 0x7f4de0068930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4dec006e90 tx=0x7f4dec006de0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 -- 172.21.15.31:0/598691626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc071cc0 msgr2=0x7f4dfc1553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc071cc0 0x7f4dfc1553c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4df800c4d0 tx=0x7f4df800c990 comp rx=0 tx=0).stop 2023-11-26T07:18:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 -- 172.21.15.31:0/598691626 shutdown_connections 2023-11-26T07:18:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4de0066470 0x7f4de0068930 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4dfc15a9c0 0x7f4dfc14da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dfc157560 0x7f4dfc155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 --2- 172.21.15.31:0/598691626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dfc071cc0 0x7f4dfc1553c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 -- 172.21.15.31:0/598691626 >> 172.21.15.31:0/598691626 conn(0x7f4dfc0b1e40 msgr2=0x7f4dfc0ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 -- 172.21.15.31:0/598691626 shutdown_connections 2023-11-26T07:18:22.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.376+0000 7f4e03df8640 1 -- 172.21.15.31:0/598691626 wait complete. 2023-11-26T07:18:22.399 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:18:22.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:18:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa558faa640 1 Processor -- start 2023-11-26T07:18:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa558faa640 1 -- start start 2023-11-26T07:18:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa558faa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5540cb9b0 0x7fa55406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa558faa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540cbd90 0x7fa55415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa558faa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa554159630 con 0x7fa5540ce5d0 2023-11-26T07:18:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa558faa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55415bba0 con 0x7fa5540cb9b0 2023-11-26T07:18:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa558faa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5540cdd50 con 0x7fa5540cbd90 2023-11-26T07:18:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa55259c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa5540cb9b0 0x7fa55406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa552d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa5540ce5d0 0x7fa55415bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:52902/0 (socket says 172.21.15.31:52902) 2023-11-26T07:18:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa552d9d640 1 -- 172.21.15.31:0/259008422 learned_addr learned my addr 172.21.15.31:0/259008422 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa551d9b640 1 --2- 172.21.15.31:0/259008422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540cbd90 0x7fa55415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa55259c640 1 -- 172.21.15.31:0/259008422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540cbd90 msgr2=0x7fa55415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:22.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa55259c640 1 --2- 172.21.15.31:0/259008422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540cbd90 0x7fa55415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa55259c640 1 -- 172.21.15.31:0/259008422 >> v1:172.21.15.97:6789/0 conn(0x7fa5540ce5d0 legacy=0x7fa55415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.544+0000 7fa55259c640 1 -- 172.21.15.31:0/259008422 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55415f6d0 con 0x7fa5540cb9b0 2023-11-26T07:18:22.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa55259c640 1 --2- 172.21.15.31:0/259008422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cb9b0 0x7fa55406e3f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa5440097c0 tx=0x7fa54401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9cc82da9c4890c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa55159a640 1 -- 172.21.15.31:0/259008422 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440ab650 con 0x7fa5540cb9b0 2023-11-26T07:18:22.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa55159a640 1 -- 172.21.15.31:0/259008422 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5440ab7b0 con 0x7fa5540cb9b0 2023-11-26T07:18:22.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa55159a640 1 -- 172.21.15.31:0/259008422 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440abb40 con 0x7fa5540cb9b0 2023-11-26T07:18:22.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa558faa640 1 -- 172.21.15.31:0/259008422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cb9b0 msgr2=0x7fa55406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa558faa640 1 --2- 172.21.15.31:0/259008422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cb9b0 0x7fa55406e3f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa5440097c0 tx=0x7fa54401c450 comp rx=0 tx=0).stop 2023-11-26T07:18:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa558faa640 1 -- 172.21.15.31:0/259008422 shutdown_connections 2023-11-26T07:18:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa558faa640 1 --2- 172.21.15.31:0/259008422 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540cbd90 0x7fa55415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa558faa640 1 --2- 172.21.15.31:0/259008422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cb9b0 0x7fa55406e3f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa558faa640 1 -- 172.21.15.31:0/259008422 >> 172.21.15.31:0/259008422 conn(0x7fa5540b0cb0 msgr2=0x7fa5540b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa558faa640 1 -- 172.21.15.31:0/259008422 shutdown_connections 2023-11-26T07:18:22.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.548+0000 7fa558faa640 1 -- 172.21.15.31:0/259008422 wait complete. 2023-11-26T07:18:22.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa558faa640 1 Processor -- start 2023-11-26T07:18:22.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa558faa640 1 -- start start 2023-11-26T07:18:22.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa558faa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5540cb9b0 0x7fa5541519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa558faa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cbd90 0x7fa554150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa558faa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540ce5d0 0x7fa55415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa558faa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa554151300 con 0x7fa5540cbd90 2023-11-26T07:18:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa558faa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5540adb10 con 0x7fa5540cb9b0 2023-11-26T07:18:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa558faa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5540addf0 con 0x7fa5540ce5d0 2023-11-26T07:18:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa551d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cbd90 0x7fa554150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa55259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5540cb9b0 0x7fa5541519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa552d9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540ce5d0 0x7fa55415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa55259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5540cb9b0 0x7fa5541519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:55396/0 (socket says 172.21.15.31:55396) 2023-11-26T07:18:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa551d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cbd90 0x7fa554150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48382/0 (socket says 172.21.15.31:48382) 2023-11-26T07:18:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa55259c640 1 -- 172.21.15.31:0/2912179465 learned_addr learned my addr 172.21.15.31:0/2912179465 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:22.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa55259c640 1 -- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540ce5d0 msgr2=0x7fa55415dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa55259c640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540ce5d0 0x7fa55415dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa55259c640 1 -- 172.21.15.31:0/2912179465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cbd90 msgr2=0x7fa554150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa55259c640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cbd90 0x7fa554150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa55259c640 1 -- 172.21.15.31:0/2912179465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa544009420 con 0x7fa5540cb9b0 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa551d9b640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cbd90 0x7fa554150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa552d9d640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540ce5d0 0x7fa55415dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa55259c640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5540cb9b0 0x7fa5541519d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa5440ad040 tx=0x7fa54401ac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa542ffd640 1 -- 172.21.15.31:0/2912179465 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440b5810 con 0x7fa5540cb9b0 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa542ffd640 1 -- 172.21.15.31:0/2912179465 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5440b5970 con 0x7fa5540cb9b0 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa542ffd640 1 -- 172.21.15.31:0/2912179465 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440b5d20 con 0x7fa5540cb9b0 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa558faa640 1 -- 172.21.15.31:0/2912179465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5540b3a30 con 0x7fa5540cb9b0 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.552+0000 7fa558faa640 1 -- 172.21.15.31:0/2912179465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5540b3ff0 con 0x7fa5540cb9b0 2023-11-26T07:18:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.556+0000 7fa558faa640 1 -- 172.21.15.31:0/2912179465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5540671f0 con 0x7fa5540cb9b0 2023-11-26T07:18:22.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.556+0000 7fa542ffd640 1 -- 172.21.15.31:0/2912179465 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa5440bf410 con 0x7fa5540cb9b0 2023-11-26T07:18:22.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.556+0000 7fa542ffd640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa5340664a0 0x7fa534068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:22.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.556+0000 7fa542ffd640 1 -- 172.21.15.31:0/2912179465 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa54411e910 con 0x7fa5540cb9b0 2023-11-26T07:18:22.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.556+0000 7fa551d9b640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa5340664a0 0x7fa534068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:22.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.556+0000 7fa551d9b640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa5340664a0 0x7fa534068960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa554156920 tx=0x7fa548008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:22.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.560+0000 7fa542ffd640 1 -- 172.21.15.31:0/2912179465 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa5440ec320 con 0x7fa5540cb9b0 2023-11-26T07:18:22.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:22.724+0000 7fa558faa640 1 -- 172.21.15.31:0/2912179465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa5540adf30 con 0x7fa5540cb9b0 2023-11-26T07:18:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.556+0000 7fa542ffd640 1 -- 172.21.15.31:0/2912179465 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa5540adf30 con 0x7fa5540cb9b0 2023-11-26T07:18:23.563 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-11-26T07:18:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 -- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa5340664a0 msgr2=0x7fa534068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa5340664a0 0x7fa534068960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa554156920 tx=0x7fa548008040 comp rx=0 tx=0).stop 2023-11-26T07:18:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 -- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5540cb9b0 msgr2=0x7fa5541519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5540cb9b0 0x7fa5541519d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa5440ad040 tx=0x7fa54401ac90 comp rx=0 tx=0).stop 2023-11-26T07:18:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 -- 172.21.15.31:0/2912179465 shutdown_connections 2023-11-26T07:18:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa5340664a0 0x7fa534068960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa5540ce5d0 0x7fa55415dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5540cbd90 0x7fa554150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 --2- 172.21.15.31:0/2912179465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5540cb9b0 0x7fa5541519d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 -- 172.21.15.31:0/2912179465 >> 172.21.15.31:0/2912179465 conn(0x7fa5540b0cb0 msgr2=0x7fa55415a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.560+0000 7fa558faa640 1 -- 172.21.15.31:0/2912179465 shutdown_connections 2023-11-26T07:18:23.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.564+0000 7fa558faa640 1 -- 172.21.15.31:0/2912179465 wait complete. 2023-11-26T07:18:23.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:18:23.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.740+0000 7fd528ddf640 1 Processor -- start 2023-11-26T07:18:23.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.740+0000 7fd528ddf640 1 -- start start 2023-11-26T07:18:23.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd528ddf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5240cdfb0 0x7fd5240abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd528ddf640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5240b2710 con 0x7fd5240cbd50 2023-11-26T07:18:23.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd528ddf640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5240b29f0 con 0x7fd52415a710 2023-11-26T07:18:23.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd528ddf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240cca80 con 0x7fd5240cdfb0 2023-11-26T07:18:23.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd522d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd5240cbd50 0x7fd5240cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54446/0 (socket says 172.21.15.31:54446) 2023-11-26T07:18:23.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd522d9d640 1 -- 172.21.15.31:0/2843829556 learned_addr learned my addr 172.21.15.31:0/2843829556 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:23.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52259c640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5240cdfb0 0x7fd5240abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513906907 0 0) 0x7fd5240b2710 con 0x7fd5240cbd50 2023-11-26T07:18:23.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd50c005740 con 0x7fd5240cbd50 2023-11-26T07:18:23.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 582198300 0 0) 0x7fd50c005740 con 0x7fd5240cbd50 2023-11-26T07:18:23.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5240cdfb0 msgr2=0x7fd5240abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:23.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5240cdfb0 0x7fd5240abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 >> v1:172.21.15.97:6789/0 conn(0x7fd52415a710 legacy=0x7fd5240ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240b2cd0 con 0x7fd5240cbd50 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fd51c001a00 con 0x7fd5240cbd50 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 >> v1:172.21.15.31:6789/0 conn(0x7fd5240cbd50 legacy=0x7fd5240cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd50c005bc0 0x7fd50c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd50c006570 0x7fd50c006a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5240cdfb0 0x7fd50c0072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5240cca80 con 0x7fd50c006570 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c00dba0 con 0x7fd50c005bc0 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c00dd60 con 0x7fd5240cdfb0 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd521d9b640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd50c006570 0x7fd50c006a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd52259c640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd50c005bc0 0x7fd50c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd522d9d640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5240cdfb0 0x7fd50c0072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd521d9b640 1 -- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5240cdfb0 msgr2=0x7fd50c0072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:23.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd521d9b640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5240cdfb0 0x7fd50c0072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd521d9b640 1 -- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd50c005bc0 msgr2=0x7fd50c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:23.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd521d9b640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd50c005bc0 0x7fd50c005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.744+0000 7fd521d9b640 1 -- 172.21.15.31:0/2843829556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c005740 con 0x7fd50c006570 2023-11-26T07:18:23.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd521d9b640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd50c006570 0x7fd50c006a40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd518001870 tx=0x7fd51801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218a62a082794999 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:23.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180052d0 con 0x7fd50c006570 2023-11-26T07:18:23.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd518005430 con 0x7fd50c006570 2023-11-26T07:18:23.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd52159a640 1 -- 172.21.15.31:0/2843829556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518003820 con 0x7fd50c006570 2023-11-26T07:18:23.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd528ddf640 1 -- 172.21.15.31:0/2843829556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd50c006570 msgr2=0x7fd50c006a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd528ddf640 1 --2- 172.21.15.31:0/2843829556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd50c006570 0x7fd50c006a40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd518001870 tx=0x7fd51801c490 comp rx=0 tx=0).stop 2023-11-26T07:18:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd522d9d640 1 -- 172.21.15.31:0/2843829556 reap_dead start 2023-11-26T07:18:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd528ddf640 1 -- 172.21.15.31:0/2843829556 shutdown_connections 2023-11-26T07:18:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd528ddf640 1 -- 172.21.15.31:0/2843829556 >> 172.21.15.31:0/2843829556 conn(0x7fd5240b0cb0 msgr2=0x7fd5240b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd528ddf640 1 -- 172.21.15.31:0/2843829556 shutdown_connections 2023-11-26T07:18:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.748+0000 7fd528ddf640 1 -- 172.21.15.31:0/2843829556 wait complete. 2023-11-26T07:18:23.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd528ddf640 1 Processor -- start 2023-11-26T07:18:23.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd528ddf640 1 -- start start 2023-11-26T07:18:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd528ddf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd50c006570 0x7fd52415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd528ddf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5241f6110 0x7fd5240c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd528ddf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5240c9900 0x7fd5240ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd528ddf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52415ac50 con 0x7fd5240c9900 2023-11-26T07:18:23.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd528ddf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52415db00 con 0x7fd5241f6110 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd528ddf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd524099a30 con 0x7fd50c006570 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd522d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5240c9900 0x7fd5240ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd522d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5240c9900 0x7fd5240ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39736/0 (socket says 172.21.15.31:39736) 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd522d9d640 1 -- 172.21.15.31:0/3998344881 learned_addr learned my addr 172.21.15.31:0/3998344881 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd52259c640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd50c006570 0x7fd52415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd521d9b640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5241f6110 0x7fd5240c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd522d9d640 1 -- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd50c006570 msgr2=0x7fd52415d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd522d9d640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd50c006570 0x7fd52415d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd522d9d640 1 -- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5241f6110 msgr2=0x7fd5240c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd522d9d640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5241f6110 0x7fd5240c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:23.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd522d9d640 1 -- 172.21.15.31:0/3998344881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5241f6660 con 0x7fd5240c9900 2023-11-26T07:18:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd522d9d640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5240c9900 0x7fd5240ccea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd51c000f90 tx=0x7fd51c00a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:23.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd512ffd640 1 -- 172.21.15.31:0/3998344881 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c007970 con 0x7fd5240c9900 2023-11-26T07:18:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd512ffd640 1 -- 172.21.15.31:0/3998344881 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51c007b10 con 0x7fd5240c9900 2023-11-26T07:18:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd528ddf640 1 -- 172.21.15.31:0/3998344881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5241f6940 con 0x7fd5240c9900 2023-11-26T07:18:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd512ffd640 1 -- 172.21.15.31:0/3998344881 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51c013630 con 0x7fd5240c9900 2023-11-26T07:18:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.752+0000 7fd528ddf640 1 -- 172.21.15.31:0/3998344881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5241f6f00 con 0x7fd5240c9900 2023-11-26T07:18:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.756+0000 7fd528ddf640 1 -- 172.21.15.31:0/3998344881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5240671f0 con 0x7fd5240c9900 2023-11-26T07:18:23.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.756+0000 7fd512ffd640 1 -- 172.21.15.31:0/3998344881 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd51c013790 con 0x7fd5240c9900 2023-11-26T07:18:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.756+0000 7fd512ffd640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd5040664a0 0x7fd504068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:23.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.756+0000 7fd512ffd640 1 -- 172.21.15.31:0/3998344881 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd51c084e10 con 0x7fd5240c9900 2023-11-26T07:18:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.756+0000 7fd52259c640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd5040664a0 0x7fd504068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:23.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.760+0000 7fd52259c640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd5040664a0 0x7fd504068960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5240ce720 tx=0x7fd514009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:23.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.760+0000 7fd512ffd640 1 -- 172.21.15.31:0/3998344881 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd51c0523c0 con 0x7fd5240c9900 2023-11-26T07:18:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:23.916+0000 7fd528ddf640 1 -- 172.21.15.31:0/3998344881 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd52406b460 con 0x7fd5240c9900 2023-11-26T07:18:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd512ffd640 1 -- 172.21.15.31:0/3998344881 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd51c0559e0 con 0x7fd5240c9900 2023-11-26T07:18:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-11-26T07:18:24.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 -- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd5040664a0 msgr2=0x7fd504068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd5040664a0 0x7fd504068960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5240ce720 tx=0x7fd514009190 comp rx=0 tx=0).stop 2023-11-26T07:18:24.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 -- 172.21.15.31:0/3998344881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5240c9900 msgr2=0x7fd5240ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5240c9900 0x7fd5240ccea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd51c000f90 tx=0x7fd51c00a730 comp rx=0 tx=0).stop 2023-11-26T07:18:24.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 -- 172.21.15.31:0/3998344881 shutdown_connections 2023-11-26T07:18:24.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd5040664a0 0x7fd504068960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5240c9900 0x7fd5240ccea0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5241f6110 0x7fd5240c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 --2- 172.21.15.31:0/3998344881 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd50c006570 0x7fd52415d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 -- 172.21.15.31:0/3998344881 >> 172.21.15.31:0/3998344881 conn(0x7fd5240b0cb0 msgr2=0x7fd5240ae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:24.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 -- 172.21.15.31:0/3998344881 shutdown_connections 2023-11-26T07:18:24.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.560+0000 7fd528ddf640 1 -- 172.21.15.31:0/3998344881 wait complete. 2023-11-26T07:18:24.610 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:18:24.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3637e6b640 1 Processor -- start 2023-11-26T07:18:24.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3637e6b640 1 -- start start 2023-11-26T07:18:24.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3637e6b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300d0510 0x7f36300d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3637e6b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36300cdfb0 con 0x7f36300ced30 2023-11-26T07:18:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3637e6b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36300d4140 con 0x7f363015d8f0 2023-11-26T07:18:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3637e6b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300d4420 con 0x7f36300d0510 2023-11-26T07:18:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3635406640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f36300ced30 0x7f36300cfe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54448/0 (socket says 172.21.15.31:54448) 2023-11-26T07:18:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3635406640 1 -- 172.21.15.31:0/530794048 learned_addr learned my addr 172.21.15.31:0/530794048 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:24.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170326384 0 0) 0x7f36300cdfb0 con 0x7f36300ced30 2023-11-26T07:18:24.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f361c003530 con 0x7f36300ced30 2023-11-26T07:18:24.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3636408640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300d0510 0x7f36300d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1989671830 0 0) 0x7f361c003530 con 0x7f36300ced30 2023-11-26T07:18:24.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300d0510 msgr2=0x7f36300d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:24.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3634c05640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300d0510 0x7f36300d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 >> v1:172.21.15.97:6789/0 conn(0x7f363015d8f0 legacy=0x7f363015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.752+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36300ce250 con 0x7f36300ced30 2023-11-26T07:18:24.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f3620001680 con 0x7f36300ced30 2023-11-26T07:18:24.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 >> v1:172.21.15.31:6789/0 conn(0x7f36300ced30 legacy=0x7f36300cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f361c0039b0 0x7f361c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300d0510 0x7f361c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 0x7f361c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300d4420 con 0x7f361c004d10 2023-11-26T07:18:24.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c00ba70 con 0x7f361c0039b0 2023-11-26T07:18:24.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c00bc30 con 0x7f36300d0510 2023-11-26T07:18:24.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3636408640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 0x7f361c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3635406640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300d0510 0x7f361c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3635c07640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f361c0039b0 0x7f361c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3636408640 1 -- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300d0510 msgr2=0x7f361c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:24.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3636408640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300d0510 0x7f361c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3636408640 1 -- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f361c0039b0 msgr2=0x7f361c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3636408640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f361c0039b0 0x7f361c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3636408640 1 -- 172.21.15.31:0/530794048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c003530 con 0x7f361c004d10 2023-11-26T07:18:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3636408640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 0x7f361c005190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3628009f90 tx=0x7f362801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56328d9849135699 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36280a5c30 con 0x7f361c004d10 2023-11-26T07:18:24.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362800f040 con 0x7f361c004d10 2023-11-26T07:18:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.756+0000 7f3634c05640 1 -- 172.21.15.31:0/530794048 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36280ad440 con 0x7f361c004d10 2023-11-26T07:18:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 -- 172.21.15.31:0/530794048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 msgr2=0x7f361c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 --2- 172.21.15.31:0/530794048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 0x7f361c005190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3628009f90 tx=0x7f362801cc20 comp rx=0 tx=0).stop 2023-11-26T07:18:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3636408640 1 -- 172.21.15.31:0/530794048 reap_dead start 2023-11-26T07:18:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 -- 172.21.15.31:0/530794048 shutdown_connections 2023-11-26T07:18:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 -- 172.21.15.31:0/530794048 >> 172.21.15.31:0/530794048 conn(0x7f363009c750 msgr2=0x7f36300d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:24.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 -- 172.21.15.31:0/530794048 shutdown_connections 2023-11-26T07:18:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 -- 172.21.15.31:0/530794048 wait complete. 2023-11-26T07:18:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 Processor -- start 2023-11-26T07:18:24.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 -- start start 2023-11-26T07:18:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 0x7f36300c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300b16f0 0x7f36300cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36300b1b70 0x7f36300ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36301560a0 con 0x7f361c004d10 2023-11-26T07:18:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300c7f10 con 0x7f36300b1b70 2023-11-26T07:18:24.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3637e6b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36300c87b0 con 0x7f36300b16f0 2023-11-26T07:18:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3635c07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 0x7f36300c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3635c07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 0x7f36300c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39752/0 (socket says 172.21.15.31:39752) 2023-11-26T07:18:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3635c07640 1 -- 172.21.15.31:0/1938405955 learned_addr learned my addr 172.21.15.31:0/1938405955 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3636408640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36300b1b70 0x7f36300ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.760+0000 7f3635406640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300b16f0 0x7f36300cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f3635c07640 1 -- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300b16f0 msgr2=0x7f36300cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f3635c07640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300b16f0 0x7f36300cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f3635c07640 1 -- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36300b1b70 msgr2=0x7f36300ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f3635c07640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36300b1b70 0x7f36300ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f3635c07640 1 -- 172.21.15.31:0/1938405955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36301f41b0 con 0x7f361c004d10 2023-11-26T07:18:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f3635c07640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 0x7f36300c8c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f362c00f850 tx=0x7f362c00fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f36267fc640 1 -- 172.21.15.31:0/1938405955 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c0050d0 con 0x7f361c004d10 2023-11-26T07:18:24.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f36267fc640 1 -- 172.21.15.31:0/1938405955 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c005230 con 0x7f361c004d10 2023-11-26T07:18:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f36267fc640 1 -- 172.21.15.31:0/1938405955 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00d730 con 0x7f361c004d10 2023-11-26T07:18:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f3637e6b640 1 -- 172.21.15.31:0/1938405955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36300c8160 con 0x7f361c004d10 2023-11-26T07:18:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f3637e6b640 1 -- 172.21.15.31:0/1938405955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36301f3410 con 0x7f361c004d10 2023-11-26T07:18:24.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f3637e6b640 1 -- 172.21.15.31:0/1938405955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36300c9180 con 0x7f361c004d10 2023-11-26T07:18:24.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f36267fc640 1 -- 172.21.15.31:0/1938405955 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f362c00d890 con 0x7f361c004d10 2023-11-26T07:18:24.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f36267fc640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f36140664a0 0x7f3614068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:24.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f36267fc640 1 -- 172.21.15.31:0/1938405955 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f362c083d10 con 0x7f361c004d10 2023-11-26T07:18:24.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.764+0000 7f3635406640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f36140664a0 0x7f3614068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:24.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.768+0000 7f3635406640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f36140664a0 0x7f3614068960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f36300d2d10 tx=0x7f3620000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:24.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.772+0000 7f36267fc640 1 -- 172.21.15.31:0/1938405955 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f362c089020 con 0x7f361c004d10 2023-11-26T07:18:24.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:24.956+0000 7f3637e6b640 1 -- 172.21.15.31:0/1938405955 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f363004e850 con 0x7f361c004d10 2023-11-26T07:18:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f36267fc640 1 -- 172.21.15.31:0/1938405955 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f362c051150 con 0x7f361c004d10 2023-11-26T07:18:25.568 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 4 and data pool 5 2023-11-26T07:18:25.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f3637e6b640 1 -- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f36140664a0 msgr2=0x7f3614068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f3637e6b640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f36140664a0 0x7f3614068960 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f36300d2d10 tx=0x7f3620000ee0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f3637e6b640 1 -- 172.21.15.31:0/1938405955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 msgr2=0x7f36300c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f3637e6b640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 0x7f36300c8c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f362c00f850 tx=0x7f362c00fc50 comp rx=0 tx=0).stop 2023-11-26T07:18:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f3637e6b640 1 -- 172.21.15.31:0/1938405955 shutdown_connections 2023-11-26T07:18:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f3637e6b640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f36140664a0 0x7f3614068960 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f3637e6b640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36300b1b70 0x7f36300ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f3637e6b640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36300b16f0 0x7f36300cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f3637e6b640 1 --2- 172.21.15.31:0/1938405955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f361c004d10 0x7f36300c8c40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.564+0000 7f3637e6b640 1 -- 172.21.15.31:0/1938405955 >> 172.21.15.31:0/1938405955 conn(0x7f363009c750 msgr2=0x7f363015dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.568+0000 7f3637e6b640 1 -- 172.21.15.31:0/1938405955 shutdown_connections 2023-11-26T07:18:25.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.568+0000 7f3637e6b640 1 -- 172.21.15.31:0/1938405955 wait complete. 2023-11-26T07:18:25.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:25.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.764+0000 7f70609aa640 1 Processor -- start 2023-11-26T07:18:25.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.764+0000 7f70609aa640 1 -- start start 2023-11-26T07:18:25.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f70609aa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f705c15d3e0 0x7f705c0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f70609aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c06ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f70609aa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c070920 con 0x7f705c071f30 2023-11-26T07:18:25.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f70609aa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c071be0 con 0x7f705c15d3e0 2023-11-26T07:18:25.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f70609aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c15d7c0 con 0x7f705c06c630 2023-11-26T07:18:25.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f705a59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f705c15d3e0 0x7f705c0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f705ad9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f705c071f30 0x7f705c0b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54460/0 (socket says 172.21.15.31:54460) 2023-11-26T07:18:25.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f705ad9d640 1 -- 172.21.15.31:0/3230900645 learned_addr learned my addr 172.21.15.31:0/3230900645 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:25.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f7059d9b640 1 --2- 172.21.15.31:0/3230900645 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c06ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f705a59c640 1 -- 172.21.15.31:0/3230900645 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 msgr2=0x7f705c06ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:25.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f705a59c640 1 --2- 172.21.15.31:0/3230900645 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c06ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f705a59c640 1 -- 172.21.15.31:0/3230900645 >> v1:172.21.15.31:6789/0 conn(0x7f705c071f30 legacy=0x7f705c0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f705a59c640 1 -- 172.21.15.31:0/3230900645 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c070bc0 con 0x7f705c15d3e0 2023-11-26T07:18:25.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.768+0000 7f7059d9b640 1 --2- 172.21.15.31:0/3230900645 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c06ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:25.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f705a59c640 1 --2- 172.21.15.31:0/3230900645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f705c15d3e0 0x7f705c0716a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f704c0097f0 tx=0x7f704c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f14ec28d0e6f90a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f705959a640 1 -- 172.21.15.31:0/3230900645 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0ab650 con 0x7f705c15d3e0 2023-11-26T07:18:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f705959a640 1 -- 172.21.15.31:0/3230900645 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c0ab7b0 con 0x7f705c15d3e0 2023-11-26T07:18:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f705959a640 1 -- 172.21.15.31:0/3230900645 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c0abb40 con 0x7f705c15d3e0 2023-11-26T07:18:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f70609aa640 1 -- 172.21.15.31:0/3230900645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f705c15d3e0 msgr2=0x7f705c0716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f70609aa640 1 --2- 172.21.15.31:0/3230900645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f705c15d3e0 0x7f705c0716a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f704c0097f0 tx=0x7f704c01c450 comp rx=0 tx=0).stop 2023-11-26T07:18:25.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f70609aa640 1 -- 172.21.15.31:0/3230900645 shutdown_connections 2023-11-26T07:18:25.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f70609aa640 1 --2- 172.21.15.31:0/3230900645 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c06ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f70609aa640 1 --2- 172.21.15.31:0/3230900645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f705c15d3e0 0x7f705c0716a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f70609aa640 1 -- 172.21.15.31:0/3230900645 >> 172.21.15.31:0/3230900645 conn(0x7f705c0b0c40 msgr2=0x7f705c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:25.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f70609aa640 1 -- 172.21.15.31:0/3230900645 shutdown_connections 2023-11-26T07:18:25.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.772+0000 7f70609aa640 1 -- 172.21.15.31:0/3230900645 wait complete. 2023-11-26T07:18:25.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 Processor -- start 2023-11-26T07:18:25.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 -- start start 2023-11-26T07:18:25.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c071f30 0x7f705c14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f705c15d3e0 0x7f705c14ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c14f4f0 con 0x7f705c15d3e0 2023-11-26T07:18:25.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c14f7d0 con 0x7f705c071f30 2023-11-26T07:18:25.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c14fab0 con 0x7f705c06c630 2023-11-26T07:18:25.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f705c15d3e0 0x7f705c14ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705a59c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f7059d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c071f30 0x7f705c14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705a59c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:41886/0 (socket says 172.21.15.31:41886) 2023-11-26T07:18:25.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705a59c640 1 -- 172.21.15.31:0/2425381034 learned_addr learned my addr 172.21.15.31:0/2425381034 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705ad9d640 1 -- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 msgr2=0x7f705c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705ad9d640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705ad9d640 1 -- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c071f30 msgr2=0x7f705c14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705ad9d640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c071f30 0x7f705c14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705ad9d640 1 -- 172.21.15.31:0/2425381034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c009420 con 0x7f705c15d3e0 2023-11-26T07:18:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705a59c640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f7059d9b640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c071f30 0x7f705c14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f705ad9d640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f705c15d3e0 0x7f705c14ede0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f705400caa0 tx=0x7f705400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f704affd640 1 -- 172.21.15.31:0/2425381034 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400ac00 con 0x7f705c15d3e0 2023-11-26T07:18:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f704affd640 1 -- 172.21.15.31:0/2425381034 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705400ad60 con 0x7f705c15d3e0 2023-11-26T07:18:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f704affd640 1 -- 172.21.15.31:0/2425381034 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054012600 con 0x7f705c15d3e0 2023-11-26T07:18:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 -- 172.21.15.31:0/2425381034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c0728d0 con 0x7f705c15d3e0 2023-11-26T07:18:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 -- 172.21.15.31:0/2425381034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c09a8d0 con 0x7f705c15d3e0 2023-11-26T07:18:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.776+0000 7f70609aa640 1 -- 172.21.15.31:0/2425381034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c14f630 con 0x7f705c15d3e0 2023-11-26T07:18:25.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.780+0000 7f704affd640 1 -- 172.21.15.31:0/2425381034 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f7054007070 con 0x7f705c15d3e0 2023-11-26T07:18:25.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.780+0000 7f704affd640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f703c066470 0x7f703c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:25.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.780+0000 7f704affd640 1 -- 172.21.15.31:0/2425381034 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7054083bf0 con 0x7f705c15d3e0 2023-11-26T07:18:25.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.780+0000 7f705a59c640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f703c066470 0x7f703c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:25.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.780+0000 7f705a59c640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f703c066470 0x7f703c068930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f704c0097f0 tx=0x7f704c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:25.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.784+0000 7f704affd640 1 -- 172.21.15.31:0/2425381034 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f705c14f630 con 0x7f705c15d3e0 2023-11-26T07:18:25.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 -- 172.21.15.31:0/2425381034 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f705c14f630 con 0x7f705c15d3e0 2023-11-26T07:18:25.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f704affd640 1 -- 172.21.15.31:0/2425381034 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16498 (secure 0 0 0) 0x7f705c14f630 con 0x7f705c15d3e0 2023-11-26T07:18:25.949 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:25.949 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:24.962966+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:22.773068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:23.967477+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 -- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f703c066470 msgr2=0x7f703c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f703c066470 0x7f703c068930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f704c0097f0 tx=0x7f704c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 -- 172.21.15.31:0/2425381034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f705c15d3e0 msgr2=0x7f705c14ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f705c15d3e0 0x7f705c14ede0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f705400caa0 tx=0x7f705400cf60 comp rx=0 tx=0).stop 2023-11-26T07:18:25.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 -- 172.21.15.31:0/2425381034 shutdown_connections 2023-11-26T07:18:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f703c066470 0x7f703c068930 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f705c15d3e0 0x7f705c14ede0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f705c071f30 0x7f705c14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 --2- 172.21.15.31:0/2425381034 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f705c06c630 0x7f705c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 -- 172.21.15.31:0/2425381034 >> 172.21.15.31:0/2425381034 conn(0x7f705c0b0c40 msgr2=0x7f705c15d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 -- 172.21.15.31:0/2425381034 shutdown_connections 2023-11-26T07:18:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:25.944+0000 7f70609aa640 1 -- 172.21.15.31:0/2425381034 wait complete. 2023-11-26T07:18:25.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049f6e3640 1 Processor -- start 2023-11-26T07:18:26.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049f6e3640 1 -- start start 2023-11-26T07:18:26.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049f6e3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f04980cead0 0x7f04980d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049f6e3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f04980d1eb0 0x7f049806bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049f6e3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04980d22c0 con 0x7f049815d7b0 2023-11-26T07:18:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049f6e3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04980d17e0 con 0x7f04980d1eb0 2023-11-26T07:18:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049f6e3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04980cefd0 con 0x7f04980cead0 2023-11-26T07:18:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049d47f640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f049815d7b0 0x7f04980d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54462/0 (socket says 172.21.15.31:54462) 2023-11-26T07:18:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049d47f640 1 -- 172.21.15.31:0/3136318169 learned_addr learned my addr 172.21.15.31:0/3136318169 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049dc80640 1 --2- 172.21.15.31:0/3136318169 >> v2:172.21.15.31:3300/0 conn(0x7f04980d1eb0 0x7f049806bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049cc7e640 1 --2- 172.21.15.31:0/3136318169 >> v2:172.21.15.97:3300/0 conn(0x7f04980cead0 0x7f04980d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f048ffff640 1 -- 172.21.15.31:0/3136318169 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537587521 0 0) 0x7f04980d22c0 con 0x7f049815d7b0 2023-11-26T07:18:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f048ffff640 1 -- 172.21.15.31:0/3136318169 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0484003530 con 0x7f049815d7b0 2023-11-26T07:18:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049dc80640 1 -- 172.21.15.31:0/3136318169 >> v2:172.21.15.97:3300/0 conn(0x7f04980cead0 msgr2=0x7f04980d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:26.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049dc80640 1 --2- 172.21.15.31:0/3136318169 >> v2:172.21.15.97:3300/0 conn(0x7f04980cead0 0x7f04980d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049dc80640 1 -- 172.21.15.31:0/3136318169 >> v1:172.21.15.31:6789/0 conn(0x7f049815d7b0 legacy=0x7f04980d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.108+0000 7f049dc80640 1 -- 172.21.15.31:0/3136318169 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04980cf2b0 con 0x7f04980d1eb0 2023-11-26T07:18:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f049dc80640 1 --2- 172.21.15.31:0/3136318169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 0x7f049806bed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f049808ba60 tx=0x7f049001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b56b6fb06ecc21 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f048ffff640 1 -- 172.21.15.31:0/3136318169 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04900a5880 con 0x7f04980d1eb0 2023-11-26T07:18:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f048ffff640 1 -- 172.21.15.31:0/3136318169 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04900a59e0 con 0x7f04980d1eb0 2023-11-26T07:18:26.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f048ffff640 1 -- 172.21.15.31:0/3136318169 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04900a5d70 con 0x7f04980d1eb0 2023-11-26T07:18:26.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f049f6e3640 1 -- 172.21.15.31:0/3136318169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 msgr2=0x7f049806bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f049f6e3640 1 --2- 172.21.15.31:0/3136318169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 0x7f049806bed0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f049808ba60 tx=0x7f049001c7c0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f049f6e3640 1 -- 172.21.15.31:0/3136318169 shutdown_connections 2023-11-26T07:18:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f049f6e3640 1 --2- 172.21.15.31:0/3136318169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 0x7f049806bed0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f049f6e3640 1 --2- 172.21.15.31:0/3136318169 >> v2:172.21.15.97:3300/0 conn(0x7f04980cead0 0x7f04980d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f049f6e3640 1 -- 172.21.15.31:0/3136318169 >> 172.21.15.31:0/3136318169 conn(0x7f04980d33b0 msgr2=0x7f04980d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f049f6e3640 1 -- 172.21.15.31:0/3136318169 shutdown_connections 2023-11-26T07:18:26.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.112+0000 7f049f6e3640 1 -- 172.21.15.31:0/3136318169 wait complete. 2023-11-26T07:18:26.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049f6e3640 1 Processor -- start 2023-11-26T07:18:26.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049f6e3640 1 -- start start 2023-11-26T07:18:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049f6e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04980cead0 0x7f0498157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049f6e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 0x7f04980cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049f6e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f049815d7b0 0x7f04980b8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049f6e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04980cbed0 con 0x7f04980d1eb0 2023-11-26T07:18:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049f6e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04981582a0 con 0x7f04980cead0 2023-11-26T07:18:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049f6e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04981584f0 con 0x7f049815d7b0 2023-11-26T07:18:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049cc7e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 0x7f04980cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049cc7e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 0x7f04980cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39786/0 (socket says 172.21.15.31:39786) 2023-11-26T07:18:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049cc7e640 1 -- 172.21.15.31:0/653145174 learned_addr learned my addr 172.21.15.31:0/653145174 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049cc7e640 1 -- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f049815d7b0 msgr2=0x7f04980b8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049cc7e640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f049815d7b0 0x7f04980b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049cc7e640 1 -- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04980cead0 msgr2=0x7f0498157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049cc7e640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04980cead0 0x7f0498157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049cc7e640 1 -- 172.21.15.31:0/653145174 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490009a70 con 0x7f04980d1eb0 2023-11-26T07:18:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049cc7e640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 0x7f04980cd2b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f048800e900 tx=0x7f048800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f048dffb640 1 -- 172.21.15.31:0/653145174 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488009690 con 0x7f04980d1eb0 2023-11-26T07:18:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f048dffb640 1 -- 172.21.15.31:0/653145174 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048800ca00 con 0x7f04980d1eb0 2023-11-26T07:18:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f048dffb640 1 -- 172.21.15.31:0/653145174 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800cdb0 con 0x7f04980d1eb0 2023-11-26T07:18:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049f6e3640 1 -- 172.21.15.31:0/653145174 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04981587d0 con 0x7f04980d1eb0 2023-11-26T07:18:26.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.116+0000 7f049f6e3640 1 -- 172.21.15.31:0/653145174 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0498158d90 con 0x7f04980d1eb0 2023-11-26T07:18:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.120+0000 7f049f6e3640 1 -- 172.21.15.31:0/653145174 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049804e850 con 0x7f04980d1eb0 2023-11-26T07:18:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.120+0000 7f048dffb640 1 -- 172.21.15.31:0/653145174 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0488013470 con 0x7f04980d1eb0 2023-11-26T07:18:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.120+0000 7f048dffb640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f047c066470 0x7f047c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.120+0000 7f048dffb640 1 -- 172.21.15.31:0/653145174 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0488084200 con 0x7f04980d1eb0 2023-11-26T07:18:26.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.120+0000 7f049d47f640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f047c066470 0x7f047c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.124+0000 7f049d47f640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f047c066470 0x7f047c068930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0494007fd0 tx=0x7f0494007940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.124+0000 7f048dffb640 1 -- 172.21.15.31:0/653145174 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0488051550 con 0x7f04980d1eb0 2023-11-26T07:18:26.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.284+0000 7f049f6e3640 1 -- 172.21.15.31:0/653145174 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f049815cfe0 con 0x7f04980d1eb0 2023-11-26T07:18:26.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f048dffb640 1 -- 172.21.15.31:0/653145174 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16498 (secure 0 0 0) 0x7f0488054b70 con 0x7f04980d1eb0 2023-11-26T07:18:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:26.292 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:24.962966+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:22.773068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:23.967477+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 -- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f047c066470 msgr2=0x7f047c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f047c066470 0x7f047c068930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0494007fd0 tx=0x7f0494007940 comp rx=0 tx=0).stop 2023-11-26T07:18:26.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 -- 172.21.15.31:0/653145174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 msgr2=0x7f04980cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 0x7f04980cd2b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f048800e900 tx=0x7f048800edc0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 -- 172.21.15.31:0/653145174 shutdown_connections 2023-11-26T07:18:26.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f047c066470 0x7f047c068930 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f049815d7b0 0x7f04980b8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04980d1eb0 0x7f04980cd2b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 --2- 172.21.15.31:0/653145174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04980cead0 0x7f0498157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 -- 172.21.15.31:0/653145174 >> 172.21.15.31:0/653145174 conn(0x7f04980d33b0 msgr2=0x7f04980d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:26.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 -- 172.21.15.31:0/653145174 shutdown_connections 2023-11-26T07:18:26.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.288+0000 7f049f6e3640 1 -- 172.21.15.31:0/653145174 wait complete. 2023-11-26T07:18:26.313 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:18:26.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.452+0000 7fceaa3a1640 1 Processor -- start 2023-11-26T07:18:26.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fceaa3a1640 1 -- start start 2023-11-26T07:18:26.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fceaa3a1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcea4146820 0x7fcea4070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fceaa3a1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcea415eb50 0x7fcea40b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fceaa3a1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea4147190 con 0x7fcea40b4210 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fceaa3a1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40b4be0 con 0x7fcea4146820 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fceaa3a1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea4146c00 con 0x7fcea415eb50 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fcea3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcea4146820 0x7fcea4070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fcea3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcea4146820 0x7fcea4070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39800/0 (socket says 172.21.15.31:39800) 2023-11-26T07:18:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fcea3fff640 1 -- 172.21.15.31:0/1724535817 learned_addr learned my addr 172.21.15.31:0/1724535817 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fcea37fe640 1 --1- 172.21.15.31:0/1724535817 >> v1:172.21.15.31:6789/0 conn(0x7fcea40b4210 0x7fcea4161180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54472/0 (socket says 172.21.15.31:54472) 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fcea893e640 1 --2- 172.21.15.31:0/1724535817 >> v2:172.21.15.97:3300/0 conn(0x7fcea415eb50 0x7fcea40b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fcea3fff640 1 -- 172.21.15.31:0/1724535817 >> v2:172.21.15.97:3300/0 conn(0x7fcea415eb50 msgr2=0x7fcea40b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fcea3fff640 1 --2- 172.21.15.31:0/1724535817 >> v2:172.21.15.97:3300/0 conn(0x7fcea415eb50 0x7fcea40b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fcea3fff640 1 -- 172.21.15.31:0/1724535817 >> v1:172.21.15.31:6789/0 conn(0x7fcea40b4210 legacy=0x7fcea4161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.456+0000 7fcea3fff640 1 -- 172.21.15.31:0/1724535817 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea406d030 con 0x7fcea4146820 2023-11-26T07:18:26.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fcea3fff640 1 --2- 172.21.15.31:0/1724535817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea4146820 0x7fcea4070a50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcea40d22d0 tx=0x7fce9401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0d2db14e336710 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fcea2ffd640 1 -- 172.21.15.31:0/1724535817 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940ab5f0 con 0x7fcea4146820 2023-11-26T07:18:26.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fcea2ffd640 1 -- 172.21.15.31:0/1724535817 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce940ab750 con 0x7fcea4146820 2023-11-26T07:18:26.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fcea2ffd640 1 -- 172.21.15.31:0/1724535817 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940abae0 con 0x7fcea4146820 2023-11-26T07:18:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fceaa3a1640 1 -- 172.21.15.31:0/1724535817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea4146820 msgr2=0x7fcea4070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fceaa3a1640 1 --2- 172.21.15.31:0/1724535817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea4146820 0x7fcea4070a50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcea40d22d0 tx=0x7fce9401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fceaa3a1640 1 -- 172.21.15.31:0/1724535817 shutdown_connections 2023-11-26T07:18:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fceaa3a1640 1 --2- 172.21.15.31:0/1724535817 >> v2:172.21.15.97:3300/0 conn(0x7fcea415eb50 0x7fcea40b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fceaa3a1640 1 --2- 172.21.15.31:0/1724535817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea4146820 0x7fcea4070a50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fceaa3a1640 1 -- 172.21.15.31:0/1724535817 >> 172.21.15.31:0/1724535817 conn(0x7fcea40b2f10 msgr2=0x7fcea40b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fceaa3a1640 1 -- 172.21.15.31:0/1724535817 shutdown_connections 2023-11-26T07:18:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fceaa3a1640 1 -- 172.21.15.31:0/1724535817 wait complete. 2023-11-26T07:18:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fceaa3a1640 1 Processor -- start 2023-11-26T07:18:26.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.460+0000 7fceaa3a1640 1 -- start start 2023-11-26T07:18:26.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fceaa3a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea40b4210 0x7fcea412a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fceaa3a1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcea4146820 0x7fcea415b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fceaa3a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea415eb50 0x7fcea4156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fceaa3a1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea413b2f0 con 0x7fcea40b4210 2023-11-26T07:18:26.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fceaa3a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4157300 con 0x7fcea415eb50 2023-11-26T07:18:26.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fceaa3a1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea41575e0 con 0x7fcea4146820 2023-11-26T07:18:26.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea40b4210 0x7fcea412a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea40b4210 0x7fcea412a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39812/0 (socket says 172.21.15.31:39812) 2023-11-26T07:18:26.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea3fff640 1 -- 172.21.15.31:0/707598700 learned_addr learned my addr 172.21.15.31:0/707598700 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea37fe640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcea4146820 0x7fcea415b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea893e640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea415eb50 0x7fcea4156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea3fff640 1 -- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcea4146820 msgr2=0x7fcea415b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea3fff640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcea4146820 0x7fcea415b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea3fff640 1 -- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea415eb50 msgr2=0x7fcea4156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea3fff640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea415eb50 0x7fcea4156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:26.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea3fff640 1 -- 172.21.15.31:0/707598700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce940094d0 con 0x7fcea40b4210 2023-11-26T07:18:26.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea3fff640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea40b4210 0x7fcea412a1f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fce940190e0 tx=0x7fce9401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea0ff9640 1 -- 172.21.15.31:0/707598700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940be5f0 con 0x7fcea40b4210 2023-11-26T07:18:26.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea0ff9640 1 -- 172.21.15.31:0/707598700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce940be750 con 0x7fcea40b4210 2023-11-26T07:18:26.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fcea0ff9640 1 -- 172.21.15.31:0/707598700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940beb00 con 0x7fcea40b4210 2023-11-26T07:18:26.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fceaa3a1640 1 -- 172.21.15.31:0/707598700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea41578c0 con 0x7fcea40b4210 2023-11-26T07:18:26.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fceaa3a1640 1 -- 172.21.15.31:0/707598700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea4139e30 con 0x7fcea40b4210 2023-11-26T07:18:26.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.464+0000 7fceaa3a1640 1 -- 172.21.15.31:0/707598700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea413a020 con 0x7fcea40b4210 2023-11-26T07:18:26.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.468+0000 7fcea0ff9640 1 -- 172.21.15.31:0/707598700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fce940b43f0 con 0x7fcea40b4210 2023-11-26T07:18:26.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.468+0000 7fcea0ff9640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fce90066470 0x7fce90068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:26.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.468+0000 7fcea0ff9640 1 -- 172.21.15.31:0/707598700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce940a8050 con 0x7fcea40b4210 2023-11-26T07:18:26.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.468+0000 7fcea37fe640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fce90066470 0x7fce90068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:26.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.468+0000 7fcea37fe640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fce90066470 0x7fce90068930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fce98000f70 tx=0x7fce98007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:26.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.472+0000 7fcea0ff9640 1 -- 172.21.15.31:0/707598700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fce940ebed0 con 0x7fcea40b4210 2023-11-26T07:18:26.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:26.668+0000 7fceaa3a1640 1 -- 172.21.15.31:0/707598700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcea404e850 con 0x7fcea40b4210 2023-11-26T07:18:27.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fcea0ff9640 1 -- 172.21.15.31:0/707598700 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fce940ef4f0 con 0x7fcea40b4210 2023-11-26T07:18:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fceaa3a1640 1 -- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fce90066470 msgr2=0x7fce90068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fceaa3a1640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fce90066470 0x7fce90068930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fce98000f70 tx=0x7fce98007c90 comp rx=0 tx=0).stop 2023-11-26T07:18:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fceaa3a1640 1 -- 172.21.15.31:0/707598700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea40b4210 msgr2=0x7fcea412a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fceaa3a1640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea40b4210 0x7fcea412a1f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fce940190e0 tx=0x7fce9401ad10 comp rx=0 tx=0).stop 2023-11-26T07:18:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fceaa3a1640 1 -- 172.21.15.31:0/707598700 shutdown_connections 2023-11-26T07:18:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fceaa3a1640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fce90066470 0x7fce90068930 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fceaa3a1640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcea415eb50 0x7fcea4156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fceaa3a1640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcea4146820 0x7fcea415b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fceaa3a1640 1 --2- 172.21.15.31:0/707598700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea40b4210 0x7fcea412a1f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.588+0000 7fceaa3a1640 1 -- 172.21.15.31:0/707598700 >> 172.21.15.31:0/707598700 conn(0x7fcea40b2f10 msgr2=0x7fcea40b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.592+0000 7fceaa3a1640 1 -- 172.21.15.31:0/707598700 shutdown_connections 2023-11-26T07:18:27.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.592+0000 7fceaa3a1640 1 -- 172.21.15.31:0/707598700 wait complete. 2023-11-26T07:18:27.619 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:18:27.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:18:27.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f123c416640 1 Processor -- start 2023-11-26T07:18:27.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f123c416640 1 -- start start 2023-11-26T07:18:27.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f123c416640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f12340cdfb0 0x7f12340abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f123c416640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f123415a710 0x7f12340ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f123c416640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12340ce8d0 con 0x7f12340cbd50 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f123c416640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12340cca80 con 0x7f123415a710 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f123c416640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12340b2710 con 0x7f12340cdfb0 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f12399b1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f123415a710 0x7f12340ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f12399b1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f123415a710 0x7f12340ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39826/0 (socket says 172.21.15.31:39826) 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f12399b1640 1 -- 172.21.15.31:0/1740317625 learned_addr learned my addr 172.21.15.31:0/1740317625 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f123a1b2640 1 --2- 172.21.15.31:0/1740317625 >> v2:172.21.15.97:3300/0 conn(0x7f12340cdfb0 0x7f12340abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f12399b1640 1 -- 172.21.15.31:0/1740317625 >> v2:172.21.15.97:3300/0 conn(0x7f12340cdfb0 msgr2=0x7f12340abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.764+0000 7f12399b1640 1 --2- 172.21.15.31:0/1740317625 >> v2:172.21.15.97:3300/0 conn(0x7f12340cdfb0 0x7f12340abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f12399b1640 1 -- 172.21.15.31:0/1740317625 >> v1:172.21.15.97:6789/0 conn(0x7f12340cbd50 legacy=0x7f12340cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f12399b1640 1 -- 172.21.15.31:0/1740317625 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12340b29f0 con 0x7f123415a710 2023-11-26T07:18:27.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f12399b1640 1 --2- 172.21.15.31:0/1740317625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f123415a710 0x7f12340ce390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f123406c270 tx=0x7f122401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592cf104c6549c84 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f12391b0640 1 -- 172.21.15.31:0/1740317625 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240aa430 con 0x7f123415a710 2023-11-26T07:18:27.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f12391b0640 1 -- 172.21.15.31:0/1740317625 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12240aa590 con 0x7f123415a710 2023-11-26T07:18:27.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f12391b0640 1 -- 172.21.15.31:0/1740317625 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12240aa920 con 0x7f123415a710 2023-11-26T07:18:27.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f123c416640 1 -- 172.21.15.31:0/1740317625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f123415a710 msgr2=0x7f12340ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f123c416640 1 --2- 172.21.15.31:0/1740317625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f123415a710 0x7f12340ce390 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f123406c270 tx=0x7f122401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f123c416640 1 -- 172.21.15.31:0/1740317625 shutdown_connections 2023-11-26T07:18:27.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f123c416640 1 --2- 172.21.15.31:0/1740317625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f123415a710 0x7f12340ce390 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f123c416640 1 --2- 172.21.15.31:0/1740317625 >> v2:172.21.15.97:3300/0 conn(0x7f12340cdfb0 0x7f12340abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f123c416640 1 -- 172.21.15.31:0/1740317625 >> 172.21.15.31:0/1740317625 conn(0x7f12340b0cb0 msgr2=0x7f12340b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:27.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f123c416640 1 -- 172.21.15.31:0/1740317625 shutdown_connections 2023-11-26T07:18:27.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.768+0000 7f123c416640 1 -- 172.21.15.31:0/1740317625 wait complete. 2023-11-26T07:18:27.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123c416640 1 Processor -- start 2023-11-26T07:18:27.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123c416640 1 -- start start 2023-11-26T07:18:27.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123c416640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12340cbd50 0x7f1234157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123c416640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12340cdfb0 0x7f12340716e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123c416640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f123415a710 0x7f12341f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123c416640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123414c7d0 con 0x7f12340cbd50 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123c416640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123414cab0 con 0x7f123415a710 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123c416640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123414cd90 con 0x7f12340cdfb0 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123a1b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12340cbd50 0x7f1234157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123a1b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12340cbd50 0x7f1234157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39832/0 (socket says 172.21.15.31:39832) 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123a1b2640 1 -- 172.21.15.31:0/962419438 learned_addr learned my addr 172.21.15.31:0/962419438 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f12399b1640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12340cdfb0 0x7f12340716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123a9b3640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f123415a710 0x7f12341f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123a1b2640 1 -- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12340cdfb0 msgr2=0x7f12340716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123a1b2640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12340cdfb0 0x7f12340716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123a1b2640 1 -- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f123415a710 msgr2=0x7f12341f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123a1b2640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f123415a710 0x7f12341f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123a1b2640 1 -- 172.21.15.31:0/962419438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240094d0 con 0x7f12340cbd50 2023-11-26T07:18:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123a1b2640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12340cbd50 0x7f1234157290 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f123000d800 tx=0x7f123000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f122affd640 1 -- 172.21.15.31:0/962419438 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123000b6c0 con 0x7f12340cbd50 2023-11-26T07:18:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f122affd640 1 -- 172.21.15.31:0/962419438 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123000b820 con 0x7f12340cbd50 2023-11-26T07:18:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123c416640 1 -- 172.21.15.31:0/962419438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123414d070 con 0x7f12340cbd50 2023-11-26T07:18:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f122affd640 1 -- 172.21.15.31:0/962419438 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123000bb90 con 0x7f12340cbd50 2023-11-26T07:18:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.772+0000 7f123c416640 1 -- 172.21.15.31:0/962419438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1234071c20 con 0x7f12340cbd50 2023-11-26T07:18:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.776+0000 7f123c416640 1 -- 172.21.15.31:0/962419438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12340671f0 con 0x7f12340cbd50 2023-11-26T07:18:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.776+0000 7f122affd640 1 -- 172.21.15.31:0/962419438 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f123000bcf0 con 0x7f12340cbd50 2023-11-26T07:18:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.776+0000 7f122affd640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1218066470 0x7f1218068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:27.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.776+0000 7f122affd640 1 -- 172.21.15.31:0/962419438 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1230084340 con 0x7f12340cbd50 2023-11-26T07:18:27.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.776+0000 7f12399b1640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1218066470 0x7f1218068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:27.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.780+0000 7f12399b1640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1218066470 0x7f1218068930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12240ae000 tx=0x7f12240b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.780+0000 7f122affd640 1 -- 172.21.15.31:0/962419438 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f12300517b0 con 0x7f12340cbd50 2023-11-26T07:18:27.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:27.976+0000 7f123c416640 1 -- 172.21.15.31:0/962419438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f123406b460 con 0x7f12340cbd50 2023-11-26T07:18:28.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f122affd640 1 -- 172.21.15.31:0/962419438 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1230054dd0 con 0x7f12340cbd50 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 -- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1218066470 msgr2=0x7f1218068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1218066470 0x7f1218068930 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12240ae000 tx=0x7f12240b5040 comp rx=0 tx=0).stop 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 -- 172.21.15.31:0/962419438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12340cbd50 msgr2=0x7f1234157290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12340cbd50 0x7f1234157290 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f123000d800 tx=0x7f123000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 -- 172.21.15.31:0/962419438 shutdown_connections 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1218066470 0x7f1218068930 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f123415a710 0x7f12341f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f12340cdfb0 0x7f12340716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 --2- 172.21.15.31:0/962419438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12340cbd50 0x7f1234157290 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 -- 172.21.15.31:0/962419438 >> 172.21.15.31:0/962419438 conn(0x7f12340b0cb0 msgr2=0x7f12340b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:28.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 -- 172.21.15.31:0/962419438 shutdown_connections 2023-11-26T07:18:28.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.596+0000 7f123c416640 1 -- 172.21.15.31:0/962419438 wait complete. 2023-11-26T07:18:28.626 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:18:28.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:28.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc20c632640 1 Processor -- start 2023-11-26T07:18:28.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc20c632640 1 -- start start 2023-11-26T07:18:28.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc20c632640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2040ac630 0x7fc2040706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc20c632640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc204070c20 0x7fc20406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc20c632640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc204145060 con 0x7fc204144c80 2023-11-26T07:18:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc20c632640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc204071fd0 con 0x7fc2040ac630 2023-11-26T07:18:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc20c632640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc204071450 con 0x7fc204070c20 2023-11-26T07:18:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc209bcd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2040ac630 0x7fc2040706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc209bcd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc2040ac630 0x7fc2040706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39838/0 (socket says 172.21.15.31:39838) 2023-11-26T07:18:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc209bcd640 1 -- 172.21.15.31:0/4126926434 learned_addr learned my addr 172.21.15.31:0/4126926434 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc20abcf640 1 --2- 172.21.15.31:0/4126926434 >> v2:172.21.15.97:3300/0 conn(0x7fc204070c20 0x7fc20406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc209bcd640 1 -- 172.21.15.31:0/4126926434 >> v2:172.21.15.97:3300/0 conn(0x7fc204070c20 msgr2=0x7fc20406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc209bcd640 1 --2- 172.21.15.31:0/4126926434 >> v2:172.21.15.97:3300/0 conn(0x7fc204070c20 0x7fc20406c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc209bcd640 1 -- 172.21.15.31:0/4126926434 >> v1:172.21.15.97:6789/0 conn(0x7fc204144c80 legacy=0x7fc2040abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.768+0000 7fc209bcd640 1 -- 172.21.15.31:0/4126926434 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc204071730 con 0x7fc2040ac630 2023-11-26T07:18:28.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc209bcd640 1 --2- 172.21.15.31:0/4126926434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040ac630 0x7fc2040706e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc2040ce840 tx=0x7fc1f401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218095f3fbe7da32 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc2093cc640 1 -- 172.21.15.31:0/4126926434 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40aa430 con 0x7fc2040ac630 2023-11-26T07:18:28.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc2093cc640 1 -- 172.21.15.31:0/4126926434 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f40aa590 con 0x7fc2040ac630 2023-11-26T07:18:28.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc2093cc640 1 -- 172.21.15.31:0/4126926434 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40aa920 con 0x7fc2040ac630 2023-11-26T07:18:28.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc20c632640 1 -- 172.21.15.31:0/4126926434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040ac630 msgr2=0x7fc2040706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc20c632640 1 --2- 172.21.15.31:0/4126926434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040ac630 0x7fc2040706e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc2040ce840 tx=0x7fc1f401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc20c632640 1 -- 172.21.15.31:0/4126926434 shutdown_connections 2023-11-26T07:18:28.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc20c632640 1 --2- 172.21.15.31:0/4126926434 >> v2:172.21.15.97:3300/0 conn(0x7fc204070c20 0x7fc20406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc20c632640 1 --2- 172.21.15.31:0/4126926434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040ac630 0x7fc2040706e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc20c632640 1 -- 172.21.15.31:0/4126926434 >> 172.21.15.31:0/4126926434 conn(0x7fc2040b0c40 msgr2=0x7fc2040b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:28.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc20c632640 1 -- 172.21.15.31:0/4126926434 shutdown_connections 2023-11-26T07:18:28.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.772+0000 7fc20c632640 1 -- 172.21.15.31:0/4126926434 wait complete. 2023-11-26T07:18:28.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20c632640 1 Processor -- start 2023-11-26T07:18:28.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20c632640 1 -- start start 2023-11-26T07:18:28.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20c632640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc204070c20 0x7fc20413cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20c632640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2040ac630 0x7fc204159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20c632640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc204144c80 0x7fc2041f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20c632640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20409b230 con 0x7fc204070c20 2023-11-26T07:18:28.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20c632640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20415a4f0 con 0x7fc2040ac630 2023-11-26T07:18:28.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20c632640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20413d5a0 con 0x7fc204144c80 2023-11-26T07:18:28.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20a3ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc204070c20 0x7fc20413cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20a3ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc204070c20 0x7fc20413cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39846/0 (socket says 172.21.15.31:39846) 2023-11-26T07:18:28.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20a3ce640 1 -- 172.21.15.31:0/1940542460 learned_addr learned my addr 172.21.15.31:0/1940542460 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:28.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc209bcd640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2040ac630 0x7fc204159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20abcf640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc204144c80 0x7fc2041f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20a3ce640 1 -- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc204144c80 msgr2=0x7fc2041f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20a3ce640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc204144c80 0x7fc2041f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20a3ce640 1 -- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2040ac630 msgr2=0x7fc204159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20a3ce640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2040ac630 0x7fc204159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20a3ce640 1 -- 172.21.15.31:0/1940542460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f40094d0 con 0x7fc204070c20 2023-11-26T07:18:28.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20abcf640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc204144c80 0x7fc2041f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:28.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20a3ce640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc204070c20 0x7fc20413cdd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc20000c3c0 tx=0x7fc20000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc1faffd640 1 -- 172.21.15.31:0/1940542460 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000052d0 con 0x7fc204070c20 2023-11-26T07:18:28.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc1faffd640 1 -- 172.21.15.31:0/1940542460 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc200005430 con 0x7fc204070c20 2023-11-26T07:18:28.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20c632640 1 -- 172.21.15.31:0/1940542460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2041f65f0 con 0x7fc204070c20 2023-11-26T07:18:28.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc1faffd640 1 -- 172.21.15.31:0/1940542460 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200003840 con 0x7fc204070c20 2023-11-26T07:18:28.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.776+0000 7fc20c632640 1 -- 172.21.15.31:0/1940542460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20413e0e0 con 0x7fc204070c20 2023-11-26T07:18:28.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.780+0000 7fc20c632640 1 -- 172.21.15.31:0/1940542460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2040b1ba0 con 0x7fc204070c20 2023-11-26T07:18:28.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.780+0000 7fc1faffd640 1 -- 172.21.15.31:0/1940542460 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc2000039a0 con 0x7fc204070c20 2023-11-26T07:18:28.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.780+0000 7fc1faffd640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1e8066470 0x7fc1e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:28.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.780+0000 7fc1faffd640 1 -- 172.21.15.31:0/1940542460 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2000833c0 con 0x7fc204070c20 2023-11-26T07:18:28.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.780+0000 7fc209bcd640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1e8066470 0x7fc1e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:28.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.784+0000 7fc209bcd640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1e8066470 0x7fc1e8068930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40ae000 tx=0x7fc1f40b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:28.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.784+0000 7fc1faffd640 1 -- 172.21.15.31:0/1940542460 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc200050850 con 0x7fc204070c20 2023-11-26T07:18:28.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 -- 172.21.15.31:0/1940542460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc20404e850 con 0x7fc204070c20 2023-11-26T07:18:28.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc1faffd640 1 -- 172.21.15.31:0/1940542460 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fc200053e70 con 0x7fc204070c20 2023-11-26T07:18:28.980 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:28.980 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3937918726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3937918726},{"type":"v1","addr":"172.21.15.97:6837","nonce":3937918726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2083665480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2083665480},{"type":"v1","addr":"172.21.15.97:6835","nonce":2083665480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1187139549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187139549},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187139549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:24.963367+0000","modified":"2023-11-26T07:18:27.980260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4036387633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4036387633},{"type":"v1","addr":"172.21.15.31:6835","nonce":4036387633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:18:28.981 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2023-11-26T07:18:28.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 -- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1e8066470 msgr2=0x7fc1e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1e8066470 0x7fc1e8068930 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40ae000 tx=0x7fc1f40b5040 comp rx=0 tx=0).stop 2023-11-26T07:18:28.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 -- 172.21.15.31:0/1940542460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc204070c20 msgr2=0x7fc20413cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:28.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc204070c20 0x7fc20413cdd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc20000c3c0 tx=0x7fc20000c880 comp rx=0 tx=0).stop 2023-11-26T07:18:28.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 -- 172.21.15.31:0/1940542460 shutdown_connections 2023-11-26T07:18:28.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1e8066470 0x7fc1e8068930 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc204144c80 0x7fc2041f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2040ac630 0x7fc204159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 --2- 172.21.15.31:0/1940542460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc204070c20 0x7fc20413cdd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:28.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 -- 172.21.15.31:0/1940542460 >> 172.21.15.31:0/1940542460 conn(0x7fc2040b0c40 msgr2=0x7fc204140a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 -- 172.21.15.31:0/1940542460 shutdown_connections 2023-11-26T07:18:28.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:28.976+0000 7fc20c632640 1 -- 172.21.15.31:0/1940542460 wait complete. 2023-11-26T07:18:29.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:29.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9202780640 1 Processor -- start 2023-11-26T07:18:29.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9202780640 1 -- start start 2023-11-26T07:18:29.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9202780640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91fc0cead0 0x7f91fc0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9202780640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91fc0d1eb0 0x7f91fc06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9202780640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91fc0cefd0 con 0x7f91fc15d7b0 2023-11-26T07:18:29.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9202780640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91fc0d17e0 con 0x7f91fc0d1eb0 2023-11-26T07:18:29.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9202780640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0d2340 con 0x7f91fc0cead0 2023-11-26T07:18:29.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f91fb7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91fc0cead0 0x7f91fc0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9200d1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91fc0d1eb0 0x7f91fc06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f91fbfff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f91fc15d7b0 0x7f91fc0d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:39792/0 (socket says 172.21.15.31:39792) 2023-11-26T07:18:29.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f91fbfff640 1 -- 172.21.15.31:0/1770649788 learned_addr learned my addr 172.21.15.31:0/1770649788 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:29.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9200d1d640 1 -- 172.21.15.31:0/1770649788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91fc0cead0 msgr2=0x7f91fc0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:29.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9200d1d640 1 --2- 172.21.15.31:0/1770649788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91fc0cead0 0x7f91fc0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9200d1d640 1 -- 172.21.15.31:0/1770649788 >> v1:172.21.15.97:6789/0 conn(0x7f91fc15d7b0 legacy=0x7f91fc0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.140+0000 7f9200d1d640 1 -- 172.21.15.31:0/1770649788 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91fc0cf2b0 con 0x7f91fc0d1eb0 2023-11-26T07:18:29.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f9200d1d640 1 --2- 172.21.15.31:0/1770649788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc0d1eb0 0x7f91fc06bed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009d90 tx=0x7f91f401c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4e65b888b710be3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f91faffd640 1 -- 172.21.15.31:0/1770649788 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40a58c0 con 0x7f91fc0d1eb0 2023-11-26T07:18:29.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f91faffd640 1 -- 172.21.15.31:0/1770649788 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f40a5a20 con 0x7f91fc0d1eb0 2023-11-26T07:18:29.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f91faffd640 1 -- 172.21.15.31:0/1770649788 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40a5db0 con 0x7f91fc0d1eb0 2023-11-26T07:18:29.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f9202780640 1 -- 172.21.15.31:0/1770649788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc0d1eb0 msgr2=0x7f91fc06bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f9202780640 1 --2- 172.21.15.31:0/1770649788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc0d1eb0 0x7f91fc06bed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009d90 tx=0x7f91f401c800 comp rx=0 tx=0).stop 2023-11-26T07:18:29.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f9202780640 1 -- 172.21.15.31:0/1770649788 shutdown_connections 2023-11-26T07:18:29.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f9202780640 1 --2- 172.21.15.31:0/1770649788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc0d1eb0 0x7f91fc06bed0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f9202780640 1 --2- 172.21.15.31:0/1770649788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91fc0cead0 0x7f91fc0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f9202780640 1 -- 172.21.15.31:0/1770649788 >> 172.21.15.31:0/1770649788 conn(0x7f91fc0d33b0 msgr2=0x7f91fc0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f9202780640 1 -- 172.21.15.31:0/1770649788 shutdown_connections 2023-11-26T07:18:29.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.144+0000 7f9202780640 1 -- 172.21.15.31:0/1770649788 wait complete. 2023-11-26T07:18:29.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9202780640 1 Processor -- start 2023-11-26T07:18:29.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9202780640 1 -- start start 2023-11-26T07:18:29.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9202780640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc0cead0 0x7f91fc0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9202780640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc0d1eb0 0x7f91fc0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9202780640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91fc15d7b0 0x7f91fc158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9202780640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0c8660 con 0x7f91fc0cead0 2023-11-26T07:18:29.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9202780640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0c8940 con 0x7f91fc0d1eb0 2023-11-26T07:18:29.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9202780640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc0c8c20 con 0x7f91fc15d7b0 2023-11-26T07:18:29.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc0cead0 0x7f91fc0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc0cead0 0x7f91fc0cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39858/0 (socket says 172.21.15.31:39858) 2023-11-26T07:18:29.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fbfff640 1 -- 172.21.15.31:0/2961168195 learned_addr learned my addr 172.21.15.31:0/2961168195 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:29.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fb7fe640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc0d1eb0 0x7f91fc0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9200d1d640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91fc15d7b0 0x7f91fc158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fbfff640 1 -- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91fc15d7b0 msgr2=0x7f91fc158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fbfff640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91fc15d7b0 0x7f91fc158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fbfff640 1 -- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc0d1eb0 msgr2=0x7f91fc0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fbfff640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc0d1eb0 0x7f91fc0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fbfff640 1 -- 172.21.15.31:0/2961168195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f4009a70 con 0x7f91fc0cead0 2023-11-26T07:18:29.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fb7fe640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc0d1eb0 0x7f91fc0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:29.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91fbfff640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc0cead0 0x7f91fc0cb4f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00c400 tx=0x7f91ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91f8ff9640 1 -- 172.21.15.31:0/2961168195 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec005cf0 con 0x7f91fc0cead0 2023-11-26T07:18:29.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91f8ff9640 1 -- 172.21.15.31:0/2961168195 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec005e50 con 0x7f91fc0cead0 2023-11-26T07:18:29.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f91f8ff9640 1 -- 172.21.15.31:0/2961168195 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec0072d0 con 0x7f91fc0cead0 2023-11-26T07:18:29.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9202780640 1 -- 172.21.15.31:0/2961168195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91fc0c8f00 con 0x7f91fc0cead0 2023-11-26T07:18:29.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.148+0000 7f9202780640 1 -- 172.21.15.31:0/2961168195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc0c9c40 con 0x7f91fc0cead0 2023-11-26T07:18:29.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.152+0000 7f9202780640 1 -- 172.21.15.31:0/2961168195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91fc04e850 con 0x7f91fc0cead0 2023-11-26T07:18:29.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.152+0000 7f91f8ff9640 1 -- 172.21.15.31:0/2961168195 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f91ec007430 con 0x7f91fc0cead0 2023-11-26T07:18:29.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.152+0000 7f91f8ff9640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91e8066470 0x7f91e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.152+0000 7f91f8ff9640 1 -- 172.21.15.31:0/2961168195 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91ec083350 con 0x7f91fc0cead0 2023-11-26T07:18:29.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.152+0000 7f91fb7fe640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91e8066470 0x7f91e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.156+0000 7f91fb7fe640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91e8066470 0x7f91e8068930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f91fc1f9430 tx=0x7f91f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.156+0000 7f91f8ff9640 1 -- 172.21.15.31:0/2961168195 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f91ec050870 con 0x7f91fc0cead0 2023-11-26T07:18:29.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.316+0000 7f9202780640 1 -- 172.21.15.31:0/2961168195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91fc0c93b0 con 0x7f91fc0cead0 2023-11-26T07:18:29.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.316+0000 7f91f8ff9640 1 -- 172.21.15.31:0/2961168195 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16485 (secure 0 0 0) 0x7f91ec053e90 con 0x7f91fc0cead0 2023-11-26T07:18:29.323 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:29.323 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:26.567047+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:22.773068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:23.967477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 -- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91e8066470 msgr2=0x7f91e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91e8066470 0x7f91e8068930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f91fc1f9430 tx=0x7f91f0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 -- 172.21.15.31:0/2961168195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc0cead0 msgr2=0x7f91fc0cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc0cead0 0x7f91fc0cb4f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00c400 tx=0x7f91ec00c8c0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 -- 172.21.15.31:0/2961168195 shutdown_connections 2023-11-26T07:18:29.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91e8066470 0x7f91e8068930 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91fc15d7b0 0x7f91fc158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91fc0d1eb0 0x7f91fc0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 --2- 172.21.15.31:0/2961168195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91fc0cead0 0x7f91fc0cb4f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 -- 172.21.15.31:0/2961168195 >> 172.21.15.31:0/2961168195 conn(0x7f91fc0d33b0 msgr2=0x7f91fc0cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 -- 172.21.15.31:0/2961168195 shutdown_connections 2023-11-26T07:18:29.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.320+0000 7f9202780640 1 -- 172.21.15.31:0/2961168195 wait complete. 2023-11-26T07:18:29.344 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:18:29.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.480+0000 7f08de994640 1 Processor -- start 2023-11-26T07:18:29.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.480+0000 7f08de994640 1 -- start start 2023-11-26T07:18:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08de994640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08d80d1b50 0x7f08d80700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08de994640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08d815c0b0 0x7f08d80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08de994640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d80d1f30 con 0x7f08d80cfe10 2023-11-26T07:18:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08de994640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d813e6f0 con 0x7f08d815c0b0 2023-11-26T07:18:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08de994640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d81407e0 con 0x7f08d80d1b50 2023-11-26T07:18:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08dcf31640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08d815c0b0 0x7f08d80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08dcf31640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08d815c0b0 0x7f08d80705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39870/0 (socket says 172.21.15.31:39870) 2023-11-26T07:18:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08dcf31640 1 -- 172.21.15.31:0/2925893040 learned_addr learned my addr 172.21.15.31:0/2925893040 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08d7fff640 1 --2- 172.21.15.31:0/2925893040 >> v2:172.21.15.97:3300/0 conn(0x7f08d80d1b50 0x7f08d80700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08dcf31640 1 -- 172.21.15.31:0/2925893040 >> v2:172.21.15.97:3300/0 conn(0x7f08d80d1b50 msgr2=0x7f08d80700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08dcf31640 1 --2- 172.21.15.31:0/2925893040 >> v2:172.21.15.97:3300/0 conn(0x7f08d80d1b50 0x7f08d80700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08dcf31640 1 -- 172.21.15.31:0/2925893040 >> v1:172.21.15.97:6789/0 conn(0x7f08d80cfe10 legacy=0x7f08d815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.484+0000 7f08dcf31640 1 -- 172.21.15.31:0/2925893040 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d815c6f0 con 0x7f08d815c0b0 2023-11-26T07:18:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08dcf31640 1 --2- 172.21.15.31:0/2925893040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d815c0b0 0x7f08d80705f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08d80d2320 tx=0x7f08d001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265b21aa389263c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08d6ffd640 1 -- 172.21.15.31:0/2925893040 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d00a5780 con 0x7f08d815c0b0 2023-11-26T07:18:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08d6ffd640 1 -- 172.21.15.31:0/2925893040 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d00a58e0 con 0x7f08d815c0b0 2023-11-26T07:18:29.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08d6ffd640 1 -- 172.21.15.31:0/2925893040 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d00a5c70 con 0x7f08d815c0b0 2023-11-26T07:18:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08de994640 1 -- 172.21.15.31:0/2925893040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d815c0b0 msgr2=0x7f08d80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08de994640 1 --2- 172.21.15.31:0/2925893040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d815c0b0 0x7f08d80705f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08d80d2320 tx=0x7f08d001c9d0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08de994640 1 -- 172.21.15.31:0/2925893040 shutdown_connections 2023-11-26T07:18:29.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08de994640 1 --2- 172.21.15.31:0/2925893040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d815c0b0 0x7f08d80705f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08de994640 1 --2- 172.21.15.31:0/2925893040 >> v2:172.21.15.97:3300/0 conn(0x7f08d80d1b50 0x7f08d80700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08de994640 1 -- 172.21.15.31:0/2925893040 >> 172.21.15.31:0/2925893040 conn(0x7f08d806e950 msgr2=0x7f08d806ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08de994640 1 -- 172.21.15.31:0/2925893040 shutdown_connections 2023-11-26T07:18:29.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08de994640 1 -- 172.21.15.31:0/2925893040 wait complete. 2023-11-26T07:18:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08de994640 1 Processor -- start 2023-11-26T07:18:29.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.488+0000 7f08de994640 1 -- start start 2023-11-26T07:18:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08de994640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cfe10 0x7f08d8158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08de994640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d80d1b50 0x7f08d81599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08de994640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d815c0b0 0x7f08d8159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08de994640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d8158cb0 con 0x7f08d80d1b50 2023-11-26T07:18:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08de994640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d80b5aa0 con 0x7f08d80cfe10 2023-11-26T07:18:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08de994640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d80b5d40 con 0x7f08d815c0b0 2023-11-26T07:18:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d80d1b50 0x7f08d81599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d80d1b50 0x7f08d81599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39878/0 (socket says 172.21.15.31:39878) 2023-11-26T07:18:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d77fe640 1 -- 172.21.15.31:0/4049610203 learned_addr learned my addr 172.21.15.31:0/4049610203 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d7fff640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cfe10 0x7f08d8158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08dcf31640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d815c0b0 0x7f08d8159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d77fe640 1 -- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d815c0b0 msgr2=0x7f08d8159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d77fe640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d815c0b0 0x7f08d8159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d77fe640 1 -- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cfe10 msgr2=0x7f08d8158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d77fe640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cfe10 0x7f08d8158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d77fe640 1 -- 172.21.15.31:0/4049610203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d0009b30 con 0x7f08d80d1b50 2023-11-26T07:18:29.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d77fe640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d80d1b50 0x7f08d81599a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00c4d0 tx=0x7f08cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d4ff9640 1 -- 172.21.15.31:0/4049610203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0052d0 con 0x7f08d80d1b50 2023-11-26T07:18:29.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d4ff9640 1 -- 172.21.15.31:0/4049610203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc005430 con 0x7f08d80d1b50 2023-11-26T07:18:29.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d4ff9640 1 -- 172.21.15.31:0/4049610203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc012830 con 0x7f08d80d1b50 2023-11-26T07:18:29.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08de994640 1 -- 172.21.15.31:0/4049610203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d81f66a0 con 0x7f08d80d1b50 2023-11-26T07:18:29.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08de994640 1 -- 172.21.15.31:0/4049610203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d806d320 con 0x7f08d80d1b50 2023-11-26T07:18:29.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08de994640 1 -- 172.21.15.31:0/4049610203 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d806d540 con 0x7f08d80d1b50 2023-11-26T07:18:29.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.492+0000 7f08d4ff9640 1 -- 172.21.15.31:0/4049610203 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f08cc012990 con 0x7f08d80d1b50 2023-11-26T07:18:29.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.496+0000 7f08d4ff9640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08c4066470 0x7f08c4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.496+0000 7f08d4ff9640 1 -- 172.21.15.31:0/4049610203 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08cc083990 con 0x7f08d80d1b50 2023-11-26T07:18:29.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.496+0000 7f08d7fff640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08c4066470 0x7f08c4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.496+0000 7f08d7fff640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08c4066470 0x7f08c4068930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f08c800b9d0 tx=0x7f08c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.500+0000 7f08d4ff9640 1 -- 172.21.15.31:0/4049610203 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f08cc050e10 con 0x7f08d80d1b50 2023-11-26T07:18:29.553 INFO:tasks.ceph.mds.a.smithi031.stderr:2023-11-26T07:18:29.548+0000 7f32fef3a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:18:29.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.656+0000 7f08de994640 1 -- 172.21.15.31:0/4049610203 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08d813f2b0 con 0x7f08c4066470 2023-11-26T07:18:29.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.664+0000 7f08d4ff9640 1 -- 172.21.15.31:0/4049610203 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283559 (secure 0 0 0) 0x7f08c800b670 con 0x7f08c4066470 2023-11-26T07:18:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:29.674 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:18:29.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 -- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08c4066470 msgr2=0x7f08c4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08c4066470 0x7f08c4068930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f08c800b9d0 tx=0x7f08c8008040 comp rx=0 tx=0).stop 2023-11-26T07:18:29.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 -- 172.21.15.31:0/4049610203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d80d1b50 msgr2=0x7f08d81599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d80d1b50 0x7f08d81599a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00c4d0 tx=0x7f08cc00c990 comp rx=0 tx=0).stop 2023-11-26T07:18:29.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 -- 172.21.15.31:0/4049610203 shutdown_connections 2023-11-26T07:18:29.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f08c4066470 0x7f08c4068930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d815c0b0 0x7f08d8159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d80d1b50 0x7f08d81599a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 --2- 172.21.15.31:0/4049610203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d80cfe10 0x7f08d8158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 -- 172.21.15.31:0/4049610203 >> 172.21.15.31:0/4049610203 conn(0x7f08d806e950 msgr2=0x7f08d815ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 -- 172.21.15.31:0/4049610203 shutdown_connections 2023-11-26T07:18:29.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.668+0000 7f08de994640 1 -- 172.21.15.31:0/4049610203 wait complete. 2023-11-26T07:18:29.693 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-11-26T07:18:28.319698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220028,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754692,"statfs":{"total":773094113280,"available":772868804608,"internally_reserved":0,"allocated":3956736,"data_stored":2240546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.761571"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574246+0000","last_change":"2023-11-26T07:18:24.574246+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574246+0000","last_undegraded":"2023-11-26T07:18:24.574246+0000","last_fullsized":"2023-11-26T07:18:24.574246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580477+0000","last_change":"2023-11-26T07:18:24.580477+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580477+0000","last_undegraded":"2023-11-26T07:18:24.580477+0000","last_fullsized":"2023-11-26T07:18:24.580477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570490+0000","last_change":"2023-11-26T07:18:24.570490+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570490+0000","last_undegraded":"2023-11-26T07:18:24.570490+0000","last_fullsized":"2023-11-26T07:18:24.570490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575149+0000","last_change":"2023-11-26T07:18:24.575149+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575149+0000","last_undegraded":"2023-11-26T07:18:24.575149+0000","last_fullsized":"2023-11-26T07:18:24.575149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572224+0000","last_change":"2023-11-26T07:18:24.572224+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572224+0000","last_undegraded":"2023-11-26T07:18:24.572224+0000","last_fullsized":"2023-11-26T07:18:24.572224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580212+0000","last_change":"2023-11-26T07:18:24.580212+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580212+0000","last_undegraded":"2023-11-26T07:18:24.580212+0000","last_fullsized":"2023-11-26T07:18:24.580212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.586557+0000","last_change":"2023-11-26T07:18:24.586557+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.586557+0000","last_undegraded":"2023-11-26T07:18:24.586557+0000","last_fullsized":"2023-11-26T07:18:24.586557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.584108+0000","last_change":"2023-11-26T07:18:24.584108+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.584108+0000","last_undegraded":"2023-11-26T07:18:24.584108+0000","last_fullsized":"2023-11-26T07:18:24.584108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575896+0000","last_change":"2023-11-26T07:18:24.575896+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575896+0000","last_undegraded":"2023-11-26T07:18:24.575896+0000","last_fullsized":"2023-11-26T07:18:24.575896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570993+0000","last_change":"2023-11-26T07:18:24.570993+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570993+0000","last_undegraded":"2023-11-26T07:18:24.570993+0000","last_fullsized":"2023-11-26T07:18:24.570993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583078+0000","last_change":"2023-11-26T07:18:24.583078+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583078+0000","last_undegraded":"2023-11-26T07:18:24.583078+0000","last_fullsized":"2023-11-26T07:18:24.583078+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571481+0000","last_change":"2023-11-26T07:18:24.571481+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571481+0000","last_undegraded":"2023-11-26T07:18:24.571481+0000","last_fullsized":"2023-11-26T07:18:24.571481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571703+0000","last_change":"2023-11-26T07:18:24.571703+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571703+0000","last_undegraded":"2023-11-26T07:18:24.571703+0000","last_fullsized":"2023-11-26T07:18:24.571703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.579692+0000","last_change":"2023-11-26T07:18:24.579692+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.579692+0000","last_undegraded":"2023-11-26T07:18:24.579692+0000","last_fullsized":"2023-11-26T07:18:24.579692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570094+0000","last_change":"2023-11-26T07:18:24.570094+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570094+0000","last_undegraded":"2023-11-26T07:18:24.570094+0000","last_fullsized":"2023-11-26T07:18:24.570094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580880+0000","last_change":"2023-11-26T07:18:24.580880+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580880+0000","last_undegraded":"2023-11-26T07:18:24.580880+0000","last_fullsized":"2023-11-26T07:18:24.580880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583314+0000","last_change":"2023-11-26T07:18:24.583314+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583314+0000","last_undegraded":"2023-11-26T07:18:24.583314+0000","last_fullsized":"2023-11-26T07:18:24.583314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.579945+0000","last_change":"2023-11-26T07:18:24.579945+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.579945+0000","last_undegraded":"2023-11-26T07:18:24.579945+0000","last_fullsized":"2023-11-26T07:18:24.579945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577346+0000","last_change":"2023-11-26T07:18:24.577346+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577346+0000","last_undegraded":"2023-11-26T07:18:24.577346+0000","last_fullsized":"2023-11-26T07:18:24.577346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571517+0000","last_change":"2023-11-26T07:18:24.571517+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571517+0000","last_undegraded":"2023-11-26T07:18:24.571517+0000","last_fullsized":"2023-11-26T07:18:24.571517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578313+0000","last_change":"2023-11-26T07:18:24.578313+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578313+0000","last_undegraded":"2023-11-26T07:18:24.578313+0000","last_fullsized":"2023-11-26T07:18:24.578313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578441+0000","last_change":"2023-11-26T07:18:24.578441+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578441+0000","last_undegraded":"2023-11-26T07:18:24.578441+0000","last_fullsized":"2023-11-26T07:18:24.578441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572722+0000","last_change":"2023-11-26T07:18:24.572722+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572722+0000","last_undegraded":"2023-11-26T07:18:24.572722+0000","last_fullsized":"2023-11-26T07:18:24.572722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570688+0000","last_change":"2023-11-26T07:18:24.570688+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570688+0000","last_undegraded":"2023-11-26T07:18:24.570688+0000","last_fullsized":"2023-11-26T07:18:24.570688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572193+0000","last_change":"2023-11-26T07:18:24.572193+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572193+0000","last_undegraded":"2023-11-26T07:18:24.572193+0000","last_fullsized":"2023-11-26T07:18:24.572193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577541+0000","last_change":"2023-11-26T07:18:24.577541+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577541+0000","last_undegraded":"2023-11-26T07:18:24.577541+0000","last_fullsized":"2023-11-26T07:18:24.577541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573219+0000","last_change":"2023-11-26T07:18:24.573219+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573219+0000","last_undegraded":"2023-11-26T07:18:24.573219+0000","last_fullsized":"2023-11-26T07:18:24.573219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570094+0000","last_change":"2023-11-26T07:18:24.570094+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570094+0000","last_undegraded":"2023-11-26T07:18:24.570094+0000","last_fullsized":"2023-11-26T07:18:24.570094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577765+0000","last_change":"2023-11-26T07:18:24.577765+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577765+0000","last_undegraded":"2023-11-26T07:18:24.577765+0000","last_fullsized":"2023-11-26T07:18:24.577765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.581657+0000","last_change":"2023-11-26T07:18:24.581657+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.581657+0000","last_undegraded":"2023-11-26T07:18:24.581657+0000","last_fullsized":"2023-11-26T07:18:24.581657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573470+0000","last_change":"2023-11-26T07:18:24.573470+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573470+0000","last_undegraded":"2023-11-26T07:18:24.573470+0000","last_fullsized":"2023-11-26T07:18:24.573470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583193+0000","last_change":"2023-11-26T07:18:24.583193+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583193+0000","last_undegraded":"2023-11-26T07:18:24.583193+0000","last_fullsized":"2023-11-26T07:18:24.583193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573749+0000","last_change":"2023-11-26T07:18:24.573749+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573749+0000","last_undegraded":"2023-11-26T07:18:24.573749+0000","last_fullsized":"2023-11-26T07:18:24.573749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.590273+0000","last_change":"2023-11-26T07:18:24.586219+0000","last_active":"2023-11-26T07:18:24.590273+0000","last_peered":"2023-11-26T07:18:24.590273+0000","last_clean":"2023-11-26T07:18:24.590273+0000","last_became_active":"2023-11-26T07:18:24.584015+0000","last_became_peered":"2023-11-26T07:18:24.584015+0000","last_unstale":"2023-11-26T07:18:24.590273+0000","last_undegraded":"2023-11-26T07:18:24.590273+0000","last_fullsized":"2023-11-26T07:18:24.590273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573126+0000","last_change":"2023-11-26T07:18:24.573126+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573126+0000","last_undegraded":"2023-11-26T07:18:24.573126+0000","last_fullsized":"2023-11-26T07:18:24.573126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597544+0000","last_change":"2023-11-26T07:18:24.592940+0000","last_active":"2023-11-26T07:18:24.597544+0000","last_peered":"2023-11-26T07:18:24.597544+0000","last_clean":"2023-11-26T07:18:24.597544+0000","last_became_active":"2023-11-26T07:18:24.592670+0000","last_became_peered":"2023-11-26T07:18:24.592670+0000","last_unstale":"2023-11-26T07:18:24.597544+0000","last_undegraded":"2023-11-26T07:18:24.597544+0000","last_fullsized":"2023-11-26T07:18:24.597544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577389+0000","last_change":"2023-11-26T07:18:24.577389+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577389+0000","last_undegraded":"2023-11-26T07:18:24.577389+0000","last_fullsized":"2023-11-26T07:18:24.577389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597822+0000","last_change":"2023-11-26T07:18:24.594822+0000","last_active":"2023-11-26T07:18:24.597822+0000","last_peered":"2023-11-26T07:18:24.597822+0000","last_clean":"2023-11-26T07:18:24.597822+0000","last_became_active":"2023-11-26T07:18:24.594466+0000","last_became_peered":"2023-11-26T07:18:24.594466+0000","last_unstale":"2023-11-26T07:18:24.597822+0000","last_undegraded":"2023-11-26T07:18:24.597822+0000","last_fullsized":"2023-11-26T07:18:24.597822+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:55:58.922003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572762+0000","last_change":"2023-11-26T07:18:24.572762+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572762+0000","last_undegraded":"2023-11-26T07:18:24.572762+0000","last_fullsized":"2023-11-26T07:18:24.572762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592346+0000","last_change":"2023-11-26T07:18:24.590286+0000","last_active":"2023-11-26T07:18:24.592346+0000","last_peered":"2023-11-26T07:18:24.592346+0000","last_clean":"2023-11-26T07:18:24.592346+0000","last_became_active":"2023-11-26T07:18:24.589675+0000","last_became_peered":"2023-11-26T07:18:24.589675+0000","last_unstale":"2023-11-26T07:18:24.592346+0000","last_undegraded":"2023-11-26T07:18:24.592346+0000","last_fullsized":"2023-11-26T07:18:24.592346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:39:04.751909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.581188+0000","last_change":"2023-11-26T07:18:24.581188+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.581188+0000","last_undegraded":"2023-11-26T07:18:24.581188+0000","last_fullsized":"2023-11-26T07:18:24.581188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591549+0000","last_change":"2023-11-26T07:18:24.588499+0000","last_active":"2023-11-26T07:18:24.591549+0000","last_peered":"2023-11-26T07:18:24.591549+0000","last_clean":"2023-11-26T07:18:24.591549+0000","last_became_active":"2023-11-26T07:18:24.588209+0000","last_became_peered":"2023-11-26T07:18:24.588209+0000","last_unstale":"2023-11-26T07:18:24.591549+0000","last_undegraded":"2023-11-26T07:18:24.591549+0000","last_fullsized":"2023-11-26T07:18:24.591549+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:24:01.800560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570150+0000","last_change":"2023-11-26T07:18:24.570150+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570150+0000","last_undegraded":"2023-11-26T07:18:24.570150+0000","last_fullsized":"2023-11-26T07:18:24.570150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594137+0000","last_change":"2023-11-26T07:18:24.593041+0000","last_active":"2023-11-26T07:18:24.594137+0000","last_peered":"2023-11-26T07:18:24.594137+0000","last_clean":"2023-11-26T07:18:24.594137+0000","last_became_active":"2023-11-26T07:18:24.592774+0000","last_became_peered":"2023-11-26T07:18:24.592774+0000","last_unstale":"2023-11-26T07:18:24.594137+0000","last_undegraded":"2023-11-26T07:18:24.594137+0000","last_fullsized":"2023-11-26T07:18:24.594137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:57.227910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575991+0000","last_change":"2023-11-26T07:18:24.575991+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575991+0000","last_undegraded":"2023-11-26T07:18:24.575991+0000","last_fullsized":"2023-11-26T07:18:24.575991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.588443+0000","last_change":"2023-11-26T07:18:24.586769+0000","last_active":"2023-11-26T07:18:24.588443+0000","last_peered":"2023-11-26T07:18:24.588443+0000","last_clean":"2023-11-26T07:18:24.588443+0000","last_became_active":"2023-11-26T07:18:24.586327+0000","last_became_peered":"2023-11-26T07:18:24.586327+0000","last_unstale":"2023-11-26T07:18:24.588443+0000","last_undegraded":"2023-11-26T07:18:24.588443+0000","last_fullsized":"2023-11-26T07:18:24.588443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:14:55.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574611+0000","last_change":"2023-11-26T07:18:24.574611+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574611+0000","last_undegraded":"2023-11-26T07:18:24.574611+0000","last_fullsized":"2023-11-26T07:18:24.574611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.586928+0000","last_change":"2023-11-26T07:18:24.581295+0000","last_active":"2023-11-26T07:18:24.586928+0000","last_peered":"2023-11-26T07:18:24.586928+0000","last_clean":"2023-11-26T07:18:24.586928+0000","last_became_active":"2023-11-26T07:18:24.580773+0000","last_became_peered":"2023-11-26T07:18:24.580773+0000","last_unstale":"2023-11-26T07:18:24.586928+0000","last_undegraded":"2023-11-26T07:18:24.586928+0000","last_fullsized":"2023-11-26T07:18:24.586928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:25.890181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570098+0000","last_change":"2023-11-26T07:18:24.570098+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570098+0000","last_undegraded":"2023-11-26T07:18:24.570098+0000","last_fullsized":"2023-11-26T07:18:24.570098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.596044+0000","last_change":"2023-11-26T07:18:24.592314+0000","last_active":"2023-11-26T07:18:24.596044+0000","last_peered":"2023-11-26T07:18:24.596044+0000","last_clean":"2023-11-26T07:18:24.596044+0000","last_became_active":"2023-11-26T07:18:24.591990+0000","last_became_peered":"2023-11-26T07:18:24.591990+0000","last_unstale":"2023-11-26T07:18:24.596044+0000","last_undegraded":"2023-11-26T07:18:24.596044+0000","last_fullsized":"2023-11-26T07:18:24.596044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:52:38.196386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577291+0000","last_change":"2023-11-26T07:18:24.577291+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577291+0000","last_undegraded":"2023-11-26T07:18:24.577291+0000","last_fullsized":"2023-11-26T07:18:24.577291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594545+0000","last_change":"2023-11-26T07:18:24.589024+0000","last_active":"2023-11-26T07:18:24.594545+0000","last_peered":"2023-11-26T07:18:24.594545+0000","last_clean":"2023-11-26T07:18:24.594545+0000","last_became_active":"2023-11-26T07:18:24.585478+0000","last_became_peered":"2023-11-26T07:18:24.585478+0000","last_unstale":"2023-11-26T07:18:24.594545+0000","last_undegraded":"2023-11-26T07:18:24.594545+0000","last_fullsized":"2023-11-26T07:18:24.594545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577248+0000","last_change":"2023-11-26T07:18:24.577248+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577248+0000","last_undegraded":"2023-11-26T07:18:24.577248+0000","last_fullsized":"2023-11-26T07:18:24.577248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594681+0000","last_change":"2023-11-26T07:18:24.592183+0000","last_active":"2023-11-26T07:18:24.594681+0000","last_peered":"2023-11-26T07:18:24.594681+0000","last_clean":"2023-11-26T07:18:24.594681+0000","last_became_active":"2023-11-26T07:18:24.591338+0000","last_became_peered":"2023-11-26T07:18:24.591338+0000","last_unstale":"2023-11-26T07:18:24.594681+0000","last_undegraded":"2023-11-26T07:18:24.594681+0000","last_fullsized":"2023-11-26T07:18:24.594681+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:53:07.256791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571005+0000","last_change":"2023-11-26T07:18:24.571005+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571005+0000","last_undegraded":"2023-11-26T07:18:24.571005+0000","last_fullsized":"2023-11-26T07:18:24.571005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597809+0000","last_change":"2023-11-26T07:18:24.594736+0000","last_active":"2023-11-26T07:18:24.597809+0000","last_peered":"2023-11-26T07:18:24.597809+0000","last_clean":"2023-11-26T07:18:24.597809+0000","last_became_active":"2023-11-26T07:18:24.594398+0000","last_became_peered":"2023-11-26T07:18:24.594398+0000","last_unstale":"2023-11-26T07:18:24.597809+0000","last_undegraded":"2023-11-26T07:18:24.597809+0000","last_fullsized":"2023-11-26T07:18:24.597809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572281+0000","last_change":"2023-11-26T07:18:24.572281+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572281+0000","last_undegraded":"2023-11-26T07:18:24.572281+0000","last_fullsized":"2023-11-26T07:18:24.572281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.588631+0000","last_change":"2023-11-26T07:18:24.583313+0000","last_active":"2023-11-26T07:18:24.588631+0000","last_peered":"2023-11-26T07:18:24.588631+0000","last_clean":"2023-11-26T07:18:24.588631+0000","last_became_active":"2023-11-26T07:18:24.583028+0000","last_became_peered":"2023-11-26T07:18:24.583028+0000","last_unstale":"2023-11-26T07:18:24.588631+0000","last_undegraded":"2023-11-26T07:18:24.588631+0000","last_fullsized":"2023-11-26T07:18:24.588631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:18:04.226180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.576716+0000","last_change":"2023-11-26T07:18:24.576716+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.576716+0000","last_undegraded":"2023-11-26T07:18:24.576716+0000","last_fullsized":"2023-11-26T07:18:24.576716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.585267+0000","last_change":"2023-11-26T07:18:24.581041+0000","last_active":"2023-11-26T07:18:24.585267+0000","last_peered":"2023-11-26T07:18:24.585267+0000","last_clean":"2023-11-26T07:18:24.585267+0000","last_became_active":"2023-11-26T07:18:24.580753+0000","last_became_peered":"2023-11-26T07:18:24.580753+0000","last_unstale":"2023-11-26T07:18:24.585267+0000","last_undegraded":"2023-11-26T07:18:24.585267+0000","last_fullsized":"2023-11-26T07:18:24.585267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.582736+0000","last_change":"2023-11-26T07:18:24.582736+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.582736+0000","last_undegraded":"2023-11-26T07:18:24.582736+0000","last_fullsized":"2023-11-26T07:18:24.582736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.600450+0000","last_change":"2023-11-26T07:18:24.596369+0000","last_active":"2023-11-26T07:18:24.600450+0000","last_peered":"2023-11-26T07:18:24.600450+0000","last_clean":"2023-11-26T07:18:24.600450+0000","last_became_active":"2023-11-26T07:18:24.595742+0000","last_became_peered":"2023-11-26T07:18:24.595742+0000","last_unstale":"2023-11-26T07:18:24.600450+0000","last_undegraded":"2023-11-26T07:18:24.600450+0000","last_fullsized":"2023-11-26T07:18:24.600450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:33.810259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572742+0000","last_change":"2023-11-26T07:18:24.572742+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572742+0000","last_undegraded":"2023-11-26T07:18:24.572742+0000","last_fullsized":"2023-11-26T07:18:24.572742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597987+0000","last_change":"2023-11-26T07:18:24.595332+0000","last_active":"2023-11-26T07:18:24.597987+0000","last_peered":"2023-11-26T07:18:24.597987+0000","last_clean":"2023-11-26T07:18:24.597987+0000","last_became_active":"2023-11-26T07:18:24.594874+0000","last_became_peered":"2023-11-26T07:18:24.594874+0000","last_unstale":"2023-11-26T07:18:24.597987+0000","last_undegraded":"2023-11-26T07:18:24.597987+0000","last_fullsized":"2023-11-26T07:18:24.597987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578391+0000","last_change":"2023-11-26T07:18:24.578391+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578391+0000","last_undegraded":"2023-11-26T07:18:24.578391+0000","last_fullsized":"2023-11-26T07:18:24.578391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.595660+0000","last_change":"2023-11-26T07:18:24.589241+0000","last_active":"2023-11-26T07:18:24.595660+0000","last_peered":"2023-11-26T07:18:24.595660+0000","last_clean":"2023-11-26T07:18:24.595660+0000","last_became_active":"2023-11-26T07:18:24.588980+0000","last_became_peered":"2023-11-26T07:18:24.588980+0000","last_unstale":"2023-11-26T07:18:24.595660+0000","last_undegraded":"2023-11-26T07:18:24.595660+0000","last_fullsized":"2023-11-26T07:18:24.595660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572015+0000","last_change":"2023-11-26T07:18:24.572015+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572015+0000","last_undegraded":"2023-11-26T07:18:24.572015+0000","last_fullsized":"2023-11-26T07:18:24.572015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591739+0000","last_change":"2023-11-26T07:18:24.589574+0000","last_active":"2023-11-26T07:18:24.591739+0000","last_peered":"2023-11-26T07:18:24.591739+0000","last_clean":"2023-11-26T07:18:24.591739+0000","last_became_active":"2023-11-26T07:18:24.589286+0000","last_became_peered":"2023-11-26T07:18:24.589286+0000","last_unstale":"2023-11-26T07:18:24.591739+0000","last_undegraded":"2023-11-26T07:18:24.591739+0000","last_fullsized":"2023-11-26T07:18:24.591739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:01:41.668016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574046+0000","last_change":"2023-11-26T07:18:24.574046+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574046+0000","last_undegraded":"2023-11-26T07:18:24.574046+0000","last_fullsized":"2023-11-26T07:18:24.574046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.593950+0000","last_change":"2023-11-26T07:18:24.590371+0000","last_active":"2023-11-26T07:18:24.593950+0000","last_peered":"2023-11-26T07:18:24.593950+0000","last_clean":"2023-11-26T07:18:24.593950+0000","last_became_active":"2023-11-26T07:18:24.590067+0000","last_became_peered":"2023-11-26T07:18:24.590067+0000","last_unstale":"2023-11-26T07:18:24.593950+0000","last_undegraded":"2023-11-26T07:18:24.593950+0000","last_fullsized":"2023-11-26T07:18:24.593950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:58.111738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571003+0000","last_change":"2023-11-26T07:18:24.571003+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571003+0000","last_undegraded":"2023-11-26T07:18:24.571003+0000","last_fullsized":"2023-11-26T07:18:24.571003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.589592+0000","last_change":"2023-11-26T07:18:24.585586+0000","last_active":"2023-11-26T07:18:24.589592+0000","last_peered":"2023-11-26T07:18:24.589592+0000","last_clean":"2023-11-26T07:18:24.589592+0000","last_became_active":"2023-11-26T07:18:24.585226+0000","last_became_peered":"2023-11-26T07:18:24.585226+0000","last_unstale":"2023-11-26T07:18:24.589592+0000","last_undegraded":"2023-11-26T07:18:24.589592+0000","last_fullsized":"2023-11-26T07:18:24.589592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:17.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573113+0000","last_change":"2023-11-26T07:18:24.573113+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573113+0000","last_undegraded":"2023-11-26T07:18:24.573113+0000","last_fullsized":"2023-11-26T07:18:24.573113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592523+0000","last_change":"2023-11-26T07:18:24.586275+0000","last_active":"2023-11-26T07:18:24.592523+0000","last_peered":"2023-11-26T07:18:24.592523+0000","last_clean":"2023-11-26T07:18:24.592523+0000","last_became_active":"2023-11-26T07:18:24.585844+0000","last_became_peered":"2023-11-26T07:18:24.585844+0000","last_unstale":"2023-11-26T07:18:24.592523+0000","last_undegraded":"2023-11-26T07:18:24.592523+0000","last_fullsized":"2023-11-26T07:18:24.592523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571207+0000","last_change":"2023-11-26T07:18:24.571207+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571207+0000","last_undegraded":"2023-11-26T07:18:24.571207+0000","last_fullsized":"2023-11-26T07:18:24.571207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.590359+0000","last_change":"2023-11-26T07:18:24.586954+0000","last_active":"2023-11-26T07:18:24.590359+0000","last_peered":"2023-11-26T07:18:24.590359+0000","last_clean":"2023-11-26T07:18:24.590359+0000","last_became_active":"2023-11-26T07:18:24.586667+0000","last_became_peered":"2023-11-26T07:18:24.586667+0000","last_unstale":"2023-11-26T07:18:24.590359+0000","last_undegraded":"2023-11-26T07:18:24.590359+0000","last_fullsized":"2023-11-26T07:18:24.590359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572705+0000","last_change":"2023-11-26T07:18:24.572705+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572705+0000","last_undegraded":"2023-11-26T07:18:24.572705+0000","last_fullsized":"2023-11-26T07:18:24.572705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594929+0000","last_change":"2023-11-26T07:18:24.593394+0000","last_active":"2023-11-26T07:18:24.594929+0000","last_peered":"2023-11-26T07:18:24.594929+0000","last_clean":"2023-11-26T07:18:24.594929+0000","last_became_active":"2023-11-26T07:18:24.593108+0000","last_became_peered":"2023-11-26T07:18:24.593108+0000","last_unstale":"2023-11-26T07:18:24.594929+0000","last_undegraded":"2023-11-26T07:18:24.594929+0000","last_fullsized":"2023-11-26T07:18:24.594929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:39:04.751909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.587065+0000","last_change":"2023-11-26T07:18:24.587065+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.587065+0000","last_undegraded":"2023-11-26T07:18:24.587065+0000","last_fullsized":"2023-11-26T07:18:24.587065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592134+0000","last_change":"2023-11-26T07:18:24.589292+0000","last_active":"2023-11-26T07:18:24.592134+0000","last_peered":"2023-11-26T07:18:24.592134+0000","last_clean":"2023-11-26T07:18:24.592134+0000","last_became_active":"2023-11-26T07:18:24.589014+0000","last_became_peered":"2023-11-26T07:18:24.589014+0000","last_unstale":"2023-11-26T07:18:24.592134+0000","last_undegraded":"2023-11-26T07:18:24.592134+0000","last_fullsized":"2023-11-26T07:18:24.592134+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:50.247115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571131+0000","last_change":"2023-11-26T07:18:24.571131+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571131+0000","last_undegraded":"2023-11-26T07:18:24.571131+0000","last_fullsized":"2023-11-26T07:18:24.571131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597278+0000","last_change":"2023-11-26T07:18:24.594254+0000","last_active":"2023-11-26T07:18:24.597278+0000","last_peered":"2023-11-26T07:18:24.597278+0000","last_clean":"2023-11-26T07:18:24.597278+0000","last_became_active":"2023-11-26T07:18:24.593932+0000","last_became_peered":"2023-11-26T07:18:24.593932+0000","last_unstale":"2023-11-26T07:18:24.597278+0000","last_undegraded":"2023-11-26T07:18:24.597278+0000","last_fullsized":"2023-11-26T07:18:24.597278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571451+0000","last_change":"2023-11-26T07:18:24.571451+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571451+0000","last_undegraded":"2023-11-26T07:18:24.571451+0000","last_fullsized":"2023-11-26T07:18:24.571451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.589132+0000","last_change":"2023-11-26T07:18:24.585493+0000","last_active":"2023-11-26T07:18:24.589132+0000","last_peered":"2023-11-26T07:18:24.589132+0000","last_clean":"2023-11-26T07:18:24.589132+0000","last_became_active":"2023-11-26T07:18:24.585141+0000","last_became_peered":"2023-11-26T07:18:24.585141+0000","last_unstale":"2023-11-26T07:18:24.589132+0000","last_undegraded":"2023-11-26T07:18:24.589132+0000","last_fullsized":"2023-11-26T07:18:24.589132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573208+0000","last_change":"2023-11-26T07:18:24.573208+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573208+0000","last_undegraded":"2023-11-26T07:18:24.573208+0000","last_fullsized":"2023-11-26T07:18:24.573208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.593649+0000","last_change":"2023-11-26T07:18:24.590400+0000","last_active":"2023-11-26T07:18:24.593649+0000","last_peered":"2023-11-26T07:18:24.593649+0000","last_clean":"2023-11-26T07:18:24.593649+0000","last_became_active":"2023-11-26T07:18:24.590083+0000","last_became_peered":"2023-11-26T07:18:24.590083+0000","last_unstale":"2023-11-26T07:18:24.593649+0000","last_undegraded":"2023-11-26T07:18:24.593649+0000","last_fullsized":"2023-11-26T07:18:24.593649+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573418+0000","last_change":"2023-11-26T07:18:24.573418+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573418+0000","last_undegraded":"2023-11-26T07:18:24.573418+0000","last_fullsized":"2023-11-26T07:18:24.573418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592541+0000","last_change":"2023-11-26T07:18:24.586412+0000","last_active":"2023-11-26T07:18:24.592541+0000","last_peered":"2023-11-26T07:18:24.592541+0000","last_clean":"2023-11-26T07:18:24.592541+0000","last_became_active":"2023-11-26T07:18:24.585386+0000","last_became_peered":"2023-11-26T07:18:24.585386+0000","last_unstale":"2023-11-26T07:18:24.592541+0000","last_undegraded":"2023-11-26T07:18:24.592541+0000","last_fullsized":"2023-11-26T07:18:24.592541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573139+0000","last_change":"2023-11-26T07:18:24.573139+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573139+0000","last_undegraded":"2023-11-26T07:18:24.573139+0000","last_fullsized":"2023-11-26T07:18:24.573139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.599965+0000","last_change":"2023-11-26T07:18:24.596404+0000","last_active":"2023-11-26T07:18:24.599965+0000","last_peered":"2023-11-26T07:18:24.599965+0000","last_clean":"2023-11-26T07:18:24.599965+0000","last_became_active":"2023-11-26T07:18:24.593975+0000","last_became_peered":"2023-11-26T07:18:24.593975+0000","last_unstale":"2023-11-26T07:18:24.599965+0000","last_undegraded":"2023-11-26T07:18:24.599965+0000","last_fullsized":"2023-11-26T07:18:24.599965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578998+0000","last_change":"2023-11-26T07:18:24.578998+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578998+0000","last_undegraded":"2023-11-26T07:18:24.578998+0000","last_fullsized":"2023-11-26T07:18:24.578998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591213+0000","last_change":"2023-11-26T07:18:24.588130+0000","last_active":"2023-11-26T07:18:24.591213+0000","last_peered":"2023-11-26T07:18:24.591213+0000","last_clean":"2023-11-26T07:18:24.591213+0000","last_became_active":"2023-11-26T07:18:24.586853+0000","last_became_peered":"2023-11-26T07:18:24.586853+0000","last_unstale":"2023-11-26T07:18:24.591213+0000","last_undegraded":"2023-11-26T07:18:24.591213+0000","last_fullsized":"2023-11-26T07:18:24.591213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:16:21.071603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578922+0000","last_change":"2023-11-26T07:18:24.578922+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578922+0000","last_undegraded":"2023-11-26T07:18:24.578922+0000","last_fullsized":"2023-11-26T07:18:24.578922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.596179+0000","last_change":"2023-11-26T07:18:24.593229+0000","last_active":"2023-11-26T07:18:24.596179+0000","last_peered":"2023-11-26T07:18:24.596179+0000","last_clean":"2023-11-26T07:18:24.596179+0000","last_became_active":"2023-11-26T07:18:24.592929+0000","last_became_peered":"2023-11-26T07:18:24.592929+0000","last_unstale":"2023-11-26T07:18:24.596179+0000","last_undegraded":"2023-11-26T07:18:24.596179+0000","last_fullsized":"2023-11-26T07:18:24.596179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.582610+0000","last_change":"2023-11-26T07:18:24.582610+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.582610+0000","last_undegraded":"2023-11-26T07:18:24.582610+0000","last_fullsized":"2023-11-26T07:18:24.582610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594151+0000","last_change":"2023-11-26T07:18:24.591226+0000","last_active":"2023-11-26T07:18:24.594151+0000","last_peered":"2023-11-26T07:18:24.594151+0000","last_clean":"2023-11-26T07:18:24.594151+0000","last_became_active":"2023-11-26T07:18:24.590903+0000","last_became_peered":"2023-11-26T07:18:24.590903+0000","last_unstale":"2023-11-26T07:18:24.594151+0000","last_undegraded":"2023-11-26T07:18:24.594151+0000","last_fullsized":"2023-11-26T07:18:24.594151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-26T07:18:23.565413+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:18:23.565413+0000","last_peered":"2023-11-26T07:18:23.565413+0000","last_clean":"2023-11-26T07:18:23.565413+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:18:23.565413+0000","last_undegraded":"2023-11-26T07:18:23.565413+0000","last_fullsized":"2023-11-26T07:18:23.565413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.746},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61499999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.83799999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.72499999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.71199999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.60099999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.51500000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.69899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.72299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.625}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54400000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.80900000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.76400000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.85399999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.874},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.79200000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.90400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.57499999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.53800000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.753}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.62},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.438},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67600000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82799999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.76500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.38400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.316},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.439}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":8.0990000000000002,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.45100000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":4.431,"5min":4.431,"15min":4.431},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.625}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.83399999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.68899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42799999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.72899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.53300000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.55200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":722736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.442}]},{"osd":2,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.49199999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73799999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.751},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:29.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:29.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f413e4e3640 1 Processor -- start 2023-11-26T07:18:29.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f413e4e3640 1 -- start start 2023-11-26T07:18:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f413e4e3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f413815d7b0 0x7f41380d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f413e4e3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41380cefd0 con 0x7f41380cead0 2023-11-26T07:18:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f413e4e3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41380cf2b0 con 0x7f41380d2080 2023-11-26T07:18:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f413e4e3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413806b3f0 con 0x7f413815d7b0 2023-11-26T07:18:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f41377fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f41380cead0 0x7f41380d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54486/0 (socket says 172.21.15.31:54486) 2023-11-26T07:18:29.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f41377fe640 1 -- 172.21.15.31:0/3972307007 learned_addr learned my addr 172.21.15.31:0/3972307007 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f4137fff640 1 --2- 172.21.15.31:0/3972307007 >> v2:172.21.15.97:3300/0 conn(0x7f413815d7b0 0x7f41380d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f4136ffd640 1 -- 172.21.15.31:0/3972307007 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226426078 0 0) 0x7f41380cefd0 con 0x7f41380cead0 2023-11-26T07:18:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f4136ffd640 1 -- 172.21.15.31:0/3972307007 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4120003530 con 0x7f41380cead0 2023-11-26T07:18:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f4136ffd640 1 -- 172.21.15.31:0/3972307007 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839142301 0 0) 0x7f41380cf2b0 con 0x7f41380d2080 2023-11-26T07:18:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f4136ffd640 1 -- 172.21.15.31:0/3972307007 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41380cefd0 con 0x7f41380d2080 2023-11-26T07:18:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f4137fff640 1 -- 172.21.15.31:0/3972307007 >> v1:172.21.15.97:6789/0 conn(0x7f41380d2080 legacy=0x7f413806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f4137fff640 1 -- 172.21.15.31:0/3972307007 >> v1:172.21.15.31:6789/0 conn(0x7f41380cead0 legacy=0x7f41380d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.872+0000 7f4137fff640 1 -- 172.21.15.31:0/3972307007 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413806b6d0 con 0x7f413815d7b0 2023-11-26T07:18:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f4137fff640 1 --2- 172.21.15.31:0/3972307007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413815d7b0 0x7f41380d10d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f41280097c0 tx=0x7f412801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc3f835b0a96eb4d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f4136ffd640 1 -- 172.21.15.31:0/3972307007 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128004230 con 0x7f413815d7b0 2023-11-26T07:18:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f4136ffd640 1 -- 172.21.15.31:0/3972307007 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41280a7050 con 0x7f413815d7b0 2023-11-26T07:18:29.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f4136ffd640 1 -- 172.21.15.31:0/3972307007 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128003620 con 0x7f413815d7b0 2023-11-26T07:18:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f413e4e3640 1 -- 172.21.15.31:0/3972307007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413815d7b0 msgr2=0x7f41380d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f413e4e3640 1 --2- 172.21.15.31:0/3972307007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413815d7b0 0x7f41380d10d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f41280097c0 tx=0x7f412801c330 comp rx=0 tx=0).stop 2023-11-26T07:18:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f413e4e3640 1 -- 172.21.15.31:0/3972307007 shutdown_connections 2023-11-26T07:18:29.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f413e4e3640 1 --2- 172.21.15.31:0/3972307007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f413815d7b0 0x7f41380d10d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f413e4e3640 1 -- 172.21.15.31:0/3972307007 >> 172.21.15.31:0/3972307007 conn(0x7f41380d33b0 msgr2=0x7f41380d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f413e4e3640 1 -- 172.21.15.31:0/3972307007 shutdown_connections 2023-11-26T07:18:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.876+0000 7f413e4e3640 1 -- 172.21.15.31:0/3972307007 wait complete. 2023-11-26T07:18:29.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413e4e3640 1 Processor -- start 2023-11-26T07:18:29.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413e4e3640 1 -- start start 2023-11-26T07:18:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413e4e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41380cead0 0x7f413815b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413e4e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f41380d2080 0x7f4138157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413e4e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f413815d7b0 0x7f4138157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413e4e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41381583a0 con 0x7f413815d7b0 2023-11-26T07:18:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413e4e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41380b6e80 con 0x7f41380cead0 2023-11-26T07:18:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413e4e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41380b7160 con 0x7f41380d2080 2023-11-26T07:18:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413ca80640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f413815d7b0 0x7f4138157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413ca80640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f413815d7b0 0x7f4138157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39888/0 (socket says 172.21.15.31:39888) 2023-11-26T07:18:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413ca80640 1 -- 172.21.15.31:0/2943367865 learned_addr learned my addr 172.21.15.31:0/2943367865 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f4137fff640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41380cead0 0x7f413815b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f41377fe640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f41380d2080 0x7f4138157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413ca80640 1 -- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f41380d2080 msgr2=0x7f4138157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413ca80640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f41380d2080 0x7f4138157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413ca80640 1 -- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41380cead0 msgr2=0x7f413815b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413ca80640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41380cead0 0x7f413815b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413ca80640 1 -- 172.21.15.31:0/2943367865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128009420 con 0x7f413815d7b0 2023-11-26T07:18:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f41377fe640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f41380d2080 0x7f4138157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413ca80640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f413815d7b0 0x7f4138157e60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f413000c730 tx=0x7f413000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f4134ff9640 1 -- 172.21.15.31:0/2943367865 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130003b80 con 0x7f413815d7b0 2023-11-26T07:18:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f4134ff9640 1 -- 172.21.15.31:0/2943367865 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4130003ce0 con 0x7f413815d7b0 2023-11-26T07:18:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413e4e3640 1 -- 172.21.15.31:0/2943367865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41380b7440 con 0x7f413815d7b0 2023-11-26T07:18:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f4134ff9640 1 -- 172.21.15.31:0/2943367865 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130005310 con 0x7f413815d7b0 2023-11-26T07:18:29.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.880+0000 7f413e4e3640 1 -- 172.21.15.31:0/2943367865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413815b990 con 0x7f413815d7b0 2023-11-26T07:18:29.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.884+0000 7f413e4e3640 1 -- 172.21.15.31:0/2943367865 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413804e850 con 0x7f413815d7b0 2023-11-26T07:18:29.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.884+0000 7f4134ff9640 1 -- 172.21.15.31:0/2943367865 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f4130005680 con 0x7f413815d7b0 2023-11-26T07:18:29.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.884+0000 7f4134ff9640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4124066470 0x7f4124068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:29.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.884+0000 7f4134ff9640 1 -- 172.21.15.31:0/2943367865 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4130083da0 con 0x7f413815d7b0 2023-11-26T07:18:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.884+0000 7f4137fff640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4124066470 0x7f4124068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:29.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.884+0000 7f4137fff640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4124066470 0x7f4124068930 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41280ad000 tx=0x7f41280a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:29.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:29.888+0000 7f4134ff9640 1 -- 172.21.15.31:0/2943367865 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4130051210 con 0x7f413815d7b0 2023-11-26T07:18:30.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.044+0000 7f413e4e3640 1 -- 172.21.15.31:0/2943367865 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f413815bbb0 con 0x7f4124066470 2023-11-26T07:18:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.052+0000 7f4134ff9640 1 -- 172.21.15.31:0/2943367865 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283559 (secure 0 0 0) 0x7f413815bbb0 con 0x7f4124066470 2023-11-26T07:18:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:30.061 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:18:30.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 -- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4124066470 msgr2=0x7f4124068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4124066470 0x7f4124068930 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41280ad000 tx=0x7f41280a6000 comp rx=0 tx=0).stop 2023-11-26T07:18:30.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 -- 172.21.15.31:0/2943367865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f413815d7b0 msgr2=0x7f4138157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f413815d7b0 0x7f4138157e60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f413000c730 tx=0x7f413000cbf0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 -- 172.21.15.31:0/2943367865 shutdown_connections 2023-11-26T07:18:30.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4124066470 0x7f4124068930 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f413815d7b0 0x7f4138157e60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f41380d2080 0x7f4138157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 --2- 172.21.15.31:0/2943367865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41380cead0 0x7f413815b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 -- 172.21.15.31:0/2943367865 >> 172.21.15.31:0/2943367865 conn(0x7f41380d33b0 msgr2=0x7f4138158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 -- 172.21.15.31:0/2943367865 shutdown_connections 2023-11-26T07:18:30.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.056+0000 7f413e4e3640 1 -- 172.21.15.31:0/2943367865 wait complete. 2023-11-26T07:18:30.079 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-11-26T07:18:28.319698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220028,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754692,"statfs":{"total":773094113280,"available":772868804608,"internally_reserved":0,"allocated":3956736,"data_stored":2240546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.761571"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574246+0000","last_change":"2023-11-26T07:18:24.574246+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574246+0000","last_undegraded":"2023-11-26T07:18:24.574246+0000","last_fullsized":"2023-11-26T07:18:24.574246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580477+0000","last_change":"2023-11-26T07:18:24.580477+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580477+0000","last_undegraded":"2023-11-26T07:18:24.580477+0000","last_fullsized":"2023-11-26T07:18:24.580477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570490+0000","last_change":"2023-11-26T07:18:24.570490+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570490+0000","last_undegraded":"2023-11-26T07:18:24.570490+0000","last_fullsized":"2023-11-26T07:18:24.570490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575149+0000","last_change":"2023-11-26T07:18:24.575149+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575149+0000","last_undegraded":"2023-11-26T07:18:24.575149+0000","last_fullsized":"2023-11-26T07:18:24.575149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572224+0000","last_change":"2023-11-26T07:18:24.572224+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572224+0000","last_undegraded":"2023-11-26T07:18:24.572224+0000","last_fullsized":"2023-11-26T07:18:24.572224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580212+0000","last_change":"2023-11-26T07:18:24.580212+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580212+0000","last_undegraded":"2023-11-26T07:18:24.580212+0000","last_fullsized":"2023-11-26T07:18:24.580212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.586557+0000","last_change":"2023-11-26T07:18:24.586557+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.586557+0000","last_undegraded":"2023-11-26T07:18:24.586557+0000","last_fullsized":"2023-11-26T07:18:24.586557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.584108+0000","last_change":"2023-11-26T07:18:24.584108+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.584108+0000","last_undegraded":"2023-11-26T07:18:24.584108+0000","last_fullsized":"2023-11-26T07:18:24.584108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575896+0000","last_change":"2023-11-26T07:18:24.575896+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575896+0000","last_undegraded":"2023-11-26T07:18:24.575896+0000","last_fullsized":"2023-11-26T07:18:24.575896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570993+0000","last_change":"2023-11-26T07:18:24.570993+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570993+0000","last_undegraded":"2023-11-26T07:18:24.570993+0000","last_fullsized":"2023-11-26T07:18:24.570993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583078+0000","last_change":"2023-11-26T07:18:24.583078+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583078+0000","last_undegraded":"2023-11-26T07:18:24.583078+0000","last_fullsized":"2023-11-26T07:18:24.583078+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571481+0000","last_change":"2023-11-26T07:18:24.571481+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571481+0000","last_undegraded":"2023-11-26T07:18:24.571481+0000","last_fullsized":"2023-11-26T07:18:24.571481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571703+0000","last_change":"2023-11-26T07:18:24.571703+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571703+0000","last_undegraded":"2023-11-26T07:18:24.571703+0000","last_fullsized":"2023-11-26T07:18:24.571703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.579692+0000","last_change":"2023-11-26T07:18:24.579692+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.579692+0000","last_undegraded":"2023-11-26T07:18:24.579692+0000","last_fullsized":"2023-11-26T07:18:24.579692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570094+0000","last_change":"2023-11-26T07:18:24.570094+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570094+0000","last_undegraded":"2023-11-26T07:18:24.570094+0000","last_fullsized":"2023-11-26T07:18:24.570094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580880+0000","last_change":"2023-11-26T07:18:24.580880+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580880+0000","last_undegraded":"2023-11-26T07:18:24.580880+0000","last_fullsized":"2023-11-26T07:18:24.580880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583314+0000","last_change":"2023-11-26T07:18:24.583314+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583314+0000","last_undegraded":"2023-11-26T07:18:24.583314+0000","last_fullsized":"2023-11-26T07:18:24.583314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.579945+0000","last_change":"2023-11-26T07:18:24.579945+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.579945+0000","last_undegraded":"2023-11-26T07:18:24.579945+0000","last_fullsized":"2023-11-26T07:18:24.579945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577346+0000","last_change":"2023-11-26T07:18:24.577346+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577346+0000","last_undegraded":"2023-11-26T07:18:24.577346+0000","last_fullsized":"2023-11-26T07:18:24.577346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571517+0000","last_change":"2023-11-26T07:18:24.571517+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571517+0000","last_undegraded":"2023-11-26T07:18:24.571517+0000","last_fullsized":"2023-11-26T07:18:24.571517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578313+0000","last_change":"2023-11-26T07:18:24.578313+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578313+0000","last_undegraded":"2023-11-26T07:18:24.578313+0000","last_fullsized":"2023-11-26T07:18:24.578313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578441+0000","last_change":"2023-11-26T07:18:24.578441+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578441+0000","last_undegraded":"2023-11-26T07:18:24.578441+0000","last_fullsized":"2023-11-26T07:18:24.578441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572722+0000","last_change":"2023-11-26T07:18:24.572722+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572722+0000","last_undegraded":"2023-11-26T07:18:24.572722+0000","last_fullsized":"2023-11-26T07:18:24.572722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570688+0000","last_change":"2023-11-26T07:18:24.570688+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570688+0000","last_undegraded":"2023-11-26T07:18:24.570688+0000","last_fullsized":"2023-11-26T07:18:24.570688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572193+0000","last_change":"2023-11-26T07:18:24.572193+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572193+0000","last_undegraded":"2023-11-26T07:18:24.572193+0000","last_fullsized":"2023-11-26T07:18:24.572193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577541+0000","last_change":"2023-11-26T07:18:24.577541+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577541+0000","last_undegraded":"2023-11-26T07:18:24.577541+0000","last_fullsized":"2023-11-26T07:18:24.577541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573219+0000","last_change":"2023-11-26T07:18:24.573219+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573219+0000","last_undegraded":"2023-11-26T07:18:24.573219+0000","last_fullsized":"2023-11-26T07:18:24.573219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570094+0000","last_change":"2023-11-26T07:18:24.570094+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570094+0000","last_undegraded":"2023-11-26T07:18:24.570094+0000","last_fullsized":"2023-11-26T07:18:24.570094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577765+0000","last_change":"2023-11-26T07:18:24.577765+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577765+0000","last_undegraded":"2023-11-26T07:18:24.577765+0000","last_fullsized":"2023-11-26T07:18:24.577765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.581657+0000","last_change":"2023-11-26T07:18:24.581657+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.581657+0000","last_undegraded":"2023-11-26T07:18:24.581657+0000","last_fullsized":"2023-11-26T07:18:24.581657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573470+0000","last_change":"2023-11-26T07:18:24.573470+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573470+0000","last_undegraded":"2023-11-26T07:18:24.573470+0000","last_fullsized":"2023-11-26T07:18:24.573470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583193+0000","last_change":"2023-11-26T07:18:24.583193+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583193+0000","last_undegraded":"2023-11-26T07:18:24.583193+0000","last_fullsized":"2023-11-26T07:18:24.583193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573749+0000","last_change":"2023-11-26T07:18:24.573749+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573749+0000","last_undegraded":"2023-11-26T07:18:24.573749+0000","last_fullsized":"2023-11-26T07:18:24.573749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.590273+0000","last_change":"2023-11-26T07:18:24.586219+0000","last_active":"2023-11-26T07:18:24.590273+0000","last_peered":"2023-11-26T07:18:24.590273+0000","last_clean":"2023-11-26T07:18:24.590273+0000","last_became_active":"2023-11-26T07:18:24.584015+0000","last_became_peered":"2023-11-26T07:18:24.584015+0000","last_unstale":"2023-11-26T07:18:24.590273+0000","last_undegraded":"2023-11-26T07:18:24.590273+0000","last_fullsized":"2023-11-26T07:18:24.590273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573126+0000","last_change":"2023-11-26T07:18:24.573126+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573126+0000","last_undegraded":"2023-11-26T07:18:24.573126+0000","last_fullsized":"2023-11-26T07:18:24.573126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597544+0000","last_change":"2023-11-26T07:18:24.592940+0000","last_active":"2023-11-26T07:18:24.597544+0000","last_peered":"2023-11-26T07:18:24.597544+0000","last_clean":"2023-11-26T07:18:24.597544+0000","last_became_active":"2023-11-26T07:18:24.592670+0000","last_became_peered":"2023-11-26T07:18:24.592670+0000","last_unstale":"2023-11-26T07:18:24.597544+0000","last_undegraded":"2023-11-26T07:18:24.597544+0000","last_fullsized":"2023-11-26T07:18:24.597544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577389+0000","last_change":"2023-11-26T07:18:24.577389+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577389+0000","last_undegraded":"2023-11-26T07:18:24.577389+0000","last_fullsized":"2023-11-26T07:18:24.577389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597822+0000","last_change":"2023-11-26T07:18:24.594822+0000","last_active":"2023-11-26T07:18:24.597822+0000","last_peered":"2023-11-26T07:18:24.597822+0000","last_clean":"2023-11-26T07:18:24.597822+0000","last_became_active":"2023-11-26T07:18:24.594466+0000","last_became_peered":"2023-11-26T07:18:24.594466+0000","last_unstale":"2023-11-26T07:18:24.597822+0000","last_undegraded":"2023-11-26T07:18:24.597822+0000","last_fullsized":"2023-11-26T07:18:24.597822+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:55:58.922003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572762+0000","last_change":"2023-11-26T07:18:24.572762+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572762+0000","last_undegraded":"2023-11-26T07:18:24.572762+0000","last_fullsized":"2023-11-26T07:18:24.572762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592346+0000","last_change":"2023-11-26T07:18:24.590286+0000","last_active":"2023-11-26T07:18:24.592346+0000","last_peered":"2023-11-26T07:18:24.592346+0000","last_clean":"2023-11-26T07:18:24.592346+0000","last_became_active":"2023-11-26T07:18:24.589675+0000","last_became_peered":"2023-11-26T07:18:24.589675+0000","last_unstale":"2023-11-26T07:18:24.592346+0000","last_undegraded":"2023-11-26T07:18:24.592346+0000","last_fullsized":"2023-11-26T07:18:24.592346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:39:04.751909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.581188+0000","last_change":"2023-11-26T07:18:24.581188+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.581188+0000","last_undegraded":"2023-11-26T07:18:24.581188+0000","last_fullsized":"2023-11-26T07:18:24.581188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591549+0000","last_change":"2023-11-26T07:18:24.588499+0000","last_active":"2023-11-26T07:18:24.591549+0000","last_peered":"2023-11-26T07:18:24.591549+0000","last_clean":"2023-11-26T07:18:24.591549+0000","last_became_active":"2023-11-26T07:18:24.588209+0000","last_became_peered":"2023-11-26T07:18:24.588209+0000","last_unstale":"2023-11-26T07:18:24.591549+0000","last_undegraded":"2023-11-26T07:18:24.591549+0000","last_fullsized":"2023-11-26T07:18:24.591549+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:24:01.800560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570150+0000","last_change":"2023-11-26T07:18:24.570150+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570150+0000","last_undegraded":"2023-11-26T07:18:24.570150+0000","last_fullsized":"2023-11-26T07:18:24.570150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594137+0000","last_change":"2023-11-26T07:18:24.593041+0000","last_active":"2023-11-26T07:18:24.594137+0000","last_peered":"2023-11-26T07:18:24.594137+0000","last_clean":"2023-11-26T07:18:24.594137+0000","last_became_active":"2023-11-26T07:18:24.592774+0000","last_became_peered":"2023-11-26T07:18:24.592774+0000","last_unstale":"2023-11-26T07:18:24.594137+0000","last_undegraded":"2023-11-26T07:18:24.594137+0000","last_fullsized":"2023-11-26T07:18:24.594137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:57.227910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575991+0000","last_change":"2023-11-26T07:18:24.575991+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575991+0000","last_undegraded":"2023-11-26T07:18:24.575991+0000","last_fullsized":"2023-11-26T07:18:24.575991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.588443+0000","last_change":"2023-11-26T07:18:24.586769+0000","last_active":"2023-11-26T07:18:24.588443+0000","last_peered":"2023-11-26T07:18:24.588443+0000","last_clean":"2023-11-26T07:18:24.588443+0000","last_became_active":"2023-11-26T07:18:24.586327+0000","last_became_peered":"2023-11-26T07:18:24.586327+0000","last_unstale":"2023-11-26T07:18:24.588443+0000","last_undegraded":"2023-11-26T07:18:24.588443+0000","last_fullsized":"2023-11-26T07:18:24.588443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:14:55.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574611+0000","last_change":"2023-11-26T07:18:24.574611+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574611+0000","last_undegraded":"2023-11-26T07:18:24.574611+0000","last_fullsized":"2023-11-26T07:18:24.574611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.586928+0000","last_change":"2023-11-26T07:18:24.581295+0000","last_active":"2023-11-26T07:18:24.586928+0000","last_peered":"2023-11-26T07:18:24.586928+0000","last_clean":"2023-11-26T07:18:24.586928+0000","last_became_active":"2023-11-26T07:18:24.580773+0000","last_became_peered":"2023-11-26T07:18:24.580773+0000","last_unstale":"2023-11-26T07:18:24.586928+0000","last_undegraded":"2023-11-26T07:18:24.586928+0000","last_fullsized":"2023-11-26T07:18:24.586928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:25.890181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570098+0000","last_change":"2023-11-26T07:18:24.570098+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570098+0000","last_undegraded":"2023-11-26T07:18:24.570098+0000","last_fullsized":"2023-11-26T07:18:24.570098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.596044+0000","last_change":"2023-11-26T07:18:24.592314+0000","last_active":"2023-11-26T07:18:24.596044+0000","last_peered":"2023-11-26T07:18:24.596044+0000","last_clean":"2023-11-26T07:18:24.596044+0000","last_became_active":"2023-11-26T07:18:24.591990+0000","last_became_peered":"2023-11-26T07:18:24.591990+0000","last_unstale":"2023-11-26T07:18:24.596044+0000","last_undegraded":"2023-11-26T07:18:24.596044+0000","last_fullsized":"2023-11-26T07:18:24.596044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:52:38.196386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577291+0000","last_change":"2023-11-26T07:18:24.577291+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577291+0000","last_undegraded":"2023-11-26T07:18:24.577291+0000","last_fullsized":"2023-11-26T07:18:24.577291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594545+0000","last_change":"2023-11-26T07:18:24.589024+0000","last_active":"2023-11-26T07:18:24.594545+0000","last_peered":"2023-11-26T07:18:24.594545+0000","last_clean":"2023-11-26T07:18:24.594545+0000","last_became_active":"2023-11-26T07:18:24.585478+0000","last_became_peered":"2023-11-26T07:18:24.585478+0000","last_unstale":"2023-11-26T07:18:24.594545+0000","last_undegraded":"2023-11-26T07:18:24.594545+0000","last_fullsized":"2023-11-26T07:18:24.594545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577248+0000","last_change":"2023-11-26T07:18:24.577248+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577248+0000","last_undegraded":"2023-11-26T07:18:24.577248+0000","last_fullsized":"2023-11-26T07:18:24.577248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594681+0000","last_change":"2023-11-26T07:18:24.592183+0000","last_active":"2023-11-26T07:18:24.594681+0000","last_peered":"2023-11-26T07:18:24.594681+0000","last_clean":"2023-11-26T07:18:24.594681+0000","last_became_active":"2023-11-26T07:18:24.591338+0000","last_became_peered":"2023-11-26T07:18:24.591338+0000","last_unstale":"2023-11-26T07:18:24.594681+0000","last_undegraded":"2023-11-26T07:18:24.594681+0000","last_fullsized":"2023-11-26T07:18:24.594681+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:53:07.256791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571005+0000","last_change":"2023-11-26T07:18:24.571005+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571005+0000","last_undegraded":"2023-11-26T07:18:24.571005+0000","last_fullsized":"2023-11-26T07:18:24.571005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597809+0000","last_change":"2023-11-26T07:18:24.594736+0000","last_active":"2023-11-26T07:18:24.597809+0000","last_peered":"2023-11-26T07:18:24.597809+0000","last_clean":"2023-11-26T07:18:24.597809+0000","last_became_active":"2023-11-26T07:18:24.594398+0000","last_became_peered":"2023-11-26T07:18:24.594398+0000","last_unstale":"2023-11-26T07:18:24.597809+0000","last_undegraded":"2023-11-26T07:18:24.597809+0000","last_fullsized":"2023-11-26T07:18:24.597809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572281+0000","last_change":"2023-11-26T07:18:24.572281+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572281+0000","last_undegraded":"2023-11-26T07:18:24.572281+0000","last_fullsized":"2023-11-26T07:18:24.572281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.588631+0000","last_change":"2023-11-26T07:18:24.583313+0000","last_active":"2023-11-26T07:18:24.588631+0000","last_peered":"2023-11-26T07:18:24.588631+0000","last_clean":"2023-11-26T07:18:24.588631+0000","last_became_active":"2023-11-26T07:18:24.583028+0000","last_became_peered":"2023-11-26T07:18:24.583028+0000","last_unstale":"2023-11-26T07:18:24.588631+0000","last_undegraded":"2023-11-26T07:18:24.588631+0000","last_fullsized":"2023-11-26T07:18:24.588631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:18:04.226180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.576716+0000","last_change":"2023-11-26T07:18:24.576716+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.576716+0000","last_undegraded":"2023-11-26T07:18:24.576716+0000","last_fullsized":"2023-11-26T07:18:24.576716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.585267+0000","last_change":"2023-11-26T07:18:24.581041+0000","last_active":"2023-11-26T07:18:24.585267+0000","last_peered":"2023-11-26T07:18:24.585267+0000","last_clean":"2023-11-26T07:18:24.585267+0000","last_became_active":"2023-11-26T07:18:24.580753+0000","last_became_peered":"2023-11-26T07:18:24.580753+0000","last_unstale":"2023-11-26T07:18:24.585267+0000","last_undegraded":"2023-11-26T07:18:24.585267+0000","last_fullsized":"2023-11-26T07:18:24.585267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.582736+0000","last_change":"2023-11-26T07:18:24.582736+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.582736+0000","last_undegraded":"2023-11-26T07:18:24.582736+0000","last_fullsized":"2023-11-26T07:18:24.582736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.600450+0000","last_change":"2023-11-26T07:18:24.596369+0000","last_active":"2023-11-26T07:18:24.600450+0000","last_peered":"2023-11-26T07:18:24.600450+0000","last_clean":"2023-11-26T07:18:24.600450+0000","last_became_active":"2023-11-26T07:18:24.595742+0000","last_became_peered":"2023-11-26T07:18:24.595742+0000","last_unstale":"2023-11-26T07:18:24.600450+0000","last_undegraded":"2023-11-26T07:18:24.600450+0000","last_fullsized":"2023-11-26T07:18:24.600450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:33.810259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572742+0000","last_change":"2023-11-26T07:18:24.572742+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572742+0000","last_undegraded":"2023-11-26T07:18:24.572742+0000","last_fullsized":"2023-11-26T07:18:24.572742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597987+0000","last_change":"2023-11-26T07:18:24.595332+0000","last_active":"2023-11-26T07:18:24.597987+0000","last_peered":"2023-11-26T07:18:24.597987+0000","last_clean":"2023-11-26T07:18:24.597987+0000","last_became_active":"2023-11-26T07:18:24.594874+0000","last_became_peered":"2023-11-26T07:18:24.594874+0000","last_unstale":"2023-11-26T07:18:24.597987+0000","last_undegraded":"2023-11-26T07:18:24.597987+0000","last_fullsized":"2023-11-26T07:18:24.597987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578391+0000","last_change":"2023-11-26T07:18:24.578391+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578391+0000","last_undegraded":"2023-11-26T07:18:24.578391+0000","last_fullsized":"2023-11-26T07:18:24.578391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.595660+0000","last_change":"2023-11-26T07:18:24.589241+0000","last_active":"2023-11-26T07:18:24.595660+0000","last_peered":"2023-11-26T07:18:24.595660+0000","last_clean":"2023-11-26T07:18:24.595660+0000","last_became_active":"2023-11-26T07:18:24.588980+0000","last_became_peered":"2023-11-26T07:18:24.588980+0000","last_unstale":"2023-11-26T07:18:24.595660+0000","last_undegraded":"2023-11-26T07:18:24.595660+0000","last_fullsized":"2023-11-26T07:18:24.595660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572015+0000","last_change":"2023-11-26T07:18:24.572015+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572015+0000","last_undegraded":"2023-11-26T07:18:24.572015+0000","last_fullsized":"2023-11-26T07:18:24.572015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591739+0000","last_change":"2023-11-26T07:18:24.589574+0000","last_active":"2023-11-26T07:18:24.591739+0000","last_peered":"2023-11-26T07:18:24.591739+0000","last_clean":"2023-11-26T07:18:24.591739+0000","last_became_active":"2023-11-26T07:18:24.589286+0000","last_became_peered":"2023-11-26T07:18:24.589286+0000","last_unstale":"2023-11-26T07:18:24.591739+0000","last_undegraded":"2023-11-26T07:18:24.591739+0000","last_fullsized":"2023-11-26T07:18:24.591739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:01:41.668016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574046+0000","last_change":"2023-11-26T07:18:24.574046+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574046+0000","last_undegraded":"2023-11-26T07:18:24.574046+0000","last_fullsized":"2023-11-26T07:18:24.574046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.593950+0000","last_change":"2023-11-26T07:18:24.590371+0000","last_active":"2023-11-26T07:18:24.593950+0000","last_peered":"2023-11-26T07:18:24.593950+0000","last_clean":"2023-11-26T07:18:24.593950+0000","last_became_active":"2023-11-26T07:18:24.590067+0000","last_became_peered":"2023-11-26T07:18:24.590067+0000","last_unstale":"2023-11-26T07:18:24.593950+0000","last_undegraded":"2023-11-26T07:18:24.593950+0000","last_fullsized":"2023-11-26T07:18:24.593950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:58.111738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571003+0000","last_change":"2023-11-26T07:18:24.571003+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571003+0000","last_undegraded":"2023-11-26T07:18:24.571003+0000","last_fullsized":"2023-11-26T07:18:24.571003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.589592+0000","last_change":"2023-11-26T07:18:24.585586+0000","last_active":"2023-11-26T07:18:24.589592+0000","last_peered":"2023-11-26T07:18:24.589592+0000","last_clean":"2023-11-26T07:18:24.589592+0000","last_became_active":"2023-11-26T07:18:24.585226+0000","last_became_peered":"2023-11-26T07:18:24.585226+0000","last_unstale":"2023-11-26T07:18:24.589592+0000","last_undegraded":"2023-11-26T07:18:24.589592+0000","last_fullsized":"2023-11-26T07:18:24.589592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:17.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573113+0000","last_change":"2023-11-26T07:18:24.573113+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573113+0000","last_undegraded":"2023-11-26T07:18:24.573113+0000","last_fullsized":"2023-11-26T07:18:24.573113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592523+0000","last_change":"2023-11-26T07:18:24.586275+0000","last_active":"2023-11-26T07:18:24.592523+0000","last_peered":"2023-11-26T07:18:24.592523+0000","last_clean":"2023-11-26T07:18:24.592523+0000","last_became_active":"2023-11-26T07:18:24.585844+0000","last_became_peered":"2023-11-26T07:18:24.585844+0000","last_unstale":"2023-11-26T07:18:24.592523+0000","last_undegraded":"2023-11-26T07:18:24.592523+0000","last_fullsized":"2023-11-26T07:18:24.592523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571207+0000","last_change":"2023-11-26T07:18:24.571207+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571207+0000","last_undegraded":"2023-11-26T07:18:24.571207+0000","last_fullsized":"2023-11-26T07:18:24.571207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.590359+0000","last_change":"2023-11-26T07:18:24.586954+0000","last_active":"2023-11-26T07:18:24.590359+0000","last_peered":"2023-11-26T07:18:24.590359+0000","last_clean":"2023-11-26T07:18:24.590359+0000","last_became_active":"2023-11-26T07:18:24.586667+0000","last_became_peered":"2023-11-26T07:18:24.586667+0000","last_unstale":"2023-11-26T07:18:24.590359+0000","last_undegraded":"2023-11-26T07:18:24.590359+0000","last_fullsized":"2023-11-26T07:18:24.590359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572705+0000","last_change":"2023-11-26T07:18:24.572705+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572705+0000","last_undegraded":"2023-11-26T07:18:24.572705+0000","last_fullsized":"2023-11-26T07:18:24.572705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594929+0000","last_change":"2023-11-26T07:18:24.593394+0000","last_active":"2023-11-26T07:18:24.594929+0000","last_peered":"2023-11-26T07:18:24.594929+0000","last_clean":"2023-11-26T07:18:24.594929+0000","last_became_active":"2023-11-26T07:18:24.593108+0000","last_became_peered":"2023-11-26T07:18:24.593108+0000","last_unstale":"2023-11-26T07:18:24.594929+0000","last_undegraded":"2023-11-26T07:18:24.594929+0000","last_fullsized":"2023-11-26T07:18:24.594929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:39:04.751909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.587065+0000","last_change":"2023-11-26T07:18:24.587065+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.587065+0000","last_undegraded":"2023-11-26T07:18:24.587065+0000","last_fullsized":"2023-11-26T07:18:24.587065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592134+0000","last_change":"2023-11-26T07:18:24.589292+0000","last_active":"2023-11-26T07:18:24.592134+0000","last_peered":"2023-11-26T07:18:24.592134+0000","last_clean":"2023-11-26T07:18:24.592134+0000","last_became_active":"2023-11-26T07:18:24.589014+0000","last_became_peered":"2023-11-26T07:18:24.589014+0000","last_unstale":"2023-11-26T07:18:24.592134+0000","last_undegraded":"2023-11-26T07:18:24.592134+0000","last_fullsized":"2023-11-26T07:18:24.592134+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:50.247115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571131+0000","last_change":"2023-11-26T07:18:24.571131+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571131+0000","last_undegraded":"2023-11-26T07:18:24.571131+0000","last_fullsized":"2023-11-26T07:18:24.571131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597278+0000","last_change":"2023-11-26T07:18:24.594254+0000","last_active":"2023-11-26T07:18:24.597278+0000","last_peered":"2023-11-26T07:18:24.597278+0000","last_clean":"2023-11-26T07:18:24.597278+0000","last_became_active":"2023-11-26T07:18:24.593932+0000","last_became_peered":"2023-11-26T07:18:24.593932+0000","last_unstale":"2023-11-26T07:18:24.597278+0000","last_undegraded":"2023-11-26T07:18:24.597278+0000","last_fullsized":"2023-11-26T07:18:24.597278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571451+0000","last_change":"2023-11-26T07:18:24.571451+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571451+0000","last_undegraded":"2023-11-26T07:18:24.571451+0000","last_fullsized":"2023-11-26T07:18:24.571451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.589132+0000","last_change":"2023-11-26T07:18:24.585493+0000","last_active":"2023-11-26T07:18:24.589132+0000","last_peered":"2023-11-26T07:18:24.589132+0000","last_clean":"2023-11-26T07:18:24.589132+0000","last_became_active":"2023-11-26T07:18:24.585141+0000","last_became_peered":"2023-11-26T07:18:24.585141+0000","last_unstale":"2023-11-26T07:18:24.589132+0000","last_undegraded":"2023-11-26T07:18:24.589132+0000","last_fullsized":"2023-11-26T07:18:24.589132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573208+0000","last_change":"2023-11-26T07:18:24.573208+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573208+0000","last_undegraded":"2023-11-26T07:18:24.573208+0000","last_fullsized":"2023-11-26T07:18:24.573208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.593649+0000","last_change":"2023-11-26T07:18:24.590400+0000","last_active":"2023-11-26T07:18:24.593649+0000","last_peered":"2023-11-26T07:18:24.593649+0000","last_clean":"2023-11-26T07:18:24.593649+0000","last_became_active":"2023-11-26T07:18:24.590083+0000","last_became_peered":"2023-11-26T07:18:24.590083+0000","last_unstale":"2023-11-26T07:18:24.593649+0000","last_undegraded":"2023-11-26T07:18:24.593649+0000","last_fullsized":"2023-11-26T07:18:24.593649+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573418+0000","last_change":"2023-11-26T07:18:24.573418+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573418+0000","last_undegraded":"2023-11-26T07:18:24.573418+0000","last_fullsized":"2023-11-26T07:18:24.573418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592541+0000","last_change":"2023-11-26T07:18:24.586412+0000","last_active":"2023-11-26T07:18:24.592541+0000","last_peered":"2023-11-26T07:18:24.592541+0000","last_clean":"2023-11-26T07:18:24.592541+0000","last_became_active":"2023-11-26T07:18:24.585386+0000","last_became_peered":"2023-11-26T07:18:24.585386+0000","last_unstale":"2023-11-26T07:18:24.592541+0000","last_undegraded":"2023-11-26T07:18:24.592541+0000","last_fullsized":"2023-11-26T07:18:24.592541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573139+0000","last_change":"2023-11-26T07:18:24.573139+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573139+0000","last_undegraded":"2023-11-26T07:18:24.573139+0000","last_fullsized":"2023-11-26T07:18:24.573139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.599965+0000","last_change":"2023-11-26T07:18:24.596404+0000","last_active":"2023-11-26T07:18:24.599965+0000","last_peered":"2023-11-26T07:18:24.599965+0000","last_clean":"2023-11-26T07:18:24.599965+0000","last_became_active":"2023-11-26T07:18:24.593975+0000","last_became_peered":"2023-11-26T07:18:24.593975+0000","last_unstale":"2023-11-26T07:18:24.599965+0000","last_undegraded":"2023-11-26T07:18:24.599965+0000","last_fullsized":"2023-11-26T07:18:24.599965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578998+0000","last_change":"2023-11-26T07:18:24.578998+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578998+0000","last_undegraded":"2023-11-26T07:18:24.578998+0000","last_fullsized":"2023-11-26T07:18:24.578998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591213+0000","last_change":"2023-11-26T07:18:24.588130+0000","last_active":"2023-11-26T07:18:24.591213+0000","last_peered":"2023-11-26T07:18:24.591213+0000","last_clean":"2023-11-26T07:18:24.591213+0000","last_became_active":"2023-11-26T07:18:24.586853+0000","last_became_peered":"2023-11-26T07:18:24.586853+0000","last_unstale":"2023-11-26T07:18:24.591213+0000","last_undegraded":"2023-11-26T07:18:24.591213+0000","last_fullsized":"2023-11-26T07:18:24.591213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:16:21.071603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578922+0000","last_change":"2023-11-26T07:18:24.578922+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578922+0000","last_undegraded":"2023-11-26T07:18:24.578922+0000","last_fullsized":"2023-11-26T07:18:24.578922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.596179+0000","last_change":"2023-11-26T07:18:24.593229+0000","last_active":"2023-11-26T07:18:24.596179+0000","last_peered":"2023-11-26T07:18:24.596179+0000","last_clean":"2023-11-26T07:18:24.596179+0000","last_became_active":"2023-11-26T07:18:24.592929+0000","last_became_peered":"2023-11-26T07:18:24.592929+0000","last_unstale":"2023-11-26T07:18:24.596179+0000","last_undegraded":"2023-11-26T07:18:24.596179+0000","last_fullsized":"2023-11-26T07:18:24.596179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.582610+0000","last_change":"2023-11-26T07:18:24.582610+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.582610+0000","last_undegraded":"2023-11-26T07:18:24.582610+0000","last_fullsized":"2023-11-26T07:18:24.582610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594151+0000","last_change":"2023-11-26T07:18:24.591226+0000","last_active":"2023-11-26T07:18:24.594151+0000","last_peered":"2023-11-26T07:18:24.594151+0000","last_clean":"2023-11-26T07:18:24.594151+0000","last_became_active":"2023-11-26T07:18:24.590903+0000","last_became_peered":"2023-11-26T07:18:24.590903+0000","last_unstale":"2023-11-26T07:18:24.594151+0000","last_undegraded":"2023-11-26T07:18:24.594151+0000","last_fullsized":"2023-11-26T07:18:24.594151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-26T07:18:23.565413+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:18:23.565413+0000","last_peered":"2023-11-26T07:18:23.565413+0000","last_clean":"2023-11-26T07:18:23.565413+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:18:23.565413+0000","last_undegraded":"2023-11-26T07:18:23.565413+0000","last_fullsized":"2023-11-26T07:18:23.565413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.746},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61499999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.83799999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.72499999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.71199999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.60099999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.51500000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.69899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.72299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.625}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54400000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.80900000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.76400000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.85399999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.874},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.79200000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.90400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.57499999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.53800000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.753}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.62},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.438},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67600000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82799999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.76500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.38400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.316},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.439}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":8.0990000000000002,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.45100000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":4.431,"5min":4.431,"15min":4.431},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.625}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.83399999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.68899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42799999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.72899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.53300000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.55200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":722736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.442}]},{"osd":2,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.49199999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73799999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.751},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:30.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:30.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:30.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:18:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.260+0000 7fd8cac4d640 1 Processor -- start 2023-11-26T07:18:30.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.260+0000 7fd8cac4d640 1 -- start start 2023-11-26T07:18:30.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8cac4d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd8c406dfd0 0x7fd8c406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8cac4d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c406e980 con 0x7fd8c40adcc0 2023-11-26T07:18:30.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8cac4d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c40ae7f0 con 0x7fd8c4143ae0 2023-11-26T07:18:30.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8cac4d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c4070670 con 0x7fd8c406dfd0 2023-11-26T07:18:30.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8c3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd8c406dfd0 0x7fd8c406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8c91ea640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd8c40adcc0 0x7fd8c40ae0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:54500/0 (socket says 172.21.15.31:54500) 2023-11-26T07:18:30.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8c91ea640 1 -- 172.21.15.31:0/1712469603 learned_addr learned my addr 172.21.15.31:0/1712469603 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:30.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8c3fff640 1 -- 172.21.15.31:0/1712469603 >> v1:172.21.15.97:6789/0 conn(0x7fd8c4143ae0 legacy=0x7fd8c40b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8c3fff640 1 -- 172.21.15.31:0/1712469603 >> v1:172.21.15.31:6789/0 conn(0x7fd8c40adcc0 legacy=0x7fd8c40ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8c3fff640 1 -- 172.21.15.31:0/1712469603 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c4070950 con 0x7fd8c406dfd0 2023-11-26T07:18:30.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.264+0000 7fd8c37fe640 1 -- 172.21.15.31:0/1712469603 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871961663 0 0) 0x7fd8c406e980 con 0x7fd8c40adcc0 2023-11-26T07:18:30.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8c3fff640 1 --2- 172.21.15.31:0/1712469603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c406dfd0 0x7fd8c406e440 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00097c0 tx=0x7fd8b001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25205f7e4c71a53 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8c37fe640 1 -- 172.21.15.31:0/1712469603 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00ab650 con 0x7fd8c406dfd0 2023-11-26T07:18:30.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8c37fe640 1 -- 172.21.15.31:0/1712469603 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b00ab7b0 con 0x7fd8c406dfd0 2023-11-26T07:18:30.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8c37fe640 1 -- 172.21.15.31:0/1712469603 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00abb40 con 0x7fd8c406dfd0 2023-11-26T07:18:30.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8cac4d640 1 -- 172.21.15.31:0/1712469603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c406dfd0 msgr2=0x7fd8c406e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8cac4d640 1 --2- 172.21.15.31:0/1712469603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c406dfd0 0x7fd8c406e440 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00097c0 tx=0x7fd8b001c450 comp rx=0 tx=0).stop 2023-11-26T07:18:30.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8cac4d640 1 -- 172.21.15.31:0/1712469603 shutdown_connections 2023-11-26T07:18:30.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8cac4d640 1 --2- 172.21.15.31:0/1712469603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c406dfd0 0x7fd8c406e440 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8cac4d640 1 -- 172.21.15.31:0/1712469603 >> 172.21.15.31:0/1712469603 conn(0x7fd8c40b0c40 msgr2=0x7fd8c40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8cac4d640 1 -- 172.21.15.31:0/1712469603 shutdown_connections 2023-11-26T07:18:30.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8cac4d640 1 -- 172.21.15.31:0/1712469603 wait complete. 2023-11-26T07:18:30.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8cac4d640 1 Processor -- start 2023-11-26T07:18:30.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.268+0000 7fd8cac4d640 1 -- start start 2023-11-26T07:18:30.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8cac4d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd8c406dfd0 0x7fd8c41408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8cac4d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c40adcc0 0x7fd8c4155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8cac4d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c4143ae0 0x7fd8c4139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8cac4d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4138330 con 0x7fd8c4143ae0 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8cac4d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4139b70 con 0x7fd8c40adcc0 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8cac4d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c4140f40 con 0x7fd8c406dfd0 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c91ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c4143ae0 0x7fd8c4139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c91ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c4143ae0 0x7fd8c4139060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39912/0 (socket says 172.21.15.31:39912) 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c91ea640 1 -- 172.21.15.31:0/3843857888 learned_addr learned my addr 172.21.15.31:0/3843857888 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c3fff640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c40adcc0 0x7fd8c4155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c89e9640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd8c406dfd0 0x7fd8c41408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c91ea640 1 -- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd8c406dfd0 msgr2=0x7fd8c41408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c91ea640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd8c406dfd0 0x7fd8c41408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c91ea640 1 -- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c40adcc0 msgr2=0x7fd8c4155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c91ea640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c40adcc0 0x7fd8c4155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c91ea640 1 -- 172.21.15.31:0/3843857888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0009420 con 0x7fd8c4143ae0 2023-11-26T07:18:30.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c91ea640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c4143ae0 0x7fd8c4139060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00c7a0 tx=0x7fd8bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c17fa640 1 -- 172.21.15.31:0/3843857888 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00a640 con 0x7fd8c4143ae0 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c17fa640 1 -- 172.21.15.31:0/3843857888 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc00a7a0 con 0x7fd8c4143ae0 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8cac4d640 1 -- 172.21.15.31:0/3843857888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c4138580 con 0x7fd8c4143ae0 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c17fa640 1 -- 172.21.15.31:0/3843857888 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc00abd0 con 0x7fd8c4143ae0 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8cac4d640 1 -- 172.21.15.31:0/3843857888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c4138b40 con 0x7fd8c4143ae0 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8cac4d640 1 -- 172.21.15.31:0/3843857888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c404e850 con 0x7fd8c4143ae0 2023-11-26T07:18:30.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.272+0000 7fd8c17fa640 1 -- 172.21.15.31:0/3843857888 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fd8bc00ad30 con 0x7fd8c4143ae0 2023-11-26T07:18:30.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.276+0000 7fd8c17fa640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8a8066470 0x7fd8a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.276+0000 7fd8c17fa640 1 -- 172.21.15.31:0/3843857888 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8bc0832c0 con 0x7fd8c4143ae0 2023-11-26T07:18:30.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.276+0000 7fd8c89e9640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8a8066470 0x7fd8a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.276+0000 7fd8c89e9640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8a8066470 0x7fd8a8068930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8007fd0 tx=0x7fd8b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.280+0000 7fd8c17fa640 1 -- 172.21.15.31:0/3843857888 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd8bc050880 con 0x7fd8c4143ae0 2023-11-26T07:18:30.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.432+0000 7fd8cac4d640 1 -- 172.21.15.31:0/3843857888 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8c4143ec0 con 0x7fd8a8066470 2023-11-26T07:18:30.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.440+0000 7fd8c17fa640 1 -- 172.21.15.31:0/3843857888 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283559 (secure 0 0 0) 0x7fd8b8001450 con 0x7fd8a8066470 2023-11-26T07:18:30.449 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:30.451 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:18:30.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 -- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8a8066470 msgr2=0x7fd8a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8a8066470 0x7fd8a8068930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8007fd0 tx=0x7fd8b8009040 comp rx=0 tx=0).stop 2023-11-26T07:18:30.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 -- 172.21.15.31:0/3843857888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c4143ae0 msgr2=0x7fd8c4139060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c4143ae0 0x7fd8c4139060 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00c7a0 tx=0x7fd8bc00cc60 comp rx=0 tx=0).stop 2023-11-26T07:18:30.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 -- 172.21.15.31:0/3843857888 shutdown_connections 2023-11-26T07:18:30.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8a8066470 0x7fd8a8068930 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c4143ae0 0x7fd8c4139060 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c40adcc0 0x7fd8c4155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 --2- 172.21.15.31:0/3843857888 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd8c406dfd0 0x7fd8c41408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 -- 172.21.15.31:0/3843857888 >> 172.21.15.31:0/3843857888 conn(0x7fd8c40b0c40 msgr2=0x7fd8c40ae0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 -- 172.21.15.31:0/3843857888 shutdown_connections 2023-11-26T07:18:30.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.444+0000 7fd8cac4d640 1 -- 172.21.15.31:0/3843857888 wait complete. 2023-11-26T07:18:30.469 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-11-26T07:18:30.320140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220028,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754692,"statfs":{"total":773094113280,"available":772868804608,"internally_reserved":0,"allocated":3956736,"data_stored":2240546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001415"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574246+0000","last_change":"2023-11-26T07:18:24.574246+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574246+0000","last_undegraded":"2023-11-26T07:18:24.574246+0000","last_fullsized":"2023-11-26T07:18:24.574246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580477+0000","last_change":"2023-11-26T07:18:24.580477+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580477+0000","last_undegraded":"2023-11-26T07:18:24.580477+0000","last_fullsized":"2023-11-26T07:18:24.580477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570490+0000","last_change":"2023-11-26T07:18:24.570490+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570490+0000","last_undegraded":"2023-11-26T07:18:24.570490+0000","last_fullsized":"2023-11-26T07:18:24.570490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575149+0000","last_change":"2023-11-26T07:18:24.575149+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575149+0000","last_undegraded":"2023-11-26T07:18:24.575149+0000","last_fullsized":"2023-11-26T07:18:24.575149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572224+0000","last_change":"2023-11-26T07:18:24.572224+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572224+0000","last_undegraded":"2023-11-26T07:18:24.572224+0000","last_fullsized":"2023-11-26T07:18:24.572224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580212+0000","last_change":"2023-11-26T07:18:24.580212+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580212+0000","last_undegraded":"2023-11-26T07:18:24.580212+0000","last_fullsized":"2023-11-26T07:18:24.580212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.586557+0000","last_change":"2023-11-26T07:18:24.586557+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.586557+0000","last_undegraded":"2023-11-26T07:18:24.586557+0000","last_fullsized":"2023-11-26T07:18:24.586557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.584108+0000","last_change":"2023-11-26T07:18:24.584108+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.584108+0000","last_undegraded":"2023-11-26T07:18:24.584108+0000","last_fullsized":"2023-11-26T07:18:24.584108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575896+0000","last_change":"2023-11-26T07:18:24.575896+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575896+0000","last_undegraded":"2023-11-26T07:18:24.575896+0000","last_fullsized":"2023-11-26T07:18:24.575896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570993+0000","last_change":"2023-11-26T07:18:24.570993+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570993+0000","last_undegraded":"2023-11-26T07:18:24.570993+0000","last_fullsized":"2023-11-26T07:18:24.570993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583078+0000","last_change":"2023-11-26T07:18:24.583078+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583078+0000","last_undegraded":"2023-11-26T07:18:24.583078+0000","last_fullsized":"2023-11-26T07:18:24.583078+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571481+0000","last_change":"2023-11-26T07:18:24.571481+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571481+0000","last_undegraded":"2023-11-26T07:18:24.571481+0000","last_fullsized":"2023-11-26T07:18:24.571481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571703+0000","last_change":"2023-11-26T07:18:24.571703+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571703+0000","last_undegraded":"2023-11-26T07:18:24.571703+0000","last_fullsized":"2023-11-26T07:18:24.571703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.579692+0000","last_change":"2023-11-26T07:18:24.579692+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.579692+0000","last_undegraded":"2023-11-26T07:18:24.579692+0000","last_fullsized":"2023-11-26T07:18:24.579692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570094+0000","last_change":"2023-11-26T07:18:24.570094+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570094+0000","last_undegraded":"2023-11-26T07:18:24.570094+0000","last_fullsized":"2023-11-26T07:18:24.570094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580880+0000","last_change":"2023-11-26T07:18:24.580880+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580880+0000","last_undegraded":"2023-11-26T07:18:24.580880+0000","last_fullsized":"2023-11-26T07:18:24.580880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583314+0000","last_change":"2023-11-26T07:18:24.583314+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583314+0000","last_undegraded":"2023-11-26T07:18:24.583314+0000","last_fullsized":"2023-11-26T07:18:24.583314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.579945+0000","last_change":"2023-11-26T07:18:24.579945+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.579945+0000","last_undegraded":"2023-11-26T07:18:24.579945+0000","last_fullsized":"2023-11-26T07:18:24.579945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577346+0000","last_change":"2023-11-26T07:18:24.577346+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577346+0000","last_undegraded":"2023-11-26T07:18:24.577346+0000","last_fullsized":"2023-11-26T07:18:24.577346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571517+0000","last_change":"2023-11-26T07:18:24.571517+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571517+0000","last_undegraded":"2023-11-26T07:18:24.571517+0000","last_fullsized":"2023-11-26T07:18:24.571517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578313+0000","last_change":"2023-11-26T07:18:24.578313+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578313+0000","last_undegraded":"2023-11-26T07:18:24.578313+0000","last_fullsized":"2023-11-26T07:18:24.578313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578441+0000","last_change":"2023-11-26T07:18:24.578441+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578441+0000","last_undegraded":"2023-11-26T07:18:24.578441+0000","last_fullsized":"2023-11-26T07:18:24.578441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572722+0000","last_change":"2023-11-26T07:18:24.572722+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572722+0000","last_undegraded":"2023-11-26T07:18:24.572722+0000","last_fullsized":"2023-11-26T07:18:24.572722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570688+0000","last_change":"2023-11-26T07:18:24.570688+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570688+0000","last_undegraded":"2023-11-26T07:18:24.570688+0000","last_fullsized":"2023-11-26T07:18:24.570688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572193+0000","last_change":"2023-11-26T07:18:24.572193+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572193+0000","last_undegraded":"2023-11-26T07:18:24.572193+0000","last_fullsized":"2023-11-26T07:18:24.572193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577541+0000","last_change":"2023-11-26T07:18:24.577541+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577541+0000","last_undegraded":"2023-11-26T07:18:24.577541+0000","last_fullsized":"2023-11-26T07:18:24.577541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573219+0000","last_change":"2023-11-26T07:18:24.573219+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573219+0000","last_undegraded":"2023-11-26T07:18:24.573219+0000","last_fullsized":"2023-11-26T07:18:24.573219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570094+0000","last_change":"2023-11-26T07:18:24.570094+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570094+0000","last_undegraded":"2023-11-26T07:18:24.570094+0000","last_fullsized":"2023-11-26T07:18:24.570094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577765+0000","last_change":"2023-11-26T07:18:24.577765+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577765+0000","last_undegraded":"2023-11-26T07:18:24.577765+0000","last_fullsized":"2023-11-26T07:18:24.577765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.581657+0000","last_change":"2023-11-26T07:18:24.581657+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.581657+0000","last_undegraded":"2023-11-26T07:18:24.581657+0000","last_fullsized":"2023-11-26T07:18:24.581657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573470+0000","last_change":"2023-11-26T07:18:24.573470+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573470+0000","last_undegraded":"2023-11-26T07:18:24.573470+0000","last_fullsized":"2023-11-26T07:18:24.573470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583193+0000","last_change":"2023-11-26T07:18:24.583193+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583193+0000","last_undegraded":"2023-11-26T07:18:24.583193+0000","last_fullsized":"2023-11-26T07:18:24.583193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573749+0000","last_change":"2023-11-26T07:18:24.573749+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573749+0000","last_undegraded":"2023-11-26T07:18:24.573749+0000","last_fullsized":"2023-11-26T07:18:24.573749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.590273+0000","last_change":"2023-11-26T07:18:24.586219+0000","last_active":"2023-11-26T07:18:24.590273+0000","last_peered":"2023-11-26T07:18:24.590273+0000","last_clean":"2023-11-26T07:18:24.590273+0000","last_became_active":"2023-11-26T07:18:24.584015+0000","last_became_peered":"2023-11-26T07:18:24.584015+0000","last_unstale":"2023-11-26T07:18:24.590273+0000","last_undegraded":"2023-11-26T07:18:24.590273+0000","last_fullsized":"2023-11-26T07:18:24.590273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573126+0000","last_change":"2023-11-26T07:18:24.573126+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573126+0000","last_undegraded":"2023-11-26T07:18:24.573126+0000","last_fullsized":"2023-11-26T07:18:24.573126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597544+0000","last_change":"2023-11-26T07:18:24.592940+0000","last_active":"2023-11-26T07:18:24.597544+0000","last_peered":"2023-11-26T07:18:24.597544+0000","last_clean":"2023-11-26T07:18:24.597544+0000","last_became_active":"2023-11-26T07:18:24.592670+0000","last_became_peered":"2023-11-26T07:18:24.592670+0000","last_unstale":"2023-11-26T07:18:24.597544+0000","last_undegraded":"2023-11-26T07:18:24.597544+0000","last_fullsized":"2023-11-26T07:18:24.597544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577389+0000","last_change":"2023-11-26T07:18:24.577389+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577389+0000","last_undegraded":"2023-11-26T07:18:24.577389+0000","last_fullsized":"2023-11-26T07:18:24.577389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597822+0000","last_change":"2023-11-26T07:18:24.594822+0000","last_active":"2023-11-26T07:18:24.597822+0000","last_peered":"2023-11-26T07:18:24.597822+0000","last_clean":"2023-11-26T07:18:24.597822+0000","last_became_active":"2023-11-26T07:18:24.594466+0000","last_became_peered":"2023-11-26T07:18:24.594466+0000","last_unstale":"2023-11-26T07:18:24.597822+0000","last_undegraded":"2023-11-26T07:18:24.597822+0000","last_fullsized":"2023-11-26T07:18:24.597822+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:55:58.922003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572762+0000","last_change":"2023-11-26T07:18:24.572762+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572762+0000","last_undegraded":"2023-11-26T07:18:24.572762+0000","last_fullsized":"2023-11-26T07:18:24.572762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592346+0000","last_change":"2023-11-26T07:18:24.590286+0000","last_active":"2023-11-26T07:18:24.592346+0000","last_peered":"2023-11-26T07:18:24.592346+0000","last_clean":"2023-11-26T07:18:24.592346+0000","last_became_active":"2023-11-26T07:18:24.589675+0000","last_became_peered":"2023-11-26T07:18:24.589675+0000","last_unstale":"2023-11-26T07:18:24.592346+0000","last_undegraded":"2023-11-26T07:18:24.592346+0000","last_fullsized":"2023-11-26T07:18:24.592346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:39:04.751909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.581188+0000","last_change":"2023-11-26T07:18:24.581188+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.581188+0000","last_undegraded":"2023-11-26T07:18:24.581188+0000","last_fullsized":"2023-11-26T07:18:24.581188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591549+0000","last_change":"2023-11-26T07:18:24.588499+0000","last_active":"2023-11-26T07:18:24.591549+0000","last_peered":"2023-11-26T07:18:24.591549+0000","last_clean":"2023-11-26T07:18:24.591549+0000","last_became_active":"2023-11-26T07:18:24.588209+0000","last_became_peered":"2023-11-26T07:18:24.588209+0000","last_unstale":"2023-11-26T07:18:24.591549+0000","last_undegraded":"2023-11-26T07:18:24.591549+0000","last_fullsized":"2023-11-26T07:18:24.591549+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:24:01.800560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570150+0000","last_change":"2023-11-26T07:18:24.570150+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570150+0000","last_undegraded":"2023-11-26T07:18:24.570150+0000","last_fullsized":"2023-11-26T07:18:24.570150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594137+0000","last_change":"2023-11-26T07:18:24.593041+0000","last_active":"2023-11-26T07:18:24.594137+0000","last_peered":"2023-11-26T07:18:24.594137+0000","last_clean":"2023-11-26T07:18:24.594137+0000","last_became_active":"2023-11-26T07:18:24.592774+0000","last_became_peered":"2023-11-26T07:18:24.592774+0000","last_unstale":"2023-11-26T07:18:24.594137+0000","last_undegraded":"2023-11-26T07:18:24.594137+0000","last_fullsized":"2023-11-26T07:18:24.594137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:57.227910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575991+0000","last_change":"2023-11-26T07:18:24.575991+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575991+0000","last_undegraded":"2023-11-26T07:18:24.575991+0000","last_fullsized":"2023-11-26T07:18:24.575991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.588443+0000","last_change":"2023-11-26T07:18:24.586769+0000","last_active":"2023-11-26T07:18:24.588443+0000","last_peered":"2023-11-26T07:18:24.588443+0000","last_clean":"2023-11-26T07:18:24.588443+0000","last_became_active":"2023-11-26T07:18:24.586327+0000","last_became_peered":"2023-11-26T07:18:24.586327+0000","last_unstale":"2023-11-26T07:18:24.588443+0000","last_undegraded":"2023-11-26T07:18:24.588443+0000","last_fullsized":"2023-11-26T07:18:24.588443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:14:55.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574611+0000","last_change":"2023-11-26T07:18:24.574611+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574611+0000","last_undegraded":"2023-11-26T07:18:24.574611+0000","last_fullsized":"2023-11-26T07:18:24.574611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.586928+0000","last_change":"2023-11-26T07:18:24.581295+0000","last_active":"2023-11-26T07:18:24.586928+0000","last_peered":"2023-11-26T07:18:24.586928+0000","last_clean":"2023-11-26T07:18:24.586928+0000","last_became_active":"2023-11-26T07:18:24.580773+0000","last_became_peered":"2023-11-26T07:18:24.580773+0000","last_unstale":"2023-11-26T07:18:24.586928+0000","last_undegraded":"2023-11-26T07:18:24.586928+0000","last_fullsized":"2023-11-26T07:18:24.586928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:25.890181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570098+0000","last_change":"2023-11-26T07:18:24.570098+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570098+0000","last_undegraded":"2023-11-26T07:18:24.570098+0000","last_fullsized":"2023-11-26T07:18:24.570098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.596044+0000","last_change":"2023-11-26T07:18:24.592314+0000","last_active":"2023-11-26T07:18:24.596044+0000","last_peered":"2023-11-26T07:18:24.596044+0000","last_clean":"2023-11-26T07:18:24.596044+0000","last_became_active":"2023-11-26T07:18:24.591990+0000","last_became_peered":"2023-11-26T07:18:24.591990+0000","last_unstale":"2023-11-26T07:18:24.596044+0000","last_undegraded":"2023-11-26T07:18:24.596044+0000","last_fullsized":"2023-11-26T07:18:24.596044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:52:38.196386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577291+0000","last_change":"2023-11-26T07:18:24.577291+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577291+0000","last_undegraded":"2023-11-26T07:18:24.577291+0000","last_fullsized":"2023-11-26T07:18:24.577291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594545+0000","last_change":"2023-11-26T07:18:24.589024+0000","last_active":"2023-11-26T07:18:24.594545+0000","last_peered":"2023-11-26T07:18:24.594545+0000","last_clean":"2023-11-26T07:18:24.594545+0000","last_became_active":"2023-11-26T07:18:24.585478+0000","last_became_peered":"2023-11-26T07:18:24.585478+0000","last_unstale":"2023-11-26T07:18:24.594545+0000","last_undegraded":"2023-11-26T07:18:24.594545+0000","last_fullsized":"2023-11-26T07:18:24.594545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577248+0000","last_change":"2023-11-26T07:18:24.577248+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577248+0000","last_undegraded":"2023-11-26T07:18:24.577248+0000","last_fullsized":"2023-11-26T07:18:24.577248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594681+0000","last_change":"2023-11-26T07:18:24.592183+0000","last_active":"2023-11-26T07:18:24.594681+0000","last_peered":"2023-11-26T07:18:24.594681+0000","last_clean":"2023-11-26T07:18:24.594681+0000","last_became_active":"2023-11-26T07:18:24.591338+0000","last_became_peered":"2023-11-26T07:18:24.591338+0000","last_unstale":"2023-11-26T07:18:24.594681+0000","last_undegraded":"2023-11-26T07:18:24.594681+0000","last_fullsized":"2023-11-26T07:18:24.594681+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:53:07.256791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571005+0000","last_change":"2023-11-26T07:18:24.571005+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571005+0000","last_undegraded":"2023-11-26T07:18:24.571005+0000","last_fullsized":"2023-11-26T07:18:24.571005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597809+0000","last_change":"2023-11-26T07:18:24.594736+0000","last_active":"2023-11-26T07:18:24.597809+0000","last_peered":"2023-11-26T07:18:24.597809+0000","last_clean":"2023-11-26T07:18:24.597809+0000","last_became_active":"2023-11-26T07:18:24.594398+0000","last_became_peered":"2023-11-26T07:18:24.594398+0000","last_unstale":"2023-11-26T07:18:24.597809+0000","last_undegraded":"2023-11-26T07:18:24.597809+0000","last_fullsized":"2023-11-26T07:18:24.597809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572281+0000","last_change":"2023-11-26T07:18:24.572281+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572281+0000","last_undegraded":"2023-11-26T07:18:24.572281+0000","last_fullsized":"2023-11-26T07:18:24.572281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.588631+0000","last_change":"2023-11-26T07:18:24.583313+0000","last_active":"2023-11-26T07:18:24.588631+0000","last_peered":"2023-11-26T07:18:24.588631+0000","last_clean":"2023-11-26T07:18:24.588631+0000","last_became_active":"2023-11-26T07:18:24.583028+0000","last_became_peered":"2023-11-26T07:18:24.583028+0000","last_unstale":"2023-11-26T07:18:24.588631+0000","last_undegraded":"2023-11-26T07:18:24.588631+0000","last_fullsized":"2023-11-26T07:18:24.588631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:18:04.226180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.576716+0000","last_change":"2023-11-26T07:18:24.576716+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.576716+0000","last_undegraded":"2023-11-26T07:18:24.576716+0000","last_fullsized":"2023-11-26T07:18:24.576716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.585267+0000","last_change":"2023-11-26T07:18:24.581041+0000","last_active":"2023-11-26T07:18:24.585267+0000","last_peered":"2023-11-26T07:18:24.585267+0000","last_clean":"2023-11-26T07:18:24.585267+0000","last_became_active":"2023-11-26T07:18:24.580753+0000","last_became_peered":"2023-11-26T07:18:24.580753+0000","last_unstale":"2023-11-26T07:18:24.585267+0000","last_undegraded":"2023-11-26T07:18:24.585267+0000","last_fullsized":"2023-11-26T07:18:24.585267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.582736+0000","last_change":"2023-11-26T07:18:24.582736+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.582736+0000","last_undegraded":"2023-11-26T07:18:24.582736+0000","last_fullsized":"2023-11-26T07:18:24.582736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.600450+0000","last_change":"2023-11-26T07:18:24.596369+0000","last_active":"2023-11-26T07:18:24.600450+0000","last_peered":"2023-11-26T07:18:24.600450+0000","last_clean":"2023-11-26T07:18:24.600450+0000","last_became_active":"2023-11-26T07:18:24.595742+0000","last_became_peered":"2023-11-26T07:18:24.595742+0000","last_unstale":"2023-11-26T07:18:24.600450+0000","last_undegraded":"2023-11-26T07:18:24.600450+0000","last_fullsized":"2023-11-26T07:18:24.600450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:33.810259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572742+0000","last_change":"2023-11-26T07:18:24.572742+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572742+0000","last_undegraded":"2023-11-26T07:18:24.572742+0000","last_fullsized":"2023-11-26T07:18:24.572742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597987+0000","last_change":"2023-11-26T07:18:24.595332+0000","last_active":"2023-11-26T07:18:24.597987+0000","last_peered":"2023-11-26T07:18:24.597987+0000","last_clean":"2023-11-26T07:18:24.597987+0000","last_became_active":"2023-11-26T07:18:24.594874+0000","last_became_peered":"2023-11-26T07:18:24.594874+0000","last_unstale":"2023-11-26T07:18:24.597987+0000","last_undegraded":"2023-11-26T07:18:24.597987+0000","last_fullsized":"2023-11-26T07:18:24.597987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578391+0000","last_change":"2023-11-26T07:18:24.578391+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578391+0000","last_undegraded":"2023-11-26T07:18:24.578391+0000","last_fullsized":"2023-11-26T07:18:24.578391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.595660+0000","last_change":"2023-11-26T07:18:24.589241+0000","last_active":"2023-11-26T07:18:24.595660+0000","last_peered":"2023-11-26T07:18:24.595660+0000","last_clean":"2023-11-26T07:18:24.595660+0000","last_became_active":"2023-11-26T07:18:24.588980+0000","last_became_peered":"2023-11-26T07:18:24.588980+0000","last_unstale":"2023-11-26T07:18:24.595660+0000","last_undegraded":"2023-11-26T07:18:24.595660+0000","last_fullsized":"2023-11-26T07:18:24.595660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572015+0000","last_change":"2023-11-26T07:18:24.572015+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572015+0000","last_undegraded":"2023-11-26T07:18:24.572015+0000","last_fullsized":"2023-11-26T07:18:24.572015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591739+0000","last_change":"2023-11-26T07:18:24.589574+0000","last_active":"2023-11-26T07:18:24.591739+0000","last_peered":"2023-11-26T07:18:24.591739+0000","last_clean":"2023-11-26T07:18:24.591739+0000","last_became_active":"2023-11-26T07:18:24.589286+0000","last_became_peered":"2023-11-26T07:18:24.589286+0000","last_unstale":"2023-11-26T07:18:24.591739+0000","last_undegraded":"2023-11-26T07:18:24.591739+0000","last_fullsized":"2023-11-26T07:18:24.591739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:01:41.668016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574046+0000","last_change":"2023-11-26T07:18:24.574046+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574046+0000","last_undegraded":"2023-11-26T07:18:24.574046+0000","last_fullsized":"2023-11-26T07:18:24.574046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.593950+0000","last_change":"2023-11-26T07:18:24.590371+0000","last_active":"2023-11-26T07:18:24.593950+0000","last_peered":"2023-11-26T07:18:24.593950+0000","last_clean":"2023-11-26T07:18:24.593950+0000","last_became_active":"2023-11-26T07:18:24.590067+0000","last_became_peered":"2023-11-26T07:18:24.590067+0000","last_unstale":"2023-11-26T07:18:24.593950+0000","last_undegraded":"2023-11-26T07:18:24.593950+0000","last_fullsized":"2023-11-26T07:18:24.593950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:58.111738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571003+0000","last_change":"2023-11-26T07:18:24.571003+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571003+0000","last_undegraded":"2023-11-26T07:18:24.571003+0000","last_fullsized":"2023-11-26T07:18:24.571003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.589592+0000","last_change":"2023-11-26T07:18:24.585586+0000","last_active":"2023-11-26T07:18:24.589592+0000","last_peered":"2023-11-26T07:18:24.589592+0000","last_clean":"2023-11-26T07:18:24.589592+0000","last_became_active":"2023-11-26T07:18:24.585226+0000","last_became_peered":"2023-11-26T07:18:24.585226+0000","last_unstale":"2023-11-26T07:18:24.589592+0000","last_undegraded":"2023-11-26T07:18:24.589592+0000","last_fullsized":"2023-11-26T07:18:24.589592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:17.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573113+0000","last_change":"2023-11-26T07:18:24.573113+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573113+0000","last_undegraded":"2023-11-26T07:18:24.573113+0000","last_fullsized":"2023-11-26T07:18:24.573113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592523+0000","last_change":"2023-11-26T07:18:24.586275+0000","last_active":"2023-11-26T07:18:24.592523+0000","last_peered":"2023-11-26T07:18:24.592523+0000","last_clean":"2023-11-26T07:18:24.592523+0000","last_became_active":"2023-11-26T07:18:24.585844+0000","last_became_peered":"2023-11-26T07:18:24.585844+0000","last_unstale":"2023-11-26T07:18:24.592523+0000","last_undegraded":"2023-11-26T07:18:24.592523+0000","last_fullsized":"2023-11-26T07:18:24.592523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571207+0000","last_change":"2023-11-26T07:18:24.571207+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571207+0000","last_undegraded":"2023-11-26T07:18:24.571207+0000","last_fullsized":"2023-11-26T07:18:24.571207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.590359+0000","last_change":"2023-11-26T07:18:24.586954+0000","last_active":"2023-11-26T07:18:24.590359+0000","last_peered":"2023-11-26T07:18:24.590359+0000","last_clean":"2023-11-26T07:18:24.590359+0000","last_became_active":"2023-11-26T07:18:24.586667+0000","last_became_peered":"2023-11-26T07:18:24.586667+0000","last_unstale":"2023-11-26T07:18:24.590359+0000","last_undegraded":"2023-11-26T07:18:24.590359+0000","last_fullsized":"2023-11-26T07:18:24.590359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572705+0000","last_change":"2023-11-26T07:18:24.572705+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572705+0000","last_undegraded":"2023-11-26T07:18:24.572705+0000","last_fullsized":"2023-11-26T07:18:24.572705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594929+0000","last_change":"2023-11-26T07:18:24.593394+0000","last_active":"2023-11-26T07:18:24.594929+0000","last_peered":"2023-11-26T07:18:24.594929+0000","last_clean":"2023-11-26T07:18:24.594929+0000","last_became_active":"2023-11-26T07:18:24.593108+0000","last_became_peered":"2023-11-26T07:18:24.593108+0000","last_unstale":"2023-11-26T07:18:24.594929+0000","last_undegraded":"2023-11-26T07:18:24.594929+0000","last_fullsized":"2023-11-26T07:18:24.594929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:39:04.751909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.587065+0000","last_change":"2023-11-26T07:18:24.587065+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.587065+0000","last_undegraded":"2023-11-26T07:18:24.587065+0000","last_fullsized":"2023-11-26T07:18:24.587065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592134+0000","last_change":"2023-11-26T07:18:24.589292+0000","last_active":"2023-11-26T07:18:24.592134+0000","last_peered":"2023-11-26T07:18:24.592134+0000","last_clean":"2023-11-26T07:18:24.592134+0000","last_became_active":"2023-11-26T07:18:24.589014+0000","last_became_peered":"2023-11-26T07:18:24.589014+0000","last_unstale":"2023-11-26T07:18:24.592134+0000","last_undegraded":"2023-11-26T07:18:24.592134+0000","last_fullsized":"2023-11-26T07:18:24.592134+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:50.247115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571131+0000","last_change":"2023-11-26T07:18:24.571131+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571131+0000","last_undegraded":"2023-11-26T07:18:24.571131+0000","last_fullsized":"2023-11-26T07:18:24.571131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597278+0000","last_change":"2023-11-26T07:18:24.594254+0000","last_active":"2023-11-26T07:18:24.597278+0000","last_peered":"2023-11-26T07:18:24.597278+0000","last_clean":"2023-11-26T07:18:24.597278+0000","last_became_active":"2023-11-26T07:18:24.593932+0000","last_became_peered":"2023-11-26T07:18:24.593932+0000","last_unstale":"2023-11-26T07:18:24.597278+0000","last_undegraded":"2023-11-26T07:18:24.597278+0000","last_fullsized":"2023-11-26T07:18:24.597278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571451+0000","last_change":"2023-11-26T07:18:24.571451+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571451+0000","last_undegraded":"2023-11-26T07:18:24.571451+0000","last_fullsized":"2023-11-26T07:18:24.571451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.589132+0000","last_change":"2023-11-26T07:18:24.585493+0000","last_active":"2023-11-26T07:18:24.589132+0000","last_peered":"2023-11-26T07:18:24.589132+0000","last_clean":"2023-11-26T07:18:24.589132+0000","last_became_active":"2023-11-26T07:18:24.585141+0000","last_became_peered":"2023-11-26T07:18:24.585141+0000","last_unstale":"2023-11-26T07:18:24.589132+0000","last_undegraded":"2023-11-26T07:18:24.589132+0000","last_fullsized":"2023-11-26T07:18:24.589132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573208+0000","last_change":"2023-11-26T07:18:24.573208+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573208+0000","last_undegraded":"2023-11-26T07:18:24.573208+0000","last_fullsized":"2023-11-26T07:18:24.573208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.593649+0000","last_change":"2023-11-26T07:18:24.590400+0000","last_active":"2023-11-26T07:18:24.593649+0000","last_peered":"2023-11-26T07:18:24.593649+0000","last_clean":"2023-11-26T07:18:24.593649+0000","last_became_active":"2023-11-26T07:18:24.590083+0000","last_became_peered":"2023-11-26T07:18:24.590083+0000","last_unstale":"2023-11-26T07:18:24.593649+0000","last_undegraded":"2023-11-26T07:18:24.593649+0000","last_fullsized":"2023-11-26T07:18:24.593649+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573418+0000","last_change":"2023-11-26T07:18:24.573418+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573418+0000","last_undegraded":"2023-11-26T07:18:24.573418+0000","last_fullsized":"2023-11-26T07:18:24.573418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592541+0000","last_change":"2023-11-26T07:18:24.586412+0000","last_active":"2023-11-26T07:18:24.592541+0000","last_peered":"2023-11-26T07:18:24.592541+0000","last_clean":"2023-11-26T07:18:24.592541+0000","last_became_active":"2023-11-26T07:18:24.585386+0000","last_became_peered":"2023-11-26T07:18:24.585386+0000","last_unstale":"2023-11-26T07:18:24.592541+0000","last_undegraded":"2023-11-26T07:18:24.592541+0000","last_fullsized":"2023-11-26T07:18:24.592541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573139+0000","last_change":"2023-11-26T07:18:24.573139+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573139+0000","last_undegraded":"2023-11-26T07:18:24.573139+0000","last_fullsized":"2023-11-26T07:18:24.573139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.599965+0000","last_change":"2023-11-26T07:18:24.596404+0000","last_active":"2023-11-26T07:18:24.599965+0000","last_peered":"2023-11-26T07:18:24.599965+0000","last_clean":"2023-11-26T07:18:24.599965+0000","last_became_active":"2023-11-26T07:18:24.593975+0000","last_became_peered":"2023-11-26T07:18:24.593975+0000","last_unstale":"2023-11-26T07:18:24.599965+0000","last_undegraded":"2023-11-26T07:18:24.599965+0000","last_fullsized":"2023-11-26T07:18:24.599965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578998+0000","last_change":"2023-11-26T07:18:24.578998+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578998+0000","last_undegraded":"2023-11-26T07:18:24.578998+0000","last_fullsized":"2023-11-26T07:18:24.578998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591213+0000","last_change":"2023-11-26T07:18:24.588130+0000","last_active":"2023-11-26T07:18:24.591213+0000","last_peered":"2023-11-26T07:18:24.591213+0000","last_clean":"2023-11-26T07:18:24.591213+0000","last_became_active":"2023-11-26T07:18:24.586853+0000","last_became_peered":"2023-11-26T07:18:24.586853+0000","last_unstale":"2023-11-26T07:18:24.591213+0000","last_undegraded":"2023-11-26T07:18:24.591213+0000","last_fullsized":"2023-11-26T07:18:24.591213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:16:21.071603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578922+0000","last_change":"2023-11-26T07:18:24.578922+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578922+0000","last_undegraded":"2023-11-26T07:18:24.578922+0000","last_fullsized":"2023-11-26T07:18:24.578922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.596179+0000","last_change":"2023-11-26T07:18:24.593229+0000","last_active":"2023-11-26T07:18:24.596179+0000","last_peered":"2023-11-26T07:18:24.596179+0000","last_clean":"2023-11-26T07:18:24.596179+0000","last_became_active":"2023-11-26T07:18:24.592929+0000","last_became_peered":"2023-11-26T07:18:24.592929+0000","last_unstale":"2023-11-26T07:18:24.596179+0000","last_undegraded":"2023-11-26T07:18:24.596179+0000","last_fullsized":"2023-11-26T07:18:24.596179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.582610+0000","last_change":"2023-11-26T07:18:24.582610+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.582610+0000","last_undegraded":"2023-11-26T07:18:24.582610+0000","last_fullsized":"2023-11-26T07:18:24.582610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594151+0000","last_change":"2023-11-26T07:18:24.591226+0000","last_active":"2023-11-26T07:18:24.594151+0000","last_peered":"2023-11-26T07:18:24.594151+0000","last_clean":"2023-11-26T07:18:24.594151+0000","last_became_active":"2023-11-26T07:18:24.590903+0000","last_became_peered":"2023-11-26T07:18:24.590903+0000","last_unstale":"2023-11-26T07:18:24.594151+0000","last_undegraded":"2023-11-26T07:18:24.594151+0000","last_fullsized":"2023-11-26T07:18:24.594151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-26T07:18:23.565413+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:18:23.565413+0000","last_peered":"2023-11-26T07:18:23.565413+0000","last_clean":"2023-11-26T07:18:23.565413+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:18:23.565413+0000","last_undegraded":"2023-11-26T07:18:23.565413+0000","last_fullsized":"2023-11-26T07:18:23.565413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.746},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61499999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.83799999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.72499999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.71199999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.60099999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.51500000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.69899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.72299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.625}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54400000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.80900000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.76400000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.85399999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.874},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.79200000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.90400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.57499999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.53800000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.753}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.62},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.438},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67600000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82799999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.76500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.38400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.316},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.439}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":8.0990000000000002,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.45100000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":4.431,"5min":4.431,"15min":4.431},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.625}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.83399999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.68899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42799999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.72899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.53300000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.55200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":722736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.442}]},{"osd":2,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.49199999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73799999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.751},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:30.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-11-26T07:18:30.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.574246+0000', 'last_change': '2023-11-26T07:18:24.574246+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.574246+0000', 'last_undegraded': '2023-11-26T07:18:24.574246+0000', 'last_fullsized': '2023-11-26T07:18:24.574246+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:18:30.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-11-26T07:18:30.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.580477+0000', 'last_change': '2023-11-26T07:18:24.580477+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.580477+0000', 'last_undegraded': '2023-11-26T07:18:24.580477+0000', 'last_fullsized': '2023-11-26T07:18:24.580477+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:30.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-11-26T07:18:30.474 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.570490+0000', 'last_change': '2023-11-26T07:18:24.570490+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.570490+0000', 'last_undegraded': '2023-11-26T07:18:24.570490+0000', 'last_fullsized': '2023-11-26T07:18:24.570490+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:30.474 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-11-26T07:18:30.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.575149+0000', 'last_change': '2023-11-26T07:18:24.575149+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.575149+0000', 'last_undegraded': '2023-11-26T07:18:24.575149+0000', 'last_fullsized': '2023-11-26T07:18:24.575149+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:30.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-11-26T07:18:30.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.572224+0000', 'last_change': '2023-11-26T07:18:24.572224+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.572224+0000', 'last_undegraded': '2023-11-26T07:18:24.572224+0000', 'last_fullsized': '2023-11-26T07:18:24.572224+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:30.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-11-26T07:18:30.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.580212+0000', 'last_change': '2023-11-26T07:18:24.580212+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.580212+0000', 'last_undegraded': '2023-11-26T07:18:24.580212+0000', 'last_fullsized': '2023-11-26T07:18:24.580212+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:30.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-11-26T07:18:30.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.586557+0000', 'last_change': '2023-11-26T07:18:24.586557+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.586557+0000', 'last_undegraded': '2023-11-26T07:18:24.586557+0000', 'last_fullsized': '2023-11-26T07:18:24.586557+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.475 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-11-26T07:18:30.475 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.584108+0000', 'last_change': '2023-11-26T07:18:24.584108+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.584108+0000', 'last_undegraded': '2023-11-26T07:18:24.584108+0000', 'last_fullsized': '2023-11-26T07:18:24.584108+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-11-26T07:18:30.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.575896+0000', 'last_change': '2023-11-26T07:18:24.575896+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.575896+0000', 'last_undegraded': '2023-11-26T07:18:24.575896+0000', 'last_fullsized': '2023-11-26T07:18:24.575896+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:30.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-11-26T07:18:30.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.570993+0000', 'last_change': '2023-11-26T07:18:24.570993+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.570993+0000', 'last_undegraded': '2023-11-26T07:18:24.570993+0000', 'last_fullsized': '2023-11-26T07:18:24.570993+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:30.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-11-26T07:18:30.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.583078+0000', 'last_change': '2023-11-26T07:18:24.583078+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.583078+0000', 'last_undegraded': '2023-11-26T07:18:24.583078+0000', 'last_fullsized': '2023-11-26T07:18:24.583078+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-11-26T07:18:30.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.571481+0000', 'last_change': '2023-11-26T07:18:24.571481+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.571481+0000', 'last_undegraded': '2023-11-26T07:18:24.571481+0000', 'last_fullsized': '2023-11-26T07:18:24.571481+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:30.476 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-11-26T07:18:30.476 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.571703+0000', 'last_change': '2023-11-26T07:18:24.571703+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.571703+0000', 'last_undegraded': '2023-11-26T07:18:24.571703+0000', 'last_fullsized': '2023-11-26T07:18:24.571703+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:30.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-11-26T07:18:30.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.579692+0000', 'last_change': '2023-11-26T07:18:24.579692+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.579692+0000', 'last_undegraded': '2023-11-26T07:18:24.579692+0000', 'last_fullsized': '2023-11-26T07:18:24.579692+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:30.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-11-26T07:18:30.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.570094+0000', 'last_change': '2023-11-26T07:18:24.570094+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.570094+0000', 'last_undegraded': '2023-11-26T07:18:24.570094+0000', 'last_fullsized': '2023-11-26T07:18:24.570094+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:30.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-11-26T07:18:30.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.580880+0000', 'last_change': '2023-11-26T07:18:24.580880+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.580880+0000', 'last_undegraded': '2023-11-26T07:18:24.580880+0000', 'last_fullsized': '2023-11-26T07:18:24.580880+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-11-26T07:18:30.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.583314+0000', 'last_change': '2023-11-26T07:18:24.583314+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.583314+0000', 'last_undegraded': '2023-11-26T07:18:24.583314+0000', 'last_fullsized': '2023-11-26T07:18:24.583314+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.477 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-11-26T07:18:30.477 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.579945+0000', 'last_change': '2023-11-26T07:18:24.579945+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.579945+0000', 'last_undegraded': '2023-11-26T07:18:24.579945+0000', 'last_fullsized': '2023-11-26T07:18:24.579945+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.577346+0000', 'last_change': '2023-11-26T07:18:24.577346+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.577346+0000', 'last_undegraded': '2023-11-26T07:18:24.577346+0000', 'last_fullsized': '2023-11-26T07:18:24.577346+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.571517+0000', 'last_change': '2023-11-26T07:18:24.571517+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.571517+0000', 'last_undegraded': '2023-11-26T07:18:24.571517+0000', 'last_fullsized': '2023-11-26T07:18:24.571517+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.578313+0000', 'last_change': '2023-11-26T07:18:24.578313+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.578313+0000', 'last_undegraded': '2023-11-26T07:18:24.578313+0000', 'last_fullsized': '2023-11-26T07:18:24.578313+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.578441+0000', 'last_change': '2023-11-26T07:18:24.578441+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.578441+0000', 'last_undegraded': '2023-11-26T07:18:24.578441+0000', 'last_fullsized': '2023-11-26T07:18:24.578441+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.572722+0000', 'last_change': '2023-11-26T07:18:24.572722+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.572722+0000', 'last_undegraded': '2023-11-26T07:18:24.572722+0000', 'last_fullsized': '2023-11-26T07:18:24.572722+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:18:30.478 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-11-26T07:18:30.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.570688+0000', 'last_change': '2023-11-26T07:18:24.570688+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.570688+0000', 'last_undegraded': '2023-11-26T07:18:24.570688+0000', 'last_fullsized': '2023-11-26T07:18:24.570688+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:30.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-11-26T07:18:30.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.572193+0000', 'last_change': '2023-11-26T07:18:24.572193+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.572193+0000', 'last_undegraded': '2023-11-26T07:18:24.572193+0000', 'last_fullsized': '2023-11-26T07:18:24.572193+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:30.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-11-26T07:18:30.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.577541+0000', 'last_change': '2023-11-26T07:18:24.577541+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.577541+0000', 'last_undegraded': '2023-11-26T07:18:24.577541+0000', 'last_fullsized': '2023-11-26T07:18:24.577541+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:30.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-11-26T07:18:30.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.573219+0000', 'last_change': '2023-11-26T07:18:24.573219+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.573219+0000', 'last_undegraded': '2023-11-26T07:18:24.573219+0000', 'last_fullsized': '2023-11-26T07:18:24.573219+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:30.479 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-11-26T07:18:30.479 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.570094+0000', 'last_change': '2023-11-26T07:18:24.570094+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.570094+0000', 'last_undegraded': '2023-11-26T07:18:24.570094+0000', 'last_fullsized': '2023-11-26T07:18:24.570094+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:30.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-11-26T07:18:30.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.577765+0000', 'last_change': '2023-11-26T07:18:24.577765+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.577765+0000', 'last_undegraded': '2023-11-26T07:18:24.577765+0000', 'last_fullsized': '2023-11-26T07:18:24.577765+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:30.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-11-26T07:18:30.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.581657+0000', 'last_change': '2023-11-26T07:18:24.581657+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.581657+0000', 'last_undegraded': '2023-11-26T07:18:24.581657+0000', 'last_fullsized': '2023-11-26T07:18:24.581657+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-11-26T07:18:30.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.573470+0000', 'last_change': '2023-11-26T07:18:24.573470+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.573470+0000', 'last_undegraded': '2023-11-26T07:18:24.573470+0000', 'last_fullsized': '2023-11-26T07:18:24.573470+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:18:30.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-11-26T07:18:30.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.583193+0000', 'last_change': '2023-11-26T07:18:24.583193+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.583193+0000', 'last_undegraded': '2023-11-26T07:18:24.583193+0000', 'last_fullsized': '2023-11-26T07:18:24.583193+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.480 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-11-26T07:18:30.480 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.573749+0000', 'last_change': '2023-11-26T07:18:24.573749+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.573749+0000', 'last_undegraded': '2023-11-26T07:18:24.573749+0000', 'last_fullsized': '2023-11-26T07:18:24.573749+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:18:30.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-11-26T07:18:30.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.573126+0000', 'last_change': '2023-11-26T07:18:24.573126+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.573126+0000', 'last_undegraded': '2023-11-26T07:18:24.573126+0000', 'last_fullsized': '2023-11-26T07:18:24.573126+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:18:30.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-11-26T07:18:30.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.577389+0000', 'last_change': '2023-11-26T07:18:24.577389+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.577389+0000', 'last_undegraded': '2023-11-26T07:18:24.577389+0000', 'last_fullsized': '2023-11-26T07:18:24.577389+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:30.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-11-26T07:18:30.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.572762+0000', 'last_change': '2023-11-26T07:18:24.572762+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.572762+0000', 'last_undegraded': '2023-11-26T07:18:24.572762+0000', 'last_fullsized': '2023-11-26T07:18:24.572762+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:30.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-11-26T07:18:30.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.581188+0000', 'last_change': '2023-11-26T07:18:24.581188+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.581188+0000', 'last_undegraded': '2023-11-26T07:18:24.581188+0000', 'last_fullsized': '2023-11-26T07:18:24.581188+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.481 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-11-26T07:18:30.481 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.570150+0000', 'last_change': '2023-11-26T07:18:24.570150+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.570150+0000', 'last_undegraded': '2023-11-26T07:18:24.570150+0000', 'last_fullsized': '2023-11-26T07:18:24.570150+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:30.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-11-26T07:18:30.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.575991+0000', 'last_change': '2023-11-26T07:18:24.575991+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.575991+0000', 'last_undegraded': '2023-11-26T07:18:24.575991+0000', 'last_fullsized': '2023-11-26T07:18:24.575991+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:30.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-11-26T07:18:30.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.574611+0000', 'last_change': '2023-11-26T07:18:24.574611+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.574611+0000', 'last_undegraded': '2023-11-26T07:18:24.574611+0000', 'last_fullsized': '2023-11-26T07:18:24.574611+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:30.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-11-26T07:18:30.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.570098+0000', 'last_change': '2023-11-26T07:18:24.570098+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.570098+0000', 'last_undegraded': '2023-11-26T07:18:24.570098+0000', 'last_fullsized': '2023-11-26T07:18:24.570098+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:30.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-11-26T07:18:30.482 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.577291+0000', 'last_change': '2023-11-26T07:18:24.577291+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.577291+0000', 'last_undegraded': '2023-11-26T07:18:24.577291+0000', 'last_fullsized': '2023-11-26T07:18:24.577291+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:30.482 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-11-26T07:18:30.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.577248+0000', 'last_change': '2023-11-26T07:18:24.577248+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.577248+0000', 'last_undegraded': '2023-11-26T07:18:24.577248+0000', 'last_fullsized': '2023-11-26T07:18:24.577248+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:30.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-11-26T07:18:30.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.571005+0000', 'last_change': '2023-11-26T07:18:24.571005+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.571005+0000', 'last_undegraded': '2023-11-26T07:18:24.571005+0000', 'last_fullsized': '2023-11-26T07:18:24.571005+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:30.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-11-26T07:18:30.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.572281+0000', 'last_change': '2023-11-26T07:18:24.572281+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.572281+0000', 'last_undegraded': '2023-11-26T07:18:24.572281+0000', 'last_fullsized': '2023-11-26T07:18:24.572281+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:30.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-11-26T07:18:30.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.576716+0000', 'last_change': '2023-11-26T07:18:24.576716+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.576716+0000', 'last_undegraded': '2023-11-26T07:18:24.576716+0000', 'last_fullsized': '2023-11-26T07:18:24.576716+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:30.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-11-26T07:18:30.483 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.582736+0000', 'last_change': '2023-11-26T07:18:24.582736+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.582736+0000', 'last_undegraded': '2023-11-26T07:18:24.582736+0000', 'last_fullsized': '2023-11-26T07:18:24.582736+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.483 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-11-26T07:18:30.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.572742+0000', 'last_change': '2023-11-26T07:18:24.572742+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.572742+0000', 'last_undegraded': '2023-11-26T07:18:24.572742+0000', 'last_fullsized': '2023-11-26T07:18:24.572742+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:18:30.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-11-26T07:18:30.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.578391+0000', 'last_change': '2023-11-26T07:18:24.578391+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.578391+0000', 'last_undegraded': '2023-11-26T07:18:24.578391+0000', 'last_fullsized': '2023-11-26T07:18:24.578391+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:30.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-11-26T07:18:30.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.572015+0000', 'last_change': '2023-11-26T07:18:24.572015+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.572015+0000', 'last_undegraded': '2023-11-26T07:18:24.572015+0000', 'last_fullsized': '2023-11-26T07:18:24.572015+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:30.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-11-26T07:18:30.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.574046+0000', 'last_change': '2023-11-26T07:18:24.574046+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.574046+0000', 'last_undegraded': '2023-11-26T07:18:24.574046+0000', 'last_fullsized': '2023-11-26T07:18:24.574046+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:18:30.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-11-26T07:18:30.484 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.571003+0000', 'last_change': '2023-11-26T07:18:24.571003+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.571003+0000', 'last_undegraded': '2023-11-26T07:18:24.571003+0000', 'last_fullsized': '2023-11-26T07:18:24.571003+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:30.484 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-11-26T07:18:30.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.573113+0000', 'last_change': '2023-11-26T07:18:24.573113+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.573113+0000', 'last_undegraded': '2023-11-26T07:18:24.573113+0000', 'last_fullsized': '2023-11-26T07:18:24.573113+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:30.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-11-26T07:18:30.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.571207+0000', 'last_change': '2023-11-26T07:18:24.571207+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.571207+0000', 'last_undegraded': '2023-11-26T07:18:24.571207+0000', 'last_fullsized': '2023-11-26T07:18:24.571207+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:30.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-11-26T07:18:30.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.572705+0000', 'last_change': '2023-11-26T07:18:24.572705+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.572705+0000', 'last_undegraded': '2023-11-26T07:18:24.572705+0000', 'last_fullsized': '2023-11-26T07:18:24.572705+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:30.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-11-26T07:18:30.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.587065+0000', 'last_change': '2023-11-26T07:18:24.587065+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.587065+0000', 'last_undegraded': '2023-11-26T07:18:24.587065+0000', 'last_fullsized': '2023-11-26T07:18:24.587065+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.485 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-11-26T07:18:30.485 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.571131+0000', 'last_change': '2023-11-26T07:18:24.571131+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.571131+0000', 'last_undegraded': '2023-11-26T07:18:24.571131+0000', 'last_fullsized': '2023-11-26T07:18:24.571131+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:18:30.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-11-26T07:18:30.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.571451+0000', 'last_change': '2023-11-26T07:18:24.571451+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.571451+0000', 'last_undegraded': '2023-11-26T07:18:24.571451+0000', 'last_fullsized': '2023-11-26T07:18:24.571451+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:18:30.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-11-26T07:18:30.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.573208+0000', 'last_change': '2023-11-26T07:18:24.573208+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.573208+0000', 'last_undegraded': '2023-11-26T07:18:24.573208+0000', 'last_fullsized': '2023-11-26T07:18:24.573208+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:18:30.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-11-26T07:18:30.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.573418+0000', 'last_change': '2023-11-26T07:18:24.573418+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.573418+0000', 'last_undegraded': '2023-11-26T07:18:24.573418+0000', 'last_fullsized': '2023-11-26T07:18:24.573418+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:18:30.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-11-26T07:18:30.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.573139+0000', 'last_change': '2023-11-26T07:18:24.573139+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.573139+0000', 'last_undegraded': '2023-11-26T07:18:24.573139+0000', 'last_fullsized': '2023-11-26T07:18:24.573139+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:18:30.486 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-11-26T07:18:30.486 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.578998+0000', 'last_change': '2023-11-26T07:18:24.578998+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.578998+0000', 'last_undegraded': '2023-11-26T07:18:24.578998+0000', 'last_fullsized': '2023-11-26T07:18:24.578998+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:18:30.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-11-26T07:18:30.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.578922+0000', 'last_change': '2023-11-26T07:18:24.578922+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.578922+0000', 'last_undegraded': '2023-11-26T07:18:24.578922+0000', 'last_fullsized': '2023-11-26T07:18:24.578922+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:18:30.487 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-11-26T07:18:30.487 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:18:24.582610+0000', 'last_change': '2023-11-26T07:18:24.582610+0000', 'last_active': '2023-11-26T07:18:24.558201+0000', 'last_peered': '2023-11-26T07:18:24.558201+0000', 'last_clean': '2023-11-26T07:18:24.558201+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:18:24.582610+0000', 'last_undegraded': '2023-11-26T07:18:24.582610+0000', 'last_fullsized': '2023-11-26T07:18:24.582610+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'last_clean_scrub_stamp': '2023-11-26T07:18:24.558201+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:18:30.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-26T07:18:30.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc994965640 1 Processor -- start 2023-11-26T07:18:30.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc994965640 1 -- start start 2023-11-26T07:18:30.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc994965640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc99015d7b0 0x7fc9900d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc994965640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc9900cead0 0x7fc9900d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc994965640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc9900d1eb0 0x7fc9900cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc994965640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9900d2340 con 0x7fc99015d7b0 2023-11-26T07:18:30.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc994965640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9900cefd0 con 0x7fc9900d1eb0 2023-11-26T07:18:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc994965640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900cf2b0 con 0x7fc9900cead0 2023-11-26T07:18:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc98e59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc99015d7b0 0x7fc9900d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc98e59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc99015d7b0 0x7fc9900d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39924/0 (socket says 172.21.15.31:39924) 2023-11-26T07:18:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc98e59c640 1 -- 172.21.15.31:0/3786591000 learned_addr learned my addr 172.21.15.31:0/3786591000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc98e59c640 1 -- 172.21.15.31:0/3786591000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc9900cead0 msgr2=0x7fc9900d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc98e59c640 1 --2- 172.21.15.31:0/3786591000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc9900cead0 0x7fc9900d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc98e59c640 1 -- 172.21.15.31:0/3786591000 >> v2:172.21.15.97:3300/0 conn(0x7fc9900d1eb0 msgr2=0x7fc9900cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc98e59c640 1 --2- 172.21.15.31:0/3786591000 >> v2:172.21.15.97:3300/0 conn(0x7fc9900d1eb0 0x7fc9900cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.656+0000 7fc98e59c640 1 -- 172.21.15.31:0/3786591000 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99006b3f0 con 0x7fc99015d7b0 2023-11-26T07:18:30.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc98e59c640 1 --2- 172.21.15.31:0/3786591000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc99015d7b0 0x7fc9900d10d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc99008c870 tx=0x7fc98001c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2a1e3a5ba60229 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc98dd9b640 1 -- 172.21.15.31:0/3786591000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800ab600 con 0x7fc99015d7b0 2023-11-26T07:18:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc98dd9b640 1 -- 172.21.15.31:0/3786591000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9800a7050 con 0x7fc99015d7b0 2023-11-26T07:18:30.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc98dd9b640 1 -- 172.21.15.31:0/3786591000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800ab980 con 0x7fc99015d7b0 2023-11-26T07:18:30.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc994965640 1 -- 172.21.15.31:0/3786591000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc99015d7b0 msgr2=0x7fc9900d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc994965640 1 --2- 172.21.15.31:0/3786591000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc99015d7b0 0x7fc9900d10d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc99008c870 tx=0x7fc98001c0f0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc994965640 1 -- 172.21.15.31:0/3786591000 shutdown_connections 2023-11-26T07:18:30.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc994965640 1 --2- 172.21.15.31:0/3786591000 >> v2:172.21.15.97:3300/0 conn(0x7fc9900d1eb0 0x7fc9900cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc994965640 1 --2- 172.21.15.31:0/3786591000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc9900cead0 0x7fc9900d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc994965640 1 --2- 172.21.15.31:0/3786591000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc99015d7b0 0x7fc9900d10d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc994965640 1 -- 172.21.15.31:0/3786591000 >> 172.21.15.31:0/3786591000 conn(0x7fc9900d33b0 msgr2=0x7fc9900d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc994965640 1 -- 172.21.15.31:0/3786591000 shutdown_connections 2023-11-26T07:18:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc994965640 1 -- 172.21.15.31:0/3786591000 wait complete. 2023-11-26T07:18:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.660+0000 7fc994965640 1 Processor -- start 2023-11-26T07:18:30.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc994965640 1 -- start start 2023-11-26T07:18:30.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc994965640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9900cead0 0x7fc99015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc994965640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9900d1eb0 0x7fc9901544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc994965640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc99015d7b0 0x7fc990153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc994965640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9901558c0 con 0x7fc9900cead0 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc994965640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc990154a30 con 0x7fc9900d1eb0 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc994965640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9901542d0 con 0x7fc99015d7b0 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9900cead0 0x7fc99015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9900cead0 0x7fc99015cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39932/0 (socket says 172.21.15.31:39932) 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98e59c640 1 -- 172.21.15.31:0/3335883447 learned_addr learned my addr 172.21.15.31:0/3335883447 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc987fff640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9900d1eb0 0x7fc9901544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98e59c640 1 -- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc99015d7b0 msgr2=0x7fc990153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98ed9d640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc99015d7b0 0x7fc990153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98e59c640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc99015d7b0 0x7fc990153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98e59c640 1 -- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9900d1eb0 msgr2=0x7fc9901544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98e59c640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9900d1eb0 0x7fc9901544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98e59c640 1 -- 172.21.15.31:0/3335883447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800094d0 con 0x7fc9900cead0 2023-11-26T07:18:30.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc987fff640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9900d1eb0 0x7fc9901544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:30.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98ed9d640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc99015d7b0 0x7fc990153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:30.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc98e59c640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9900cead0 0x7fc99015cb60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc9800ae040 tx=0x7fc98001a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc986ffd640 1 -- 172.21.15.31:0/3335883447 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800b6620 con 0x7fc9900cead0 2023-11-26T07:18:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc986ffd640 1 -- 172.21.15.31:0/3335883447 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9800b67a0 con 0x7fc9900cead0 2023-11-26T07:18:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc994965640 1 -- 172.21.15.31:0/3335883447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc990154c40 con 0x7fc9900cead0 2023-11-26T07:18:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc986ffd640 1 -- 172.21.15.31:0/3335883447 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800b6b50 con 0x7fc9900cead0 2023-11-26T07:18:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc994965640 1 -- 172.21.15.31:0/3335883447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc990155200 con 0x7fc9900cead0 2023-11-26T07:18:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.664+0000 7fc994965640 1 -- 172.21.15.31:0/3335883447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99004e850 con 0x7fc9900cead0 2023-11-26T07:18:30.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.668+0000 7fc986ffd640 1 -- 172.21.15.31:0/3335883447 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc9800b2040 con 0x7fc9900cead0 2023-11-26T07:18:30.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.668+0000 7fc986ffd640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc970066470 0x7fc970068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:30.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.668+0000 7fc986ffd640 1 -- 172.21.15.31:0/3335883447 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc98011e470 con 0x7fc9900cead0 2023-11-26T07:18:30.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.668+0000 7fc987fff640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc970066470 0x7fc970068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:30.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.668+0000 7fc987fff640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc970066470 0x7fc970068930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9900d3ea0 tx=0x7fc978008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:30.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.672+0000 7fc986ffd640 1 -- 172.21.15.31:0/3335883447 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc9800eb850 con 0x7fc9900cead0 2023-11-26T07:18:30.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.884+0000 7fc994965640 1 -- 172.21.15.31:0/3335883447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc9900d3930 con 0x7fc9900cead0 2023-11-26T07:18:30.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.884+0000 7fc986ffd640 1 -- 172.21.15.31:0/3335883447 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1580 (secure 0 0 0) 0x7fc9800eee70 con 0x7fc9900cead0 2023-11-26T07:18:30.890 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:30.890 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1700983039,"num_in_osds":8,"osd_in_since":1700983029,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225308672,"bytes_avail":772868804608,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5106}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-11-26T07:18:22.318388+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-26T07:18:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 -- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc970066470 msgr2=0x7fc970068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc970066470 0x7fc970068930 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9900d3ea0 tx=0x7fc978008040 comp rx=0 tx=0).stop 2023-11-26T07:18:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 -- 172.21.15.31:0/3335883447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9900cead0 msgr2=0x7fc99015cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9900cead0 0x7fc99015cb60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc9800ae040 tx=0x7fc98001a930 comp rx=0 tx=0).stop 2023-11-26T07:18:30.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 -- 172.21.15.31:0/3335883447 shutdown_connections 2023-11-26T07:18:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc970066470 0x7fc970068930 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc99015d7b0 0x7fc990153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9900d1eb0 0x7fc9901544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 --2- 172.21.15.31:0/3335883447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9900cead0 0x7fc99015cb60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 -- 172.21.15.31:0/3335883447 >> 172.21.15.31:0/3335883447 conn(0x7fc9900d33b0 msgr2=0x7fc990159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 -- 172.21.15.31:0/3335883447 shutdown_connections 2023-11-26T07:18:30.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:30.888+0000 7fc994965640 1 -- 172.21.15.31:0/3335883447 wait complete. 2023-11-26T07:18:30.911 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-26T07:18:30.911 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:31.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.048+0000 7f19826ea640 1 Processor -- start 2023-11-26T07:18:31.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.048+0000 7f19826ea640 1 -- start start 2023-11-26T07:18:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.048+0000 7f19826ea640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f197c157560 0x7f197c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.048+0000 7f19826ea640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c157940 con 0x7f197c071cc0 2023-11-26T07:18:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.048+0000 7f19826ea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c0b4e90 con 0x7f197c15a9c0 2023-11-26T07:18:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.048+0000 7f19826ea640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197c0b4ca0 con 0x7f197c157560 2023-11-26T07:18:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.048+0000 7f197bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f197c157560 0x7f197c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.048+0000 7f197bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f197c157560 0x7f197c0b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39938/0 (socket says 172.21.15.31:39938) 2023-11-26T07:18:31.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.048+0000 7f197bfff640 1 -- 172.21.15.31:0/3796845516 learned_addr learned my addr 172.21.15.31:0/3796845516 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:31.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f197bfff640 1 -- 172.21.15.31:0/3796845516 >> v1:172.21.15.97:6789/0 conn(0x7f197c15a9c0 legacy=0x7f197c0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:31.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f197bfff640 1 -- 172.21.15.31:0/3796845516 >> v1:172.21.15.31:6789/0 conn(0x7f197c071cc0 legacy=0x7f197c0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f197bfff640 1 -- 172.21.15.31:0/3796845516 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c15ae70 con 0x7f197c157560 2023-11-26T07:18:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f197bfff640 1 --2- 172.21.15.31:0/3796845516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c157560 0x7f197c0b4760 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f196c009870 tx=0x7f196c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925005f1be2eef94 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f197affd640 1 -- 172.21.15.31:0/3796845516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c0ac630 con 0x7f197c157560 2023-11-26T07:18:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f197affd640 1 -- 172.21.15.31:0/3796845516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c0ac790 con 0x7f197c157560 2023-11-26T07:18:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f197affd640 1 -- 172.21.15.31:0/3796845516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c0acb20 con 0x7f197c157560 2023-11-26T07:18:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f19826ea640 1 -- 172.21.15.31:0/3796845516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c157560 msgr2=0x7f197c0b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f19826ea640 1 --2- 172.21.15.31:0/3796845516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c157560 0x7f197c0b4760 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f196c009870 tx=0x7f196c01c450 comp rx=0 tx=0).stop 2023-11-26T07:18:31.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f19826ea640 1 -- 172.21.15.31:0/3796845516 shutdown_connections 2023-11-26T07:18:31.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f19826ea640 1 --2- 172.21.15.31:0/3796845516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c157560 0x7f197c0b4760 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f19826ea640 1 -- 172.21.15.31:0/3796845516 >> 172.21.15.31:0/3796845516 conn(0x7f197c0b1e40 msgr2=0x7f197c070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:31.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f19826ea640 1 -- 172.21.15.31:0/3796845516 shutdown_connections 2023-11-26T07:18:31.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.052+0000 7f19826ea640 1 -- 172.21.15.31:0/3796845516 wait complete. 2023-11-26T07:18:31.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f19826ea640 1 Processor -- start 2023-11-26T07:18:31.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f19826ea640 1 -- start start 2023-11-26T07:18:31.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f19826ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f197c071cc0 0x7f197c151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f19826ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f197c157560 0x7f197c1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f19826ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c15a9c0 0x7f197c152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f19826ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c158bc0 con 0x7f197c15a9c0 2023-11-26T07:18:31.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f19826ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c158ea0 con 0x7f197c157560 2023-11-26T07:18:31.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f19826ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c159180 con 0x7f197c071cc0 2023-11-26T07:18:31.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1980c87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c15a9c0 0x7f197c152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1980c87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c15a9c0 0x7f197c152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39946/0 (socket says 172.21.15.31:39946) 2023-11-26T07:18:31.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1980c87640 1 -- 172.21.15.31:0/2427276386 learned_addr learned my addr 172.21.15.31:0/2427276386 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:31.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f197bfff640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f197c071cc0 0x7f197c151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f197b7fe640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f197c157560 0x7f197c1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1980c87640 1 -- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f197c071cc0 msgr2=0x7f197c151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1980c87640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f197c071cc0 0x7f197c151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1980c87640 1 -- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f197c157560 msgr2=0x7f197c1584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1980c87640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f197c157560 0x7f197c1584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1980c87640 1 -- 172.21.15.31:0/2427276386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c0094d0 con 0x7f197c15a9c0 2023-11-26T07:18:31.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f197b7fe640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f197c157560 0x7f197c1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:31.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1980c87640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c15a9c0 0x7f197c152860 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f197400c770 tx=0x7f197400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:31.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1978ff9640 1 -- 172.21.15.31:0/2427276386 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974003b80 con 0x7f197c15a9c0 2023-11-26T07:18:31.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.056+0000 7f1978ff9640 1 -- 172.21.15.31:0/2427276386 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1974003ce0 con 0x7f197c15a9c0 2023-11-26T07:18:31.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.060+0000 7f1978ff9640 1 -- 172.21.15.31:0/2427276386 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197400aa90 con 0x7f197c15a9c0 2023-11-26T07:18:31.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.060+0000 7f19826ea640 1 -- 172.21.15.31:0/2427276386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c159460 con 0x7f197c15a9c0 2023-11-26T07:18:31.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.060+0000 7f19826ea640 1 -- 172.21.15.31:0/2427276386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c152da0 con 0x7f197c15a9c0 2023-11-26T07:18:31.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.060+0000 7f19826ea640 1 -- 172.21.15.31:0/2427276386 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c04e850 con 0x7f197c15a9c0 2023-11-26T07:18:31.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.060+0000 7f1978ff9640 1 -- 172.21.15.31:0/2427276386 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f197400abf0 con 0x7f197c15a9c0 2023-11-26T07:18:31.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.060+0000 7f1978ff9640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1968066470 0x7f1968068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:31.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.060+0000 7f1978ff9640 1 -- 172.21.15.31:0/2427276386 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1974083cc0 con 0x7f197c15a9c0 2023-11-26T07:18:31.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.060+0000 7f197bfff640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1968066470 0x7f1968068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:31.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.064+0000 7f197bfff640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1968066470 0x7f1968068930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f196c0ae000 tx=0x7f196c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:31.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.064+0000 7f1978ff9640 1 -- 172.21.15.31:0/2427276386 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f197408a080 con 0x7f197c15a9c0 2023-11-26T07:18:31.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.220+0000 7f19826ea640 1 -- 172.21.15.31:0/2427276386 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f197c159010 con 0x7f1968066470 2023-11-26T07:18:31.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f1978ff9640 1 -- 172.21.15.31:0/2427276386 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283559 (secure 0 0 0) 0x7f197c159010 con 0x7f1968066470 2023-11-26T07:18:31.235 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:31.237 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:18:31.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 -- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1968066470 msgr2=0x7f1968068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1968066470 0x7f1968068930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f196c0ae000 tx=0x7f196c008040 comp rx=0 tx=0).stop 2023-11-26T07:18:31.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 -- 172.21.15.31:0/2427276386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c15a9c0 msgr2=0x7f197c152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:31.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c15a9c0 0x7f197c152860 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f197400c770 tx=0x7f197400cc30 comp rx=0 tx=0).stop 2023-11-26T07:18:31.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 -- 172.21.15.31:0/2427276386 shutdown_connections 2023-11-26T07:18:31.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1968066470 0x7f1968068930 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c15a9c0 0x7f197c152860 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f197c157560 0x7f197c1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 --2- 172.21.15.31:0/2427276386 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f197c071cc0 0x7f197c151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:31.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 -- 172.21.15.31:0/2427276386 >> 172.21.15.31:0/2427276386 conn(0x7f197c0b1e40 msgr2=0x7f197c15c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:31.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.228+0000 7f19826ea640 1 -- 172.21.15.31:0/2427276386 shutdown_connections 2023-11-26T07:18:31.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:31.232+0000 7f19826ea640 1 -- 172.21.15.31:0/2427276386 wait complete. 2023-11-26T07:18:31.255 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-11-26T07:18:30.320140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220028,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754692,"statfs":{"total":773094113280,"available":772868804608,"internally_reserved":0,"allocated":3956736,"data_stored":2240546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001415"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574246+0000","last_change":"2023-11-26T07:18:24.574246+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574246+0000","last_undegraded":"2023-11-26T07:18:24.574246+0000","last_fullsized":"2023-11-26T07:18:24.574246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580477+0000","last_change":"2023-11-26T07:18:24.580477+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580477+0000","last_undegraded":"2023-11-26T07:18:24.580477+0000","last_fullsized":"2023-11-26T07:18:24.580477+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570490+0000","last_change":"2023-11-26T07:18:24.570490+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570490+0000","last_undegraded":"2023-11-26T07:18:24.570490+0000","last_fullsized":"2023-11-26T07:18:24.570490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575149+0000","last_change":"2023-11-26T07:18:24.575149+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575149+0000","last_undegraded":"2023-11-26T07:18:24.575149+0000","last_fullsized":"2023-11-26T07:18:24.575149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572224+0000","last_change":"2023-11-26T07:18:24.572224+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572224+0000","last_undegraded":"2023-11-26T07:18:24.572224+0000","last_fullsized":"2023-11-26T07:18:24.572224+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580212+0000","last_change":"2023-11-26T07:18:24.580212+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580212+0000","last_undegraded":"2023-11-26T07:18:24.580212+0000","last_fullsized":"2023-11-26T07:18:24.580212+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.586557+0000","last_change":"2023-11-26T07:18:24.586557+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.586557+0000","last_undegraded":"2023-11-26T07:18:24.586557+0000","last_fullsized":"2023-11-26T07:18:24.586557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.584108+0000","last_change":"2023-11-26T07:18:24.584108+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.584108+0000","last_undegraded":"2023-11-26T07:18:24.584108+0000","last_fullsized":"2023-11-26T07:18:24.584108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575896+0000","last_change":"2023-11-26T07:18:24.575896+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575896+0000","last_undegraded":"2023-11-26T07:18:24.575896+0000","last_fullsized":"2023-11-26T07:18:24.575896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570993+0000","last_change":"2023-11-26T07:18:24.570993+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570993+0000","last_undegraded":"2023-11-26T07:18:24.570993+0000","last_fullsized":"2023-11-26T07:18:24.570993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583078+0000","last_change":"2023-11-26T07:18:24.583078+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583078+0000","last_undegraded":"2023-11-26T07:18:24.583078+0000","last_fullsized":"2023-11-26T07:18:24.583078+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571481+0000","last_change":"2023-11-26T07:18:24.571481+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571481+0000","last_undegraded":"2023-11-26T07:18:24.571481+0000","last_fullsized":"2023-11-26T07:18:24.571481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571703+0000","last_change":"2023-11-26T07:18:24.571703+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571703+0000","last_undegraded":"2023-11-26T07:18:24.571703+0000","last_fullsized":"2023-11-26T07:18:24.571703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.579692+0000","last_change":"2023-11-26T07:18:24.579692+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.579692+0000","last_undegraded":"2023-11-26T07:18:24.579692+0000","last_fullsized":"2023-11-26T07:18:24.579692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570094+0000","last_change":"2023-11-26T07:18:24.570094+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570094+0000","last_undegraded":"2023-11-26T07:18:24.570094+0000","last_fullsized":"2023-11-26T07:18:24.570094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.580880+0000","last_change":"2023-11-26T07:18:24.580880+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.580880+0000","last_undegraded":"2023-11-26T07:18:24.580880+0000","last_fullsized":"2023-11-26T07:18:24.580880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583314+0000","last_change":"2023-11-26T07:18:24.583314+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583314+0000","last_undegraded":"2023-11-26T07:18:24.583314+0000","last_fullsized":"2023-11-26T07:18:24.583314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.579945+0000","last_change":"2023-11-26T07:18:24.579945+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.579945+0000","last_undegraded":"2023-11-26T07:18:24.579945+0000","last_fullsized":"2023-11-26T07:18:24.579945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577346+0000","last_change":"2023-11-26T07:18:24.577346+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577346+0000","last_undegraded":"2023-11-26T07:18:24.577346+0000","last_fullsized":"2023-11-26T07:18:24.577346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571517+0000","last_change":"2023-11-26T07:18:24.571517+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571517+0000","last_undegraded":"2023-11-26T07:18:24.571517+0000","last_fullsized":"2023-11-26T07:18:24.571517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578313+0000","last_change":"2023-11-26T07:18:24.578313+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578313+0000","last_undegraded":"2023-11-26T07:18:24.578313+0000","last_fullsized":"2023-11-26T07:18:24.578313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578441+0000","last_change":"2023-11-26T07:18:24.578441+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578441+0000","last_undegraded":"2023-11-26T07:18:24.578441+0000","last_fullsized":"2023-11-26T07:18:24.578441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572722+0000","last_change":"2023-11-26T07:18:24.572722+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572722+0000","last_undegraded":"2023-11-26T07:18:24.572722+0000","last_fullsized":"2023-11-26T07:18:24.572722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570688+0000","last_change":"2023-11-26T07:18:24.570688+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570688+0000","last_undegraded":"2023-11-26T07:18:24.570688+0000","last_fullsized":"2023-11-26T07:18:24.570688+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572193+0000","last_change":"2023-11-26T07:18:24.572193+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572193+0000","last_undegraded":"2023-11-26T07:18:24.572193+0000","last_fullsized":"2023-11-26T07:18:24.572193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577541+0000","last_change":"2023-11-26T07:18:24.577541+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577541+0000","last_undegraded":"2023-11-26T07:18:24.577541+0000","last_fullsized":"2023-11-26T07:18:24.577541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573219+0000","last_change":"2023-11-26T07:18:24.573219+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573219+0000","last_undegraded":"2023-11-26T07:18:24.573219+0000","last_fullsized":"2023-11-26T07:18:24.573219+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570094+0000","last_change":"2023-11-26T07:18:24.570094+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570094+0000","last_undegraded":"2023-11-26T07:18:24.570094+0000","last_fullsized":"2023-11-26T07:18:24.570094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577765+0000","last_change":"2023-11-26T07:18:24.577765+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577765+0000","last_undegraded":"2023-11-26T07:18:24.577765+0000","last_fullsized":"2023-11-26T07:18:24.577765+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.581657+0000","last_change":"2023-11-26T07:18:24.581657+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.581657+0000","last_undegraded":"2023-11-26T07:18:24.581657+0000","last_fullsized":"2023-11-26T07:18:24.581657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573470+0000","last_change":"2023-11-26T07:18:24.573470+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573470+0000","last_undegraded":"2023-11-26T07:18:24.573470+0000","last_fullsized":"2023-11-26T07:18:24.573470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.583193+0000","last_change":"2023-11-26T07:18:24.583193+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.583193+0000","last_undegraded":"2023-11-26T07:18:24.583193+0000","last_fullsized":"2023-11-26T07:18:24.583193+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573749+0000","last_change":"2023-11-26T07:18:24.573749+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573749+0000","last_undegraded":"2023-11-26T07:18:24.573749+0000","last_fullsized":"2023-11-26T07:18:24.573749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.590273+0000","last_change":"2023-11-26T07:18:24.586219+0000","last_active":"2023-11-26T07:18:24.590273+0000","last_peered":"2023-11-26T07:18:24.590273+0000","last_clean":"2023-11-26T07:18:24.590273+0000","last_became_active":"2023-11-26T07:18:24.584015+0000","last_became_peered":"2023-11-26T07:18:24.584015+0000","last_unstale":"2023-11-26T07:18:24.590273+0000","last_undegraded":"2023-11-26T07:18:24.590273+0000","last_fullsized":"2023-11-26T07:18:24.590273+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573126+0000","last_change":"2023-11-26T07:18:24.573126+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573126+0000","last_undegraded":"2023-11-26T07:18:24.573126+0000","last_fullsized":"2023-11-26T07:18:24.573126+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597544+0000","last_change":"2023-11-26T07:18:24.592940+0000","last_active":"2023-11-26T07:18:24.597544+0000","last_peered":"2023-11-26T07:18:24.597544+0000","last_clean":"2023-11-26T07:18:24.597544+0000","last_became_active":"2023-11-26T07:18:24.592670+0000","last_became_peered":"2023-11-26T07:18:24.592670+0000","last_unstale":"2023-11-26T07:18:24.597544+0000","last_undegraded":"2023-11-26T07:18:24.597544+0000","last_fullsized":"2023-11-26T07:18:24.597544+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577389+0000","last_change":"2023-11-26T07:18:24.577389+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577389+0000","last_undegraded":"2023-11-26T07:18:24.577389+0000","last_fullsized":"2023-11-26T07:18:24.577389+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597822+0000","last_change":"2023-11-26T07:18:24.594822+0000","last_active":"2023-11-26T07:18:24.597822+0000","last_peered":"2023-11-26T07:18:24.597822+0000","last_clean":"2023-11-26T07:18:24.597822+0000","last_became_active":"2023-11-26T07:18:24.594466+0000","last_became_peered":"2023-11-26T07:18:24.594466+0000","last_unstale":"2023-11-26T07:18:24.597822+0000","last_undegraded":"2023-11-26T07:18:24.597822+0000","last_fullsized":"2023-11-26T07:18:24.597822+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:55:58.922003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572762+0000","last_change":"2023-11-26T07:18:24.572762+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572762+0000","last_undegraded":"2023-11-26T07:18:24.572762+0000","last_fullsized":"2023-11-26T07:18:24.572762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592346+0000","last_change":"2023-11-26T07:18:24.590286+0000","last_active":"2023-11-26T07:18:24.592346+0000","last_peered":"2023-11-26T07:18:24.592346+0000","last_clean":"2023-11-26T07:18:24.592346+0000","last_became_active":"2023-11-26T07:18:24.589675+0000","last_became_peered":"2023-11-26T07:18:24.589675+0000","last_unstale":"2023-11-26T07:18:24.592346+0000","last_undegraded":"2023-11-26T07:18:24.592346+0000","last_fullsized":"2023-11-26T07:18:24.592346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:39:04.751909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.581188+0000","last_change":"2023-11-26T07:18:24.581188+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.581188+0000","last_undegraded":"2023-11-26T07:18:24.581188+0000","last_fullsized":"2023-11-26T07:18:24.581188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591549+0000","last_change":"2023-11-26T07:18:24.588499+0000","last_active":"2023-11-26T07:18:24.591549+0000","last_peered":"2023-11-26T07:18:24.591549+0000","last_clean":"2023-11-26T07:18:24.591549+0000","last_became_active":"2023-11-26T07:18:24.588209+0000","last_became_peered":"2023-11-26T07:18:24.588209+0000","last_unstale":"2023-11-26T07:18:24.591549+0000","last_undegraded":"2023-11-26T07:18:24.591549+0000","last_fullsized":"2023-11-26T07:18:24.591549+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:24:01.800560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570150+0000","last_change":"2023-11-26T07:18:24.570150+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570150+0000","last_undegraded":"2023-11-26T07:18:24.570150+0000","last_fullsized":"2023-11-26T07:18:24.570150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594137+0000","last_change":"2023-11-26T07:18:24.593041+0000","last_active":"2023-11-26T07:18:24.594137+0000","last_peered":"2023-11-26T07:18:24.594137+0000","last_clean":"2023-11-26T07:18:24.594137+0000","last_became_active":"2023-11-26T07:18:24.592774+0000","last_became_peered":"2023-11-26T07:18:24.592774+0000","last_unstale":"2023-11-26T07:18:24.594137+0000","last_undegraded":"2023-11-26T07:18:24.594137+0000","last_fullsized":"2023-11-26T07:18:24.594137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:57.227910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.575991+0000","last_change":"2023-11-26T07:18:24.575991+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.575991+0000","last_undegraded":"2023-11-26T07:18:24.575991+0000","last_fullsized":"2023-11-26T07:18:24.575991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.588443+0000","last_change":"2023-11-26T07:18:24.586769+0000","last_active":"2023-11-26T07:18:24.588443+0000","last_peered":"2023-11-26T07:18:24.588443+0000","last_clean":"2023-11-26T07:18:24.588443+0000","last_became_active":"2023-11-26T07:18:24.586327+0000","last_became_peered":"2023-11-26T07:18:24.586327+0000","last_unstale":"2023-11-26T07:18:24.588443+0000","last_undegraded":"2023-11-26T07:18:24.588443+0000","last_fullsized":"2023-11-26T07:18:24.588443+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:14:55.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574611+0000","last_change":"2023-11-26T07:18:24.574611+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574611+0000","last_undegraded":"2023-11-26T07:18:24.574611+0000","last_fullsized":"2023-11-26T07:18:24.574611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.586928+0000","last_change":"2023-11-26T07:18:24.581295+0000","last_active":"2023-11-26T07:18:24.586928+0000","last_peered":"2023-11-26T07:18:24.586928+0000","last_clean":"2023-11-26T07:18:24.586928+0000","last_became_active":"2023-11-26T07:18:24.580773+0000","last_became_peered":"2023-11-26T07:18:24.580773+0000","last_unstale":"2023-11-26T07:18:24.586928+0000","last_undegraded":"2023-11-26T07:18:24.586928+0000","last_fullsized":"2023-11-26T07:18:24.586928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:25.890181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.570098+0000","last_change":"2023-11-26T07:18:24.570098+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.570098+0000","last_undegraded":"2023-11-26T07:18:24.570098+0000","last_fullsized":"2023-11-26T07:18:24.570098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.596044+0000","last_change":"2023-11-26T07:18:24.592314+0000","last_active":"2023-11-26T07:18:24.596044+0000","last_peered":"2023-11-26T07:18:24.596044+0000","last_clean":"2023-11-26T07:18:24.596044+0000","last_became_active":"2023-11-26T07:18:24.591990+0000","last_became_peered":"2023-11-26T07:18:24.591990+0000","last_unstale":"2023-11-26T07:18:24.596044+0000","last_undegraded":"2023-11-26T07:18:24.596044+0000","last_fullsized":"2023-11-26T07:18:24.596044+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:52:38.196386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577291+0000","last_change":"2023-11-26T07:18:24.577291+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577291+0000","last_undegraded":"2023-11-26T07:18:24.577291+0000","last_fullsized":"2023-11-26T07:18:24.577291+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594545+0000","last_change":"2023-11-26T07:18:24.589024+0000","last_active":"2023-11-26T07:18:24.594545+0000","last_peered":"2023-11-26T07:18:24.594545+0000","last_clean":"2023-11-26T07:18:24.594545+0000","last_became_active":"2023-11-26T07:18:24.585478+0000","last_became_peered":"2023-11-26T07:18:24.585478+0000","last_unstale":"2023-11-26T07:18:24.594545+0000","last_undegraded":"2023-11-26T07:18:24.594545+0000","last_fullsized":"2023-11-26T07:18:24.594545+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.577248+0000","last_change":"2023-11-26T07:18:24.577248+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.577248+0000","last_undegraded":"2023-11-26T07:18:24.577248+0000","last_fullsized":"2023-11-26T07:18:24.577248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594681+0000","last_change":"2023-11-26T07:18:24.592183+0000","last_active":"2023-11-26T07:18:24.594681+0000","last_peered":"2023-11-26T07:18:24.594681+0000","last_clean":"2023-11-26T07:18:24.594681+0000","last_became_active":"2023-11-26T07:18:24.591338+0000","last_became_peered":"2023-11-26T07:18:24.591338+0000","last_unstale":"2023-11-26T07:18:24.594681+0000","last_undegraded":"2023-11-26T07:18:24.594681+0000","last_fullsized":"2023-11-26T07:18:24.594681+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:53:07.256791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571005+0000","last_change":"2023-11-26T07:18:24.571005+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571005+0000","last_undegraded":"2023-11-26T07:18:24.571005+0000","last_fullsized":"2023-11-26T07:18:24.571005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597809+0000","last_change":"2023-11-26T07:18:24.594736+0000","last_active":"2023-11-26T07:18:24.597809+0000","last_peered":"2023-11-26T07:18:24.597809+0000","last_clean":"2023-11-26T07:18:24.597809+0000","last_became_active":"2023-11-26T07:18:24.594398+0000","last_became_peered":"2023-11-26T07:18:24.594398+0000","last_unstale":"2023-11-26T07:18:24.597809+0000","last_undegraded":"2023-11-26T07:18:24.597809+0000","last_fullsized":"2023-11-26T07:18:24.597809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572281+0000","last_change":"2023-11-26T07:18:24.572281+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572281+0000","last_undegraded":"2023-11-26T07:18:24.572281+0000","last_fullsized":"2023-11-26T07:18:24.572281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.588631+0000","last_change":"2023-11-26T07:18:24.583313+0000","last_active":"2023-11-26T07:18:24.588631+0000","last_peered":"2023-11-26T07:18:24.588631+0000","last_clean":"2023-11-26T07:18:24.588631+0000","last_became_active":"2023-11-26T07:18:24.583028+0000","last_became_peered":"2023-11-26T07:18:24.583028+0000","last_unstale":"2023-11-26T07:18:24.588631+0000","last_undegraded":"2023-11-26T07:18:24.588631+0000","last_fullsized":"2023-11-26T07:18:24.588631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:18:04.226180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.576716+0000","last_change":"2023-11-26T07:18:24.576716+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.576716+0000","last_undegraded":"2023-11-26T07:18:24.576716+0000","last_fullsized":"2023-11-26T07:18:24.576716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.585267+0000","last_change":"2023-11-26T07:18:24.581041+0000","last_active":"2023-11-26T07:18:24.585267+0000","last_peered":"2023-11-26T07:18:24.585267+0000","last_clean":"2023-11-26T07:18:24.585267+0000","last_became_active":"2023-11-26T07:18:24.580753+0000","last_became_peered":"2023-11-26T07:18:24.580753+0000","last_unstale":"2023-11-26T07:18:24.585267+0000","last_undegraded":"2023-11-26T07:18:24.585267+0000","last_fullsized":"2023-11-26T07:18:24.585267+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.582736+0000","last_change":"2023-11-26T07:18:24.582736+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.582736+0000","last_undegraded":"2023-11-26T07:18:24.582736+0000","last_fullsized":"2023-11-26T07:18:24.582736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.600450+0000","last_change":"2023-11-26T07:18:24.596369+0000","last_active":"2023-11-26T07:18:24.600450+0000","last_peered":"2023-11-26T07:18:24.600450+0000","last_clean":"2023-11-26T07:18:24.600450+0000","last_became_active":"2023-11-26T07:18:24.595742+0000","last_became_peered":"2023-11-26T07:18:24.595742+0000","last_unstale":"2023-11-26T07:18:24.600450+0000","last_undegraded":"2023-11-26T07:18:24.600450+0000","last_fullsized":"2023-11-26T07:18:24.600450+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:33.810259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572742+0000","last_change":"2023-11-26T07:18:24.572742+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572742+0000","last_undegraded":"2023-11-26T07:18:24.572742+0000","last_fullsized":"2023-11-26T07:18:24.572742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597987+0000","last_change":"2023-11-26T07:18:24.595332+0000","last_active":"2023-11-26T07:18:24.597987+0000","last_peered":"2023-11-26T07:18:24.597987+0000","last_clean":"2023-11-26T07:18:24.597987+0000","last_became_active":"2023-11-26T07:18:24.594874+0000","last_became_peered":"2023-11-26T07:18:24.594874+0000","last_unstale":"2023-11-26T07:18:24.597987+0000","last_undegraded":"2023-11-26T07:18:24.597987+0000","last_fullsized":"2023-11-26T07:18:24.597987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578391+0000","last_change":"2023-11-26T07:18:24.578391+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578391+0000","last_undegraded":"2023-11-26T07:18:24.578391+0000","last_fullsized":"2023-11-26T07:18:24.578391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.595660+0000","last_change":"2023-11-26T07:18:24.589241+0000","last_active":"2023-11-26T07:18:24.595660+0000","last_peered":"2023-11-26T07:18:24.595660+0000","last_clean":"2023-11-26T07:18:24.595660+0000","last_became_active":"2023-11-26T07:18:24.588980+0000","last_became_peered":"2023-11-26T07:18:24.588980+0000","last_unstale":"2023-11-26T07:18:24.595660+0000","last_undegraded":"2023-11-26T07:18:24.595660+0000","last_fullsized":"2023-11-26T07:18:24.595660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572015+0000","last_change":"2023-11-26T07:18:24.572015+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572015+0000","last_undegraded":"2023-11-26T07:18:24.572015+0000","last_fullsized":"2023-11-26T07:18:24.572015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591739+0000","last_change":"2023-11-26T07:18:24.589574+0000","last_active":"2023-11-26T07:18:24.591739+0000","last_peered":"2023-11-26T07:18:24.591739+0000","last_clean":"2023-11-26T07:18:24.591739+0000","last_became_active":"2023-11-26T07:18:24.589286+0000","last_became_peered":"2023-11-26T07:18:24.589286+0000","last_unstale":"2023-11-26T07:18:24.591739+0000","last_undegraded":"2023-11-26T07:18:24.591739+0000","last_fullsized":"2023-11-26T07:18:24.591739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:01:41.668016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.574046+0000","last_change":"2023-11-26T07:18:24.574046+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.574046+0000","last_undegraded":"2023-11-26T07:18:24.574046+0000","last_fullsized":"2023-11-26T07:18:24.574046+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.593950+0000","last_change":"2023-11-26T07:18:24.590371+0000","last_active":"2023-11-26T07:18:24.593950+0000","last_peered":"2023-11-26T07:18:24.593950+0000","last_clean":"2023-11-26T07:18:24.593950+0000","last_became_active":"2023-11-26T07:18:24.590067+0000","last_became_peered":"2023-11-26T07:18:24.590067+0000","last_unstale":"2023-11-26T07:18:24.593950+0000","last_undegraded":"2023-11-26T07:18:24.593950+0000","last_fullsized":"2023-11-26T07:18:24.593950+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:58.111738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571003+0000","last_change":"2023-11-26T07:18:24.571003+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571003+0000","last_undegraded":"2023-11-26T07:18:24.571003+0000","last_fullsized":"2023-11-26T07:18:24.571003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.589592+0000","last_change":"2023-11-26T07:18:24.585586+0000","last_active":"2023-11-26T07:18:24.589592+0000","last_peered":"2023-11-26T07:18:24.589592+0000","last_clean":"2023-11-26T07:18:24.589592+0000","last_became_active":"2023-11-26T07:18:24.585226+0000","last_became_peered":"2023-11-26T07:18:24.585226+0000","last_unstale":"2023-11-26T07:18:24.589592+0000","last_undegraded":"2023-11-26T07:18:24.589592+0000","last_fullsized":"2023-11-26T07:18:24.589592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:17.423432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573113+0000","last_change":"2023-11-26T07:18:24.573113+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573113+0000","last_undegraded":"2023-11-26T07:18:24.573113+0000","last_fullsized":"2023-11-26T07:18:24.573113+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592523+0000","last_change":"2023-11-26T07:18:24.586275+0000","last_active":"2023-11-26T07:18:24.592523+0000","last_peered":"2023-11-26T07:18:24.592523+0000","last_clean":"2023-11-26T07:18:24.592523+0000","last_became_active":"2023-11-26T07:18:24.585844+0000","last_became_peered":"2023-11-26T07:18:24.585844+0000","last_unstale":"2023-11-26T07:18:24.592523+0000","last_undegraded":"2023-11-26T07:18:24.592523+0000","last_fullsized":"2023-11-26T07:18:24.592523+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571207+0000","last_change":"2023-11-26T07:18:24.571207+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571207+0000","last_undegraded":"2023-11-26T07:18:24.571207+0000","last_fullsized":"2023-11-26T07:18:24.571207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.590359+0000","last_change":"2023-11-26T07:18:24.586954+0000","last_active":"2023-11-26T07:18:24.590359+0000","last_peered":"2023-11-26T07:18:24.590359+0000","last_clean":"2023-11-26T07:18:24.590359+0000","last_became_active":"2023-11-26T07:18:24.586667+0000","last_became_peered":"2023-11-26T07:18:24.586667+0000","last_unstale":"2023-11-26T07:18:24.590359+0000","last_undegraded":"2023-11-26T07:18:24.590359+0000","last_fullsized":"2023-11-26T07:18:24.590359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.572705+0000","last_change":"2023-11-26T07:18:24.572705+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.572705+0000","last_undegraded":"2023-11-26T07:18:24.572705+0000","last_fullsized":"2023-11-26T07:18:24.572705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594929+0000","last_change":"2023-11-26T07:18:24.593394+0000","last_active":"2023-11-26T07:18:24.594929+0000","last_peered":"2023-11-26T07:18:24.594929+0000","last_clean":"2023-11-26T07:18:24.594929+0000","last_became_active":"2023-11-26T07:18:24.593108+0000","last_became_peered":"2023-11-26T07:18:24.593108+0000","last_unstale":"2023-11-26T07:18:24.594929+0000","last_undegraded":"2023-11-26T07:18:24.594929+0000","last_fullsized":"2023-11-26T07:18:24.594929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:39:04.751909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.587065+0000","last_change":"2023-11-26T07:18:24.587065+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.587065+0000","last_undegraded":"2023-11-26T07:18:24.587065+0000","last_fullsized":"2023-11-26T07:18:24.587065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592134+0000","last_change":"2023-11-26T07:18:24.589292+0000","last_active":"2023-11-26T07:18:24.592134+0000","last_peered":"2023-11-26T07:18:24.592134+0000","last_clean":"2023-11-26T07:18:24.592134+0000","last_became_active":"2023-11-26T07:18:24.589014+0000","last_became_peered":"2023-11-26T07:18:24.589014+0000","last_unstale":"2023-11-26T07:18:24.592134+0000","last_undegraded":"2023-11-26T07:18:24.592134+0000","last_fullsized":"2023-11-26T07:18:24.592134+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:50.247115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571131+0000","last_change":"2023-11-26T07:18:24.571131+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571131+0000","last_undegraded":"2023-11-26T07:18:24.571131+0000","last_fullsized":"2023-11-26T07:18:24.571131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.597278+0000","last_change":"2023-11-26T07:18:24.594254+0000","last_active":"2023-11-26T07:18:24.597278+0000","last_peered":"2023-11-26T07:18:24.597278+0000","last_clean":"2023-11-26T07:18:24.597278+0000","last_became_active":"2023-11-26T07:18:24.593932+0000","last_became_peered":"2023-11-26T07:18:24.593932+0000","last_unstale":"2023-11-26T07:18:24.597278+0000","last_undegraded":"2023-11-26T07:18:24.597278+0000","last_fullsized":"2023-11-26T07:18:24.597278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.571451+0000","last_change":"2023-11-26T07:18:24.571451+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.571451+0000","last_undegraded":"2023-11-26T07:18:24.571451+0000","last_fullsized":"2023-11-26T07:18:24.571451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.589132+0000","last_change":"2023-11-26T07:18:24.585493+0000","last_active":"2023-11-26T07:18:24.589132+0000","last_peered":"2023-11-26T07:18:24.589132+0000","last_clean":"2023-11-26T07:18:24.589132+0000","last_became_active":"2023-11-26T07:18:24.585141+0000","last_became_peered":"2023-11-26T07:18:24.585141+0000","last_unstale":"2023-11-26T07:18:24.589132+0000","last_undegraded":"2023-11-26T07:18:24.589132+0000","last_fullsized":"2023-11-26T07:18:24.589132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573208+0000","last_change":"2023-11-26T07:18:24.573208+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573208+0000","last_undegraded":"2023-11-26T07:18:24.573208+0000","last_fullsized":"2023-11-26T07:18:24.573208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.593649+0000","last_change":"2023-11-26T07:18:24.590400+0000","last_active":"2023-11-26T07:18:24.593649+0000","last_peered":"2023-11-26T07:18:24.593649+0000","last_clean":"2023-11-26T07:18:24.593649+0000","last_became_active":"2023-11-26T07:18:24.590083+0000","last_became_peered":"2023-11-26T07:18:24.590083+0000","last_unstale":"2023-11-26T07:18:24.593649+0000","last_undegraded":"2023-11-26T07:18:24.593649+0000","last_fullsized":"2023-11-26T07:18:24.593649+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573418+0000","last_change":"2023-11-26T07:18:24.573418+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573418+0000","last_undegraded":"2023-11-26T07:18:24.573418+0000","last_fullsized":"2023-11-26T07:18:24.573418+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.592541+0000","last_change":"2023-11-26T07:18:24.586412+0000","last_active":"2023-11-26T07:18:24.592541+0000","last_peered":"2023-11-26T07:18:24.592541+0000","last_clean":"2023-11-26T07:18:24.592541+0000","last_became_active":"2023-11-26T07:18:24.585386+0000","last_became_peered":"2023-11-26T07:18:24.585386+0000","last_unstale":"2023-11-26T07:18:24.592541+0000","last_undegraded":"2023-11-26T07:18:24.592541+0000","last_fullsized":"2023-11-26T07:18:24.592541+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:13.757309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.573139+0000","last_change":"2023-11-26T07:18:24.573139+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.573139+0000","last_undegraded":"2023-11-26T07:18:24.573139+0000","last_fullsized":"2023-11-26T07:18:24.573139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.599965+0000","last_change":"2023-11-26T07:18:24.596404+0000","last_active":"2023-11-26T07:18:24.599965+0000","last_peered":"2023-11-26T07:18:24.599965+0000","last_clean":"2023-11-26T07:18:24.599965+0000","last_became_active":"2023-11-26T07:18:24.593975+0000","last_became_peered":"2023-11-26T07:18:24.593975+0000","last_unstale":"2023-11-26T07:18:24.599965+0000","last_undegraded":"2023-11-26T07:18:24.599965+0000","last_fullsized":"2023-11-26T07:18:24.599965+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:21.470530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578998+0000","last_change":"2023-11-26T07:18:24.578998+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578998+0000","last_undegraded":"2023-11-26T07:18:24.578998+0000","last_fullsized":"2023-11-26T07:18:24.578998+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.591213+0000","last_change":"2023-11-26T07:18:24.588130+0000","last_active":"2023-11-26T07:18:24.591213+0000","last_peered":"2023-11-26T07:18:24.591213+0000","last_clean":"2023-11-26T07:18:24.591213+0000","last_became_active":"2023-11-26T07:18:24.586853+0000","last_became_peered":"2023-11-26T07:18:24.586853+0000","last_unstale":"2023-11-26T07:18:24.591213+0000","last_undegraded":"2023-11-26T07:18:24.591213+0000","last_fullsized":"2023-11-26T07:18:24.591213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:16:21.071603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.578922+0000","last_change":"2023-11-26T07:18:24.578922+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.578922+0000","last_undegraded":"2023-11-26T07:18:24.578922+0000","last_fullsized":"2023-11-26T07:18:24.578922+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.596179+0000","last_change":"2023-11-26T07:18:24.593229+0000","last_active":"2023-11-26T07:18:24.596179+0000","last_peered":"2023-11-26T07:18:24.596179+0000","last_clean":"2023-11-26T07:18:24.596179+0000","last_became_active":"2023-11-26T07:18:24.592929+0000","last_became_peered":"2023-11-26T07:18:24.592929+0000","last_unstale":"2023-11-26T07:18:24.596179+0000","last_undegraded":"2023-11-26T07:18:24.596179+0000","last_fullsized":"2023-11-26T07:18:24.596179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:38.200743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2023-11-26T07:18:24.582610+0000","last_change":"2023-11-26T07:18:24.582610+0000","last_active":"2023-11-26T07:18:24.558201+0000","last_peered":"2023-11-26T07:18:24.558201+0000","last_clean":"2023-11-26T07:18:24.558201+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:18:24.582610+0000","last_undegraded":"2023-11-26T07:18:24.582610+0000","last_fullsized":"2023-11-26T07:18:24.582610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-11-26T07:18:24.594151+0000","last_change":"2023-11-26T07:18:24.591226+0000","last_active":"2023-11-26T07:18:24.594151+0000","last_peered":"2023-11-26T07:18:24.594151+0000","last_clean":"2023-11-26T07:18:24.594151+0000","last_became_active":"2023-11-26T07:18:24.590903+0000","last_became_peered":"2023-11-26T07:18:24.590903+0000","last_unstale":"2023-11-26T07:18:24.594151+0000","last_undegraded":"2023-11-26T07:18:24.594151+0000","last_fullsized":"2023-11-26T07:18:24.594151+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:46:59.788037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-11-26T07:18:23.565413+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:18:23.565413+0000","last_peered":"2023-11-26T07:18:23.565413+0000","last_clean":"2023-11-26T07:18:23.565413+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:18:23.565413+0000","last_undegraded":"2023-11-26T07:18:23.565413+0000","last_fullsized":"2023-11-26T07:18:23.565413+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":723755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.746},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61499999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.83799999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.72499999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.71199999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.60099999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.51500000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.69899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.72299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.625}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54400000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.80900000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.76400000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.85399999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.874},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.79200000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.90400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.57499999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.53800000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.753}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.81399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.62},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.41299999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.438},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67600000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.82799999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.76500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":127120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.38400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.316},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.439}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":8.0990000000000002,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.45100000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":4.431,"5min":4.431,"15min":4.431},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.625}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.83399999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.68899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42799999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.72899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.53300000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.55200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":722736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.442}]},{"osd":2,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57599999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.49199999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73799999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.751},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.71099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:34.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:18:34.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc5a0817640 1 Processor -- start 2023-11-26T07:18:34.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc5a0817640 1 -- start start 2023-11-26T07:18:34.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc5a0817640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc59c0d3760 0x7fc59c0d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc5a0817640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc59c0cfa20 0x7fc59c0cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc5a0817640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59c0d15f0 con 0x7fc59c15d880 2023-11-26T07:18:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc5a0817640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59c0d4110 con 0x7fc59c0d3760 2023-11-26T07:18:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc5a0817640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59c0d0390 con 0x7fc59c0cfa20 2023-11-26T07:18:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc59b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc59c15d880 0x7fc59c0b6730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57104/0 (socket says 172.21.15.31:57104) 2023-11-26T07:18:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc59b7fe640 1 -- 172.21.15.31:0/2985322416 learned_addr learned my addr 172.21.15.31:0/2985322416 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc59affd640 1 --2- 172.21.15.31:0/2985322416 >> v2:172.21.15.31:3300/0 conn(0x7fc59c0d3760 0x7fc59c0d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc59bfff640 1 --2- 172.21.15.31:0/2985322416 >> v2:172.21.15.97:3300/0 conn(0x7fc59c0cfa20 0x7fc59c0cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc59a7fc640 1 -- 172.21.15.31:0/2985322416 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717910492 0 0) 0x7fc59c0d15f0 con 0x7fc59c15d880 2023-11-26T07:18:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc59a7fc640 1 -- 172.21.15.31:0/2985322416 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc584003530 con 0x7fc59c15d880 2023-11-26T07:18:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc59affd640 1 -- 172.21.15.31:0/2985322416 >> v2:172.21.15.97:3300/0 conn(0x7fc59c0cfa20 msgr2=0x7fc59c0cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc59affd640 1 --2- 172.21.15.31:0/2985322416 >> v2:172.21.15.97:3300/0 conn(0x7fc59c0cfa20 0x7fc59c0cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc59affd640 1 -- 172.21.15.31:0/2985322416 >> v1:172.21.15.31:6789/0 conn(0x7fc59c15d880 legacy=0x7fc59c0b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.412+0000 7fc59affd640 1 -- 172.21.15.31:0/2985322416 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c0ce000 con 0x7fc59c0d3760 2023-11-26T07:18:34.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc59affd640 1 --2- 172.21.15.31:0/2985322416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c0d3760 0x7fc59c0d3bd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc59c08c580 tx=0x7fc59001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84aa49307caf4d25 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc59a7fc640 1 -- 172.21.15.31:0/2985322416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5900ab660 con 0x7fc59c0d3760 2023-11-26T07:18:34.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc59a7fc640 1 -- 172.21.15.31:0/2985322416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5900ab7c0 con 0x7fc59c0d3760 2023-11-26T07:18:34.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc59a7fc640 1 -- 172.21.15.31:0/2985322416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5900abb50 con 0x7fc59c0d3760 2023-11-26T07:18:34.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc5a0817640 1 -- 172.21.15.31:0/2985322416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c0d3760 msgr2=0x7fc59c0d3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc5a0817640 1 --2- 172.21.15.31:0/2985322416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c0d3760 0x7fc59c0d3bd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc59c08c580 tx=0x7fc59001c460 comp rx=0 tx=0).stop 2023-11-26T07:18:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc5a0817640 1 -- 172.21.15.31:0/2985322416 shutdown_connections 2023-11-26T07:18:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc5a0817640 1 --2- 172.21.15.31:0/2985322416 >> v2:172.21.15.97:3300/0 conn(0x7fc59c0cfa20 0x7fc59c0cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc5a0817640 1 --2- 172.21.15.31:0/2985322416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c0d3760 0x7fc59c0d3bd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc5a0817640 1 -- 172.21.15.31:0/2985322416 >> 172.21.15.31:0/2985322416 conn(0x7fc59c06b9f0 msgr2=0x7fc59c06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc5a0817640 1 -- 172.21.15.31:0/2985322416 shutdown_connections 2023-11-26T07:18:34.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc5a0817640 1 -- 172.21.15.31:0/2985322416 wait complete. 2023-11-26T07:18:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.416+0000 7fc5a0817640 1 Processor -- start 2023-11-26T07:18:34.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc5a0817640 1 -- start start 2023-11-26T07:18:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc5a0817640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc59c0cfa20 0x7fc59c1f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc5a0817640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc59c0d3760 0x7fc59c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc5a0817640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c15d880 0x7fc59c154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc5a0817640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c1f9e30 con 0x7fc59c15d880 2023-11-26T07:18:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc5a0817640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c158b20 con 0x7fc59c0d3760 2023-11-26T07:18:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc5a0817640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0c8c80 con 0x7fc59c0cfa20 2023-11-26T07:18:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c15d880 0x7fc59c154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c15d880 0x7fc59c154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55568/0 (socket says 172.21.15.31:55568) 2023-11-26T07:18:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59bfff640 1 -- 172.21.15.31:0/3572672310 learned_addr learned my addr 172.21.15.31:0/3572672310 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59b7fe640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc59c0cfa20 0x7fc59c1f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59affd640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc59c0d3760 0x7fc59c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59bfff640 1 -- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc59c0cfa20 msgr2=0x7fc59c1f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59bfff640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc59c0cfa20 0x7fc59c1f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59bfff640 1 -- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc59c0d3760 msgr2=0x7fc59c0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59bfff640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc59c0d3760 0x7fc59c0c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59bfff640 1 -- 172.21.15.31:0/3572672310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc590009420 con 0x7fc59c15d880 2023-11-26T07:18:34.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc59bfff640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c15d880 0x7fc59c154eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc59400efc0 tx=0x7fc59400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc583fff640 1 -- 172.21.15.31:0/3572672310 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594014480 con 0x7fc59c15d880 2023-11-26T07:18:34.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc583fff640 1 -- 172.21.15.31:0/3572672310 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59400f040 con 0x7fc59c15d880 2023-11-26T07:18:34.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc583fff640 1 -- 172.21.15.31:0/3572672310 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594014820 con 0x7fc59c15d880 2023-11-26T07:18:34.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc5a0817640 1 -- 172.21.15.31:0/3572672310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59c1553f0 con 0x7fc59c15d880 2023-11-26T07:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc5a0817640 1 -- 172.21.15.31:0/3572672310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59c155970 con 0x7fc59c15d880 2023-11-26T07:18:34.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.420+0000 7fc5a0817640 1 -- 172.21.15.31:0/3572672310 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59c158cb0 con 0x7fc59c15d880 2023-11-26T07:18:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.424+0000 7fc583fff640 1 -- 172.21.15.31:0/3572672310 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc594014980 con 0x7fc59c15d880 2023-11-26T07:18:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.424+0000 7fc583fff640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc5880664a0 0x7fc588068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.424+0000 7fc583fff640 1 -- 172.21.15.31:0/3572672310 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc594084970 con 0x7fc59c15d880 2023-11-26T07:18:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.424+0000 7fc59b7fe640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc5880664a0 0x7fc588068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.424+0000 7fc59b7fe640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc5880664a0 0x7fc588068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0011f0 tx=0x7fc58c007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.428+0000 7fc583fff640 1 -- 172.21.15.31:0/3572672310 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc594051e00 con 0x7fc59c15d880 2023-11-26T07:18:34.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.584+0000 7fc5a0817640 1 -- 172.21.15.31:0/3572672310 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc59c0d1b20 con 0x7fc5880664a0 2023-11-26T07:18:34.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.592+0000 7fc583fff640 1 -- 172.21.15.31:0/3572672310 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288870 (secure 0 0 0) 0x7fc58c001470 con 0x7fc5880664a0 2023-11-26T07:18:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:34.604 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:18:34.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 -- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc5880664a0 msgr2=0x7fc588068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc5880664a0 0x7fc588068960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc58c0011f0 tx=0x7fc58c007d40 comp rx=0 tx=0).stop 2023-11-26T07:18:34.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 -- 172.21.15.31:0/3572672310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c15d880 msgr2=0x7fc59c154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c15d880 0x7fc59c154eb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc59400efc0 tx=0x7fc59400c490 comp rx=0 tx=0).stop 2023-11-26T07:18:34.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 -- 172.21.15.31:0/3572672310 shutdown_connections 2023-11-26T07:18:34.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc5880664a0 0x7fc588068960 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc59c15d880 0x7fc59c154eb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc59c0d3760 0x7fc59c0c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 --2- 172.21.15.31:0/3572672310 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc59c0cfa20 0x7fc59c1f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 -- 172.21.15.31:0/3572672310 >> 172.21.15.31:0/3572672310 conn(0x7fc59c06b9f0 msgr2=0x7fc59c1601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:34.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 -- 172.21.15.31:0/3572672310 shutdown_connections 2023-11-26T07:18:34.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.596+0000 7fc5a0817640 1 -- 172.21.15.31:0/3572672310 wait complete. 2023-11-26T07:18:34.622 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-11-26T07:18:34.321449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220328,"kb_used_data":4132,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754392,"statfs":{"total":773094113280,"available":772868497408,"internally_reserved":0,"allocated":4231168,"data_stored":2356233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.753360"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572846+0000","last_change":"2023-11-26T07:18:25.577585+0000","last_active":"2023-11-26T07:18:26.572846+0000","last_peered":"2023-11-26T07:18:26.572846+0000","last_clean":"2023-11-26T07:18:26.572846+0000","last_became_active":"2023-11-26T07:18:25.577168+0000","last_became_peered":"2023-11-26T07:18:25.577168+0000","last_unstale":"2023-11-26T07:18:26.572846+0000","last_undegraded":"2023-11-26T07:18:26.572846+0000","last_fullsized":"2023-11-26T07:18:26.572846+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:53:08.264292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.231136+0000","last_change":"2023-11-26T07:18:25.601065+0000","last_active":"2023-11-26T07:18:27.231136+0000","last_peered":"2023-11-26T07:18:27.231136+0000","last_clean":"2023-11-26T07:18:27.231136+0000","last_became_active":"2023-11-26T07:18:25.600339+0000","last_became_peered":"2023-11-26T07:18:25.600339+0000","last_unstale":"2023-11-26T07:18:27.231136+0000","last_undegraded":"2023-11-26T07:18:27.231136+0000","last_fullsized":"2023-11-26T07:18:27.231136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:48:49.369077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.229035+0000","last_change":"2023-11-26T07:18:25.584087+0000","last_active":"2023-11-26T07:18:27.229035+0000","last_peered":"2023-11-26T07:18:27.229035+0000","last_clean":"2023-11-26T07:18:27.229035+0000","last_became_active":"2023-11-26T07:18:25.583120+0000","last_became_peered":"2023-11-26T07:18:25.583120+0000","last_unstale":"2023-11-26T07:18:27.229035+0000","last_undegraded":"2023-11-26T07:18:27.229035+0000","last_fullsized":"2023-11-26T07:18:27.229035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:08:19.267272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230760+0000","last_change":"2023-11-26T07:18:25.608288+0000","last_active":"2023-11-26T07:18:27.230760+0000","last_peered":"2023-11-26T07:18:27.230760+0000","last_clean":"2023-11-26T07:18:27.230760+0000","last_became_active":"2023-11-26T07:18:25.607809+0000","last_became_peered":"2023-11-26T07:18:25.607809+0000","last_unstale":"2023-11-26T07:18:27.230760+0000","last_undegraded":"2023-11-26T07:18:27.230760+0000","last_fullsized":"2023-11-26T07:18:27.230760+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:11:58.141078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572138+0000","last_change":"2023-11-26T07:18:25.604101+0000","last_active":"2023-11-26T07:18:26.572138+0000","last_peered":"2023-11-26T07:18:26.572138+0000","last_clean":"2023-11-26T07:18:26.572138+0000","last_became_active":"2023-11-26T07:18:25.603815+0000","last_became_peered":"2023-11-26T07:18:25.603815+0000","last_unstale":"2023-11-26T07:18:26.572138+0000","last_undegraded":"2023-11-26T07:18:26.572138+0000","last_fullsized":"2023-11-26T07:18:26.572138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:14.764810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230053+0000","last_change":"2023-11-26T07:18:25.602729+0000","last_active":"2023-11-26T07:18:27.230053+0000","last_peered":"2023-11-26T07:18:27.230053+0000","last_clean":"2023-11-26T07:18:27.230053+0000","last_became_active":"2023-11-26T07:18:25.600515+0000","last_became_peered":"2023-11-26T07:18:25.600515+0000","last_unstale":"2023-11-26T07:18:27.230053+0000","last_undegraded":"2023-11-26T07:18:27.230053+0000","last_fullsized":"2023-11-26T07:18:27.230053+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:27:47.448055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230161+0000","last_change":"2023-11-26T07:18:25.597792+0000","last_active":"2023-11-26T07:18:27.230161+0000","last_peered":"2023-11-26T07:18:27.230161+0000","last_clean":"2023-11-26T07:18:27.230161+0000","last_became_active":"2023-11-26T07:18:25.597435+0000","last_became_peered":"2023-11-26T07:18:25.597435+0000","last_unstale":"2023-11-26T07:18:27.230161+0000","last_undegraded":"2023-11-26T07:18:27.230161+0000","last_fullsized":"2023-11-26T07:18:27.230161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:38:27.653798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.229801+0000","last_change":"2023-11-26T07:18:25.608357+0000","last_active":"2023-11-26T07:18:27.229801+0000","last_peered":"2023-11-26T07:18:27.229801+0000","last_clean":"2023-11-26T07:18:27.229801+0000","last_became_active":"2023-11-26T07:18:25.608088+0000","last_became_peered":"2023-11-26T07:18:25.608088+0000","last_unstale":"2023-11-26T07:18:27.229801+0000","last_undegraded":"2023-11-26T07:18:27.229801+0000","last_fullsized":"2023-11-26T07:18:27.229801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:43:13.913095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.573361+0000","last_change":"2023-11-26T07:18:25.609312+0000","last_active":"2023-11-26T07:18:26.573361+0000","last_peered":"2023-11-26T07:18:26.573361+0000","last_clean":"2023-11-26T07:18:26.573361+0000","last_became_active":"2023-11-26T07:18:25.609040+0000","last_became_peered":"2023-11-26T07:18:25.609040+0000","last_unstale":"2023-11-26T07:18:26.573361+0000","last_undegraded":"2023-11-26T07:18:26.573361+0000","last_fullsized":"2023-11-26T07:18:26.573361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:36:47.106097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230536+0000","last_change":"2023-11-26T07:18:25.607575+0000","last_active":"2023-11-26T07:18:27.230536+0000","last_peered":"2023-11-26T07:18:27.230536+0000","last_clean":"2023-11-26T07:18:27.230536+0000","last_became_active":"2023-11-26T07:18:25.607264+0000","last_became_peered":"2023-11-26T07:18:25.607264+0000","last_unstale":"2023-11-26T07:18:27.230536+0000","last_undegraded":"2023-11-26T07:18:27.230536+0000","last_fullsized":"2023-11-26T07:18:27.230536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:36:47.106097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230692+0000","last_change":"2023-11-26T07:18:25.614229+0000","last_active":"2023-11-26T07:18:27.230692+0000","last_peered":"2023-11-26T07:18:27.230692+0000","last_clean":"2023-11-26T07:18:27.230692+0000","last_became_active":"2023-11-26T07:18:25.613985+0000","last_became_peered":"2023-11-26T07:18:25.613985+0000","last_unstale":"2023-11-26T07:18:27.230692+0000","last_undegraded":"2023-11-26T07:18:27.230692+0000","last_fullsized":"2023-11-26T07:18:27.230692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:23:04.014784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.228845+0000","last_change":"2023-11-26T07:18:25.607161+0000","last_active":"2023-11-26T07:18:27.228845+0000","last_peered":"2023-11-26T07:18:27.228845+0000","last_clean":"2023-11-26T07:18:27.228845+0000","last_became_active":"2023-11-26T07:18:25.606704+0000","last_became_peered":"2023-11-26T07:18:25.606704+0000","last_unstale":"2023-11-26T07:18:27.228845+0000","last_undegraded":"2023-11-26T07:18:27.228845+0000","last_fullsized":"2023-11-26T07:18:27.228845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:37.447188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572139+0000","last_change":"2023-11-26T07:18:25.611768+0000","last_active":"2023-11-26T07:18:26.572139+0000","last_peered":"2023-11-26T07:18:26.572139+0000","last_clean":"2023-11-26T07:18:26.572139+0000","last_became_active":"2023-11-26T07:18:25.611527+0000","last_became_peered":"2023-11-26T07:18:25.611527+0000","last_unstale":"2023-11-26T07:18:26.572139+0000","last_undegraded":"2023-11-26T07:18:26.572139+0000","last_fullsized":"2023-11-26T07:18:26.572139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:22.478031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230182+0000","last_change":"2023-11-26T07:18:25.600298+0000","last_active":"2023-11-26T07:18:27.230182+0000","last_peered":"2023-11-26T07:18:27.230182+0000","last_clean":"2023-11-26T07:18:27.230182+0000","last_became_active":"2023-11-26T07:18:25.595956+0000","last_became_peered":"2023-11-26T07:18:25.595956+0000","last_unstale":"2023-11-26T07:18:27.230182+0000","last_undegraded":"2023-11-26T07:18:27.230182+0000","last_fullsized":"2023-11-26T07:18:27.230182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:55:21.347422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.228689+0000","last_change":"2023-11-26T07:18:25.576246+0000","last_active":"2023-11-26T07:18:27.228689+0000","last_peered":"2023-11-26T07:18:27.228689+0000","last_clean":"2023-11-26T07:18:27.228689+0000","last_became_active":"2023-11-26T07:18:25.575357+0000","last_became_peered":"2023-11-26T07:18:25.575357+0000","last_unstale":"2023-11-26T07:18:27.228689+0000","last_undegraded":"2023-11-26T07:18:27.228689+0000","last_fullsized":"2023-11-26T07:18:27.228689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:43:13.913095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230233+0000","last_change":"2023-11-26T07:18:25.610185+0000","last_active":"2023-11-26T07:18:27.230233+0000","last_peered":"2023-11-26T07:18:27.230233+0000","last_clean":"2023-11-26T07:18:27.230233+0000","last_became_active":"2023-11-26T07:18:25.609504+0000","last_became_peered":"2023-11-26T07:18:25.609504+0000","last_unstale":"2023-11-26T07:18:27.230233+0000","last_undegraded":"2023-11-26T07:18:27.230233+0000","last_fullsized":"2023-11-26T07:18:27.230233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:48:49.369077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230397+0000","last_change":"2023-11-26T07:18:25.595669+0000","last_active":"2023-11-26T07:18:27.230397+0000","last_peered":"2023-11-26T07:18:27.230397+0000","last_clean":"2023-11-26T07:18:27.230397+0000","last_became_active":"2023-11-26T07:18:25.595300+0000","last_became_peered":"2023-11-26T07:18:25.595300+0000","last_unstale":"2023-11-26T07:18:27.230397+0000","last_undegraded":"2023-11-26T07:18:27.230397+0000","last_fullsized":"2023-11-26T07:18:27.230397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:42.303842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.231255+0000","last_change":"2023-11-26T07:18:25.611589+0000","last_active":"2023-11-26T07:18:27.231255+0000","last_peered":"2023-11-26T07:18:27.231255+0000","last_clean":"2023-11-26T07:18:27.231255+0000","last_became_active":"2023-11-26T07:18:25.611298+0000","last_became_peered":"2023-11-26T07:18:25.611298+0000","last_unstale":"2023-11-26T07:18:27.231255+0000","last_undegraded":"2023-11-26T07:18:27.231255+0000","last_fullsized":"2023-11-26T07:18:27.231255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:12.355159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230098+0000","last_change":"2023-11-26T07:18:25.614022+0000","last_active":"2023-11-26T07:18:27.230098+0000","last_peered":"2023-11-26T07:18:27.230098+0000","last_clean":"2023-11-26T07:18:27.230098+0000","last_became_active":"2023-11-26T07:18:25.613707+0000","last_became_peered":"2023-11-26T07:18:25.613707+0000","last_unstale":"2023-11-26T07:18:27.230098+0000","last_undegraded":"2023-11-26T07:18:27.230098+0000","last_fullsized":"2023-11-26T07:18:27.230098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:37.447188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572298+0000","last_change":"2023-11-26T07:18:25.593263+0000","last_active":"2023-11-26T07:18:26.572298+0000","last_peered":"2023-11-26T07:18:26.572298+0000","last_clean":"2023-11-26T07:18:26.572298+0000","last_became_active":"2023-11-26T07:18:25.592061+0000","last_became_peered":"2023-11-26T07:18:25.592061+0000","last_unstale":"2023-11-26T07:18:26.572298+0000","last_undegraded":"2023-11-26T07:18:26.572298+0000","last_fullsized":"2023-11-26T07:18:26.572298+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:13:39.208244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572016+0000","last_change":"2023-11-26T07:18:25.604014+0000","last_active":"2023-11-26T07:18:26.572016+0000","last_peered":"2023-11-26T07:18:26.572016+0000","last_clean":"2023-11-26T07:18:26.572016+0000","last_became_active":"2023-11-26T07:18:25.603689+0000","last_became_peered":"2023-11-26T07:18:25.603689+0000","last_unstale":"2023-11-26T07:18:26.572016+0000","last_undegraded":"2023-11-26T07:18:26.572016+0000","last_fullsized":"2023-11-26T07:18:26.572016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:16:50.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572125+0000","last_change":"2023-11-26T07:18:25.601234+0000","last_active":"2023-11-26T07:18:26.572125+0000","last_peered":"2023-11-26T07:18:26.572125+0000","last_clean":"2023-11-26T07:18:26.572125+0000","last_became_active":"2023-11-26T07:18:25.600928+0000","last_became_peered":"2023-11-26T07:18:25.600928+0000","last_unstale":"2023-11-26T07:18:26.572125+0000","last_undegraded":"2023-11-26T07:18:26.572125+0000","last_fullsized":"2023-11-26T07:18:26.572125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:45:38.690049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.229598+0000","last_change":"2023-11-26T07:18:25.592182+0000","last_active":"2023-11-26T07:18:27.229598+0000","last_peered":"2023-11-26T07:18:27.229598+0000","last_clean":"2023-11-26T07:18:27.229598+0000","last_became_active":"2023-11-26T07:18:25.591864+0000","last_became_peered":"2023-11-26T07:18:25.591864+0000","last_unstale":"2023-11-26T07:18:27.229598+0000","last_undegraded":"2023-11-26T07:18:27.229598+0000","last_fullsized":"2023-11-26T07:18:27.229598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:16:50.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.228555+0000","last_change":"2023-11-26T07:18:25.587158+0000","last_active":"2023-11-26T07:18:27.228555+0000","last_peered":"2023-11-26T07:18:27.228555+0000","last_clean":"2023-11-26T07:18:27.228555+0000","last_became_active":"2023-11-26T07:18:25.586847+0000","last_became_peered":"2023-11-26T07:18:25.586847+0000","last_unstale":"2023-11-26T07:18:27.228555+0000","last_undegraded":"2023-11-26T07:18:27.228555+0000","last_fullsized":"2023-11-26T07:18:27.228555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:27:47.448055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230624+0000","last_change":"2023-11-26T07:18:25.593861+0000","last_active":"2023-11-26T07:18:27.230624+0000","last_peered":"2023-11-26T07:18:27.230624+0000","last_clean":"2023-11-26T07:18:27.230624+0000","last_became_active":"2023-11-26T07:18:25.592882+0000","last_became_peered":"2023-11-26T07:18:25.592882+0000","last_unstale":"2023-11-26T07:18:27.230624+0000","last_undegraded":"2023-11-26T07:18:27.230624+0000","last_fullsized":"2023-11-26T07:18:27.230624+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:51.434007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.231205+0000","last_change":"2023-11-26T07:18:25.600872+0000","last_active":"2023-11-26T07:18:27.231205+0000","last_peered":"2023-11-26T07:18:27.231205+0000","last_clean":"2023-11-26T07:18:27.231205+0000","last_became_active":"2023-11-26T07:18:25.600150+0000","last_became_peered":"2023-11-26T07:18:25.600150+0000","last_unstale":"2023-11-26T07:18:27.231205+0000","last_undegraded":"2023-11-26T07:18:27.231205+0000","last_fullsized":"2023-11-26T07:18:27.231205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:42.303842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230921+0000","last_change":"2023-11-26T07:18:25.608008+0000","last_active":"2023-11-26T07:18:27.230921+0000","last_peered":"2023-11-26T07:18:27.230921+0000","last_clean":"2023-11-26T07:18:27.230921+0000","last_became_active":"2023-11-26T07:18:25.607238+0000","last_became_peered":"2023-11-26T07:18:25.607238+0000","last_unstale":"2023-11-26T07:18:27.230921+0000","last_undegraded":"2023-11-26T07:18:27.230921+0000","last_fullsized":"2023-11-26T07:18:27.230921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:24:25.111217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.228877+0000","last_change":"2023-11-26T07:18:25.597767+0000","last_active":"2023-11-26T07:18:27.228877+0000","last_peered":"2023-11-26T07:18:27.228877+0000","last_clean":"2023-11-26T07:18:27.228877+0000","last_became_active":"2023-11-26T07:18:25.597490+0000","last_became_peered":"2023-11-26T07:18:25.597490+0000","last_unstale":"2023-11-26T07:18:27.228877+0000","last_undegraded":"2023-11-26T07:18:27.228877+0000","last_fullsized":"2023-11-26T07:18:27.228877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:55:21.347422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572168+0000","last_change":"2023-11-26T07:18:25.604858+0000","last_active":"2023-11-26T07:18:26.572168+0000","last_peered":"2023-11-26T07:18:26.572168+0000","last_clean":"2023-11-26T07:18:26.572168+0000","last_became_active":"2023-11-26T07:18:25.603308+0000","last_became_peered":"2023-11-26T07:18:25.603308+0000","last_unstale":"2023-11-26T07:18:26.572168+0000","last_undegraded":"2023-11-26T07:18:26.572168+0000","last_fullsized":"2023-11-26T07:18:26.572168+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:57:53.233364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230226+0000","last_change":"2023-11-26T07:18:25.590333+0000","last_active":"2023-11-26T07:18:27.230226+0000","last_peered":"2023-11-26T07:18:27.230226+0000","last_clean":"2023-11-26T07:18:27.230226+0000","last_became_active":"2023-11-26T07:18:25.590026+0000","last_became_peered":"2023-11-26T07:18:25.590026+0000","last_unstale":"2023-11-26T07:18:27.230226+0000","last_undegraded":"2023-11-26T07:18:27.230226+0000","last_fullsized":"2023-11-26T07:18:27.230226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:08:19.267272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572847+0000","last_change":"2023-11-26T07:18:25.588646+0000","last_active":"2023-11-26T07:18:26.572847+0000","last_peered":"2023-11-26T07:18:26.572847+0000","last_clean":"2023-11-26T07:18:26.572847+0000","last_became_active":"2023-11-26T07:18:25.587792+0000","last_became_peered":"2023-11-26T07:18:25.587792+0000","last_unstale":"2023-11-26T07:18:26.572847+0000","last_undegraded":"2023-11-26T07:18:26.572847+0000","last_fullsized":"2023-11-26T07:18:26.572847+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:51.254616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.229844+0000","last_change":"2023-11-26T07:18:25.610636+0000","last_active":"2023-11-26T07:18:27.229844+0000","last_peered":"2023-11-26T07:18:27.229844+0000","last_clean":"2023-11-26T07:18:27.229844+0000","last_became_active":"2023-11-26T07:18:25.610370+0000","last_became_peered":"2023-11-26T07:18:25.610370+0000","last_unstale":"2023-11-26T07:18:27.229844+0000","last_undegraded":"2023-11-26T07:18:27.229844+0000","last_fullsized":"2023-11-26T07:18:27.229844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:27:47.448055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.231409+0000","last_change":"2023-11-26T07:18:25.598460+0000","last_active":"2023-11-26T07:18:27.231409+0000","last_peered":"2023-11-26T07:18:27.231409+0000","last_clean":"2023-11-26T07:18:27.231409+0000","last_became_active":"2023-11-26T07:18:25.597862+0000","last_became_peered":"2023-11-26T07:18:25.597862+0000","last_unstale":"2023-11-26T07:18:27.231409+0000","last_undegraded":"2023-11-26T07:18:27.231409+0000","last_fullsized":"2023-11-26T07:18:27.231409+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:57:53.233364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.591735+0000","last_change":"2023-11-26T07:18:24.586219+0000","last_active":"2023-11-26T07:18:25.591735+0000","last_peered":"2023-11-26T07:18:25.591735+0000","last_clean":"2023-11-26T07:18:25.591735+0000","last_became_active":"2023-11-26T07:18:24.584015+0000","last_became_peered":"2023-11-26T07:18:24.584015+0000","last_unstale":"2023-11-26T07:18:25.591735+0000","last_undegraded":"2023-11-26T07:18:25.591735+0000","last_fullsized":"2023-11-26T07:18:25.591735+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:39:49.112182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572993+0000","last_change":"2023-11-26T07:18:25.599296+0000","last_active":"2023-11-26T07:18:26.572993+0000","last_peered":"2023-11-26T07:18:26.572993+0000","last_clean":"2023-11-26T07:18:26.572993+0000","last_became_active":"2023-11-26T07:18:25.599009+0000","last_became_peered":"2023-11-26T07:18:25.599009+0000","last_unstale":"2023-11-26T07:18:26.572993+0000","last_undegraded":"2023-11-26T07:18:26.572993+0000","last_fullsized":"2023-11-26T07:18:26.572993+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:03:35.641061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.599178+0000","last_change":"2023-11-26T07:18:24.592940+0000","last_active":"2023-11-26T07:18:25.599178+0000","last_peered":"2023-11-26T07:18:25.599178+0000","last_clean":"2023-11-26T07:18:25.599178+0000","last_became_active":"2023-11-26T07:18:24.592670+0000","last_became_peered":"2023-11-26T07:18:24.592670+0000","last_unstale":"2023-11-26T07:18:25.599178+0000","last_undegraded":"2023-11-26T07:18:25.599178+0000","last_fullsized":"2023-11-26T07:18:25.599178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:53:07.256791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572508+0000","last_change":"2023-11-26T07:18:25.607463+0000","last_active":"2023-11-26T07:18:26.572508+0000","last_peered":"2023-11-26T07:18:26.572508+0000","last_clean":"2023-11-26T07:18:26.572508+0000","last_became_active":"2023-11-26T07:18:25.607215+0000","last_became_peered":"2023-11-26T07:18:25.607215+0000","last_unstale":"2023-11-26T07:18:26.572508+0000","last_undegraded":"2023-11-26T07:18:26.572508+0000","last_fullsized":"2023-11-26T07:18:26.572508+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:12:10.758526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.616660+0000","last_change":"2023-11-26T07:18:24.594822+0000","last_active":"2023-11-26T07:18:25.616660+0000","last_peered":"2023-11-26T07:18:25.616660+0000","last_clean":"2023-11-26T07:18:25.616660+0000","last_became_active":"2023-11-26T07:18:24.594466+0000","last_became_peered":"2023-11-26T07:18:24.594466+0000","last_unstale":"2023-11-26T07:18:25.616660+0000","last_undegraded":"2023-11-26T07:18:25.616660+0000","last_fullsized":"2023-11-26T07:18:25.616660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:24:24.103716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.231075+0000","last_change":"2023-11-26T07:18:25.608196+0000","last_active":"2023-11-26T07:18:27.231075+0000","last_peered":"2023-11-26T07:18:27.231075+0000","last_clean":"2023-11-26T07:18:27.231075+0000","last_became_active":"2023-11-26T07:18:25.607715+0000","last_became_peered":"2023-11-26T07:18:25.607715+0000","last_unstale":"2023-11-26T07:18:27.231075+0000","last_undegraded":"2023-11-26T07:18:27.231075+0000","last_fullsized":"2023-11-26T07:18:27.231075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:44:32.702096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.610349+0000","last_change":"2023-11-26T07:18:24.590286+0000","last_active":"2023-11-26T07:18:25.610349+0000","last_peered":"2023-11-26T07:18:25.610349+0000","last_clean":"2023-11-26T07:18:25.610349+0000","last_became_active":"2023-11-26T07:18:24.589675+0000","last_became_peered":"2023-11-26T07:18:24.589675+0000","last_unstale":"2023-11-26T07:18:25.610349+0000","last_undegraded":"2023-11-26T07:18:25.610349+0000","last_fullsized":"2023-11-26T07:18:25.610349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:28.295792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230140+0000","last_change":"2023-11-26T07:18:25.604973+0000","last_active":"2023-11-26T07:18:27.230140+0000","last_peered":"2023-11-26T07:18:27.230140+0000","last_clean":"2023-11-26T07:18:27.230140+0000","last_became_active":"2023-11-26T07:18:25.604718+0000","last_became_peered":"2023-11-26T07:18:25.604718+0000","last_unstale":"2023-11-26T07:18:27.230140+0000","last_undegraded":"2023-11-26T07:18:27.230140+0000","last_fullsized":"2023-11-26T07:18:27.230140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:20:13.131760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.612955+0000","last_change":"2023-11-26T07:18:24.588499+0000","last_active":"2023-11-26T07:18:25.612955+0000","last_peered":"2023-11-26T07:18:25.612955+0000","last_clean":"2023-11-26T07:18:25.612955+0000","last_became_active":"2023-11-26T07:18:24.588209+0000","last_became_peered":"2023-11-26T07:18:24.588209+0000","last_unstale":"2023-11-26T07:18:25.612955+0000","last_undegraded":"2023-11-26T07:18:25.612955+0000","last_fullsized":"2023-11-26T07:18:25.612955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:25:16.010749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.228834+0000","last_change":"2023-11-26T07:18:25.598504+0000","last_active":"2023-11-26T07:18:27.228834+0000","last_peered":"2023-11-26T07:18:27.228834+0000","last_clean":"2023-11-26T07:18:27.228834+0000","last_became_active":"2023-11-26T07:18:25.598214+0000","last_became_peered":"2023-11-26T07:18:25.598214+0000","last_unstale":"2023-11-26T07:18:27.228834+0000","last_undegraded":"2023-11-26T07:18:27.228834+0000","last_fullsized":"2023-11-26T07:18:27.228834+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:20:13.131760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.611007+0000","last_change":"2023-11-26T07:18:24.593041+0000","last_active":"2023-11-26T07:18:25.611007+0000","last_peered":"2023-11-26T07:18:25.611007+0000","last_clean":"2023-11-26T07:18:25.611007+0000","last_became_active":"2023-11-26T07:18:24.592774+0000","last_became_peered":"2023-11-26T07:18:24.592774+0000","last_unstale":"2023-11-26T07:18:25.611007+0000","last_undegraded":"2023-11-26T07:18:25.611007+0000","last_fullsized":"2023-11-26T07:18:25.611007+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:14:55.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.571911+0000","last_change":"2023-11-26T07:18:25.602968+0000","last_active":"2023-11-26T07:18:26.571911+0000","last_peered":"2023-11-26T07:18:26.571911+0000","last_clean":"2023-11-26T07:18:26.571911+0000","last_became_active":"2023-11-26T07:18:25.602702+0000","last_became_peered":"2023-11-26T07:18:25.602702+0000","last_unstale":"2023-11-26T07:18:26.571911+0000","last_undegraded":"2023-11-26T07:18:26.571911+0000","last_fullsized":"2023-11-26T07:18:26.571911+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:03:35.641061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.596219+0000","last_change":"2023-11-26T07:18:24.586769+0000","last_active":"2023-11-26T07:18:25.596219+0000","last_peered":"2023-11-26T07:18:25.596219+0000","last_clean":"2023-11-26T07:18:25.596219+0000","last_became_active":"2023-11-26T07:18:24.586327+0000","last_became_peered":"2023-11-26T07:18:24.586327+0000","last_unstale":"2023-11-26T07:18:25.596219+0000","last_undegraded":"2023-11-26T07:18:25.596219+0000","last_fullsized":"2023-11-26T07:18:25.596219+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:43:12.905594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230004+0000","last_change":"2023-11-26T07:18:25.579385+0000","last_active":"2023-11-26T07:18:27.230004+0000","last_peered":"2023-11-26T07:18:27.230004+0000","last_clean":"2023-11-26T07:18:27.230004+0000","last_became_active":"2023-11-26T07:18:25.579170+0000","last_became_peered":"2023-11-26T07:18:25.579170+0000","last_unstale":"2023-11-26T07:18:27.230004+0000","last_undegraded":"2023-11-26T07:18:27.230004+0000","last_fullsized":"2023-11-26T07:18:27.230004+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:12:10.758526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.583938+0000","last_change":"2023-11-26T07:18:24.581295+0000","last_active":"2023-11-26T07:18:25.583938+0000","last_peered":"2023-11-26T07:18:25.583938+0000","last_clean":"2023-11-26T07:18:25.583938+0000","last_became_active":"2023-11-26T07:18:24.580773+0000","last_became_peered":"2023-11-26T07:18:24.580773+0000","last_unstale":"2023-11-26T07:18:25.583938+0000","last_undegraded":"2023-11-26T07:18:25.583938+0000","last_fullsized":"2023-11-26T07:18:25.583938+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:16.638606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.229094+0000","last_change":"2023-11-26T07:18:25.602757+0000","last_active":"2023-11-26T07:18:27.229094+0000","last_peered":"2023-11-26T07:18:27.229094+0000","last_clean":"2023-11-26T07:18:27.229094+0000","last_became_active":"2023-11-26T07:18:25.602449+0000","last_became_peered":"2023-11-26T07:18:25.602449+0000","last_unstale":"2023-11-26T07:18:27.229094+0000","last_undegraded":"2023-11-26T07:18:27.229094+0000","last_fullsized":"2023-11-26T07:18:27.229094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:48:49.369077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.596179+0000","last_change":"2023-11-26T07:18:24.592314+0000","last_active":"2023-11-26T07:18:25.596179+0000","last_peered":"2023-11-26T07:18:25.596179+0000","last_clean":"2023-11-26T07:18:25.596179+0000","last_became_active":"2023-11-26T07:18:24.591990+0000","last_became_peered":"2023-11-26T07:18:24.591990+0000","last_unstale":"2023-11-26T07:18:25.596179+0000","last_undegraded":"2023-11-26T07:18:25.596179+0000","last_fullsized":"2023-11-26T07:18:25.596179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:44:31.694595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230425+0000","last_change":"2023-11-26T07:18:25.612082+0000","last_active":"2023-11-26T07:18:27.230425+0000","last_peered":"2023-11-26T07:18:27.230425+0000","last_clean":"2023-11-26T07:18:27.230425+0000","last_became_active":"2023-11-26T07:18:25.611770+0000","last_became_peered":"2023-11-26T07:18:25.611770+0000","last_unstale":"2023-11-26T07:18:27.230425+0000","last_undegraded":"2023-11-26T07:18:27.230425+0000","last_fullsized":"2023-11-26T07:18:27.230425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:38:27.653798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.586876+0000","last_change":"2023-11-26T07:18:24.589024+0000","last_active":"2023-11-26T07:18:25.586876+0000","last_peered":"2023-11-26T07:18:25.586876+0000","last_clean":"2023-11-26T07:18:25.586876+0000","last_became_active":"2023-11-26T07:18:24.585478+0000","last_became_peered":"2023-11-26T07:18:24.585478+0000","last_unstale":"2023-11-26T07:18:25.586876+0000","last_undegraded":"2023-11-26T07:18:25.586876+0000","last_fullsized":"2023-11-26T07:18:25.586876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:03:34.633560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230383+0000","last_change":"2023-11-26T07:18:25.603855+0000","last_active":"2023-11-26T07:18:27.230383+0000","last_peered":"2023-11-26T07:18:27.230383+0000","last_clean":"2023-11-26T07:18:27.230383+0000","last_became_active":"2023-11-26T07:18:25.599162+0000","last_became_peered":"2023-11-26T07:18:25.599162+0000","last_unstale":"2023-11-26T07:18:27.230383+0000","last_undegraded":"2023-11-26T07:18:27.230383+0000","last_fullsized":"2023-11-26T07:18:27.230383+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:20:13.131760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.596475+0000","last_change":"2023-11-26T07:18:24.592183+0000","last_active":"2023-11-26T07:18:25.596475+0000","last_peered":"2023-11-26T07:18:25.596475+0000","last_clean":"2023-11-26T07:18:25.596475+0000","last_became_active":"2023-11-26T07:18:24.591338+0000","last_became_peered":"2023-11-26T07:18:24.591338+0000","last_unstale":"2023-11-26T07:18:25.596475+0000","last_undegraded":"2023-11-26T07:18:25.596475+0000","last_fullsized":"2023-11-26T07:18:25.596475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:20:04.215891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.228936+0000","last_change":"2023-11-26T07:18:25.588432+0000","last_active":"2023-11-26T07:18:27.228936+0000","last_peered":"2023-11-26T07:18:27.228936+0000","last_clean":"2023-11-26T07:18:27.228936+0000","last_became_active":"2023-11-26T07:18:25.588114+0000","last_became_peered":"2023-11-26T07:18:25.588114+0000","last_unstale":"2023-11-26T07:18:27.228936+0000","last_undegraded":"2023-11-26T07:18:27.228936+0000","last_fullsized":"2023-11-26T07:18:27.228936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:38:27.653798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.619402+0000","last_change":"2023-11-26T07:18:24.594736+0000","last_active":"2023-11-26T07:18:25.619402+0000","last_peered":"2023-11-26T07:18:25.619402+0000","last_clean":"2023-11-26T07:18:25.619402+0000","last_became_active":"2023-11-26T07:18:24.594398+0000","last_became_peered":"2023-11-26T07:18:24.594398+0000","last_unstale":"2023-11-26T07:18:25.619402+0000","last_undegraded":"2023-11-26T07:18:25.619402+0000","last_fullsized":"2023-11-26T07:18:25.619402+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:20:04.215891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230223+0000","last_change":"2023-11-26T07:18:25.579490+0000","last_active":"2023-11-26T07:18:27.230223+0000","last_peered":"2023-11-26T07:18:27.230223+0000","last_clean":"2023-11-26T07:18:27.230223+0000","last_became_active":"2023-11-26T07:18:25.579224+0000","last_became_peered":"2023-11-26T07:18:25.579224+0000","last_unstale":"2023-11-26T07:18:27.230223+0000","last_undegraded":"2023-11-26T07:18:27.230223+0000","last_fullsized":"2023-11-26T07:18:27.230223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:06:19.891238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.610100+0000","last_change":"2023-11-26T07:18:24.583313+0000","last_active":"2023-11-26T07:18:25.610100+0000","last_peered":"2023-11-26T07:18:25.610100+0000","last_clean":"2023-11-26T07:18:25.610100+0000","last_became_active":"2023-11-26T07:18:24.583028+0000","last_became_peered":"2023-11-26T07:18:24.583028+0000","last_unstale":"2023-11-26T07:18:25.610100+0000","last_undegraded":"2023-11-26T07:18:25.610100+0000","last_fullsized":"2023-11-26T07:18:25.610100+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:33.810259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230736+0000","last_change":"2023-11-26T07:18:25.600844+0000","last_active":"2023-11-26T07:18:27.230736+0000","last_peered":"2023-11-26T07:18:27.230736+0000","last_clean":"2023-11-26T07:18:27.230736+0000","last_became_active":"2023-11-26T07:18:25.600393+0000","last_became_peered":"2023-11-26T07:18:25.600393+0000","last_unstale":"2023-11-26T07:18:27.230736+0000","last_undegraded":"2023-11-26T07:18:27.230736+0000","last_fullsized":"2023-11-26T07:18:27.230736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:08:19.267272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.573428+0000","last_change":"2023-11-26T07:18:24.581041+0000","last_active":"2023-11-26T07:18:25.573428+0000","last_peered":"2023-11-26T07:18:25.573428+0000","last_clean":"2023-11-26T07:18:25.573428+0000","last_became_active":"2023-11-26T07:18:24.580753+0000","last_became_peered":"2023-11-26T07:18:24.580753+0000","last_unstale":"2023-11-26T07:18:25.573428+0000","last_undegraded":"2023-11-26T07:18:25.573428+0000","last_fullsized":"2023-11-26T07:18:25.573428+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:50.247115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230060+0000","last_change":"2023-11-26T07:18:25.610686+0000","last_active":"2023-11-26T07:18:27.230060+0000","last_peered":"2023-11-26T07:18:27.230060+0000","last_clean":"2023-11-26T07:18:27.230060+0000","last_became_active":"2023-11-26T07:18:25.610428+0000","last_became_peered":"2023-11-26T07:18:25.610428+0000","last_unstale":"2023-11-26T07:18:27.230060+0000","last_undegraded":"2023-11-26T07:18:27.230060+0000","last_fullsized":"2023-11-26T07:18:27.230060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:55:21.347422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.593337+0000","last_change":"2023-11-26T07:18:24.596369+0000","last_active":"2023-11-26T07:18:25.593337+0000","last_peered":"2023-11-26T07:18:25.593337+0000","last_clean":"2023-11-26T07:18:25.593337+0000","last_became_active":"2023-11-26T07:18:24.595742+0000","last_became_peered":"2023-11-26T07:18:24.595742+0000","last_unstale":"2023-11-26T07:18:25.593337+0000","last_undegraded":"2023-11-26T07:18:25.593337+0000","last_fullsized":"2023-11-26T07:18:25.593337+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:11:50.426506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.573000+0000","last_change":"2023-11-26T07:18:25.598237+0000","last_active":"2023-11-26T07:18:26.573000+0000","last_peered":"2023-11-26T07:18:26.573000+0000","last_clean":"2023-11-26T07:18:26.573000+0000","last_became_active":"2023-11-26T07:18:25.597914+0000","last_became_peered":"2023-11-26T07:18:25.597914+0000","last_unstale":"2023-11-26T07:18:26.573000+0000","last_undegraded":"2023-11-26T07:18:26.573000+0000","last_fullsized":"2023-11-26T07:18:26.573000+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:16:50.893903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.617172+0000","last_change":"2023-11-26T07:18:24.595332+0000","last_active":"2023-11-26T07:18:25.617172+0000","last_peered":"2023-11-26T07:18:25.617172+0000","last_clean":"2023-11-26T07:18:25.617172+0000","last_became_active":"2023-11-26T07:18:24.594874+0000","last_became_peered":"2023-11-26T07:18:24.594874+0000","last_unstale":"2023-11-26T07:18:25.617172+0000","last_undegraded":"2023-11-26T07:18:25.617172+0000","last_fullsized":"2023-11-26T07:18:25.617172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:11:57.133577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.573255+0000","last_change":"2023-11-26T07:18:25.594341+0000","last_active":"2023-11-26T07:18:26.573255+0000","last_peered":"2023-11-26T07:18:26.573255+0000","last_clean":"2023-11-26T07:18:26.573255+0000","last_became_active":"2023-11-26T07:18:25.594048+0000","last_became_peered":"2023-11-26T07:18:25.594048+0000","last_unstale":"2023-11-26T07:18:26.573255+0000","last_undegraded":"2023-11-26T07:18:26.573255+0000","last_fullsized":"2023-11-26T07:18:26.573255+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:06:19.891238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.602728+0000","last_change":"2023-11-26T07:18:24.589241+0000","last_active":"2023-11-26T07:18:25.602728+0000","last_peered":"2023-11-26T07:18:25.602728+0000","last_clean":"2023-11-26T07:18:25.602728+0000","last_became_active":"2023-11-26T07:18:24.588980+0000","last_became_peered":"2023-11-26T07:18:24.588980+0000","last_unstale":"2023-11-26T07:18:25.602728+0000","last_undegraded":"2023-11-26T07:18:25.602728+0000","last_fullsized":"2023-11-26T07:18:25.602728+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:03:34.633560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.229156+0000","last_change":"2023-11-26T07:18:25.602748+0000","last_active":"2023-11-26T07:18:27.229156+0000","last_peered":"2023-11-26T07:18:27.229156+0000","last_clean":"2023-11-26T07:18:27.229156+0000","last_became_active":"2023-11-26T07:18:25.602437+0000","last_became_peered":"2023-11-26T07:18:25.602437+0000","last_unstale":"2023-11-26T07:18:27.229156+0000","last_undegraded":"2023-11-26T07:18:27.229156+0000","last_fullsized":"2023-11-26T07:18:27.229156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:33:42.303842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.610086+0000","last_change":"2023-11-26T07:18:24.589574+0000","last_active":"2023-11-26T07:18:25.610086+0000","last_peered":"2023-11-26T07:18:25.610086+0000","last_clean":"2023-11-26T07:18:25.610086+0000","last_became_active":"2023-11-26T07:18:24.589286+0000","last_became_peered":"2023-11-26T07:18:24.589286+0000","last_unstale":"2023-11-26T07:18:25.610086+0000","last_undegraded":"2023-11-26T07:18:25.610086+0000","last_fullsized":"2023-11-26T07:18:25.610086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:50.247115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230898+0000","last_change":"2023-11-26T07:18:25.608111+0000","last_active":"2023-11-26T07:18:27.230898+0000","last_peered":"2023-11-26T07:18:27.230898+0000","last_clean":"2023-11-26T07:18:27.230898+0000","last_became_active":"2023-11-26T07:18:25.607469+0000","last_became_peered":"2023-11-26T07:18:25.607469+0000","last_unstale":"2023-11-26T07:18:27.230898+0000","last_undegraded":"2023-11-26T07:18:27.230898+0000","last_fullsized":"2023-11-26T07:18:27.230898+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:20:05.223393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.608317+0000","last_change":"2023-11-26T07:18:24.590371+0000","last_active":"2023-11-26T07:18:25.608317+0000","last_peered":"2023-11-26T07:18:25.608317+0000","last_clean":"2023-11-26T07:18:25.608317+0000","last_became_active":"2023-11-26T07:18:24.590067+0000","last_became_peered":"2023-11-26T07:18:24.590067+0000","last_unstale":"2023-11-26T07:18:25.608317+0000","last_undegraded":"2023-11-26T07:18:25.608317+0000","last_fullsized":"2023-11-26T07:18:25.608317+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:53:07.256791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572332+0000","last_change":"2023-11-26T07:18:25.598264+0000","last_active":"2023-11-26T07:18:26.572332+0000","last_peered":"2023-11-26T07:18:26.572332+0000","last_clean":"2023-11-26T07:18:26.572332+0000","last_became_active":"2023-11-26T07:18:25.597934+0000","last_became_peered":"2023-11-26T07:18:25.597934+0000","last_unstale":"2023-11-26T07:18:26.572332+0000","last_undegraded":"2023-11-26T07:18:26.572332+0000","last_fullsized":"2023-11-26T07:18:26.572332+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:52:39.203887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.591824+0000","last_change":"2023-11-26T07:18:24.585586+0000","last_active":"2023-11-26T07:18:25.591824+0000","last_peered":"2023-11-26T07:18:25.591824+0000","last_clean":"2023-11-26T07:18:25.591824+0000","last_became_active":"2023-11-26T07:18:24.585226+0000","last_became_peered":"2023-11-26T07:18:24.585226+0000","last_unstale":"2023-11-26T07:18:25.591824+0000","last_undegraded":"2023-11-26T07:18:25.591824+0000","last_fullsized":"2023-11-26T07:18:25.591824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:39:49.112182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572137+0000","last_change":"2023-11-26T07:18:25.602900+0000","last_active":"2023-11-26T07:18:26.572137+0000","last_peered":"2023-11-26T07:18:26.572137+0000","last_clean":"2023-11-26T07:18:26.572137+0000","last_became_active":"2023-11-26T07:18:25.602649+0000","last_became_peered":"2023-11-26T07:18:25.602649+0000","last_unstale":"2023-11-26T07:18:26.572137+0000","last_undegraded":"2023-11-26T07:18:26.572137+0000","last_fullsized":"2023-11-26T07:18:26.572137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:47:00.795538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.610872+0000","last_change":"2023-11-26T07:18:24.586275+0000","last_active":"2023-11-26T07:18:25.610872+0000","last_peered":"2023-11-26T07:18:25.610872+0000","last_clean":"2023-11-26T07:18:25.610872+0000","last_became_active":"2023-11-26T07:18:24.585844+0000","last_became_peered":"2023-11-26T07:18:24.585844+0000","last_unstale":"2023-11-26T07:18:25.610872+0000","last_undegraded":"2023-11-26T07:18:25.610872+0000","last_fullsized":"2023-11-26T07:18:25.610872+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:25:16.010749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572292+0000","last_change":"2023-11-26T07:18:25.600991+0000","last_active":"2023-11-26T07:18:26.572292+0000","last_peered":"2023-11-26T07:18:26.572292+0000","last_clean":"2023-11-26T07:18:26.572292+0000","last_became_active":"2023-11-26T07:18:25.600702+0000","last_became_peered":"2023-11-26T07:18:25.600702+0000","last_unstale":"2023-11-26T07:18:26.572292+0000","last_undegraded":"2023-11-26T07:18:26.572292+0000","last_fullsized":"2023-11-26T07:18:26.572292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:11:17.646107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.571402+0000","last_change":"2023-11-26T07:18:24.586954+0000","last_active":"2023-11-26T07:18:25.571402+0000","last_peered":"2023-11-26T07:18:25.571402+0000","last_clean":"2023-11-26T07:18:25.571402+0000","last_became_active":"2023-11-26T07:18:24.586667+0000","last_became_peered":"2023-11-26T07:18:24.586667+0000","last_unstale":"2023-11-26T07:18:25.571402+0000","last_undegraded":"2023-11-26T07:18:25.571402+0000","last_fullsized":"2023-11-26T07:18:25.571402+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:14:55.655337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.571950+0000","last_change":"2023-11-26T07:18:25.600946+0000","last_active":"2023-11-26T07:18:26.571950+0000","last_peered":"2023-11-26T07:18:26.571950+0000","last_clean":"2023-11-26T07:18:26.571950+0000","last_became_active":"2023-11-26T07:18:25.600668+0000","last_became_peered":"2023-11-26T07:18:25.600668+0000","last_unstale":"2023-11-26T07:18:26.571950+0000","last_undegraded":"2023-11-26T07:18:26.571950+0000","last_fullsized":"2023-11-26T07:18:26.571950+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:10:50.241906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.584021+0000","last_change":"2023-11-26T07:18:24.593394+0000","last_active":"2023-11-26T07:18:25.584021+0000","last_peered":"2023-11-26T07:18:25.584021+0000","last_clean":"2023-11-26T07:18:25.584021+0000","last_became_active":"2023-11-26T07:18:24.593108+0000","last_became_peered":"2023-11-26T07:18:24.593108+0000","last_unstale":"2023-11-26T07:18:25.584021+0000","last_undegraded":"2023-11-26T07:18:25.584021+0000","last_fullsized":"2023-11-26T07:18:25.584021+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:52:38.196386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230016+0000","last_change":"2023-11-26T07:18:25.598749+0000","last_active":"2023-11-26T07:18:27.230016+0000","last_peered":"2023-11-26T07:18:27.230016+0000","last_clean":"2023-11-26T07:18:27.230016+0000","last_became_active":"2023-11-26T07:18:25.597808+0000","last_became_peered":"2023-11-26T07:18:25.597808+0000","last_unstale":"2023-11-26T07:18:27.230016+0000","last_undegraded":"2023-11-26T07:18:27.230016+0000","last_fullsized":"2023-11-26T07:18:27.230016+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:20:37.447188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.572383+0000","last_change":"2023-11-26T07:18:24.589292+0000","last_active":"2023-11-26T07:18:25.572383+0000","last_peered":"2023-11-26T07:18:25.572383+0000","last_clean":"2023-11-26T07:18:25.572383+0000","last_became_active":"2023-11-26T07:18:24.589014+0000","last_became_peered":"2023-11-26T07:18:24.589014+0000","last_unstale":"2023-11-26T07:18:25.572383+0000","last_undegraded":"2023-11-26T07:18:25.572383+0000","last_fullsized":"2023-11-26T07:18:25.572383+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:12:09.751025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572349+0000","last_change":"2023-11-26T07:18:25.596028+0000","last_active":"2023-11-26T07:18:26.572349+0000","last_peered":"2023-11-26T07:18:26.572349+0000","last_clean":"2023-11-26T07:18:26.572349+0000","last_became_active":"2023-11-26T07:18:25.595701+0000","last_became_peered":"2023-11-26T07:18:25.595701+0000","last_unstale":"2023-11-26T07:18:26.572349+0000","last_undegraded":"2023-11-26T07:18:26.572349+0000","last_fullsized":"2023-11-26T07:18:26.572349+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:55:59.929505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.593416+0000","last_change":"2023-11-26T07:18:24.594254+0000","last_active":"2023-11-26T07:18:25.593416+0000","last_peered":"2023-11-26T07:18:25.593416+0000","last_clean":"2023-11-26T07:18:25.593416+0000","last_became_active":"2023-11-26T07:18:24.593932+0000","last_became_peered":"2023-11-26T07:18:24.593932+0000","last_unstale":"2023-11-26T07:18:25.593416+0000","last_undegraded":"2023-11-26T07:18:25.593416+0000","last_fullsized":"2023-11-26T07:18:25.593416+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:45:37.682548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.229196+0000","last_change":"2023-11-26T07:18:25.583426+0000","last_active":"2023-11-26T07:18:27.229196+0000","last_peered":"2023-11-26T07:18:27.229196+0000","last_clean":"2023-11-26T07:18:27.229196+0000","last_became_active":"2023-11-26T07:18:25.583092+0000","last_became_peered":"2023-11-26T07:18:25.583092+0000","last_unstale":"2023-11-26T07:18:27.229196+0000","last_undegraded":"2023-11-26T07:18:27.229196+0000","last_fullsized":"2023-11-26T07:18:27.229196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:12.355159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.231519+0000","last_change":"2023-11-26T07:18:24.585493+0000","last_active":"2023-11-26T07:18:27.231519+0000","last_peered":"2023-11-26T07:18:27.231519+0000","last_clean":"2023-11-26T07:18:27.231519+0000","last_became_active":"2023-11-26T07:18:24.585141+0000","last_became_peered":"2023-11-26T07:18:24.585141+0000","last_unstale":"2023-11-26T07:18:27.231519+0000","last_undegraded":"2023-11-26T07:18:27.231519+0000","last_fullsized":"2023-11-26T07:18:27.231519+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:07:50.247115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230978+0000","last_change":"2023-11-26T07:18:25.597354+0000","last_active":"2023-11-26T07:18:27.230978+0000","last_peered":"2023-11-26T07:18:27.230978+0000","last_clean":"2023-11-26T07:18:27.230978+0000","last_became_active":"2023-11-26T07:18:25.597042+0000","last_became_peered":"2023-11-26T07:18:25.597042+0000","last_unstale":"2023-11-26T07:18:27.230978+0000","last_undegraded":"2023-11-26T07:18:27.230978+0000","last_fullsized":"2023-11-26T07:18:27.230978+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:45:38.690049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.573490+0000","last_change":"2023-11-26T07:18:24.590400+0000","last_active":"2023-11-26T07:18:25.573490+0000","last_peered":"2023-11-26T07:18:25.573490+0000","last_clean":"2023-11-26T07:18:25.573490+0000","last_became_active":"2023-11-26T07:18:24.590083+0000","last_became_peered":"2023-11-26T07:18:24.590083+0000","last_unstale":"2023-11-26T07:18:25.573490+0000","last_undegraded":"2023-11-26T07:18:25.573490+0000","last_fullsized":"2023-11-26T07:18:25.573490+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:16:49.886402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.573023+0000","last_change":"2023-11-26T07:18:25.599222+0000","last_active":"2023-11-26T07:18:26.573023+0000","last_peered":"2023-11-26T07:18:26.573023+0000","last_clean":"2023-11-26T07:18:26.573023+0000","last_became_active":"2023-11-26T07:18:25.598952+0000","last_became_peered":"2023-11-26T07:18:25.598952+0000","last_unstale":"2023-11-26T07:18:26.573023+0000","last_undegraded":"2023-11-26T07:18:26.573023+0000","last_fullsized":"2023-11-26T07:18:26.573023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:14:13.237623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.582594+0000","last_change":"2023-11-26T07:18:24.586412+0000","last_active":"2023-11-26T07:18:25.582594+0000","last_peered":"2023-11-26T07:18:25.582594+0000","last_clean":"2023-11-26T07:18:25.582594+0000","last_became_active":"2023-11-26T07:18:24.585386+0000","last_became_peered":"2023-11-26T07:18:24.585386+0000","last_unstale":"2023-11-26T07:18:25.582594+0000","last_undegraded":"2023-11-26T07:18:25.582594+0000","last_fullsized":"2023-11-26T07:18:25.582594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:39:49.112182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230176+0000","last_change":"2023-11-26T07:18:25.598014+0000","last_active":"2023-11-26T07:18:27.230176+0000","last_peered":"2023-11-26T07:18:27.230176+0000","last_clean":"2023-11-26T07:18:27.230176+0000","last_became_active":"2023-11-26T07:18:25.597620+0000","last_became_peered":"2023-11-26T07:18:25.597620+0000","last_unstale":"2023-11-26T07:18:27.230176+0000","last_undegraded":"2023-11-26T07:18:27.230176+0000","last_fullsized":"2023-11-26T07:18:27.230176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:14:13.237623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.573563+0000","last_change":"2023-11-26T07:18:24.596404+0000","last_active":"2023-11-26T07:18:25.573563+0000","last_peered":"2023-11-26T07:18:25.573563+0000","last_clean":"2023-11-26T07:18:25.573563+0000","last_became_active":"2023-11-26T07:18:24.593975+0000","last_became_peered":"2023-11-26T07:18:24.593975+0000","last_unstale":"2023-11-26T07:18:25.573563+0000","last_undegraded":"2023-11-26T07:18:25.573563+0000","last_fullsized":"2023-11-26T07:18:25.573563+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:14:12.230122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.231425+0000","last_change":"2023-11-26T07:18:25.603659+0000","last_active":"2023-11-26T07:18:27.231425+0000","last_peered":"2023-11-26T07:18:27.231425+0000","last_clean":"2023-11-26T07:18:27.231425+0000","last_became_active":"2023-11-26T07:18:25.603321+0000","last_became_peered":"2023-11-26T07:18:25.603321+0000","last_unstale":"2023-11-26T07:18:27.231425+0000","last_undegraded":"2023-11-26T07:18:27.231425+0000","last_fullsized":"2023-11-26T07:18:27.231425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:23:04.014784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.584025+0000","last_change":"2023-11-26T07:18:24.588130+0000","last_active":"2023-11-26T07:18:25.584025+0000","last_peered":"2023-11-26T07:18:25.584025+0000","last_clean":"2023-11-26T07:18:25.584025+0000","last_became_active":"2023-11-26T07:18:24.586853+0000","last_became_peered":"2023-11-26T07:18:24.586853+0000","last_unstale":"2023-11-26T07:18:25.584025+0000","last_undegraded":"2023-11-26T07:18:25.584025+0000","last_fullsized":"2023-11-26T07:18:25.584025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:55:58.922003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:26.572092+0000","last_change":"2023-11-26T07:18:25.602033+0000","last_active":"2023-11-26T07:18:26.572092+0000","last_peered":"2023-11-26T07:18:26.572092+0000","last_clean":"2023-11-26T07:18:26.572092+0000","last_became_active":"2023-11-26T07:18:25.595875+0000","last_became_peered":"2023-11-26T07:18:25.595875+0000","last_unstale":"2023-11-26T07:18:26.572092+0000","last_undegraded":"2023-11-26T07:18:26.572092+0000","last_fullsized":"2023-11-26T07:18:26.572092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:14:13.237623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.593216+0000","last_change":"2023-11-26T07:18:24.593229+0000","last_active":"2023-11-26T07:18:25.593216+0000","last_peered":"2023-11-26T07:18:25.593216+0000","last_clean":"2023-11-26T07:18:25.593216+0000","last_became_active":"2023-11-26T07:18:24.592929+0000","last_became_peered":"2023-11-26T07:18:24.592929+0000","last_unstale":"2023-11-26T07:18:25.593216+0000","last_undegraded":"2023-11-26T07:18:25.593216+0000","last_fullsized":"2023-11-26T07:18:25.593216+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:36:46.098596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-26T07:18:27.230496+0000","last_change":"2023-11-26T07:18:25.608505+0000","last_active":"2023-11-26T07:18:27.230496+0000","last_peered":"2023-11-26T07:18:27.230496+0000","last_clean":"2023-11-26T07:18:27.230496+0000","last_became_active":"2023-11-26T07:18:25.608077+0000","last_became_peered":"2023-11-26T07:18:25.608077+0000","last_unstale":"2023-11-26T07:18:27.230496+0000","last_undegraded":"2023-11-26T07:18:27.230496+0000","last_fullsized":"2023-11-26T07:18:27.230496+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:24.558201+0000","last_clean_scrub_stamp":"2023-11-26T07:18:24.558201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:12.355159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.571501+0000","last_change":"2023-11-26T07:18:24.591226+0000","last_active":"2023-11-26T07:18:25.571501+0000","last_peered":"2023-11-26T07:18:25.571501+0000","last_clean":"2023-11-26T07:18:25.571501+0000","last_became_active":"2023-11-26T07:18:24.590903+0000","last_became_peered":"2023-11-26T07:18:24.590903+0000","last_unstale":"2023-11-26T07:18:25.571501+0000","last_undegraded":"2023-11-26T07:18:25.571501+0000","last_fullsized":"2023-11-26T07:18:25.571501+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:18:23.550699+0000","last_clean_scrub_stamp":"2023-11-26T07:18:23.550699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:49:33.810259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":174,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-26T07:18:25.580442+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:18:25.580442+0000","last_peered":"2023-11-26T07:18:25.580442+0000","last_clean":"2023-11-26T07:18:25.580442+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:18:25.580442+0000","last_undegraded":"2023-11-26T07:18:25.580442+0000","last_fullsized":"2023-11-26T07:18:25.580442+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":970752,"data_stored":736920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56000000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.755}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.80200000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.77100000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.91700000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.88800000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.94799999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":147078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.629}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.69699999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.52500000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.55500000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.41399999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54500000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.501},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67300000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.62}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.68799999999999994}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.64300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":146566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.71399999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.81899999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.89000000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":1}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":1.012}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.98299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.379}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.307},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.39700000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.47599999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.78200000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.81399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.71299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":1.655}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.52000000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":8.0990000000000002,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.67700000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":4.431,"5min":4.431,"15min":4.431},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":1.7050000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.77300000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":1.7310000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":1.7549999999999999},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.58799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.57799999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.44600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.48099999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.65000000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.66300000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.69399999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":962560,"data_stored":735811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.376},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44800000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51600000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57599999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.503}]},{"osd":7,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.67600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:18:34.627 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:18:34.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:18:34.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4bddf640 1 Processor -- start 2023-11-26T07:18:34.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4bddf640 1 -- start start 2023-11-26T07:18:34.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4bddf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cb9b0 0x7f6a4406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4bddf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6a440ce5d0 0x7f6a4415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4bddf640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a44159630 con 0x7f6a440cbd90 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4bddf640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4415bba0 con 0x7f6a440ce5d0 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4bddf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440cdd50 con 0x7f6a440cb9b0 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4a37c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6a440ce5d0 0x7f6a4415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4a37c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6a440ce5d0 0x7f6a4415bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55580/0 (socket says 172.21.15.31:55580) 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4a37c640 1 -- 172.21.15.31:0/637054828 learned_addr learned my addr 172.21.15.31:0/637054828 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4937a640 1 --1- 172.21.15.31:0/637054828 >> v1:172.21.15.31:6789/0 conn(0x7f6a440cbd90 0x7f6a4415f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57116/0 (socket says 172.21.15.31:57116) 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a49b7b640 1 --2- 172.21.15.31:0/637054828 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cb9b0 0x7f6a4406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4a37c640 1 -- 172.21.15.31:0/637054828 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cb9b0 msgr2=0x7f6a4406e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4a37c640 1 --2- 172.21.15.31:0/637054828 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cb9b0 0x7f6a4406e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4a37c640 1 -- 172.21.15.31:0/637054828 >> v1:172.21.15.31:6789/0 conn(0x7f6a440cbd90 legacy=0x7f6a4415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.812+0000 7f6a4a37c640 1 -- 172.21.15.31:0/637054828 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440cd1d0 con 0x7f6a440ce5d0 2023-11-26T07:18:34.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a4a37c640 1 --2- 172.21.15.31:0/637054828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440ce5d0 0x7f6a4415bfd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009e50 tx=0x7f6a3c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc90d1611d2bb73 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a48b79640 1 -- 172.21.15.31:0/637054828 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0a5880 con 0x7f6a440ce5d0 2023-11-26T07:18:34.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a48b79640 1 -- 172.21.15.31:0/637054828 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0a59e0 con 0x7f6a440ce5d0 2023-11-26T07:18:34.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a48b79640 1 -- 172.21.15.31:0/637054828 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0a5d70 con 0x7f6a440ce5d0 2023-11-26T07:18:34.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a4bddf640 1 -- 172.21.15.31:0/637054828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440ce5d0 msgr2=0x7f6a4415bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a4bddf640 1 --2- 172.21.15.31:0/637054828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440ce5d0 0x7f6a4415bfd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009e50 tx=0x7f6a3c01c9e0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a4bddf640 1 -- 172.21.15.31:0/637054828 shutdown_connections 2023-11-26T07:18:34.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a4bddf640 1 --2- 172.21.15.31:0/637054828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440ce5d0 0x7f6a4415bfd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a4bddf640 1 --2- 172.21.15.31:0/637054828 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cb9b0 0x7f6a4406e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a4bddf640 1 -- 172.21.15.31:0/637054828 >> 172.21.15.31:0/637054828 conn(0x7f6a440b0cb0 msgr2=0x7f6a440b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:34.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a4bddf640 1 -- 172.21.15.31:0/637054828 shutdown_connections 2023-11-26T07:18:34.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.816+0000 7f6a4bddf640 1 -- 172.21.15.31:0/637054828 wait complete. 2023-11-26T07:18:34.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4bddf640 1 Processor -- start 2023-11-26T07:18:34.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4bddf640 1 -- start start 2023-11-26T07:18:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4bddf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440cb9b0 0x7f6a441fa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4bddf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cbd90 0x7f6a44164440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4bddf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a440ce5d0 0x7f6a44158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4bddf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44153300 con 0x7f6a440cb9b0 2023-11-26T07:18:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4bddf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44164980 con 0x7f6a440ce5d0 2023-11-26T07:18:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4bddf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44152960 con 0x7f6a440cbd90 2023-11-26T07:18:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a49b7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440cb9b0 0x7f6a441fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a49b7b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440cb9b0 0x7f6a441fa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55590/0 (socket says 172.21.15.31:55590) 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a49b7b640 1 -- 172.21.15.31:0/2218800597 learned_addr learned my addr 172.21.15.31:0/2218800597 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4937a640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cbd90 0x7f6a44164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4a37c640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a440ce5d0 0x7f6a44158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a49b7b640 1 -- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cbd90 msgr2=0x7f6a44164440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a49b7b640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cbd90 0x7f6a44164440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a49b7b640 1 -- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a440ce5d0 msgr2=0x7f6a44158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a49b7b640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a440ce5d0 0x7f6a44158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a49b7b640 1 -- 172.21.15.31:0/2218800597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c009b30 con 0x7f6a440cb9b0 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4937a640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cbd90 0x7f6a44164440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4a37c640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a440ce5d0 0x7f6a44158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a49b7b640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440cb9b0 0x7f6a441fa820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000d800 tx=0x7f6a4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a3a7fc640 1 -- 172.21.15.31:0/2218800597 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000b6c0 con 0x7f6a440cb9b0 2023-11-26T07:18:34.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a3a7fc640 1 -- 172.21.15.31:0/2218800597 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4000b820 con 0x7f6a440cb9b0 2023-11-26T07:18:34.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4bddf640 1 -- 172.21.15.31:0/2218800597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a44152c40 con 0x7f6a440cb9b0 2023-11-26T07:18:34.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a3a7fc640 1 -- 172.21.15.31:0/2218800597 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000bb90 con 0x7f6a440cb9b0 2023-11-26T07:18:34.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.820+0000 7f6a4bddf640 1 -- 172.21.15.31:0/2218800597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a44164b70 con 0x7f6a440cb9b0 2023-11-26T07:18:34.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.824+0000 7f6a4bddf640 1 -- 172.21.15.31:0/2218800597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a440671f0 con 0x7f6a440cb9b0 2023-11-26T07:18:34.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.824+0000 7f6a3a7fc640 1 -- 172.21.15.31:0/2218800597 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f6a4000bcf0 con 0x7f6a440cb9b0 2023-11-26T07:18:34.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.824+0000 7f6a3a7fc640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6a280664c0 0x7f6a28068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:34.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.824+0000 7f6a3a7fc640 1 -- 172.21.15.31:0/2218800597 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a40084420 con 0x7f6a440cb9b0 2023-11-26T07:18:34.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.824+0000 7f6a4937a640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6a280664c0 0x7f6a28068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:34.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.824+0000 7f6a4937a640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6a280664c0 0x7f6a28068980 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6a440b4290 tx=0x7f6a34007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:34.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:34.828+0000 7f6a3a7fc640 1 -- 172.21.15.31:0/2218800597 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6a40051880 con 0x7f6a440cb9b0 2023-11-26T07:18:35.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.036+0000 7f6a4bddf640 1 -- 172.21.15.31:0/2218800597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6a4406b460 con 0x7f6a440cb9b0 2023-11-26T07:18:35.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.040+0000 7f6a3a7fc640 1 -- 172.21.15.31:0/2218800597 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6a40054ea0 con 0x7f6a440cb9b0 2023-11-26T07:18:35.046 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-11-26T07:18:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA== 2023-11-26T07:18:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2023-11-26T07:18:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2023-11-26T07:18:35.046 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:18:35.047 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-11-26T07:18:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 -- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6a280664c0 msgr2=0x7f6a28068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6a280664c0 0x7f6a28068980 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6a440b4290 tx=0x7f6a34007d50 comp rx=0 tx=0).stop 2023-11-26T07:18:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 -- 172.21.15.31:0/2218800597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440cb9b0 msgr2=0x7f6a441fa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440cb9b0 0x7f6a441fa820 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000d800 tx=0x7f6a4000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 -- 172.21.15.31:0/2218800597 shutdown_connections 2023-11-26T07:18:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6a280664c0 0x7f6a28068980 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6a440ce5d0 0x7f6a44158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6a440cbd90 0x7f6a44164440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 --2- 172.21.15.31:0/2218800597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6a440cb9b0 0x7f6a441fa820 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 -- 172.21.15.31:0/2218800597 >> 172.21.15.31:0/2218800597 conn(0x7f6a440b0cb0 msgr2=0x7f6a441597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 -- 172.21.15.31:0/2218800597 shutdown_connections 2023-11-26T07:18:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.044+0000 7f6a4bddf640 1 -- 172.21.15.31:0/2218800597 wait complete. 2023-11-26T07:18:35.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:35.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82af0d6640 1 Processor -- start 2023-11-26T07:18:35.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82af0d6640 1 -- start start 2023-11-26T07:18:35.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82af0d6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a815d530 0x7f82a80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82af0d6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f82a80cf1b0 0x7f82a80cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82af0d6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f82a80d02f0 0x7f82a81611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82af0d6640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a80d0780 con 0x7f82a80d02f0 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82af0d6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a80d12e0 con 0x7f82a80cf1b0 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82af0d6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a80d15c0 con 0x7f82a815d530 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82ad673640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f82a80d02f0 0x7f82a81611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82ad673640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f82a80d02f0 0x7f82a81611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55606/0 (socket says 172.21.15.31:55606) 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82ad673640 1 -- 172.21.15.31:0/126427047 learned_addr learned my addr 172.21.15.31:0/126427047 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82a7fff640 1 --2- 172.21.15.31:0/126427047 >> v2:172.21.15.97:3300/0 conn(0x7f82a80cf1b0 0x7f82a80cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82ace72640 1 --2- 172.21.15.31:0/126427047 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a815d530 0x7f82a80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82ad673640 1 -- 172.21.15.31:0/126427047 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a815d530 msgr2=0x7f82a80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82ad673640 1 --2- 172.21.15.31:0/126427047 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a815d530 0x7f82a80cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82ad673640 1 -- 172.21.15.31:0/126427047 >> v2:172.21.15.97:3300/0 conn(0x7f82a80cf1b0 msgr2=0x7f82a80cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82ad673640 1 --2- 172.21.15.31:0/126427047 >> v2:172.21.15.97:3300/0 conn(0x7f82a80cf1b0 0x7f82a80cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.204+0000 7f82ad673640 1 -- 172.21.15.31:0/126427047 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a80d18a0 con 0x7f82a80d02f0 2023-11-26T07:18:35.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82ad673640 1 --2- 172.21.15.31:0/126427047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 0x7f82a81611a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f82a808c2a0 tx=0x7f82a001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63357d8f0cf3e67 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82a77fe640 1 -- 172.21.15.31:0/126427047 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a00af400 con 0x7f82a80d02f0 2023-11-26T07:18:35.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82a77fe640 1 -- 172.21.15.31:0/126427047 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a00ab050 con 0x7f82a80d02f0 2023-11-26T07:18:35.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82a77fe640 1 -- 172.21.15.31:0/126427047 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a00af780 con 0x7f82a80d02f0 2023-11-26T07:18:35.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82af0d6640 1 -- 172.21.15.31:0/126427047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 msgr2=0x7f82a81611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82af0d6640 1 --2- 172.21.15.31:0/126427047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 0x7f82a81611a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f82a808c2a0 tx=0x7f82a001f1c0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82af0d6640 1 -- 172.21.15.31:0/126427047 shutdown_connections 2023-11-26T07:18:35.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82af0d6640 1 --2- 172.21.15.31:0/126427047 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 0x7f82a81611a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82af0d6640 1 --2- 172.21.15.31:0/126427047 >> v2:172.21.15.97:3300/0 conn(0x7f82a80cf1b0 0x7f82a80cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82af0d6640 1 --2- 172.21.15.31:0/126427047 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a815d530 0x7f82a80cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82af0d6640 1 -- 172.21.15.31:0/126427047 >> 172.21.15.31:0/126427047 conn(0x7f82a809c6d0 msgr2=0x7f82a80d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82af0d6640 1 -- 172.21.15.31:0/126427047 shutdown_connections 2023-11-26T07:18:35.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.208+0000 7f82af0d6640 1 -- 172.21.15.31:0/126427047 wait complete. 2023-11-26T07:18:35.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82af0d6640 1 Processor -- start 2023-11-26T07:18:35.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82af0d6640 1 -- start start 2023-11-26T07:18:35.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82af0d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a80cf1b0 0x7f82a80c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82af0d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 0x7f82a80ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82af0d6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a815d530 0x7f82a815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82af0d6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a80cac10 con 0x7f82a80d02f0 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82af0d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a80caef0 con 0x7f82a80cf1b0 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82af0d6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a80cb1d0 con 0x7f82a815d530 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 0x7f82a80ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 0x7f82a80ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55608/0 (socket says 172.21.15.31:55608) 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a7fff640 1 -- 172.21.15.31:0/105161587 learned_addr learned my addr 172.21.15.31:0/105161587 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82ad673640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a815d530 0x7f82a815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82ace72640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a80cf1b0 0x7f82a80c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a7fff640 1 -- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a815d530 msgr2=0x7f82a815a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a7fff640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a815d530 0x7f82a815a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a7fff640 1 -- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a80cf1b0 msgr2=0x7f82a80c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a7fff640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a80cf1b0 0x7f82a80c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a7fff640 1 -- 172.21.15.31:0/105161587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a0009b30 con 0x7f82a80d02f0 2023-11-26T07:18:35.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a7fff640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 0x7f82a80ccfc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f829400e8b0 tx=0x7f829400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a57fa640 1 -- 172.21.15.31:0/105161587 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829400c940 con 0x7f82a80d02f0 2023-11-26T07:18:35.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a57fa640 1 -- 172.21.15.31:0/105161587 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829400caa0 con 0x7f82a80d02f0 2023-11-26T07:18:35.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82af0d6640 1 -- 172.21.15.31:0/105161587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a80cb4b0 con 0x7f82a80d02f0 2023-11-26T07:18:35.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.212+0000 7f82a57fa640 1 -- 172.21.15.31:0/105161587 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829400ce30 con 0x7f82a80d02f0 2023-11-26T07:18:35.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.216+0000 7f82af0d6640 1 -- 172.21.15.31:0/105161587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a80cb790 con 0x7f82a80d02f0 2023-11-26T07:18:35.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.216+0000 7f82af0d6640 1 -- 172.21.15.31:0/105161587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a804e850 con 0x7f82a80d02f0 2023-11-26T07:18:35.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.216+0000 7f82a57fa640 1 -- 172.21.15.31:0/105161587 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8294013470 con 0x7f82a80d02f0 2023-11-26T07:18:35.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.216+0000 7f82a57fa640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f828c066470 0x7f828c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.216+0000 7f82a57fa640 1 -- 172.21.15.31:0/105161587 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8294083f20 con 0x7f82a80d02f0 2023-11-26T07:18:35.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.216+0000 7f82ace72640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f828c066470 0x7f828c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.216+0000 7f82ace72640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f828c066470 0x7f828c068930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f829c009ac0 tx=0x7f829c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.220+0000 7f82a57fa640 1 -- 172.21.15.31:0/105161587 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82940514e0 con 0x7f82a80d02f0 2023-11-26T07:18:35.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.412+0000 7f82af0d6640 1 -- 172.21.15.31:0/105161587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82a80cf980 con 0x7f82a80d02f0 2023-11-26T07:18:35.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.412+0000 7f82a57fa640 1 -- 172.21.15.31:0/105161587 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f8294054b00 con 0x7f82a80d02f0 2023-11-26T07:18:35.418 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:35.418 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3937918726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3937918726},{"type":"v1","addr":"172.21.15.97:6837","nonce":3937918726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2083665480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2083665480},{"type":"v1","addr":"172.21.15.97:6835","nonce":2083665480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1187139549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187139549},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187139549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:24.963367+0000","modified":"2023-11-26T07:18:27.980260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4036387633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4036387633},{"type":"v1","addr":"172.21.15.31:6835","nonce":4036387633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:18:35.418 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2023-11-26T07:18:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.412+0000 7f82af0d6640 1 -- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f828c066470 msgr2=0x7f828c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.412+0000 7f82af0d6640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f828c066470 0x7f828c068930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f829c009ac0 tx=0x7f829c009190 comp rx=0 tx=0).stop 2023-11-26T07:18:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.412+0000 7f82af0d6640 1 -- 172.21.15.31:0/105161587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 msgr2=0x7f82a80ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.412+0000 7f82af0d6640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 0x7f82a80ccfc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f829400e8b0 tx=0x7f829400ed70 comp rx=0 tx=0).stop 2023-11-26T07:18:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.416+0000 7f82af0d6640 1 -- 172.21.15.31:0/105161587 shutdown_connections 2023-11-26T07:18:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.416+0000 7f82af0d6640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f828c066470 0x7f828c068930 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.416+0000 7f82af0d6640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f82a815d530 0x7f82a815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.416+0000 7f82af0d6640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80d02f0 0x7f82a80ccfc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.416+0000 7f82af0d6640 1 --2- 172.21.15.31:0/105161587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82a80cf1b0 0x7f82a80c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.416+0000 7f82af0d6640 1 -- 172.21.15.31:0/105161587 >> 172.21.15.31:0/105161587 conn(0x7f82a809c6d0 msgr2=0x7f82a80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.416+0000 7f82af0d6640 1 -- 172.21.15.31:0/105161587 shutdown_connections 2023-11-26T07:18:35.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.416+0000 7f82af0d6640 1 -- 172.21.15.31:0/105161587 wait complete. 2023-11-26T07:18:35.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:18:24.963367+0000', 'modified': '2023-11-26T07:18:27.980260+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5106}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5106': {'gid': 5106, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/4036387633', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6834', 'nonce': 4036387633}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 4036387633}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:18:35.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:18:35.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-26T07:18:35.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f653951d640 1 Processor -- start 2023-11-26T07:18:35.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f653951d640 1 -- start start 2023-11-26T07:18:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f653951d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65340b07a0 0x7f65341619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f653951d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6534161f20 0x7f65340b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f653951d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f65340b2870 0x7f65340b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f653951d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65340b0b80 con 0x7f65340b07a0 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f653951d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6534161290 con 0x7f65340b2870 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f653951d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6534161550 con 0x7f6534161f20 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f6532ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65340b07a0 0x7f65341619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f6532ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65340b07a0 0x7f65341619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55618/0 (socket says 172.21.15.31:55618) 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f6532ffd640 1 -- 172.21.15.31:0/1774668292 learned_addr learned my addr 172.21.15.31:0/1774668292 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f65337fe640 1 --2- 172.21.15.31:0/1774668292 >> v2:172.21.15.97:3300/0 conn(0x7f65340b2870 0x7f65340b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f652bfff640 1 --2- 172.21.15.31:0/1774668292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6534161f20 0x7f65340b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f6532ffd640 1 -- 172.21.15.31:0/1774668292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6534161f20 msgr2=0x7f65340b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f6532ffd640 1 --2- 172.21.15.31:0/1774668292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6534161f20 0x7f65340b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f6532ffd640 1 -- 172.21.15.31:0/1774668292 >> v2:172.21.15.97:3300/0 conn(0x7f65340b2870 msgr2=0x7f65340b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:35.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f6532ffd640 1 --2- 172.21.15.31:0/1774668292 >> v2:172.21.15.97:3300/0 conn(0x7f65340b2870 0x7f65340b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f6532ffd640 1 -- 172.21.15.31:0/1774668292 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65340c65a0 con 0x7f65340b07a0 2023-11-26T07:18:35.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.984+0000 7f652bfff640 1 --2- 172.21.15.31:0/1774668292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6534161f20 0x7f65340b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:35.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f6532ffd640 1 --2- 172.21.15.31:0/1774668292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b07a0 0x7f65341619e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f653406c650 tx=0x7f652401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e6a3c8b07da7cb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f65327fc640 1 -- 172.21.15.31:0/1774668292 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65240aa5f0 con 0x7f65340b07a0 2023-11-26T07:18:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f65327fc640 1 -- 172.21.15.31:0/1774668292 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65240aa750 con 0x7f65340b07a0 2023-11-26T07:18:35.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f65327fc640 1 -- 172.21.15.31:0/1774668292 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65240aaae0 con 0x7f65340b07a0 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f653951d640 1 -- 172.21.15.31:0/1774668292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b07a0 msgr2=0x7f65341619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f653951d640 1 --2- 172.21.15.31:0/1774668292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b07a0 0x7f65341619e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f653406c650 tx=0x7f652401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f653951d640 1 -- 172.21.15.31:0/1774668292 shutdown_connections 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f653951d640 1 --2- 172.21.15.31:0/1774668292 >> v2:172.21.15.97:3300/0 conn(0x7f65340b2870 0x7f65340b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f653951d640 1 --2- 172.21.15.31:0/1774668292 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6534161f20 0x7f65340b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f653951d640 1 --2- 172.21.15.31:0/1774668292 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b07a0 0x7f65341619e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f653951d640 1 -- 172.21.15.31:0/1774668292 >> 172.21.15.31:0/1774668292 conn(0x7f65340b1d50 msgr2=0x7f65340b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f653951d640 1 -- 172.21.15.31:0/1774668292 shutdown_connections 2023-11-26T07:18:35.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.988+0000 7f653951d640 1 -- 172.21.15.31:0/1774668292 wait complete. 2023-11-26T07:18:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f653951d640 1 Processor -- start 2023-11-26T07:18:35.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f653951d640 1 -- start start 2023-11-26T07:18:35.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f653951d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65340b07a0 0x7f6534158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f653951d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b2870 0x7f65340af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f653951d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6534161f20 0x7f65340aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f653951d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653415efb0 con 0x7f65340b2870 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f653951d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653415f290 con 0x7f6534161f20 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f653951d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653415f570 con 0x7f65340b07a0 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b2870 0x7f65340af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b2870 0x7f65340af9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55632/0 (socket says 172.21.15.31:55632) 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652bfff640 1 -- 172.21.15.31:0/594277571 learned_addr learned my addr 172.21.15.31:0/594277571 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f65337fe640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6534161f20 0x7f65340aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f6532ffd640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65340b07a0 0x7f6534158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:35.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652bfff640 1 -- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65340b07a0 msgr2=0x7f6534158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652bfff640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65340b07a0 0x7f6534158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652bfff640 1 -- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6534161f20 msgr2=0x7f65340aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652bfff640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6534161f20 0x7f65340aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652bfff640 1 -- 172.21.15.31:0/594277571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65240094d0 con 0x7f65340b2870 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652bfff640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b2870 0x7f65340af9e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f651c00c7f0 tx=0x7f651c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652b7fe640 1 -- 172.21.15.31:0/594277571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c004220 con 0x7f65340b2870 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652b7fe640 1 -- 172.21.15.31:0/594277571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c004380 con 0x7f65340b2870 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f652b7fe640 1 -- 172.21.15.31:0/594277571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c003840 con 0x7f65340b2870 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f653951d640 1 -- 172.21.15.31:0/594277571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653415f850 con 0x7f65340b2870 2023-11-26T07:18:36.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.992+0000 7f653951d640 1 -- 172.21.15.31:0/594277571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65341f63e0 con 0x7f65340b2870 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f652b7fe640 1 -- 172.21.15.31:0/594277571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f651c003bb0 con 0x7f65340b2870 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f652b7fe640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65140664a0 0x7f6514068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f652b7fe640 1 -- 172.21.15.31:0/594277571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f651c083d60 con 0x7f65340b2870 2023-11-26T07:18:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f6532ffd640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65140664a0 0x7f6514068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f653c882000 1 Processor -- start 2023-11-26T07:18:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f653c882000 1 -- start start 2023-11-26T07:18:36.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f6532ffd640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65140664a0 0x7f6514068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f65240ae000 tx=0x7f65240b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f653c882000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24bc90 0x55690c24c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f653c882000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c24c5b0 0x55690c201de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f653c882000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c202320 0x55690c202700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f653c882000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690c06c130 con 0x55690c202320 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f653c882000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690c0599a0 con 0x55690c24c5b0 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f653c882000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55690c056ed0 con 0x55690c24bc90 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:35.996+0000 7f65337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c202320 0x55690c202700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f65337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c202320 0x55690c202700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55634/0 (socket says 172.21.15.31:55634) 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f65337fe640 1 -- 172.21.15.31:0/2454146015 learned_addr learned my addr 172.21.15.31:0/2454146015 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f65337fe640 1 -- 172.21.15.31:0/2454146015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24bc90 msgr2=0x55690c24c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f6532ffd640 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24bc90 0x55690c24c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f65337fe640 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24bc90 0x55690c24c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f65337fe640 1 -- 172.21.15.31:0/2454146015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c24c5b0 msgr2=0x55690c201de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f652bfff640 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c24c5b0 0x55690c201de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f65337fe640 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c24c5b0 0x55690c201de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f65337fe640 1 -- 172.21.15.31:0/2454146015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55690bd4a890 con 0x55690c202320 2023-11-26T07:18:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f6532ffd640 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24bc90 0x55690c24c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f652bfff640 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c24c5b0 0x55690c201de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f65337fe640 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c202320 0x55690c202700 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f652c009f30 tx=0x7f652c003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65db771cbf6637d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f6528ff9640 1 -- 172.21.15.31:0/2454146015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c015420 con 0x55690c202320 2023-11-26T07:18:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f6528ff9640 1 -- 172.21.15.31:0/2454146015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652c015580 con 0x55690c202320 2023-11-26T07:18:36.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f6528ff9640 1 -- 172.21.15.31:0/2454146015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c0158f0 con 0x55690c202320 2023-11-26T07:18:36.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 -- 172.21.15.31:0/2454146015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c202320 msgr2=0x55690c202700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c202320 0x55690c202700 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f652c009f30 tx=0x7f652c003e30 comp rx=0 tx=0).stop 2023-11-26T07:18:36.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 -- 172.21.15.31:0/2454146015 shutdown_connections 2023-11-26T07:18:36.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c202320 0x55690c202700 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c24c5b0 0x55690c201de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 --2- 172.21.15.31:0/2454146015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24bc90 0x55690c24c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 -- 172.21.15.31:0/2454146015 >> 172.21.15.31:0/2454146015 conn(0x55690c1fe2d0 msgr2=0x55690c2006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 -- 172.21.15.31:0/2454146015 shutdown_connections 2023-11-26T07:18:36.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 -- 172.21.15.31:0/2454146015 wait complete. 2023-11-26T07:18:36.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 -- start start 2023-11-26T07:18:36.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 Processor -- start 2023-11-26T07:18:36.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c202320 0x55690c206d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c207280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24c5b0 0x55690c21a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690c0527c0 con 0x55690c24bc90 2023-11-26T07:18:36.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690c00abf0 con 0x55690c202320 2023-11-26T07:18:36.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55690c010a90 con 0x55690c24c5b0 2023-11-26T07:18:36.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f653c882000 10 client.0 _ll_register_callbacks cb 0x55690c1ed3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:18:36.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f652bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c207280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f652bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c207280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55638/0 (socket says 172.21.15.31:55638) 2023-11-26T07:18:36.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.000+0000 7f652bfff640 1 -- 172.21.15.31:0/3761018875 learned_addr learned my addr 172.21.15.31:0/3761018875 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:36.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f65337fe640 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24c5b0 0x55690c21a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6532ffd640 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c202320 0x55690c206d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f652bfff640 1 -- 172.21.15.31:0/3761018875 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24c5b0 msgr2=0x55690c21a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f652bfff640 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24c5b0 0x55690c21a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f652bfff640 1 -- 172.21.15.31:0/3761018875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c202320 msgr2=0x55690c206d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f652bfff640 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c202320 0x55690c206d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f652bfff640 1 -- 172.21.15.31:0/3761018875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652c011040 con 0x55690c24bc90 2023-11-26T07:18:36.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f652bfff640 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c207280 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f651c05cbe0 tx=0x7f651c05b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f13774b31e50b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:18:36.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6511ffb640 1 -- 172.21.15.31:0/3761018875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c06c070 con 0x55690c24bc90 2023-11-26T07:18:36.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6511ffb640 1 -- 172.21.15.31:0/3761018875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f651c05d110 con 0x55690c24bc90 2023-11-26T07:18:36.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 -- 172.21.15.31:0/3761018875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55690c014470 con 0x55690c24bc90 2023-11-26T07:18:36.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6511ffb640 1 -- 172.21.15.31:0/3761018875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f651c05ce80 con 0x55690c24bc90 2023-11-26T07:18:36.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 10 client.5313 fetch_fsmap learned FSMap version 17 2023-11-26T07:18:36.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 -- 172.21.15.31:0/3761018875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55690c013190 con 0x55690c24bc90 2023-11-26T07:18:36.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6511ffb640 1 -- 172.21.15.31:0/3761018875 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55690c014470 con 0x55690c24bc90 2023-11-26T07:18:36.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6511ffb640 1 -- 172.21.15.31:0/3761018875 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f651c068b60 con 0x55690c24bc90 2023-11-26T07:18:36.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 10 client.5313 fetch_fsmap finished waiting for FSMap version 17 2023-11-26T07:18:36.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 10 client.5313 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:18:36.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 20 client.5313 populate_metadata read hostname 'smithi031' 2023-11-26T07:18:36.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.31:6834/4036387633,v1:172.21.15.31:6835/4036387633] conn(0x55690c21b320 0x55690c25df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 4 client.5313 mds_command: new command op to 5106 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:18:36.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 -- 172.21.15.31:0/3761018875 --> [v2:172.21.15.31:6834/4036387633,v1:172.21.15.31:6835/4036387633] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55690c017b50 con 0x55690c21b320 2023-11-26T07:18:36.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6532ffd640 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.31:6834/4036387633,v1:172.21.15.31:6835/4036387633] conn(0x55690c21b320 0x55690c25df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6532ffd640 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.31:6834/4036387633,v1:172.21.15.31:6835/4036387633] conn(0x55690c21b320 0x55690c25df50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9492653e5c22a382 server_cookie=e213c46cf73e06e5 in_seq=0 out_seq=0 2023-11-26T07:18:36.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6511ffb640 10 client.5313 ms_handle_connect on v2:172.21.15.31:6834/4036387633 2023-11-26T07:18:36.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6511ffb640 1 -- 172.21.15.31:0/3761018875 <== mds.0 v2:172.21.15.31:6834/4036387633 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55690c017b50 con 0x55690c21b320 2023-11-26T07:18:36.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6511ffb640 10 client.5313 handle_command_reply: tid=0 2023-11-26T07:18:36.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 client.5313 shutdown 2023-11-26T07:18:36.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 -- 172.21.15.31:0/3761018875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 msgr2=0x55690c207280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c207280 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f651c05cbe0 tx=0x7f651c05b190 comp rx=0 tx=0).stop 2023-11-26T07:18:36.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 -- 172.21.15.31:0/3761018875 shutdown_connections 2023-11-26T07:18:36.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.31:6834/4036387633,v1:172.21.15.31:6835/4036387633] conn(0x55690c21b320 0x55690c25df50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24c5b0 0x55690c21a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c207280 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 --2- 172.21.15.31:0/3761018875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c202320 0x55690c206d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 -- 172.21.15.31:0/3761018875 >> 172.21.15.31:0/3761018875 conn(0x55690c1fe2d0 msgr2=0x55690c2006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f6511ffb640 0 client.5313 ms_handle_reset on v2:172.21.15.31:6834/4036387633 2023-11-26T07:18:36.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 -- 172.21.15.31:0/3761018875 shutdown_connections 2023-11-26T07:18:36.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 1 -- 172.21.15.31:0/3761018875 wait complete. 2023-11-26T07:18:36.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.004+0000 7f653c882000 20 client.5313 trim_cache size 0 max 16384 2023-11-26T07:18:36.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.020+0000 7f653c882000 1 Processor -- start 2023-11-26T07:18:36.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.020+0000 7f653c882000 1 -- start start 2023-11-26T07:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c202320 0x55690c24aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c249b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24c5b0 0x55690c24a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690bd0b940 con 0x55690c24bc90 2023-11-26T07:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690c207fb0 con 0x55690c202320 2023-11-26T07:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55690c015d60 con 0x55690c24c5b0 2023-11-26T07:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f652bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c249b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f652bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c249b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55652/0 (socket says 172.21.15.31:55652) 2023-11-26T07:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f652bfff640 1 -- 172.21.15.31:0/3728657066 learned_addr learned my addr 172.21.15.31:0/3728657066 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:36.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f6532ffd640 1 --2- 172.21.15.31:0/3728657066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c202320 0x55690c24aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f65337fe640 1 --2- 172.21.15.31:0/3728657066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24c5b0 0x55690c24a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f652bfff640 1 -- 172.21.15.31:0/3728657066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24c5b0 msgr2=0x55690c24a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f652bfff640 1 --2- 172.21.15.31:0/3728657066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24c5b0 0x55690c24a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f652bfff640 1 -- 172.21.15.31:0/3728657066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c202320 msgr2=0x55690c24aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f652bfff640 1 --2- 172.21.15.31:0/3728657066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c202320 0x55690c24aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f652bfff640 1 -- 172.21.15.31:0/3728657066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55690bff9490 con 0x55690c24bc90 2023-11-26T07:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f652bfff640 1 --2- 172.21.15.31:0/3728657066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c249b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f651c007410 tx=0x7f651c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f55902f37896ea server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f65297fa640 1 -- 172.21.15.31:0/3728657066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c06c070 con 0x55690c24bc90 2023-11-26T07:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f65297fa640 1 -- 172.21.15.31:0/3728657066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c057030 con 0x55690c24bc90 2023-11-26T07:18:36.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f65297fa640 1 -- 172.21.15.31:0/3728657066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c05cec0 con 0x55690c24bc90 2023-11-26T07:18:36.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- 172.21.15.31:0/3728657066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 msgr2=0x55690c249b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 --2- 172.21.15.31:0/3728657066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c249b30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f651c007410 tx=0x7f651c01b000 comp rx=0 tx=0).stop 2023-11-26T07:18:36.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- 172.21.15.31:0/3728657066 shutdown_connections 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 --2- 172.21.15.31:0/3728657066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c24c5b0 0x55690c24a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 --2- 172.21.15.31:0/3728657066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24bc90 0x55690c249b30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 --2- 172.21.15.31:0/3728657066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c202320 0x55690c24aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- 172.21.15.31:0/3728657066 >> 172.21.15.31:0/3728657066 conn(0x55690c1fe2d0 msgr2=0x55690c259280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- 172.21.15.31:0/3728657066 shutdown_connections 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- 172.21.15.31:0/3728657066 wait complete. 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- start start 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 Processor -- start 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c202320 0x55690c203a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24c5b0 0x55690c22e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c22ed20 0x55690c22f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690bffd3b0 con 0x55690c24c5b0 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55690c04e5c0 con 0x55690c22ed20 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.024+0000 7f653c882000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55690c051fd0 con 0x55690c202320 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 10 client.0 _ll_register_callbacks cb 0x55690c0a0c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f652bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24c5b0 0x55690c22e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f652bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24c5b0 0x55690c22e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55668/0 (socket says 172.21.15.31:55668) 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f652bfff640 1 -- 172.21.15.31:0/281841699 learned_addr learned my addr 172.21.15.31:0/281841699 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f65337fe640 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c22ed20 0x55690c22f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6532ffd640 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c202320 0x55690c203a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f652bfff640 1 -- 172.21.15.31:0/281841699 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c202320 msgr2=0x55690c203a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f652bfff640 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c202320 0x55690c203a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f652bfff640 1 -- 172.21.15.31:0/281841699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c22ed20 msgr2=0x55690c22f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f652bfff640 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c22ed20 0x55690c22f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f652bfff640 1 -- 172.21.15.31:0/281841699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55690c013190 con 0x55690c24c5b0 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f652bfff640 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24c5b0 0x55690c22e7e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f651c007410 tx=0x7f651c082150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c7a7b3968b1238 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6511ffb640 1 -- 172.21.15.31:0/281841699 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c054820 con 0x55690c24c5b0 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6511ffb640 1 -- 172.21.15.31:0/281841699 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f651c05d110 con 0x55690c24c5b0 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 -- 172.21.15.31:0/281841699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55690c051110 con 0x55690c24c5b0 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6511ffb640 1 -- 172.21.15.31:0/281841699 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f651c053e20 con 0x55690c24c5b0 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 10 client.5319 fetch_fsmap learned FSMap version 17 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 -- 172.21.15.31:0/281841699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55690bd4acc0 con 0x55690c24c5b0 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6511ffb640 1 -- 172.21.15.31:0/281841699 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55690c051110 con 0x55690c24c5b0 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6511ffb640 1 -- 172.21.15.31:0/281841699 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f651c053f80 con 0x55690c24c5b0 2023-11-26T07:18:36.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 10 client.5319 fetch_fsmap finished waiting for FSMap version 17 2023-11-26T07:18:36.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 10 client.5319 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:18:36.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 20 client.5319 populate_metadata read hostname 'smithi031' 2023-11-26T07:18:36.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.31:6834/4036387633,v1:172.21.15.31:6835/4036387633] conn(0x55690c25e370 0x55690c25e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 4 client.5319 mds_command: new command op to 5106 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:18:36.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 -- 172.21.15.31:0/281841699 --> [v2:172.21.15.31:6834/4036387633,v1:172.21.15.31:6835/4036387633] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55690c0809a0 con 0x55690c25e370 2023-11-26T07:18:36.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6532ffd640 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.31:6834/4036387633,v1:172.21.15.31:6835/4036387633] conn(0x55690c25e370 0x55690c25e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6511ffb640 10 client.5319 ms_handle_connect on v2:172.21.15.31:6834/4036387633 2023-11-26T07:18:36.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6532ffd640 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.31:6834/4036387633,v1:172.21.15.31:6835/4036387633] conn(0x55690c25e370 0x55690c25e750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71b19437bc2f3d0e server_cookie=30f59853d3f343c in_seq=0 out_seq=0 2023-11-26T07:18:36.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6511ffb640 1 -- 172.21.15.31:0/281841699 <== mds.0 v2:172.21.15.31:6834/4036387633 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55690c0809a0 con 0x55690c25e370 2023-11-26T07:18:36.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6511ffb640 10 client.5319 handle_command_reply: tid=0 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 client.5319 shutdown 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 -- 172.21.15.31:0/281841699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24c5b0 msgr2=0x55690c22e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24c5b0 0x55690c22e7e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f651c007410 tx=0x7f651c082150 comp rx=0 tx=0).stop 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 -- 172.21.15.31:0/281841699 shutdown_connections 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.31:6834/4036387633,v1:172.21.15.31:6835/4036387633] conn(0x55690c25e370 0x55690c25e750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55690c22ed20 0x55690c22f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55690c24c5b0 0x55690c22e7e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f6511ffb640 0 client.5319 ms_handle_reset on v2:172.21.15.31:6834/4036387633 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 --2- 172.21.15.31:0/281841699 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55690c202320 0x55690c203a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 -- 172.21.15.31:0/281841699 >> 172.21.15.31:0/281841699 conn(0x55690c1fe2d0 msgr2=0x55690c2006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 -- 172.21.15.31:0/281841699 shutdown_connections 2023-11-26T07:18:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 1 -- 172.21.15.31:0/281841699 wait complete. 2023-11-26T07:18:36.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653c882000 20 client.5319 trim_cache size 0 max 16384 2023-11-26T07:18:36.040 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:18:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "4c2b4df1-71b1-4841-8807-af144d2610e7", 2023-11-26T07:18:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-11-26T07:18:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5106, 2023-11-26T07:18:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-11-26T07:18:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-11-26T07:18:36.040 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-11-26T07:18:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 10.465485497, 2023-11-26T07:18:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 17, 2023-11-26T07:18:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 33, 2023-11-26T07:18:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 33, 2023-11-26T07:18:36.041 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 18.312842925999998 2023-11-26T07:18:36.041 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:18:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 -- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65140664a0 msgr2=0x7f6514068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65140664a0 0x7f6514068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f65240ae000 tx=0x7f65240b5040 comp rx=0 tx=0).stop 2023-11-26T07:18:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 -- 172.21.15.31:0/594277571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b2870 msgr2=0x7f65340af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b2870 0x7f65340af9e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f651c00c7f0 tx=0x7f651c00ccb0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 -- 172.21.15.31:0/594277571 shutdown_connections 2023-11-26T07:18:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65140664a0 0x7f6514068960 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6534161f20 0x7f65340aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65340b2870 0x7f65340af9e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 --2- 172.21.15.31:0/594277571 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65340b07a0 0x7f6534158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 -- 172.21.15.31:0/594277571 >> 172.21.15.31:0/594277571 conn(0x7f65340b1d50 msgr2=0x7f653415df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.028+0000 7f653951d640 1 -- 172.21.15.31:0/594277571 shutdown_connections 2023-11-26T07:18:36.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.032+0000 7f653951d640 1 -- 172.21.15.31:0/594277571 wait complete. 2023-11-26T07:18:36.055 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:18:36.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.192+0000 7f344cc6b640 1 Processor -- start 2023-11-26T07:18:36.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.192+0000 7f344cc6b640 1 -- start start 2023-11-26T07:18:36.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.192+0000 7f344cc6b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34480716a0 0x7f3448071f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.192+0000 7f344cc6b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3448072470 0x7f34480b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.192+0000 7f344cc6b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3448071b30 con 0x7f344815d3e0 2023-11-26T07:18:36.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.192+0000 7f344cc6b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34480728e0 con 0x7f3448072470 2023-11-26T07:18:36.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.192+0000 7f344cc6b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3448070920 con 0x7f34480716a0 2023-11-26T07:18:36.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.192+0000 7f3446d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3448072470 0x7f34480b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.192+0000 7f3446d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3448072470 0x7f34480b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55680/0 (socket says 172.21.15.31:55680) 2023-11-26T07:18:36.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.192+0000 7f3446d9d640 1 -- 172.21.15.31:0/35329373 learned_addr learned my addr 172.21.15.31:0/35329373 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:36.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f343ffff640 1 --2- 172.21.15.31:0/35329373 >> v2:172.21.15.97:3300/0 conn(0x7f34480716a0 0x7f3448071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f3446d9d640 1 -- 172.21.15.31:0/35329373 >> v2:172.21.15.97:3300/0 conn(0x7f34480716a0 msgr2=0x7f3448071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f3446d9d640 1 --2- 172.21.15.31:0/35329373 >> v2:172.21.15.97:3300/0 conn(0x7f34480716a0 0x7f3448071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f3446d9d640 1 -- 172.21.15.31:0/35329373 >> v1:172.21.15.97:6789/0 conn(0x7f344815d3e0 legacy=0x7f344806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:36.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f3446d9d640 1 -- 172.21.15.31:0/35329373 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344815d7c0 con 0x7f3448072470 2023-11-26T07:18:36.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f3446d9d640 1 --2- 172.21.15.31:0/35329373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3448072470 0x7f34480b1f40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f344806bf20 tx=0x7f344001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3965a7359bf3b5c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f3445d9b640 1 -- 172.21.15.31:0/35329373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34480b2c90 con 0x7f3448072470 2023-11-26T07:18:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f3445d9b640 1 -- 172.21.15.31:0/35329373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34400a5860 con 0x7f3448072470 2023-11-26T07:18:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f3445d9b640 1 -- 172.21.15.31:0/35329373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34400a5bf0 con 0x7f3448072470 2023-11-26T07:18:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f344cc6b640 1 -- 172.21.15.31:0/35329373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3448072470 msgr2=0x7f34480b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f344cc6b640 1 --2- 172.21.15.31:0/35329373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3448072470 0x7f34480b1f40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f344806bf20 tx=0x7f344001c9a0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f344cc6b640 1 -- 172.21.15.31:0/35329373 shutdown_connections 2023-11-26T07:18:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f344cc6b640 1 --2- 172.21.15.31:0/35329373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3448072470 0x7f34480b1f40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f344cc6b640 1 --2- 172.21.15.31:0/35329373 >> v2:172.21.15.97:3300/0 conn(0x7f34480716a0 0x7f3448071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.196+0000 7f344cc6b640 1 -- 172.21.15.31:0/35329373 >> 172.21.15.31:0/35329373 conn(0x7f34480b0c40 msgr2=0x7f34480b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344cc6b640 1 -- 172.21.15.31:0/35329373 shutdown_connections 2023-11-26T07:18:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344cc6b640 1 -- 172.21.15.31:0/35329373 wait complete. 2023-11-26T07:18:36.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344cc6b640 1 Processor -- start 2023-11-26T07:18:36.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344cc6b640 1 -- start start 2023-11-26T07:18:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344cc6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34480716a0 0x7f34480ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344cc6b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3448072470 0x7f344814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344cc6b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344815d3e0 0x7f344814cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344cc6b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344814f0e0 con 0x7f344815d3e0 2023-11-26T07:18:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344cc6b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34480ac7d0 con 0x7f34480716a0 2023-11-26T07:18:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344cc6b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34481547f0 con 0x7f3448072470 2023-11-26T07:18:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f3446d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344815d3e0 0x7f344814cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34480716a0 0x7f34480ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34480716a0 0x7f34480ad220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51288/0 (socket says 172.21.15.31:51288) 2023-11-26T07:18:36.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f3446d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344815d3e0 0x7f344814cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55692/0 (socket says 172.21.15.31:55692) 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344659c640 1 -- 172.21.15.31:0/3725844571 learned_addr learned my addr 172.21.15.31:0/3725844571 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f343ffff640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3448072470 0x7f344814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f3446d9d640 1 -- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3448072470 msgr2=0x7f344814e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f3446d9d640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3448072470 0x7f344814e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f3446d9d640 1 -- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34480716a0 msgr2=0x7f34480ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f3446d9d640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34480716a0 0x7f34480ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f3446d9d640 1 -- 172.21.15.31:0/3725844571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3440009b30 con 0x7f344815d3e0 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f344659c640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34480716a0 0x7f34480ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.200+0000 7f3446d9d640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344815d3e0 0x7f344814cd90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f34400ab000 tx=0x7f3440019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.204+0000 7f343effd640 1 -- 172.21.15.31:0/3725844571 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34400b8670 con 0x7f344815d3e0 2023-11-26T07:18:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.204+0000 7f343effd640 1 -- 172.21.15.31:0/3725844571 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34400b87d0 con 0x7f344815d3e0 2023-11-26T07:18:36.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.204+0000 7f343effd640 1 -- 172.21.15.31:0/3725844571 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34400b8b40 con 0x7f344815d3e0 2023-11-26T07:18:36.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.204+0000 7f344cc6b640 1 -- 172.21.15.31:0/3725844571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3448154ad0 con 0x7f344815d3e0 2023-11-26T07:18:36.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.204+0000 7f344cc6b640 1 -- 172.21.15.31:0/3725844571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34480acd00 con 0x7f344815d3e0 2023-11-26T07:18:36.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.204+0000 7f344cc6b640 1 -- 172.21.15.31:0/3725844571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3448154c60 con 0x7f344815d3e0 2023-11-26T07:18:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.204+0000 7f343effd640 1 -- 172.21.15.31:0/3725844571 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f34400b8ca0 con 0x7f344815d3e0 2023-11-26T07:18:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.204+0000 7f343effd640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3428066470 0x7f3428068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.204+0000 7f343effd640 1 -- 172.21.15.31:0/3725844571 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f344011f440 con 0x7f344815d3e0 2023-11-26T07:18:36.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.204+0000 7f344659c640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3428066470 0x7f3428068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:36.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.208+0000 7f344659c640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3428066470 0x7f3428068930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3438001c00 tx=0x7f3438009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:36.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.208+0000 7f343effd640 1 -- 172.21.15.31:0/3725844571 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f34400eca00 con 0x7f344815d3e0 2023-11-26T07:18:36.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.400+0000 7f344cc6b640 1 -- 172.21.15.31:0/3725844571 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f34480acef0 con 0x7f344815d3e0 2023-11-26T07:18:36.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.400+0000 7f343effd640 1 -- 172.21.15.31:0/3725844571 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f34400f0020 con 0x7f344815d3e0 2023-11-26T07:18:36.407 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:18:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 -- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3428066470 msgr2=0x7f3428068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3428066470 0x7f3428068930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3438001c00 tx=0x7f3438009040 comp rx=0 tx=0).stop 2023-11-26T07:18:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 -- 172.21.15.31:0/3725844571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344815d3e0 msgr2=0x7f344814cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344815d3e0 0x7f344814cd90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f34400ab000 tx=0x7f3440019740 comp rx=0 tx=0).stop 2023-11-26T07:18:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 -- 172.21.15.31:0/3725844571 shutdown_connections 2023-11-26T07:18:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3428066470 0x7f3428068930 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f344815d3e0 0x7f344814cd90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3448072470 0x7f344814e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 --2- 172.21.15.31:0/3725844571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34480716a0 0x7f34480ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 -- 172.21.15.31:0/3725844571 >> 172.21.15.31:0/3725844571 conn(0x7f34480b0c40 msgr2=0x7f34480aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 -- 172.21.15.31:0/3725844571 shutdown_connections 2023-11-26T07:18:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:36.404+0000 7f344cc6b640 1 -- 172.21.15.31:0/3725844571 wait complete. 2023-11-26T07:18:36.428 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:18:36.429 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:18:36.429 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-11-26T07:18:36.429 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:18:36.429 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:36.429 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:18:36.429 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:18:36.430 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:18:36.430 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:18:36.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:36.430 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-11-26T07:18:36.441 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:18:36.442 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:18:36.442 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:18:36.442 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:36.442 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-11-26T07:18:36.443 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:36.443 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:36.443 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:36.443 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-11-26T07:18:36.444 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:36.444 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:36.444 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-11-26T07:18:36.444 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:18:36.444 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:36.444 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-11-26T07:18:36.445 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-26T07:18:36.445 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42323sec preferred_lft 42323sec 2023-11-26T07:18:36.445 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-11-26T07:18:36.445 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:36.445 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:36.445 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:36.445 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-11-26T07:18:36.446 INFO:teuthology.orchestra.run.smithi031.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:18:36.446 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 02:42:1b:e4:a9:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:36.446 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:18:36.446 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:36.446 INFO:teuthology.orchestra.run.smithi031.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:18:36.446 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:36.446 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:18:36.446 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:36.447 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:18:36.447 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:36.447 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:18:36.447 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:18:36.447 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:18:36.447 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:36.447 INFO:teuthology.orchestra.run.smithi031.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:18:36.447 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:18:36.448 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:18:36.448 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:36.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:36.448 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:18:36.504 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-11-26T07:18:36.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:36.506 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-11-26T07:18:36.562 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-11-26T07:18:36.563 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:18:36.563 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-26T07:18:36.563 INFO:teuthology.orchestra.run.smithi031.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:18:36.563 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-26T07:18:36.563 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:18:36.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:36.563 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:18:36.564 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:18:36.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-26T07:18:36.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-26T07:18:36.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-26T07:18:36.564 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:18:36.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:36.658 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:18:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:18:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:18:36.668 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:18:36.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:36.668 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-11-26T07:18:36.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:36.740 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:36.796 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:18:36.796 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:36.797 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:36.849 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:18:36.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:36.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:18:36.947 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:18:36.947 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:18:36.959 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:18:36.959 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:18:36.959 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:18:36.960 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:18:36.960 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:18:36.963 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:18:36.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:36.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:36.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:36.987 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:18:37.054 INFO:teuthology.orchestra.run.smithi031.stdout:client5328 2023-11-26T07:18:37.055 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:37.055 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:37.105 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:18:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.284+0000 7f0c5a72a640 1 Processor -- start 2023-11-26T07:18:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.284+0000 7f0c5a72a640 1 -- start start 2023-11-26T07:18:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.288+0000 7f0c5a72a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0c5415d510 0x7f0c540b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.288+0000 7f0c5a72a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5406cf50 con 0x7f0c540715b0 2023-11-26T07:18:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.288+0000 7f0c5a72a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5406cc20 con 0x7f0c540b3cc0 2023-11-26T07:18:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.288+0000 7f0c5a72a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5415d8f0 con 0x7f0c5415d510 2023-11-26T07:18:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.288+0000 7f0c53fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0c5415d510 0x7f0c540b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.288+0000 7f0c58cc7640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0c540715b0 0x7f0c540ad770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57120/0 (socket says 172.21.15.31:57120) 2023-11-26T07:18:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.288+0000 7f0c58cc7640 1 -- 172.21.15.31:0/2498331805 learned_addr learned my addr 172.21.15.31:0/2498331805 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.288+0000 7f0c53fff640 1 -- 172.21.15.31:0/2498331805 >> v1:172.21.15.97:6789/0 conn(0x7f0c540b3cc0 legacy=0x7f0c5406d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.288+0000 7f0c53fff640 1 -- 172.21.15.31:0/2498331805 >> v1:172.21.15.31:6789/0 conn(0x7f0c540715b0 legacy=0x7f0c540ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.288+0000 7f0c53fff640 1 -- 172.21.15.31:0/2498331805 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c540b31d0 con 0x7f0c5415d510 2023-11-26T07:18:37.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c53fff640 1 --2- 172.21.15.31:0/2498331805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 0x7f0c540b29a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0c440097c0 tx=0x7f0c4401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942208a63c46171f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c52ffd640 1 -- 172.21.15.31:0/2498331805 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c440ad660 con 0x7f0c5415d510 2023-11-26T07:18:37.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c52ffd640 1 -- 172.21.15.31:0/2498331805 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c440ad7c0 con 0x7f0c5415d510 2023-11-26T07:18:37.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c52ffd640 1 -- 172.21.15.31:0/2498331805 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c440adb50 con 0x7f0c5415d510 2023-11-26T07:18:37.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c5a72a640 1 -- 172.21.15.31:0/2498331805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 msgr2=0x7f0c540b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c5a72a640 1 --2- 172.21.15.31:0/2498331805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 0x7f0c540b29a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0c440097c0 tx=0x7f0c4401c450 comp rx=0 tx=0).stop 2023-11-26T07:18:37.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c5a72a640 1 -- 172.21.15.31:0/2498331805 shutdown_connections 2023-11-26T07:18:37.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c5a72a640 1 --2- 172.21.15.31:0/2498331805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 0x7f0c540b29a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c5a72a640 1 -- 172.21.15.31:0/2498331805 >> 172.21.15.31:0/2498331805 conn(0x7f0c540b1e40 msgr2=0x7f0c54070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:37.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c5a72a640 1 -- 172.21.15.31:0/2498331805 shutdown_connections 2023-11-26T07:18:37.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c5a72a640 1 -- 172.21.15.31:0/2498331805 wait complete. 2023-11-26T07:18:37.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c5a72a640 1 Processor -- start 2023-11-26T07:18:37.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.292+0000 7f0c5a72a640 1 -- start start 2023-11-26T07:18:37.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c5a72a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c540715b0 0x7f0c54151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c5a72a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c540b3cc0 0x7f0c540cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c5a72a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 0x7f0c540b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c5a72a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540cd5f0 con 0x7f0c5415d510 2023-11-26T07:18:37.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c5a72a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540cc9e0 con 0x7f0c540715b0 2023-11-26T07:18:37.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c5a72a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540cccc0 con 0x7f0c540b3cc0 2023-11-26T07:18:37.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c58cc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 0x7f0c540b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c53fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c540715b0 0x7f0c54151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c58cc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 0x7f0c540b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55708/0 (socket says 172.21.15.31:55708) 2023-11-26T07:18:37.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c58cc7640 1 -- 172.21.15.31:0/333859368 learned_addr learned my addr 172.21.15.31:0/333859368 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:37.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c537fe640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c540b3cc0 0x7f0c540cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c58cc7640 1 -- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c540b3cc0 msgr2=0x7f0c540cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c58cc7640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c540b3cc0 0x7f0c540cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c58cc7640 1 -- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c540715b0 msgr2=0x7f0c54151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c58cc7640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c540715b0 0x7f0c54151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c58cc7640 1 -- 172.21.15.31:0/333859368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44009420 con 0x7f0c5415d510 2023-11-26T07:18:37.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c58cc7640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 0x7f0c540b0660 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00cbc0 tx=0x7f0c4c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c50ff9640 1 -- 172.21.15.31:0/333859368 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00ad40 con 0x7f0c5415d510 2023-11-26T07:18:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c50ff9640 1 -- 172.21.15.31:0/333859368 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c012400 con 0x7f0c5415d510 2023-11-26T07:18:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c50ff9640 1 -- 172.21.15.31:0/333859368 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c00aea0 con 0x7f0c5415d510 2023-11-26T07:18:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c5a72a640 1 -- 172.21.15.31:0/333859368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c54071f50 con 0x7f0c5415d510 2023-11-26T07:18:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c5a72a640 1 -- 172.21.15.31:0/333859368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c540cd280 con 0x7f0c5415d510 2023-11-26T07:18:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c5a72a640 1 -- 172.21.15.31:0/333859368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c540cd760 con 0x7f0c5415d510 2023-11-26T07:18:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.296+0000 7f0c50ff9640 1 -- 172.21.15.31:0/333859368 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f0c4c007070 con 0x7f0c5415d510 2023-11-26T07:18:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.300+0000 7f0c50ff9640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066470 0x7f0c40068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.300+0000 7f0c50ff9640 1 -- 172.21.15.31:0/333859368 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c4c083c60 con 0x7f0c5415d510 2023-11-26T07:18:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.300+0000 7f0c53fff640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066470 0x7f0c40068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:37.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.300+0000 7f0c53fff640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066470 0x7f0c40068930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c440af000 tx=0x7f0c440a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.304+0000 7f0c50ff9640 1 -- 172.21.15.31:0/333859368 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0c4c051200 con 0x7f0c5415d510 2023-11-26T07:18:37.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 -- 172.21.15.31:0/333859368 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0c5404e850 con 0x7f0c5415d510 2023-11-26T07:18:37.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c50ff9640 1 -- 172.21.15.31:0/333859368 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0c4c017070 con 0x7f0c5415d510 2023-11-26T07:18:37.504 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:18:37.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 -- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066470 msgr2=0x7f0c40068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066470 0x7f0c40068930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0c440af000 tx=0x7f0c440a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:37.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 -- 172.21.15.31:0/333859368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 msgr2=0x7f0c540b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:37.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 0x7f0c540b0660 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00cbc0 tx=0x7f0c4c00a430 comp rx=0 tx=0).stop 2023-11-26T07:18:37.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 -- 172.21.15.31:0/333859368 shutdown_connections 2023-11-26T07:18:37.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066470 0x7f0c40068930 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c5415d510 0x7f0c540b0660 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c540b3cc0 0x7f0c540cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 --2- 172.21.15.31:0/333859368 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c540715b0 0x7f0c54151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:37.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 -- 172.21.15.31:0/333859368 >> 172.21.15.31:0/333859368 conn(0x7f0c540b1e40 msgr2=0x7f0c540ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:37.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 -- 172.21.15.31:0/333859368 shutdown_connections 2023-11-26T07:18:37.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:37.500+0000 7f0c5a72a640 1 -- 172.21.15.31:0/333859368 wait complete. 2023-11-26T07:18:37.525 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:18:37.525 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:18:37.526 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-26T07:18:37.526 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:18:37.526 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:37.526 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:18:37.526 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:18:37.526 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:18:37.526 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:18:37.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:37.526 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-26T07:18:37.537 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:18:37.537 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:18:37.537 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:18:37.537 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:37.537 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-26T07:18:37.537 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:37.537 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:37.537 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42345sec preferred_lft 42345sec 2023-11-26T07:18:37.538 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:61:bc:5b:3d brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:37.539 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:18:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:18:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:18:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:37.540 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:18:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:18:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:18:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:18:37.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:37.541 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:18:37.605 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-26T07:18:37.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:37.608 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-26T07:18:37.663 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-26T07:18:37.663 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:18:37.663 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:18:37.663 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:18:37.663 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:18:37.664 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:18:37.664 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:37.664 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:37.664 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:18:37.664 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:18:37.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:37.664 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:18:37.664 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:18:37.664 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:18:37.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:18:37.665 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:18:37.665 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:18:37.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:37.763 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:18:37.776 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:18:37.776 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:18:37.776 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:18:37.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:37.776 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-26T07:18:37.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:37.842 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:37.893 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:18:37.894 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:18:37.894 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:37.943 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:18:37.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:37.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:18:38.040 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:18:38.041 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:18:38.053 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:18:38.053 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:18:38.054 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:18:38.054 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 1@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:18:38.054 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:18:38.057 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:18:38.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:38.057 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:38.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:38.078 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:18:38.131 INFO:teuthology.orchestra.run.smithi097.stdout:client4907 2023-11-26T07:18:38.131 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:38.131 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:38.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:18:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40bbc06640 1 Processor -- start 2023-11-26T07:18:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40bbc06640 1 -- start start 2023-11-26T07:18:38.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40bbc06640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40b415d8f0 0x7f40b415e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40bbc06640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f40b40ced30 0x7f40b40cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40bbc06640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cf1e0 0x7f40b40d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40bbc06640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b40d4420 con 0x7f40b415d8f0 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40bbc06640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b415ecc0 con 0x7f40b40ced30 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40bbc06640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40cdfb0 con 0x7f40b40cf1e0 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40b99a2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40b415d8f0 0x7f40b415e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40b99a2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40b415d8f0 0x7f40b415e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55716/0 (socket says 172.21.15.31:55716) 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40b99a2640 1 -- 172.21.15.31:0/1716654705 learned_addr learned my addr 172.21.15.31:0/1716654705 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40b91a1640 1 --2- 172.21.15.31:0/1716654705 >> v2:172.21.15.97:3300/0 conn(0x7f40b40ced30 0x7f40b40cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40ba1a3640 1 --2- 172.21.15.31:0/1716654705 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cf1e0 0x7f40b40d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40b99a2640 1 -- 172.21.15.31:0/1716654705 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cf1e0 msgr2=0x7f40b40d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40b99a2640 1 --2- 172.21.15.31:0/1716654705 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cf1e0 0x7f40b40d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40b99a2640 1 -- 172.21.15.31:0/1716654705 >> v2:172.21.15.97:3300/0 conn(0x7f40b40ced30 msgr2=0x7f40b40cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40b99a2640 1 --2- 172.21.15.31:0/1716654705 >> v2:172.21.15.97:3300/0 conn(0x7f40b40ced30 0x7f40b40cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.324+0000 7f40b99a2640 1 -- 172.21.15.31:0/1716654705 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b40d4140 con 0x7f40b415d8f0 2023-11-26T07:18:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40b99a2640 1 --2- 172.21.15.31:0/1716654705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 0x7f40b415e780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009870 tx=0x7f40b001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e08727641058347 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40b89a0640 1 -- 172.21.15.31:0/1716654705 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b00aa5f0 con 0x7f40b415d8f0 2023-11-26T07:18:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40b89a0640 1 -- 172.21.15.31:0/1716654705 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b00aa750 con 0x7f40b415d8f0 2023-11-26T07:18:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40b89a0640 1 -- 172.21.15.31:0/1716654705 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b00aaae0 con 0x7f40b415d8f0 2023-11-26T07:18:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40bbc06640 1 -- 172.21.15.31:0/1716654705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 msgr2=0x7f40b415e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40bbc06640 1 --2- 172.21.15.31:0/1716654705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 0x7f40b415e780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009870 tx=0x7f40b001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40bbc06640 1 -- 172.21.15.31:0/1716654705 shutdown_connections 2023-11-26T07:18:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40bbc06640 1 --2- 172.21.15.31:0/1716654705 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cf1e0 0x7f40b40d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40bbc06640 1 --2- 172.21.15.31:0/1716654705 >> v2:172.21.15.97:3300/0 conn(0x7f40b40ced30 0x7f40b40cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40bbc06640 1 --2- 172.21.15.31:0/1716654705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 0x7f40b415e780 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40bbc06640 1 -- 172.21.15.31:0/1716654705 >> 172.21.15.31:0/1716654705 conn(0x7f40b409c750 msgr2=0x7f40b40d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40bbc06640 1 -- 172.21.15.31:0/1716654705 shutdown_connections 2023-11-26T07:18:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.328+0000 7f40bbc06640 1 -- 172.21.15.31:0/1716654705 wait complete. 2023-11-26T07:18:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 Processor -- start 2023-11-26T07:18:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 -- start start 2023-11-26T07:18:38.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b40ced30 0x7f40b41561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cf1e0 0x7f40b41534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 0x7f40b4153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b4151c30 con 0x7f40b415d8f0 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b41566e0 con 0x7f40b40ced30 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b4150b90 con 0x7f40b40cf1e0 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40ba1a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 0x7f40b4153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40b99a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b40ced30 0x7f40b41561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40ba1a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 0x7f40b4153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55732/0 (socket says 172.21.15.31:55732) 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40b99a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b40ced30 0x7f40b41561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51308/0 (socket says 172.21.15.31:51308) 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40ba1a3640 1 -- 172.21.15.31:0/3104788799 learned_addr learned my addr 172.21.15.31:0/3104788799 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40b91a1640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cf1e0 0x7f40b41534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40ba1a3640 1 -- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cf1e0 msgr2=0x7f40b41534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40ba1a3640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cf1e0 0x7f40b41534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40ba1a3640 1 -- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b40ced30 msgr2=0x7f40b41561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:38.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40ba1a3640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b40ced30 0x7f40b41561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:38.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40ba1a3640 1 -- 172.21.15.31:0/3104788799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00094d0 con 0x7f40b415d8f0 2023-11-26T07:18:38.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40ba1a3640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 0x7f40b4153a00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009fd0 tx=0x7f40ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40aa7fc640 1 -- 172.21.15.31:0/3104788799 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac014420 con 0x7f40b415d8f0 2023-11-26T07:18:38.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40aa7fc640 1 -- 172.21.15.31:0/3104788799 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac014580 con 0x7f40b415d8f0 2023-11-26T07:18:38.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40aa7fc640 1 -- 172.21.15.31:0/3104788799 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac014930 con 0x7f40b415d8f0 2023-11-26T07:18:38.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 -- 172.21.15.31:0/3104788799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b4150e70 con 0x7f40b415d8f0 2023-11-26T07:18:38.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 -- 172.21.15.31:0/3104788799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b4151430 con 0x7f40b415d8f0 2023-11-26T07:18:38.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.332+0000 7f40bbc06640 1 -- 172.21.15.31:0/3104788799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b404e850 con 0x7f40b415d8f0 2023-11-26T07:18:38.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.336+0000 7f40aa7fc640 1 -- 172.21.15.31:0/3104788799 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f40ac014a90 con 0x7f40b415d8f0 2023-11-26T07:18:38.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.336+0000 7f40aa7fc640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4098066470 0x7f4098068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:38.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.336+0000 7f40aa7fc640 1 -- 172.21.15.31:0/3104788799 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40ac0846b0 con 0x7f40b415d8f0 2023-11-26T07:18:38.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.336+0000 7f40b99a2640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4098066470 0x7f4098068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:38.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.336+0000 7f40b99a2640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4098066470 0x7f4098068930 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f40b00ae000 tx=0x7f40b00b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:38.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.340+0000 7f40aa7fc640 1 -- 172.21.15.31:0/3104788799 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f40ac051b50 con 0x7f40b415d8f0 2023-11-26T07:18:38.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:38.528+0000 7f40bbc06640 1 -- 172.21.15.31:0/3104788799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f40b40cf9d0 con 0x7f40b415d8f0 2023-11-26T07:18:39.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.020+0000 7f40aa7fc640 1 -- 172.21.15.31:0/3104788799 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7f40ac055170 con 0x7f40b415d8f0 2023-11-26T07:18:39.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 -- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4098066470 msgr2=0x7f4098068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4098066470 0x7f4098068930 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f40b00ae000 tx=0x7f40b00b5040 comp rx=0 tx=0).stop 2023-11-26T07:18:39.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 -- 172.21.15.31:0/3104788799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 msgr2=0x7f40b4153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 0x7f40b4153a00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009fd0 tx=0x7f40ac00e4b0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 -- 172.21.15.31:0/3104788799 shutdown_connections 2023-11-26T07:18:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4098066470 0x7f4098068930 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b415d8f0 0x7f40b4153a00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cf1e0 0x7f40b41534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 --2- 172.21.15.31:0/3104788799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b40ced30 0x7f40b41561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 -- 172.21.15.31:0/3104788799 >> 172.21.15.31:0/3104788799 conn(0x7f40b409c750 msgr2=0x7f40b40b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 -- 172.21.15.31:0/3104788799 shutdown_connections 2023-11-26T07:18:39.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.024+0000 7f40bbc06640 1 -- 172.21.15.31:0/3104788799 wait complete. 2023-11-26T07:18:39.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:18:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4ab180640 1 Processor -- start 2023-11-26T07:18:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4ab180640 1 -- start start 2023-11-26T07:18:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4ab180640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb4a40b5830 0x7fb4a415e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4ab180640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a40af630 con 0x7fb4a40acd70 2023-11-26T07:18:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4ab180640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a40acb10 con 0x7fb4a40ad8e0 2023-11-26T07:18:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4ab180640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a40ae5b0 con 0x7fb4a40b5830 2023-11-26T07:18:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a3fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb4a40acd70 0x7fb4a40ad1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57130/0 (socket says 172.21.15.31:57130) 2023-11-26T07:18:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a3fff640 1 -- 172.21.15.31:0/3291958974 learned_addr learned my addr 172.21.15.31:0/3291958974 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:39.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a8f1c640 1 --2- 172.21.15.31:0/3291958974 >> v2:172.21.15.97:3300/0 conn(0x7fb4a40b5830 0x7fb4a415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798452277 0 0) 0x7fb490001120 con 0x7fb4a40acd70 2023-11-26T07:18:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb494003530 con 0x7fb4a40acd70 2023-11-26T07:18:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406062501 0 0) 0x7fb4a40acb10 con 0x7fb4a40ad8e0 2023-11-26T07:18:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb490001120 con 0x7fb4a40ad8e0 2023-11-26T07:18:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2995936094 0 0) 0x7fb494003530 con 0x7fb4a40acd70 2023-11-26T07:18:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 >> v2:172.21.15.97:3300/0 conn(0x7fb4a40b5830 msgr2=0x7fb4a415e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 --2- 172.21.15.31:0/3291958974 >> v2:172.21.15.97:3300/0 conn(0x7fb4a40b5830 0x7fb4a415e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 >> v1:172.21.15.97:6789/0 conn(0x7fb4a40ad8e0 legacy=0x7fb4a406d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a40ae870 con 0x7fb4a40acd70 2023-11-26T07:18:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fb490001810 con 0x7fb4a40acd70 2023-11-26T07:18:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 >> v1:172.21.15.31:6789/0 conn(0x7fb4a40acd70 legacy=0x7fb4a40ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb494003b10 0x7fb494003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4940047d0 0x7fb494004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb494005190 0x7fb49400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40ae5b0 con 0x7fb4940047d0 2023-11-26T07:18:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49400c010 con 0x7fb494005190 2023-11-26T07:18:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49400c1d0 con 0x7fb494003b10 2023-11-26T07:18:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a3fff640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4940047d0 0x7fb494004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a8f1c640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb494003b10 0x7fb494003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a971d640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb494005190 0x7fb49400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a3fff640 1 -- 172.21.15.31:0/3291958974 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb494003b10 msgr2=0x7fb494003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:39.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a3fff640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb494003b10 0x7fb494003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a3fff640 1 -- 172.21.15.31:0/3291958974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb494005190 msgr2=0x7fb49400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a3fff640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb494005190 0x7fb49400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a3fff640 1 -- 172.21.15.31:0/3291958974 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb494003530 con 0x7fb4940047d0 2023-11-26T07:18:39.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.192+0000 7fb4a971d640 1 -- 172.21.15.31:0/3291958974 reap_dead start 2023-11-26T07:18:39.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4a3fff640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4940047d0 0x7fb494004c50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb490007fd0 tx=0x7fb49001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9b8571584068a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4900058e0 con 0x7fb4940047d0 2023-11-26T07:18:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb490005a60 con 0x7fb4940047d0 2023-11-26T07:18:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4a37fe640 1 -- 172.21.15.31:0/3291958974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490005e50 con 0x7fb4940047d0 2023-11-26T07:18:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4ab180640 1 -- 172.21.15.31:0/3291958974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4940047d0 msgr2=0x7fb494004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4ab180640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4940047d0 0x7fb494004c50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb490007fd0 tx=0x7fb49001cba0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4ab180640 1 -- 172.21.15.31:0/3291958974 shutdown_connections 2023-11-26T07:18:39.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4ab180640 1 --2- 172.21.15.31:0/3291958974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4940047d0 0x7fb494004c50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4ab180640 1 -- 172.21.15.31:0/3291958974 >> 172.21.15.31:0/3291958974 conn(0x7fb4a4071d80 msgr2=0x7fb4a406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4ab180640 1 -- 172.21.15.31:0/3291958974 shutdown_connections 2023-11-26T07:18:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.196+0000 7fb4ab180640 1 -- 172.21.15.31:0/3291958974 wait complete. 2023-11-26T07:18:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4ab180640 1 Processor -- start 2023-11-26T07:18:39.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4ab180640 1 -- start start 2023-11-26T07:18:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4ab180640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4940047d0 0x7fb4a415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4ab180640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a41582d0 0x7fb4a41f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4ab180640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4a4158750 0x7fb4a4149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4ab180640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a41f74f0 con 0x7fb4a41582d0 2023-11-26T07:18:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4ab180640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a415b290 con 0x7fb4a4158750 2023-11-26T07:18:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4ab180640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a415c2e0 con 0x7fb4940047d0 2023-11-26T07:18:39.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a41582d0 0x7fb4a41f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a41582d0 0x7fb4a41f68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55760/0 (socket says 172.21.15.31:55760) 2023-11-26T07:18:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a3fff640 1 -- 172.21.15.31:0/1864061291 learned_addr learned my addr 172.21.15.31:0/1864061291 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a8f1c640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4940047d0 0x7fb4a415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a971d640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4a4158750 0x7fb4a4149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a3fff640 1 -- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4940047d0 msgr2=0x7fb4a415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a3fff640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4940047d0 0x7fb4a415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a3fff640 1 -- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4a4158750 msgr2=0x7fb4a4149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a3fff640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4a4158750 0x7fb4a4149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a3fff640 1 -- 172.21.15.31:0/1864061291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a415c5c0 con 0x7fb4a41582d0 2023-11-26T07:18:39.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a971d640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4a4158750 0x7fb4a4149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:39.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a8f1c640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4940047d0 0x7fb4a415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:39.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a3fff640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a41582d0 0x7fb4a41f68d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb49001c9d0 tx=0x7fb494005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a17fa640 1 -- 172.21.15.31:0/1864061291 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490004cc0 con 0x7fb4a41582d0 2023-11-26T07:18:39.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.200+0000 7fb4a17fa640 1 -- 172.21.15.31:0/1864061291 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4900b2070 con 0x7fb4a41582d0 2023-11-26T07:18:39.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.204+0000 7fb4ab180640 1 -- 172.21.15.31:0/1864061291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a415c8a0 con 0x7fb4a41582d0 2023-11-26T07:18:39.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.204+0000 7fb4a17fa640 1 -- 172.21.15.31:0/1864061291 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4900a59d0 con 0x7fb4a41582d0 2023-11-26T07:18:39.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.204+0000 7fb4ab180640 1 -- 172.21.15.31:0/1864061291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a41f6e10 con 0x7fb4a41582d0 2023-11-26T07:18:39.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.204+0000 7fb4ab180640 1 -- 172.21.15.31:0/1864061291 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a404e850 con 0x7fb4a41582d0 2023-11-26T07:18:39.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.204+0000 7fb4a17fa640 1 -- 172.21.15.31:0/1864061291 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fb4900a5b30 con 0x7fb4a41582d0 2023-11-26T07:18:39.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.204+0000 7fb4a17fa640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb488066470 0x7fb488068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.204+0000 7fb4a17fa640 1 -- 172.21.15.31:0/1864061291 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb49011d930 con 0x7fb4a41582d0 2023-11-26T07:18:39.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.204+0000 7fb4a8f1c640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb488066470 0x7fb488068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.204+0000 7fb4a8f1c640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb488066470 0x7fb488068930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb498006d50 tx=0x7fb498008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.208+0000 7fb4a17fa640 1 -- 172.21.15.31:0/1864061291 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4900ead10 con 0x7fb4a41582d0 2023-11-26T07:18:39.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.392+0000 7fb4ab180640 1 -- 172.21.15.31:0/1864061291 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4a40af110 con 0x7fb488066470 2023-11-26T07:18:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.392+0000 7fb4a17fa640 1 -- 172.21.15.31:0/1864061291 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb4980138b0 con 0x7fb488066470 2023-11-26T07:18:39.399 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:18:39.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:39.399 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-11-26T07:18:39.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:39.399 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:18:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 -- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb488066470 msgr2=0x7fb488068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb488066470 0x7fb488068930 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb498006d50 tx=0x7fb498008040 comp rx=0 tx=0).stop 2023-11-26T07:18:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 -- 172.21.15.31:0/1864061291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a41582d0 msgr2=0x7fb4a41f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a41582d0 0x7fb4a41f68d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb49001c9d0 tx=0x7fb494005800 comp rx=0 tx=0).stop 2023-11-26T07:18:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 -- 172.21.15.31:0/1864061291 shutdown_connections 2023-11-26T07:18:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb488066470 0x7fb488068930 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4a4158750 0x7fb4a4149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4a41582d0 0x7fb4a41f68d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 --2- 172.21.15.31:0/1864061291 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4940047d0 0x7fb4a415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 -- 172.21.15.31:0/1864061291 >> 172.21.15.31:0/1864061291 conn(0x7fb4a4071d80 msgr2=0x7fb4a406e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 -- 172.21.15.31:0/1864061291 shutdown_connections 2023-11-26T07:18:39.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.396+0000 7fb4ab180640 1 -- 172.21.15.31:0/1864061291 wait complete. 2023-11-26T07:18:39.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:18:39.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.560+0000 7fc551279640 1 Processor -- start 2023-11-26T07:18:39.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.560+0000 7fc551279640 1 -- start start 2023-11-26T07:18:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc551279640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc54c15a710 0x7fc54c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc551279640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c0cbd50 0x7fc54c0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc551279640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c0b2710 con 0x7fc54c0cdfb0 2023-11-26T07:18:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc551279640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c0cca80 con 0x7fc54c15a710 2023-11-26T07:18:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc551279640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0b29f0 con 0x7fc54c0cbd50 2023-11-26T07:18:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc54a7fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc54c15a710 0x7fc54c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc54a7fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc54c15a710 0x7fc54c0abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55764/0 (socket says 172.21.15.31:55764) 2023-11-26T07:18:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc54a7fc640 1 -- 172.21.15.31:0/1520389564 learned_addr learned my addr 172.21.15.31:0/1520389564 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc54b7fe640 1 --2- 172.21.15.31:0/1520389564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c0cbd50 0x7fc54c0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc54a7fc640 1 -- 172.21.15.31:0/1520389564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c0cbd50 msgr2=0x7fc54c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc54a7fc640 1 --2- 172.21.15.31:0/1520389564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c0cbd50 0x7fc54c0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc54a7fc640 1 -- 172.21.15.31:0/1520389564 >> v1:172.21.15.97:6789/0 conn(0x7fc54c0cdfb0 legacy=0x7fc54c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc54a7fc640 1 -- 172.21.15.31:0/1520389564 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c0b2cd0 con 0x7fc54c15a710 2023-11-26T07:18:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc54a7fc640 1 --2- 172.21.15.31:0/1520389564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c15a710 0x7fc54c0abf90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc540009870 tx=0x7fc54001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17f27627fec20f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.564+0000 7fc549ffb640 1 -- 172.21.15.31:0/1520389564 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400a95f0 con 0x7fc54c15a710 2023-11-26T07:18:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc549ffb640 1 -- 172.21.15.31:0/1520389564 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5400a9750 con 0x7fc54c15a710 2023-11-26T07:18:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc549ffb640 1 -- 172.21.15.31:0/1520389564 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400a9ae0 con 0x7fc54c15a710 2023-11-26T07:18:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 -- 172.21.15.31:0/1520389564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c15a710 msgr2=0x7fc54c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 --2- 172.21.15.31:0/1520389564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c15a710 0x7fc54c0abf90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc540009870 tx=0x7fc54001c510 comp rx=0 tx=0).stop 2023-11-26T07:18:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 -- 172.21.15.31:0/1520389564 shutdown_connections 2023-11-26T07:18:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 --2- 172.21.15.31:0/1520389564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c0cbd50 0x7fc54c0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 --2- 172.21.15.31:0/1520389564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c15a710 0x7fc54c0abf90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 -- 172.21.15.31:0/1520389564 >> 172.21.15.31:0/1520389564 conn(0x7fc54c0b0cb0 msgr2=0x7fc54c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 -- 172.21.15.31:0/1520389564 shutdown_connections 2023-11-26T07:18:39.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 -- 172.21.15.31:0/1520389564 wait complete. 2023-11-26T07:18:39.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 Processor -- start 2023-11-26T07:18:39.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 -- start start 2023-11-26T07:18:39.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c0cbd50 0x7fc54c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc54c0cdfb0 0x7fc54c09a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c15a710 0x7fc54c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c13e2e0 con 0x7fc54c0cbd50 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c09af00 con 0x7fc54c0cdfb0 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc551279640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c09b1e0 con 0x7fc54c15a710 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc54affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c0cbd50 0x7fc54c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc54affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c0cbd50 0x7fc54c15d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55768/0 (socket says 172.21.15.31:55768) 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.568+0000 7fc54affd640 1 -- 172.21.15.31:0/3987824413 learned_addr learned my addr 172.21.15.31:0/3987824413 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc54a7fc640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc54c0cdfb0 0x7fc54c09a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc54b7fe640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c15a710 0x7fc54c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc54affd640 1 -- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c15a710 msgr2=0x7fc54c0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc54affd640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c15a710 0x7fc54c0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc54affd640 1 -- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc54c0cdfb0 msgr2=0x7fc54c09a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc54affd640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc54c0cdfb0 0x7fc54c09a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc54affd640 1 -- 172.21.15.31:0/3987824413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5400094d0 con 0x7fc54c0cbd50 2023-11-26T07:18:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc54a7fc640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc54c0cdfb0 0x7fc54c09a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc54b7fe640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c15a710 0x7fc54c0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc54affd640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c0cbd50 0x7fc54c15d630 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c500 tx=0x7fc53c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc5337fe640 1 -- 172.21.15.31:0/3987824413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c005220 con 0x7fc54c0cbd50 2023-11-26T07:18:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc5337fe640 1 -- 172.21.15.31:0/3987824413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c005380 con 0x7fc54c0cbd50 2023-11-26T07:18:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc551279640 1 -- 172.21.15.31:0/3987824413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c09b4c0 con 0x7fc54c0cbd50 2023-11-26T07:18:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc5337fe640 1 -- 172.21.15.31:0/3987824413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c007270 con 0x7fc54c0cbd50 2023-11-26T07:18:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc551279640 1 -- 172.21.15.31:0/3987824413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c13e880 con 0x7fc54c0cbd50 2023-11-26T07:18:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc551279640 1 -- 172.21.15.31:0/3987824413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c0671f0 con 0x7fc54c0cbd50 2023-11-26T07:18:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.572+0000 7fc5337fe640 1 -- 172.21.15.31:0/3987824413 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fc53c0073d0 con 0x7fc54c0cbd50 2023-11-26T07:18:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.576+0000 7fc5337fe640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc538066470 0x7fc538068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.576+0000 7fc5337fe640 1 -- 172.21.15.31:0/3987824413 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc53c083800 con 0x7fc54c0cbd50 2023-11-26T07:18:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.576+0000 7fc54a7fc640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc538066470 0x7fc538068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.576+0000 7fc54a7fc640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc538066470 0x7fc538068930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc5400ae000 tx=0x7fc5400a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:39.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.580+0000 7fc5337fe640 1 -- 172.21.15.31:0/3987824413 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc53c050bf0 con 0x7fc54c0cbd50 2023-11-26T07:18:39.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 -- 172.21.15.31:0/3987824413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc54c06b460 con 0x7fc54c0cbd50 2023-11-26T07:18:39.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc5337fe640 1 -- 172.21.15.31:0/3987824413 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc53c054210 con 0x7fc54c0cbd50 2023-11-26T07:18:39.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 -- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc538066470 msgr2=0x7fc538068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc538066470 0x7fc538068930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc5400ae000 tx=0x7fc5400a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:39.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 -- 172.21.15.31:0/3987824413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c0cbd50 msgr2=0x7fc54c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:39.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c0cbd50 0x7fc54c15d630 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c500 tx=0x7fc53c00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 -- 172.21.15.31:0/3987824413 shutdown_connections 2023-11-26T07:18:39.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc538066470 0x7fc538068930 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc54c15a710 0x7fc54c0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc54c0cdfb0 0x7fc54c09a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 --2- 172.21.15.31:0/3987824413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54c0cbd50 0x7fc54c15d630 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:39.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 -- 172.21.15.31:0/3987824413 >> 172.21.15.31:0/3987824413 conn(0x7fc54c0b0cb0 msgr2=0x7fc54c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:39.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 -- 172.21.15.31:0/3987824413 shutdown_connections 2023-11-26T07:18:39.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:39.736+0000 7fc551279640 1 -- 172.21.15.31:0/3987824413 wait complete. 2023-11-26T07:18:39.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:39.763 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp64210/subvol63760) 2023-11-26T07:18:39.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:39.792 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-11-26T07:18:39.845 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-11-26T07:18:39.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:39.846 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp64210/subvol63760) 2023-11-26T07:18:40.027 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:40.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp64210/subvol63760/.meta 2023-11-26T07:18:40.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol63760 subvol_grp64210 2023-11-26T07:18:40.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.236+0000 7f29a37f8640 1 Processor -- start 2023-11-26T07:18:40.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.236+0000 7f29a37f8640 1 -- start start 2023-11-26T07:18:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.236+0000 7f29a37f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f299c0cb9b0 0x7f299c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.236+0000 7f29a37f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cbd90 0x7f299c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.236+0000 7f29a37f8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f299c0ce5d0 0x7f299c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.236+0000 7f29a37f8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c15bba0 con 0x7f299c0ce5d0 2023-11-26T07:18:40.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.236+0000 7f29a37f8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c0cdd50 con 0x7f299c0cb9b0 2023-11-26T07:18:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.236+0000 7f29a37f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c159630 con 0x7f299c0cbd90 2023-11-26T07:18:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1d95640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f299c0ce5d0 0x7f299c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a0d93640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cbd90 0x7f299c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1594640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f299c0cb9b0 0x7f299c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1594640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f299c0cb9b0 0x7f299c06e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51356/0 (socket says 172.21.15.31:51356) 2023-11-26T07:18:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1594640 1 -- 172.21.15.31:0/3590138586 learned_addr learned my addr 172.21.15.31:0/3590138586 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1594640 1 -- 172.21.15.31:0/3590138586 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cbd90 msgr2=0x7f299c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1594640 1 --2- 172.21.15.31:0/3590138586 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cbd90 0x7f299c15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1594640 1 -- 172.21.15.31:0/3590138586 >> v2:172.21.15.31:3300/0 conn(0x7f299c0ce5d0 msgr2=0x7f299c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:40.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1594640 1 --2- 172.21.15.31:0/3590138586 >> v2:172.21.15.31:3300/0 conn(0x7f299c0ce5d0 0x7f299c15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1594640 1 -- 172.21.15.31:0/3590138586 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c15f6d0 con 0x7f299c0cb9b0 2023-11-26T07:18:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1d95640 1 --2- 172.21.15.31:0/3590138586 >> v2:172.21.15.31:3300/0 conn(0x7f299c0ce5d0 0x7f299c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a0d93640 1 --2- 172.21.15.31:0/3590138586 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cbd90 0x7f299c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:40.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f29a1594640 1 --2- 172.21.15.31:0/3590138586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299c0cb9b0 0x7f299c06e3f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f299c06c360 tx=0x7f299801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f36a4957c0450742 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.240+0000 7f2993fff640 1 -- 172.21.15.31:0/3590138586 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980ab670 con 0x7f299c0cb9b0 2023-11-26T07:18:40.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f2993fff640 1 -- 172.21.15.31:0/3590138586 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29980ab7d0 con 0x7f299c0cb9b0 2023-11-26T07:18:40.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f2993fff640 1 -- 172.21.15.31:0/3590138586 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980abb60 con 0x7f299c0cb9b0 2023-11-26T07:18:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 -- 172.21.15.31:0/3590138586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299c0cb9b0 msgr2=0x7f299c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 --2- 172.21.15.31:0/3590138586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299c0cb9b0 0x7f299c06e3f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f299c06c360 tx=0x7f299801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 -- 172.21.15.31:0/3590138586 shutdown_connections 2023-11-26T07:18:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 --2- 172.21.15.31:0/3590138586 >> v2:172.21.15.31:3300/0 conn(0x7f299c0ce5d0 0x7f299c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 --2- 172.21.15.31:0/3590138586 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cbd90 0x7f299c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 --2- 172.21.15.31:0/3590138586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299c0cb9b0 0x7f299c06e3f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 -- 172.21.15.31:0/3590138586 >> 172.21.15.31:0/3590138586 conn(0x7f299c0b0cb0 msgr2=0x7f299c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 -- 172.21.15.31:0/3590138586 shutdown_connections 2023-11-26T07:18:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 -- 172.21.15.31:0/3590138586 wait complete. 2023-11-26T07:18:40.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 Processor -- start 2023-11-26T07:18:40.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 -- start start 2023-11-26T07:18:40.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cb9b0 0x7f299c0b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299c0cbd90 0x7f299c1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f299c0ce5d0 0x7f299c150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c158fd0 con 0x7f299c0ce5d0 2023-11-26T07:18:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c14d600 con 0x7f299c0cbd90 2023-11-26T07:18:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.244+0000 7f29a37f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c1584e0 con 0x7f299c0cb9b0 2023-11-26T07:18:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1d95640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f299c0ce5d0 0x7f299c150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1594640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cb9b0 0x7f299c0b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1d95640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f299c0ce5d0 0x7f299c150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55796/0 (socket says 172.21.15.31:55796) 2023-11-26T07:18:40.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1d95640 1 -- 172.21.15.31:0/3624291972 learned_addr learned my addr 172.21.15.31:0/3624291972 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a0d93640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299c0cbd90 0x7f299c1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1d95640 1 -- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cb9b0 msgr2=0x7f299c0b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1d95640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cb9b0 0x7f299c0b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1d95640 1 -- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299c0cbd90 msgr2=0x7f299c1551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1d95640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299c0cbd90 0x7f299c1551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1d95640 1 -- 172.21.15.31:0/3624291972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c009420 con 0x7f299c0ce5d0 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1594640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cb9b0 0x7f299c0b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a1d95640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f299c0ce5d0 0x7f299c150d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f299400efc0 tx=0x7f299400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f2991ffb640 1 -- 172.21.15.31:0/3624291972 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994009cd0 con 0x7f299c0ce5d0 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f2991ffb640 1 -- 172.21.15.31:0/3624291972 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299400f040 con 0x7f299c0ce5d0 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f2991ffb640 1 -- 172.21.15.31:0/3624291972 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940039a0 con 0x7f299c0ce5d0 2023-11-26T07:18:40.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a37f8640 1 -- 172.21.15.31:0/3624291972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2994009a70 con 0x7f299c0ce5d0 2023-11-26T07:18:40.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a37f8640 1 -- 172.21.15.31:0/3624291972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29980094d0 con 0x7f299c0ce5d0 2023-11-26T07:18:40.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f29a37f8640 1 -- 172.21.15.31:0/3624291972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299c0671f0 con 0x7f299c0ce5d0 2023-11-26T07:18:40.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f2991ffb640 1 -- 172.21.15.31:0/3624291972 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f2994014480 con 0x7f299c0ce5d0 2023-11-26T07:18:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.248+0000 7f2991ffb640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29800664a0 0x7f2980068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.252+0000 7f2991ffb640 1 -- 172.21.15.31:0/3624291972 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2994084920 con 0x7f299c0ce5d0 2023-11-26T07:18:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.252+0000 7f29a1594640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29800664a0 0x7f2980068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.252+0000 7f29a1594640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29800664a0 0x7f2980068960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f29980af000 tx=0x7f29980b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.252+0000 7f2991ffb640 1 -- 172.21.15.31:0/3624291972 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2994051db0 con 0x7f299c0ce5d0 2023-11-26T07:18:40.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.440+0000 7f29a37f8640 1 -- 172.21.15.31:0/3624291972 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol63760", "group_name": "subvol_grp64210", "target": ["mon-mgr", ""]}) v1 -- 0x7f299c0ccea0 con 0x7f29800664a0 2023-11-26T07:18:40.452 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:40.446+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:40.452 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:40.446+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:40.452 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:40.446+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:40.453 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:40.446+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:40.453 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:40.446+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:40.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.612+0000 7f2991ffb640 1 -- 172.21.15.31:0/3624291972 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f29980ab470 con 0x7f29800664a0 2023-11-26T07:18:40.619 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/subvol_grp64210/subvol63760 2023-11-26T07:18:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 -- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29800664a0 msgr2=0x7f2980068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29800664a0 0x7f2980068960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f29980af000 tx=0x7f29980b6040 comp rx=0 tx=0).stop 2023-11-26T07:18:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 -- 172.21.15.31:0/3624291972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f299c0ce5d0 msgr2=0x7f299c150d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f299c0ce5d0 0x7f299c150d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f299400efc0 tx=0x7f299400c490 comp rx=0 tx=0).stop 2023-11-26T07:18:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 -- 172.21.15.31:0/3624291972 shutdown_connections 2023-11-26T07:18:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29800664a0 0x7f2980068960 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f299c0ce5d0 0x7f299c150d10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f299c0cbd90 0x7f299c1551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 --2- 172.21.15.31:0/3624291972 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f299c0cb9b0 0x7f299c0b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 -- 172.21.15.31:0/3624291972 >> 172.21.15.31:0/3624291972 conn(0x7f299c0b0cb0 msgr2=0x7f299c15f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 -- 172.21.15.31:0/3624291972 shutdown_connections 2023-11-26T07:18:40.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.616+0000 7f29a37f8640 1 -- 172.21.15.31:0/3624291972 wait complete. 2023-11-26T07:18:40.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol63760 subvol_grp64210 2023-11-26T07:18:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.780+0000 7f5a3789a640 1 Processor -- start 2023-11-26T07:18:40.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.780+0000 7f5a3789a640 1 -- start start 2023-11-26T07:18:40.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a3789a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a300b4040 0x7f5a300b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a3789a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a30162190 0x7f5a301625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a3789a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a300b2590 con 0x7f5a300b3c60 2023-11-26T07:18:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a3789a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a300b2870 con 0x7f5a300b4040 2023-11-26T07:18:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a3789a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300b2b50 con 0x7f5a30162190 2023-11-26T07:18:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a34e35640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a300b4040 0x7f5a300b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a35636640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5a300b3c60 0x7f5a300ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57134/0 (socket says 172.21.15.31:57134) 2023-11-26T07:18:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a35636640 1 -- 172.21.15.31:0/2345110547 learned_addr learned my addr 172.21.15.31:0/2345110547 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:40.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a35e37640 1 --2- 172.21.15.31:0/2345110547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a30162190 0x7f5a301625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a34e35640 1 -- 172.21.15.31:0/2345110547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a30162190 msgr2=0x7f5a301625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a34e35640 1 --2- 172.21.15.31:0/2345110547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a30162190 0x7f5a301625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a34e35640 1 -- 172.21.15.31:0/2345110547 >> v1:172.21.15.31:6789/0 conn(0x7f5a300b3c60 legacy=0x7f5a300ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a34e35640 1 -- 172.21.15.31:0/2345110547 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a300b2e30 con 0x7f5a300b4040 2023-11-26T07:18:40.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.784+0000 7f5a27fff640 1 -- 172.21.15.31:0/2345110547 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284261746 0 0) 0x7f5a300b2590 con 0x7f5a300b3c60 2023-11-26T07:18:40.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a34e35640 1 --2- 172.21.15.31:0/2345110547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b4040 0x7f5a300b1fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a200097c0 tx=0x7f5a2001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcf5d7826ca26f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a27fff640 1 -- 172.21.15.31:0/2345110547 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a200ab670 con 0x7f5a300b4040 2023-11-26T07:18:40.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a27fff640 1 -- 172.21.15.31:0/2345110547 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a200ab7d0 con 0x7f5a300b4040 2023-11-26T07:18:40.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a27fff640 1 -- 172.21.15.31:0/2345110547 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a200abb60 con 0x7f5a300b4040 2023-11-26T07:18:40.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a3789a640 1 -- 172.21.15.31:0/2345110547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b4040 msgr2=0x7f5a300b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a3789a640 1 --2- 172.21.15.31:0/2345110547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b4040 0x7f5a300b1fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a200097c0 tx=0x7f5a2001c450 comp rx=0 tx=0).stop 2023-11-26T07:18:40.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a3789a640 1 -- 172.21.15.31:0/2345110547 shutdown_connections 2023-11-26T07:18:40.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a3789a640 1 --2- 172.21.15.31:0/2345110547 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a30162190 0x7f5a301625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a3789a640 1 --2- 172.21.15.31:0/2345110547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b4040 0x7f5a300b1fb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a3789a640 1 -- 172.21.15.31:0/2345110547 >> 172.21.15.31:0/2345110547 conn(0x7f5a300b0cb0 msgr2=0x7f5a300b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:40.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a3789a640 1 -- 172.21.15.31:0/2345110547 shutdown_connections 2023-11-26T07:18:40.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a3789a640 1 -- 172.21.15.31:0/2345110547 wait complete. 2023-11-26T07:18:40.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a3789a640 1 Processor -- start 2023-11-26T07:18:40.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.788+0000 7f5a3789a640 1 -- start start 2023-11-26T07:18:40.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a3789a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b3c60 0x7f5a30155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a3789a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a30162190 0x7f5a30070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a3789a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30070bd0 0x7f5a3015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a3789a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30156140 con 0x7f5a300b3c60 2023-11-26T07:18:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a3789a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a30071110 con 0x7f5a30070bd0 2023-11-26T07:18:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a3789a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a300713f0 con 0x7f5a30162190 2023-11-26T07:18:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35636640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b3c60 0x7f5a30155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35636640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b3c60 0x7f5a30155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55828/0 (socket says 172.21.15.31:55828) 2023-11-26T07:18:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35636640 1 -- 172.21.15.31:0/2346318524 learned_addr learned my addr 172.21.15.31:0/2346318524 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a34e35640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a30162190 0x7f5a30070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35e37640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30070bd0 0x7f5a3015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35636640 1 -- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a30162190 msgr2=0x7f5a30070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35636640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a30162190 0x7f5a30070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35636640 1 -- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30070bd0 msgr2=0x7f5a3015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35636640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30070bd0 0x7f5a3015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35636640 1 -- 172.21.15.31:0/2346318524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a20009420 con 0x7f5a300b3c60 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35e37640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30070bd0 0x7f5a3015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a35636640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b3c60 0x7f5a30155c00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0011f0 tx=0x7f5a2c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a25ffb640 1 -- 172.21.15.31:0/2346318524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0074e0 con 0x7f5a300b3c60 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a25ffb640 1 -- 172.21.15.31:0/2346318524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2c007640 con 0x7f5a300b3c60 2023-11-26T07:18:40.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a25ffb640 1 -- 172.21.15.31:0/2346318524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c004a10 con 0x7f5a300b3c60 2023-11-26T07:18:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a3789a640 1 -- 172.21.15.31:0/2346318524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a300716d0 con 0x7f5a300b3c60 2023-11-26T07:18:40.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a3789a640 1 -- 172.21.15.31:0/2346318524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a30071c90 con 0x7f5a300b3c60 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a3789a640 1 -- 172.21.15.31:0/2346318524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a300671f0 con 0x7f5a300b3c60 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.792+0000 7f5a25ffb640 1 -- 172.21.15.31:0/2346318524 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f5a2c004b70 con 0x7f5a300b3c60 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.796+0000 7f5a25ffb640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a140664a0 0x7f5a14068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.796+0000 7f5a25ffb640 1 -- 172.21.15.31:0/2346318524 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a2c082f80 con 0x7f5a300b3c60 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.796+0000 7f5a34e35640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a140664a0 0x7f5a14068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:40.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.796+0000 7f5a34e35640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a140664a0 0x7f5a14068960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5a200af000 tx=0x7f5a200a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:40.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.800+0000 7f5a25ffb640 1 -- 172.21.15.31:0/2346318524 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5a2c0504a0 con 0x7f5a300b3c60 2023-11-26T07:18:40.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.980+0000 7f5a3789a640 1 -- 172.21.15.31:0/2346318524 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol63760", "group_name": "subvol_grp64210", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a3006b460 con 0x7f5a140664a0 2023-11-26T07:18:40.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.984+0000 7f5a25ffb640 1 -- 172.21.15.31:0/2346318524 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5a3006b460 con 0x7f5a140664a0 2023-11-26T07:18:40.990 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/subvol_grp64210/subvol63760 2023-11-26T07:18:40.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 -- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a140664a0 msgr2=0x7f5a14068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a140664a0 0x7f5a14068960 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5a200af000 tx=0x7f5a200a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 -- 172.21.15.31:0/2346318524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b3c60 msgr2=0x7f5a30155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b3c60 0x7f5a30155c00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0011f0 tx=0x7f5a2c00c9b0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 -- 172.21.15.31:0/2346318524 shutdown_connections 2023-11-26T07:18:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a140664a0 0x7f5a14068960 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a30070bd0 0x7f5a3015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a30162190 0x7f5a30070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 --2- 172.21.15.31:0/2346318524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a300b3c60 0x7f5a30155c00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 -- 172.21.15.31:0/2346318524 >> 172.21.15.31:0/2346318524 conn(0x7f5a300b0cb0 msgr2=0x7f5a300afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 -- 172.21.15.31:0/2346318524 shutdown_connections 2023-11-26T07:18:40.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:40.988+0000 7f5a3789a640 1 -- 172.21.15.31:0/2346318524 wait complete. 2023-11-26T07:18:41.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:41.011 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/6fa8c0c8a2165e8445119b98da353393.meta) 2023-11-26T07:18:41.040 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-11-26T07:18:41.040 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-11-26T07:18:41.040 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-11-26T07:18:41.040 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/subvol_grp64210/subvol63760 2023-11-26T07:18:41.040 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-11-26T07:18:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:41.042 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol63760 subvol_grp64210 2023-11-26T07:18:41.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.224+0000 7f32fe594640 1 Processor -- start 2023-11-26T07:18:41.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.224+0000 7f32fe594640 1 -- start start 2023-11-26T07:18:41.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.224+0000 7f32fe594640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f32f8159f00 0x7f32f80ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.224+0000 7f32fe594640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32f80ae680 0x7f32f80cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.224+0000 7f32fe594640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f8072950 con 0x7f32f80cd920 2023-11-26T07:18:41.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.224+0000 7f32fe594640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f806e2b0 con 0x7f32f80ae680 2023-11-26T07:18:41.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.224+0000 7f32fe594640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f806c6f0 con 0x7f32f8159f00 2023-11-26T07:18:41.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.224+0000 7f32f77fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32f80ae680 0x7f32f80cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.224+0000 7f32fcb31640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f32f80cd920 0x7f32f815d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57136/0 (socket says 172.21.15.31:57136) 2023-11-26T07:18:41.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.224+0000 7f32fcb31640 1 -- 172.21.15.31:0/3420027416 learned_addr learned my addr 172.21.15.31:0/3420027416 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:41.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f7fff640 1 --2- 172.21.15.31:0/3420027416 >> v2:172.21.15.97:3300/0 conn(0x7f32f8159f00 0x7f32f80ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f6ffd640 1 -- 172.21.15.31:0/3420027416 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311240154 0 0) 0x7f32f8072950 con 0x7f32f80cd920 2023-11-26T07:18:41.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f6ffd640 1 -- 172.21.15.31:0/3420027416 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e0003530 con 0x7f32f80cd920 2023-11-26T07:18:41.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f77fe640 1 -- 172.21.15.31:0/3420027416 >> v2:172.21.15.97:3300/0 conn(0x7f32f8159f00 msgr2=0x7f32f80ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:41.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f77fe640 1 --2- 172.21.15.31:0/3420027416 >> v2:172.21.15.97:3300/0 conn(0x7f32f8159f00 0x7f32f80ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f77fe640 1 -- 172.21.15.31:0/3420027416 >> v1:172.21.15.31:6789/0 conn(0x7f32f80cd920 legacy=0x7f32f815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f77fe640 1 -- 172.21.15.31:0/3420027416 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f806e550 con 0x7f32f80ae680 2023-11-26T07:18:41.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f7fff640 1 --2- 172.21.15.31:0/3420027416 >> v2:172.21.15.97:3300/0 conn(0x7f32f8159f00 0x7f32f80ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:41.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f77fe640 1 --2- 172.21.15.31:0/3420027416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f80ae680 0x7f32f80cd3e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f32f806c110 tx=0x7f32ec01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7842888ccf4ee19 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f6ffd640 1 -- 172.21.15.31:0/3420027416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0ab600 con 0x7f32f80ae680 2023-11-26T07:18:41.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f6ffd640 1 -- 172.21.15.31:0/3420027416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec0a7050 con 0x7f32f80ae680 2023-11-26T07:18:41.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.228+0000 7f32f6ffd640 1 -- 172.21.15.31:0/3420027416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0ab980 con 0x7f32f80ae680 2023-11-26T07:18:41.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 -- 172.21.15.31:0/3420027416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f80ae680 msgr2=0x7f32f80cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 --2- 172.21.15.31:0/3420027416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f80ae680 0x7f32f80cd3e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f32f806c110 tx=0x7f32ec01c410 comp rx=0 tx=0).stop 2023-11-26T07:18:41.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 -- 172.21.15.31:0/3420027416 shutdown_connections 2023-11-26T07:18:41.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 --2- 172.21.15.31:0/3420027416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f80ae680 0x7f32f80cd3e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 --2- 172.21.15.31:0/3420027416 >> v2:172.21.15.97:3300/0 conn(0x7f32f8159f00 0x7f32f80ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 -- 172.21.15.31:0/3420027416 >> 172.21.15.31:0/3420027416 conn(0x7f32f80b0cb0 msgr2=0x7f32f80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:41.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 -- 172.21.15.31:0/3420027416 shutdown_connections 2023-11-26T07:18:41.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 -- 172.21.15.31:0/3420027416 wait complete. 2023-11-26T07:18:41.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 Processor -- start 2023-11-26T07:18:41.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 -- start start 2023-11-26T07:18:41.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f80ae680 0x7f32f81f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32f80cd920 0x7f32f814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f814fec0 0x7f32f80afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f814f490 con 0x7f32f814fec0 2023-11-26T07:18:41.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f81f72f0 con 0x7f32f80ae680 2023-11-26T07:18:41.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fe594640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f814eb60 con 0x7f32f80cd920 2023-11-26T07:18:41.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fcb31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f814fec0 0x7f32f80afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fcb31640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f814fec0 0x7f32f80afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55844/0 (socket says 172.21.15.31:55844) 2023-11-26T07:18:41.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fcb31640 1 -- 172.21.15.31:0/2777318442 learned_addr learned my addr 172.21.15.31:0/2777318442 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:41.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32f7fff640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f80ae680 0x7f32f81f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32f77fe640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32f80cd920 0x7f32f814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fcb31640 1 -- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32f80cd920 msgr2=0x7f32f814a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fcb31640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32f80cd920 0x7f32f814a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fcb31640 1 -- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f80ae680 msgr2=0x7f32f81f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fcb31640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f80ae680 0x7f32f81f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fcb31640 1 -- 172.21.15.31:0/2777318442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec009420 con 0x7f32f814fec0 2023-11-26T07:18:41.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.232+0000 7f32fcb31640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f814fec0 0x7f32f80afcc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f32f000cb40 tx=0x7f32f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.236+0000 7f32f4ff9640 1 -- 172.21.15.31:0/2777318442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f000aca0 con 0x7f32f814fec0 2023-11-26T07:18:41.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.236+0000 7f32f4ff9640 1 -- 172.21.15.31:0/2777318442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f000ae00 con 0x7f32f814fec0 2023-11-26T07:18:41.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.236+0000 7f32f4ff9640 1 -- 172.21.15.31:0/2777318442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0012620 con 0x7f32f814fec0 2023-11-26T07:18:41.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.236+0000 7f32fe594640 1 -- 172.21.15.31:0/2777318442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f814ee00 con 0x7f32f814fec0 2023-11-26T07:18:41.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.236+0000 7f32fe594640 1 -- 172.21.15.31:0/2777318442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f814f6c0 con 0x7f32f814fec0 2023-11-26T07:18:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.236+0000 7f32fe594640 1 -- 172.21.15.31:0/2777318442 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f80671f0 con 0x7f32f814fec0 2023-11-26T07:18:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.236+0000 7f32f4ff9640 1 -- 172.21.15.31:0/2777318442 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f32f0007070 con 0x7f32f814fec0 2023-11-26T07:18:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.236+0000 7f32f4ff9640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32e4066470 0x7f32e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.236+0000 7f32f4ff9640 1 -- 172.21.15.31:0/2777318442 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32f0083ef0 con 0x7f32f814fec0 2023-11-26T07:18:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.236+0000 7f32f7fff640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32e4066470 0x7f32e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.240+0000 7f32f7fff640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32e4066470 0x7f32e4068930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f32e80045f0 tx=0x7f32e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.240+0000 7f32f4ff9640 1 -- 172.21.15.31:0/2777318442 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f32f0051380 con 0x7f32f814fec0 2023-11-26T07:18:41.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.420+0000 7f32fe594640 1 -- 172.21.15.31:0/2777318442 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol63760", "group_name": "subvol_grp64210", "target": ["mon-mgr", ""]}) v1 -- 0x7f32f806b460 con 0x7f32e4066470 2023-11-26T07:18:41.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32f4ff9640 1 -- 172.21.15.31:0/2777318442 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f32f806b460 con 0x7f32e4066470 2023-11-26T07:18:41.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 -- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32e4066470 msgr2=0x7f32e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32e4066470 0x7f32e4068930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f32e80045f0 tx=0x7f32e8008040 comp rx=0 tx=0).stop 2023-11-26T07:18:41.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 -- 172.21.15.31:0/2777318442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f814fec0 msgr2=0x7f32f80afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f814fec0 0x7f32f80afcc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f32f000cb40 tx=0x7f32f000a060 comp rx=0 tx=0).stop 2023-11-26T07:18:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 -- 172.21.15.31:0/2777318442 shutdown_connections 2023-11-26T07:18:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32e4066470 0x7f32e4068930 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f814fec0 0x7f32f80afcc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32f80cd920 0x7f32f814a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 --2- 172.21.15.31:0/2777318442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f80ae680 0x7f32f81f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 -- 172.21.15.31:0/2777318442 >> 172.21.15.31:0/2777318442 conn(0x7f32f80b0cb0 msgr2=0x7f32f8154ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 -- 172.21.15.31:0/2777318442 shutdown_connections 2023-11-26T07:18:41.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.440+0000 7f32fe594640 1 -- 172.21.15.31:0/2777318442 wait complete. 2023-11-26T07:18:41.449 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:41.442+0000 7f240399c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:41.449 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:41.442+0000 7f240399c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:41.449 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:41.442+0000 7f240399c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:41.449 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:41.442+0000 7f240399c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:41.449 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:41.442+0000 7f240399c640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:41.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:41.467 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-26T07:18:41.469 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:41.462+0000 7f240319b640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:41.469 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:41.462+0000 7f240319b640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:41.469 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:41.462+0000 7f240319b640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:41.469 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:41.462+0000 7f240319b640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:41.470 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:18:41.462+0000 7f240319b640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:18:41.489 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-11-26T07:18:41.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp64210 2023-11-26T07:18:41.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.672+0000 7f8b917d0640 1 Processor -- start 2023-11-26T07:18:41.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.672+0000 7f8b917d0640 1 -- start start 2023-11-26T07:18:41.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b917d0640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0b4c40 0x7f8b8c0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b917d0640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c0b6250 con 0x7f8b8c15d2a0 2023-11-26T07:18:41.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b917d0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c161cf0 con 0x7f8b8c0d10e0 2023-11-26T07:18:41.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b917d0640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0ce8c0 con 0x7f8b8c0b4c40 2023-11-26T07:18:41.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8b8c15d2a0 0x7f8b8c0d26e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57146/0 (socket says 172.21.15.31:57146) 2023-11-26T07:18:41.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8bfff640 1 -- 172.21.15.31:0/1231392612 learned_addr learned my addr 172.21.15.31:0/1231392612 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:41.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8b7fe640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0b4c40 0x7f8b8c0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228907641 0 0) 0x7f8b8c0b6250 con 0x7f8b8c15d2a0 2023-11-26T07:18:41.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b74003530 con 0x7f8b8c15d2a0 2023-11-26T07:18:41.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651933020 0 0) 0x7f8b8c161cf0 con 0x7f8b8c0d10e0 2023-11-26T07:18:41.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b8c0b6250 con 0x7f8b8c0d10e0 2023-11-26T07:18:41.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 197935645 0 0) 0x7f8b74003530 con 0x7f8b8c15d2a0 2023-11-26T07:18:41.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0b4c40 msgr2=0x7f8b8c0b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0b4c40 0x7f8b8c0b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 >> v1:172.21.15.97:6789/0 conn(0x7f8b8c0d10e0 legacy=0x7f8b8c0d14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c161170 con 0x7f8b8c15d2a0 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f8b7c0016a0 con 0x7f8b8c15d2a0 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8b7fe640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0b4c40 0x7f8b8c0b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 >> v1:172.21.15.31:6789/0 conn(0x7f8b8c15d2a0 legacy=0x7f8b8c0d26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b74003b40 0x7f8b74003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b740047d0 0x7f8b74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0b4c40 0x7f8b74005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0ce8c0 con 0x7f8b740047d0 2023-11-26T07:18:41.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7400b9c0 con 0x7f8b74003b40 2023-11-26T07:18:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7400bb80 con 0x7f8b8c0b4c40 2023-11-26T07:18:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8b7fe640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b740047d0 0x7f8b74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8bfff640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b74003b40 0x7f8b74003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b90fcf640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0b4c40 0x7f8b74005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8b7fe640 1 -- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0b4c40 msgr2=0x7f8b74005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8b7fe640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0b4c40 0x7f8b74005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8b7fe640 1 -- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b74003b40 msgr2=0x7f8b74003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8b7fe640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b74003b40 0x7f8b74003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b8b7fe640 1 -- 172.21.15.31:0/1231392612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b80009420 con 0x7f8b740047d0 2023-11-26T07:18:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.676+0000 7f8b90fcf640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0b4c40 0x7f8b74005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:41.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b8b7fe640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b740047d0 0x7f8b74004c50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009c80 tx=0x7f8b8001cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3895d56107bb864 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b800a5850 con 0x7f8b740047d0 2023-11-26T07:18:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b800a59b0 con 0x7f8b740047d0 2023-11-26T07:18:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b8affd640 1 -- 172.21.15.31:0/1231392612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b800a5d60 con 0x7f8b740047d0 2023-11-26T07:18:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b917d0640 1 -- 172.21.15.31:0/1231392612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b740047d0 msgr2=0x7f8b74004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b917d0640 1 --2- 172.21.15.31:0/1231392612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b740047d0 0x7f8b74004c50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009c80 tx=0x7f8b8001cb90 comp rx=0 tx=0).stop 2023-11-26T07:18:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b90fcf640 1 -- 172.21.15.31:0/1231392612 reap_dead start 2023-11-26T07:18:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b917d0640 1 -- 172.21.15.31:0/1231392612 shutdown_connections 2023-11-26T07:18:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b917d0640 1 -- 172.21.15.31:0/1231392612 >> 172.21.15.31:0/1231392612 conn(0x7f8b8c0668a0 msgr2=0x7f8b8c0cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b917d0640 1 -- 172.21.15.31:0/1231392612 shutdown_connections 2023-11-26T07:18:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.680+0000 7f8b917d0640 1 -- 172.21.15.31:0/1231392612 wait complete. 2023-11-26T07:18:41.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 Processor -- start 2023-11-26T07:18:41.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 -- start start 2023-11-26T07:18:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b740047d0 0x7f8b8c15a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0cd4b0 0x7f8b8c0a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c0aa5c0 0x7f8b8c0aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0aaf90 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c15ab30 con 0x7f8b740047d0 2023-11-26T07:18:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0cc410 con 0x7f8b8c0cd4b0 2023-11-26T07:18:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b90fcf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c0aa5c0 0x7f8b8c0aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b90fcf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c0aa5c0 0x7f8b8c0aaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55854/0 (socket says 172.21.15.31:55854) 2023-11-26T07:18:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b90fcf640 1 -- 172.21.15.31:0/2254012344 learned_addr learned my addr 172.21.15.31:0/2254012344 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b8b7fe640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0cd4b0 0x7f8b8c0a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b8bfff640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b740047d0 0x7f8b8c15a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b90fcf640 1 -- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0cd4b0 msgr2=0x7f8b8c0a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b90fcf640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0cd4b0 0x7f8b8c0a9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b90fcf640 1 -- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b740047d0 msgr2=0x7f8b8c15a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b90fcf640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b740047d0 0x7f8b8c15a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b90fcf640 1 -- 172.21.15.31:0/2254012344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c0cc6f0 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b90fcf640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c0aa5c0 0x7f8b8c0aaa50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8b84007e70 tx=0x7f8b8400f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b88ff9640 1 -- 172.21.15.31:0/2254012344 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400fda0 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b88ff9640 1 -- 172.21.15.31:0/2254012344 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b84015410 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b88ff9640 1 -- 172.21.15.31:0/2254012344 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b840157c0 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 -- 172.21.15.31:0/2254012344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c0cc9d0 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 -- 172.21.15.31:0/2254012344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c0ccf90 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.684+0000 7f8b917d0640 1 -- 172.21.15.31:0/2254012344 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c0cd180 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.688+0000 7f8b88ff9640 1 -- 172.21.15.31:0/2254012344 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f8b84015920 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.688+0000 7f8b88ff9640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b780664a0 0x7f8b78068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:41.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.688+0000 7f8b88ff9640 1 -- 172.21.15.31:0/2254012344 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b84087280 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.688+0000 7f8b8bfff640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b780664a0 0x7f8b78068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:41.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.688+0000 7f8b8bfff640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b780664a0 0x7f8b78068960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c10 tx=0x7f8b7c001e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:41.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.692+0000 7f8b88ff9640 1 -- 172.21.15.31:0/2254012344 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b84054710 con 0x7f8b8c0aa5c0 2023-11-26T07:18:41.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.876+0000 7f8b917d0640 1 -- 172.21.15.31:0/2254012344 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp64210", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b8c161a10 con 0x7f8b780664a0 2023-11-26T07:18:41.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.880+0000 7f8b88ff9640 1 -- 172.21.15.31:0/2254012344 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8b7c001680 con 0x7f8b780664a0 2023-11-26T07:18:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 -- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b780664a0 msgr2=0x7f8b78068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b780664a0 0x7f8b78068960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c10 tx=0x7f8b7c001e50 comp rx=0 tx=0).stop 2023-11-26T07:18:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 -- 172.21.15.31:0/2254012344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c0aa5c0 msgr2=0x7f8b8c0aaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c0aa5c0 0x7f8b8c0aaa50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8b84007e70 tx=0x7f8b8400f540 comp rx=0 tx=0).stop 2023-11-26T07:18:41.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 -- 172.21.15.31:0/2254012344 shutdown_connections 2023-11-26T07:18:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8b780664a0 0x7f8b78068960 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b8c0aa5c0 0x7f8b8c0aaa50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8b8c0cd4b0 0x7f8b8c0a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 --2- 172.21.15.31:0/2254012344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b740047d0 0x7f8b8c15a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 -- 172.21.15.31:0/2254012344 >> 172.21.15.31:0/2254012344 conn(0x7f8b8c0668a0 msgr2=0x7f8b8c0cfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 -- 172.21.15.31:0/2254012344 shutdown_connections 2023-11-26T07:18:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:41.884+0000 7f8b917d0640 1 -- 172.21.15.31:0/2254012344 wait complete. 2023-11-26T07:18:41.907 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:18:41.907 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:18:41.998 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:18:42.098 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:18:42.098 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:42.098 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:42.107 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:42.107 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:42.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:42.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7ec079640 1 Processor -- start 2023-11-26T07:18:42.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7ec079640 1 -- start start 2023-11-26T07:18:42.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7ec079640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa7e415f540 0x7fa7e40b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7ec079640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e406cd40 0x7fa7e4070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7ec079640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e40b2920 con 0x7fa7e40b1f40 2023-11-26T07:18:42.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7ec079640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e40b2c00 con 0x7fa7e415f540 2023-11-26T07:18:42.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7ec079640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40b2ee0 con 0x7fa7e406cd40 2023-11-26T07:18:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7e9e15640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa7e415f540 0x7fa7e40b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7ea616640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e406cd40 0x7fa7e4070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7e9614640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fa7e40b1f40 0x7fa7e406c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:39722/0 (socket says 172.21.15.31:39722) 2023-11-26T07:18:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.340+0000 7fa7e9614640 1 -- 172.21.15.31:0/3414679779 learned_addr learned my addr 172.21.15.31:0/3414679779 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7ea616640 1 -- 172.21.15.31:0/3414679779 >> v2:172.21.15.97:3300/0 conn(0x7fa7e415f540 msgr2=0x7fa7e40b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7ea616640 1 --2- 172.21.15.31:0/3414679779 >> v2:172.21.15.97:3300/0 conn(0x7fa7e415f540 0x7fa7e40b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7ea616640 1 -- 172.21.15.31:0/3414679779 >> v1:172.21.15.97:6789/0 conn(0x7fa7e40b1f40 legacy=0x7fa7e406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7ea616640 1 -- 172.21.15.31:0/3414679779 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e40b31c0 con 0x7fa7e406cd40 2023-11-26T07:18:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7e9e15640 1 --2- 172.21.15.31:0/3414679779 >> v2:172.21.15.97:3300/0 conn(0x7fa7e415f540 0x7fa7e40b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7ea616640 1 --2- 172.21.15.31:0/3414679779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e406cd40 0x7fa7e4070880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009d90 tx=0x7fa7dc01c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5553846668a71e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:42.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7e8e13640 1 -- 172.21.15.31:0/3414679779 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc0a5a00 con 0x7fa7e406cd40 2023-11-26T07:18:42.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7e8e13640 1 -- 172.21.15.31:0/3414679779 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc0a5b60 con 0x7fa7e406cd40 2023-11-26T07:18:42.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7e8e13640 1 -- 172.21.15.31:0/3414679779 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc0ad440 con 0x7fa7e406cd40 2023-11-26T07:18:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7ec079640 1 -- 172.21.15.31:0/3414679779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e406cd40 msgr2=0x7fa7e4070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.344+0000 7fa7ec079640 1 --2- 172.21.15.31:0/3414679779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e406cd40 0x7fa7e4070880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009d90 tx=0x7fa7dc01c920 comp rx=0 tx=0).stop 2023-11-26T07:18:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 -- 172.21.15.31:0/3414679779 shutdown_connections 2023-11-26T07:18:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 --2- 172.21.15.31:0/3414679779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e406cd40 0x7fa7e4070880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 --2- 172.21.15.31:0/3414679779 >> v2:172.21.15.97:3300/0 conn(0x7fa7e415f540 0x7fa7e40b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 -- 172.21.15.31:0/3414679779 >> 172.21.15.31:0/3414679779 conn(0x7fa7e40b0c40 msgr2=0x7fa7e40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 -- 172.21.15.31:0/3414679779 shutdown_connections 2023-11-26T07:18:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 -- 172.21.15.31:0/3414679779 wait complete. 2023-11-26T07:18:42.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 Processor -- start 2023-11-26T07:18:42.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 -- start start 2023-11-26T07:18:42.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e406cd40 0x7fa7e40ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e40b1f40 0x7fa7e4151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e415f540 0x7fa7e41584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4157200 con 0x7fa7e406cd40 2023-11-26T07:18:42.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4151c40 con 0x7fa7e415f540 2023-11-26T07:18:42.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ec079640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4157450 con 0x7fa7e40b1f40 2023-11-26T07:18:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7e9e15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e406cd40 0x7fa7e40ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7e9e15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e406cd40 0x7fa7e40ce480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55856/0 (socket says 172.21.15.31:55856) 2023-11-26T07:18:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7e9e15640 1 -- 172.21.15.31:0/3300459017 learned_addr learned my addr 172.21.15.31:0/3300459017 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7ea616640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e415f540 0x7fa7e41584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7e9614640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e40b1f40 0x7fa7e4151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7e9e15640 1 -- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e40b1f40 msgr2=0x7fa7e4151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7e9e15640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e40b1f40 0x7fa7e4151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7e9e15640 1 -- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e415f540 msgr2=0x7fa7e41584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7e9e15640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e415f540 0x7fa7e41584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.348+0000 7fa7e9e15640 1 -- 172.21.15.31:0/3300459017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e0009420 con 0x7fa7e406cd40 2023-11-26T07:18:42.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7e9e15640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e406cd40 0x7fa7e40ce480 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000e980 tx=0x7fa7e000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:42.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7da7fc640 1 -- 172.21.15.31:0/3300459017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000ca00 con 0x7fa7e406cd40 2023-11-26T07:18:42.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7da7fc640 1 -- 172.21.15.31:0/3300459017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e000cb60 con 0x7fa7e406cd40 2023-11-26T07:18:42.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7ec079640 1 -- 172.21.15.31:0/3300459017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc009a70 con 0x7fa7e406cd40 2023-11-26T07:18:42.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7da7fc640 1 -- 172.21.15.31:0/3300459017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0013470 con 0x7fa7e406cd40 2023-11-26T07:18:42.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7ec079640 1 -- 172.21.15.31:0/3300459017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e4157a10 con 0x7fa7e406cd40 2023-11-26T07:18:42.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7ec079640 1 -- 172.21.15.31:0/3300459017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e404e850 con 0x7fa7e406cd40 2023-11-26T07:18:42.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7da7fc640 1 -- 172.21.15.31:0/3300459017 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7fa7e00135d0 con 0x7fa7e406cd40 2023-11-26T07:18:42.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7da7fc640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7c8066470 0x7fa7c8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7da7fc640 1 -- 172.21.15.31:0/3300459017 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7e0084050 con 0x7fa7e406cd40 2023-11-26T07:18:42.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.352+0000 7fa7e9614640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7c8066470 0x7fa7c8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.356+0000 7fa7e9614640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7c8066470 0x7fa7c8068930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4160780 tx=0x7fa7d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:42.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.356+0000 7fa7da7fc640 1 -- 172.21.15.31:0/3300459017 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7e00b5280 con 0x7fa7e406cd40 2023-11-26T07:18:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.516+0000 7fa7ec079640 1 -- 172.21.15.31:0/3300459017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7e40ced70 con 0x7fa7e406cd40 2023-11-26T07:18:42.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.516+0000 7fa7da7fc640 1 -- 172.21.15.31:0/3300459017 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16485 (secure 0 0 0) 0x7fa7e0051570 con 0x7fa7e406cd40 2023-11-26T07:18:42.523 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:42.523 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:26.567047+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:22.773068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:23.967477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:42.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 -- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7c8066470 msgr2=0x7fa7c8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7c8066470 0x7fa7c8068930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4160780 tx=0x7fa7d4009040 comp rx=0 tx=0).stop 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 -- 172.21.15.31:0/3300459017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e406cd40 msgr2=0x7fa7e40ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e406cd40 0x7fa7e40ce480 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000e980 tx=0x7fa7e000ee40 comp rx=0 tx=0).stop 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 -- 172.21.15.31:0/3300459017 shutdown_connections 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7c8066470 0x7fa7c8068930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7e415f540 0x7fa7e41584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7e40b1f40 0x7fa7e4151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 --2- 172.21.15.31:0/3300459017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7e406cd40 0x7fa7e40ce480 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 -- 172.21.15.31:0/3300459017 >> 172.21.15.31:0/3300459017 conn(0x7fa7e40b0c40 msgr2=0x7fa7e4155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 -- 172.21.15.31:0/3300459017 shutdown_connections 2023-11-26T07:18:42.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.520+0000 7fa7ec079640 1 -- 172.21.15.31:0/3300459017 wait complete. 2023-11-26T07:18:42.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:18:22.773068+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:18:23.967477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:42.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.684+0000 7f29c2017640 1 Processor -- start 2023-11-26T07:18:42.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.684+0000 7f29c2017640 1 -- start start 2023-11-26T07:18:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.684+0000 7f29c2017640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29bc0b07a0 0x7f29bc1619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.684+0000 7f29c2017640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29bc161f20 0x7f29bc0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.684+0000 7f29c2017640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f29bc0b2870 0x7f29bc0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.684+0000 7f29c2017640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc0b0b80 con 0x7f29bc161f20 2023-11-26T07:18:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.684+0000 7f29c2017640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc161290 con 0x7f29bc0b2870 2023-11-26T07:18:42.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.684+0000 7f29c2017640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc161550 con 0x7f29bc0b07a0 2023-11-26T07:18:42.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.684+0000 7f29baffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29bc161f20 0x7f29bc0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29baffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29bc161f20 0x7f29bc0b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55868/0 (socket says 172.21.15.31:55868) 2023-11-26T07:18:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29baffd640 1 -- 172.21.15.31:0/3601774342 learned_addr learned my addr 172.21.15.31:0/3601774342 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29bb7fe640 1 --2- 172.21.15.31:0/3601774342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29bc0b07a0 0x7f29bc1619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29bbfff640 1 --2- 172.21.15.31:0/3601774342 >> v2:172.21.15.97:3300/0 conn(0x7f29bc0b2870 0x7f29bc0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29baffd640 1 -- 172.21.15.31:0/3601774342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29bc0b07a0 msgr2=0x7f29bc1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29baffd640 1 --2- 172.21.15.31:0/3601774342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29bc0b07a0 0x7f29bc1619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29baffd640 1 -- 172.21.15.31:0/3601774342 >> v2:172.21.15.97:3300/0 conn(0x7f29bc0b2870 msgr2=0x7f29bc0b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29baffd640 1 --2- 172.21.15.31:0/3601774342 >> v2:172.21.15.97:3300/0 conn(0x7f29bc0b2870 0x7f29bc0b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29baffd640 1 -- 172.21.15.31:0/3601774342 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0c65a0 con 0x7f29bc161f20 2023-11-26T07:18:42.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29baffd640 1 --2- 172.21.15.31:0/3601774342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc161f20 0x7f29bc0b0ee0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f29bc06c650 tx=0x7f29b001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a893b68a511e1bff server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:42.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29ba7fc640 1 -- 172.21.15.31:0/3601774342 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00aa5f0 con 0x7f29bc161f20 2023-11-26T07:18:42.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29ba7fc640 1 -- 172.21.15.31:0/3601774342 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b00aa750 con 0x7f29bc161f20 2023-11-26T07:18:42.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.688+0000 7f29ba7fc640 1 -- 172.21.15.31:0/3601774342 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00aaae0 con 0x7f29bc161f20 2023-11-26T07:18:42.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 -- 172.21.15.31:0/3601774342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc161f20 msgr2=0x7f29bc0b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 --2- 172.21.15.31:0/3601774342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc161f20 0x7f29bc0b0ee0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f29bc06c650 tx=0x7f29b001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 -- 172.21.15.31:0/3601774342 shutdown_connections 2023-11-26T07:18:42.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 --2- 172.21.15.31:0/3601774342 >> v2:172.21.15.97:3300/0 conn(0x7f29bc0b2870 0x7f29bc0b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 --2- 172.21.15.31:0/3601774342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc161f20 0x7f29bc0b0ee0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 --2- 172.21.15.31:0/3601774342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29bc0b07a0 0x7f29bc1619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 -- 172.21.15.31:0/3601774342 >> 172.21.15.31:0/3601774342 conn(0x7f29bc0b1d50 msgr2=0x7f29bc0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 -- 172.21.15.31:0/3601774342 shutdown_connections 2023-11-26T07:18:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 -- 172.21.15.31:0/3601774342 wait complete. 2023-11-26T07:18:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 Processor -- start 2023-11-26T07:18:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 -- start start 2023-11-26T07:18:42.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc0b07a0 0x7f29bc0af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc0b2870 0x7f29bc0af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29bc161f20 0x7f29bc14e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc159920 con 0x7f29bc0b07a0 2023-11-26T07:18:42.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0b2e40 con 0x7f29bc0b2870 2023-11-26T07:18:42.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29c2017640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0afdb0 con 0x7f29bc161f20 2023-11-26T07:18:42.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc0b07a0 0x7f29bc0af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc0b07a0 0x7f29bc0af330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55876/0 (socket says 172.21.15.31:55876) 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29bb7fe640 1 -- 172.21.15.31:0/2300016497 learned_addr learned my addr 172.21.15.31:0/2300016497 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29bbfff640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29bc161f20 0x7f29bc14e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29baffd640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc0b2870 0x7f29bc0af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29bb7fe640 1 -- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29bc161f20 msgr2=0x7f29bc14e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29bb7fe640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29bc161f20 0x7f29bc14e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29bb7fe640 1 -- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc0b2870 msgr2=0x7f29bc0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29bb7fe640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc0b2870 0x7f29bc0af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.692+0000 7f29bb7fe640 1 -- 172.21.15.31:0/2300016497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b00094d0 con 0x7f29bc0b07a0 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29bb7fe640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc0b07a0 0x7f29bc0af330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00e8b0 tx=0x7f29ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29a3fff640 1 -- 172.21.15.31:0/2300016497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00c940 con 0x7f29bc0b07a0 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29a3fff640 1 -- 172.21.15.31:0/2300016497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac00caa0 con 0x7f29bc0b07a0 2023-11-26T07:18:42.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29a3fff640 1 -- 172.21.15.31:0/2300016497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00ce30 con 0x7f29bc0b07a0 2023-11-26T07:18:42.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29c2017640 1 -- 172.21.15.31:0/2300016497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc157070 con 0x7f29bc0b07a0 2023-11-26T07:18:42.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29c2017640 1 -- 172.21.15.31:0/2300016497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc159540 con 0x7f29bc0b07a0 2023-11-26T07:18:42.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29c2017640 1 -- 172.21.15.31:0/2300016497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc04e850 con 0x7f29bc0b07a0 2023-11-26T07:18:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29a3fff640 1 -- 172.21.15.31:0/2300016497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f29ac013470 con 0x7f29bc0b07a0 2023-11-26T07:18:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29a3fff640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29a8066470 0x7f29a8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29a3fff640 1 -- 172.21.15.31:0/2300016497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29ac083ff0 con 0x7f29bc0b07a0 2023-11-26T07:18:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.696+0000 7f29baffd640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29a8066470 0x7f29a8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.700+0000 7f29baffd640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29a8066470 0x7f29a8068930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f29b00ae000 tx=0x7f29b00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:42.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.700+0000 7f29a3fff640 1 -- 172.21.15.31:0/2300016497 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f29ac0b5280 con 0x7f29bc0b07a0 2023-11-26T07:18:42.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.860+0000 7f29c2017640 1 -- 172.21.15.31:0/2300016497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29bc15eb90 con 0x7f29bc0b07a0 2023-11-26T07:18:42.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.860+0000 7f29a3fff640 1 -- 172.21.15.31:0/2300016497 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29ac021020 con 0x7f29bc0b07a0 2023-11-26T07:18:42.866 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:42.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 -- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29a8066470 msgr2=0x7f29a8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29a8066470 0x7f29a8068930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f29b00ae000 tx=0x7f29b00a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:42.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 -- 172.21.15.31:0/2300016497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc0b07a0 msgr2=0x7f29bc0af330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:42.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc0b07a0 0x7f29bc0af330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00e8b0 tx=0x7f29ac00ed70 comp rx=0 tx=0).stop 2023-11-26T07:18:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 -- 172.21.15.31:0/2300016497 shutdown_connections 2023-11-26T07:18:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f29a8066470 0x7f29a8068930 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f29bc161f20 0x7f29bc14e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29bc0b2870 0x7f29bc0af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 --2- 172.21.15.31:0/2300016497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29bc0b07a0 0x7f29bc0af330 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 -- 172.21.15.31:0/2300016497 >> 172.21.15.31:0/2300016497 conn(0x7f29bc0b1d50 msgr2=0x7f29bc167c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 -- 172.21.15.31:0/2300016497 shutdown_connections 2023-11-26T07:18:42.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:42.864+0000 7f29c2017640 1 -- 172.21.15.31:0/2300016497 wait complete. 2023-11-26T07:18:42.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:43.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f957912a640 1 Processor -- start 2023-11-26T07:18:43.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f957912a640 1 -- start start 2023-11-26T07:18:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f957912a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9574144d50 0x7f95740b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f957912a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f957415f5f0 0x7f95740d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f957912a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957415fcb0 con 0x7f95740b5160 2023-11-26T07:18:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f957912a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95740b5790 con 0x7f9574144d50 2023-11-26T07:18:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f957912a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95740b5a70 con 0x7f957415f5f0 2023-11-26T07:18:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f9572d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9574144d50 0x7f95740b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f956bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f95740b5160 0x7f957415eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57162/0 (socket says 172.21.15.31:57162) 2023-11-26T07:18:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f956bfff640 1 -- 172.21.15.31:0/1740705329 learned_addr learned my addr 172.21.15.31:0/1740705329 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f957359e640 1 --2- 172.21.15.31:0/1740705329 >> v2:172.21.15.97:3300/0 conn(0x7f957415f5f0 0x7f95740d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f9572d9d640 1 -- 172.21.15.31:0/1740705329 >> v2:172.21.15.97:3300/0 conn(0x7f957415f5f0 msgr2=0x7f95740d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f9572d9d640 1 --2- 172.21.15.31:0/1740705329 >> v2:172.21.15.97:3300/0 conn(0x7f957415f5f0 0x7f95740d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f9572d9d640 1 -- 172.21.15.31:0/1740705329 >> v1:172.21.15.31:6789/0 conn(0x7f95740b5160 legacy=0x7f957415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f9572d9d640 1 -- 172.21.15.31:0/1740705329 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95740d0150 con 0x7f9574144d50 2023-11-26T07:18:43.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f957259c640 1 -- 172.21.15.31:0/1740705329 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727617494 0 0) 0x7f957415fcb0 con 0x7f95740b5160 2023-11-26T07:18:43.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.028+0000 7f957359e640 1 --2- 172.21.15.31:0/1740705329 >> v2:172.21.15.97:3300/0 conn(0x7f957415f5f0 0x7f95740d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:43.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f9572d9d640 1 --2- 172.21.15.31:0/1740705329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 0x7f95740b4c20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f95740b3e50 tx=0x7f956401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effc34256d9a0ba4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957259c640 1 -- 172.21.15.31:0/1740705329 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640ac640 con 0x7f9574144d50 2023-11-26T07:18:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957259c640 1 -- 172.21.15.31:0/1740705329 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95640ac7a0 con 0x7f9574144d50 2023-11-26T07:18:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957259c640 1 -- 172.21.15.31:0/1740705329 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640acb30 con 0x7f9574144d50 2023-11-26T07:18:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957912a640 1 -- 172.21.15.31:0/1740705329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 msgr2=0x7f95740b4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957912a640 1 --2- 172.21.15.31:0/1740705329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 0x7f95740b4c20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f95740b3e50 tx=0x7f956401c460 comp rx=0 tx=0).stop 2023-11-26T07:18:43.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957912a640 1 -- 172.21.15.31:0/1740705329 shutdown_connections 2023-11-26T07:18:43.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957912a640 1 --2- 172.21.15.31:0/1740705329 >> v2:172.21.15.97:3300/0 conn(0x7f957415f5f0 0x7f95740d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957912a640 1 --2- 172.21.15.31:0/1740705329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 0x7f95740b4c20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957912a640 1 -- 172.21.15.31:0/1740705329 >> 172.21.15.31:0/1740705329 conn(0x7f95740b2f10 msgr2=0x7f95740b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:43.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957912a640 1 -- 172.21.15.31:0/1740705329 shutdown_connections 2023-11-26T07:18:43.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.032+0000 7f957912a640 1 -- 172.21.15.31:0/1740705329 wait complete. 2023-11-26T07:18:43.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f957912a640 1 Processor -- start 2023-11-26T07:18:43.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f957912a640 1 -- start start 2023-11-26T07:18:43.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f957912a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95740b5160 0x7f9574142a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f957912a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 0x7f95741411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f957912a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f957415f5f0 0x7f957415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f957912a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95741425a0 con 0x7f9574144d50 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f957912a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95741416f0 con 0x7f95740b5160 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f957912a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9574141990 con 0x7f957415f5f0 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f956bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 0x7f95741411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f956bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 0x7f95741411b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55898/0 (socket says 172.21.15.31:55898) 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f956bfff640 1 -- 172.21.15.31:0/879615948 learned_addr learned my addr 172.21.15.31:0/879615948 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f9572d9d640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95740b5160 0x7f9574142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f957359e640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f957415f5f0 0x7f957415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f956bfff640 1 -- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f957415f5f0 msgr2=0x7f957415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f956bfff640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f957415f5f0 0x7f957415d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f956bfff640 1 -- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95740b5160 msgr2=0x7f9574142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f956bfff640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95740b5160 0x7f9574142a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f956bfff640 1 -- 172.21.15.31:0/879615948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9564009420 con 0x7f9574144d50 2023-11-26T07:18:43.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.036+0000 7f956bfff640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 0x7f95741411b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f955c0011f0 tx=0x7f955c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.040+0000 7f956b7fe640 1 -- 172.21.15.31:0/879615948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c0074e0 con 0x7f9574144d50 2023-11-26T07:18:43.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.040+0000 7f956b7fe640 1 -- 172.21.15.31:0/879615948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c007640 con 0x7f9574144d50 2023-11-26T07:18:43.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.040+0000 7f956b7fe640 1 -- 172.21.15.31:0/879615948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c004a10 con 0x7f9574144d50 2023-11-26T07:18:43.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.040+0000 7f957912a640 1 -- 172.21.15.31:0/879615948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9574141c70 con 0x7f9574144d50 2023-11-26T07:18:43.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.040+0000 7f957912a640 1 -- 172.21.15.31:0/879615948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9574142230 con 0x7f9574144d50 2023-11-26T07:18:43.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.040+0000 7f957912a640 1 -- 172.21.15.31:0/879615948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9574142710 con 0x7f9574144d50 2023-11-26T07:18:43.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.040+0000 7f956b7fe640 1 -- 172.21.15.31:0/879615948 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 85852+0+0 (secure 0 0 0) 0x7f955c004b70 con 0x7f9574144d50 2023-11-26T07:18:43.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.040+0000 7f956b7fe640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f95540664a0 0x7f9554068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.040+0000 7f956b7fe640 1 -- 172.21.15.31:0/879615948 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f955c082f90 con 0x7f9574144d50 2023-11-26T07:18:43.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.040+0000 7f9572d9d640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f95540664a0 0x7f9554068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.044+0000 7f9572d9d640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f95540664a0 0x7f9554068960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95640ae000 tx=0x7f95640a6070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.048+0000 7f956b7fe640 1 -- 172.21.15.31:0/879615948 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f955c050870 con 0x7f9574144d50 2023-11-26T07:18:43.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.052+0000 7f956b7fe640 1 -- 172.21.15.31:0/879615948 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f955c0b4280 con 0x7f9574144d50 2023-11-26T07:18:43.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.212+0000 7f957912a640 1 -- 172.21.15.31:0/879615948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f957404e850 con 0x7f9574144d50 2023-11-26T07:18:43.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.212+0000 7f956b7fe640 1 -- 172.21.15.31:0/879615948 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f955c06ddc0 con 0x7f9574144d50 2023-11-26T07:18:43.219 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 -- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f95540664a0 msgr2=0x7f9554068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f95540664a0 0x7f9554068960 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95640ae000 tx=0x7f95640a6070 comp rx=0 tx=0).stop 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 -- 172.21.15.31:0/879615948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 msgr2=0x7f95741411b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 0x7f95741411b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f955c0011f0 tx=0x7f955c00c9b0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 -- 172.21.15.31:0/879615948 shutdown_connections 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f95540664a0 0x7f9554068960 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f957415f5f0 0x7f957415d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9574144d50 0x7f95741411b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 --2- 172.21.15.31:0/879615948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95740b5160 0x7f9574142a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 -- 172.21.15.31:0/879615948 >> 172.21.15.31:0/879615948 conn(0x7f95740b2f10 msgr2=0x7f95740b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:43.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 -- 172.21.15.31:0/879615948 shutdown_connections 2023-11-26T07:18:43.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.216+0000 7f957912a640 1 -- 172.21.15.31:0/879615948 wait complete. 2023-11-26T07:18:43.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:43.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.380+0000 7f6856232640 1 Processor -- start 2023-11-26T07:18:43.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.380+0000 7f6856232640 1 -- start start 2023-11-26T07:18:43.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f6856232640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68500b5830 0x7f685015e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f6856232640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f68500ad8e0 0x7f685006d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f6856232640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500af630 con 0x7f68500acd70 2023-11-26T07:18:43.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f6856232640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500acb10 con 0x7f68500b5830 2023-11-26T07:18:43.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f6856232640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500ae5b0 con 0x7f68500ad8e0 2023-11-26T07:18:43.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f684f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68500b5830 0x7f685015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f684f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f68500b5830 0x7f685015e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55902/0 (socket says 172.21.15.31:55902) 2023-11-26T07:18:43.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f684f7fe640 1 -- 172.21.15.31:0/2581562162 learned_addr learned my addr 172.21.15.31:0/2581562162 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:43.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f684ffff640 1 --2- 172.21.15.31:0/2581562162 >> v2:172.21.15.97:3300/0 conn(0x7f68500ad8e0 0x7f685006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f684f7fe640 1 -- 172.21.15.31:0/2581562162 >> v2:172.21.15.97:3300/0 conn(0x7f68500ad8e0 msgr2=0x7f685006d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:43.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f684f7fe640 1 --2- 172.21.15.31:0/2581562162 >> v2:172.21.15.97:3300/0 conn(0x7f68500ad8e0 0x7f685006d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f684f7fe640 1 -- 172.21.15.31:0/2581562162 >> v1:172.21.15.97:6789/0 conn(0x7f68500acd70 legacy=0x7f68500ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:43.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f684f7fe640 1 -- 172.21.15.31:0/2581562162 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500ae870 con 0x7f68500b5830 2023-11-26T07:18:43.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.384+0000 7f684f7fe640 1 --2- 172.21.15.31:0/2581562162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 0x7f685015e690 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f685006bfa0 tx=0x7f684001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9315af1c9d1f1c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f684e7fc640 1 -- 172.21.15.31:0/2581562162 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400aa620 con 0x7f68500b5830 2023-11-26T07:18:43.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f684e7fc640 1 -- 172.21.15.31:0/2581562162 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68400aa780 con 0x7f68500b5830 2023-11-26T07:18:43.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f684e7fc640 1 -- 172.21.15.31:0/2581562162 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400aab10 con 0x7f68500b5830 2023-11-26T07:18:43.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 -- 172.21.15.31:0/2581562162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 msgr2=0x7f685015e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 --2- 172.21.15.31:0/2581562162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 0x7f685015e690 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f685006bfa0 tx=0x7f684001c4c0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 -- 172.21.15.31:0/2581562162 shutdown_connections 2023-11-26T07:18:43.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 --2- 172.21.15.31:0/2581562162 >> v2:172.21.15.97:3300/0 conn(0x7f68500ad8e0 0x7f685006d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 --2- 172.21.15.31:0/2581562162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 0x7f685015e690 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 -- 172.21.15.31:0/2581562162 >> 172.21.15.31:0/2581562162 conn(0x7f6850071d80 msgr2=0x7f685006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:43.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 -- 172.21.15.31:0/2581562162 shutdown_connections 2023-11-26T07:18:43.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 -- 172.21.15.31:0/2581562162 wait complete. 2023-11-26T07:18:43.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 Processor -- start 2023-11-26T07:18:43.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 -- start start 2023-11-26T07:18:43.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68500acd70 0x7f6850154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f68500ad8e0 0x7f685015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 0x7f685013b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685013b210 con 0x7f68500b5830 2023-11-26T07:18:43.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6850157b50 con 0x7f68500acd70 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.388+0000 7f6856232640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6850157590 con 0x7f68500ad8e0 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 0x7f685013b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 0x7f685013b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55912/0 (socket says 172.21.15.31:55912) 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684ffff640 1 -- 172.21.15.31:0/4070081328 learned_addr learned my addr 172.21.15.31:0/4070081328 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684f7fe640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68500acd70 0x7f6850154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684effd640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f68500ad8e0 0x7f685015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684ffff640 1 -- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f68500ad8e0 msgr2=0x7f685015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684ffff640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f68500ad8e0 0x7f685015d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684ffff640 1 -- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68500acd70 msgr2=0x7f6850154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684ffff640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68500acd70 0x7f6850154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684ffff640 1 -- 172.21.15.31:0/4070081328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400094d0 con 0x7f68500b5830 2023-11-26T07:18:43.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f684ffff640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 0x7f685013b6a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6848007e40 tx=0x7f684800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f6837fff640 1 -- 172.21.15.31:0/4070081328 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848014620 con 0x7f68500b5830 2023-11-26T07:18:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f6837fff640 1 -- 172.21.15.31:0/4070081328 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684800ee90 con 0x7f68500b5830 2023-11-26T07:18:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f6837fff640 1 -- 172.21.15.31:0/4070081328 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68480149e0 con 0x7f68500b5830 2023-11-26T07:18:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f6856232640 1 -- 172.21.15.31:0/4070081328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6850157870 con 0x7f68500b5830 2023-11-26T07:18:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f6856232640 1 -- 172.21.15.31:0/4070081328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685014a550 con 0x7f68500b5830 2023-11-26T07:18:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f6856232640 1 -- 172.21.15.31:0/4070081328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685013acd0 con 0x7f68500b5830 2023-11-26T07:18:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.392+0000 7f6837fff640 1 -- 172.21.15.31:0/4070081328 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f6848014b40 con 0x7f68500b5830 2023-11-26T07:18:43.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.396+0000 7f6837fff640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f683c066620 0x7f683c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.396+0000 7f6837fff640 1 -- 172.21.15.31:0/4070081328 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6848084a20 con 0x7f68500b5830 2023-11-26T07:18:43.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.396+0000 7f684f7fe640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f683c066620 0x7f683c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.396+0000 7f684f7fe640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f683c066620 0x7f683c068ae0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f68400ae000 tx=0x7f68400b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.400+0000 7f6837fff640 1 -- 172.21.15.31:0/4070081328 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6848051cc0 con 0x7f68500b5830 2023-11-26T07:18:43.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.556+0000 7f6856232640 1 -- 172.21.15.31:0/4070081328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f685004e850 con 0x7f68500b5830 2023-11-26T07:18:43.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.556+0000 7f6837fff640 1 -- 172.21.15.31:0/4070081328 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68480552e0 con 0x7f68500b5830 2023-11-26T07:18:43.563 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:18:43.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 -- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f683c066620 msgr2=0x7f683c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f683c066620 0x7f683c068ae0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f68400ae000 tx=0x7f68400b5040 comp rx=0 tx=0).stop 2023-11-26T07:18:43.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 -- 172.21.15.31:0/4070081328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 msgr2=0x7f685013b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 0x7f685013b6a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6848007e40 tx=0x7f684800e4b0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 -- 172.21.15.31:0/4070081328 shutdown_connections 2023-11-26T07:18:43.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f683c066620 0x7f683c068ae0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68500b5830 0x7f685013b6a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f68500ad8e0 0x7f685015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 --2- 172.21.15.31:0/4070081328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68500acd70 0x7f6850154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 -- 172.21.15.31:0/4070081328 >> 172.21.15.31:0/4070081328 conn(0x7f6850071d80 msgr2=0x7f68501606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:43.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 -- 172.21.15.31:0/4070081328 shutdown_connections 2023-11-26T07:18:43.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.560+0000 7f6856232640 1 -- 172.21.15.31:0/4070081328 wait complete. 2023-11-26T07:18:43.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f1891217640 1 Processor -- start 2023-11-26T07:18:43.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f1891217640 1 -- start start 2023-11-26T07:18:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f1891217640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188c0ce5d0 0x7f188c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f1891217640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c159630 con 0x7f188c06e3f0 2023-11-26T07:18:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f1891217640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c0cd1d0 con 0x7f188c0cb9b0 2023-11-26T07:18:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f1891217640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c06e860 con 0x7f188c0ce5d0 2023-11-26T07:18:43.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f1883fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f188c06e3f0 0x7f188c15f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47514/0 (socket says 172.21.15.31:47514) 2023-11-26T07:18:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f1883fff640 1 -- 172.21.15.31:0/914221960 learned_addr learned my addr 172.21.15.31:0/914221960 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f188b59e640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188c0ce5d0 0x7f188c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730403035 0 0) 0x7f188c159630 con 0x7f188c06e3f0 2023-11-26T07:18:43.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.724+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1878003530 con 0x7f188c06e3f0 2023-11-26T07:18:43.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3631598777 0 0) 0x7f1878003530 con 0x7f188c06e3f0 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188c0ce5d0 msgr2=0x7f188c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188c0ce5d0 0x7f188c15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 >> v1:172.21.15.97:6789/0 conn(0x7f188c0cb9b0 legacy=0x7f188c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0cd4b0 con 0x7f188c06e3f0 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f1874001680 con 0x7f188c06e3f0 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 >> v1:172.21.15.31:6789/0 conn(0x7f188c06e3f0 legacy=0x7f188c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18780039b0 0x7f1878003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18780047d0 0x7f1878004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188c0ce5d0 0x7f1878005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c06e860 con 0x7f18780047d0 2023-11-26T07:18:43.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187800ba40 con 0x7f18780039b0 2023-11-26T07:18:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187800bbd0 con 0x7f188c0ce5d0 2023-11-26T07:18:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f1883fff640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18780047d0 0x7f1878004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188ad9d640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18780039b0 0x7f1878003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f188b59e640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188c0ce5d0 0x7f1878005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f1883fff640 1 -- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188c0ce5d0 msgr2=0x7f1878005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f1883fff640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188c0ce5d0 0x7f1878005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f1883fff640 1 -- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18780039b0 msgr2=0x7f1878003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f1883fff640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18780039b0 0x7f1878003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f1883fff640 1 -- 172.21.15.31:0/914221960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1878003530 con 0x7f18780047d0 2023-11-26T07:18:43.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.728+0000 7f1883fff640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18780047d0 0x7f1878004c50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f187400c160 tx=0x7f187401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727cf1d307de9dac server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874005ae0 con 0x7f18780047d0 2023-11-26T07:18:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1874005c40 con 0x7f18780047d0 2023-11-26T07:18:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f188a59c640 1 -- 172.21.15.31:0/914221960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18740a5810 con 0x7f18780047d0 2023-11-26T07:18:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 -- 172.21.15.31:0/914221960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18780047d0 msgr2=0x7f1878004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 --2- 172.21.15.31:0/914221960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18780047d0 0x7f1878004c50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f187400c160 tx=0x7f187401cba0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f188b59e640 1 -- 172.21.15.31:0/914221960 reap_dead start 2023-11-26T07:18:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 -- 172.21.15.31:0/914221960 shutdown_connections 2023-11-26T07:18:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 -- 172.21.15.31:0/914221960 >> 172.21.15.31:0/914221960 conn(0x7f188c0b0cb0 msgr2=0x7f188c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 -- 172.21.15.31:0/914221960 shutdown_connections 2023-11-26T07:18:43.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 -- 172.21.15.31:0/914221960 wait complete. 2023-11-26T07:18:43.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 Processor -- start 2023-11-26T07:18:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 -- start start 2023-11-26T07:18:43.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f18780047d0 0x7f188c1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f188c15ac60 0x7f188c099d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188c09a250 0x7f188c0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c8710 con 0x7f188c15ac60 2023-11-26T07:18:43.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c89f0 con 0x7f188c09a250 2023-11-26T07:18:43.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.732+0000 7f1891217640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c8cd0 con 0x7f18780047d0 2023-11-26T07:18:43.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1883fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f188c15ac60 0x7f188c099d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1883fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f188c15ac60 0x7f188c099d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55940/0 (socket says 172.21.15.31:55940) 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1883fff640 1 -- 172.21.15.31:0/1762451848 learned_addr learned my addr 172.21.15.31:0/1762451848 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f188ad9d640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f18780047d0 0x7f188c1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f188b59e640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188c09a250 0x7f188c0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1883fff640 1 -- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f18780047d0 msgr2=0x7f188c1556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1883fff640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f18780047d0 0x7f188c1556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1883fff640 1 -- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188c09a250 msgr2=0x7f188c0ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1883fff640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188c09a250 0x7f188c0ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1883fff640 1 -- 172.21.15.31:0/1762451848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0c8fb0 con 0x7f188c15ac60 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f188ad9d640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f18780047d0 0x7f188c1556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f188b59e640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188c09a250 0x7f188c0ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:43.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1883fff640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f188c15ac60 0x7f188c099d10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f187400c160 tx=0x7f1878005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f18837fe640 1 -- 172.21.15.31:0/1762451848 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18740b4070 con 0x7f188c15ac60 2023-11-26T07:18:43.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f18837fe640 1 -- 172.21.15.31:0/1762451848 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18740a5810 con 0x7f188c15ac60 2023-11-26T07:18:43.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1891217640 1 -- 172.21.15.31:0/1762451848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c0c9290 con 0x7f188c15ac60 2023-11-26T07:18:43.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f18837fe640 1 -- 172.21.15.31:0/1762451848 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18740a5ba0 con 0x7f188c15ac60 2023-11-26T07:18:43.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1891217640 1 -- 172.21.15.31:0/1762451848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c0c8330 con 0x7f188c15ac60 2023-11-26T07:18:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f1891217640 1 -- 172.21.15.31:0/1762451848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c0671f0 con 0x7f188c15ac60 2023-11-26T07:18:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f18837fe640 1 -- 172.21.15.31:0/1762451848 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f18740a5d00 con 0x7f188c15ac60 2023-11-26T07:18:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.736+0000 7f18837fe640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f186c066620 0x7f186c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.740+0000 7f18837fe640 1 -- 172.21.15.31:0/1762451848 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f187411df00 con 0x7f188c15ac60 2023-11-26T07:18:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.740+0000 7f188ad9d640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f186c066620 0x7f186c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:43.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.740+0000 7f188ad9d640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f186c066620 0x7f186c068ae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f187c007fd0 tx=0x7f187c005130 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:43.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.740+0000 7f18837fe640 1 -- 172.21.15.31:0/1762451848 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f18740eb060 con 0x7f188c15ac60 2023-11-26T07:18:43.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 -- 172.21.15.31:0/1762451848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f188c06b460 con 0x7f188c15ac60 2023-11-26T07:18:43.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f18837fe640 1 -- 172.21.15.31:0/1762451848 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f18740ee680 con 0x7f188c15ac60 2023-11-26T07:18:43.917 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:18:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 -- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f186c066620 msgr2=0x7f186c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f186c066620 0x7f186c068ae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f187c007fd0 tx=0x7f187c005130 comp rx=0 tx=0).stop 2023-11-26T07:18:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 -- 172.21.15.31:0/1762451848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f188c15ac60 msgr2=0x7f188c099d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f188c15ac60 0x7f188c099d10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f187400c160 tx=0x7f1878005800 comp rx=0 tx=0).stop 2023-11-26T07:18:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 -- 172.21.15.31:0/1762451848 shutdown_connections 2023-11-26T07:18:43.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f186c066620 0x7f186c068ae0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f188c09a250 0x7f188c0ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f188c15ac60 0x7f188c099d10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 --2- 172.21.15.31:0/1762451848 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f18780047d0 0x7f188c1556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:43.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 -- 172.21.15.31:0/1762451848 >> 172.21.15.31:0/1762451848 conn(0x7f188c0b0cb0 msgr2=0x7f188c0af040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:43.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 -- 172.21.15.31:0/1762451848 shutdown_connections 2023-11-26T07:18:43.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:43.912+0000 7f1891217640 1 -- 172.21.15.31:0/1762451848 wait complete. 2023-11-26T07:18:43.938 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:18:43.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:43.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:18:43.997 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:18:43.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:43.997 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:18:44.015 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:44.015 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:44.029 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:44.029 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:44.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:44.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0f72a640 1 Processor -- start 2023-11-26T07:18:44.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0f72a640 1 -- start start 2023-11-26T07:18:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0f72a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbb0806dc90 0x7fbb080b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0f72a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 0x7fbb080ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0f72a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbb080b2040 0x7fbb08165570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0f72a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb08161ef0 con 0x7fbb0806dc90 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0f72a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb080b25b0 con 0x7fbb080b2040 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0f72a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb080b2890 con 0x7fbb0806e070 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0d4c6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbb0806dc90 0x7fbb080b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0d4c6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbb0806dc90 0x7fbb080b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55946/0 (socket says 172.21.15.31:55946) 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0d4c6640 1 -- 172.21.15.31:0/3882761325 learned_addr learned my addr 172.21.15.31:0/3882761325 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0ccc5640 1 --2- 172.21.15.31:0/3882761325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 0x7fbb080ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0dcc7640 1 --2- 172.21.15.31:0/3882761325 >> v2:172.21.15.97:3300/0 conn(0x7fbb080b2040 0x7fbb08165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0d4c6640 1 -- 172.21.15.31:0/3882761325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 msgr2=0x7fbb080ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0d4c6640 1 --2- 172.21.15.31:0/3882761325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 0x7fbb080ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0d4c6640 1 -- 172.21.15.31:0/3882761325 >> v2:172.21.15.97:3300/0 conn(0x7fbb080b2040 msgr2=0x7fbb08165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:44.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0d4c6640 1 --2- 172.21.15.31:0/3882761325 >> v2:172.21.15.97:3300/0 conn(0x7fbb080b2040 0x7fbb08165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0d4c6640 1 -- 172.21.15.31:0/3882761325 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb080b2b70 con 0x7fbb0806dc90 2023-11-26T07:18:44.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0dcc7640 1 --2- 172.21.15.31:0/3882761325 >> v2:172.21.15.97:3300/0 conn(0x7fbb080b2040 0x7fbb08165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:44.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.219+0000 7fbb0ccc5640 1 --2- 172.21.15.31:0/3882761325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 0x7fbb080ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbb0d4c6640 1 --2- 172.21.15.31:0/3882761325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb0806dc90 0x7fbb080b3650 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbb08072a40 tx=0x7fbb0401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e7845bd5a8675a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbafffff640 1 -- 172.21.15.31:0/3882761325 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040a89b0 con 0x7fbb0806dc90 2023-11-26T07:18:44.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbafffff640 1 -- 172.21.15.31:0/3882761325 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb040a8b10 con 0x7fbb0806dc90 2023-11-26T07:18:44.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbafffff640 1 -- 172.21.15.31:0/3882761325 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040a8ea0 con 0x7fbb0806dc90 2023-11-26T07:18:44.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbb0f72a640 1 -- 172.21.15.31:0/3882761325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb0806dc90 msgr2=0x7fbb080b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbb0f72a640 1 --2- 172.21.15.31:0/3882761325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb0806dc90 0x7fbb080b3650 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbb08072a40 tx=0x7fbb0401ecf0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbb0f72a640 1 -- 172.21.15.31:0/3882761325 shutdown_connections 2023-11-26T07:18:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbb0f72a640 1 --2- 172.21.15.31:0/3882761325 >> v2:172.21.15.97:3300/0 conn(0x7fbb080b2040 0x7fbb08165570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbb0f72a640 1 --2- 172.21.15.31:0/3882761325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 0x7fbb080ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbb0f72a640 1 --2- 172.21.15.31:0/3882761325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb0806dc90 0x7fbb080b3650 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbb0f72a640 1 -- 172.21.15.31:0/3882761325 >> 172.21.15.31:0/3882761325 conn(0x7fbb0806a5b0 msgr2=0x7fbb0806a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbb0f72a640 1 -- 172.21.15.31:0/3882761325 shutdown_connections 2023-11-26T07:18:44.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.223+0000 7fbb0f72a640 1 -- 172.21.15.31:0/3882761325 wait complete. 2023-11-26T07:18:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0f72a640 1 Processor -- start 2023-11-26T07:18:44.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0f72a640 1 -- start start 2023-11-26T07:18:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0f72a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0806dc90 0x7fbb0809a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0f72a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 0x7fbb081f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0f72a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb080b2040 0x7fbb080943d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0f72a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb08093b20 con 0x7fbb080b2040 2023-11-26T07:18:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0f72a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081f84a0 con 0x7fbb0806dc90 2023-11-26T07:18:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0f72a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb08099660 con 0x7fbb0806e070 2023-11-26T07:18:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0dcc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb080b2040 0x7fbb080943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0dcc7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb080b2040 0x7fbb080943d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55950/0 (socket says 172.21.15.31:55950) 2023-11-26T07:18:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0dcc7640 1 -- 172.21.15.31:0/2014761750 learned_addr learned my addr 172.21.15.31:0/2014761750 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0d4c6640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0806dc90 0x7fbb0809a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0ccc5640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 0x7fbb081f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0d4c6640 1 -- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 msgr2=0x7fbb081f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0d4c6640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 0x7fbb081f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0d4c6640 1 -- 172.21.15.31:0/2014761750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb080b2040 msgr2=0x7fbb080943d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0d4c6640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb080b2040 0x7fbb080943d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0d4c6640 1 -- 172.21.15.31:0/2014761750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb040094d0 con 0x7fbb0806dc90 2023-11-26T07:18:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0dcc7640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb080b2040 0x7fbb080943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0d4c6640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0806dc90 0x7fbb0809a6e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbb040af000 tx=0x7fbb0401ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbafdffb640 1 -- 172.21.15.31:0/2014761750 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040ba600 con 0x7fbb0806dc90 2023-11-26T07:18:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbafdffb640 1 -- 172.21.15.31:0/2014761750 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb040ba780 con 0x7fbb0806dc90 2023-11-26T07:18:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0f72a640 1 -- 172.21.15.31:0/2014761750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb08099940 con 0x7fbb0806dc90 2023-11-26T07:18:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbafdffb640 1 -- 172.21.15.31:0/2014761750 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040bab30 con 0x7fbb0806dc90 2023-11-26T07:18:44.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.227+0000 7fbb0f72a640 1 -- 172.21.15.31:0/2014761750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb08099f00 con 0x7fbb0806dc90 2023-11-26T07:18:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.231+0000 7fbb0f72a640 1 -- 172.21.15.31:0/2014761750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0804e850 con 0x7fbb0806dc90 2023-11-26T07:18:44.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.231+0000 7fbafdffb640 1 -- 172.21.15.31:0/2014761750 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fbb040bac90 con 0x7fbb0806dc90 2023-11-26T07:18:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.231+0000 7fbafdffb640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbaec066620 0x7fbaec068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.231+0000 7fbafdffb640 1 -- 172.21.15.31:0/2014761750 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb04122630 con 0x7fbb0806dc90 2023-11-26T07:18:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.231+0000 7fbb0ccc5640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbaec066620 0x7fbaec068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.231+0000 7fbb0ccc5640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbaec066620 0x7fbaec068ae0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb080ad660 tx=0x7fbaf8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.235+0000 7fbafdffb640 1 -- 172.21.15.31:0/2014761750 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbb040ef840 con 0x7fbb0806dc90 2023-11-26T07:18:44.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.395+0000 7fbb0f72a640 1 -- 172.21.15.31:0/2014761750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb080727f0 con 0x7fbb0806dc90 2023-11-26T07:18:44.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbafdffb640 1 -- 172.21.15.31:0/2014761750 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16485 (secure 0 0 0) 0x7fbb040f2e60 con 0x7fbb0806dc90 2023-11-26T07:18:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:44.405 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:26.567047+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:22.773068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:23.967477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 -- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbaec066620 msgr2=0x7fbaec068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbaec066620 0x7fbaec068ae0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbb080ad660 tx=0x7fbaf8008040 comp rx=0 tx=0).stop 2023-11-26T07:18:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 -- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0806dc90 msgr2=0x7fbb0809a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0806dc90 0x7fbb0809a6e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbb040af000 tx=0x7fbb0401ba90 comp rx=0 tx=0).stop 2023-11-26T07:18:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 -- 172.21.15.31:0/2014761750 shutdown_connections 2023-11-26T07:18:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbaec066620 0x7fbaec068ae0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb080b2040 0x7fbb080943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbb0806e070 0x7fbb081f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 --2- 172.21.15.31:0/2014761750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb0806dc90 0x7fbb0809a6e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 -- 172.21.15.31:0/2014761750 >> 172.21.15.31:0/2014761750 conn(0x7fbb0806a5b0 msgr2=0x7fbb080af210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:44.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 -- 172.21.15.31:0/2014761750 shutdown_connections 2023-11-26T07:18:44.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.399+0000 7fbb0f72a640 1 -- 172.21.15.31:0/2014761750 wait complete. 2023-11-26T07:18:44.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:18:22.773068+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:18:23.967477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:44.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8c54d6640 1 Processor -- start 2023-11-26T07:18:44.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8c54d6640 1 -- start start 2023-11-26T07:18:44.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8c54d6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd8c00ce4b0 0x7fd8c00caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8c54d6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd8c006c520 0x7fd8c006c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8c54d6640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c006ce70 con 0x7fd8c015d610 2023-11-26T07:18:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8c54d6640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c00cbfe0 con 0x7fd8c006c520 2023-11-26T07:18:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8c54d6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c006b970 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8beffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd8c015d610 0x7fd8c00cb8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47524/0 (socket says 172.21.15.31:47524) 2023-11-26T07:18:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8beffd640 1 -- 172.21.15.31:0/694257259 learned_addr learned my addr 172.21.15.31:0/694257259 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8bf7fe640 1 --2- 172.21.15.31:0/694257259 >> v2:172.21.15.31:3300/0 conn(0x7fd8c006c520 0x7fd8c006c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8be7fc640 1 --2- 172.21.15.31:0/694257259 >> v2:172.21.15.97:3300/0 conn(0x7fd8c00ce4b0 0x7fd8c00caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8bdffb640 1 -- 172.21.15.31:0/694257259 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12836134 0 0) 0x7fd8c006ce70 con 0x7fd8c015d610 2023-11-26T07:18:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8bdffb640 1 -- 172.21.15.31:0/694257259 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a8003530 con 0x7fd8c015d610 2023-11-26T07:18:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8bf7fe640 1 -- 172.21.15.31:0/694257259 >> v2:172.21.15.97:3300/0 conn(0x7fd8c00ce4b0 msgr2=0x7fd8c00caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8bf7fe640 1 --2- 172.21.15.31:0/694257259 >> v2:172.21.15.97:3300/0 conn(0x7fd8c00ce4b0 0x7fd8c00caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8bf7fe640 1 -- 172.21.15.31:0/694257259 >> v1:172.21.15.31:6789/0 conn(0x7fd8c015d610 legacy=0x7fd8c00cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.567+0000 7fd8bf7fe640 1 -- 172.21.15.31:0/694257259 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c006bc50 con 0x7fd8c006c520 2023-11-26T07:18:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8bf7fe640 1 --2- 172.21.15.31:0/694257259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c006c520 0x7fd8c006c930 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd8c006e210 tx=0x7fd8b801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211e0e65f40ebf1b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8bdffb640 1 -- 172.21.15.31:0/694257259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80a57b0 con 0x7fd8c006c520 2023-11-26T07:18:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8bdffb640 1 -- 172.21.15.31:0/694257259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b80a5910 con 0x7fd8c006c520 2023-11-26T07:18:44.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8bdffb640 1 -- 172.21.15.31:0/694257259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80a5ca0 con 0x7fd8c006c520 2023-11-26T07:18:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8c54d6640 1 -- 172.21.15.31:0/694257259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c006c520 msgr2=0x7fd8c006c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8c54d6640 1 --2- 172.21.15.31:0/694257259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c006c520 0x7fd8c006c930 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd8c006e210 tx=0x7fd8b801c910 comp rx=0 tx=0).stop 2023-11-26T07:18:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8c54d6640 1 -- 172.21.15.31:0/694257259 shutdown_connections 2023-11-26T07:18:44.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8c54d6640 1 --2- 172.21.15.31:0/694257259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c006c520 0x7fd8c006c930 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8c54d6640 1 --2- 172.21.15.31:0/694257259 >> v2:172.21.15.97:3300/0 conn(0x7fd8c00ce4b0 0x7fd8c00caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8c54d6640 1 -- 172.21.15.31:0/694257259 >> 172.21.15.31:0/694257259 conn(0x7fd8c00b2f10 msgr2=0x7fd8c00b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:44.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8c54d6640 1 -- 172.21.15.31:0/694257259 shutdown_connections 2023-11-26T07:18:44.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.571+0000 7fd8c54d6640 1 -- 172.21.15.31:0/694257259 wait complete. 2023-11-26T07:18:44.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 Processor -- start 2023-11-26T07:18:44.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 -- start start 2023-11-26T07:18:44.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c006c520 0x7fd8c014ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c00ce4b0 0x7fd8c0150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd8c015d610 0x7fd8c014a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0154b60 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c014e420 con 0x7fd8c006c520 2023-11-26T07:18:44.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c014e700 con 0x7fd8c015d610 2023-11-26T07:18:44.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8be7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c00ce4b0 0x7fd8c0150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8be7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c00ce4b0 0x7fd8c0150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55964/0 (socket says 172.21.15.31:55964) 2023-11-26T07:18:44.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8be7fc640 1 -- 172.21.15.31:0/680499802 learned_addr learned my addr 172.21.15.31:0/680499802 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8beffd640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c006c520 0x7fd8c014ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8be7fc640 1 -- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd8c015d610 msgr2=0x7fd8c014a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8be7fc640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd8c015d610 0x7fd8c014a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8be7fc640 1 -- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c006c520 msgr2=0x7fd8c014ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8be7fc640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c006c520 0x7fd8c014ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8be7fc640 1 -- 172.21.15.31:0/680499802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b8009a70 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8beffd640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c006c520 0x7fd8c014ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8be7fc640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c00ce4b0 0x7fd8c0150480 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400e930 tx=0x7fd8b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8a77fe640 1 -- 172.21.15.31:0/680499802 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4009690 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8a77fe640 1 -- 172.21.15.31:0/680499802 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b400ca00 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8a77fe640 1 -- 172.21.15.31:0/680499802 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400cd70 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 -- 172.21.15.31:0/680499802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c014e9e0 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 -- 172.21.15.31:0/680499802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c0152660 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.575+0000 7fd8c54d6640 1 -- 172.21.15.31:0/680499802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c014ec00 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.579+0000 7fd8a77fe640 1 -- 172.21.15.31:0/680499802 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd8b4013470 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.579+0000 7fd8a77fe640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8ac066650 0x7fd8ac068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.579+0000 7fd8a77fe640 1 -- 172.21.15.31:0/680499802 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8b40844e0 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.579+0000 7fd8beffd640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8ac066650 0x7fd8ac068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.579+0000 7fd8beffd640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8ac066650 0x7fd8ac068b10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00011f0 tx=0x7fd8b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.583+0000 7fd8a77fe640 1 -- 172.21.15.31:0/680499802 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd8b4051700 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.739+0000 7fd8c54d6640 1 -- 172.21.15.31:0/680499802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8c004e850 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8a77fe640 1 -- 172.21.15.31:0/680499802 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8b4054d20 con 0x7fd8c00ce4b0 2023-11-26T07:18:44.748 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 -- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8ac066650 msgr2=0x7fd8ac068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8ac066650 0x7fd8ac068b10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00011f0 tx=0x7fd8b0009040 comp rx=0 tx=0).stop 2023-11-26T07:18:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 -- 172.21.15.31:0/680499802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c00ce4b0 msgr2=0x7fd8c0150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c00ce4b0 0x7fd8c0150480 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400e930 tx=0x7fd8b400edf0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 -- 172.21.15.31:0/680499802 shutdown_connections 2023-11-26T07:18:44.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd8ac066650 0x7fd8ac068b10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd8c015d610 0x7fd8c014a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8c00ce4b0 0x7fd8c0150480 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 --2- 172.21.15.31:0/680499802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd8c006c520 0x7fd8c014ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 -- 172.21.15.31:0/680499802 >> 172.21.15.31:0/680499802 conn(0x7fd8c00b2f10 msgr2=0x7fd8c015c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:44.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 -- 172.21.15.31:0/680499802 shutdown_connections 2023-11-26T07:18:44.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.743+0000 7fd8c54d6640 1 -- 172.21.15.31:0/680499802 wait complete. 2023-11-26T07:18:44.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:44.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.907+0000 7f67c2980640 1 Processor -- start 2023-11-26T07:18:44.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.907+0000 7f67c2980640 1 -- start start 2023-11-26T07:18:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67c2980640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 0x7f67bc0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67c2980640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc0b2710 con 0x7f67bc15a710 2023-11-26T07:18:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67c2980640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc0b29f0 con 0x7f67bc0cbd50 2023-11-26T07:18:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67c2980640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0cca80 con 0x7f67bc0cdfb0 2023-11-26T07:18:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67bb7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f67bc15a710 0x7f67bc0ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47538/0 (socket says 172.21.15.31:47538) 2023-11-26T07:18:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67bb7fe640 1 -- 172.21.15.31:0/3335741811 learned_addr learned my addr 172.21.15.31:0/3335741811 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:44.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67bbfff640 1 --2- 172.21.15.31:0/3335741811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 0x7f67bc0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67baffd640 1 -- 172.21.15.31:0/3335741811 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111524806 0 0) 0x7f67bc0b2710 con 0x7f67bc15a710 2023-11-26T07:18:44.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67baffd640 1 -- 172.21.15.31:0/3335741811 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67a4005740 con 0x7f67bc15a710 2023-11-26T07:18:44.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67baffd640 1 -- 172.21.15.31:0/3335741811 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108022929 0 0) 0x7f67bc0b29f0 con 0x7f67bc0cbd50 2023-11-26T07:18:44.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67baffd640 1 -- 172.21.15.31:0/3335741811 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67bc0b2710 con 0x7f67bc0cbd50 2023-11-26T07:18:44.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67bbfff640 1 -- 172.21.15.31:0/3335741811 >> v1:172.21.15.97:6789/0 conn(0x7f67bc0cbd50 legacy=0x7f67bc0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67bbfff640 1 -- 172.21.15.31:0/3335741811 >> v1:172.21.15.31:6789/0 conn(0x7f67bc15a710 legacy=0x7f67bc0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.911+0000 7f67bbfff640 1 -- 172.21.15.31:0/3335741811 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc0b2cd0 con 0x7f67bc0cdfb0 2023-11-26T07:18:44.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67bbfff640 1 --2- 172.21.15.31:0/3335741811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 0x7f67bc0abf90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0097f0 tx=0x7f67ac01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a1f1ccf78d2697 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67baffd640 1 -- 172.21.15.31:0/3335741811 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0aa610 con 0x7f67bc0cdfb0 2023-11-26T07:18:44.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67baffd640 1 -- 172.21.15.31:0/3335741811 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac0a6040 con 0x7f67bc0cdfb0 2023-11-26T07:18:44.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67baffd640 1 -- 172.21.15.31:0/3335741811 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0aa990 con 0x7f67bc0cdfb0 2023-11-26T07:18:44.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67c2980640 1 -- 172.21.15.31:0/3335741811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 msgr2=0x7f67bc0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67c2980640 1 --2- 172.21.15.31:0/3335741811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 0x7f67bc0abf90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0097f0 tx=0x7f67ac01c490 comp rx=0 tx=0).stop 2023-11-26T07:18:44.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67c2980640 1 -- 172.21.15.31:0/3335741811 shutdown_connections 2023-11-26T07:18:44.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67c2980640 1 --2- 172.21.15.31:0/3335741811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 0x7f67bc0abf90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67c2980640 1 -- 172.21.15.31:0/3335741811 >> 172.21.15.31:0/3335741811 conn(0x7f67bc0b0cb0 msgr2=0x7f67bc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:44.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67c2980640 1 -- 172.21.15.31:0/3335741811 shutdown_connections 2023-11-26T07:18:44.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67c2980640 1 -- 172.21.15.31:0/3335741811 wait complete. 2023-11-26T07:18:44.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67c2980640 1 Processor -- start 2023-11-26T07:18:44.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.915+0000 7f67c2980640 1 -- start start 2023-11-26T07:18:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c2980640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67bc0cbd50 0x7f67bc153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c2980640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 0x7f67bc152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c2980640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc15a710 0x7f67bc1635a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c2980640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc15c890 con 0x7f67bc15a710 2023-11-26T07:18:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c2980640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc15cb70 con 0x7f67bc0cbd50 2023-11-26T07:18:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c2980640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc158ad0 con 0x7f67bc0cdfb0 2023-11-26T07:18:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c0f1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc15a710 0x7f67bc1635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c0f1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc15a710 0x7f67bc1635a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55972/0 (socket says 172.21.15.31:55972) 2023-11-26T07:18:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c0f1d640 1 -- 172.21.15.31:0/1750749049 learned_addr learned my addr 172.21.15.31:0/1750749049 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67bbfff640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67bc0cbd50 0x7f67bc153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67bb7fe640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 0x7f67bc152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c0f1d640 1 -- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 msgr2=0x7f67bc152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c0f1d640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 0x7f67bc152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c0f1d640 1 -- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67bc0cbd50 msgr2=0x7f67bc153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c0f1d640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67bc0cbd50 0x7f67bc153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c0f1d640 1 -- 172.21.15.31:0/1750749049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac009420 con 0x7f67bc15a710 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67bbfff640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67bc0cbd50 0x7f67bc153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c0f1d640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc15a710 0x7f67bc1635a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f67b400c790 tx=0x7f67b400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67b8ff9640 1 -- 172.21.15.31:0/1750749049 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b400cea0 con 0x7f67bc15a710 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67b8ff9640 1 -- 172.21.15.31:0/1750749049 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b4003b80 con 0x7f67bc15a710 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67b8ff9640 1 -- 172.21.15.31:0/1750749049 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b400a850 con 0x7f67bc15a710 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c2980640 1 -- 172.21.15.31:0/1750749049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc1534e0 con 0x7f67bc15a710 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c2980640 1 -- 172.21.15.31:0/1750749049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc154040 con 0x7f67bc15a710 2023-11-26T07:18:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.919+0000 7f67c2980640 1 -- 172.21.15.31:0/1750749049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc0671f0 con 0x7f67bc15a710 2023-11-26T07:18:44.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.923+0000 7f67b8ff9640 1 -- 172.21.15.31:0/1750749049 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f67b400a9b0 con 0x7f67bc15a710 2023-11-26T07:18:44.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.923+0000 7f67b8ff9640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67a8066620 0x7f67a8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:44.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.923+0000 7f67b8ff9640 1 -- 172.21.15.31:0/1750749049 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67b4083d00 con 0x7f67bc15a710 2023-11-26T07:18:44.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.923+0000 7f67bbfff640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67a8066620 0x7f67a8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:44.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.923+0000 7f67bbfff640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67a8066620 0x7f67a8068ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0ae000 tx=0x7f67ac0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:44.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:44.927+0000 7f67b8ff9640 1 -- 172.21.15.31:0/1750749049 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f67b4051040 con 0x7f67bc15a710 2023-11-26T07:18:45.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.083+0000 7f67c2980640 1 -- 172.21.15.31:0/1750749049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67bc158930 con 0x7f67bc15a710 2023-11-26T07:18:45.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.083+0000 7f67b8ff9640 1 -- 172.21.15.31:0/1750749049 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67bc158930 con 0x7f67bc15a710 2023-11-26T07:18:45.090 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:18:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 -- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67a8066620 msgr2=0x7f67a8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67a8066620 0x7f67a8068ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0ae000 tx=0x7f67ac0a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 -- 172.21.15.31:0/1750749049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc15a710 msgr2=0x7f67bc1635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc15a710 0x7f67bc1635a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f67b400c790 tx=0x7f67b400cc50 comp rx=0 tx=0).stop 2023-11-26T07:18:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 -- 172.21.15.31:0/1750749049 shutdown_connections 2023-11-26T07:18:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67a8066620 0x7f67a8068ae0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67bc15a710 0x7f67bc1635a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67bc0cdfb0 0x7f67bc152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 --2- 172.21.15.31:0/1750749049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67bc0cbd50 0x7f67bc153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 -- 172.21.15.31:0/1750749049 >> 172.21.15.31:0/1750749049 conn(0x7f67bc0b0cb0 msgr2=0x7f67bc0ace00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:45.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 -- 172.21.15.31:0/1750749049 shutdown_connections 2023-11-26T07:18:45.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.087+0000 7f67c2980640 1 -- 172.21.15.31:0/1750749049 wait complete. 2023-11-26T07:18:45.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:45.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2ff5017640 1 Processor -- start 2023-11-26T07:18:45.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2ff5017640 1 -- start start 2023-11-26T07:18:45.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2ff5017640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ff00cdfb0 0x7f2ff00abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2ff5017640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ff00cbd50 0x7f2ff00cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2ff5017640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff00b2710 con 0x7f2ff015a710 2023-11-26T07:18:45.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2ff5017640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff00cca80 con 0x7f2ff00cdfb0 2023-11-26T07:18:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2ff5017640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00b29f0 con 0x7f2ff00cbd50 2023-11-26T07:18:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2fef59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ff00cbd50 0x7f2ff00cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2fe7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2ff015a710 0x7f2ff00ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:52380/0 (socket says 172.21.15.31:52380) 2023-11-26T07:18:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2feed9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2ff00cdfb0 0x7f2ff00abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2fe7fff640 1 -- 172.21.15.31:0/1845062129 learned_addr learned my addr 172.21.15.31:0/1845062129 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:45.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2feed9d640 1 -- 172.21.15.31:0/1845062129 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ff00cbd50 msgr2=0x7f2ff00cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:45.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2feed9d640 1 --2- 172.21.15.31:0/1845062129 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ff00cbd50 0x7f2ff00cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2feed9d640 1 -- 172.21.15.31:0/1845062129 >> v1:172.21.15.97:6789/0 conn(0x7f2ff015a710 legacy=0x7f2ff00ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.251+0000 7f2feed9d640 1 -- 172.21.15.31:0/1845062129 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff00b2cd0 con 0x7f2ff00cdfb0 2023-11-26T07:18:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2feed9d640 1 --2- 172.21.15.31:0/1845062129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff00cdfb0 0x7f2ff00abf90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00097c0 tx=0x7f2fe001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb2636e2a7462a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2fee59c640 1 -- 172.21.15.31:0/1845062129 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00ab670 con 0x7f2ff00cdfb0 2023-11-26T07:18:45.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2fee59c640 1 -- 172.21.15.31:0/1845062129 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe00ab7d0 con 0x7f2ff00cdfb0 2023-11-26T07:18:45.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2fee59c640 1 -- 172.21.15.31:0/1845062129 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00abb60 con 0x7f2ff00cdfb0 2023-11-26T07:18:45.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2ff5017640 1 -- 172.21.15.31:0/1845062129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff00cdfb0 msgr2=0x7f2ff00abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2ff5017640 1 --2- 172.21.15.31:0/1845062129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff00cdfb0 0x7f2ff00abf90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00097c0 tx=0x7f2fe001c450 comp rx=0 tx=0).stop 2023-11-26T07:18:45.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2ff5017640 1 -- 172.21.15.31:0/1845062129 shutdown_connections 2023-11-26T07:18:45.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2ff5017640 1 --2- 172.21.15.31:0/1845062129 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ff00cbd50 0x7f2ff00cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2ff5017640 1 --2- 172.21.15.31:0/1845062129 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff00cdfb0 0x7f2ff00abf90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2ff5017640 1 -- 172.21.15.31:0/1845062129 >> 172.21.15.31:0/1845062129 conn(0x7f2ff00b0cb0 msgr2=0x7f2ff00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:45.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2ff5017640 1 -- 172.21.15.31:0/1845062129 shutdown_connections 2023-11-26T07:18:45.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.255+0000 7f2ff5017640 1 -- 172.21.15.31:0/1845062129 wait complete. 2023-11-26T07:18:45.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2ff5017640 1 Processor -- start 2023-11-26T07:18:45.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2ff5017640 1 -- start start 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2ff5017640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff00cbd50 0x7f2ff0099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2ff5017640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ff00cdfb0 0x7f2ff009b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2ff5017640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff015a710 0x7f2ff0166620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2ff5017640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff015b0e0 con 0x7f2ff015a710 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2ff5017640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff009b7f0 con 0x7f2ff00cbd50 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2ff5017640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff01598e0 con 0x7f2ff00cdfb0 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fef59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff015a710 0x7f2ff0166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fef59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff015a710 0x7f2ff0166620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55976/0 (socket says 172.21.15.31:55976) 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fef59e640 1 -- 172.21.15.31:0/3136841793 learned_addr learned my addr 172.21.15.31:0/3136841793 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2feed9d640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff00cbd50 0x7f2ff0099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fe7fff640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ff00cdfb0 0x7f2ff009b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fef59e640 1 -- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ff00cdfb0 msgr2=0x7f2ff009b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fef59e640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ff00cdfb0 0x7f2ff009b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fef59e640 1 -- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff00cbd50 msgr2=0x7f2ff0099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fef59e640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff00cbd50 0x7f2ff0099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fef59e640 1 -- 172.21.15.31:0/3136841793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe0009420 con 0x7f2ff015a710 2023-11-26T07:18:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fef59e640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff015a710 0x7f2ff0166620 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800efc0 tx=0x7f2fe800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.259+0000 7f2fe77fe640 1 -- 172.21.15.31:0/3136841793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8014480 con 0x7f2ff015a710 2023-11-26T07:18:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.263+0000 7f2fe77fe640 1 -- 172.21.15.31:0/3136841793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe800f040 con 0x7f2ff015a710 2023-11-26T07:18:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.263+0000 7f2ff5017640 1 -- 172.21.15.31:0/3136841793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff015ab40 con 0x7f2ff015a710 2023-11-26T07:18:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.263+0000 7f2fe77fe640 1 -- 172.21.15.31:0/3136841793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8014820 con 0x7f2ff015a710 2023-11-26T07:18:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.263+0000 7f2ff5017640 1 -- 172.21.15.31:0/3136841793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff015d6c0 con 0x7f2ff015a710 2023-11-26T07:18:45.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.263+0000 7f2ff5017640 1 -- 172.21.15.31:0/3136841793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff00671f0 con 0x7f2ff015a710 2023-11-26T07:18:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.263+0000 7f2fe77fe640 1 -- 172.21.15.31:0/3136841793 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2fe8014980 con 0x7f2ff015a710 2023-11-26T07:18:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.263+0000 7f2fe77fe640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fd0066620 0x7f2fd0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.263+0000 7f2fe77fe640 1 -- 172.21.15.31:0/3136841793 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fe8084d50 con 0x7f2ff015a710 2023-11-26T07:18:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.263+0000 7f2feed9d640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fd0066620 0x7f2fd0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.263+0000 7f2feed9d640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fd0066620 0x7f2fd0068ae0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00af000 tx=0x7f2fe00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:45.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.267+0000 7f2fe77fe640 1 -- 172.21.15.31:0/3136841793 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2fe8051e50 con 0x7f2ff015a710 2023-11-26T07:18:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.427+0000 7f2ff5017640 1 -- 172.21.15.31:0/3136841793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ff015d4f0 con 0x7f2ff015a710 2023-11-26T07:18:45.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.427+0000 7f2fe77fe640 1 -- 172.21.15.31:0/3136841793 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2fe80180a0 con 0x7f2ff015a710 2023-11-26T07:18:45.434 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:18:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.427+0000 7f2ff5017640 1 -- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fd0066620 msgr2=0x7f2fd0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.427+0000 7f2ff5017640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fd0066620 0x7f2fd0068ae0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00af000 tx=0x7f2fe00a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.427+0000 7f2ff5017640 1 -- 172.21.15.31:0/3136841793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff015a710 msgr2=0x7f2ff0166620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.427+0000 7f2ff5017640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff015a710 0x7f2ff0166620 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800efc0 tx=0x7f2fe800c490 comp rx=0 tx=0).stop 2023-11-26T07:18:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.431+0000 7f2ff5017640 1 -- 172.21.15.31:0/3136841793 shutdown_connections 2023-11-26T07:18:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.431+0000 7f2ff5017640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fd0066620 0x7f2fd0068ae0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.431+0000 7f2ff5017640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ff015a710 0x7f2ff0166620 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.431+0000 7f2ff5017640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2ff00cdfb0 0x7f2ff009b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.431+0000 7f2ff5017640 1 --2- 172.21.15.31:0/3136841793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ff00cbd50 0x7f2ff0099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.431+0000 7f2ff5017640 1 -- 172.21.15.31:0/3136841793 >> 172.21.15.31:0/3136841793 conn(0x7f2ff00b0cb0 msgr2=0x7f2ff00ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:45.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.431+0000 7f2ff5017640 1 -- 172.21.15.31:0/3136841793 shutdown_connections 2023-11-26T07:18:45.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.431+0000 7f2ff5017640 1 -- 172.21.15.31:0/3136841793 wait complete. 2023-11-26T07:18:45.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:18:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.591+0000 7f7009baa640 1 Processor -- start 2023-11-26T07:18:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.591+0000 7f7009baa640 1 -- start start 2023-11-26T07:18:45.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7009baa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7004159f00 0x7f70040ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7009baa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7004070890 0x7f7004070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7009baa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7004072950 con 0x7f70040ae680 2023-11-26T07:18:45.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7009baa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700406e2b0 con 0x7f7004070890 2023-11-26T07:18:45.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7009baa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700406c6f0 con 0x7f7004159f00 2023-11-26T07:18:45.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7002ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f70040ae680 0x7f70040cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47554/0 (socket says 172.21.15.31:47554) 2023-11-26T07:18:45.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7002ffd640 1 -- 172.21.15.31:0/1701776566 learned_addr learned my addr 172.21.15.31:0/1701776566 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f70037fe640 1 --2- 172.21.15.31:0/1701776566 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7004159f00 0x7f70040ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7003fff640 1 --2- 172.21.15.31:0/1701776566 >> v2:172.21.15.97:3300/0 conn(0x7f7004070890 0x7f7004070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f70027fc640 1 -- 172.21.15.31:0/1701776566 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160231688 0 0) 0x7f7004072950 con 0x7f70040ae680 2023-11-26T07:18:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f70027fc640 1 -- 172.21.15.31:0/1701776566 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fec005740 con 0x7f70040ae680 2023-11-26T07:18:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f70027fc640 1 -- 172.21.15.31:0/1701776566 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3586129209 0 0) 0x7f6fec005740 con 0x7f70040ae680 2023-11-26T07:18:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7003fff640 1 -- 172.21.15.31:0/1701776566 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7004159f00 msgr2=0x7f70040ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7003fff640 1 --2- 172.21.15.31:0/1701776566 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7004159f00 0x7f70040ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7003fff640 1 -- 172.21.15.31:0/1701776566 >> v1:172.21.15.31:6789/0 conn(0x7f70040ae680 legacy=0x7f70040cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.595+0000 7f7003fff640 1 -- 172.21.15.31:0/1701776566 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700406e550 con 0x7f7004070890 2023-11-26T07:18:45.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7003fff640 1 --2- 172.21.15.31:0/1701776566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004070890 0x7f7004070cf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009d90 tx=0x7f6ffc01c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce0ac41d917b2cdd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:45.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f70027fc640 1 -- 172.21.15.31:0/1701776566 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8001680 con 0x7f7004070890 2023-11-26T07:18:45.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f70027fc640 1 -- 172.21.15.31:0/1701776566 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc0a55c0 con 0x7f7004070890 2023-11-26T07:18:45.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f70027fc640 1 -- 172.21.15.31:0/1701776566 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc0a5950 con 0x7f7004070890 2023-11-26T07:18:45.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7009baa640 1 -- 172.21.15.31:0/1701776566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004070890 msgr2=0x7f7004070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7009baa640 1 --2- 172.21.15.31:0/1701776566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004070890 0x7f7004070cf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009d90 tx=0x7f6ffc01c840 comp rx=0 tx=0).stop 2023-11-26T07:18:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7009baa640 1 -- 172.21.15.31:0/1701776566 shutdown_connections 2023-11-26T07:18:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7009baa640 1 --2- 172.21.15.31:0/1701776566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7004070890 0x7f7004070cf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7009baa640 1 --2- 172.21.15.31:0/1701776566 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7004159f00 0x7f70040ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7009baa640 1 -- 172.21.15.31:0/1701776566 >> 172.21.15.31:0/1701776566 conn(0x7f70040b0cb0 msgr2=0x7f70040b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7009baa640 1 -- 172.21.15.31:0/1701776566 shutdown_connections 2023-11-26T07:18:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7009baa640 1 -- 172.21.15.31:0/1701776566 wait complete. 2023-11-26T07:18:45.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7009baa640 1 Processor -- start 2023-11-26T07:18:45.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.599+0000 7f7009baa640 1 -- start start 2023-11-26T07:18:45.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7009baa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7004070890 0x7f700414d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7009baa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70040ae680 0x7f700415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7009baa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700415cf20 0x7f700415a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7009baa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7004152920 con 0x7f700415cf20 2023-11-26T07:18:45.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7009baa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7004153780 con 0x7f70040ae680 2023-11-26T07:18:45.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7009baa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7004152b70 con 0x7f7004070890 2023-11-26T07:18:45.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7003fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700415cf20 0x7f700415a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7003fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700415cf20 0x7f700415a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55978/0 (socket says 172.21.15.31:55978) 2023-11-26T07:18:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7003fff640 1 -- 172.21.15.31:0/4129085556 learned_addr learned my addr 172.21.15.31:0/4129085556 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7002ffd640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70040ae680 0x7f700415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f70037fe640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7004070890 0x7f700414d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7003fff640 1 -- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7004070890 msgr2=0x7f700414d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7003fff640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7004070890 0x7f700414d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7003fff640 1 -- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70040ae680 msgr2=0x7f700415c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7003fff640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70040ae680 0x7f700415c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7003fff640 1 -- 172.21.15.31:0/4129085556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc009a70 con 0x7f700415cf20 2023-11-26T07:18:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7003fff640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700415cf20 0x7f700415a690 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009d90 tx=0x7f6ffc0195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:45.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f6febfff640 1 -- 172.21.15.31:0/4129085556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc0b9650 con 0x7f700415cf20 2023-11-26T07:18:45.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f6febfff640 1 -- 172.21.15.31:0/4129085556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc0b97b0 con 0x7f700415cf20 2023-11-26T07:18:45.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f6febfff640 1 -- 172.21.15.31:0/4129085556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc0b9b40 con 0x7f700415cf20 2023-11-26T07:18:45.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7009baa640 1 -- 172.21.15.31:0/4129085556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7004152e50 con 0x7f700415cf20 2023-11-26T07:18:45.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7009baa640 1 -- 172.21.15.31:0/4129085556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7004153410 con 0x7f700415cf20 2023-11-26T07:18:45.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.603+0000 7f7009baa640 1 -- 172.21.15.31:0/4129085556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70040671f0 con 0x7f700415cf20 2023-11-26T07:18:45.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.607+0000 7f6febfff640 1 -- 172.21.15.31:0/4129085556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f6ffc0b9ca0 con 0x7f700415cf20 2023-11-26T07:18:45.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.607+0000 7f6febfff640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff0066620 0x7f6ff0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:45.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.607+0000 7f6febfff640 1 -- 172.21.15.31:0/4129085556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ffc11f3b0 con 0x7f700415cf20 2023-11-26T07:18:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.607+0000 7f70037fe640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff0066620 0x7f6ff0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.607+0000 7f70037fe640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff0066620 0x7f6ff0068ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40045b0 tx=0x7f6ff400a500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:45.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.611+0000 7f6febfff640 1 -- 172.21.15.31:0/4129085556 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ffc0ec6e0 con 0x7f700415cf20 2023-11-26T07:18:45.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.783+0000 7f7009baa640 1 -- 172.21.15.31:0/4129085556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f700406b460 con 0x7f700415cf20 2023-11-26T07:18:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.783+0000 7f6febfff640 1 -- 172.21.15.31:0/4129085556 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6ffc009450 con 0x7f700415cf20 2023-11-26T07:18:45.791 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:18:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 -- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff0066620 msgr2=0x7f6ff0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff0066620 0x7f6ff0068ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ff40045b0 tx=0x7f6ff400a500 comp rx=0 tx=0).stop 2023-11-26T07:18:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 -- 172.21.15.31:0/4129085556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700415cf20 msgr2=0x7f700415a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:45.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700415cf20 0x7f700415a690 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009d90 tx=0x7f6ffc0195e0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 -- 172.21.15.31:0/4129085556 shutdown_connections 2023-11-26T07:18:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6ff0066620 0x7f6ff0068ae0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f700415cf20 0x7f700415a690 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70040ae680 0x7f700415c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 --2- 172.21.15.31:0/4129085556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7004070890 0x7f700414d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 -- 172.21.15.31:0/4129085556 >> 172.21.15.31:0/4129085556 conn(0x7f70040b0cb0 msgr2=0x7f70040b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 -- 172.21.15.31:0/4129085556 shutdown_connections 2023-11-26T07:18:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:45.787+0000 7f7009baa640 1 -- 172.21.15.31:0/4129085556 wait complete. 2023-11-26T07:18:45.812 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:18:45.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:45.812 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:18:45.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:18:45.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:18:45.844 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:18:45.896 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:45.896 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:45.906 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:18:45.906 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:45.906 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:45.948 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:18:45.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.083+0000 7f2fe4e68640 1 Processor -- start 2023-11-26T07:18:46.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fe4e68640 1 -- start start 2023-11-26T07:18:46.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fe4e68640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fe00ac630 0x7f2fe00706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fe4e68640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fe0070c20 0x7f2fe006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fe4e68640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe0145060 con 0x7f2fe0144c80 2023-11-26T07:18:46.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fe4e68640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe0071fd0 con 0x7f2fe00ac630 2023-11-26T07:18:46.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fe4e68640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe0071450 con 0x7f2fe0070c20 2023-11-26T07:18:46.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fd7fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fe00ac630 0x7f2fe00706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fd7fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fe00ac630 0x7f2fe00706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55988/0 (socket says 172.21.15.31:55988) 2023-11-26T07:18:46.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fd7fff640 1 -- 172.21.15.31:0/1917314313 learned_addr learned my addr 172.21.15.31:0/1917314313 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:46.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fded9d640 1 --2- 172.21.15.31:0/1917314313 >> v2:172.21.15.97:3300/0 conn(0x7f2fe0070c20 0x7f2fe006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fd7fff640 1 -- 172.21.15.31:0/1917314313 >> v2:172.21.15.97:3300/0 conn(0x7f2fe0070c20 msgr2=0x7f2fe006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fd7fff640 1 --2- 172.21.15.31:0/1917314313 >> v2:172.21.15.97:3300/0 conn(0x7f2fe0070c20 0x7f2fe006c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fd7fff640 1 -- 172.21.15.31:0/1917314313 >> v1:172.21.15.97:6789/0 conn(0x7f2fe0144c80 legacy=0x7f2fe00abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.087+0000 7f2fd7fff640 1 -- 172.21.15.31:0/1917314313 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe0071730 con 0x7f2fe00ac630 2023-11-26T07:18:46.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fd7fff640 1 --2- 172.21.15.31:0/1917314313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe00ac630 0x7f2fe00706e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00ce840 tx=0x7f2fc801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ec707e5ded0f7c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fddd9b640 1 -- 172.21.15.31:0/1917314313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80aa430 con 0x7f2fe00ac630 2023-11-26T07:18:46.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fddd9b640 1 -- 172.21.15.31:0/1917314313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc80aa590 con 0x7f2fe00ac630 2023-11-26T07:18:46.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fddd9b640 1 -- 172.21.15.31:0/1917314313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80aa920 con 0x7f2fe00ac630 2023-11-26T07:18:46.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1917314313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe00ac630 msgr2=0x7f2fe00706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fe4e68640 1 --2- 172.21.15.31:0/1917314313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe00ac630 0x7f2fe00706e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00ce840 tx=0x7f2fc801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1917314313 shutdown_connections 2023-11-26T07:18:46.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fe4e68640 1 --2- 172.21.15.31:0/1917314313 >> v2:172.21.15.97:3300/0 conn(0x7f2fe0070c20 0x7f2fe006c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fe4e68640 1 --2- 172.21.15.31:0/1917314313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe00ac630 0x7f2fe00706e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1917314313 >> 172.21.15.31:0/1917314313 conn(0x7f2fe00b0c40 msgr2=0x7f2fe00b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:46.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1917314313 shutdown_connections 2023-11-26T07:18:46.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1917314313 wait complete. 2023-11-26T07:18:46.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fe4e68640 1 Processor -- start 2023-11-26T07:18:46.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.091+0000 7f2fe4e68640 1 -- start start 2023-11-26T07:18:46.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fe4e68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fe0070c20 0x7f2fe015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fe4e68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fe00ac630 0x7f2fe0141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fe4e68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe0144c80 0x7f2fe01f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fe4e68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe015c170 con 0x7f2fe0144c80 2023-11-26T07:18:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fe4e68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe015bf10 con 0x7f2fe00ac630 2023-11-26T07:18:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fe4e68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe015e290 con 0x7f2fe0070c20 2023-11-26T07:18:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fded9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe0144c80 0x7f2fe01f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fded9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe0144c80 0x7f2fe01f5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55996/0 (socket says 172.21.15.31:55996) 2023-11-26T07:18:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fded9d640 1 -- 172.21.15.31:0/1958194231 learned_addr learned my addr 172.21.15.31:0/1958194231 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fd7fff640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fe00ac630 0x7f2fe0141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fde59c640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fe0070c20 0x7f2fe015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fded9d640 1 -- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fe0070c20 msgr2=0x7f2fe015b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fded9d640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fe0070c20 0x7f2fe015b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fded9d640 1 -- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fe00ac630 msgr2=0x7f2fe0141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fded9d640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fe00ac630 0x7f2fe0141430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fded9d640 1 -- 172.21.15.31:0/1958194231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc80094d0 con 0x7f2fe0144c80 2023-11-26T07:18:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fded9d640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe0144c80 0x7f2fe01f5d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800efc0 tx=0x7f2fd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fd6ffd640 1 -- 172.21.15.31:0/1958194231 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8014480 con 0x7f2fe0144c80 2023-11-26T07:18:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fd6ffd640 1 -- 172.21.15.31:0/1958194231 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd800f040 con 0x7f2fe0144c80 2023-11-26T07:18:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1958194231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe015e570 con 0x7f2fe0144c80 2023-11-26T07:18:46.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fd6ffd640 1 -- 172.21.15.31:0/1958194231 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8014820 con 0x7f2fe0144c80 2023-11-26T07:18:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1958194231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe0141970 con 0x7f2fe0144c80 2023-11-26T07:18:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.095+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1958194231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe00b1ba0 con 0x7f2fe0144c80 2023-11-26T07:18:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.099+0000 7f2fd6ffd640 1 -- 172.21.15.31:0/1958194231 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2fd8014980 con 0x7f2fe0144c80 2023-11-26T07:18:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.099+0000 7f2fd6ffd640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fc0066620 0x7f2fc0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.099+0000 7f2fd6ffd640 1 -- 172.21.15.31:0/1958194231 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fd8084cf0 con 0x7f2fe0144c80 2023-11-26T07:18:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.099+0000 7f2fde59c640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fc0066620 0x7f2fc0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.099+0000 7f2fde59c640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fc0066620 0x7f2fc0068ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0007fd0 tx=0x7f2fd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.103+0000 7f2fd6ffd640 1 -- 172.21.15.31:0/1958194231 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2fd8051e80 con 0x7f2fe0144c80 2023-11-26T07:18:46.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.287+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1958194231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fe004e850 con 0x7f2fe0144c80 2023-11-26T07:18:46.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fd6ffd640 1 -- 172.21.15.31:0/1958194231 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f2fd8018090 con 0x7f2fe0144c80 2023-11-26T07:18:46.296 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:46.296 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3937918726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3937918726},{"type":"v1","addr":"172.21.15.97:6837","nonce":3937918726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2083665480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2083665480},{"type":"v1","addr":"172.21.15.97:6835","nonce":2083665480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1187139549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187139549},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187139549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:24.963367+0000","modified":"2023-11-26T07:18:27.980260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4036387633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4036387633},{"type":"v1","addr":"172.21.15.31:6835","nonce":4036387633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:18:46.297 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2023-11-26T07:18:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fc0066620 msgr2=0x7f2fc0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fc0066620 0x7f2fc0068ae0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0007fd0 tx=0x7f2fd0009040 comp rx=0 tx=0).stop 2023-11-26T07:18:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1958194231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe0144c80 msgr2=0x7f2fe01f5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe0144c80 0x7f2fe01f5d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800efc0 tx=0x7f2fd800c490 comp rx=0 tx=0).stop 2023-11-26T07:18:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1958194231 shutdown_connections 2023-11-26T07:18:46.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fc0066620 0x7f2fc0068ae0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe0144c80 0x7f2fe01f5d80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fe00ac630 0x7f2fe0141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 --2- 172.21.15.31:0/1958194231 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fe0070c20 0x7f2fe015b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1958194231 >> 172.21.15.31:0/1958194231 conn(0x7f2fe00b0c40 msgr2=0x7f2fe01643d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:46.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1958194231 shutdown_connections 2023-11-26T07:18:46.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.291+0000 7f2fe4e68640 1 -- 172.21.15.31:0/1958194231 wait complete. 2023-11-26T07:18:46.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:46.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf746b640 1 Processor -- start 2023-11-26T07:18:46.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf746b640 1 -- start start 2023-11-26T07:18:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf746b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4bf015d530 0x7f4bf00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf746b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4bf00cf1b0 0x7f4bf00cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf746b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf00d02f0 0x7f4bf01611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf746b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf00d07a0 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf746b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf00d12e0 con 0x7f4bf015d530 2023-11-26T07:18:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf746b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00d15c0 con 0x7f4bf00d02f0 2023-11-26T07:18:46.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf4a06640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4bf00cf1b0 0x7f4bf00cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf4a06640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4bf00cf1b0 0x7f4bf00cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56000/0 (socket says 172.21.15.31:56000) 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf4a06640 1 -- 172.21.15.31:0/4164376994 learned_addr learned my addr 172.21.15.31:0/4164376994 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf5207640 1 --2- 172.21.15.31:0/4164376994 >> v2:172.21.15.97:3300/0 conn(0x7f4bf015d530 0x7f4bf00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf5a08640 1 --2- 172.21.15.31:0/4164376994 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf00d02f0 0x7f4bf01611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf4a06640 1 -- 172.21.15.31:0/4164376994 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf00d02f0 msgr2=0x7f4bf01611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf4a06640 1 --2- 172.21.15.31:0/4164376994 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf00d02f0 0x7f4bf01611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf4a06640 1 -- 172.21.15.31:0/4164376994 >> v2:172.21.15.97:3300/0 conn(0x7f4bf015d530 msgr2=0x7f4bf00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf4a06640 1 --2- 172.21.15.31:0/4164376994 >> v2:172.21.15.97:3300/0 conn(0x7f4bf015d530 0x7f4bf00cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.455+0000 7f4bf4a06640 1 -- 172.21.15.31:0/4164376994 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00d18a0 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4bf4a06640 1 --2- 172.21.15.31:0/4164376994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 0x7f4bf00cfdb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009870 tx=0x7f4be001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171680bdb062ca42 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4be7fff640 1 -- 172.21.15.31:0/4164376994 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be00a89b0 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4be7fff640 1 -- 172.21.15.31:0/4164376994 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be00a8b10 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4be7fff640 1 -- 172.21.15.31:0/4164376994 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be00a8ea0 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4bf746b640 1 -- 172.21.15.31:0/4164376994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 msgr2=0x7f4bf00cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4bf746b640 1 --2- 172.21.15.31:0/4164376994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 0x7f4bf00cfdb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009870 tx=0x7f4be001ecf0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4bf746b640 1 -- 172.21.15.31:0/4164376994 shutdown_connections 2023-11-26T07:18:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4bf746b640 1 --2- 172.21.15.31:0/4164376994 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf00d02f0 0x7f4bf01611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4bf746b640 1 --2- 172.21.15.31:0/4164376994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 0x7f4bf00cfdb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4bf746b640 1 --2- 172.21.15.31:0/4164376994 >> v2:172.21.15.97:3300/0 conn(0x7f4bf015d530 0x7f4bf00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4bf746b640 1 -- 172.21.15.31:0/4164376994 >> 172.21.15.31:0/4164376994 conn(0x7f4bf009c6d0 msgr2=0x7f4bf00d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4bf746b640 1 -- 172.21.15.31:0/4164376994 shutdown_connections 2023-11-26T07:18:46.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.459+0000 7f4bf746b640 1 -- 172.21.15.31:0/4164376994 wait complete. 2023-11-26T07:18:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 Processor -- start 2023-11-26T07:18:46.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 -- start start 2023-11-26T07:18:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 0x7f4bf00b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf00d02f0 0x7f4bf015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf015d530 0x7f4bf0160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00b71e0 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00c7fa0 con 0x7f4bf00d02f0 2023-11-26T07:18:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00b8990 con 0x7f4bf015d530 2023-11-26T07:18:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5207640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 0x7f4bf00b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5207640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 0x7f4bf00b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56012/0 (socket says 172.21.15.31:56012) 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5207640 1 -- 172.21.15.31:0/2203614554 learned_addr learned my addr 172.21.15.31:0/2203614554 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf4a06640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf00d02f0 0x7f4bf015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5207640 1 -- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf015d530 msgr2=0x7f4bf0160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5a08640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf015d530 0x7f4bf0160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5207640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf015d530 0x7f4bf0160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5207640 1 -- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf00d02f0 msgr2=0x7f4bf015ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5207640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf00d02f0 0x7f4bf015ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5207640 1 -- 172.21.15.31:0/2203614554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00094d0 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5a08640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf015d530 0x7f4bf0160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf4a06640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf00d02f0 0x7f4bf015ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf5207640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 0x7f4bf00b7600 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00d5f0 tx=0x7f4bec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4be5ffb640 1 -- 172.21.15.31:0/2203614554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0125f0 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4be5ffb640 1 -- 172.21.15.31:0/2203614554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec012750 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4be5ffb640 1 -- 172.21.15.31:0/2203614554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec012ae0 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 -- 172.21.15.31:0/2203614554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf00b86b0 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 -- 172.21.15.31:0/2203614554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf00b6e00 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.463+0000 7f4bf746b640 1 -- 172.21.15.31:0/2203614554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf004e850 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.467+0000 7f4be5ffb640 1 -- 172.21.15.31:0/2203614554 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f4bec012c40 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.467+0000 7f4be5ffb640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bd4066620 0x7f4bd4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.467+0000 7f4be5ffb640 1 -- 172.21.15.31:0/2203614554 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bec084680 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.467+0000 7f4bf4a06640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bd4066620 0x7f4bd4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.467+0000 7f4bf4a06640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bd4066620 0x7f4bd4068ae0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4be00b2000 tx=0x7f4be0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.471+0000 7f4be5ffb640 1 -- 172.21.15.31:0/2203614554 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4bec051870 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.627+0000 7f4bf746b640 1 -- 172.21.15.31:0/2203614554 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bf00cf980 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.627+0000 7f4be5ffb640 1 -- 172.21.15.31:0/2203614554 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16485 (secure 0 0 0) 0x7f4bec007380 con 0x7f4bf00cf1b0 2023-11-26T07:18:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:46.634 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:26.567047+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:22.773068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:23.967477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 -- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bd4066620 msgr2=0x7f4bd4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bd4066620 0x7f4bd4068ae0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4be00b2000 tx=0x7f4be0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 -- 172.21.15.31:0/2203614554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 msgr2=0x7f4bf00b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 0x7f4bf00b7600 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00d5f0 tx=0x7f4bec00dab0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 -- 172.21.15.31:0/2203614554 shutdown_connections 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bd4066620 0x7f4bd4068ae0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4bf015d530 0x7f4bf0160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bf00d02f0 0x7f4bf015ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 --2- 172.21.15.31:0/2203614554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bf00cf1b0 0x7f4bf00b7600 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 -- 172.21.15.31:0/2203614554 >> 172.21.15.31:0/2203614554 conn(0x7f4bf009c6d0 msgr2=0x7f4bf00b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 -- 172.21.15.31:0/2203614554 shutdown_connections 2023-11-26T07:18:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.631+0000 7f4bf746b640 1 -- 172.21.15.31:0/2203614554 wait complete. 2023-11-26T07:18:46.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:18:22.773068+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:18:23.967477+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:46.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:46.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f33069df640 1 Processor -- start 2023-11-26T07:18:46.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f33069df640 1 -- start start 2023-11-26T07:18:46.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f33069df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cb9b0 0x7f330006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f33069df640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f33000cbd90 0x7f330015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f33069df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3300159630 con 0x7f33000ce5d0 2023-11-26T07:18:46.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f33069df640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330015bba0 con 0x7f33000cbd90 2023-11-26T07:18:46.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f33069df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33000cdd50 con 0x7f33000cb9b0 2023-11-26T07:18:46.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f32fffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cb9b0 0x7f330006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f32fffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cb9b0 0x7f330006e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:56748/0 (socket says 172.21.15.31:56748) 2023-11-26T07:18:46.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f32fffff640 1 -- 172.21.15.31:0/2236656352 learned_addr learned my addr 172.21.15.31:0/2236656352 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:46.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.795+0000 7f32ff7fe640 1 --2- 172.21.15.31:0/2236656352 >> v2:172.21.15.97:3300/0 conn(0x7f33000cbd90 0x7f330015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.799+0000 7f32fffff640 1 -- 172.21.15.31:0/2236656352 >> v2:172.21.15.97:3300/0 conn(0x7f33000cbd90 msgr2=0x7f330015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:46.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.799+0000 7f32fffff640 1 --2- 172.21.15.31:0/2236656352 >> v2:172.21.15.97:3300/0 conn(0x7f33000cbd90 0x7f330015f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.799+0000 7f32fffff640 1 -- 172.21.15.31:0/2236656352 >> v1:172.21.15.97:6789/0 conn(0x7f33000ce5d0 legacy=0x7f330015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.799+0000 7f32feffd640 1 -- 172.21.15.31:0/2236656352 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29361072 0 0) 0x7f3300159630 con 0x7f33000ce5d0 2023-11-26T07:18:46.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.799+0000 7f32fffff640 1 -- 172.21.15.31:0/2236656352 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330015f6d0 con 0x7f33000cb9b0 2023-11-26T07:18:46.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.799+0000 7f32fffff640 1 --2- 172.21.15.31:0/2236656352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cb9b0 0x7f330006e3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009870 tx=0x7f32f001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4f68cff1976ce0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.799+0000 7f32feffd640 1 -- 172.21.15.31:0/2236656352 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f00aa670 con 0x7f33000cb9b0 2023-11-26T07:18:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.799+0000 7f32feffd640 1 -- 172.21.15.31:0/2236656352 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f00aa7d0 con 0x7f33000cb9b0 2023-11-26T07:18:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.799+0000 7f32feffd640 1 -- 172.21.15.31:0/2236656352 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f00aab60 con 0x7f33000cb9b0 2023-11-26T07:18:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 -- 172.21.15.31:0/2236656352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cb9b0 msgr2=0x7f330006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 --2- 172.21.15.31:0/2236656352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cb9b0 0x7f330006e3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009870 tx=0x7f32f001c510 comp rx=0 tx=0).stop 2023-11-26T07:18:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 -- 172.21.15.31:0/2236656352 shutdown_connections 2023-11-26T07:18:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 --2- 172.21.15.31:0/2236656352 >> v2:172.21.15.97:3300/0 conn(0x7f33000cbd90 0x7f330015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 --2- 172.21.15.31:0/2236656352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cb9b0 0x7f330006e3f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 -- 172.21.15.31:0/2236656352 >> 172.21.15.31:0/2236656352 conn(0x7f33000b0cb0 msgr2=0x7f33000b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:46.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 -- 172.21.15.31:0/2236656352 shutdown_connections 2023-11-26T07:18:46.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 -- 172.21.15.31:0/2236656352 wait complete. 2023-11-26T07:18:46.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 Processor -- start 2023-11-26T07:18:46.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 -- start start 2023-11-26T07:18:46.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33000cb9b0 0x7f3300165f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cbd90 0x7f33000abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33000ce5d0 0x7f3300147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3300155440 con 0x7f33000ce5d0 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3300164ea0 con 0x7f33000cb9b0 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f33069df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3300165180 con 0x7f33000cbd90 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f3304f7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33000ce5d0 0x7f3300147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f3304f7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33000ce5d0 0x7f3300147a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56022/0 (socket says 172.21.15.31:56022) 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f3304f7c640 1 -- 172.21.15.31:0/2107362230 learned_addr learned my addr 172.21.15.31:0/2107362230 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f32ff7fe640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cbd90 0x7f33000abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.803+0000 7f32fffff640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33000cb9b0 0x7f3300165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f3304f7c640 1 -- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cbd90 msgr2=0x7f33000abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f3304f7c640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cbd90 0x7f33000abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f3304f7c640 1 -- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33000cb9b0 msgr2=0x7f3300165f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f3304f7c640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33000cb9b0 0x7f3300165f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f3304f7c640 1 -- 172.21.15.31:0/2107362230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f00094d0 con 0x7f33000ce5d0 2023-11-26T07:18:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f32ff7fe640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cbd90 0x7f33000abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f32fffff640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33000cb9b0 0x7f3300165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:46.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f3304f7c640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33000ce5d0 0x7f3300147a30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f32f8002240 tx=0x7f32f800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f32fcff9640 1 -- 172.21.15.31:0/2107362230 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f8003db0 con 0x7f33000ce5d0 2023-11-26T07:18:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f32fcff9640 1 -- 172.21.15.31:0/2107362230 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f800ce30 con 0x7f33000ce5d0 2023-11-26T07:18:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f33069df640 1 -- 172.21.15.31:0/2107362230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3300165460 con 0x7f33000ce5d0 2023-11-26T07:18:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f32fcff9640 1 -- 172.21.15.31:0/2107362230 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f8005270 con 0x7f33000ce5d0 2023-11-26T07:18:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f33069df640 1 -- 172.21.15.31:0/2107362230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3300165a20 con 0x7f33000ce5d0 2023-11-26T07:18:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f33069df640 1 -- 172.21.15.31:0/2107362230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33000671f0 con 0x7f33000ce5d0 2023-11-26T07:18:46.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f32fcff9640 1 -- 172.21.15.31:0/2107362230 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f32f80053d0 con 0x7f33000ce5d0 2023-11-26T07:18:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.807+0000 7f32fcff9640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32ec066620 0x7f32ec068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:46.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.811+0000 7f32fcff9640 1 -- 172.21.15.31:0/2107362230 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32f8083930 con 0x7f33000ce5d0 2023-11-26T07:18:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.811+0000 7f32fffff640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32ec066620 0x7f32ec068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:46.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.811+0000 7f32fffff640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32ec066620 0x7f32ec068ae0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f32f00ae000 tx=0x7f32f00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:46.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.811+0000 7f32fcff9640 1 -- 172.21.15.31:0/2107362230 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f32f8050b40 con 0x7f33000ce5d0 2023-11-26T07:18:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 -- 172.21.15.31:0/2107362230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f330006b460 con 0x7f33000ce5d0 2023-11-26T07:18:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f32fcff9640 1 -- 172.21.15.31:0/2107362230 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32f8017070 con 0x7f33000ce5d0 2023-11-26T07:18:46.978 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 -- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32ec066620 msgr2=0x7f32ec068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32ec066620 0x7f32ec068ae0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f32f00ae000 tx=0x7f32f00a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 -- 172.21.15.31:0/2107362230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33000ce5d0 msgr2=0x7f3300147a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33000ce5d0 0x7f3300147a30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f32f8002240 tx=0x7f32f800cbe0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 -- 172.21.15.31:0/2107362230 shutdown_connections 2023-11-26T07:18:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f32ec066620 0x7f32ec068ae0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33000ce5d0 0x7f3300147a30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33000cbd90 0x7f33000abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 --2- 172.21.15.31:0/2107362230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33000cb9b0 0x7f3300165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 -- 172.21.15.31:0/2107362230 >> 172.21.15.31:0/2107362230 conn(0x7f33000b0cb0 msgr2=0x7f330015e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 -- 172.21.15.31:0/2107362230 shutdown_connections 2023-11-26T07:18:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:46.971+0000 7f33069df640 1 -- 172.21.15.31:0/2107362230 wait complete. 2023-11-26T07:18:46.998 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:18:47.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33f59d1640 1 Processor -- start 2023-11-26T07:18:47.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33f59d1640 1 -- start start 2023-11-26T07:18:47.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33f59d1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f33f00cdfb0 0x7f33f00abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33f59d1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f33f015a710 0x7f33f00ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33f59d1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f00ce8d0 con 0x7f33f00cbd50 2023-11-26T07:18:47.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33f59d1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f00cca80 con 0x7f33f00cdfb0 2023-11-26T07:18:47.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33f59d1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f00b2710 con 0x7f33f015a710 2023-11-26T07:18:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33eeffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f33f00cdfb0 0x7f33f00abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33ef7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f33f00cbd50 0x7f33f00cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47564/0 (socket says 172.21.15.31:47564) 2023-11-26T07:18:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33eeffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f33f00cdfb0 0x7f33f00abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56026/0 (socket says 172.21.15.31:56026) 2023-11-26T07:18:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33ef7fe640 1 -- 172.21.15.31:0/622068437 learned_addr learned my addr 172.21.15.31:0/622068437 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33ee7fc640 1 --2- 172.21.15.31:0/622068437 >> v2:172.21.15.97:3300/0 conn(0x7f33f015a710 0x7f33f00ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33eeffd640 1 -- 172.21.15.31:0/622068437 >> v2:172.21.15.97:3300/0 conn(0x7f33f015a710 msgr2=0x7f33f00ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33eeffd640 1 --2- 172.21.15.31:0/622068437 >> v2:172.21.15.97:3300/0 conn(0x7f33f015a710 0x7f33f00ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33eeffd640 1 -- 172.21.15.31:0/622068437 >> v1:172.21.15.31:6789/0 conn(0x7f33f00cbd50 legacy=0x7f33f00cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.139+0000 7f33eeffd640 1 -- 172.21.15.31:0/622068437 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f00b29f0 con 0x7f33f00cdfb0 2023-11-26T07:18:47.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33eeffd640 1 --2- 172.21.15.31:0/622068437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f00cdfb0 0x7f33f00abf90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f33f00d0040 tx=0x7f33e001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcc99f7dbf22db2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33edffb640 1 -- 172.21.15.31:0/622068437 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e00a7070 con 0x7f33f00cdfb0 2023-11-26T07:18:47.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33edffb640 1 -- 172.21.15.31:0/622068437 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e00ab430 con 0x7f33f00cdfb0 2023-11-26T07:18:47.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33edffb640 1 -- 172.21.15.31:0/622068437 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e00ab7c0 con 0x7f33f00cdfb0 2023-11-26T07:18:47.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33f59d1640 1 -- 172.21.15.31:0/622068437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f00cdfb0 msgr2=0x7f33f00abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33f59d1640 1 --2- 172.21.15.31:0/622068437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f00cdfb0 0x7f33f00abf90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f33f00d0040 tx=0x7f33e001c430 comp rx=0 tx=0).stop 2023-11-26T07:18:47.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33f59d1640 1 -- 172.21.15.31:0/622068437 shutdown_connections 2023-11-26T07:18:47.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33f59d1640 1 --2- 172.21.15.31:0/622068437 >> v2:172.21.15.97:3300/0 conn(0x7f33f015a710 0x7f33f00ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33f59d1640 1 --2- 172.21.15.31:0/622068437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f00cdfb0 0x7f33f00abf90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33f59d1640 1 -- 172.21.15.31:0/622068437 >> 172.21.15.31:0/622068437 conn(0x7f33f00b0cb0 msgr2=0x7f33f00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:47.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33f59d1640 1 -- 172.21.15.31:0/622068437 shutdown_connections 2023-11-26T07:18:47.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.143+0000 7f33f59d1640 1 -- 172.21.15.31:0/622068437 wait complete. 2023-11-26T07:18:47.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33f59d1640 1 Processor -- start 2023-11-26T07:18:47.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33f59d1640 1 -- start start 2023-11-26T07:18:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33f59d1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33f00cbd50 0x7f33f014dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33f59d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f00cdfb0 0x7f33f0163f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33f59d1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f015a710 0x7f33f01644c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33f59d1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00722c0 con 0x7f33f015a710 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33f59d1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00715f0 con 0x7f33f00cdfb0 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33f59d1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f0071310 con 0x7f33f00cbd50 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f015a710 0x7f33f01644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f015a710 0x7f33f01644c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56028/0 (socket says 172.21.15.31:56028) 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ef7fe640 1 -- 172.21.15.31:0/2648091722 learned_addr learned my addr 172.21.15.31:0/2648091722 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33eeffd640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33f00cbd50 0x7f33f014dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ee7fc640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f00cdfb0 0x7f33f0163f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ef7fe640 1 -- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33f00cbd50 msgr2=0x7f33f014dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ef7fe640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33f00cbd50 0x7f33f014dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ef7fe640 1 -- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f00cdfb0 msgr2=0x7f33f0163f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ef7fe640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f00cdfb0 0x7f33f0163f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ef7fe640 1 -- 172.21.15.31:0/2648091722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e0009420 con 0x7f33f015a710 2023-11-26T07:18:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33eeffd640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33f00cbd50 0x7f33f014dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ee7fc640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f00cdfb0 0x7f33f0163f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:47.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33ef7fe640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f015a710 0x7f33f01644c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f33e800cbf0 tx=0x7f33e800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33d7fff640 1 -- 172.21.15.31:0/2648091722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800ad40 con 0x7f33f015a710 2023-11-26T07:18:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33d7fff640 1 -- 172.21.15.31:0/2648091722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e8012400 con 0x7f33f015a710 2023-11-26T07:18:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33d7fff640 1 -- 172.21.15.31:0/2648091722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800aea0 con 0x7f33f015a710 2023-11-26T07:18:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33f59d1640 1 -- 172.21.15.31:0/2648091722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f0151da0 con 0x7f33f015a710 2023-11-26T07:18:47.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.147+0000 7f33f59d1640 1 -- 172.21.15.31:0/2648091722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f0152360 con 0x7f33f015a710 2023-11-26T07:18:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.151+0000 7f33f59d1640 1 -- 172.21.15.31:0/2648091722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f00671f0 con 0x7f33f015a710 2023-11-26T07:18:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.151+0000 7f33d7fff640 1 -- 172.21.15.31:0/2648091722 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f33e8007070 con 0x7f33f015a710 2023-11-26T07:18:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.151+0000 7f33d7fff640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f33dc066620 0x7f33dc068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.151+0000 7f33d7fff640 1 -- 172.21.15.31:0/2648091722 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33e8083f70 con 0x7f33f015a710 2023-11-26T07:18:47.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.151+0000 7f33eeffd640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f33dc066620 0x7f33dc068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.155+0000 7f33eeffd640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f33dc066620 0x7f33dc068ae0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f33e00af000 tx=0x7f33e00a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.155+0000 7f33d7fff640 1 -- 172.21.15.31:0/2648091722 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f33e80b5280 con 0x7f33f015a710 2023-11-26T07:18:47.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.311+0000 7f33f59d1640 1 -- 172.21.15.31:0/2648091722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33f006b460 con 0x7f33f015a710 2023-11-26T07:18:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.311+0000 7f33d7fff640 1 -- 172.21.15.31:0/2648091722 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33e8017020 con 0x7f33f015a710 2023-11-26T07:18:47.319 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:18:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 -- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f33dc066620 msgr2=0x7f33dc068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f33dc066620 0x7f33dc068ae0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f33e00af000 tx=0x7f33e00a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 -- 172.21.15.31:0/2648091722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f015a710 msgr2=0x7f33f01644c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f015a710 0x7f33f01644c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f33e800cbf0 tx=0x7f33e800a430 comp rx=0 tx=0).stop 2023-11-26T07:18:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 -- 172.21.15.31:0/2648091722 shutdown_connections 2023-11-26T07:18:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f33dc066620 0x7f33dc068ae0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33f015a710 0x7f33f01644c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33f00cdfb0 0x7f33f0163f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 --2- 172.21.15.31:0/2648091722 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f33f00cbd50 0x7f33f014dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 -- 172.21.15.31:0/2648091722 >> 172.21.15.31:0/2648091722 conn(0x7f33f00b0cb0 msgr2=0x7f33f0159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 -- 172.21.15.31:0/2648091722 shutdown_connections 2023-11-26T07:18:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.315+0000 7f33f59d1640 1 -- 172.21.15.31:0/2648091722 wait complete. 2023-11-26T07:18:47.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:18:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.479+0000 7fe72e279640 1 Processor -- start 2023-11-26T07:18:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.479+0000 7fe72e279640 1 -- start start 2023-11-26T07:18:47.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.479+0000 7fe72e279640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7280cc460 0x7fe7280abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.479+0000 7fe72e279640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7280b2710 con 0x7fe7280cdfb0 2023-11-26T07:18:47.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.479+0000 7fe72e279640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7280b29f0 con 0x7fe72815a710 2023-11-26T07:18:47.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.479+0000 7fe72e279640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7280cca80 con 0x7fe7280cc460 2023-11-26T07:18:47.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.479+0000 7fe727fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe7280cdfb0 0x7fe7280ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47570/0 (socket says 172.21.15.31:47570) 2023-11-26T07:18:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.479+0000 7fe727fff640 1 -- 172.21.15.31:0/1950856234 learned_addr learned my addr 172.21.15.31:0/1950856234 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe72c816640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7280cc460 0x7fe7280abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249114102 0 0) 0x7fe7280b2710 con 0x7fe7280cdfb0 2023-11-26T07:18:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe710003530 con 0x7fe7280cdfb0 2023-11-26T07:18:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1133317943 0 0) 0x7fe710003530 con 0x7fe7280cdfb0 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7280cc460 msgr2=0x7fe7280abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7280cc460 0x7fe7280abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 >> v1:172.21.15.97:6789/0 conn(0x7fe72815a710 legacy=0x7fe7280cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7280b2cd0 con 0x7fe7280cdfb0 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fe718001680 con 0x7fe7280cdfb0 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 >> v1:172.21.15.31:6789/0 conn(0x7fe7280cdfb0 legacy=0x7fe7280ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7100039b0 0x7fe710003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7280cc460 0x7fe7100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710004d10 0x7fe710005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7280cca80 con 0x7fe710004d10 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71000ba40 con 0x7fe7100039b0 2023-11-26T07:18:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71000dde0 con 0x7fe7280cc460 2023-11-26T07:18:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe72c816640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710004d10 0x7fe710005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe7277fe640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7280cc460 0x7fe7100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe727fff640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7100039b0 0x7fe710003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe72c816640 1 -- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7280cc460 msgr2=0x7fe7100047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe72c816640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7280cc460 0x7fe7100047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe72c816640 1 -- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7100039b0 msgr2=0x7fe710003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe72c816640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7100039b0 0x7fe710003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe72c816640 1 -- 172.21.15.31:0/1950856234 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe710003530 con 0x7fe710004d10 2023-11-26T07:18:47.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.483+0000 7fe7277fe640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7280cc460 0x7fe7100047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe72c816640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710004d10 0x7fe710005190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe72000e170 tx=0x7fe72001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47ecf6890ce3aa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7200a5c50 con 0x7fe710004d10 2023-11-26T07:18:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72000f040 con 0x7fe710004d10 2023-11-26T07:18:47.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe726ffd640 1 -- 172.21.15.31:0/1950856234 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7200ad440 con 0x7fe710004d10 2023-11-26T07:18:47.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe72e279640 1 -- 172.21.15.31:0/1950856234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710004d10 msgr2=0x7fe710005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe72e279640 1 --2- 172.21.15.31:0/1950856234 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710004d10 0x7fe710005190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe72000e170 tx=0x7fe72001cc40 comp rx=0 tx=0).stop 2023-11-26T07:18:47.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe72c816640 1 -- 172.21.15.31:0/1950856234 reap_dead start 2023-11-26T07:18:47.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe72e279640 1 -- 172.21.15.31:0/1950856234 shutdown_connections 2023-11-26T07:18:47.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe72e279640 1 -- 172.21.15.31:0/1950856234 >> 172.21.15.31:0/1950856234 conn(0x7fe7280b0cb0 msgr2=0x7fe7280b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:47.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe72e279640 1 -- 172.21.15.31:0/1950856234 shutdown_connections 2023-11-26T07:18:47.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe72e279640 1 -- 172.21.15.31:0/1950856234 wait complete. 2023-11-26T07:18:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe72e279640 1 Processor -- start 2023-11-26T07:18:47.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.487+0000 7fe72e279640 1 -- start start 2023-11-26T07:18:47.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72e279640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe72814da90 0x7fe728150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72e279640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72815cd00 0x7fe7281f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72e279640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7280afca0 0x7fe7280b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72e279640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7281f6a80 con 0x7fe7280afca0 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72e279640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7280b0660 con 0x7fe72815cd00 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72e279640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72815dad0 con 0x7fe72814da90 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72c816640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7280afca0 0x7fe7280b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72c816640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7280afca0 0x7fe7280b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56054/0 (socket says 172.21.15.31:56054) 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72c816640 1 -- 172.21.15.31:0/4157744373 learned_addr learned my addr 172.21.15.31:0/4157744373 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe7277fe640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72815cd00 0x7fe7281f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe727fff640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe72814da90 0x7fe728150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72c816640 1 -- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe72814da90 msgr2=0x7fe728150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72c816640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe72814da90 0x7fe728150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72c816640 1 -- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72815cd00 msgr2=0x7fe7281f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72c816640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72815cd00 0x7fe7281f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72c816640 1 -- 172.21.15.31:0/4157744373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72815d510 con 0x7fe7280afca0 2023-11-26T07:18:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72c816640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7280afca0 0x7fe7280b0120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe72000e170 tx=0x7fe720009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe724ff9640 1 -- 172.21.15.31:0/4157744373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7200ba070 con 0x7fe7280afca0 2023-11-26T07:18:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe724ff9640 1 -- 172.21.15.31:0/4157744373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7200039a0 con 0x7fe7280afca0 2023-11-26T07:18:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe724ff9640 1 -- 172.21.15.31:0/4157744373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720003b30 con 0x7fe7280afca0 2023-11-26T07:18:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72e279640 1 -- 172.21.15.31:0/4157744373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72815d7f0 con 0x7fe7280afca0 2023-11-26T07:18:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72e279640 1 -- 172.21.15.31:0/4157744373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7281f73e0 con 0x7fe7280afca0 2023-11-26T07:18:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe72e279640 1 -- 172.21.15.31:0/4157744373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7280671f0 con 0x7fe7280afca0 2023-11-26T07:18:47.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.491+0000 7fe724ff9640 1 -- 172.21.15.31:0/4157744373 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fe7200b4040 con 0x7fe7280afca0 2023-11-26T07:18:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.495+0000 7fe724ff9640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe714066650 0x7fe714068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.495+0000 7fe724ff9640 1 -- 172.21.15.31:0/4157744373 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe72011dc70 con 0x7fe7280afca0 2023-11-26T07:18:47.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.495+0000 7fe727fff640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe714066650 0x7fe714068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.495+0000 7fe727fff640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe714066650 0x7fe714068b10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe718000c10 tx=0x7fe718000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.499+0000 7fe724ff9640 1 -- 172.21.15.31:0/4157744373 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe7200eafd0 con 0x7fe7280afca0 2023-11-26T07:18:47.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.655+0000 7fe72e279640 1 -- 172.21.15.31:0/4157744373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe72806b460 con 0x7fe7280afca0 2023-11-26T07:18:47.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.655+0000 7fe724ff9640 1 -- 172.21.15.31:0/4157744373 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7200b20a0 con 0x7fe7280afca0 2023-11-26T07:18:47.662 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:18:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.655+0000 7fe72e279640 1 -- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe714066650 msgr2=0x7fe714068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe714066650 0x7fe714068b10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe718000c10 tx=0x7fe718000ee0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 -- 172.21.15.31:0/4157744373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7280afca0 msgr2=0x7fe7280b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7280afca0 0x7fe7280b0120 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe72000e170 tx=0x7fe720009750 comp rx=0 tx=0).stop 2023-11-26T07:18:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 -- 172.21.15.31:0/4157744373 shutdown_connections 2023-11-26T07:18:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe714066650 0x7fe714068b10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7280afca0 0x7fe7280b0120 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72815cd00 0x7fe7281f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 --2- 172.21.15.31:0/4157744373 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe72814da90 0x7fe728150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 -- 172.21.15.31:0/4157744373 >> 172.21.15.31:0/4157744373 conn(0x7fe7280b0cb0 msgr2=0x7fe7280734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 -- 172.21.15.31:0/4157744373 shutdown_connections 2023-11-26T07:18:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.659+0000 7fe72e279640 1 -- 172.21.15.31:0/4157744373 wait complete. 2023-11-26T07:18:47.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f269e165640 1 Processor -- start 2023-11-26T07:18:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f269e165640 1 -- start start 2023-11-26T07:18:47.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f269e165640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f269815d7b0 0x7f26980d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f269e165640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26980cefd0 con 0x7f26980d2080 2023-11-26T07:18:47.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f269e165640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26980cf2b0 con 0x7f26980cead0 2023-11-26T07:18:47.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f269e165640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269806b3f0 con 0x7f269815d7b0 2023-11-26T07:18:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f2697fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f26980d2080 0x7f269806bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47578/0 (socket says 172.21.15.31:47578) 2023-11-26T07:18:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f2697fff640 1 -- 172.21.15.31:0/1541788959 learned_addr learned my addr 172.21.15.31:0/1541788959 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f26977fe640 1 --2- 172.21.15.31:0/1541788959 >> v2:172.21.15.97:3300/0 conn(0x7f269815d7b0 0x7f26980d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f26967fc640 1 -- 172.21.15.31:0/1541788959 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143865943 0 0) 0x7f26980cefd0 con 0x7f26980d2080 2023-11-26T07:18:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f26967fc640 1 -- 172.21.15.31:0/1541788959 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2680003530 con 0x7f26980d2080 2023-11-26T07:18:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f26977fe640 1 -- 172.21.15.31:0/1541788959 >> v1:172.21.15.97:6789/0 conn(0x7f26980cead0 legacy=0x7f26980d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f26977fe640 1 -- 172.21.15.31:0/1541788959 >> v1:172.21.15.31:6789/0 conn(0x7f26980d2080 legacy=0x7f269806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.823+0000 7f26977fe640 1 -- 172.21.15.31:0/1541788959 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269806b6d0 con 0x7f269815d7b0 2023-11-26T07:18:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f26977fe640 1 --2- 172.21.15.31:0/1541788959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269815d7b0 0x7f26980d10d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f26880097c0 tx=0x7f268801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2912ad1e6e29b0f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f26967fc640 1 -- 172.21.15.31:0/1541788959 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688004230 con 0x7f269815d7b0 2023-11-26T07:18:47.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f26967fc640 1 -- 172.21.15.31:0/1541788959 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26880a7050 con 0x7f269815d7b0 2023-11-26T07:18:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f26967fc640 1 -- 172.21.15.31:0/1541788959 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688003620 con 0x7f269815d7b0 2023-11-26T07:18:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f269e165640 1 -- 172.21.15.31:0/1541788959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269815d7b0 msgr2=0x7f26980d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f269e165640 1 --2- 172.21.15.31:0/1541788959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269815d7b0 0x7f26980d10d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f26880097c0 tx=0x7f268801c330 comp rx=0 tx=0).stop 2023-11-26T07:18:47.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f269e165640 1 -- 172.21.15.31:0/1541788959 shutdown_connections 2023-11-26T07:18:47.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f269e165640 1 --2- 172.21.15.31:0/1541788959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269815d7b0 0x7f26980d10d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f269e165640 1 -- 172.21.15.31:0/1541788959 >> 172.21.15.31:0/1541788959 conn(0x7f26980d33b0 msgr2=0x7f26980d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:47.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f269e165640 1 -- 172.21.15.31:0/1541788959 shutdown_connections 2023-11-26T07:18:47.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.827+0000 7f269e165640 1 -- 172.21.15.31:0/1541788959 wait complete. 2023-11-26T07:18:47.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f269e165640 1 Processor -- start 2023-11-26T07:18:47.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f269e165640 1 -- start start 2023-11-26T07:18:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f269e165640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26980cead0 0x7f26980c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f269e165640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26980d2080 0x7f26980c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f269e165640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269815d7b0 0x7f2698151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f269e165640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26980ca920 con 0x7f26980cead0 2023-11-26T07:18:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f269e165640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26980b7eb0 con 0x7f269815d7b0 2023-11-26T07:18:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f269e165640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26980b9190 con 0x7f26980d2080 2023-11-26T07:18:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f26977fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26980cead0 0x7f26980c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f26977fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26980cead0 0x7f26980c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56058/0 (socket says 172.21.15.31:56058) 2023-11-26T07:18:47.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f26977fe640 1 -- 172.21.15.31:0/3410865346 learned_addr learned my addr 172.21.15.31:0/3410865346 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:47.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f2696ffd640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26980d2080 0x7f26980c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f2697fff640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269815d7b0 0x7f2698151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f26977fe640 1 -- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26980d2080 msgr2=0x7f26980c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f26977fe640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26980d2080 0x7f26980c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f26977fe640 1 -- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269815d7b0 msgr2=0x7f2698151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:47.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f26977fe640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269815d7b0 0x7f2698151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:47.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f26977fe640 1 -- 172.21.15.31:0/3410865346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2688009420 con 0x7f26980cead0 2023-11-26T07:18:47.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f2697fff640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269815d7b0 0x7f2698151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:47.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f26977fe640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26980cead0 0x7f26980c9750 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2688018f40 tx=0x7f26880190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f267ffff640 1 -- 172.21.15.31:0/3410865346 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688003b70 con 0x7f26980cead0 2023-11-26T07:18:47.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f267ffff640 1 -- 172.21.15.31:0/3410865346 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2688003cd0 con 0x7f26980cead0 2023-11-26T07:18:47.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f267ffff640 1 -- 172.21.15.31:0/3410865346 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26880ab660 con 0x7f26980cead0 2023-11-26T07:18:47.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.831+0000 7f269e165640 1 -- 172.21.15.31:0/3410865346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26980cc310 con 0x7f26980cead0 2023-11-26T07:18:47.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.835+0000 7f269e165640 1 -- 172.21.15.31:0/3410865346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26980c9c90 con 0x7f26980cead0 2023-11-26T07:18:47.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.835+0000 7f269e165640 1 -- 172.21.15.31:0/3410865346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269804e850 con 0x7f26980cead0 2023-11-26T07:18:47.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.835+0000 7f267ffff640 1 -- 172.21.15.31:0/3410865346 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f26880b2050 con 0x7f26980cead0 2023-11-26T07:18:47.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.835+0000 7f267ffff640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2684066620 0x7f2684068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:47.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.835+0000 7f267ffff640 1 -- 172.21.15.31:0/3410865346 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f268811e520 con 0x7f26980cead0 2023-11-26T07:18:47.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.835+0000 7f2696ffd640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2684066620 0x7f2684068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:47.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.839+0000 7f2696ffd640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2684066620 0x7f2684068ae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f26980b2430 tx=0x7f268c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:47.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:47.839+0000 7f267ffff640 1 -- 172.21.15.31:0/3410865346 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f26880eb620 con 0x7f26980cead0 2023-11-26T07:18:48.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.027+0000 7f269e165640 1 -- 172.21.15.31:0/3410865346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26980cc0f0 con 0x7f26980cead0 2023-11-26T07:18:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.027+0000 7f267ffff640 1 -- 172.21.15.31:0/3410865346 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f26880eec40 con 0x7f26980cead0 2023-11-26T07:18:48.035 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:48.035 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3937918726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3937918726},{"type":"v1","addr":"172.21.15.97:6837","nonce":3937918726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2083665480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2083665480},{"type":"v1","addr":"172.21.15.97:6835","nonce":2083665480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1187139549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187139549},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187139549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:24.963367+0000","modified":"2023-11-26T07:18:27.980260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4036387633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4036387633},{"type":"v1","addr":"172.21.15.31:6835","nonce":4036387633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:18:48.035 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2023-11-26T07:18:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 -- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2684066620 msgr2=0x7f2684068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2684066620 0x7f2684068ae0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f26980b2430 tx=0x7f268c009040 comp rx=0 tx=0).stop 2023-11-26T07:18:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 -- 172.21.15.31:0/3410865346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26980cead0 msgr2=0x7f26980c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26980cead0 0x7f26980c9750 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2688018f40 tx=0x7f26880190d0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 -- 172.21.15.31:0/3410865346 shutdown_connections 2023-11-26T07:18:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2684066620 0x7f2684068ae0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f269815d7b0 0x7f2698151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f26980d2080 0x7f26980c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 --2- 172.21.15.31:0/3410865346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26980cead0 0x7f26980c9750 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 -- 172.21.15.31:0/3410865346 >> 172.21.15.31:0/3410865346 conn(0x7f26980d33b0 msgr2=0x7f26980b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 -- 172.21.15.31:0/3410865346 shutdown_connections 2023-11-26T07:18:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.031+0000 7f269e165640 1 -- 172.21.15.31:0/3410865346 wait complete. 2023-11-26T07:18:48.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:48.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.195+0000 7fac6d9aa640 1 Processor -- start 2023-11-26T07:18:48.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.195+0000 7fac6d9aa640 1 -- start start 2023-11-26T07:18:48.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.195+0000 7fac6d9aa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac680cfdb0 0x7fac680d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.195+0000 7fac6d9aa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fac681654c0 0x7fac681678b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.195+0000 7fac6d9aa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac680d12e0 con 0x7fac6815d530 2023-11-26T07:18:48.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.195+0000 7fac6d9aa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac680d0760 con 0x7fac680cfdb0 2023-11-26T07:18:48.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.195+0000 7fac6d9aa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac680d15c0 con 0x7fac681654c0 2023-11-26T07:18:48.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.195+0000 7fac5ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac680cfdb0 0x7fac680d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.195+0000 7fac5ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fac680cfdb0 0x7fac680d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56068/0 (socket says 172.21.15.31:56068) 2023-11-26T07:18:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.195+0000 7fac5ffff640 1 -- 172.21.15.31:0/3031636349 learned_addr learned my addr 172.21.15.31:0/3031636349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.199+0000 7fac677fe640 1 --2- 172.21.15.31:0/3031636349 >> v2:172.21.15.97:3300/0 conn(0x7fac681654c0 0x7fac681678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.199+0000 7fac5ffff640 1 -- 172.21.15.31:0/3031636349 >> v2:172.21.15.97:3300/0 conn(0x7fac681654c0 msgr2=0x7fac681678b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.199+0000 7fac5ffff640 1 --2- 172.21.15.31:0/3031636349 >> v2:172.21.15.97:3300/0 conn(0x7fac681654c0 0x7fac681678b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.199+0000 7fac5ffff640 1 -- 172.21.15.31:0/3031636349 >> v1:172.21.15.97:6789/0 conn(0x7fac6815d530 legacy=0x7fac680cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.199+0000 7fac5ffff640 1 -- 172.21.15.31:0/3031636349 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac680d18a0 con 0x7fac680cfdb0 2023-11-26T07:18:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.199+0000 7fac677fe640 1 --2- 172.21.15.31:0/3031636349 >> v2:172.21.15.97:3300/0 conn(0x7fac681654c0 0x7fac681678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.199+0000 7fac5ffff640 1 --2- 172.21.15.31:0/3031636349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac680cfdb0 0x7fac680d0220 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fac50009870 tx=0x7fac5001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91eed729675a9cc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.199+0000 7fac667fc640 1 -- 172.21.15.31:0/3031636349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500a89f0 con 0x7fac680cfdb0 2023-11-26T07:18:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.199+0000 7fac667fc640 1 -- 172.21.15.31:0/3031636349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac500a8b50 con 0x7fac680cfdb0 2023-11-26T07:18:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.199+0000 7fac667fc640 1 -- 172.21.15.31:0/3031636349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500b0440 con 0x7fac680cfdb0 2023-11-26T07:18:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3031636349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac680cfdb0 msgr2=0x7fac680d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 --2- 172.21.15.31:0/3031636349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac680cfdb0 0x7fac680d0220 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fac50009870 tx=0x7fac5001ecf0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3031636349 shutdown_connections 2023-11-26T07:18:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 --2- 172.21.15.31:0/3031636349 >> v2:172.21.15.97:3300/0 conn(0x7fac681654c0 0x7fac681678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 --2- 172.21.15.31:0/3031636349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac680cfdb0 0x7fac680d0220 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3031636349 >> 172.21.15.31:0/3031636349 conn(0x7fac6809c6d0 msgr2=0x7fac680d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3031636349 shutdown_connections 2023-11-26T07:18:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3031636349 wait complete. 2023-11-26T07:18:48.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 Processor -- start 2023-11-26T07:18:48.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 -- start start 2023-11-26T07:18:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680cfdb0 0x7fac6815a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac6815d530 0x7fac6815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac681654c0 0x7fac6815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac68147460 con 0x7fac681654c0 2023-11-26T07:18:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac681445d0 con 0x7fac680cfdb0 2023-11-26T07:18:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac6d9aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac68147880 con 0x7fac6815d530 2023-11-26T07:18:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac66ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680cfdb0 0x7fac6815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac677fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac681654c0 0x7fac6815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac6815d530 0x7fac6815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac66ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680cfdb0 0x7fac6815a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:34942/0 (socket says 172.21.15.31:34942) 2023-11-26T07:18:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac5ffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac6815d530 0x7fac6815eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:56778/0 (socket says 172.21.15.31:56778) 2023-11-26T07:18:48.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.203+0000 7fac66ffd640 1 -- 172.21.15.31:0/3764771956 learned_addr learned my addr 172.21.15.31:0/3764771956 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac677fe640 1 -- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac6815d530 msgr2=0x7fac6815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac677fe640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac6815d530 0x7fac6815eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac677fe640 1 -- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680cfdb0 msgr2=0x7fac6815a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac677fe640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680cfdb0 0x7fac6815a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac677fe640 1 -- 172.21.15.31:0/3764771956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac500094d0 con 0x7fac681654c0 2023-11-26T07:18:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac677fe640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac681654c0 0x7fac6815f0c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fac6000cdc0 tx=0x7fac6000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac64ff9640 1 -- 172.21.15.31:0/3764771956 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6000d840 con 0x7fac681654c0 2023-11-26T07:18:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac64ff9640 1 -- 172.21.15.31:0/3764771956 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6000d9e0 con 0x7fac681654c0 2023-11-26T07:18:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac64ff9640 1 -- 172.21.15.31:0/3764771956 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6000dd90 con 0x7fac681654c0 2023-11-26T07:18:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3764771956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac68146ac0 con 0x7fac681654c0 2023-11-26T07:18:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3764771956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac68147080 con 0x7fac681654c0 2023-11-26T07:18:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3764771956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac681475a0 con 0x7fac681654c0 2023-11-26T07:18:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.207+0000 7fac64ff9640 1 -- 172.21.15.31:0/3764771956 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fac60015440 con 0x7fac681654c0 2023-11-26T07:18:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.211+0000 7fac64ff9640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac48066620 0x7fac48068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.211+0000 7fac64ff9640 1 -- 172.21.15.31:0/3764771956 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac60084920 con 0x7fac681654c0 2023-11-26T07:18:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.211+0000 7fac66ffd640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac48066620 0x7fac48068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.211+0000 7fac66ffd640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac48066620 0x7fac48068ae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac580018b0 tx=0x7fac58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.215+0000 7fac64ff9640 1 -- 172.21.15.31:0/3764771956 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fac681475a0 con 0x7fac681654c0 2023-11-26T07:18:48.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3764771956 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac681475a0 con 0x7fac681654c0 2023-11-26T07:18:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac64ff9640 1 -- 172.21.15.31:0/3764771956 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16485 (secure 0 0 0) 0x7fac681475a0 con 0x7fac681654c0 2023-11-26T07:18:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:48.381 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:26.567047+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:22.773068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:23.967477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:48.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac48066620 msgr2=0x7fac48068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac48066620 0x7fac48068ae0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fac580018b0 tx=0x7fac58009040 comp rx=0 tx=0).stop 2023-11-26T07:18:48.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3764771956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac681654c0 msgr2=0x7fac6815f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac681654c0 0x7fac6815f0c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fac6000cdc0 tx=0x7fac6000a710 comp rx=0 tx=0).stop 2023-11-26T07:18:48.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3764771956 shutdown_connections 2023-11-26T07:18:48.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac48066620 0x7fac48068ae0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac681654c0 0x7fac6815f0c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac6815d530 0x7fac6815eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 --2- 172.21.15.31:0/3764771956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac680cfdb0 0x7fac6815a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3764771956 >> 172.21.15.31:0/3764771956 conn(0x7fac6809c6d0 msgr2=0x7fac680b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3764771956 shutdown_connections 2023-11-26T07:18:48.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.375+0000 7fac6d9aa640 1 -- 172.21.15.31:0/3764771956 wait complete. 2023-11-26T07:18:48.403 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:18:48.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:48.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.539+0000 7ff8d17d0640 1 Processor -- start 2023-11-26T07:18:48.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.539+0000 7ff8d17d0640 1 -- start start 2023-11-26T07:18:48.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.539+0000 7ff8d17d0640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8cc072100 0x7ff8cc15e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.539+0000 7ff8d17d0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8cc0724e0 0x7ff8cc0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.539+0000 7ff8d17d0640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8cc15ea40 con 0x7ff8cc0ce1b0 2023-11-26T07:18:48.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.539+0000 7ff8d17d0640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8cc15b440 con 0x7ff8cc0724e0 2023-11-26T07:18:48.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.539+0000 7ff8d17d0640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc13d3f0 con 0x7ff8cc072100 2023-11-26T07:18:48.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8cb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8cc0724e0 0x7ff8cc0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8cb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8cc0724e0 0x7ff8cc0cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56094/0 (socket says 172.21.15.31:56094) 2023-11-26T07:18:48.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8cb7fe640 1 -- 172.21.15.31:0/76657458 learned_addr learned my addr 172.21.15.31:0/76657458 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:48.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8cbfff640 1 --2- 172.21.15.31:0/76657458 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8cc072100 0x7ff8cc15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8cb7fe640 1 -- 172.21.15.31:0/76657458 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8cc072100 msgr2=0x7ff8cc15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8cb7fe640 1 --2- 172.21.15.31:0/76657458 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8cc072100 0x7ff8cc15e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8cb7fe640 1 -- 172.21.15.31:0/76657458 >> v1:172.21.15.97:6789/0 conn(0x7ff8cc0ce1b0 legacy=0x7ff8cc0ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8cb7fe640 1 -- 172.21.15.31:0/76657458 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc15a7c0 con 0x7ff8cc0724e0 2023-11-26T07:18:48.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8cb7fe640 1 --2- 172.21.15.31:0/76657458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 0x7ff8cc0cdc70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000bb40 tx=0x7ff8c001eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab915a947960e99a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8caffd640 1 -- 172.21.15.31:0/76657458 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00a7a80 con 0x7ff8cc0724e0 2023-11-26T07:18:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8caffd640 1 -- 172.21.15.31:0/76657458 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c00a7be0 con 0x7ff8cc0724e0 2023-11-26T07:18:48.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.543+0000 7ff8caffd640 1 -- 172.21.15.31:0/76657458 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00af440 con 0x7ff8cc0724e0 2023-11-26T07:18:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 -- 172.21.15.31:0/76657458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 msgr2=0x7ff8cc0cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 --2- 172.21.15.31:0/76657458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 0x7ff8cc0cdc70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000bb40 tx=0x7ff8c001eaf0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 -- 172.21.15.31:0/76657458 shutdown_connections 2023-11-26T07:18:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 --2- 172.21.15.31:0/76657458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 0x7ff8cc0cdc70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 --2- 172.21.15.31:0/76657458 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8cc072100 0x7ff8cc15e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 -- 172.21.15.31:0/76657458 >> 172.21.15.31:0/76657458 conn(0x7ff8cc0b0e00 msgr2=0x7ff8cc0b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 -- 172.21.15.31:0/76657458 shutdown_connections 2023-11-26T07:18:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 -- 172.21.15.31:0/76657458 wait complete. 2023-11-26T07:18:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 Processor -- start 2023-11-26T07:18:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 -- start start 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc072100 0x7ff8cc13ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 0x7ff8cc0b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8cc0ce1b0 0x7ff8cc1fa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc13cf40 con 0x7ff8cc0724e0 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0b0930 con 0x7ff8cc072100 2023-11-26T07:18:48.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d17d0640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0b06b0 con 0x7ff8cc0ce1b0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8cb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 0x7ff8cc0b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8cb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 0x7ff8cc0b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56110/0 (socket says 172.21.15.31:56110) 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8cb7fe640 1 -- 172.21.15.31:0/1015149067 learned_addr learned my addr 172.21.15.31:0/1015149067 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8cbfff640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc072100 0x7ff8cc13ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8d0fcf640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8cc0ce1b0 0x7ff8cc1fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8cb7fe640 1 -- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8cc0ce1b0 msgr2=0x7ff8cc1fa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8cb7fe640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8cc0ce1b0 0x7ff8cc1fa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8cb7fe640 1 -- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc072100 msgr2=0x7ff8cc13ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8cb7fe640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc072100 0x7ff8cc13ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.547+0000 7ff8cb7fe640 1 -- 172.21.15.31:0/1015149067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c000b770 con 0x7ff8cc0724e0 2023-11-26T07:18:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8cb7fe640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 0x7ff8cc0b0170 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b2c0 tx=0x7ff8c001b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8c8ff9640 1 -- 172.21.15.31:0/1015149067 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00ba660 con 0x7ff8cc0724e0 2023-11-26T07:18:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8c8ff9640 1 -- 172.21.15.31:0/1015149067 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c00ba800 con 0x7ff8cc0724e0 2023-11-26T07:18:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8d17d0640 1 -- 172.21.15.31:0/1015149067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc0cee90 con 0x7ff8cc0724e0 2023-11-26T07:18:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8c8ff9640 1 -- 172.21.15.31:0/1015149067 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00babb0 con 0x7ff8cc0724e0 2023-11-26T07:18:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8d17d0640 1 -- 172.21.15.31:0/1015149067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc0b2ce0 con 0x7ff8cc0724e0 2023-11-26T07:18:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8d17d0640 1 -- 172.21.15.31:0/1015149067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc074a00 con 0x7ff8cc0724e0 2023-11-26T07:18:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8c8ff9640 1 -- 172.21.15.31:0/1015149067 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff8c00a7d50 con 0x7ff8cc0724e0 2023-11-26T07:18:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8c8ff9640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff8b8066620 0x7ff8b8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8c8ff9640 1 -- 172.21.15.31:0/1015149067 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8c011ef50 con 0x7ff8cc0724e0 2023-11-26T07:18:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.551+0000 7ff8cbfff640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff8b8066620 0x7ff8b8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.555+0000 7ff8cbfff640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff8b8066620 0x7ff8b8068ae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00a970 tx=0x7ff8bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.555+0000 7ff8c8ff9640 1 -- 172.21.15.31:0/1015149067 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff8c00ec140 con 0x7ff8cc0724e0 2023-11-26T07:18:48.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 -- 172.21.15.31:0/1015149067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8cc070500 con 0x7ff8cc0724e0 2023-11-26T07:18:48.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8c8ff9640 1 -- 172.21.15.31:0/1015149067 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7ff8c00bf020 con 0x7ff8cc0724e0 2023-11-26T07:18:48.754 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:48.754 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3937918726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3937918726},{"type":"v1","addr":"172.21.15.97:6837","nonce":3937918726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2083665480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2083665480},{"type":"v1","addr":"172.21.15.97:6835","nonce":2083665480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1187139549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187139549},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187139549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:24.963367+0000","modified":"2023-11-26T07:18:27.980260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4036387633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4036387633},{"type":"v1","addr":"172.21.15.31:6835","nonce":4036387633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:18:48.754 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2023-11-26T07:18:48.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 -- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff8b8066620 msgr2=0x7ff8b8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff8b8066620 0x7ff8b8068ae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00a970 tx=0x7ff8bc008040 comp rx=0 tx=0).stop 2023-11-26T07:18:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 -- 172.21.15.31:0/1015149067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 msgr2=0x7ff8cc0b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 0x7ff8cc0b0170 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000b2c0 tx=0x7ff8c001b890 comp rx=0 tx=0).stop 2023-11-26T07:18:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 -- 172.21.15.31:0/1015149067 shutdown_connections 2023-11-26T07:18:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff8b8066620 0x7ff8b8068ae0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8cc0ce1b0 0x7ff8cc1fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8cc0724e0 0x7ff8cc0b0170 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 --2- 172.21.15.31:0/1015149067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8cc072100 0x7ff8cc13ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 -- 172.21.15.31:0/1015149067 >> 172.21.15.31:0/1015149067 conn(0x7ff8cc0b0e00 msgr2=0x7ff8cc0ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.747+0000 7ff8d17d0640 1 -- 172.21.15.31:0/1015149067 shutdown_connections 2023-11-26T07:18:48.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.751+0000 7ff8d17d0640 1 -- 172.21.15.31:0/1015149067 wait complete. 2023-11-26T07:18:48.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:48.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.911+0000 7fa654d01640 1 Processor -- start 2023-11-26T07:18:48.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.911+0000 7fa654d01640 1 -- start start 2023-11-26T07:18:48.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.911+0000 7fa654d01640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6500715b0 0x7fa650071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.911+0000 7fa654d01640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65006cf50 con 0x7fa65015d510 2023-11-26T07:18:48.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.911+0000 7fa654d01640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65006cc20 con 0x7fa6500b3cc0 2023-11-26T07:18:48.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.911+0000 7fa654d01640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65015d8f0 con 0x7fa6500715b0 2023-11-26T07:18:48.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.911+0000 7fa64e59c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa65015d510 0x7fa6500b29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47580/0 (socket says 172.21.15.31:47580) 2023-11-26T07:18:48.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.911+0000 7fa64e59c640 1 -- 172.21.15.31:0/2198459392 learned_addr learned my addr 172.21.15.31:0/2198459392 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.911+0000 7fa64ed9d640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6500715b0 0x7fa650071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860428914 0 0) 0x7fa65006cf50 con 0x7fa65015d510 2023-11-26T07:18:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa63c003530 con 0x7fa65015d510 2023-11-26T07:18:48.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558696873 0 0) 0x7fa65006cc20 con 0x7fa6500b3cc0 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa65006cf50 con 0x7fa6500b3cc0 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3559768988 0 0) 0x7fa63c003530 con 0x7fa65015d510 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6500715b0 msgr2=0x7fa650071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6500715b0 0x7fa650071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 >> v1:172.21.15.97:6789/0 conn(0x7fa6500b3cc0 legacy=0x7fa65006d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500b31d0 con 0x7fa65015d510 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fa640001680 con 0x7fa65015d510 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 >> v1:172.21.15.31:6789/0 conn(0x7fa65015d510 legacy=0x7fa6500b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c003b40 0x7fa63c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0047d0 0x7fa63c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6500715b0 0x7fa63c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65015d8f0 con 0x7fa63c0047d0 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c00ba40 con 0x7fa63c003b40 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c00bc00 con 0x7fa6500715b0 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa647fff640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0047d0 0x7fa63c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64e59c640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c003b40 0x7fa63c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64ed9d640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6500715b0 0x7fa63c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa647fff640 1 -- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6500715b0 msgr2=0x7fa63c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa647fff640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6500715b0 0x7fa63c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa647fff640 1 -- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c003b40 msgr2=0x7fa63c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa647fff640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c003b40 0x7fa63c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa647fff640 1 -- 172.21.15.31:0/2198459392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c003530 con 0x7fa63c0047d0 2023-11-26T07:18:48.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.915+0000 7fa64ed9d640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6500715b0 0x7fa63c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa647fff640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0047d0 0x7fa63c004c50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa6500b4170 tx=0x7fa63801ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec07ae362865175 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380052d0 con 0x7fa63c0047d0 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa638005450 con 0x7fa63c0047d0 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa64dd9b640 1 -- 172.21.15.31:0/2198459392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380a8910 con 0x7fa63c0047d0 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 -- 172.21.15.31:0/2198459392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0047d0 msgr2=0x7fa63c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 --2- 172.21.15.31:0/2198459392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa63c0047d0 0x7fa63c004c50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa6500b4170 tx=0x7fa63801ecb0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa64ed9d640 1 -- 172.21.15.31:0/2198459392 reap_dead start 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 -- 172.21.15.31:0/2198459392 shutdown_connections 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 -- 172.21.15.31:0/2198459392 >> 172.21.15.31:0/2198459392 conn(0x7fa6500b1e40 msgr2=0x7fa650070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 -- 172.21.15.31:0/2198459392 shutdown_connections 2023-11-26T07:18:48.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 -- 172.21.15.31:0/2198459392 wait complete. 2023-11-26T07:18:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 Processor -- start 2023-11-26T07:18:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 -- start start 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c0047d0 0x7fa65015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6500ce590 0x7fa6501491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa65015bc40 0x7fa65015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650071fa0 con 0x7fa6500ce590 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65015c600 con 0x7fa63c0047d0 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.919+0000 7fa654d01640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65015a6e0 con 0x7fa65015bc40 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa647fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6500ce590 0x7fa6501491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa647fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6500ce590 0x7fa6501491d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56126/0 (socket says 172.21.15.31:56126) 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa647fff640 1 -- 172.21.15.31:0/2009227794 learned_addr learned my addr 172.21.15.31:0/2009227794 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa64e59c640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c0047d0 0x7fa65015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa64ed9d640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa65015bc40 0x7fa65015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa647fff640 1 -- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa65015bc40 msgr2=0x7fa65015c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa647fff640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa65015bc40 0x7fa65015c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa647fff640 1 -- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c0047d0 msgr2=0x7fa65015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa647fff640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c0047d0 0x7fa65015a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa647fff640 1 -- 172.21.15.31:0/2009227794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65015a9c0 con 0x7fa6500ce590 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa647fff640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6500ce590 0x7fa6501491d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa63801eae0 tx=0x7fa63c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa646ffd640 1 -- 172.21.15.31:0/2009227794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380a8d70 con 0x7fa6500ce590 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa646ffd640 1 -- 172.21.15.31:0/2009227794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6380050d0 con 0x7fa6500ce590 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa646ffd640 1 -- 172.21.15.31:0/2009227794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638003600 con 0x7fa6500ce590 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa654d01640 1 -- 172.21.15.31:0/2009227794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65015aca0 con 0x7fa6500ce590 2023-11-26T07:18:48.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa654d01640 1 -- 172.21.15.31:0/2009227794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65015b260 con 0x7fa6500ce590 2023-11-26T07:18:48.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa654d01640 1 -- 172.21.15.31:0/2009227794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65004e850 con 0x7fa6500ce590 2023-11-26T07:18:48.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.923+0000 7fa646ffd640 1 -- 172.21.15.31:0/2009227794 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fa638003780 con 0x7fa6500ce590 2023-11-26T07:18:48.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.927+0000 7fa646ffd640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa630066650 0x7fa630068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:48.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.927+0000 7fa646ffd640 1 -- 172.21.15.31:0/2009227794 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa638121100 con 0x7fa6500ce590 2023-11-26T07:18:48.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.927+0000 7fa64e59c640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa630066650 0x7fa630068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:48.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.927+0000 7fa64e59c640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa630066650 0x7fa630068b10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa640001ec0 tx=0x7fa640009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:48.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:48.931+0000 7fa646ffd640 1 -- 172.21.15.31:0/2009227794 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6380ee1d0 con 0x7fa6500ce590 2023-11-26T07:18:49.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.119+0000 7fa654d01640 1 -- 172.21.15.31:0/2009227794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa65004e850 con 0x7fa6500ce590 2023-11-26T07:18:49.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.119+0000 7fa646ffd640 1 -- 172.21.15.31:0/2009227794 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fa6380c0090 con 0x7fa6500ce590 2023-11-26T07:18:49.127 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:49.127 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3937918726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3937918726},{"type":"v1","addr":"172.21.15.97:6837","nonce":3937918726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2083665480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2083665480},{"type":"v1","addr":"172.21.15.97:6835","nonce":2083665480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1187139549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187139549},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187139549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:18:24.963367+0000","modified":"2023-11-26T07:18:27.980260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/4036387633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4036387633},{"type":"v1","addr":"172.21.15.31:6835","nonce":4036387633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-26T07:18:49.127 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2023-11-26T07:18:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 -- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa630066650 msgr2=0x7fa630068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa630066650 0x7fa630068b10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa640001ec0 tx=0x7fa640009040 comp rx=0 tx=0).stop 2023-11-26T07:18:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 -- 172.21.15.31:0/2009227794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6500ce590 msgr2=0x7fa6501491d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6500ce590 0x7fa6501491d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa63801eae0 tx=0x7fa63c005800 comp rx=0 tx=0).stop 2023-11-26T07:18:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 -- 172.21.15.31:0/2009227794 shutdown_connections 2023-11-26T07:18:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa630066650 0x7fa630068b10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa65015bc40 0x7fa65015c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6500ce590 0x7fa6501491d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 --2- 172.21.15.31:0/2009227794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa63c0047d0 0x7fa65015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 -- 172.21.15.31:0/2009227794 >> 172.21.15.31:0/2009227794 conn(0x7fa6500b1e40 msgr2=0x7fa6500afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 -- 172.21.15.31:0/2009227794 shutdown_connections 2023-11-26T07:18:49.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.123+0000 7fa654d01640 1 -- 172.21.15.31:0/2009227794 wait complete. 2023-11-26T07:18:49.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.283+0000 7f885893e640 1 Processor -- start 2023-11-26T07:18:49.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.283+0000 7f885893e640 1 -- start start 2023-11-26T07:18:49.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f885893e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f885415d7b0 0x7f88540d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f885893e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f88540d2080 0x7f885406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f885893e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88540cefd0 con 0x7f88540cead0 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f885893e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88540cf2b0 con 0x7f88540d2080 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f885893e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885406b3f0 con 0x7f885415d7b0 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f8852d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f88540d2080 0x7f885406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f8852d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f88540d2080 0x7f885406bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56138/0 (socket says 172.21.15.31:56138) 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f8852d9d640 1 -- 172.21.15.31:0/2255173799 learned_addr learned my addr 172.21.15.31:0/2255173799 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f885259c640 1 --2- 172.21.15.31:0/2255173799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f885415d7b0 0x7f88540d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f8852d9d640 1 -- 172.21.15.31:0/2255173799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f885415d7b0 msgr2=0x7f88540d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f8852d9d640 1 --2- 172.21.15.31:0/2255173799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f885415d7b0 0x7f88540d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f8852d9d640 1 -- 172.21.15.31:0/2255173799 >> v1:172.21.15.97:6789/0 conn(0x7f88540cead0 legacy=0x7f88540d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.287+0000 7f8852d9d640 1 -- 172.21.15.31:0/2255173799 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885406b6d0 con 0x7f88540d2080 2023-11-26T07:18:49.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f8852d9d640 1 --2- 172.21.15.31:0/2255173799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88540d2080 0x7f885406bed0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f884c009e50 tx=0x7f884c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1310f56d30bbdb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f8851d9b640 1 -- 172.21.15.31:0/2255173799 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0a57a0 con 0x7f88540d2080 2023-11-26T07:18:49.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f8851d9b640 1 -- 172.21.15.31:0/2255173799 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c0a5900 con 0x7f88540d2080 2023-11-26T07:18:49.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f8851d9b640 1 -- 172.21.15.31:0/2255173799 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0a5c90 con 0x7f88540d2080 2023-11-26T07:18:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 -- 172.21.15.31:0/2255173799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88540d2080 msgr2=0x7f885406bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 --2- 172.21.15.31:0/2255173799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88540d2080 0x7f885406bed0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f884c009e50 tx=0x7f884c01c8c0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 -- 172.21.15.31:0/2255173799 shutdown_connections 2023-11-26T07:18:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 --2- 172.21.15.31:0/2255173799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88540d2080 0x7f885406bed0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 --2- 172.21.15.31:0/2255173799 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f885415d7b0 0x7f88540d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 -- 172.21.15.31:0/2255173799 >> 172.21.15.31:0/2255173799 conn(0x7f88540d33b0 msgr2=0x7f88540d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 -- 172.21.15.31:0/2255173799 shutdown_connections 2023-11-26T07:18:49.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 -- 172.21.15.31:0/2255173799 wait complete. 2023-11-26T07:18:49.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 Processor -- start 2023-11-26T07:18:49.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 -- start start 2023-11-26T07:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88540cead0 0x7f88540b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88540d2080 0x7f88541569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885415d7b0 0x7f88540ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.291+0000 7f885893e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540c89c0 con 0x7f88540d2080 2023-11-26T07:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f885893e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88540b6e80 con 0x7f885415d7b0 2023-11-26T07:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f885893e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88540c8020 con 0x7f88540cead0 2023-11-26T07:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88540d2080 0x7f88541569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f8852d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885415d7b0 0x7f88540ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f885259c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88540cead0 0x7f88540b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f8852d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885415d7b0 0x7f88540ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:34970/0 (socket says 172.21.15.31:34970) 2023-11-26T07:18:49.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f8852d9d640 1 -- 172.21.15.31:0/1428317731 learned_addr learned my addr 172.21.15.31:0/1428317731 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884bfff640 1 -- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88540cead0 msgr2=0x7f88540b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884bfff640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88540cead0 0x7f88540b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884bfff640 1 -- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885415d7b0 msgr2=0x7f88540ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884bfff640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885415d7b0 0x7f88540ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884bfff640 1 -- 172.21.15.31:0/1428317731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c009b30 con 0x7f88540d2080 2023-11-26T07:18:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f8852d9d640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885415d7b0 0x7f88540ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884bfff640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88540d2080 0x7f88541569e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c3f0 tx=0x7f883c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884affd640 1 -- 172.21.15.31:0/1428317731 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0052d0 con 0x7f88540d2080 2023-11-26T07:18:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884affd640 1 -- 172.21.15.31:0/1428317731 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c005430 con 0x7f88540d2080 2023-11-26T07:18:49.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f885893e640 1 -- 172.21.15.31:0/1428317731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88540c8300 con 0x7f88540d2080 2023-11-26T07:18:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884affd640 1 -- 172.21.15.31:0/1428317731 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c007260 con 0x7f88540d2080 2023-11-26T07:18:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f885893e640 1 -- 172.21.15.31:0/1428317731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8854156f20 con 0x7f88540d2080 2023-11-26T07:18:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f885893e640 1 -- 172.21.15.31:0/1428317731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885404e850 con 0x7f88540d2080 2023-11-26T07:18:49.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.295+0000 7f884affd640 1 -- 172.21.15.31:0/1428317731 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f883c0073c0 con 0x7f88540d2080 2023-11-26T07:18:49.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.299+0000 7f884affd640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8834066650 0x7f8834068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.299+0000 7f884affd640 1 -- 172.21.15.31:0/1428317731 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f883c083360 con 0x7f88540d2080 2023-11-26T07:18:49.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.299+0000 7f885259c640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8834066650 0x7f8834068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.299+0000 7f885259c640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8834066650 0x7f8834068b10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f88440079c0 tx=0x7f8844008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.303+0000 7f884affd640 1 -- 172.21.15.31:0/1428317731 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f883c050620 con 0x7f88540d2080 2023-11-26T07:18:49.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.459+0000 7f885893e640 1 -- 172.21.15.31:0/1428317731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88540ce1b0 con 0x7f88540d2080 2023-11-26T07:18:49.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.459+0000 7f884affd640 1 -- 172.21.15.31:0/1428317731 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16485 (secure 0 0 0) 0x7f883c053c40 con 0x7f88540d2080 2023-11-26T07:18:49.468 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:49.468 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:26.567047+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:22.773068+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-26T07:18:23.967477+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 -- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8834066650 msgr2=0x7f8834068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8834066650 0x7f8834068b10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f88440079c0 tx=0x7f8844008040 comp rx=0 tx=0).stop 2023-11-26T07:18:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 -- 172.21.15.31:0/1428317731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88540d2080 msgr2=0x7f88541569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88540d2080 0x7f88541569e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c3f0 tx=0x7f883c00c8b0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 -- 172.21.15.31:0/1428317731 shutdown_connections 2023-11-26T07:18:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8834066650 0x7f8834068b10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885415d7b0 0x7f88540ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88540d2080 0x7f88541569e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 --2- 172.21.15.31:0/1428317731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f88540cead0 0x7f88540b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 -- 172.21.15.31:0/1428317731 >> 172.21.15.31:0/1428317731 conn(0x7f88540d33b0 msgr2=0x7f88540d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 -- 172.21.15.31:0/1428317731 shutdown_connections 2023-11-26T07:18:49.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.463+0000 7f885893e640 1 -- 172.21.15.31:0/1428317731 wait complete. 2023-11-26T07:18:49.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:18:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.627+0000 7fc3728ad640 1 Processor -- start 2023-11-26T07:18:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc3728ad640 1 -- start start 2023-11-26T07:18:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc3728ad640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc36c146820 0x7fc36c070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc3728ad640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc36c15eb50 0x7fc36c0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc3728ad640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36c147190 con 0x7fc36c0b4210 2023-11-26T07:18:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc3728ad640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36c0b4be0 con 0x7fc36c146820 2023-11-26T07:18:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc3728ad640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c146c00 con 0x7fc36c15eb50 2023-11-26T07:18:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc36bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc36c146820 0x7fc36c070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc36b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc36c0b4210 0x7fc36c161180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47588/0 (socket says 172.21.15.31:47588) 2023-11-26T07:18:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc36b7fe640 1 -- 172.21.15.31:0/1570406317 learned_addr learned my addr 172.21.15.31:0/1570406317 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc370e4a640 1 --2- 172.21.15.31:0/1570406317 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc36c15eb50 0x7fc36c0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc36bfff640 1 -- 172.21.15.31:0/1570406317 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc36c15eb50 msgr2=0x7fc36c0b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc36bfff640 1 --2- 172.21.15.31:0/1570406317 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc36c15eb50 0x7fc36c0b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc36bfff640 1 -- 172.21.15.31:0/1570406317 >> v1:172.21.15.31:6789/0 conn(0x7fc36c0b4210 legacy=0x7fc36c161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc36bfff640 1 -- 172.21.15.31:0/1570406317 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c06d030 con 0x7fc36c146820 2023-11-26T07:18:49.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.631+0000 7fc36affd640 1 -- 172.21.15.31:0/1570406317 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907521493 0 0) 0x7fc36c147190 con 0x7fc36c0b4210 2023-11-26T07:18:49.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc36bfff640 1 --2- 172.21.15.31:0/1570406317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c146820 0x7fc36c070a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0097c0 tx=0x7fc35c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7d17e83f66482d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc36affd640 1 -- 172.21.15.31:0/1570406317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0ab600 con 0x7fc36c146820 2023-11-26T07:18:49.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc36affd640 1 -- 172.21.15.31:0/1570406317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c0a7050 con 0x7fc36c146820 2023-11-26T07:18:49.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc36affd640 1 -- 172.21.15.31:0/1570406317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0ab980 con 0x7fc36c146820 2023-11-26T07:18:49.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc3728ad640 1 -- 172.21.15.31:0/1570406317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c146820 msgr2=0x7fc36c070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc3728ad640 1 --2- 172.21.15.31:0/1570406317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c146820 0x7fc36c070a50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc35c0097c0 tx=0x7fc35c01c450 comp rx=0 tx=0).stop 2023-11-26T07:18:49.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc3728ad640 1 -- 172.21.15.31:0/1570406317 shutdown_connections 2023-11-26T07:18:49.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc3728ad640 1 --2- 172.21.15.31:0/1570406317 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc36c15eb50 0x7fc36c0b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc3728ad640 1 --2- 172.21.15.31:0/1570406317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c146820 0x7fc36c070a50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc3728ad640 1 -- 172.21.15.31:0/1570406317 >> 172.21.15.31:0/1570406317 conn(0x7fc36c0b2f10 msgr2=0x7fc36c0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc3728ad640 1 -- 172.21.15.31:0/1570406317 shutdown_connections 2023-11-26T07:18:49.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc3728ad640 1 -- 172.21.15.31:0/1570406317 wait complete. 2023-11-26T07:18:49.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc3728ad640 1 Processor -- start 2023-11-26T07:18:49.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.635+0000 7fc3728ad640 1 -- start start 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc3728ad640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b4210 0x7fc36c140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc3728ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c146820 0x7fc36c145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc3728ad640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc36c15eb50 0x7fc36c13ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc3728ad640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c145e50 con 0x7fc36c0b4210 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc3728ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c141070 con 0x7fc36c146820 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc3728ad640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c141350 con 0x7fc36c15eb50 2023-11-26T07:18:49.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc36bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b4210 0x7fc36c140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc36bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b4210 0x7fc36c140990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56168/0 (socket says 172.21.15.31:56168) 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc36bfff640 1 -- 172.21.15.31:0/1862049840 learned_addr learned my addr 172.21.15.31:0/1862049840 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc36b7fe640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c146820 0x7fc36c145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc370e4a640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc36c15eb50 0x7fc36c13ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc36bfff640 1 -- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc36c15eb50 msgr2=0x7fc36c13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc36bfff640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc36c15eb50 0x7fc36c13ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc36bfff640 1 -- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c146820 msgr2=0x7fc36c145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc36bfff640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c146820 0x7fc36c145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc36bfff640 1 -- 172.21.15.31:0/1862049840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c009420 con 0x7fc36c0b4210 2023-11-26T07:18:49.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc36bfff640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b4210 0x7fc36c140990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc35c01ab00 tx=0x7fc35c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc368ff9640 1 -- 172.21.15.31:0/1862049840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c006e20 con 0x7fc36c0b4210 2023-11-26T07:18:49.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc368ff9640 1 -- 172.21.15.31:0/1862049840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c004030 con 0x7fc36c0b4210 2023-11-26T07:18:49.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc3728ad640 1 -- 172.21.15.31:0/1862049840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c141630 con 0x7fc36c0b4210 2023-11-26T07:18:49.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc368ff9640 1 -- 172.21.15.31:0/1862049840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c0043a0 con 0x7fc36c0b4210 2023-11-26T07:18:49.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc3728ad640 1 -- 172.21.15.31:0/1862049840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c141bf0 con 0x7fc36c0b4210 2023-11-26T07:18:49.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.639+0000 7fc3728ad640 1 -- 172.21.15.31:0/1862049840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c141de0 con 0x7fc36c0b4210 2023-11-26T07:18:49.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.643+0000 7fc368ff9640 1 -- 172.21.15.31:0/1862049840 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fc35c0b2040 con 0x7fc36c0b4210 2023-11-26T07:18:49.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.643+0000 7fc368ff9640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc358066620 0x7fc358068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:49.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.643+0000 7fc368ff9640 1 -- 172.21.15.31:0/1862049840 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc35c11d570 con 0x7fc36c0b4210 2023-11-26T07:18:49.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.643+0000 7fc36b7fe640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc358066620 0x7fc358068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:49.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.643+0000 7fc36b7fe640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc358066620 0x7fc358068ae0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc36c168040 tx=0x7fc360009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:49.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.647+0000 7fc368ff9640 1 -- 172.21.15.31:0/1862049840 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc35c0ea5c0 con 0x7fc36c0b4210 2023-11-26T07:18:49.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.839+0000 7fc3728ad640 1 -- 172.21.15.31:0/1862049840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc36c04e850 con 0x7fc36c0b4210 2023-11-26T07:18:49.844 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-11-26T07:18:49.839+0000 7fe86499d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:18:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.839+0000 7fc368ff9640 1 -- 172.21.15.31:0/1862049840 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc35c0b0090 con 0x7fc36c0b4210 2023-11-26T07:18:49.847 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:18:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 -- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc358066620 msgr2=0x7fc358068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc358066620 0x7fc358068ae0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc36c168040 tx=0x7fc360009040 comp rx=0 tx=0).stop 2023-11-26T07:18:49.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 -- 172.21.15.31:0/1862049840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b4210 msgr2=0x7fc36c140990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b4210 0x7fc36c140990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc35c01ab00 tx=0x7fc35c0191f0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 -- 172.21.15.31:0/1862049840 shutdown_connections 2023-11-26T07:18:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc358066620 0x7fc358068ae0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc36c15eb50 0x7fc36c13ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc36c146820 0x7fc36c145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 --2- 172.21.15.31:0/1862049840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc36c0b4210 0x7fc36c140990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 -- 172.21.15.31:0/1862049840 >> 172.21.15.31:0/1862049840 conn(0x7fc36c0b2f10 msgr2=0x7fc36c15ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 -- 172.21.15.31:0/1862049840 shutdown_connections 2023-11-26T07:18:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:49.843+0000 7fc3728ad640 1 -- 172.21.15.31:0/1862049840 wait complete. 2023-11-26T07:18:49.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:18:49.918 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-11-26T07:18:50.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f43556e3640 1 Processor -- start 2023-11-26T07:18:50.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f43556e3640 1 -- start start 2023-11-26T07:18:50.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f43556e3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f435006e3f0 0x7f435015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f43556e3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4350159630 con 0x7f43500cb9b0 2023-11-26T07:18:50.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f43556e3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435015f6d0 con 0x7f43500ce5d0 2023-11-26T07:18:50.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f43556e3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435006e860 con 0x7f435006e3f0 2023-11-26T07:18:50.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f4347fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f435006e3f0 0x7f435015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f4347fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f435006e3f0 0x7f435015f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56190/0 (socket says 172.21.15.31:56190) 2023-11-26T07:18:50.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f4347fff640 1 -- 172.21.15.31:0/478194043 learned_addr learned my addr 172.21.15.31:0/478194043 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:50.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f4347fff640 1 -- 172.21.15.31:0/478194043 >> v1:172.21.15.97:6789/0 conn(0x7f43500ce5d0 legacy=0x7f435015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f4347fff640 1 -- 172.21.15.31:0/478194043 >> v1:172.21.15.31:6789/0 conn(0x7f43500cb9b0 legacy=0x7f43500cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.011+0000 7f4347fff640 1 -- 172.21.15.31:0/478194043 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500cd1d0 con 0x7f435006e3f0 2023-11-26T07:18:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f4347fff640 1 --2- 172.21.15.31:0/478194043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f435006e3f0 0x7f435015f190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4338009870 tx=0x7f433801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53d89436109ff1c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f434e7fc640 1 -- 172.21.15.31:0/478194043 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43380ab600 con 0x7f435006e3f0 2023-11-26T07:18:50.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f434e7fc640 1 -- 172.21.15.31:0/478194043 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43380a7050 con 0x7f435006e3f0 2023-11-26T07:18:50.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f434e7fc640 1 -- 172.21.15.31:0/478194043 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43380ab980 con 0x7f435006e3f0 2023-11-26T07:18:50.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f43556e3640 1 -- 172.21.15.31:0/478194043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f435006e3f0 msgr2=0x7f435015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f43556e3640 1 --2- 172.21.15.31:0/478194043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f435006e3f0 0x7f435015f190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4338009870 tx=0x7f433801c540 comp rx=0 tx=0).stop 2023-11-26T07:18:50.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f43556e3640 1 -- 172.21.15.31:0/478194043 shutdown_connections 2023-11-26T07:18:50.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f43556e3640 1 --2- 172.21.15.31:0/478194043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f435006e3f0 0x7f435015f190 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f43556e3640 1 -- 172.21.15.31:0/478194043 >> 172.21.15.31:0/478194043 conn(0x7f43500b0cb0 msgr2=0x7f43500b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:50.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f43556e3640 1 -- 172.21.15.31:0/478194043 shutdown_connections 2023-11-26T07:18:50.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.015+0000 7f43556e3640 1 -- 172.21.15.31:0/478194043 wait complete. 2023-11-26T07:18:50.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43556e3640 1 Processor -- start 2023-11-26T07:18:50.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43556e3640 1 -- start start 2023-11-26T07:18:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43556e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f435006e3f0 0x7f4350142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43556e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500cb9b0 0x7f4350071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43556e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43500ce5d0 0x7f4350071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43556e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435015de10 con 0x7f43500ce5d0 2023-11-26T07:18:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43556e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435015d200 con 0x7f43500cb9b0 2023-11-26T07:18:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43556e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435015d4e0 con 0x7f435006e3f0 2023-11-26T07:18:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f434f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43500ce5d0 0x7f4350071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f434effd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f435006e3f0 0x7f4350142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f434f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43500ce5d0 0x7f4350071bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56202/0 (socket says 172.21.15.31:56202) 2023-11-26T07:18:50.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f434f7fe640 1 -- 172.21.15.31:0/3816775811 learned_addr learned my addr 172.21.15.31:0/3816775811 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:50.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f4347fff640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500cb9b0 0x7f4350071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f434f7fe640 1 -- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f435006e3f0 msgr2=0x7f4350142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f434f7fe640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f435006e3f0 0x7f4350142360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f434f7fe640 1 -- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500cb9b0 msgr2=0x7f4350071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f434f7fe640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500cb9b0 0x7f4350071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f434f7fe640 1 -- 172.21.15.31:0/3816775811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380094d0 con 0x7f43500ce5d0 2023-11-26T07:18:50.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f434f7fe640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43500ce5d0 0x7f4350071bd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f434800c730 tx=0x7f434800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43477fe640 1 -- 172.21.15.31:0/3816775811 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348003b80 con 0x7f43500ce5d0 2023-11-26T07:18:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43477fe640 1 -- 172.21.15.31:0/3816775811 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348003ce0 con 0x7f43500ce5d0 2023-11-26T07:18:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43477fe640 1 -- 172.21.15.31:0/3816775811 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434800aa30 con 0x7f43500ce5d0 2023-11-26T07:18:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.019+0000 7f43556e3640 1 -- 172.21.15.31:0/3816775811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435015d7c0 con 0x7f43500ce5d0 2023-11-26T07:18:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.023+0000 7f43556e3640 1 -- 172.21.15.31:0/3816775811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43501428a0 con 0x7f43500ce5d0 2023-11-26T07:18:50.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.023+0000 7f43556e3640 1 -- 172.21.15.31:0/3816775811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43500671f0 con 0x7f43500ce5d0 2023-11-26T07:18:50.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.023+0000 7f43477fe640 1 -- 172.21.15.31:0/3816775811 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f434800ab90 con 0x7f43500ce5d0 2023-11-26T07:18:50.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.023+0000 7f43477fe640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4330066620 0x7f4330068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:50.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.023+0000 7f43477fe640 1 -- 172.21.15.31:0/3816775811 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4348083e70 con 0x7f43500ce5d0 2023-11-26T07:18:50.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.027+0000 7f434effd640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4330066620 0x7f4330068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:50.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.027+0000 7f434effd640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4330066620 0x7f4330068ae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4340007fd0 tx=0x7f4340009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:50.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.027+0000 7f43477fe640 1 -- 172.21.15.31:0/3816775811 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f43480510c0 con 0x7f43500ce5d0 2023-11-26T07:18:50.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.223+0000 7f43556e3640 1 -- 172.21.15.31:0/3816775811 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f435006b460 con 0x7f43500ce5d0 2023-11-26T07:18:50.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.843+0000 7f43477fe640 1 -- 172.21.15.31:0/3816775811 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f43480546e0 con 0x7f43500ce5d0 2023-11-26T07:18:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.843+0000 7f43556e3640 1 -- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4330066620 msgr2=0x7f4330068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.843+0000 7f43556e3640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4330066620 0x7f4330068ae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4340007fd0 tx=0x7f4340009040 comp rx=0 tx=0).stop 2023-11-26T07:18:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.847+0000 7f43556e3640 1 -- 172.21.15.31:0/3816775811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43500ce5d0 msgr2=0x7f4350071bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.847+0000 7f43556e3640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43500ce5d0 0x7f4350071bd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f434800c730 tx=0x7f434800cbf0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.847+0000 7f43556e3640 1 -- 172.21.15.31:0/3816775811 shutdown_connections 2023-11-26T07:18:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.847+0000 7f43556e3640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4330066620 0x7f4330068ae0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.847+0000 7f43556e3640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43500ce5d0 0x7f4350071bd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.847+0000 7f43556e3640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43500cb9b0 0x7f4350071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.847+0000 7f43556e3640 1 --2- 172.21.15.31:0/3816775811 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f435006e3f0 0x7f4350142360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.847+0000 7f43556e3640 1 -- 172.21.15.31:0/3816775811 >> 172.21.15.31:0/3816775811 conn(0x7f43500b0cb0 msgr2=0x7f43500cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.847+0000 7f43556e3640 1 -- 172.21.15.31:0/3816775811 shutdown_connections 2023-11-26T07:18:50.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:50.847+0000 7f43556e3640 1 -- 172.21.15.31:0/3816775811 wait complete. 2023-11-26T07:18:50.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:18:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1f3a1640 1 Processor -- start 2023-11-26T07:18:51.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1f3a1640 1 -- start start 2023-11-26T07:18:51.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1f3a1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde180cb9b0 0x7fde1806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:51.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1f3a1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde18159630 con 0x7fde180ce5d0 2023-11-26T07:18:51.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1f3a1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde180cd1d0 con 0x7fde180cbd90 2023-11-26T07:18:51.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1f3a1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1815bba0 con 0x7fde180cb9b0 2023-11-26T07:18:51.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1d93e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fde180ce5d0 0x7fde1815bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47602/0 (socket says 172.21.15.31:47602) 2023-11-26T07:18:51.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1d13d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fde180cb9b0 0x7fde1806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:51.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1d93e640 1 -- 172.21.15.31:0/969372489 learned_addr learned my addr 172.21.15.31:0/969372489 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:51.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1d13d640 1 -- 172.21.15.31:0/969372489 >> v1:172.21.15.97:6789/0 conn(0x7fde180cbd90 legacy=0x7fde1815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:51.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1d13d640 1 -- 172.21.15.31:0/969372489 >> v1:172.21.15.31:6789/0 conn(0x7fde180ce5d0 legacy=0x7fde1815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.059+0000 7fde1d13d640 1 -- 172.21.15.31:0/969372489 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde180cd4b0 con 0x7fde180cb9b0 2023-11-26T07:18:51.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde1d13d640 1 --2- 172.21.15.31:0/969372489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 0x7fde1806e3f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fde140097c0 tx=0x7fde1401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269ada39666f2c05 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:51.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde0ffff640 1 -- 172.21.15.31:0/969372489 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140ab600 con 0x7fde180cb9b0 2023-11-26T07:18:51.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde0ffff640 1 -- 172.21.15.31:0/969372489 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde140a7050 con 0x7fde180cb9b0 2023-11-26T07:18:51.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde0ffff640 1 -- 172.21.15.31:0/969372489 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140ab980 con 0x7fde180cb9b0 2023-11-26T07:18:51.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde1f3a1640 1 -- 172.21.15.31:0/969372489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 msgr2=0x7fde1806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde1f3a1640 1 --2- 172.21.15.31:0/969372489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 0x7fde1806e3f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fde140097c0 tx=0x7fde1401c440 comp rx=0 tx=0).stop 2023-11-26T07:18:51.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde1f3a1640 1 -- 172.21.15.31:0/969372489 shutdown_connections 2023-11-26T07:18:51.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde1f3a1640 1 --2- 172.21.15.31:0/969372489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 0x7fde1806e3f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:51.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde1f3a1640 1 -- 172.21.15.31:0/969372489 >> 172.21.15.31:0/969372489 conn(0x7fde180b0cb0 msgr2=0x7fde180b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:51.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde1f3a1640 1 -- 172.21.15.31:0/969372489 shutdown_connections 2023-11-26T07:18:51.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.063+0000 7fde1f3a1640 1 -- 172.21.15.31:0/969372489 wait complete. 2023-11-26T07:18:51.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1f3a1640 1 Processor -- start 2023-11-26T07:18:51.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1f3a1640 1 -- start start 2023-11-26T07:18:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1f3a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 0x7fde180ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1f3a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde180cbd90 0x7fde18150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1f3a1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fde180ce5d0 0x7fde180afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1f3a1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde18153570 con 0x7fde180cb9b0 2023-11-26T07:18:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1f3a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde18153810 con 0x7fde180cbd90 2023-11-26T07:18:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1f3a1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde18153af0 con 0x7fde180ce5d0 2023-11-26T07:18:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d13d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 0x7fde180ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d13d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 0x7fde180ae9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56216/0 (socket says 172.21.15.31:56216) 2023-11-26T07:18:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d13d640 1 -- 172.21.15.31:0/2934827730 learned_addr learned my addr 172.21.15.31:0/2934827730 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:51.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1c93c640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde180cbd90 0x7fde18150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d93e640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fde180ce5d0 0x7fde180afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d13d640 1 -- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fde180ce5d0 msgr2=0x7fde180afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d13d640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fde180ce5d0 0x7fde180afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d13d640 1 -- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde180cbd90 msgr2=0x7fde18150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d13d640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde180cbd90 0x7fde18150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d13d640 1 -- 172.21.15.31:0/2934827730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde14009420 con 0x7fde180cb9b0 2023-11-26T07:18:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d93e640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fde180ce5d0 0x7fde180afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:51.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1d13d640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 0x7fde180ae9d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fde140ae040 tx=0x7fde1401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:51.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde0dffb640 1 -- 172.21.15.31:0/2934827730 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140b7650 con 0x7fde180cb9b0 2023-11-26T07:18:51.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde0dffb640 1 -- 172.21.15.31:0/2934827730 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde140abe90 con 0x7fde180cb9b0 2023-11-26T07:18:51.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde0dffb640 1 -- 172.21.15.31:0/2934827730 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde140b7a10 con 0x7fde180cb9b0 2023-11-26T07:18:51.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.067+0000 7fde1f3a1640 1 -- 172.21.15.31:0/2934827730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde18153dd0 con 0x7fde180cb9b0 2023-11-26T07:18:51.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.071+0000 7fde1f3a1640 1 -- 172.21.15.31:0/2934827730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde18154390 con 0x7fde180cb9b0 2023-11-26T07:18:51.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.071+0000 7fde1f3a1640 1 -- 172.21.15.31:0/2934827730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde180671f0 con 0x7fde180cb9b0 2023-11-26T07:18:51.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.071+0000 7fde0dffb640 1 -- 172.21.15.31:0/2934827730 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fde140b7b70 con 0x7fde180cb9b0 2023-11-26T07:18:51.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.071+0000 7fde0dffb640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddfc066620 0x7fddfc068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:51.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.075+0000 7fde0dffb640 1 -- 172.21.15.31:0/2934827730 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fde1411ebe0 con 0x7fde180cb9b0 2023-11-26T07:18:51.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.075+0000 7fde1c93c640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddfc066620 0x7fddfc068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:51.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.075+0000 7fde1c93c640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddfc066620 0x7fddfc068ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fde180b4290 tx=0x7fde08007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:51.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.075+0000 7fde0dffb640 1 -- 172.21.15.31:0/2934827730 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fde140ebbd0 con 0x7fde180cb9b0 2023-11-26T07:18:51.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:51.235+0000 7fde1f3a1640 1 -- 172.21.15.31:0/2934827730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fde180cc1e0 con 0x7fde180cb9b0 2023-11-26T07:18:52.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.087+0000 7fde0dffb640 1 -- 172.21.15.31:0/2934827730 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7fde180cc1e0 con 0x7fde180cb9b0 2023-11-26T07:18:52.095 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:18:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.091+0000 7fde1f3a1640 1 -- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddfc066620 msgr2=0x7fddfc068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.091+0000 7fde1f3a1640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddfc066620 0x7fddfc068ae0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fde180b4290 tx=0x7fde08007e40 comp rx=0 tx=0).stop 2023-11-26T07:18:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.091+0000 7fde1f3a1640 1 -- 172.21.15.31:0/2934827730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 msgr2=0x7fde180ae9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.091+0000 7fde1f3a1640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 0x7fde180ae9d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fde140ae040 tx=0x7fde1401ac80 comp rx=0 tx=0).stop 2023-11-26T07:18:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.095+0000 7fde1f3a1640 1 -- 172.21.15.31:0/2934827730 shutdown_connections 2023-11-26T07:18:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.095+0000 7fde1f3a1640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddfc066620 0x7fddfc068ae0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.095+0000 7fde1f3a1640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fde180ce5d0 0x7fde180afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.095+0000 7fde1f3a1640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde180cbd90 0x7fde18150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.095+0000 7fde1f3a1640 1 --2- 172.21.15.31:0/2934827730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde180cb9b0 0x7fde180ae9d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.095+0000 7fde1f3a1640 1 -- 172.21.15.31:0/2934827730 >> 172.21.15.31:0/2934827730 conn(0x7fde180b0cb0 msgr2=0x7fde1815bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:52.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.095+0000 7fde1f3a1640 1 -- 172.21.15.31:0/2934827730 shutdown_connections 2023-11-26T07:18:52.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.095+0000 7fde1f3a1640 1 -- 172.21.15.31:0/2934827730 wait complete. 2023-11-26T07:18:52.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:18:52.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fd518d640 1 Processor -- start 2023-11-26T07:18:52.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fd518d640 1 -- start start 2023-11-26T07:18:52.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fd518d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fd0161f20 0x7f2fd00cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fd518d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fd00d02f0 0x7f2fd00cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fd518d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd00d0740 con 0x7f2fd015d7c0 2023-11-26T07:18:52.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fd518d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd00d0f90 con 0x7f2fd00d02f0 2023-11-26T07:18:52.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fd518d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00d11a0 con 0x7f2fd0161f20 2023-11-26T07:18:52.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fcf59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fd00d02f0 0x7f2fd00cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:52.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fcf59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fd00d02f0 0x7f2fd00cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56222/0 (socket says 172.21.15.31:56222) 2023-11-26T07:18:52.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fcf59e640 1 -- 172.21.15.31:0/2022039054 learned_addr learned my addr 172.21.15.31:0/2022039054 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:52.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fc7fff640 1 --2- 172.21.15.31:0/2022039054 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fd0161f20 0x7f2fd00cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:52.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fcf59e640 1 -- 172.21.15.31:0/2022039054 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fd0161f20 msgr2=0x7f2fd00cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:52.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fcf59e640 1 --2- 172.21.15.31:0/2022039054 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fd0161f20 0x7f2fd00cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fcf59e640 1 -- 172.21.15.31:0/2022039054 >> v1:172.21.15.97:6789/0 conn(0x7f2fd015d7c0 legacy=0x7f2fd00d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.275+0000 7f2fcf59e640 1 -- 172.21.15.31:0/2022039054 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd00d1480 con 0x7f2fd00d02f0 2023-11-26T07:18:52.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fcf59e640 1 --2- 172.21.15.31:0/2022039054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 0x7f2fd00cdf60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009e50 tx=0x7f2fc801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab277c12ae11d12 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:52.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fce59c640 1 -- 172.21.15.31:0/2022039054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80a5810 con 0x7f2fd00d02f0 2023-11-26T07:18:52.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fce59c640 1 -- 172.21.15.31:0/2022039054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc80a5970 con 0x7f2fd00d02f0 2023-11-26T07:18:52.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fce59c640 1 -- 172.21.15.31:0/2022039054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80a5d00 con 0x7f2fd00d02f0 2023-11-26T07:18:52.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fd518d640 1 -- 172.21.15.31:0/2022039054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 msgr2=0x7f2fd00cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fd518d640 1 --2- 172.21.15.31:0/2022039054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 0x7f2fd00cdf60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009e50 tx=0x7f2fc801c950 comp rx=0 tx=0).stop 2023-11-26T07:18:52.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fd518d640 1 -- 172.21.15.31:0/2022039054 shutdown_connections 2023-11-26T07:18:52.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fd518d640 1 --2- 172.21.15.31:0/2022039054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 0x7f2fd00cdf60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fd518d640 1 --2- 172.21.15.31:0/2022039054 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fd0161f20 0x7f2fd00cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fd518d640 1 -- 172.21.15.31:0/2022039054 >> 172.21.15.31:0/2022039054 conn(0x7f2fd008afc0 msgr2=0x7f2fd00d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:52.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fd518d640 1 -- 172.21.15.31:0/2022039054 shutdown_connections 2023-11-26T07:18:52.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.279+0000 7f2fd518d640 1 -- 172.21.15.31:0/2022039054 wait complete. 2023-11-26T07:18:52.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fd518d640 1 Processor -- start 2023-11-26T07:18:52.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fd518d640 1 -- start start 2023-11-26T07:18:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fd518d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 0x7f2fd00b8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fd518d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd015d7c0 0x7f2fd00b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fd518d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fd0158480 0x7f2fd01554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fd518d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd015c8b0 con 0x7f2fd00d02f0 2023-11-26T07:18:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fd518d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00b6520 con 0x7f2fd015d7c0 2023-11-26T07:18:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fd518d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd00b6ff0 con 0x7f2fd0158480 2023-11-26T07:18:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 0x7f2fd00b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fced9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 0x7f2fd00b8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56226/0 (socket says 172.21.15.31:56226) 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fced9d640 1 -- 172.21.15.31:0/762310011 learned_addr learned my addr 172.21.15.31:0/762310011 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fc7fff640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd015d7c0 0x7f2fd00b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fced9d640 1 -- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fd0158480 msgr2=0x7f2fd01554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fced9d640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fd0158480 0x7f2fd01554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fced9d640 1 -- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd015d7c0 msgr2=0x7f2fd00b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fced9d640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd015d7c0 0x7f2fd00b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fced9d640 1 -- 172.21.15.31:0/762310011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8009b30 con 0x7f2fd00d02f0 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fced9d640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 0x7f2fd00b8620 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000c420 tx=0x7f2fc000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fc77fe640 1 -- 172.21.15.31:0/762310011 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00052d0 con 0x7f2fd00d02f0 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fc77fe640 1 -- 172.21.15.31:0/762310011 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc0005430 con 0x7f2fd00d02f0 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fc77fe640 1 -- 172.21.15.31:0/762310011 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0007260 con 0x7f2fd00d02f0 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fd518d640 1 -- 172.21.15.31:0/762310011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd00b6750 con 0x7f2fd00d02f0 2023-11-26T07:18:52.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.283+0000 7f2fd518d640 1 -- 172.21.15.31:0/762310011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd00b6d10 con 0x7f2fd00d02f0 2023-11-26T07:18:52.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.287+0000 7f2fd518d640 1 -- 172.21.15.31:0/762310011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd004e850 con 0x7f2fd00d02f0 2023-11-26T07:18:52.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.287+0000 7f2fc77fe640 1 -- 172.21.15.31:0/762310011 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2fc00073c0 con 0x7f2fd00d02f0 2023-11-26T07:18:52.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.287+0000 7f2fc77fe640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fb0066620 0x7f2fb0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:52.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.287+0000 7f2fc77fe640 1 -- 172.21.15.31:0/762310011 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f2fc0083240 con 0x7f2fd00d02f0 2023-11-26T07:18:52.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.287+0000 7f2fc7fff640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fb0066620 0x7f2fb0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:52.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.287+0000 7f2fc7fff640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fb0066620 0x7f2fb0068ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0168520 tx=0x7f2fb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:52.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.291+0000 7f2fc77fe640 1 -- 172.21.15.31:0/762310011 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2fc0050590 con 0x7f2fd00d02f0 2023-11-26T07:18:52.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:52.455+0000 7f2fd518d640 1 -- 172.21.15.31:0/762310011 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2fd00d0740 con 0x7f2fd00d02f0 2023-11-26T07:18:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.095+0000 7f2fc77fe640 1 -- 172.21.15.31:0/762310011 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2fd00d0740 con 0x7f2fd00d02f0 2023-11-26T07:18:53.101 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-11-26T07:18:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.095+0000 7f2fd518d640 1 -- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fb0066620 msgr2=0x7f2fb0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.095+0000 7f2fd518d640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fb0066620 0x7f2fb0068ae0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0168520 tx=0x7f2fb8008040 comp rx=0 tx=0).stop 2023-11-26T07:18:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.095+0000 7f2fd518d640 1 -- 172.21.15.31:0/762310011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 msgr2=0x7f2fd00b8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.095+0000 7f2fd518d640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 0x7f2fd00b8620 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000c420 tx=0x7f2fc000c8e0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.099+0000 7f2fd518d640 1 -- 172.21.15.31:0/762310011 shutdown_connections 2023-11-26T07:18:53.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.099+0000 7f2fd518d640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2fb0066620 0x7f2fb0068ae0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.099+0000 7f2fd518d640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fd0158480 0x7f2fd01554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.099+0000 7f2fd518d640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fd015d7c0 0x7f2fd00b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.099+0000 7f2fd518d640 1 --2- 172.21.15.31:0/762310011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fd00d02f0 0x7f2fd00b8620 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.099+0000 7f2fd518d640 1 -- 172.21.15.31:0/762310011 >> 172.21.15.31:0/762310011 conn(0x7f2fd008afc0 msgr2=0x7f2fd00b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:53.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.099+0000 7f2fd518d640 1 -- 172.21.15.31:0/762310011 shutdown_connections 2023-11-26T07:18:53.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.099+0000 7f2fd518d640 1 -- 172.21.15.31:0/762310011 wait complete. 2023-11-26T07:18:53.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:18:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa474ae4640 1 Processor -- start 2023-11-26T07:18:53.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa474ae4640 1 -- start start 2023-11-26T07:18:53.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa474ae4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa47015d8f0 0x7fa47015e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa474ae4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4700ced30 0x7fa4700cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa474ae4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47015ecc0 con 0x7fa4700cf1e0 2023-11-26T07:18:53.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa474ae4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4700d4420 con 0x7fa4700ced30 2023-11-26T07:18:53.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa474ae4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4700cdfb0 con 0x7fa47015d8f0 2023-11-26T07:18:53.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa467fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4700ced30 0x7fa4700cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa46ed9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa4700cf1e0 0x7fa4700d1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47616/0 (socket says 172.21.15.31:47616) 2023-11-26T07:18:53.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa46ed9d640 1 -- 172.21.15.31:0/3853469142 learned_addr learned my addr 172.21.15.31:0/3853469142 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:53.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa46e59c640 1 --2- 172.21.15.31:0/3853469142 >> v2:172.21.15.97:3300/0 conn(0x7fa47015d8f0 0x7fa47015e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa467fff640 1 -- 172.21.15.31:0/3853469142 >> v2:172.21.15.97:3300/0 conn(0x7fa47015d8f0 msgr2=0x7fa47015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:53.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa467fff640 1 --2- 172.21.15.31:0/3853469142 >> v2:172.21.15.97:3300/0 conn(0x7fa47015d8f0 0x7fa47015e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa467fff640 1 -- 172.21.15.31:0/3853469142 >> v1:172.21.15.31:6789/0 conn(0x7fa4700cf1e0 legacy=0x7fa4700d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.279+0000 7fa467fff640 1 -- 172.21.15.31:0/3853469142 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4700d4140 con 0x7fa4700ced30 2023-11-26T07:18:53.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa467fff640 1 --2- 172.21.15.31:0/3853469142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700ced30 0x7fa4700cfe00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa4580097c0 tx=0x7fa45801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45fa4ab94ed28e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa46dd9b640 1 -- 172.21.15.31:0/3853469142 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580ab610 con 0x7fa4700ced30 2023-11-26T07:18:53.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa46dd9b640 1 -- 172.21.15.31:0/3853469142 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4580ab770 con 0x7fa4700ced30 2023-11-26T07:18:53.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa46dd9b640 1 -- 172.21.15.31:0/3853469142 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580abb00 con 0x7fa4700ced30 2023-11-26T07:18:53.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa474ae4640 1 -- 172.21.15.31:0/3853469142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700ced30 msgr2=0x7fa4700cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa474ae4640 1 --2- 172.21.15.31:0/3853469142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700ced30 0x7fa4700cfe00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa4580097c0 tx=0x7fa45801c410 comp rx=0 tx=0).stop 2023-11-26T07:18:53.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa474ae4640 1 -- 172.21.15.31:0/3853469142 shutdown_connections 2023-11-26T07:18:53.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa474ae4640 1 --2- 172.21.15.31:0/3853469142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700ced30 0x7fa4700cfe00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa474ae4640 1 --2- 172.21.15.31:0/3853469142 >> v2:172.21.15.97:3300/0 conn(0x7fa47015d8f0 0x7fa47015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa474ae4640 1 -- 172.21.15.31:0/3853469142 >> 172.21.15.31:0/3853469142 conn(0x7fa47009c750 msgr2=0x7fa4700d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:53.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa474ae4640 1 -- 172.21.15.31:0/3853469142 shutdown_connections 2023-11-26T07:18:53.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.283+0000 7fa474ae4640 1 -- 172.21.15.31:0/3853469142 wait complete. 2023-11-26T07:18:53.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa474ae4640 1 Processor -- start 2023-11-26T07:18:53.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa474ae4640 1 -- start start 2023-11-26T07:18:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa474ae4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4700ced30 0x7fa4700c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa474ae4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700cf1e0 0x7fa4700b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa474ae4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa47015d8f0 0x7fa4700b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa474ae4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47015d1e0 con 0x7fa4700cf1e0 2023-11-26T07:18:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa474ae4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47015ad90 con 0x7fa4700ced30 2023-11-26T07:18:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa474ae4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47015c560 con 0x7fa47015d8f0 2023-11-26T07:18:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa467fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700cf1e0 0x7fa4700b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa467fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700cf1e0 0x7fa4700b7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56242/0 (socket says 172.21.15.31:56242) 2023-11-26T07:18:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa467fff640 1 -- 172.21.15.31:0/2835303319 learned_addr learned my addr 172.21.15.31:0/2835303319 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:53.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa46e59c640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4700ced30 0x7fa4700c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa46ed9d640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa47015d8f0 0x7fa4700b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa467fff640 1 -- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa47015d8f0 msgr2=0x7fa4700b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa467fff640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa47015d8f0 0x7fa4700b82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa467fff640 1 -- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4700ced30 msgr2=0x7fa4700c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa467fff640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4700ced30 0x7fa4700c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa467fff640 1 -- 172.21.15.31:0/2835303319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa458009420 con 0x7fa4700cf1e0 2023-11-26T07:18:53.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa467fff640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700cf1e0 0x7fa4700b7d60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa45801a040 tx=0x7fa45801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa466ffd640 1 -- 172.21.15.31:0/2835303319 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580bf610 con 0x7fa4700cf1e0 2023-11-26T07:18:53.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa466ffd640 1 -- 172.21.15.31:0/2835303319 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4580bf790 con 0x7fa4700cf1e0 2023-11-26T07:18:53.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa466ffd640 1 -- 172.21.15.31:0/2835303319 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4580bfb40 con 0x7fa4700cf1e0 2023-11-26T07:18:53.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.287+0000 7fa474ae4640 1 -- 172.21.15.31:0/2835303319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47015c840 con 0x7fa4700cf1e0 2023-11-26T07:18:53.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.291+0000 7fa474ae4640 1 -- 172.21.15.31:0/2835303319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47015ce00 con 0x7fa4700cf1e0 2023-11-26T07:18:53.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.291+0000 7fa474ae4640 1 -- 172.21.15.31:0/2835303319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47004e850 con 0x7fa4700cf1e0 2023-11-26T07:18:53.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.291+0000 7fa466ffd640 1 -- 172.21.15.31:0/2835303319 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fa4580b5400 con 0x7fa4700cf1e0 2023-11-26T07:18:53.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.291+0000 7fa466ffd640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa450066670 0x7fa450068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.291+0000 7fa466ffd640 1 -- 172.21.15.31:0/2835303319 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa45811ef40 con 0x7fa4700cf1e0 2023-11-26T07:18:53.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.291+0000 7fa46e59c640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa450066670 0x7fa450068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.295+0000 7fa46e59c640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa450066670 0x7fa450068b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa460009b60 tx=0x7fa460009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.295+0000 7fa466ffd640 1 -- 172.21.15.31:0/2835303319 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa4580ec590 con 0x7fa4700cf1e0 2023-11-26T07:18:53.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.467+0000 7fa474ae4640 1 -- 172.21.15.31:0/2835303319 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa4700cf9d0 con 0x7fa4700cf1e0 2023-11-26T07:18:53.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa466ffd640 1 -- 172.21.15.31:0/2835303319 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa4580efbb0 con 0x7fa4700cf1e0 2023-11-26T07:18:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 -- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa450066670 msgr2=0x7fa450068b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa450066670 0x7fa450068b30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa460009b60 tx=0x7fa460009190 comp rx=0 tx=0).stop 2023-11-26T07:18:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 -- 172.21.15.31:0/2835303319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700cf1e0 msgr2=0x7fa4700b7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700cf1e0 0x7fa4700b7d60 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa45801a040 tx=0x7fa45801ac50 comp rx=0 tx=0).stop 2023-11-26T07:18:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 -- 172.21.15.31:0/2835303319 shutdown_connections 2023-11-26T07:18:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa450066670 0x7fa450068b30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa47015d8f0 0x7fa4700b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4700cf1e0 0x7fa4700b7d60 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 --2- 172.21.15.31:0/2835303319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4700ced30 0x7fa4700c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 -- 172.21.15.31:0/2835303319 >> 172.21.15.31:0/2835303319 conn(0x7fa47009c750 msgr2=0x7fa4700b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 -- 172.21.15.31:0/2835303319 shutdown_connections 2023-11-26T07:18:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.471+0000 7fa474ae4640 1 -- 172.21.15.31:0/2835303319 wait complete. 2023-11-26T07:18:53.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:18:53.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.635+0000 7f0981772640 1 Processor -- start 2023-11-26T07:18:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.635+0000 7f0981772640 1 -- start start 2023-11-26T07:18:53.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f0981772640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f097c144c80 0x7f097c0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f0981772640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f097c0ac460 0x7f097c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f0981772640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097c145060 con 0x7f097c070c20 2023-11-26T07:18:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f0981772640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f097c071fd0 con 0x7f097c0ac460 2023-11-26T07:18:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f0981772640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0ac8c0 con 0x7f097c144c80 2023-11-26T07:18:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f097c070c20 0x7f097c15eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:47622/0 (socket says 172.21.15.31:47622) 2023-11-26T07:18:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097b7fe640 1 -- 172.21.15.31:0/3479364383 learned_addr learned my addr 172.21.15.31:0/3479364383 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f0973fff640 1 --2- 172.21.15.31:0/3479364383 >> v2:172.21.15.97:3300/0 conn(0x7f097c0ac460 0x7f097c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097affd640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f097c144c80 0x7f097c0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229078461 0 0) 0x7f097c145060 con 0x7f097c070c20 2023-11-26T07:18:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0968003530 con 0x7f097c070c20 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 197852858 0 0) 0x7f0968003530 con 0x7f097c070c20 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f097c144c80 msgr2=0x7f097c0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f097c144c80 0x7f097c0abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 >> v2:172.21.15.97:3300/0 conn(0x7f097c0ac460 msgr2=0x7f097c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 --2- 172.21.15.31:0/3479364383 >> v2:172.21.15.97:3300/0 conn(0x7f097c0ac460 0x7f097c0706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c071450 con 0x7f097c070c20 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f0974001a00 con 0x7f097c070c20 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 >> v1:172.21.15.31:6789/0 conn(0x7f097c070c20 legacy=0x7f097c15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f097c144c80 0x7f0968003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968003f50 0x7f09680047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0968004d10 0x7f0968005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c071fd0 con 0x7f0968004d10 2023-11-26T07:18:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0ac8c0 con 0x7f0968003f50 2023-11-26T07:18:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096800b990 con 0x7f097c144c80 2023-11-26T07:18:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097b7fe640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0968004d10 0x7f0968005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097affd640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f097c144c80 0x7f0968003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f0973fff640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968003f50 0x7f09680047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097b7fe640 1 -- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f097c144c80 msgr2=0x7f0968003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097b7fe640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f097c144c80 0x7f0968003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097b7fe640 1 -- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968003f50 msgr2=0x7f09680047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097b7fe640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0968003f50 0x7f09680047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.639+0000 7f097b7fe640 1 -- 172.21.15.31:0/3479364383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968003530 con 0x7f0968004d10 2023-11-26T07:18:53.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f097b7fe640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0968004d10 0x7f0968005190 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f097400a7c0 tx=0x7f097401d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd07d6d3273522ba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09740ad630 con 0x7f0968004d10 2023-11-26T07:18:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09740a8070 con 0x7f0968004d10 2023-11-26T07:18:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f097a7fc640 1 -- 172.21.15.31:0/3479364383 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09740ada30 con 0x7f0968004d10 2023-11-26T07:18:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f0981772640 1 -- 172.21.15.31:0/3479364383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0968004d10 msgr2=0x7f0968005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f0981772640 1 --2- 172.21.15.31:0/3479364383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0968004d10 0x7f0968005190 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f097400a7c0 tx=0x7f097401d280 comp rx=0 tx=0).stop 2023-11-26T07:18:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f097b7fe640 1 -- 172.21.15.31:0/3479364383 reap_dead start 2023-11-26T07:18:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f0981772640 1 -- 172.21.15.31:0/3479364383 shutdown_connections 2023-11-26T07:18:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f0981772640 1 -- 172.21.15.31:0/3479364383 >> 172.21.15.31:0/3479364383 conn(0x7f097c0b0c40 msgr2=0x7f097c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f0981772640 1 -- 172.21.15.31:0/3479364383 shutdown_connections 2023-11-26T07:18:53.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f0981772640 1 -- 172.21.15.31:0/3479364383 wait complete. 2023-11-26T07:18:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f0981772640 1 Processor -- start 2023-11-26T07:18:53.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.643+0000 7f0981772640 1 -- start start 2023-11-26T07:18:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0981772640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0968004d10 0x7f097c15d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0981772640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f097c15e5c0 0x7f097c0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0981772640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f097c15b9a0 0x7f097c15be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0981772640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c15c360 con 0x7f097c15b9a0 2023-11-26T07:18:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0981772640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0b2580 con 0x7f097c15e5c0 2023-11-26T07:18:53.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0981772640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c0b2820 con 0x7f0968004d10 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f097c15b9a0 0x7f097c15be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f097c15b9a0 0x7f097c15be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56250/0 (socket says 172.21.15.31:56250) 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097b7fe640 1 -- 172.21.15.31:0/679803480 learned_addr learned my addr 172.21.15.31:0/679803480 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097affd640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0968004d10 0x7f097c15d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0973fff640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f097c15e5c0 0x7f097c0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097b7fe640 1 -- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0968004d10 msgr2=0x7f097c15d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097b7fe640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0968004d10 0x7f097c15d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097b7fe640 1 -- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f097c15e5c0 msgr2=0x7f097c0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097b7fe640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f097c15e5c0 0x7f097c0b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097b7fe640 1 -- 172.21.15.31:0/679803480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c0b2b00 con 0x7f097c15b9a0 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0973fff640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f097c15e5c0 0x7f097c0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:53.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097affd640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0968004d10 0x7f097c15d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:53.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f097b7fe640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f097c15b9a0 0x7f097c15be20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f097400a760 tx=0x7f0968005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f09737fe640 1 -- 172.21.15.31:0/679803480 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09740017c0 con 0x7f097c15b9a0 2023-11-26T07:18:53.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f09737fe640 1 -- 172.21.15.31:0/679803480 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974001920 con 0x7f097c15b9a0 2023-11-26T07:18:53.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f09737fe640 1 -- 172.21.15.31:0/679803480 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974004db0 con 0x7f097c15b9a0 2023-11-26T07:18:53.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0981772640 1 -- 172.21.15.31:0/679803480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097c0b2de0 con 0x7f097c15b9a0 2023-11-26T07:18:53.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0981772640 1 -- 172.21.15.31:0/679803480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097c0b33a0 con 0x7f097c15b9a0 2023-11-26T07:18:53.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.647+0000 7f0981772640 1 -- 172.21.15.31:0/679803480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097c0b1ba0 con 0x7f097c15b9a0 2023-11-26T07:18:53.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.651+0000 7f09737fe640 1 -- 172.21.15.31:0/679803480 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f09740aa030 con 0x7f097c15b9a0 2023-11-26T07:18:53.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.651+0000 7f09737fe640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f095c066650 0x7f095c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:53.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.651+0000 7f09737fe640 1 -- 172.21.15.31:0/679803480 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f097411dcb0 con 0x7f097c15b9a0 2023-11-26T07:18:53.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.651+0000 7f097affd640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f095c066650 0x7f095c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:53.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.651+0000 7f097affd640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f095c066650 0x7f095c068b10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f096c009be0 tx=0x7f096c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:53.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.655+0000 7f09737fe640 1 -- 172.21.15.31:0/679803480 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f09740eaf70 con 0x7f097c15b9a0 2023-11-26T07:18:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:53.867+0000 7f0981772640 1 -- 172.21.15.31:0/679803480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f097c0aca00 con 0x7f097c15b9a0 2023-11-26T07:18:54.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f09737fe640 1 -- 172.21.15.31:0/679803480 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v100) v1 ==== 158+0+0 (secure 0 0 0) 0x7f09740bc030 con 0x7f097c15b9a0 2023-11-26T07:18:54.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 -- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f095c066650 msgr2=0x7f095c068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f095c066650 0x7f095c068b10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f096c009be0 tx=0x7f096c008040 comp rx=0 tx=0).stop 2023-11-26T07:18:54.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 -- 172.21.15.31:0/679803480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f097c15b9a0 msgr2=0x7f097c15be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f097c15b9a0 0x7f097c15be20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f097400a760 tx=0x7f0968005800 comp rx=0 tx=0).stop 2023-11-26T07:18:54.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 -- 172.21.15.31:0/679803480 shutdown_connections 2023-11-26T07:18:54.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f095c066650 0x7f095c068b10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f097c15b9a0 0x7f097c15be20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f097c15e5c0 0x7f097c0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 --2- 172.21.15.31:0/679803480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0968004d10 0x7f097c15d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 -- 172.21.15.31:0/679803480 >> 172.21.15.31:0/679803480 conn(0x7f097c0b0c40 msgr2=0x7f097c142d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:54.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 -- 172.21.15.31:0/679803480 shutdown_connections 2023-11-26T07:18:54.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.099+0000 7f0981772640 1 -- 172.21.15.31:0/679803480 wait complete. 2023-11-26T07:18:54.126 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2023-11-26T07:18:54.126 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:18:54.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:54.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd16285a640 1 Processor -- start 2023-11-26T07:18:54.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd16285a640 1 -- start start 2023-11-26T07:18:54.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd16285a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd15c144c80 0x7fd15c0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd16285a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd15c070df0 0x7fd15c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd16285a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15c145060 con 0x7fd15c0ac460 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd16285a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15c071fd0 con 0x7fd15c144c80 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd16285a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15c0ac8c0 con 0x7fd15c070df0 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd15bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd15c144c80 0x7fd15c0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd15bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd15c144c80 0x7fd15c0abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51078/0 (socket says 172.21.15.31:51078) 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd15bfff640 1 -- 172.21.15.31:0/4072051163 learned_addr learned my addr 172.21.15.31:0/4072051163 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd15b7fe640 1 --1- 172.21.15.31:0/4072051163 >> v1:172.21.15.31:6789/0 conn(0x7fd15c0ac460 0x7fd15c0706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42452/0 (socket says 172.21.15.31:42452) 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.263+0000 7fd160df7640 1 --2- 172.21.15.31:0/4072051163 >> v2:172.21.15.97:3300/0 conn(0x7fd15c070df0 0x7fd15c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.267+0000 7fd15bfff640 1 -- 172.21.15.31:0/4072051163 >> v2:172.21.15.97:3300/0 conn(0x7fd15c070df0 msgr2=0x7fd15c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.267+0000 7fd15bfff640 1 --2- 172.21.15.31:0/4072051163 >> v2:172.21.15.97:3300/0 conn(0x7fd15c070df0 0x7fd15c06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.267+0000 7fd15bfff640 1 -- 172.21.15.31:0/4072051163 >> v1:172.21.15.31:6789/0 conn(0x7fd15c0ac460 legacy=0x7fd15c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.267+0000 7fd15bfff640 1 -- 172.21.15.31:0/4072051163 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c071450 con 0x7fd15c144c80 2023-11-26T07:18:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.267+0000 7fd15affd640 1 -- 172.21.15.31:0/4072051163 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779517620 0 0) 0x7fd15c145060 con 0x7fd15c0ac460 2023-11-26T07:18:54.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.267+0000 7fd15bfff640 1 --2- 172.21.15.31:0/4072051163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c144c80 0x7fd15c0abf20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0ce840 tx=0x7fd14c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be2b7c2939452d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.267+0000 7fd15affd640 1 -- 172.21.15.31:0/4072051163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0ab5f0 con 0x7fd15c144c80 2023-11-26T07:18:54.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.267+0000 7fd15affd640 1 -- 172.21.15.31:0/4072051163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14c0ab750 con 0x7fd15c144c80 2023-11-26T07:18:54.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.267+0000 7fd15affd640 1 -- 172.21.15.31:0/4072051163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0abae0 con 0x7fd15c144c80 2023-11-26T07:18:54.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 -- 172.21.15.31:0/4072051163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c144c80 msgr2=0x7fd15c0abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 --2- 172.21.15.31:0/4072051163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c144c80 0x7fd15c0abf20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd15c0ce840 tx=0x7fd14c01c500 comp rx=0 tx=0).stop 2023-11-26T07:18:54.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 -- 172.21.15.31:0/4072051163 shutdown_connections 2023-11-26T07:18:54.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 --2- 172.21.15.31:0/4072051163 >> v2:172.21.15.97:3300/0 conn(0x7fd15c070df0 0x7fd15c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 --2- 172.21.15.31:0/4072051163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c144c80 0x7fd15c0abf20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 -- 172.21.15.31:0/4072051163 >> 172.21.15.31:0/4072051163 conn(0x7fd15c0b0c40 msgr2=0x7fd15c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:54.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 -- 172.21.15.31:0/4072051163 shutdown_connections 2023-11-26T07:18:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 -- 172.21.15.31:0/4072051163 wait complete. 2023-11-26T07:18:54.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 Processor -- start 2023-11-26T07:18:54.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 -- start start 2023-11-26T07:18:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd15c070df0 0x7fd15c0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c0ac460 0x7fd15c13c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c144c80 0x7fd15c13e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c158e40 con 0x7fd15c0ac460 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c0b3230 con 0x7fd15c144c80 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd16285a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c0b3510 con 0x7fd15c070df0 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd15b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c0ac460 0x7fd15c13c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd15b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c0ac460 0x7fd15c13c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51094/0 (socket says 172.21.15.31:51094) 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd15b7fe640 1 -- 172.21.15.31:0/1458560021 learned_addr learned my addr 172.21.15.31:0/1458560021 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd160df7640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c144c80 0x7fd15c13e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd15bfff640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd15c070df0 0x7fd15c0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd15b7fe640 1 -- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd15c070df0 msgr2=0x7fd15c0b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.271+0000 7fd15b7fe640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd15c070df0 0x7fd15c0b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd15b7fe640 1 -- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c144c80 msgr2=0x7fd15c13e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd15b7fe640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c144c80 0x7fd15c13e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd15b7fe640 1 -- 172.21.15.31:0/1458560021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c0094d0 con 0x7fd15c0ac460 2023-11-26T07:18:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd160df7640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c144c80 0x7fd15c13e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd15b7fe640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c0ac460 0x7fd15c13c740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd1500011f0 tx=0x7fd15000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd158ff9640 1 -- 172.21.15.31:0/1458560021 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1500074e0 con 0x7fd15c0ac460 2023-11-26T07:18:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd158ff9640 1 -- 172.21.15.31:0/1458560021 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd150007640 con 0x7fd15c0ac460 2023-11-26T07:18:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd158ff9640 1 -- 172.21.15.31:0/1458560021 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150004a10 con 0x7fd15c0ac460 2023-11-26T07:18:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd16285a640 1 -- 172.21.15.31:0/1458560021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c0b37f0 con 0x7fd15c0ac460 2023-11-26T07:18:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd16285a640 1 -- 172.21.15.31:0/1458560021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15c09bc50 con 0x7fd15c0ac460 2023-11-26T07:18:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd16285a640 1 -- 172.21.15.31:0/1458560021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15c158ff0 con 0x7fd15c0ac460 2023-11-26T07:18:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd158ff9640 1 -- 172.21.15.31:0/1458560021 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd150004b70 con 0x7fd15c0ac460 2023-11-26T07:18:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd158ff9640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd148066650 0x7fd148068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd158ff9640 1 -- 172.21.15.31:0/1458560021 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd150083100 con 0x7fd15c0ac460 2023-11-26T07:18:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.275+0000 7fd15bfff640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd148066650 0x7fd148068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.279+0000 7fd15bfff640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd148066650 0x7fd148068b10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0af000 tx=0x7fd14c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.279+0000 7fd158ff9640 1 -- 172.21.15.31:0/1458560021 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd15c158ff0 con 0x7fd15c0ac460 2023-11-26T07:18:54.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.439+0000 7fd16285a640 1 -- 172.21.15.31:0/1458560021 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd15c0b3d10 con 0x7fd15c0ac460 2023-11-26T07:18:54.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.439+0000 7fd158ff9640 1 -- 172.21.15.31:0/1458560021 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+12902 (secure 0 0 0) 0x7fd150050560 con 0x7fd15c0ac460 2023-11-26T07:18:54.446 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:54.446 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:53.094238+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6835/4036387633":"2023-11-27T07:18:49.843694+0000","172.21.15.31:6834/4036387633":"2023-11-27T07:18:49.843694+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:54.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.439+0000 7fd16285a640 1 -- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd148066650 msgr2=0x7fd148068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.439+0000 7fd16285a640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd148066650 0x7fd148068b10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0af000 tx=0x7fd14c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:18:54.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.439+0000 7fd16285a640 1 -- 172.21.15.31:0/1458560021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c0ac460 msgr2=0x7fd15c13c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.439+0000 7fd16285a640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c0ac460 0x7fd15c13c740 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd1500011f0 tx=0x7fd15000c9b0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.443+0000 7fd16285a640 1 -- 172.21.15.31:0/1458560021 shutdown_connections 2023-11-26T07:18:54.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.443+0000 7fd16285a640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd148066650 0x7fd148068b10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.443+0000 7fd16285a640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd15c144c80 0x7fd15c13e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.443+0000 7fd16285a640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd15c0ac460 0x7fd15c13c740 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.443+0000 7fd16285a640 1 --2- 172.21.15.31:0/1458560021 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd15c070df0 0x7fd15c0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.443+0000 7fd16285a640 1 -- 172.21.15.31:0/1458560021 >> 172.21.15.31:0/1458560021 conn(0x7fd15c0b0c40 msgr2=0x7fd15c0add30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:54.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.443+0000 7fd16285a640 1 -- 172.21.15.31:0/1458560021 shutdown_connections 2023-11-26T07:18:54.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.443+0000 7fd16285a640 1 -- 172.21.15.31:0/1458560021 wait complete. 2023-11-26T07:18:54.467 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:54.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:54.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.607+0000 7fc1fcbec640 1 Processor -- start 2023-11-26T07:18:54.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.607+0000 7fc1fcbec640 1 -- start start 2023-11-26T07:18:54.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1fcbec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1f80cb9b0 0x7fc1f806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1fcbec640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc1f80cbd90 0x7fc1f815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1fcbec640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc1f80ce5d0 0x7fc1f815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1fcbec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f815bba0 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1fcbec640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f80cdd50 con 0x7fc1f80cbd90 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1fcbec640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f8159630 con 0x7fc1f80ce5d0 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1f659c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1f80cb9b0 0x7fc1f806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1f659c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc1f80cb9b0 0x7fc1f806e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51106/0 (socket says 172.21.15.31:51106) 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1f659c640 1 -- 172.21.15.31:0/1218347134 learned_addr learned my addr 172.21.15.31:0/1218347134 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1f6d9d640 1 --2- 172.21.15.31:0/1218347134 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc1f80ce5d0 0x7fc1f815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1effff640 1 --2- 172.21.15.31:0/1218347134 >> v2:172.21.15.97:3300/0 conn(0x7fc1f80cbd90 0x7fc1f815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1f659c640 1 -- 172.21.15.31:0/1218347134 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc1f80ce5d0 msgr2=0x7fc1f815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:54.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1f659c640 1 --2- 172.21.15.31:0/1218347134 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc1f80ce5d0 0x7fc1f815bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1f659c640 1 -- 172.21.15.31:0/1218347134 >> v2:172.21.15.97:3300/0 conn(0x7fc1f80cbd90 msgr2=0x7fc1f815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:54.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1f659c640 1 --2- 172.21.15.31:0/1218347134 >> v2:172.21.15.97:3300/0 conn(0x7fc1f80cbd90 0x7fc1f815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1f659c640 1 -- 172.21.15.31:0/1218347134 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f815f6d0 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.611+0000 7fc1f6d9d640 1 --2- 172.21.15.31:0/1218347134 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc1f80ce5d0 0x7fc1f815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:54.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1f659c640 1 --2- 172.21.15.31:0/1218347134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 0x7fc1f806e3f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80b5200 tx=0x7fc1e801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bacec25dd10c95 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1f5d9b640 1 -- 172.21.15.31:0/1218347134 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80ab670 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1f5d9b640 1 -- 172.21.15.31:0/1218347134 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e80ab7d0 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1f5d9b640 1 -- 172.21.15.31:0/1218347134 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80abb60 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 -- 172.21.15.31:0/1218347134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 msgr2=0x7fc1f806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 --2- 172.21.15.31:0/1218347134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 0x7fc1f806e3f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80b5200 tx=0x7fc1e801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 -- 172.21.15.31:0/1218347134 shutdown_connections 2023-11-26T07:18:54.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 --2- 172.21.15.31:0/1218347134 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc1f80ce5d0 0x7fc1f815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 --2- 172.21.15.31:0/1218347134 >> v2:172.21.15.97:3300/0 conn(0x7fc1f80cbd90 0x7fc1f815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 --2- 172.21.15.31:0/1218347134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 0x7fc1f806e3f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 -- 172.21.15.31:0/1218347134 >> 172.21.15.31:0/1218347134 conn(0x7fc1f80b0cb0 msgr2=0x7fc1f80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:54.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 -- 172.21.15.31:0/1218347134 shutdown_connections 2023-11-26T07:18:54.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 -- 172.21.15.31:0/1218347134 wait complete. 2023-11-26T07:18:54.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 Processor -- start 2023-11-26T07:18:54.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.615+0000 7fc1fcbec640 1 -- start start 2023-11-26T07:18:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1fcbec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 0x7fc1f8157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1fcbec640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc1f80cbd90 0x7fc1f80abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1fcbec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1f80ce5d0 0x7fc1f8157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1fcbec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f809a080 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1fcbec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80b2ea0 con 0x7fc1f80ce5d0 2023-11-26T07:18:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1fcbec640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80b3180 con 0x7fc1f80cbd90 2023-11-26T07:18:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1f659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 0x7fc1f8157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1f659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 0x7fc1f8157920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51110/0 (socket says 172.21.15.31:51110) 2023-11-26T07:18:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1f659c640 1 -- 172.21.15.31:0/4146828338 learned_addr learned my addr 172.21.15.31:0/4146828338 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1f659c640 1 -- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc1f80cbd90 msgr2=0x7fc1f80abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1f659c640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc1f80cbd90 0x7fc1f80abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1f659c640 1 -- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1f80ce5d0 msgr2=0x7fc1f8157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1f659c640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1f80ce5d0 0x7fc1f8157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1f659c640 1 -- 172.21.15.31:0/4146828338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e80094d0 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1f659c640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 0x7fc1f8157920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc1e801ab80 tx=0x7fc1e801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1eeffd640 1 -- 172.21.15.31:0/4146828338 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80b4800 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1eeffd640 1 -- 172.21.15.31:0/4146828338 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e80b4960 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1eeffd640 1 -- 172.21.15.31:0/4146828338 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80b4cf0 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1fcbec640 1 -- 172.21.15.31:0/4146828338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f8099a30 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1fcbec640 1 -- 172.21.15.31:0/4146828338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f8158150 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1fcbec640 1 -- 172.21.15.31:0/4146828338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f80671f0 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.619+0000 7fc1eeffd640 1 -- 172.21.15.31:0/4146828338 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fc1e80b3380 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.623+0000 7fc1eeffd640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1d8066620 0x7fc1d8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.623+0000 7fc1eeffd640 1 -- 172.21.15.31:0/4146828338 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc1e811e7b0 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.623+0000 7fc1effff640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1d8066620 0x7fc1d8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.623+0000 7fc1effff640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1d8066620 0x7fc1d8068ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80cc850 tx=0x7fc1e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.627+0000 7fc1eeffd640 1 -- 172.21.15.31:0/4146828338 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc1e80ebbb0 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.783+0000 7fc1fcbec640 1 -- 172.21.15.31:0/4146828338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1f806b460 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1eeffd640 1 -- 172.21.15.31:0/4146828338 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1e80c4020 con 0x7fc1f80cb9b0 2023-11-26T07:18:54.792 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 -- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1d8066620 msgr2=0x7fc1d8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1d8066620 0x7fc1d8068ae0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80cc850 tx=0x7fc1e0008040 comp rx=0 tx=0).stop 2023-11-26T07:18:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 -- 172.21.15.31:0/4146828338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 msgr2=0x7fc1f8157920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 0x7fc1f8157920 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc1e801ab80 tx=0x7fc1e801ad10 comp rx=0 tx=0).stop 2023-11-26T07:18:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 -- 172.21.15.31:0/4146828338 shutdown_connections 2023-11-26T07:18:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc1d8066620 0x7fc1d8068ae0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1f80ce5d0 0x7fc1f8157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc1f80cbd90 0x7fc1f80abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 --2- 172.21.15.31:0/4146828338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1f80cb9b0 0x7fc1f8157920 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 -- 172.21.15.31:0/4146828338 >> 172.21.15.31:0/4146828338 conn(0x7fc1f80b0cb0 msgr2=0x7fc1f8071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 -- 172.21.15.31:0/4146828338 shutdown_connections 2023-11-26T07:18:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.787+0000 7fc1fcbec640 1 -- 172.21.15.31:0/4146828338 wait complete. 2023-11-26T07:18:54.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-11-26T07:18:54.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.947+0000 7f7a84b5f640 1 Processor -- start 2023-11-26T07:18:54.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.947+0000 7f7a84b5f640 1 -- start start 2023-11-26T07:18:54.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a84b5f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7a800d3760 0x7f7a800d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a84b5f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 0x7f7a800cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a84b5f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a800d15f0 con 0x7f7a8015d880 2023-11-26T07:18:54.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a84b5f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a800d4110 con 0x7f7a800d3760 2023-11-26T07:18:54.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a84b5f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800d03d0 con 0x7f7a800cfa20 2023-11-26T07:18:54.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a7ffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 0x7f7a800cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a7ffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 0x7f7a800cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:60036/0 (socket says 172.21.15.31:60036) 2023-11-26T07:18:54.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a7ffff640 1 -- 172.21.15.31:0/3492395598 learned_addr learned my addr 172.21.15.31:0/3492395598 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:54.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a7effd640 1 --2- 172.21.15.31:0/3492395598 >> v2:172.21.15.97:3300/0 conn(0x7f7a800d3760 0x7f7a800d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a7ffff640 1 -- 172.21.15.31:0/3492395598 >> v2:172.21.15.97:3300/0 conn(0x7f7a800d3760 msgr2=0x7f7a800d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:54.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a7ffff640 1 --2- 172.21.15.31:0/3492395598 >> v2:172.21.15.97:3300/0 conn(0x7f7a800d3760 0x7f7a800d3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a7ffff640 1 -- 172.21.15.31:0/3492395598 >> v1:172.21.15.97:6789/0 conn(0x7f7a8015d880 legacy=0x7f7a800b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a7ffff640 1 -- 172.21.15.31:0/3492395598 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a800ce000 con 0x7f7a800cfa20 2023-11-26T07:18:54.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.951+0000 7f7a7e7fc640 1 -- 172.21.15.31:0/3492395598 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861682783 0 0) 0x7f7a800d15f0 con 0x7f7a8015d880 2023-11-26T07:18:54.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a7ffff640 1 --2- 172.21.15.31:0/3492395598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 0x7f7a800cfe90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009e50 tx=0x7f7a7801ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf4c047db24897d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a7e7fc640 1 -- 172.21.15.31:0/3492395598 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a780a58e0 con 0x7f7a800cfa20 2023-11-26T07:18:54.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a7e7fc640 1 -- 172.21.15.31:0/3492395598 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a780a5a40 con 0x7f7a800cfa20 2023-11-26T07:18:54.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a7e7fc640 1 -- 172.21.15.31:0/3492395598 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a780a5dd0 con 0x7f7a800cfa20 2023-11-26T07:18:54.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3492395598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 msgr2=0x7f7a800cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a84b5f640 1 --2- 172.21.15.31:0/3492395598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 0x7f7a800cfe90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a78009e50 tx=0x7f7a7801ca10 comp rx=0 tx=0).stop 2023-11-26T07:18:54.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3492395598 shutdown_connections 2023-11-26T07:18:54.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a84b5f640 1 --2- 172.21.15.31:0/3492395598 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 0x7f7a800cfe90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a84b5f640 1 --2- 172.21.15.31:0/3492395598 >> v2:172.21.15.97:3300/0 conn(0x7f7a800d3760 0x7f7a800d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3492395598 >> 172.21.15.31:0/3492395598 conn(0x7f7a8006b9f0 msgr2=0x7f7a8006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:54.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3492395598 shutdown_connections 2023-11-26T07:18:54.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3492395598 wait complete. 2023-11-26T07:18:54.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a84b5f640 1 Processor -- start 2023-11-26T07:18:54.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.955+0000 7f7a84b5f640 1 -- start start 2023-11-26T07:18:54.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a84b5f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 0x7f7a8015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a84b5f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800d3760 0x7f7a800b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a84b5f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8015d880 0x7f7a800b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a84b5f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801572d0 con 0x7f7a8015d880 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a84b5f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80155ef0 con 0x7f7a800d3760 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a84b5f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8015d070 con 0x7f7a800cfa20 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8015d880 0x7f7a800b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8015d880 0x7f7a800b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51116/0 (socket says 172.21.15.31:51116) 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7ffff640 1 -- 172.21.15.31:0/3704894439 learned_addr learned my addr 172.21.15.31:0/3704894439 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7f7fe640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 0x7f7a8015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7effd640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800d3760 0x7f7a800b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7ffff640 1 -- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 msgr2=0x7f7a8015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7ffff640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 0x7f7a8015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7ffff640 1 -- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800d3760 msgr2=0x7f7a800b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7ffff640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800d3760 0x7f7a800b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:54.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7ffff640 1 -- 172.21.15.31:0/3704894439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a78009b30 con 0x7f7a8015d880 2023-11-26T07:18:54.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a7ffff640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8015d880 0x7f7a800b2ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7a780af040 tx=0x7f7a780197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a67fff640 1 -- 172.21.15.31:0/3704894439 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a780b8670 con 0x7f7a8015d880 2023-11-26T07:18:54.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a67fff640 1 -- 172.21.15.31:0/3704894439 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a780b87f0 con 0x7f7a8015d880 2023-11-26T07:18:54.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a67fff640 1 -- 172.21.15.31:0/3704894439 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a780b8ba0 con 0x7f7a8015d880 2023-11-26T07:18:54.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3704894439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a80156d50 con 0x7f7a8015d880 2023-11-26T07:18:54.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3704894439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a80156ff0 con 0x7f7a8015d880 2023-11-26T07:18:54.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.959+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3704894439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a80156060 con 0x7f7a8015d880 2023-11-26T07:18:54.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.963+0000 7f7a67fff640 1 -- 172.21.15.31:0/3704894439 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f7a780b8d00 con 0x7f7a8015d880 2023-11-26T07:18:54.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.963+0000 7f7a67fff640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a6c066620 0x7f7a6c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:54.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.963+0000 7f7a67fff640 1 -- 172.21.15.31:0/3704894439 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7a7811f3c0 con 0x7f7a8015d880 2023-11-26T07:18:54.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.963+0000 7f7a7f7fe640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a6c066620 0x7f7a6c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:54.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.963+0000 7f7a7f7fe640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a6c066620 0x7f7a6c068ae0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a70007fd0 tx=0x7f7a70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:54.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:54.967+0000 7f7a67fff640 1 -- 172.21.15.31:0/3704894439 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7a780ec8d0 con 0x7f7a8015d880 2023-11-26T07:18:55.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:55.175+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3704894439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f7a80074a00 con 0x7f7a8015d880 2023-11-26T07:18:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.107+0000 7f7a67fff640 1 -- 172.21.15.31:0/3704894439 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v102) v1 ==== 143+0+0 (secure 0 0 0) 0x7f7a780efef0 con 0x7f7a8015d880 2023-11-26T07:18:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a6c066620 msgr2=0x7f7a6c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a6c066620 0x7f7a6c068ae0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7a70007fd0 tx=0x7f7a70009040 comp rx=0 tx=0).stop 2023-11-26T07:18:56.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3704894439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8015d880 msgr2=0x7f7a800b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8015d880 0x7f7a800b2ff0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7a780af040 tx=0x7f7a780197b0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3704894439 shutdown_connections 2023-11-26T07:18:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7a6c066620 0x7f7a6c068ae0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a8015d880 0x7f7a800b2ff0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a800d3760 0x7f7a800b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 --2- 172.21.15.31:0/3704894439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7a800cfa20 0x7f7a8015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3704894439 >> 172.21.15.31:0/3704894439 conn(0x7f7a8006b9f0 msgr2=0x7f7a80161670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3704894439 shutdown_connections 2023-11-26T07:18:56.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.111+0000 7f7a84b5f640 1 -- 172.21.15.31:0/3704894439 wait complete. 2023-11-26T07:18:56.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:18:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91c0d57640 1 Processor -- start 2023-11-26T07:18:56.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91c0d57640 1 -- start start 2023-11-26T07:18:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91c0d57640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f91bc15a710 0x7f91bc0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91c0d57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cbd50 0x7f91bc0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91c0d57640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc0b2710 con 0x7f91bc0cdfb0 2023-11-26T07:18:56.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91c0d57640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91bc0cca80 con 0x7f91bc15a710 2023-11-26T07:18:56.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91c0d57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0b29f0 con 0x7f91bc0cbd50 2023-11-26T07:18:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91bad9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cbd50 0x7f91bc0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91bad9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cbd50 0x7f91bc0cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:60052/0 (socket says 172.21.15.31:60052) 2023-11-26T07:18:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91bad9d640 1 -- 172.21.15.31:0/1107050534 learned_addr learned my addr 172.21.15.31:0/1107050534 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91b3fff640 1 --2- 172.21.15.31:0/1107050534 >> v2:172.21.15.97:3300/0 conn(0x7f91bc15a710 0x7f91bc0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91bad9d640 1 -- 172.21.15.31:0/1107050534 >> v2:172.21.15.97:3300/0 conn(0x7f91bc15a710 msgr2=0x7f91bc0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91bad9d640 1 --2- 172.21.15.31:0/1107050534 >> v2:172.21.15.97:3300/0 conn(0x7f91bc15a710 0x7f91bc0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91bad9d640 1 -- 172.21.15.31:0/1107050534 >> v1:172.21.15.97:6789/0 conn(0x7f91bc0cdfb0 legacy=0x7f91bc0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.287+0000 7f91bad9d640 1 -- 172.21.15.31:0/1107050534 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc0b2cd0 con 0x7f91bc0cbd50 2023-11-26T07:18:56.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91bad9d640 1 --2- 172.21.15.31:0/1107050534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cbd50 0x7f91bc0cc1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009e50 tx=0x7f91b401ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b00284af2aa2a951 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91b9d9b640 1 -- 172.21.15.31:0/1107050534 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b40a7ae0 con 0x7f91bc0cbd50 2023-11-26T07:18:56.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91b9d9b640 1 -- 172.21.15.31:0/1107050534 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b40a7c40 con 0x7f91bc0cbd50 2023-11-26T07:18:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91b9d9b640 1 -- 172.21.15.31:0/1107050534 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b40af3f0 con 0x7f91bc0cbd50 2023-11-26T07:18:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91c0d57640 1 -- 172.21.15.31:0/1107050534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cbd50 msgr2=0x7f91bc0cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91c0d57640 1 --2- 172.21.15.31:0/1107050534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cbd50 0x7f91bc0cc1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009e50 tx=0x7f91b401ebf0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91c0d57640 1 -- 172.21.15.31:0/1107050534 shutdown_connections 2023-11-26T07:18:56.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91c0d57640 1 --2- 172.21.15.31:0/1107050534 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cbd50 0x7f91bc0cc1e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91c0d57640 1 --2- 172.21.15.31:0/1107050534 >> v2:172.21.15.97:3300/0 conn(0x7f91bc15a710 0x7f91bc0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91c0d57640 1 -- 172.21.15.31:0/1107050534 >> 172.21.15.31:0/1107050534 conn(0x7f91bc0b0cb0 msgr2=0x7f91bc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:56.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91c0d57640 1 -- 172.21.15.31:0/1107050534 shutdown_connections 2023-11-26T07:18:56.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.291+0000 7f91c0d57640 1 -- 172.21.15.31:0/1107050534 wait complete. 2023-11-26T07:18:56.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91c0d57640 1 Processor -- start 2023-11-26T07:18:56.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91c0d57640 1 -- start start 2023-11-26T07:18:56.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91c0d57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91bc0cbd50 0x7f91bc155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91c0d57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cdfb0 0x7f91bc154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91c0d57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc15a710 0x7f91bc15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91c0d57640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc1574d0 con 0x7f91bc0cbd50 2023-11-26T07:18:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91c0d57640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc155340 con 0x7f91bc15a710 2023-11-26T07:18:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91c0d57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc15d390 con 0x7f91bc0cdfb0 2023-11-26T07:18:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91ba59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91bc0cbd50 0x7f91bc155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91ba59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91bc0cbd50 0x7f91bc155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51122/0 (socket says 172.21.15.31:51122) 2023-11-26T07:18:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91ba59c640 1 -- 172.21.15.31:0/1827034163 learned_addr learned my addr 172.21.15.31:0/1827034163 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91bad9d640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc15a710 0x7f91bc15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91b3fff640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cdfb0 0x7f91bc154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91ba59c640 1 -- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cdfb0 msgr2=0x7f91bc154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91ba59c640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cdfb0 0x7f91bc154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91ba59c640 1 -- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc15a710 msgr2=0x7f91bc15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91ba59c640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc15a710 0x7f91bc15ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91ba59c640 1 -- 172.21.15.31:0/1827034163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b4009b30 con 0x7f91bc0cbd50 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91bad9d640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc15a710 0x7f91bc15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91ba59c640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91bc0cbd50 0x7f91bc155530 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00c560 tx=0x7f91ac00ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91b2ffd640 1 -- 172.21.15.31:0/1827034163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac005220 con 0x7f91bc0cbd50 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91b2ffd640 1 -- 172.21.15.31:0/1827034163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac005380 con 0x7f91bc0cbd50 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91b2ffd640 1 -- 172.21.15.31:0/1827034163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac007270 con 0x7f91bc0cbd50 2023-11-26T07:18:56.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91c0d57640 1 -- 172.21.15.31:0/1827034163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc155da0 con 0x7f91bc0cbd50 2023-11-26T07:18:56.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.295+0000 7f91c0d57640 1 -- 172.21.15.31:0/1827034163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc156ba0 con 0x7f91bc0cbd50 2023-11-26T07:18:56.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.299+0000 7f91c0d57640 1 -- 172.21.15.31:0/1827034163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc0671f0 con 0x7f91bc0cbd50 2023-11-26T07:18:56.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.299+0000 7f91b2ffd640 1 -- 172.21.15.31:0/1827034163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f91ac0073d0 con 0x7f91bc0cbd50 2023-11-26T07:18:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.299+0000 7f91b2ffd640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f919c066650 0x7f919c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.299+0000 7f91b2ffd640 1 -- 172.21.15.31:0/1827034163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f91ac083660 con 0x7f91bc0cbd50 2023-11-26T07:18:56.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.299+0000 7f91b3fff640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f919c066650 0x7f919c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.299+0000 7f91b3fff640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f919c066650 0x7f919c068b10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0cb670 tx=0x7f91a4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.303+0000 7f91b2ffd640 1 -- 172.21.15.31:0/1827034163 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f91ac050c00 con 0x7f91bc0cbd50 2023-11-26T07:18:56.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.463+0000 7f91c0d57640 1 -- 172.21.15.31:0/1827034163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f91bc06b460 con 0x7f91bc0cbd50 2023-11-26T07:18:56.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91b2ffd640 1 -- 172.21.15.31:0/1827034163 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91ac054220 con 0x7f91bc0cbd50 2023-11-26T07:18:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 -- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f919c066650 msgr2=0x7f919c068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f919c066650 0x7f919c068b10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0cb670 tx=0x7f91a4009190 comp rx=0 tx=0).stop 2023-11-26T07:18:56.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 -- 172.21.15.31:0/1827034163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91bc0cbd50 msgr2=0x7f91bc155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91bc0cbd50 0x7f91bc155530 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00c560 tx=0x7f91ac00ca20 comp rx=0 tx=0).stop 2023-11-26T07:18:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 -- 172.21.15.31:0/1827034163 shutdown_connections 2023-11-26T07:18:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f919c066650 0x7f919c068b10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91bc15a710 0x7f91bc15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f91bc0cdfb0 0x7f91bc154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 --2- 172.21.15.31:0/1827034163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91bc0cbd50 0x7f91bc155530 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 -- 172.21.15.31:0/1827034163 >> 172.21.15.31:0/1827034163 conn(0x7f91bc0b0cb0 msgr2=0x7f91bc0ac2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 -- 172.21.15.31:0/1827034163 shutdown_connections 2023-11-26T07:18:56.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.467+0000 7f91c0d57640 1 -- 172.21.15.31:0/1827034163 wait complete. 2023-11-26T07:18:56.495 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:18:56.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:18:56.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:18:56.646 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:18:56.646 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:56.646 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:56.655 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:18:56.656 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:56.656 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:56.665 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:18:56.666 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:18:56.666 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:56.708 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:18:56.708 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:18:56.708 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:18:56.719 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:18:56.720 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:18:56.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:18:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5919be7640 1 Processor -- start 2023-11-26T07:18:56.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5919be7640 1 -- start start 2023-11-26T07:18:56.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5919be7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5914071cc0 0x7f59140b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5919be7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59140b2ee0 0x7f59140adee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5919be7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59140ad740 con 0x7f591406e410 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5919be7640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59140b3e20 con 0x7f59140b2ee0 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5919be7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591415dde0 con 0x7f5914071cc0 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5913fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59140b2ee0 0x7f59140adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5913fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59140b2ee0 0x7f59140adee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51128/0 (socket says 172.21.15.31:51128) 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5913fff640 1 -- 172.21.15.31:0/438292327 learned_addr learned my addr 172.21.15.31:0/438292327 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5912ffd640 1 --2- 172.21.15.31:0/438292327 >> v2:172.21.15.97:3300/0 conn(0x7f5914071cc0 0x7f59140b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5913fff640 1 -- 172.21.15.31:0/438292327 >> v2:172.21.15.97:3300/0 conn(0x7f5914071cc0 msgr2=0x7f59140b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5913fff640 1 --2- 172.21.15.31:0/438292327 >> v2:172.21.15.97:3300/0 conn(0x7f5914071cc0 0x7f59140b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5913fff640 1 -- 172.21.15.31:0/438292327 >> v1:172.21.15.97:6789/0 conn(0x7f591406e410 legacy=0x7f59140715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5913fff640 1 -- 172.21.15.31:0/438292327 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59140b3580 con 0x7f59140b2ee0 2023-11-26T07:18:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.887+0000 7f5912ffd640 1 --2- 172.21.15.31:0/438292327 >> v2:172.21.15.97:3300/0 conn(0x7f5914071cc0 0x7f59140b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:18:56.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f5913fff640 1 --2- 172.21.15.31:0/438292327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59140b2ee0 0x7f59140adee0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5914158020 tx=0x7f590c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d74b4c3bc3df31e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f59127fc640 1 -- 172.21.15.31:0/438292327 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c0a5740 con 0x7f59140b2ee0 2023-11-26T07:18:56.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f59127fc640 1 -- 172.21.15.31:0/438292327 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c0a58a0 con 0x7f59140b2ee0 2023-11-26T07:18:56.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f59127fc640 1 -- 172.21.15.31:0/438292327 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c0a5c30 con 0x7f59140b2ee0 2023-11-26T07:18:56.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f5919be7640 1 -- 172.21.15.31:0/438292327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59140b2ee0 msgr2=0x7f59140adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f5919be7640 1 --2- 172.21.15.31:0/438292327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59140b2ee0 0x7f59140adee0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5914158020 tx=0x7f590c01c990 comp rx=0 tx=0).stop 2023-11-26T07:18:56.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f5919be7640 1 -- 172.21.15.31:0/438292327 shutdown_connections 2023-11-26T07:18:56.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f5919be7640 1 --2- 172.21.15.31:0/438292327 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59140b2ee0 0x7f59140adee0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f5919be7640 1 --2- 172.21.15.31:0/438292327 >> v2:172.21.15.97:3300/0 conn(0x7f5914071cc0 0x7f59140b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f5919be7640 1 -- 172.21.15.31:0/438292327 >> 172.21.15.31:0/438292327 conn(0x7f59140b1e40 msgr2=0x7f5914070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:56.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f5919be7640 1 -- 172.21.15.31:0/438292327 shutdown_connections 2023-11-26T07:18:56.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.891+0000 7f5919be7640 1 -- 172.21.15.31:0/438292327 wait complete. 2023-11-26T07:18:56.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5919be7640 1 Processor -- start 2023-11-26T07:18:56.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5919be7640 1 -- start start 2023-11-26T07:18:56.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5919be7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f591406e410 0x7f5914148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5919be7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5914071cc0 0x7f591414aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5919be7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59140b2ee0 0x7f591414afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5919be7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591415eb70 con 0x7f5914071cc0 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5919be7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591414b8b0 con 0x7f59140b2ee0 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5919be7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591414bb90 con 0x7f591406e410 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5912ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5914071cc0 0x7f591414aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5912ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5914071cc0 0x7f591414aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51132/0 (socket says 172.21.15.31:51132) 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5912ffd640 1 -- 172.21.15.31:0/2438632337 learned_addr learned my addr 172.21.15.31:0/2438632337 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5913fff640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59140b2ee0 0x7f591414afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f59137fe640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f591406e410 0x7f5914148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5912ffd640 1 -- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f591406e410 msgr2=0x7f5914148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5912ffd640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f591406e410 0x7f5914148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5912ffd640 1 -- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59140b2ee0 msgr2=0x7f591414afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5912ffd640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59140b2ee0 0x7f591414afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5912ffd640 1 -- 172.21.15.31:0/2438632337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c009b30 con 0x7f5914071cc0 2023-11-26T07:18:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5912ffd640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5914071cc0 0x7f591414aa70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f590800c7c0 tx=0x7f590800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f58fbfff640 1 -- 172.21.15.31:0/2438632337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5908007a30 con 0x7f5914071cc0 2023-11-26T07:18:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f58fbfff640 1 -- 172.21.15.31:0/2438632337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5908007b90 con 0x7f5914071cc0 2023-11-26T07:18:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5919be7640 1 -- 172.21.15.31:0/2438632337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591414be70 con 0x7f5914071cc0 2023-11-26T07:18:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f58fbfff640 1 -- 172.21.15.31:0/2438632337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590800ce70 con 0x7f5914071cc0 2023-11-26T07:18:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.895+0000 7f5919be7640 1 -- 172.21.15.31:0/2438632337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591414b4f0 con 0x7f5914071cc0 2023-11-26T07:18:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.899+0000 7f5919be7640 1 -- 172.21.15.31:0/2438632337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591404e850 con 0x7f5914071cc0 2023-11-26T07:18:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.899+0000 7f58fbfff640 1 -- 172.21.15.31:0/2438632337 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f59080113f0 con 0x7f5914071cc0 2023-11-26T07:18:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.899+0000 7f58fbfff640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5900066620 0x7f5900068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.899+0000 7f58fbfff640 1 -- 172.21.15.31:0/2438632337 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5908084380 con 0x7f5914071cc0 2023-11-26T07:18:56.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.899+0000 7f59137fe640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5900066620 0x7f5900068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:56.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.899+0000 7f59137fe640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5900066620 0x7f5900068ae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5904007fd0 tx=0x7f5904007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:56.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:56.903+0000 7f58fbfff640 1 -- 172.21.15.31:0/2438632337 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5908051840 con 0x7f5914071cc0 2023-11-26T07:18:57.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 -- 172.21.15.31:0/2438632337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f591415ef90 con 0x7f5914071cc0 2023-11-26T07:18:57.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f58fbfff640 1 -- 172.21.15.31:0/2438632337 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f5908054e60 con 0x7f5914071cc0 2023-11-26T07:18:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:57.102 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3937918726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3937918726},{"type":"v1","addr":"172.21.15.97:6837","nonce":3937918726}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2083665480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2083665480},{"type":"v1","addr":"172.21.15.97:6835","nonce":2083665480}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4984,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1187139549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1187139549},{"type":"v1","addr":"172.21.15.31:6833","nonce":1187139549}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5538,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3113556084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3113556084},{"type":"v1","addr":"172.21.15.31:6835","nonce":3113556084}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2023-11-26T07:18:57.102 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2023-11-26T07:18:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 -- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5900066620 msgr2=0x7f5900068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5900066620 0x7f5900068ae0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5904007fd0 tx=0x7f5904007e40 comp rx=0 tx=0).stop 2023-11-26T07:18:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 -- 172.21.15.31:0/2438632337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5914071cc0 msgr2=0x7f591414aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5914071cc0 0x7f591414aa70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f590800c7c0 tx=0x7f590800cc80 comp rx=0 tx=0).stop 2023-11-26T07:18:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 -- 172.21.15.31:0/2438632337 shutdown_connections 2023-11-26T07:18:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5900066620 0x7f5900068ae0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59140b2ee0 0x7f591414afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5914071cc0 0x7f591414aa70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 --2- 172.21.15.31:0/2438632337 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f591406e410 0x7f5914148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.095+0000 7f5919be7640 1 -- 172.21.15.31:0/2438632337 >> 172.21.15.31:0/2438632337 conn(0x7f59140b1e40 msgr2=0x7f59140cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.099+0000 7f5919be7640 1 -- 172.21.15.31:0/2438632337 shutdown_connections 2023-11-26T07:18:57.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.099+0000 7f5919be7640 1 -- 172.21.15.31:0/2438632337 wait complete. 2023-11-26T07:18:57.124 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:18:57.124 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:18:57.125 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:18:57.125 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:57.125 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:18:57.125 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:18:57.125 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:18:57.126 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:57.126 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:18:57.126 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:18:57.126 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:18:57.126 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:57.126 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:18:57.127 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:18:57.127 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:18:57.127 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:18:57.136 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:18:57.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:18:57.138 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:18:57.138 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:18:57.141 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:18:57.141 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:18:57.142 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:18:57.142 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:18:57.186 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:18:57.186 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:57.189 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:18:57.189 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:57.189 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:18:57.190 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:57.192 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:18:57.192 DEBUG:teuthology.parallel:result is None 2023-11-26T07:18:57.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:18:57.275 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-11-26T07:18:57.278 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-11-26T07:18:57.284 INFO:tasks.ceph.mds.d.smithi097.stdout:starting mds.d at 2023-11-26T07:18:57.284 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2023-11-26T07:18:57.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.335+0000 7fddcfae3640 1 Processor -- start 2023-11-26T07:18:57.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.335+0000 7fddcfae3640 1 -- start start 2023-11-26T07:18:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.335+0000 7fddcfae3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddc80cc460 0x7fddc80abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.335+0000 7fddcfae3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc80b2710 con 0x7fddc815a710 2023-11-26T07:18:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.335+0000 7fddcfae3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc80b29f0 con 0x7fddc80cdfb0 2023-11-26T07:18:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.335+0000 7fddcfae3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc80cca80 con 0x7fddc80cc460 2023-11-26T07:18:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.335+0000 7fddce080640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddc80cc460 0x7fddc80abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.335+0000 7fddcd07e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fddc815a710 0x7fddc80cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42466/0 (socket says 172.21.15.31:42466) 2023-11-26T07:18:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.335+0000 7fddcd07e640 1 -- 172.21.15.31:0/879783393 learned_addr learned my addr 172.21.15.31:0/879783393 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.335+0000 7fddcd87f640 1 --1- 172.21.15.31:0/879783393 >> v1:172.21.15.97:6789/0 conn(0x7fddc80cdfb0 0x7fddc80ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:51832/0 (socket says 172.21.15.31:51832) 2023-11-26T07:18:57.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.339+0000 7fddce080640 1 -- 172.21.15.31:0/879783393 >> v1:172.21.15.97:6789/0 conn(0x7fddc80cdfb0 legacy=0x7fddc80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.339+0000 7fddce080640 1 -- 172.21.15.31:0/879783393 >> v1:172.21.15.31:6789/0 conn(0x7fddc815a710 legacy=0x7fddc80cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.339+0000 7fddce080640 1 -- 172.21.15.31:0/879783393 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc80b2cd0 con 0x7fddc80cc460 2023-11-26T07:18:57.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.339+0000 7fddcc87d640 1 -- 172.21.15.31:0/879783393 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949902373 0 0) 0x7fddc80b29f0 con 0x7fddc80cdfb0 2023-11-26T07:18:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.339+0000 7fddce080640 1 --2- 172.21.15.31:0/879783393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cc460 0x7fddc80abf90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009d90 tx=0x7fddc001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364747c52a558a91 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.339+0000 7fddcc87d640 1 -- 172.21.15.31:0/879783393 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc00a5a00 con 0x7fddc80cc460 2023-11-26T07:18:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.339+0000 7fddcc87d640 1 -- 172.21.15.31:0/879783393 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc00a5b60 con 0x7fddc80cc460 2023-11-26T07:18:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.339+0000 7fddcc87d640 1 -- 172.21.15.31:0/879783393 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc00ad440 con 0x7fddc80cc460 2023-11-26T07:18:57.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.339+0000 7fddcfae3640 1 -- 172.21.15.31:0/879783393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cc460 msgr2=0x7fddc80abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.339+0000 7fddcfae3640 1 --2- 172.21.15.31:0/879783393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cc460 0x7fddc80abf90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009d90 tx=0x7fddc001c920 comp rx=0 tx=0).stop 2023-11-26T07:18:57.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 -- 172.21.15.31:0/879783393 shutdown_connections 2023-11-26T07:18:57.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 --2- 172.21.15.31:0/879783393 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cc460 0x7fddc80abf90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 -- 172.21.15.31:0/879783393 >> 172.21.15.31:0/879783393 conn(0x7fddc80b0cb0 msgr2=0x7fddc80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:57.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 -- 172.21.15.31:0/879783393 shutdown_connections 2023-11-26T07:18:57.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 -- 172.21.15.31:0/879783393 wait complete. 2023-11-26T07:18:57.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 Processor -- start 2023-11-26T07:18:57.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 -- start start 2023-11-26T07:18:57.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddc80cc460 0x7fddc81f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cdfb0 0x7fddc809cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc815a710 0x7fddc809a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc815c6a0 con 0x7fddc80cdfb0 2023-11-26T07:18:57.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc809aef0 con 0x7fddc815a710 2023-11-26T07:18:57.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcfae3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc815c050 con 0x7fddc80cc460 2023-11-26T07:18:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcd07e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cdfb0 0x7fddc809cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcd07e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cdfb0 0x7fddc809cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51188/0 (socket says 172.21.15.31:51188) 2023-11-26T07:18:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcd07e640 1 -- 172.21.15.31:0/706858505 learned_addr learned my addr 172.21.15.31:0/706858505 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcd87f640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddc80cc460 0x7fddc81f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddce080640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc815a710 0x7fddc809a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcd07e640 1 -- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddc80cc460 msgr2=0x7fddc81f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcd07e640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddc80cc460 0x7fddc81f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcd07e640 1 -- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc815a710 msgr2=0x7fddc809a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:57.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcd07e640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc815a710 0x7fddc809a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcd07e640 1 -- 172.21.15.31:0/706858505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0009a70 con 0x7fddc80cdfb0 2023-11-26T07:18:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddce080640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc815a710 0x7fddc809a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:18:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.343+0000 7fddcd07e640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cdfb0 0x7fddc809cc20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fddb8001ec0 tx=0x7fddb800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.347+0000 7fddbe7fc640 1 -- 172.21.15.31:0/706858505 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8005cf0 con 0x7fddc80cdfb0 2023-11-26T07:18:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.347+0000 7fddbe7fc640 1 -- 172.21.15.31:0/706858505 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8005e50 con 0x7fddc80cdfb0 2023-11-26T07:18:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.347+0000 7fddbe7fc640 1 -- 172.21.15.31:0/706858505 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80072d0 con 0x7fddc80cdfb0 2023-11-26T07:18:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.347+0000 7fddcfae3640 1 -- 172.21.15.31:0/706858505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc815c330 con 0x7fddc80cdfb0 2023-11-26T07:18:57.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.347+0000 7fddcfae3640 1 -- 172.21.15.31:0/706858505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc809b1d0 con 0x7fddc80cdfb0 2023-11-26T07:18:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.347+0000 7fddcfae3640 1 -- 172.21.15.31:0/706858505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc80671f0 con 0x7fddc80cdfb0 2023-11-26T07:18:57.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.347+0000 7fddbe7fc640 1 -- 172.21.15.31:0/706858505 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fddb8007430 con 0x7fddc80cdfb0 2023-11-26T07:18:57.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.347+0000 7fddbe7fc640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddac066650 0x7fddac068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:57.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.347+0000 7fddbe7fc640 1 -- 172.21.15.31:0/706858505 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fddb80831c0 con 0x7fddc80cdfb0 2023-11-26T07:18:57.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.347+0000 7fddcd87f640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddac066650 0x7fddac068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:57.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.351+0000 7fddcd87f640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddac066650 0x7fddac068b10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fddc4007fd0 tx=0x7fddc4007d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:57.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.355+0000 7fddbe7fc640 1 -- 172.21.15.31:0/706858505 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fddb8050890 con 0x7fddc80cdfb0 2023-11-26T07:18:57.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:57.523+0000 7fddcfae3640 1 -- 172.21.15.31:0/706858505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fddc806b460 con 0x7fddc80cdfb0 2023-11-26T07:18:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddbe7fc640 1 -- 172.21.15.31:0/706858505 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) v1 ==== 97+0+0 (secure 0 0 0) 0x7fddb8053eb0 con 0x7fddc80cdfb0 2023-11-26T07:18:58.136 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-11-26T07:18:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddcfae3640 1 -- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddac066650 msgr2=0x7fddac068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddcfae3640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddac066650 0x7fddac068b10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fddc4007fd0 tx=0x7fddc4007d60 comp rx=0 tx=0).stop 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddcfae3640 1 -- 172.21.15.31:0/706858505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cdfb0 msgr2=0x7fddc809cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddcfae3640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cdfb0 0x7fddc809cc20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fddb8001ec0 tx=0x7fddb800c890 comp rx=0 tx=0).stop 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddcfae3640 1 -- 172.21.15.31:0/706858505 shutdown_connections 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddcfae3640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddac066650 0x7fddac068b10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddcfae3640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddc815a710 0x7fddc809a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddcfae3640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddc80cdfb0 0x7fddc809cc20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddcfae3640 1 --2- 172.21.15.31:0/706858505 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddc80cc460 0x7fddc81f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.131+0000 7fddcfae3640 1 -- 172.21.15.31:0/706858505 >> 172.21.15.31:0/706858505 conn(0x7fddc80b0cb0 msgr2=0x7fddc80b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.135+0000 7fddcfae3640 1 -- 172.21.15.31:0/706858505 shutdown_connections 2023-11-26T07:18:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.135+0000 7fddcfae3640 1 -- 172.21.15.31:0/706858505 wait complete. 2023-11-26T07:18:58.163 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:18:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2ca0fec640 1 Processor -- start 2023-11-26T07:18:58.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2ca0fec640 1 -- start start 2023-11-26T07:18:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2ca0fec640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2c9c0cead0 0x7f2c9c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2ca0fec640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9c0d22c0 con 0x7f2c9c0d1eb0 2023-11-26T07:18:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2ca0fec640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9c0cefd0 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2ca0fec640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c0d17e0 con 0x7f2c9c0cead0 2023-11-26T07:18:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2c9ad9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2c9c0d1eb0 0x7f2c9c06bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42472/0 (socket says 172.21.15.31:42472) 2023-11-26T07:18:58.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2c9ad9d640 1 -- 172.21.15.31:0/3138355918 learned_addr learned my addr 172.21.15.31:0/3138355918 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:18:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2c93fff640 1 --2- 172.21.15.31:0/3138355918 >> v2:172.21.15.31:3300/0 conn(0x7f2c9c0cead0 0x7f2c9c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2c99d9b640 1 -- 172.21.15.31:0/3138355918 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1160041914 0 0) 0x7f2c9c0d22c0 con 0x7f2c9c0d1eb0 2023-11-26T07:18:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2c93fff640 1 -- 172.21.15.31:0/3138355918 >> v1:172.21.15.97:6789/0 conn(0x7f2c9c15d7b0 legacy=0x7f2c9c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2c93fff640 1 -- 172.21.15.31:0/3138355918 >> v1:172.21.15.31:6789/0 conn(0x7f2c9c0d1eb0 legacy=0x7f2c9c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.303+0000 7f2c93fff640 1 -- 172.21.15.31:0/3138355918 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0cf2b0 con 0x7f2c9c0cead0 2023-11-26T07:18:58.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2c93fff640 1 --2- 172.21.15.31:0/3138355918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c0cead0 0x7f2c9c0d1970 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2c840097c0 tx=0x7f2c8401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf5dde6e5a2cbe3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:58.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2c99d9b640 1 -- 172.21.15.31:0/3138355918 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840ab600 con 0x7f2c9c0cead0 2023-11-26T07:18:58.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2c99d9b640 1 -- 172.21.15.31:0/3138355918 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c840a7050 con 0x7f2c9c0cead0 2023-11-26T07:18:58.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2c99d9b640 1 -- 172.21.15.31:0/3138355918 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840ab980 con 0x7f2c9c0cead0 2023-11-26T07:18:58.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2ca0fec640 1 -- 172.21.15.31:0/3138355918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c0cead0 msgr2=0x7f2c9c0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2ca0fec640 1 --2- 172.21.15.31:0/3138355918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c0cead0 0x7f2c9c0d1970 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2c840097c0 tx=0x7f2c8401c330 comp rx=0 tx=0).stop 2023-11-26T07:18:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2ca0fec640 1 -- 172.21.15.31:0/3138355918 shutdown_connections 2023-11-26T07:18:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2ca0fec640 1 --2- 172.21.15.31:0/3138355918 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c0cead0 0x7f2c9c0d1970 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2ca0fec640 1 -- 172.21.15.31:0/3138355918 >> 172.21.15.31:0/3138355918 conn(0x7f2c9c0d33b0 msgr2=0x7f2c9c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2ca0fec640 1 -- 172.21.15.31:0/3138355918 shutdown_connections 2023-11-26T07:18:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.307+0000 7f2ca0fec640 1 -- 172.21.15.31:0/3138355918 wait complete. 2023-11-26T07:18:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2ca0fec640 1 Processor -- start 2023-11-26T07:18:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2ca0fec640 1 -- start start 2023-11-26T07:18:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2ca0fec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0cead0 0x7f2c9c0b8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2ca0fec640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2c9c0d1eb0 0x7f2c9c0cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2ca0fec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c15d7b0 0x7f2c9c0c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2ca0fec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0b88d0 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2ca0fec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0c6ee0 con 0x7f2c9c0cead0 2023-11-26T07:18:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2ca0fec640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0c7130 con 0x7f2c9c0d1eb0 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c9ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c15d7b0 0x7f2c9c0c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c9ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c15d7b0 0x7f2c9c0c9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51206/0 (socket says 172.21.15.31:51206) 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c9ad9d640 1 -- 172.21.15.31:0/406476808 learned_addr learned my addr 172.21.15.31:0/406476808 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c93fff640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2c9c0d1eb0 0x7f2c9c0cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c9a59c640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0cead0 0x7f2c9c0b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c9ad9d640 1 -- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2c9c0d1eb0 msgr2=0x7f2c9c0cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c9ad9d640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2c9c0d1eb0 0x7f2c9c0cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c9ad9d640 1 -- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0cead0 msgr2=0x7f2c9c0b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c9ad9d640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0cead0 0x7f2c9c0b8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c9ad9d640 1 -- 172.21.15.31:0/406476808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c84009420 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c9ad9d640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c15d7b0 0x7f2c9c0c9210 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400c780 tx=0x7f2c9400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:58.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c92ffd640 1 -- 172.21.15.31:0/406476808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9400ce30 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c92ffd640 1 -- 172.21.15.31:0/406476808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9400a640 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2c92ffd640 1 -- 172.21.15.31:0/406476808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9400a9f0 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2ca0fec640 1 -- 172.21.15.31:0/406476808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c0c7410 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.311+0000 7f2ca0fec640 1 -- 172.21.15.31:0/406476808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c0c79d0 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.315+0000 7f2ca0fec640 1 -- 172.21.15.31:0/406476808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c0c7bc0 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.315+0000 7f2c92ffd640 1 -- 172.21.15.31:0/406476808 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2c9400ab50 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.315+0000 7f2c92ffd640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c7c066620 0x7f2c7c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.315+0000 7f2c92ffd640 1 -- 172.21.15.31:0/406476808 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2c94083190 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.315+0000 7f2c9a59c640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c7c066620 0x7f2c7c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.315+0000 7f2c9a59c640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c7c066620 0x7f2c7c068ae0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0b44a0 tx=0x7f2c8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.319+0000 7f2c92ffd640 1 -- 172.21.15.31:0/406476808 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c94050780 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 -- 172.21.15.31:0/406476808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2c9c04e850 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2c92ffd640 1 -- 172.21.15.31:0/406476808 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2c94053da0 con 0x7f2c9c15d7b0 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlRJmWGBAAuUAS68YFt9PaqDV8RFi+6w==", 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:18:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlIdH1JBAAmPlAQXq78ht4pJf6I7Cd5w==", 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlfuXoHxAAf7vKZr+LVZaN6HR3qiks5w==", 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlWmgkLBAAojAzX06O7kbwmj154qUGog==", 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDU8GJlc/bqEBAAueOQvSeK6MW+WOT746yQzA==", 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDX8GJlEaW6FxAA8q4rEKSJtIl/Hr2uCNSMUg==", 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDa8GJldV90HxAAQRgcLf8ABSU4Sfb3A1Xa4g==", 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDd8GJl2erPJhAAoC71eDwCOmskb+VCLYQyUw==", 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDj8GJllq08LxAAGuz9+Ou/ybWxdkN3PSMaEQ==", 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDm8GJlww4eNxAAjRaBq45nrNwJPlopyOhXNg==", 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDq8GJl2jxiAhAAhu2OULy84YRJYxTmShoVBg==", 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDt8GJld8HNCBAA3deA4HKTqRggbVc73Y8efg==", 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA==", 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlEBAQOhAAx5TsMN17L2fNm+j3yqUIOQ==", 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:18:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl/CN+NhAAeyry8gIOtxwLxCQb0t1eKw==", 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:18:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDR8GJlA9e7BBAATe3WYA79PgfLc5UHgKGkpw==", 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDP8GJlfT+4NhAACA0t23bQtWvdFLsB9X+X8w==", 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-11-26T07:18:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlnh/pExAA6qyPM7So2BLbxbGbquLZZA==", 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlYhb/DxAAx1G3Ve23IdZ7cX9nPb2WhA==", 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:18:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:18:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:18:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:18:58.560 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:18:58.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 -- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c7c066620 msgr2=0x7f2c7c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c7c066620 0x7f2c7c068ae0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0b44a0 tx=0x7f2c8c009040 comp rx=0 tx=0).stop 2023-11-26T07:18:58.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 -- 172.21.15.31:0/406476808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c15d7b0 msgr2=0x7f2c9c0c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c15d7b0 0x7f2c9c0c9210 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2c9400c780 tx=0x7f2c9400cc40 comp rx=0 tx=0).stop 2023-11-26T07:18:58.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 -- 172.21.15.31:0/406476808 shutdown_connections 2023-11-26T07:18:58.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2c7c066620 0x7f2c7c068ae0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c9c15d7b0 0x7f2c9c0c9210 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2c9c0d1eb0 0x7f2c9c0cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 --2- 172.21.15.31:0/406476808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c0cead0 0x7f2c9c0b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 -- 172.21.15.31:0/406476808 >> 172.21.15.31:0/406476808 conn(0x7f2c9c0d33b0 msgr2=0x7f2c9c0b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 -- 172.21.15.31:0/406476808 shutdown_connections 2023-11-26T07:18:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.539+0000 7f2ca0fec640 1 -- 172.21.15.31:0/406476808 wait complete. 2023-11-26T07:18:58.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:18:58.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facbe52a640 1 Processor -- start 2023-11-26T07:18:58.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facbe52a640 1 -- start start 2023-11-26T07:18:58.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facbe52a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facb8144c80 0x7facb80abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facbe52a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facb80ac460 0x7facb80706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facbe52a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb8145060 con 0x7facb8070c20 2023-11-26T07:18:58.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facbe52a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb8071fd0 con 0x7facb80ac460 2023-11-26T07:18:58.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facbe52a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb80ac8c0 con 0x7facb8144c80 2023-11-26T07:18:58.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facb77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facb80ac460 0x7facb80706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facb7fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facb8144c80 0x7facb80abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facb77fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facb80ac460 0x7facb80706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:38198/0 (socket says 172.21.15.31:38198) 2023-11-26T07:18:58.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facb77fe640 1 -- 172.21.15.31:0/1380177180 learned_addr learned my addr 172.21.15.31:0/1380177180 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:58.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facbcac7640 1 --1- 172.21.15.31:0/1380177180 >> v1:172.21.15.97:6789/0 conn(0x7facb8070c20 0x7facb815eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:51848/0 (socket says 172.21.15.31:51848) 2023-11-26T07:18:58.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facb77fe640 1 -- 172.21.15.31:0/1380177180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facb8144c80 msgr2=0x7facb80abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:58.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facb77fe640 1 --2- 172.21.15.31:0/1380177180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facb8144c80 0x7facb80abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facb77fe640 1 -- 172.21.15.31:0/1380177180 >> v1:172.21.15.97:6789/0 conn(0x7facb8070c20 legacy=0x7facb815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.703+0000 7facb77fe640 1 -- 172.21.15.31:0/1380177180 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8071450 con 0x7facb80ac460 2023-11-26T07:18:58.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facb77fe640 1 --2- 172.21.15.31:0/1380177180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb80ac460 0x7facb80706e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7facac009870 tx=0x7facac01c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68cca6dc048c8356 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:58.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facb6ffd640 1 -- 172.21.15.31:0/1380177180 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac0aa670 con 0x7facb80ac460 2023-11-26T07:18:58.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facb6ffd640 1 -- 172.21.15.31:0/1380177180 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facac0aa7d0 con 0x7facb80ac460 2023-11-26T07:18:58.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facb6ffd640 1 -- 172.21.15.31:0/1380177180 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facac0aab60 con 0x7facb80ac460 2023-11-26T07:18:58.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facbe52a640 1 -- 172.21.15.31:0/1380177180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb80ac460 msgr2=0x7facb80706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facbe52a640 1 --2- 172.21.15.31:0/1380177180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb80ac460 0x7facb80706e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7facac009870 tx=0x7facac01c510 comp rx=0 tx=0).stop 2023-11-26T07:18:58.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facbe52a640 1 -- 172.21.15.31:0/1380177180 shutdown_connections 2023-11-26T07:18:58.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facbe52a640 1 --2- 172.21.15.31:0/1380177180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb80ac460 0x7facb80706e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facbe52a640 1 --2- 172.21.15.31:0/1380177180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facb8144c80 0x7facb80abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facbe52a640 1 -- 172.21.15.31:0/1380177180 >> 172.21.15.31:0/1380177180 conn(0x7facb80b0c40 msgr2=0x7facb80b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:58.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facbe52a640 1 -- 172.21.15.31:0/1380177180 shutdown_connections 2023-11-26T07:18:58.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.707+0000 7facbe52a640 1 -- 172.21.15.31:0/1380177180 wait complete. 2023-11-26T07:18:58.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facbe52a640 1 Processor -- start 2023-11-26T07:18:58.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facbe52a640 1 -- start start 2023-11-26T07:18:58.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facbe52a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb8070c20 0x7facb8148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facbe52a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facb80ac460 0x7facb813ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facbe52a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8144c80 0x7facb813f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facbe52a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb814a2f0 con 0x7facb8070c20 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facbe52a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb813c4c0 con 0x7facb8144c80 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facbe52a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb81499c0 con 0x7facb80ac460 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb8070c20 0x7facb8148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb8070c20 0x7facb8148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51210/0 (socket says 172.21.15.31:51210) 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb7fff640 1 -- 172.21.15.31:0/481206314 learned_addr learned my addr 172.21.15.31:0/481206314 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facbcac7640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8144c80 0x7facb813f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb77fe640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facb80ac460 0x7facb813ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb7fff640 1 -- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facb80ac460 msgr2=0x7facb813ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb7fff640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facb80ac460 0x7facb813ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb7fff640 1 -- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8144c80 msgr2=0x7facb813f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb7fff640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8144c80 0x7facb813f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb7fff640 1 -- 172.21.15.31:0/481206314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac0094d0 con 0x7facb8070c20 2023-11-26T07:18:58.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb7fff640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb8070c20 0x7facb8148f30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faca800e8a0 tx=0x7faca800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb4ff9640 1 -- 172.21.15.31:0/481206314 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca8009510 con 0x7facb8070c20 2023-11-26T07:18:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb4ff9640 1 -- 172.21.15.31:0/481206314 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca800c940 con 0x7facb8070c20 2023-11-26T07:18:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facb4ff9640 1 -- 172.21.15.31:0/481206314 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca800ccb0 con 0x7facb8070c20 2023-11-26T07:18:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.711+0000 7facbe52a640 1 -- 172.21.15.31:0/481206314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb8149ca0 con 0x7facb8070c20 2023-11-26T07:18:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.715+0000 7facbe52a640 1 -- 172.21.15.31:0/481206314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb8149470 con 0x7facb8070c20 2023-11-26T07:18:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.715+0000 7facbe52a640 1 -- 172.21.15.31:0/481206314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb80b1ba0 con 0x7facb8070c20 2023-11-26T07:18:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.715+0000 7facb4ff9640 1 -- 172.21.15.31:0/481206314 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7faca8013470 con 0x7facb8070c20 2023-11-26T07:18:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.715+0000 7facb4ff9640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faca4066620 0x7faca4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.715+0000 7facb4ff9640 1 -- 172.21.15.31:0/481206314 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faca8083e60 con 0x7facb8070c20 2023-11-26T07:18:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.715+0000 7facb77fe640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faca4066620 0x7faca4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:58.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.719+0000 7facb77fe640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faca4066620 0x7faca4068ae0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7facac0ae000 tx=0x7facac0b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:58.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.719+0000 7facb4ff9640 1 -- 172.21.15.31:0/481206314 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faca8051470 con 0x7facb8070c20 2023-11-26T07:18:58.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.879+0000 7facbe52a640 1 -- 172.21.15.31:0/481206314 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facb804e850 con 0x7facb8070c20 2023-11-26T07:18:58.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.879+0000 7facb4ff9640 1 -- 172.21.15.31:0/481206314 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+12773 (secure 0 0 0) 0x7faca8054a90 con 0x7facb8070c20 2023-11-26T07:18:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:58.887 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:18:58.144383+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:18:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 -- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faca4066620 msgr2=0x7faca4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faca4066620 0x7faca4068ae0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7facac0ae000 tx=0x7facac0b5040 comp rx=0 tx=0).stop 2023-11-26T07:18:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 -- 172.21.15.31:0/481206314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb8070c20 msgr2=0x7facb8148f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb8070c20 0x7facb8148f30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faca800e8a0 tx=0x7faca800eca0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 -- 172.21.15.31:0/481206314 shutdown_connections 2023-11-26T07:18:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faca4066620 0x7faca4068ae0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facb8144c80 0x7facb813f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facb80ac460 0x7facb813ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 --2- 172.21.15.31:0/481206314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facb8070c20 0x7facb8148f30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:58.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 -- 172.21.15.31:0/481206314 >> 172.21.15.31:0/481206314 conn(0x7facb80b0c40 msgr2=0x7facb80b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:58.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 -- 172.21.15.31:0/481206314 shutdown_connections 2023-11-26T07:18:58.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:58.883+0000 7facbe52a640 1 -- 172.21.15.31:0/481206314 wait complete. 2023-11-26T07:18:58.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:18:58.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f24459c1640 1 Processor -- start 2023-11-26T07:18:59.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f24459c1640 1 -- start start 2023-11-26T07:18:59.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f24459c1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24400cdfb0 0x7f24400abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f24459c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24400cbd50 0x7f24400cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f24459c1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24400b2710 con 0x7f244015a710 2023-11-26T07:18:59.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f24459c1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24400cca80 con 0x7f24400cdfb0 2023-11-26T07:18:59.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f24459c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400b29f0 con 0x7f24400cbd50 2023-11-26T07:18:59.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243f7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24400cbd50 0x7f24400cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243effd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f24400cdfb0 0x7f24400abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243f7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24400cbd50 0x7f24400cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:60158/0 (socket says 172.21.15.31:60158) 2023-11-26T07:18:59.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243e7fc640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f244015a710 0x7f24400ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:51854/0 (socket says 172.21.15.31:51854) 2023-11-26T07:18:59.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243f7fe640 1 -- 172.21.15.31:0/1084653852 learned_addr learned my addr 172.21.15.31:0/1084653852 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:59.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243dffb640 1 -- 172.21.15.31:0/1084653852 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939021847 0 0) 0x7f24400b2710 con 0x7f244015a710 2023-11-26T07:18:59.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243dffb640 1 -- 172.21.15.31:0/1084653852 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2428005740 con 0x7f244015a710 2023-11-26T07:18:59.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243f7fe640 1 -- 172.21.15.31:0/1084653852 >> v2:172.21.15.97:3300/0 conn(0x7f24400cdfb0 msgr2=0x7f24400abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243f7fe640 1 --2- 172.21.15.31:0/1084653852 >> v2:172.21.15.97:3300/0 conn(0x7f24400cdfb0 0x7f24400abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243f7fe640 1 -- 172.21.15.31:0/1084653852 >> v1:172.21.15.97:6789/0 conn(0x7f244015a710 legacy=0x7f24400ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243f7fe640 1 -- 172.21.15.31:0/1084653852 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400b2cd0 con 0x7f24400cbd50 2023-11-26T07:18:59.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.051+0000 7f243effd640 1 --2- 172.21.15.31:0/1084653852 >> v2:172.21.15.97:3300/0 conn(0x7f24400cdfb0 0x7f24400abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:59.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f243f7fe640 1 --2- 172.21.15.31:0/1084653852 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24400cbd50 0x7f24400cc1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2438009e50 tx=0x7f243801c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c65eb433cdba5ec server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:59.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f243dffb640 1 -- 172.21.15.31:0/1084653852 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24380a58d0 con 0x7f24400cbd50 2023-11-26T07:18:59.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f243dffb640 1 -- 172.21.15.31:0/1084653852 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24380a5a30 con 0x7f24400cbd50 2023-11-26T07:18:59.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f243dffb640 1 -- 172.21.15.31:0/1084653852 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24380a5dc0 con 0x7f24400cbd50 2023-11-26T07:18:59.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f24459c1640 1 -- 172.21.15.31:0/1084653852 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24400cbd50 msgr2=0x7f24400cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f24459c1640 1 --2- 172.21.15.31:0/1084653852 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24400cbd50 0x7f24400cc1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2438009e50 tx=0x7f243801c9e0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f24459c1640 1 -- 172.21.15.31:0/1084653852 shutdown_connections 2023-11-26T07:18:59.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f24459c1640 1 --2- 172.21.15.31:0/1084653852 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f24400cbd50 0x7f24400cc1e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f24459c1640 1 --2- 172.21.15.31:0/1084653852 >> v2:172.21.15.97:3300/0 conn(0x7f24400cdfb0 0x7f24400abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f24459c1640 1 -- 172.21.15.31:0/1084653852 >> 172.21.15.31:0/1084653852 conn(0x7f24400b0cb0 msgr2=0x7f24400b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:59.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f24459c1640 1 -- 172.21.15.31:0/1084653852 shutdown_connections 2023-11-26T07:18:59.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.055+0000 7f24459c1640 1 -- 172.21.15.31:0/1084653852 wait complete. 2023-11-26T07:18:59.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 Processor -- start 2023-11-26T07:18:59.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 -- start start 2023-11-26T07:18:59.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24400cbd50 0x7f2440154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24400cdfb0 0x7f244014d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f244015a710 0x7f244015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401517d0 con 0x7f24400cdfb0 2023-11-26T07:18:59.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2440154bc0 con 0x7f24400cbd50 2023-11-26T07:18:59.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2440158e40 con 0x7f244015a710 2023-11-26T07:18:59.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24400cdfb0 0x7f244014d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24400cdfb0 0x7f244014d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51212/0 (socket says 172.21.15.31:51212) 2023-11-26T07:18:59.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243e7fc640 1 -- 172.21.15.31:0/2145089950 learned_addr learned my addr 172.21.15.31:0/2145089950 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:59.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243f7fe640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f244015a710 0x7f244015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243effd640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24400cbd50 0x7f2440154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243e7fc640 1 -- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f244015a710 msgr2=0x7f244015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243e7fc640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f244015a710 0x7f244015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243e7fc640 1 -- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24400cbd50 msgr2=0x7f2440154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243e7fc640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24400cbd50 0x7f2440154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243e7fc640 1 -- 172.21.15.31:0/2145089950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2430009420 con 0x7f24400cdfb0 2023-11-26T07:18:59.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f243e7fc640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24400cdfb0 0x7f244014d1b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f243400c460 tx=0x7f243400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:59.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f2427fff640 1 -- 172.21.15.31:0/2145089950 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434007480 con 0x7f24400cdfb0 2023-11-26T07:18:59.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f2427fff640 1 -- 172.21.15.31:0/2145089950 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2434007600 con 0x7f24400cdfb0 2023-11-26T07:18:59.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 -- 172.21.15.31:0/2145089950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2438009b30 con 0x7f24400cdfb0 2023-11-26T07:18:59.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f2427fff640 1 -- 172.21.15.31:0/2145089950 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434004a30 con 0x7f24400cdfb0 2023-11-26T07:18:59.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 -- 172.21.15.31:0/2145089950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244015d890 con 0x7f24400cdfb0 2023-11-26T07:18:59.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.059+0000 7f24459c1640 1 -- 172.21.15.31:0/2145089950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24400671f0 con 0x7f24400cdfb0 2023-11-26T07:18:59.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.063+0000 7f2427fff640 1 -- 172.21.15.31:0/2145089950 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2434004b90 con 0x7f24400cdfb0 2023-11-26T07:18:59.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.063+0000 7f2427fff640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f242c066620 0x7f242c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.063+0000 7f2427fff640 1 -- 172.21.15.31:0/2145089950 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f24340830d0 con 0x7f24400cdfb0 2023-11-26T07:18:59.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.063+0000 7f243effd640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f242c066620 0x7f242c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.063+0000 7f243effd640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f242c066620 0x7f242c068ae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2440154440 tx=0x7f2430009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:59.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.067+0000 7f2427fff640 1 -- 172.21.15.31:0/2145089950 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2434050770 con 0x7f24400cdfb0 2023-11-26T07:18:59.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 -- 172.21.15.31:0/2145089950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f244015eea0 con 0x7f24400cdfb0 2023-11-26T07:18:59.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f2427fff640 1 -- 172.21.15.31:0/2145089950 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2434053d90 con 0x7f24400cdfb0 2023-11-26T07:18:59.233 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:18:59.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 -- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f242c066620 msgr2=0x7f242c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f242c066620 0x7f242c068ae0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2440154440 tx=0x7f2430009190 comp rx=0 tx=0).stop 2023-11-26T07:18:59.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 -- 172.21.15.31:0/2145089950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24400cdfb0 msgr2=0x7f244014d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24400cdfb0 0x7f244014d1b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f243400c460 tx=0x7f243400c920 comp rx=0 tx=0).stop 2023-11-26T07:18:59.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 -- 172.21.15.31:0/2145089950 shutdown_connections 2023-11-26T07:18:59.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f242c066620 0x7f242c068ae0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f244015a710 0x7f244015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24400cdfb0 0x7f244014d1b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 --2- 172.21.15.31:0/2145089950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24400cbd50 0x7f2440154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 -- 172.21.15.31:0/2145089950 >> 172.21.15.31:0/2145089950 conn(0x7f24400b0cb0 msgr2=0x7f24400ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:59.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 -- 172.21.15.31:0/2145089950 shutdown_connections 2023-11-26T07:18:59.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.227+0000 7f24459c1640 1 -- 172.21.15.31:0/2145089950 wait complete. 2023-11-26T07:18:59.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:18:59.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d6b05a640 1 Processor -- start 2023-11-26T07:18:59.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d6b05a640 1 -- start start 2023-11-26T07:18:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d6b05a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d6415d7b0 0x7f2d640d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d6b05a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d640cead0 0x7f2d640d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d6b05a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d640cefd0 con 0x7f2d640d1eb0 2023-11-26T07:18:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d6b05a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d640d2340 con 0x7f2d640cead0 2023-11-26T07:18:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d6b05a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640cf2b0 con 0x7f2d6415d7b0 2023-11-26T07:18:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d63fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d640cead0 0x7f2d640d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d63fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d640cead0 0x7f2d640d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51214/0 (socket says 172.21.15.31:51214) 2023-11-26T07:18:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d63fff640 1 -- 172.21.15.31:0/3950276028 learned_addr learned my addr 172.21.15.31:0/3950276028 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d68df6640 1 --2- 172.21.15.31:0/3950276028 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d6415d7b0 0x7f2d640d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d63fff640 1 -- 172.21.15.31:0/3950276028 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d6415d7b0 msgr2=0x7f2d640d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:59.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d63fff640 1 --2- 172.21.15.31:0/3950276028 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d6415d7b0 0x7f2d640d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d63fff640 1 -- 172.21.15.31:0/3950276028 >> v1:172.21.15.97:6789/0 conn(0x7f2d640d1eb0 legacy=0x7f2d640cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.391+0000 7f2d63fff640 1 -- 172.21.15.31:0/3950276028 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6406b3f0 con 0x7f2d640cead0 2023-11-26T07:18:59.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d63fff640 1 --2- 172.21.15.31:0/3950276028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640cead0 0x7f2d640d1970 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009870 tx=0x7f2d5001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af841c88d6cd1bd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:59.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d637fe640 1 -- 172.21.15.31:0/3950276028 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d500ab5f0 con 0x7f2d640cead0 2023-11-26T07:18:59.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d637fe640 1 -- 172.21.15.31:0/3950276028 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d500ab750 con 0x7f2d640cead0 2023-11-26T07:18:59.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d637fe640 1 -- 172.21.15.31:0/3950276028 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d500abae0 con 0x7f2d640cead0 2023-11-26T07:18:59.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3950276028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640cead0 msgr2=0x7f2d640d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d6b05a640 1 --2- 172.21.15.31:0/3950276028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640cead0 0x7f2d640d1970 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2d50009870 tx=0x7f2d5001c3f0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3950276028 shutdown_connections 2023-11-26T07:18:59.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d6b05a640 1 --2- 172.21.15.31:0/3950276028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640cead0 0x7f2d640d1970 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d6b05a640 1 --2- 172.21.15.31:0/3950276028 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d6415d7b0 0x7f2d640d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3950276028 >> 172.21.15.31:0/3950276028 conn(0x7f2d640d33b0 msgr2=0x7f2d640d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:59.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.395+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3950276028 shutdown_connections 2023-11-26T07:18:59.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3950276028 wait complete. 2023-11-26T07:18:59.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d6b05a640 1 Processor -- start 2023-11-26T07:18:59.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d6b05a640 1 -- start start 2023-11-26T07:18:59.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d6b05a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d640cead0 0x7f2d640cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d6b05a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640d1eb0 0x7f2d64153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d6b05a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d6415d7b0 0x7f2d640b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d6b05a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640ce4c0 con 0x7f2d640d1eb0 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d6b05a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640b9270 con 0x7f2d640cead0 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d6b05a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640b81d0 con 0x7f2d6415d7b0 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640d1eb0 0x7f2d64153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d63fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640d1eb0 0x7f2d64153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51218/0 (socket says 172.21.15.31:51218) 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d63fff640 1 -- 172.21.15.31:0/3499196048 learned_addr learned my addr 172.21.15.31:0/3499196048 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d68df6640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d640cead0 0x7f2d640cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d695f7640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d6415d7b0 0x7f2d640b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d63fff640 1 -- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d6415d7b0 msgr2=0x7f2d640b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d63fff640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d6415d7b0 0x7f2d640b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d63fff640 1 -- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d640cead0 msgr2=0x7f2d640cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d63fff640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d640cead0 0x7f2d640cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d63fff640 1 -- 172.21.15.31:0/3499196048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d500094d0 con 0x7f2d640d1eb0 2023-11-26T07:18:59.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d63fff640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640d1eb0 0x7f2d64153ac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2d5001a020 tx=0x7f2d5001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:59.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.399+0000 7f2d617fa640 1 -- 172.21.15.31:0/3499196048 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d500be5f0 con 0x7f2d640d1eb0 2023-11-26T07:18:59.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.403+0000 7f2d617fa640 1 -- 172.21.15.31:0/3499196048 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d500be750 con 0x7f2d640d1eb0 2023-11-26T07:18:59.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.403+0000 7f2d617fa640 1 -- 172.21.15.31:0/3499196048 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d500beb00 con 0x7f2d640d1eb0 2023-11-26T07:18:59.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.403+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3499196048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d640b84b0 con 0x7f2d640d1eb0 2023-11-26T07:18:59.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.403+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3499196048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d640b8a70 con 0x7f2d640d1eb0 2023-11-26T07:18:59.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.403+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3499196048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6404e850 con 0x7f2d640d1eb0 2023-11-26T07:18:59.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.403+0000 7f2d617fa640 1 -- 172.21.15.31:0/3499196048 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2d500b43f0 con 0x7f2d640d1eb0 2023-11-26T07:18:59.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.403+0000 7f2d617fa640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d48066650 0x7f2d48068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.403+0000 7f2d617fa640 1 -- 172.21.15.31:0/3499196048 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2d500a8050 con 0x7f2d640d1eb0 2023-11-26T07:18:59.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.403+0000 7f2d68df6640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d48066650 0x7f2d48068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.407+0000 7f2d68df6640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d48066650 0x7f2d48068b10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009b60 tx=0x7f2d58009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:59.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.407+0000 7f2d617fa640 1 -- 172.21.15.31:0/3499196048 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d500ebea0 con 0x7f2d640d1eb0 2023-11-26T07:18:59.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3499196048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2d64156f00 con 0x7f2d640d1eb0 2023-11-26T07:18:59.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d617fa640 1 -- 172.21.15.31:0/3499196048 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2d500ef4c0 con 0x7f2d640d1eb0 2023-11-26T07:18:59.577 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:18:59.577 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:18:59.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:18:59.577 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-11-26T07:18:59.578 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-11-26T07:18:59.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:18:59.578 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:18:59.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d48066650 msgr2=0x7f2d48068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d48066650 0x7f2d48068b10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009b60 tx=0x7f2d58009190 comp rx=0 tx=0).stop 2023-11-26T07:18:59.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3499196048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640d1eb0 msgr2=0x7f2d64153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640d1eb0 0x7f2d64153ac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2d5001a020 tx=0x7f2d5001ac30 comp rx=0 tx=0).stop 2023-11-26T07:18:59.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3499196048 shutdown_connections 2023-11-26T07:18:59.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d48066650 0x7f2d48068b10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d6415d7b0 0x7f2d640b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d640d1eb0 0x7f2d64153ac0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 --2- 172.21.15.31:0/3499196048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d640cead0 0x7f2d640cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3499196048 >> 172.21.15.31:0/3499196048 conn(0x7f2d640d33b0 msgr2=0x7f2d640b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:59.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3499196048 shutdown_connections 2023-11-26T07:18:59.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.571+0000 7f2d6b05a640 1 -- 172.21.15.31:0/3499196048 wait complete. 2023-11-26T07:18:59.599 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:18:59.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:18:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.739+0000 7f499572a640 1 Processor -- start 2023-11-26T07:18:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.739+0000 7f499572a640 1 -- start start 2023-11-26T07:18:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f499572a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4990161f20 0x7f49900cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f499572a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49900d02f0 0x7f49900cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f499572a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49900d0740 con 0x7f499015d7c0 2023-11-26T07:18:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f499572a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49900d0f90 con 0x7f4990161f20 2023-11-26T07:18:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f499572a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900d11a0 con 0x7f49900d02f0 2023-11-26T07:18:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f4987fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4990161f20 0x7f49900cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f498f7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49900d02f0 0x7f49900cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f4987fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4990161f20 0x7f49900cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:38230/0 (socket says 172.21.15.31:38230) 2023-11-26T07:18:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f4987fff640 1 -- 172.21.15.31:0/2162436657 learned_addr learned my addr 172.21.15.31:0/2162436657 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:59.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f4987fff640 1 -- 172.21.15.31:0/2162436657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49900d02f0 msgr2=0x7f49900cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:18:59.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f4987fff640 1 --2- 172.21.15.31:0/2162436657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49900d02f0 0x7f49900cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f4987fff640 1 -- 172.21.15.31:0/2162436657 >> v1:172.21.15.97:6789/0 conn(0x7f499015d7c0 legacy=0x7f49900d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f4987fff640 1 -- 172.21.15.31:0/2162436657 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900d1480 con 0x7f4990161f20 2023-11-26T07:18:59.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f498e7fc640 1 -- 172.21.15.31:0/2162436657 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759795641 0 0) 0x7f4980001120 con 0x7f499015d7c0 2023-11-26T07:18:59.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.743+0000 7f498f7fe640 1 --2- 172.21.15.31:0/2162436657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49900d02f0 0x7f49900cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:18:59.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f4987fff640 1 --2- 172.21.15.31:0/2162436657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4990161f20 0x7f49900cfdb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4978006fd0 tx=0x7f497801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e016bc866932f13 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:59.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f498e7fc640 1 -- 172.21.15.31:0/2162436657 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780ab660 con 0x7f4990161f20 2023-11-26T07:18:59.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f498e7fc640 1 -- 172.21.15.31:0/2162436657 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49780ab7c0 con 0x7f4990161f20 2023-11-26T07:18:59.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f498e7fc640 1 -- 172.21.15.31:0/2162436657 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780abb50 con 0x7f4990161f20 2023-11-26T07:18:59.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f499572a640 1 -- 172.21.15.31:0/2162436657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4990161f20 msgr2=0x7f49900cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f499572a640 1 --2- 172.21.15.31:0/2162436657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4990161f20 0x7f49900cfdb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4978006fd0 tx=0x7f497801c460 comp rx=0 tx=0).stop 2023-11-26T07:18:59.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f499572a640 1 -- 172.21.15.31:0/2162436657 shutdown_connections 2023-11-26T07:18:59.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f499572a640 1 --2- 172.21.15.31:0/2162436657 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49900d02f0 0x7f49900cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f499572a640 1 --2- 172.21.15.31:0/2162436657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4990161f20 0x7f49900cfdb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f499572a640 1 -- 172.21.15.31:0/2162436657 >> 172.21.15.31:0/2162436657 conn(0x7f499008afc0 msgr2=0x7f49900d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:18:59.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f499572a640 1 -- 172.21.15.31:0/2162436657 shutdown_connections 2023-11-26T07:18:59.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f499572a640 1 -- 172.21.15.31:0/2162436657 wait complete. 2023-11-26T07:18:59.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f499572a640 1 Processor -- start 2023-11-26T07:18:59.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.747+0000 7f499572a640 1 -- start start 2023-11-26T07:18:59.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f499572a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49900d02f0 0x7f499015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f499572a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f499015d7c0 0x7f4990153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f499572a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4990154410 0x7f4990156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f499572a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49901557c0 con 0x7f49900d02f0 2023-11-26T07:18:59.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f499572a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4990154270 con 0x7f4990154410 2023-11-26T07:18:59.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f499572a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4990154c40 con 0x7f499015d7c0 2023-11-26T07:18:59.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f498effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49900d02f0 0x7f499015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f498effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49900d02f0 0x7f499015bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51224/0 (socket says 172.21.15.31:51224) 2023-11-26T07:18:59.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f498effd640 1 -- 172.21.15.31:0/1298703032 learned_addr learned my addr 172.21.15.31:0/1298703032 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:18:59.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f4987fff640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f499015d7c0 0x7f4990153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f498f7fe640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4990154410 0x7f4990156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f498effd640 1 -- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f499015d7c0 msgr2=0x7f4990153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f498effd640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f499015d7c0 0x7f4990153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f498effd640 1 -- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4990154410 msgr2=0x7f4990156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:18:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f498effd640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4990154410 0x7f4990156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:18:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f498effd640 1 -- 172.21.15.31:0/1298703032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4988009a70 con 0x7f49900d02f0 2023-11-26T07:18:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f498effd640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49900d02f0 0x7f499015bd90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f498000c780 tx=0x7f498000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f49877fe640 1 -- 172.21.15.31:0/1298703032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980005270 con 0x7f49900d02f0 2023-11-26T07:18:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f49877fe640 1 -- 172.21.15.31:0/1298703032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49800053f0 con 0x7f49900d02f0 2023-11-26T07:18:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f49877fe640 1 -- 172.21.15.31:0/1298703032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980003840 con 0x7f49900d02f0 2023-11-26T07:18:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f499572a640 1 -- 172.21.15.31:0/1298703032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4978009480 con 0x7f49900d02f0 2023-11-26T07:18:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f499572a640 1 -- 172.21.15.31:0/1298703032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4990155200 con 0x7f49900d02f0 2023-11-26T07:18:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f499572a640 1 -- 172.21.15.31:0/1298703032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499004e850 con 0x7f49900d02f0 2023-11-26T07:18:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.751+0000 7f49877fe640 1 -- 172.21.15.31:0/1298703032 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f49800039a0 con 0x7f49900d02f0 2023-11-26T07:18:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.755+0000 7f49877fe640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4970066650 0x7f4970068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:18:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.755+0000 7f49877fe640 1 -- 172.21.15.31:0/1298703032 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4980083130 con 0x7f49900d02f0 2023-11-26T07:18:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.755+0000 7f4987fff640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4970066650 0x7f4970068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:18:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.755+0000 7f4987fff640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4970066650 0x7f4970068b10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f49780af000 tx=0x7f49780b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:18:59.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.759+0000 7f49877fe640 1 -- 172.21.15.31:0/1298703032 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4980050690 con 0x7f49900d02f0 2023-11-26T07:18:59.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:18:59.915+0000 7f499572a640 1 -- 172.21.15.31:0/1298703032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f49900d0740 con 0x7f49900d02f0 2023-11-26T07:19:00.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.143+0000 7f49877fe640 1 -- 172.21.15.31:0/1298703032 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f49900d0740 con 0x7f49900d02f0 2023-11-26T07:19:00.151 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-11-26T07:19:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 -- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4970066650 msgr2=0x7f4970068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4970066650 0x7f4970068b10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f49780af000 tx=0x7f49780b6040 comp rx=0 tx=0).stop 2023-11-26T07:19:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 -- 172.21.15.31:0/1298703032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49900d02f0 msgr2=0x7f499015bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49900d02f0 0x7f499015bd90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f498000c780 tx=0x7f498000cb80 comp rx=0 tx=0).stop 2023-11-26T07:19:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 -- 172.21.15.31:0/1298703032 shutdown_connections 2023-11-26T07:19:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4970066650 0x7f4970068b10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4990154410 0x7f4990156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:00.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f499015d7c0 0x7f4990153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:00.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 --2- 172.21.15.31:0/1298703032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49900d02f0 0x7f499015bd90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:00.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 -- 172.21.15.31:0/1298703032 >> 172.21.15.31:0/1298703032 conn(0x7f499008afc0 msgr2=0x7f499006ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:00.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 -- 172.21.15.31:0/1298703032 shutdown_connections 2023-11-26T07:19:00.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.147+0000 7f499572a640 1 -- 172.21.15.31:0/1298703032 wait complete. 2023-11-26T07:19:00.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:19:00.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:19:00.230 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:19:00.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0c005a640 1 Processor -- start 2023-11-26T07:19:00.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0c005a640 1 -- start start 2023-11-26T07:19:00.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0c005a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0b8154940 0x7ff0b806da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:00.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0c005a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0b815e6a0 0x7ff0b80ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:00.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0c005a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b80cd870 con 0x7ff0b806df50 2023-11-26T07:19:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0c005a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b8154d20 con 0x7ff0b8154940 2023-11-26T07:19:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0c005a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b815ecc0 con 0x7ff0b815e6a0 2023-11-26T07:19:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0bddf6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0b8154940 0x7ff0b806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0bddf6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0b8154940 0x7ff0b806da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51232/0 (socket says 172.21.15.31:51232) 2023-11-26T07:19:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0bddf6640 1 -- 172.21.15.31:0/578854017 learned_addr learned my addr 172.21.15.31:0/578854017 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0be5f7640 1 --2- 172.21.15.31:0/578854017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0b815e6a0 0x7ff0b80ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0bddf6640 1 -- 172.21.15.31:0/578854017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0b815e6a0 msgr2=0x7ff0b80ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0bddf6640 1 --2- 172.21.15.31:0/578854017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0b815e6a0 0x7ff0b80ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0bddf6640 1 -- 172.21.15.31:0/578854017 >> v1:172.21.15.97:6789/0 conn(0x7ff0b806df50 legacy=0x7ff0b815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:00.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.323+0000 7ff0bddf6640 1 -- 172.21.15.31:0/578854017 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b80ccf40 con 0x7ff0b8154940 2023-11-26T07:19:00.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0bddf6640 1 --2- 172.21.15.31:0/578854017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8154940 0x7ff0b806da10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009870 tx=0x7ff0b401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851fb794b61e6285 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:00.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0bcdf4640 1 -- 172.21.15.31:0/578854017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40ab600 con 0x7ff0b8154940 2023-11-26T07:19:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0bcdf4640 1 -- 172.21.15.31:0/578854017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b40a7050 con 0x7ff0b8154940 2023-11-26T07:19:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0bcdf4640 1 -- 172.21.15.31:0/578854017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40ab980 con 0x7ff0b8154940 2023-11-26T07:19:00.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0c005a640 1 -- 172.21.15.31:0/578854017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8154940 msgr2=0x7ff0b806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0c005a640 1 --2- 172.21.15.31:0/578854017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8154940 0x7ff0b806da10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009870 tx=0x7ff0b401c330 comp rx=0 tx=0).stop 2023-11-26T07:19:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0c005a640 1 -- 172.21.15.31:0/578854017 shutdown_connections 2023-11-26T07:19:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0c005a640 1 --2- 172.21.15.31:0/578854017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0b815e6a0 0x7ff0b80ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0c005a640 1 --2- 172.21.15.31:0/578854017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b8154940 0x7ff0b806da10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0c005a640 1 -- 172.21.15.31:0/578854017 >> 172.21.15.31:0/578854017 conn(0x7ff0b80b0cb0 msgr2=0x7ff0b80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.327+0000 7ff0c005a640 1 -- 172.21.15.31:0/578854017 shutdown_connections 2023-11-26T07:19:00.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0c005a640 1 -- 172.21.15.31:0/578854017 wait complete. 2023-11-26T07:19:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0c005a640 1 Processor -- start 2023-11-26T07:19:00.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0c005a640 1 -- start start 2023-11-26T07:19:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0c005a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0b806df50 0x7ff0b81473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0c005a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b8154940 0x7ff0b814ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:00.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0c005a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b815e6a0 0x7ff0b81f9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0c005a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b81478f0 con 0x7ff0b815e6a0 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0c005a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b814e430 con 0x7ff0b8154940 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0c005a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8150710 con 0x7ff0b806df50 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0be5f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b815e6a0 0x7ff0b81f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0bddf6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0b806df50 0x7ff0b81473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0be5f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b815e6a0 0x7ff0b81f9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51242/0 (socket says 172.21.15.31:51242) 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0be5f7640 1 -- 172.21.15.31:0/655891791 learned_addr learned my addr 172.21.15.31:0/655891791 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0bd5f5640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b8154940 0x7ff0b814ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0be5f7640 1 -- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0b806df50 msgr2=0x7ff0b81473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0be5f7640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0b806df50 0x7ff0b81473b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0be5f7640 1 -- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b8154940 msgr2=0x7ff0b814ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:00.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0be5f7640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b8154940 0x7ff0b814ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0be5f7640 1 -- 172.21.15.31:0/655891791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b40094d0 con 0x7ff0b815e6a0 2023-11-26T07:19:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0be5f7640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b815e6a0 0x7ff0b81f9830 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000efc0 tx=0x7ff0b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0ae7fc640 1 -- 172.21.15.31:0/655891791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0009ba0 con 0x7ff0b815e6a0 2023-11-26T07:19:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0ae7fc640 1 -- 172.21.15.31:0/655891791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b000f040 con 0x7ff0b815e6a0 2023-11-26T07:19:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0c005a640 1 -- 172.21.15.31:0/655891791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b814db40 con 0x7ff0b815e6a0 2023-11-26T07:19:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.331+0000 7ff0ae7fc640 1 -- 172.21.15.31:0/655891791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00146c0 con 0x7ff0b815e6a0 2023-11-26T07:19:00.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.335+0000 7ff0c005a640 1 -- 172.21.15.31:0/655891791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b814dde0 con 0x7ff0b815e6a0 2023-11-26T07:19:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.335+0000 7ff0c005a640 1 -- 172.21.15.31:0/655891791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b814e5c0 con 0x7ff0b815e6a0 2023-11-26T07:19:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.335+0000 7ff0ae7fc640 1 -- 172.21.15.31:0/655891791 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff0b0014820 con 0x7ff0b815e6a0 2023-11-26T07:19:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.335+0000 7ff0ae7fc640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff09c066620 0x7ff09c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:00.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.335+0000 7ff0ae7fc640 1 -- 172.21.15.31:0/655891791 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff0b0084aa0 con 0x7ff0b815e6a0 2023-11-26T07:19:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.335+0000 7ff0bddf6640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff09c066620 0x7ff09c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:00.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.335+0000 7ff0bddf6640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff09c066620 0x7ff09c068ae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40ad000 tx=0x7ff0b40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:00.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.339+0000 7ff0ae7fc640 1 -- 172.21.15.31:0/655891791 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0b0051e10 con 0x7ff0b815e6a0 2023-11-26T07:19:00.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:00.499+0000 7ff0c005a640 1 -- 172.21.15.31:0/655891791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff0b80671f0 con 0x7ff0b815e6a0 2023-11-26T07:19:01.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.151+0000 7ff0ae7fc640 1 -- 172.21.15.31:0/655891791 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff0b00180d0 con 0x7ff0b815e6a0 2023-11-26T07:19:01.159 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-11-26T07:19:01.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 -- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff09c066620 msgr2=0x7ff09c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff09c066620 0x7ff09c068ae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40ad000 tx=0x7ff0b40a6000 comp rx=0 tx=0).stop 2023-11-26T07:19:01.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 -- 172.21.15.31:0/655891791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b815e6a0 msgr2=0x7ff0b81f9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b815e6a0 0x7ff0b81f9830 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000efc0 tx=0x7ff0b000c490 comp rx=0 tx=0).stop 2023-11-26T07:19:01.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 -- 172.21.15.31:0/655891791 shutdown_connections 2023-11-26T07:19:01.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff09c066620 0x7ff09c068ae0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b815e6a0 0x7ff0b81f9830 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b8154940 0x7ff0b814ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 --2- 172.21.15.31:0/655891791 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0b806df50 0x7ff0b81473b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 -- 172.21.15.31:0/655891791 >> 172.21.15.31:0/655891791 conn(0x7ff0b80b0cb0 msgr2=0x7ff0b80ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:01.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 -- 172.21.15.31:0/655891791 shutdown_connections 2023-11-26T07:19:01.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.155+0000 7ff0c005a640 1 -- 172.21.15.31:0/655891791 wait complete. 2023-11-26T07:19:01.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:19:01.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.347+0000 7ffa68157640 1 Processor -- start 2023-11-26T07:19:01.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.347+0000 7ffa68157640 1 -- start start 2023-11-26T07:19:01.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.347+0000 7ffa68157640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffa600cdfb0 0x7ffa600abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.347+0000 7ffa68157640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa6015a710 0x7ffa600ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.347+0000 7ffa68157640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa600ce8d0 con 0x7ffa600cbd50 2023-11-26T07:19:01.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.347+0000 7ffa68157640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa600cca80 con 0x7ffa600cdfb0 2023-11-26T07:19:01.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.347+0000 7ffa68157640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600b2710 con 0x7ffa6015a710 2023-11-26T07:19:01.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa666f4640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffa600cbd50 0x7ffa600cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42486/0 (socket says 172.21.15.31:42486) 2023-11-26T07:19:01.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa666f4640 1 -- 172.21.15.31:0/162623980 learned_addr learned my addr 172.21.15.31:0/162623980 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa656f2640 1 --2- 172.21.15.31:0/162623980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa6015a710 0x7ffa600ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa65ef3640 1 --2- 172.21.15.31:0/162623980 >> v2:172.21.15.97:3300/0 conn(0x7ffa600cdfb0 0x7ffa600abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa64ef1640 1 -- 172.21.15.31:0/162623980 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418525575 0 0) 0x7ffa600ce8d0 con 0x7ffa600cbd50 2023-11-26T07:19:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa64ef1640 1 -- 172.21.15.31:0/162623980 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa4c003530 con 0x7ffa600cbd50 2023-11-26T07:19:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa656f2640 1 -- 172.21.15.31:0/162623980 >> v2:172.21.15.97:3300/0 conn(0x7ffa600cdfb0 msgr2=0x7ffa600abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa656f2640 1 --2- 172.21.15.31:0/162623980 >> v2:172.21.15.97:3300/0 conn(0x7ffa600cdfb0 0x7ffa600abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa656f2640 1 -- 172.21.15.31:0/162623980 >> v1:172.21.15.31:6789/0 conn(0x7ffa600cbd50 legacy=0x7ffa600cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa656f2640 1 -- 172.21.15.31:0/162623980 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600b29f0 con 0x7ffa6015a710 2023-11-26T07:19:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa65ef3640 1 --2- 172.21.15.31:0/162623980 >> v2:172.21.15.97:3300/0 conn(0x7ffa600cdfb0 0x7ffa600abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa656f2640 1 --2- 172.21.15.31:0/162623980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa6015a710 0x7ffa600ce390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa500097c0 tx=0x7ffa5001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33f9016bf1b565d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.351+0000 7ffa64ef1640 1 -- 172.21.15.31:0/162623980 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500ab650 con 0x7ffa6015a710 2023-11-26T07:19:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa64ef1640 1 -- 172.21.15.31:0/162623980 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa500ab7b0 con 0x7ffa6015a710 2023-11-26T07:19:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa64ef1640 1 -- 172.21.15.31:0/162623980 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500abb40 con 0x7ffa6015a710 2023-11-26T07:19:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 -- 172.21.15.31:0/162623980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa6015a710 msgr2=0x7ffa600ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 --2- 172.21.15.31:0/162623980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa6015a710 0x7ffa600ce390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa500097c0 tx=0x7ffa5001c450 comp rx=0 tx=0).stop 2023-11-26T07:19:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 -- 172.21.15.31:0/162623980 shutdown_connections 2023-11-26T07:19:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 --2- 172.21.15.31:0/162623980 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa6015a710 0x7ffa600ce390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 --2- 172.21.15.31:0/162623980 >> v2:172.21.15.97:3300/0 conn(0x7ffa600cdfb0 0x7ffa600abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 -- 172.21.15.31:0/162623980 >> 172.21.15.31:0/162623980 conn(0x7ffa600b0cb0 msgr2=0x7ffa600b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 -- 172.21.15.31:0/162623980 shutdown_connections 2023-11-26T07:19:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 -- 172.21.15.31:0/162623980 wait complete. 2023-11-26T07:19:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 Processor -- start 2023-11-26T07:19:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 -- start start 2023-11-26T07:19:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa600cbd50 0x7ffa60154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa600cdfb0 0x7ffa60156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa6015a710 0x7ffa601567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa601527b0 con 0x7ffa600cdfb0 2023-11-26T07:19:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa60156d00 con 0x7ffa6015a710 2023-11-26T07:19:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa68157640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa60151ba0 con 0x7ffa600cbd50 2023-11-26T07:19:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa656f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa600cdfb0 0x7ffa60156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa656f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa600cdfb0 0x7ffa60156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51254/0 (socket says 172.21.15.31:51254) 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa656f2640 1 -- 172.21.15.31:0/783763759 learned_addr learned my addr 172.21.15.31:0/783763759 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa65ef3640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa600cbd50 0x7ffa60154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.355+0000 7ffa666f4640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa6015a710 0x7ffa601567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa656f2640 1 -- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa600cbd50 msgr2=0x7ffa60154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa656f2640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa600cbd50 0x7ffa60154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa656f2640 1 -- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa6015a710 msgr2=0x7ffa601567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa656f2640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa6015a710 0x7ffa601567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa656f2640 1 -- 172.21.15.31:0/783763759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c009420 con 0x7ffa600cdfb0 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa666f4640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa6015a710 0x7ffa601567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa656f2640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa600cdfb0 0x7ffa60156280 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ffa500ad040 tx=0x7ffa5001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa567fc640 1 -- 172.21.15.31:0/783763759 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500bf610 con 0x7ffa600cdfb0 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa567fc640 1 -- 172.21.15.31:0/783763759 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa500bf7b0 con 0x7ffa600cdfb0 2023-11-26T07:19:01.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa68157640 1 -- 172.21.15.31:0/783763759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa50009420 con 0x7ffa600cdfb0 2023-11-26T07:19:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa567fc640 1 -- 172.21.15.31:0/783763759 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500bfb60 con 0x7ffa600cdfb0 2023-11-26T07:19:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa68157640 1 -- 172.21.15.31:0/783763759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa60152160 con 0x7ffa600cdfb0 2023-11-26T07:19:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa68157640 1 -- 172.21.15.31:0/783763759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa600671f0 con 0x7ffa600cdfb0 2023-11-26T07:19:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.359+0000 7ffa567fc640 1 -- 172.21.15.31:0/783763759 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ffa500b5400 con 0x7ffa600cdfb0 2023-11-26T07:19:01.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.363+0000 7ffa567fc640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa44066620 0x7ffa44068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:01.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.363+0000 7ffa567fc640 1 -- 172.21.15.31:0/783763759 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ffa5011f430 con 0x7ffa600cdfb0 2023-11-26T07:19:01.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.363+0000 7ffa65ef3640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa44066620 0x7ffa44068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.363+0000 7ffa65ef3640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa44066620 0x7ffa44068ae0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffa600b1340 tx=0x7ffa5c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:01.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.367+0000 7ffa567fc640 1 -- 172.21.15.31:0/783763759 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffa500ec8d0 con 0x7ffa600cdfb0 2023-11-26T07:19:01.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:01.555+0000 7ffa68157640 1 -- 172.21.15.31:0/783763759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ffa6006b460 con 0x7ffa600cdfb0 2023-11-26T07:19:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.155+0000 7ffa567fc640 1 -- 172.21.15.31:0/783763759 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) v1 ==== 172+0+0 (secure 0 0 0) 0x7ffa500efef0 con 0x7ffa600cdfb0 2023-11-26T07:19:02.161 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 6 and data pool 7 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 -- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa44066620 msgr2=0x7ffa44068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa44066620 0x7ffa44068ae0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ffa600b1340 tx=0x7ffa5c009190 comp rx=0 tx=0).stop 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 -- 172.21.15.31:0/783763759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa600cdfb0 msgr2=0x7ffa60156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa600cdfb0 0x7ffa60156280 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ffa500ad040 tx=0x7ffa5001ac90 comp rx=0 tx=0).stop 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 -- 172.21.15.31:0/783763759 shutdown_connections 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa44066620 0x7ffa44068ae0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.164 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa6015a710 0x7ffa601567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa600cdfb0 0x7ffa60156280 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 --2- 172.21.15.31:0/783763759 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa600cbd50 0x7ffa60154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 -- 172.21.15.31:0/783763759 >> 172.21.15.31:0/783763759 conn(0x7ffa600b0cb0 msgr2=0x7ffa60164610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 -- 172.21.15.31:0/783763759 shutdown_connections 2023-11-26T07:19:02.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.159+0000 7ffa68157640 1 -- 172.21.15.31:0/783763759 wait complete. 2023-11-26T07:19:02.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f032485a640 1 Processor -- start 2023-11-26T07:19:02.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f032485a640 1 -- start start 2023-11-26T07:19:02.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f032485a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f03201662f0 0x7f03201686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f032485a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03200b0a70 con 0x7f032015d8b0 2023-11-26T07:19:02.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f032485a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03200b4100 con 0x7f03200ce5c0 2023-11-26T07:19:02.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f032485a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03200cddb0 con 0x7f03201662f0 2023-11-26T07:19:02.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031e59c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f032015d8b0 0x7f03200b4be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42498/0 (socket says 172.21.15.31:42498) 2023-11-26T07:19:02.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031e59c640 1 -- 172.21.15.31:0/215342836 learned_addr learned my addr 172.21.15.31:0/215342836 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:02.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031ed9d640 1 --2- 172.21.15.31:0/215342836 >> v2:172.21.15.97:3300/0 conn(0x7f03201662f0 0x7f03201686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031d59a640 1 -- 172.21.15.31:0/215342836 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008221348 0 0) 0x7f03200b0a70 con 0x7f032015d8b0 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031d59a640 1 -- 172.21.15.31:0/215342836 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0308003530 con 0x7f032015d8b0 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031d59a640 1 -- 172.21.15.31:0/215342836 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479356147 0 0) 0x7f03200b4100 con 0x7f03200ce5c0 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031d59a640 1 -- 172.21.15.31:0/215342836 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03200b0a70 con 0x7f03200ce5c0 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031ed9d640 1 -- 172.21.15.31:0/215342836 >> v1:172.21.15.97:6789/0 conn(0x7f03200ce5c0 legacy=0x7f03200cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031ed9d640 1 -- 172.21.15.31:0/215342836 >> v1:172.21.15.31:6789/0 conn(0x7f032015d8b0 legacy=0x7f03200b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031d59a640 1 -- 172.21.15.31:0/215342836 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1132011948 0 0) 0x7f0308003530 con 0x7f032015d8b0 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.335+0000 7f031ed9d640 1 -- 172.21.15.31:0/215342836 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03200b43e0 con 0x7f03201662f0 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f031ed9d640 1 --2- 172.21.15.31:0/215342836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03201662f0 0x7f03201686e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0318009d90 tx=0x7f031801f060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e78134c2095c876 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f031d59a640 1 -- 172.21.15.31:0/215342836 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03100016a0 con 0x7f03201662f0 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f031d59a640 1 -- 172.21.15.31:0/215342836 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03180a8bf0 con 0x7f03201662f0 2023-11-26T07:19:02.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f031d59a640 1 -- 172.21.15.31:0/215342836 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180b0440 con 0x7f03201662f0 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f032485a640 1 -- 172.21.15.31:0/215342836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03201662f0 msgr2=0x7f03201686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f032485a640 1 --2- 172.21.15.31:0/215342836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03201662f0 0x7f03201686e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0318009d90 tx=0x7f031801f060 comp rx=0 tx=0).stop 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f032485a640 1 -- 172.21.15.31:0/215342836 shutdown_connections 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f032485a640 1 --2- 172.21.15.31:0/215342836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03201662f0 0x7f03201686e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f032485a640 1 -- 172.21.15.31:0/215342836 >> 172.21.15.31:0/215342836 conn(0x7f03200b5570 msgr2=0x7f03200b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f032485a640 1 -- 172.21.15.31:0/215342836 shutdown_connections 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.339+0000 7f032485a640 1 -- 172.21.15.31:0/215342836 wait complete. 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f032485a640 1 Processor -- start 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f032485a640 1 -- start start 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f032485a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03200ce5c0 0x7f032013d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f032485a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f032015d8b0 0x7f032013fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f032485a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03201662f0 0x7f032006e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f032485a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032013dac0 con 0x7f03201662f0 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f032485a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0320158ac0 con 0x7f03200ce5c0 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f032485a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0320158d80 con 0x7f032015d8b0 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03201662f0 0x7f032006e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03201662f0 0x7f032006e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51268/0 (socket says 172.21.15.31:51268) 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031ed9d640 1 -- 172.21.15.31:0/4093635845 learned_addr learned my addr 172.21.15.31:0/4093635845 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031e59c640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03200ce5c0 0x7f032013d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031dd9b640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f032015d8b0 0x7f032013fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031ed9d640 1 -- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f032015d8b0 msgr2=0x7f032013fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031ed9d640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f032015d8b0 0x7f032013fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031ed9d640 1 -- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03200ce5c0 msgr2=0x7f032013d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031ed9d640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03200ce5c0 0x7f032013d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031ed9d640 1 -- 172.21.15.31:0/4093635845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318009a70 con 0x7f03201662f0 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031dd9b640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f032015d8b0 0x7f032013fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f031ed9d640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03201662f0 0x7f032006e050 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0318009e80 tx=0x7f031801be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f030effd640 1 -- 172.21.15.31:0/4093635845 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180b0e30 con 0x7f03201662f0 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f030effd640 1 -- 172.21.15.31:0/4093635845 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0318009450 con 0x7f03201662f0 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f032485a640 1 -- 172.21.15.31:0/4093635845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0320140a80 con 0x7f03201662f0 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f030effd640 1 -- 172.21.15.31:0/4093635845 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180ba610 con 0x7f03201662f0 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.343+0000 7f032485a640 1 -- 172.21.15.31:0/4093635845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03201401e0 con 0x7f03201662f0 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.347+0000 7f032485a640 1 -- 172.21.15.31:0/4093635845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0320158f30 con 0x7f03201662f0 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.347+0000 7f030effd640 1 -- 172.21.15.31:0/4093635845 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f03180b7050 con 0x7f03201662f0 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.347+0000 7f030effd640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0300066620 0x7f0300068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.347+0000 7f030effd640 1 -- 172.21.15.31:0/4093635845 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03181223d0 con 0x7f03201662f0 2023-11-26T07:19:02.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.347+0000 7f031e59c640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0300066620 0x7f0300068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.347+0000 7f031e59c640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0300066620 0x7f0300068ae0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0310000fc0 tx=0x7f0310001b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.351+0000 7f030effd640 1 -- 172.21.15.31:0/4093635845 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0320158f30 con 0x7f03201662f0 2023-11-26T07:19:02.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.515+0000 7f032485a640 1 -- 172.21.15.31:0/4093635845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f032015b960 con 0x7f03201662f0 2023-11-26T07:19:02.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.515+0000 7f030effd640 1 -- 172.21.15.31:0/4093635845 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16442 (secure 0 0 0) 0x7f03180ef6e0 con 0x7f03201662f0 2023-11-26T07:19:02.523 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:02.523 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":43,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:19:01.563255+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:59.966450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:00.550730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 -- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0300066620 msgr2=0x7f0300068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0300066620 0x7f0300068ae0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0310000fc0 tx=0x7f0310001b10 comp rx=0 tx=0).stop 2023-11-26T07:19:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 -- 172.21.15.31:0/4093635845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03201662f0 msgr2=0x7f032006e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03201662f0 0x7f032006e050 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0318009e80 tx=0x7f031801be00 comp rx=0 tx=0).stop 2023-11-26T07:19:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 -- 172.21.15.31:0/4093635845 shutdown_connections 2023-11-26T07:19:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0300066620 0x7f0300068ae0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03201662f0 0x7f032006e050 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f032015d8b0 0x7f032013fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 --2- 172.21.15.31:0/4093635845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03200ce5c0 0x7f032013d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 -- 172.21.15.31:0/4093635845 >> 172.21.15.31:0/4093635845 conn(0x7f03200b5570 msgr2=0x7f03200cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 -- 172.21.15.31:0/4093635845 shutdown_connections 2023-11-26T07:19:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.519+0000 7f032485a640 1 -- 172.21.15.31:0/4093635845 wait complete. 2023-11-26T07:19:02.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:02.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.679+0000 7fab5812a640 1 Processor -- start 2023-11-26T07:19:02.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.679+0000 7fab5812a640 1 -- start start 2023-11-26T07:19:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab5812a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fab500ac630 0x7fab500706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab5812a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab50070c20 0x7fab5006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab5812a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab50145060 con 0x7fab50144c80 2023-11-26T07:19:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab5812a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab50071fd0 con 0x7fab50070c20 2023-11-26T07:19:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab5812a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab50071450 con 0x7fab500ac630 2023-11-26T07:19:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab566c7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fab50070c20 0x7fab5006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab55ec6640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fab50144c80 0x7fab500abf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:42504/0 (socket says 172.21.15.31:42504) 2023-11-26T07:19:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab55ec6640 1 -- 172.21.15.31:0/2866541014 learned_addr learned my addr 172.21.15.31:0/2866541014 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab556c5640 1 --2- 172.21.15.31:0/2866541014 >> v2:172.21.15.97:3300/0 conn(0x7fab500ac630 0x7fab500706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab566c7640 1 -- 172.21.15.31:0/2866541014 >> v2:172.21.15.97:3300/0 conn(0x7fab500ac630 msgr2=0x7fab500706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab566c7640 1 --2- 172.21.15.31:0/2866541014 >> v2:172.21.15.97:3300/0 conn(0x7fab500ac630 0x7fab500706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab566c7640 1 -- 172.21.15.31:0/2866541014 >> v1:172.21.15.31:6789/0 conn(0x7fab50144c80 legacy=0x7fab500abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab566c7640 1 -- 172.21.15.31:0/2866541014 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50071730 con 0x7fab50070c20 2023-11-26T07:19:02.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.683+0000 7fab54ec4640 1 -- 172.21.15.31:0/2866541014 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699420895 0 0) 0x7fab50145060 con 0x7fab50144c80 2023-11-26T07:19:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab566c7640 1 --2- 172.21.15.31:0/2866541014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50070c20 0x7fab5006c910 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fab50072d10 tx=0x7fab4801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3d65607ea06119 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab54ec4640 1 -- 172.21.15.31:0/2866541014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab480a59a0 con 0x7fab50070c20 2023-11-26T07:19:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab54ec4640 1 -- 172.21.15.31:0/2866541014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab480a5b00 con 0x7fab50070c20 2023-11-26T07:19:02.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab54ec4640 1 -- 172.21.15.31:0/2866541014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab480a5e90 con 0x7fab50070c20 2023-11-26T07:19:02.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab5812a640 1 -- 172.21.15.31:0/2866541014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50070c20 msgr2=0x7fab5006c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab5812a640 1 --2- 172.21.15.31:0/2866541014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50070c20 0x7fab5006c910 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fab50072d10 tx=0x7fab4801c8e0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab5812a640 1 -- 172.21.15.31:0/2866541014 shutdown_connections 2023-11-26T07:19:02.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab5812a640 1 --2- 172.21.15.31:0/2866541014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50070c20 0x7fab5006c910 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab5812a640 1 --2- 172.21.15.31:0/2866541014 >> v2:172.21.15.97:3300/0 conn(0x7fab500ac630 0x7fab500706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab5812a640 1 -- 172.21.15.31:0/2866541014 >> 172.21.15.31:0/2866541014 conn(0x7fab500b0c40 msgr2=0x7fab500b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab5812a640 1 -- 172.21.15.31:0/2866541014 shutdown_connections 2023-11-26T07:19:02.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab5812a640 1 -- 172.21.15.31:0/2866541014 wait complete. 2023-11-26T07:19:02.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab5812a640 1 Processor -- start 2023-11-26T07:19:02.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.687+0000 7fab5812a640 1 -- start start 2023-11-26T07:19:02.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab5812a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab50070c20 0x7fab5015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab5812a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab500ac630 0x7fab50099df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab5812a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50144c80 0x7fab500af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab5812a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015bd70 con 0x7fab50144c80 2023-11-26T07:19:02.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab5812a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015bf80 con 0x7fab500ac630 2023-11-26T07:19:02.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab5812a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5015c260 con 0x7fab50070c20 2023-11-26T07:19:02.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab55ec6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab50070c20 0x7fab5015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab556c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab500ac630 0x7fab50099df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab566c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50144c80 0x7fab500af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab556c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab500ac630 0x7fab50099df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:38302/0 (socket says 172.21.15.31:38302) 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab55ec6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab50070c20 0x7fab5015a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:60270/0 (socket says 172.21.15.31:60270) 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab556c5640 1 -- 172.21.15.31:0/1911846169 learned_addr learned my addr 172.21.15.31:0/1911846169 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab556c5640 1 -- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab50070c20 msgr2=0x7fab5015a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab556c5640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab50070c20 0x7fab5015a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab556c5640 1 -- 172.21.15.31:0/1911846169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50144c80 msgr2=0x7fab500af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab556c5640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50144c80 0x7fab500af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab556c5640 1 -- 172.21.15.31:0/1911846169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48009a70 con 0x7fab500ac630 2023-11-26T07:19:02.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab556c5640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab500ac630 0x7fab50099df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab4000e930 tx=0x7fab4000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab467fc640 1 -- 172.21.15.31:0/1911846169 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40009690 con 0x7fab500ac630 2023-11-26T07:19:02.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab467fc640 1 -- 172.21.15.31:0/1911846169 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4000ca00 con 0x7fab500ac630 2023-11-26T07:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab467fc640 1 -- 172.21.15.31:0/1911846169 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4000cd70 con 0x7fab500ac630 2023-11-26T07:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab5812a640 1 -- 172.21.15.31:0/1911846169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5015c540 con 0x7fab500ac630 2023-11-26T07:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab5812a640 1 -- 172.21.15.31:0/1911846169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5015cb00 con 0x7fab500ac630 2023-11-26T07:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.691+0000 7fab5812a640 1 -- 172.21.15.31:0/1911846169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5015ccc0 con 0x7fab500ac630 2023-11-26T07:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.695+0000 7fab467fc640 1 -- 172.21.15.31:0/1911846169 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fab40013470 con 0x7fab500ac630 2023-11-26T07:19:02.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.695+0000 7fab467fc640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fab34066620 0x7fab34068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:02.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.695+0000 7fab467fc640 1 -- 172.21.15.31:0/1911846169 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab400842a0 con 0x7fab500ac630 2023-11-26T07:19:02.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.695+0000 7fab55ec6640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fab34066620 0x7fab34068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:02.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.695+0000 7fab55ec6640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fab34066620 0x7fab34068ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fab4c000f70 tx=0x7fab4c007df0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:02.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.699+0000 7fab467fc640 1 -- 172.21.15.31:0/1911846169 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fab400516b0 con 0x7fab500ac630 2023-11-26T07:19:02.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.859+0000 7fab5812a640 1 -- 172.21.15.31:0/1911846169 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab500b1ba0 con 0x7fab500ac630 2023-11-26T07:19:02.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.859+0000 7fab467fc640 1 -- 172.21.15.31:0/1911846169 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16442 (secure 0 0 0) 0x7fab40054cd0 con 0x7fab500ac630 2023-11-26T07:19:02.866 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:02.867 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":43,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:19:01.563255+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:59.966450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:00.550730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 -- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fab34066620 msgr2=0x7fab34068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fab34066620 0x7fab34068ae0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fab4c000f70 tx=0x7fab4c007df0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 -- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab500ac630 msgr2=0x7fab50099df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab500ac630 0x7fab50099df0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fab4000e930 tx=0x7fab4000edf0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 -- 172.21.15.31:0/1911846169 shutdown_connections 2023-11-26T07:19:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fab34066620 0x7fab34068ae0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab50144c80 0x7fab500af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab500ac630 0x7fab50099df0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 --2- 172.21.15.31:0/1911846169 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fab50070c20 0x7fab5015a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 -- 172.21.15.31:0/1911846169 >> 172.21.15.31:0/1911846169 conn(0x7fab500b0c40 msgr2=0x7fab50165680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 -- 172.21.15.31:0/1911846169 shutdown_connections 2023-11-26T07:19:02.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:02.863+0000 7fab5812a640 1 -- 172.21.15.31:0/1911846169 wait complete. 2023-11-26T07:19:02.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:19:03.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbcc501640 1 Processor -- start 2023-11-26T07:19:03.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbcc501640 1 -- start start 2023-11-26T07:19:03.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbcc501640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffbc40b5830 0x7ffbc415e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbcc501640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbc40ad8e0 0x7ffbc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbcc501640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc40af630 con 0x7ffbc40acd70 2023-11-26T07:19:03.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbcc501640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc40acb10 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbcc501640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc40ae5b0 con 0x7ffbc40b5830 2023-11-26T07:19:03.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbcaa9e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbc40ad8e0 0x7ffbc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbcaa9e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbc40ad8e0 0x7ffbc406d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51300/0 (socket says 172.21.15.31:51300) 2023-11-26T07:19:03.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbcaa9e640 1 -- 172.21.15.31:0/780572976 learned_addr learned my addr 172.21.15.31:0/780572976 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:03.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.031+0000 7ffbca29d640 1 --2- 172.21.15.31:0/780572976 >> v2:172.21.15.97:3300/0 conn(0x7ffbc40b5830 0x7ffbc415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.035+0000 7ffbcaa9e640 1 -- 172.21.15.31:0/780572976 >> v2:172.21.15.97:3300/0 conn(0x7ffbc40b5830 msgr2=0x7ffbc415e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:03.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.035+0000 7ffbcaa9e640 1 --2- 172.21.15.31:0/780572976 >> v2:172.21.15.97:3300/0 conn(0x7ffbc40b5830 0x7ffbc415e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.035+0000 7ffbcaa9e640 1 -- 172.21.15.31:0/780572976 >> v1:172.21.15.97:6789/0 conn(0x7ffbc40acd70 legacy=0x7ffbc40ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:03.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.035+0000 7ffbcaa9e640 1 -- 172.21.15.31:0/780572976 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40ae870 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.035+0000 7ffbcaa9e640 1 --2- 172.21.15.31:0/780572976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 0x7ffbc406d190 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4070a60 tx=0x7ffbbc01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1851918c450f52b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:03.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.035+0000 7ffbc929b640 1 -- 172.21.15.31:0/780572976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0a5890 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.035+0000 7ffbc929b640 1 -- 172.21.15.31:0/780572976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc0a59f0 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.035+0000 7ffbc929b640 1 -- 172.21.15.31:0/780572976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0a5d80 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.035+0000 7ffbcc501640 1 -- 172.21.15.31:0/780572976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 msgr2=0x7ffbc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:03.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.035+0000 7ffbcc501640 1 --2- 172.21.15.31:0/780572976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 0x7ffbc406d190 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4070a60 tx=0x7ffbbc01c9d0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 -- 172.21.15.31:0/780572976 shutdown_connections 2023-11-26T07:19:03.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 --2- 172.21.15.31:0/780572976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 0x7ffbc406d190 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 --2- 172.21.15.31:0/780572976 >> v2:172.21.15.97:3300/0 conn(0x7ffbc40b5830 0x7ffbc415e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 -- 172.21.15.31:0/780572976 >> 172.21.15.31:0/780572976 conn(0x7ffbc4071d80 msgr2=0x7ffbc406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:03.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 -- 172.21.15.31:0/780572976 shutdown_connections 2023-11-26T07:19:03.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 -- 172.21.15.31:0/780572976 wait complete. 2023-11-26T07:19:03.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 Processor -- start 2023-11-26T07:19:03.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 -- start start 2023-11-26T07:19:03.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbc40acd70 0x7ffbc414b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 0x7ffbc414bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbc40b5830 0x7ffbc414c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4153840 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc415b290 con 0x7ffbc40acd70 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcc501640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4153a90 con 0x7ffbc40b5830 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbc9a9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 0x7ffbc414bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbc9a9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 0x7ffbc414bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51310/0 (socket says 172.21.15.31:51310) 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbc9a9c640 1 -- 172.21.15.31:0/1059963586 learned_addr learned my addr 172.21.15.31:0/1059963586 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbca29d640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbc40acd70 0x7ffbc414b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbcaa9e640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbc40b5830 0x7ffbc414c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbc9a9c640 1 -- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbc40b5830 msgr2=0x7ffbc414c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbc9a9c640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbc40b5830 0x7ffbc414c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbc9a9c640 1 -- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbc40acd70 msgr2=0x7ffbc414b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbc9a9c640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbc40acd70 0x7ffbc414b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbc9a9c640 1 -- 172.21.15.31:0/1059963586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc009b30 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.039+0000 7ffbc9a9c640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 0x7ffbc414bde0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c3c0 tx=0x7ffbb400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:03.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.043+0000 7ffbbaffd640 1 -- 172.21.15.31:0/1059963586 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40052d0 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.043+0000 7ffbbaffd640 1 -- 172.21.15.31:0/1059963586 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4005430 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.043+0000 7ffbbaffd640 1 -- 172.21.15.31:0/1059963586 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4003840 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.043+0000 7ffbcc501640 1 -- 172.21.15.31:0/1059963586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc4153d70 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.043+0000 7ffbcc501640 1 -- 172.21.15.31:0/1059963586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc414d020 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.043+0000 7ffbcc501640 1 -- 172.21.15.31:0/1059963586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc404e850 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.043+0000 7ffbbaffd640 1 -- 172.21.15.31:0/1059963586 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ffbb40039a0 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.043+0000 7ffbbaffd640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffba8066620 0x7ffba8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.043+0000 7ffbbaffd640 1 -- 172.21.15.31:0/1059963586 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbb4083410 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.043+0000 7ffbca29d640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffba8066620 0x7ffba8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:03.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.047+0000 7ffbca29d640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffba8066620 0x7ffba8068ae0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009b60 tx=0x7ffbc0009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:03.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.047+0000 7ffbbaffd640 1 -- 172.21.15.31:0/1059963586 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffbb4050760 con 0x7ffbc40ad8e0 2023-11-26T07:19:03.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:03.251+0000 7ffbcc501640 1 -- 172.21.15.31:0/1059963586 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffbc40af630 con 0x7ffbc40ad8e0 2023-11-26T07:19:04.146 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-11-26T07:19:04.139+0000 7fbbaf364640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:19:04.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.175+0000 7ffbbaffd640 1 -- 172.21.15.31:0/1059963586 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffbc40af630 con 0x7ffbc40ad8e0 2023-11-26T07:19:04.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 -- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffba8066620 msgr2=0x7ffba8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffba8066620 0x7ffba8068ae0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009b60 tx=0x7ffbc0009190 comp rx=0 tx=0).stop 2023-11-26T07:19:04.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 -- 172.21.15.31:0/1059963586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 msgr2=0x7ffbc414bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 0x7ffbc414bde0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c3c0 tx=0x7ffbb400c880 comp rx=0 tx=0).stop 2023-11-26T07:19:04.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 -- 172.21.15.31:0/1059963586 shutdown_connections 2023-11-26T07:19:04.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffba8066620 0x7ffba8068ae0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbc40b5830 0x7ffbc414c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbc40ad8e0 0x7ffbc414bde0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 --2- 172.21.15.31:0/1059963586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbc40acd70 0x7ffbc414b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 -- 172.21.15.31:0/1059963586 >> 172.21.15.31:0/1059963586 conn(0x7ffbc4071d80 msgr2=0x7ffbc415edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:04.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 -- 172.21.15.31:0/1059963586 shutdown_connections 2023-11-26T07:19:04.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.179+0000 7ffbcc501640 1 -- 172.21.15.31:0/1059963586 wait complete. 2023-11-26T07:19:04.208 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:19:04.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:19:04.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcffa632640 1 Processor -- start 2023-11-26T07:19:04.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcffa632640 1 -- start start 2023-11-26T07:19:04.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcffa632640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff406e3f0 0x7fcff415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcffa632640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcff40ce5d0 0x7fcff415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcffa632640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff4159630 con 0x7fcff40cb9b0 2023-11-26T07:19:04.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcffa632640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff415bba0 con 0x7fcff40ce5d0 2023-11-26T07:19:04.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcffa632640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40cdd50 con 0x7fcff406e3f0 2023-11-26T07:19:04.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcff8bcf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcff40ce5d0 0x7fcff415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcff8bcf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcff40ce5d0 0x7fcff415bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51206/0 (socket says 172.21.15.31:51206) 2023-11-26T07:19:04.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcff37fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff406e3f0 0x7fcff415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcff8bcf640 1 -- 172.21.15.31:0/2901709374 learned_addr learned my addr 172.21.15.31:0/2901709374 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:04.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcff3fff640 1 --1- 172.21.15.31:0/2901709374 >> v1:172.21.15.97:6789/0 conn(0x7fcff40cb9b0 0x7fcff40cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:47192/0 (socket says 172.21.15.31:47192) 2023-11-26T07:19:04.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcff8bcf640 1 -- 172.21.15.31:0/2901709374 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff406e3f0 msgr2=0x7fcff415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:04.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcff8bcf640 1 --2- 172.21.15.31:0/2901709374 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff406e3f0 0x7fcff415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcff8bcf640 1 -- 172.21.15.31:0/2901709374 >> v1:172.21.15.97:6789/0 conn(0x7fcff40cb9b0 legacy=0x7fcff40cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.355+0000 7fcff8bcf640 1 -- 172.21.15.31:0/2901709374 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff415f6d0 con 0x7fcff40ce5d0 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcff8bcf640 1 --2- 172.21.15.31:0/2901709374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff40ce5d0 0x7fcff415bfd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009e50 tx=0x7fcfec01ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8c23664e9fcc69d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:04.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcff2ffd640 1 -- 172.21.15.31:0/2901709374 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0a57b0 con 0x7fcff40ce5d0 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcff2ffd640 1 -- 172.21.15.31:0/2901709374 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec0a5910 con 0x7fcff40ce5d0 2023-11-26T07:19:04.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcff2ffd640 1 -- 172.21.15.31:0/2901709374 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0a5ca0 con 0x7fcff40ce5d0 2023-11-26T07:19:04.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcffa632640 1 -- 172.21.15.31:0/2901709374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff40ce5d0 msgr2=0x7fcff415bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcffa632640 1 --2- 172.21.15.31:0/2901709374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff40ce5d0 0x7fcff415bfd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009e50 tx=0x7fcfec01ca10 comp rx=0 tx=0).stop 2023-11-26T07:19:04.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcffa632640 1 -- 172.21.15.31:0/2901709374 shutdown_connections 2023-11-26T07:19:04.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcffa632640 1 --2- 172.21.15.31:0/2901709374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff40ce5d0 0x7fcff415bfd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcffa632640 1 --2- 172.21.15.31:0/2901709374 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff406e3f0 0x7fcff415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcffa632640 1 -- 172.21.15.31:0/2901709374 >> 172.21.15.31:0/2901709374 conn(0x7fcff40b0cb0 msgr2=0x7fcff40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcffa632640 1 -- 172.21.15.31:0/2901709374 shutdown_connections 2023-11-26T07:19:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.359+0000 7fcffa632640 1 -- 172.21.15.31:0/2901709374 wait complete. 2023-11-26T07:19:04.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 Processor -- start 2023-11-26T07:19:04.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 -- start start 2023-11-26T07:19:04.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff406e3f0 0x7fcff415e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff40cb9b0 0x7fcff40af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff40ce5d0 0x7fcff40aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff409b680 con 0x7fcff406e3f0 2023-11-26T07:19:04.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff4158e00 con 0x7fcff40cb9b0 2023-11-26T07:19:04.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff409aa00 con 0x7fcff40ce5d0 2023-11-26T07:19:04.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff406e3f0 0x7fcff415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff406e3f0 0x7fcff415e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51552/0 (socket says 172.21.15.31:51552) 2023-11-26T07:19:04.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff3fff640 1 -- 172.21.15.31:0/933509287 learned_addr learned my addr 172.21.15.31:0/933509287 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:04.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff37fe640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff40cb9b0 0x7fcff40af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff8bcf640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff40ce5d0 0x7fcff40aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff3fff640 1 -- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff40ce5d0 msgr2=0x7fcff40aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff3fff640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff40ce5d0 0x7fcff40aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff3fff640 1 -- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff40cb9b0 msgr2=0x7fcff40af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:04.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff3fff640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff40cb9b0 0x7fcff40af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:04.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff3fff640 1 -- 172.21.15.31:0/933509287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec009b30 con 0x7fcff406e3f0 2023-11-26T07:19:04.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff37fe640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff40cb9b0 0x7fcff40af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:04.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff8bcf640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff40ce5d0 0x7fcff40aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:04.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff3fff640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff406e3f0 0x7fcff415e040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c490 tx=0x7fcfe400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:04.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff0ff9640 1 -- 172.21.15.31:0/933509287 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4007480 con 0x7fcff406e3f0 2023-11-26T07:19:04.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff0ff9640 1 -- 172.21.15.31:0/933509287 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe40075e0 con 0x7fcff406e3f0 2023-11-26T07:19:04.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcff0ff9640 1 -- 172.21.15.31:0/933509287 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4004a10 con 0x7fcff406e3f0 2023-11-26T07:19:04.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 -- 172.21.15.31:0/933509287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff409ace0 con 0x7fcff406e3f0 2023-11-26T07:19:04.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 -- 172.21.15.31:0/933509287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff409b2a0 con 0x7fcff406e3f0 2023-11-26T07:19:04.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.363+0000 7fcffa632640 1 -- 172.21.15.31:0/933509287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff40671f0 con 0x7fcff406e3f0 2023-11-26T07:19:04.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.367+0000 7fcff0ff9640 1 -- 172.21.15.31:0/933509287 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fcfe4004b70 con 0x7fcff406e3f0 2023-11-26T07:19:04.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.367+0000 7fcff0ff9640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfe0066620 0x7fcfe0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:04.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.367+0000 7fcff0ff9640 1 -- 172.21.15.31:0/933509287 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfe4083320 con 0x7fcff406e3f0 2023-11-26T07:19:04.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.367+0000 7fcff37fe640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfe0066620 0x7fcfe0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:04.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.367+0000 7fcff37fe640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfe0066620 0x7fcfe0068ae0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcff41592b0 tx=0x7fcfe8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:04.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.371+0000 7fcff0ff9640 1 -- 172.21.15.31:0/933509287 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcfe4050590 con 0x7fcff406e3f0 2023-11-26T07:19:04.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:04.559+0000 7fcffa632640 1 -- 172.21.15.31:0/933509287 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcff406b460 con 0x7fcff406e3f0 2023-11-26T07:19:05.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.183+0000 7fcff0ff9640 1 -- 172.21.15.31:0/933509287 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcfe4053bb0 con 0x7fcff406e3f0 2023-11-26T07:19:05.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 -- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfe0066620 msgr2=0x7fcfe0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfe0066620 0x7fcfe0068ae0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcff41592b0 tx=0x7fcfe8008040 comp rx=0 tx=0).stop 2023-11-26T07:19:05.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 -- 172.21.15.31:0/933509287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff406e3f0 msgr2=0x7fcff415e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff406e3f0 0x7fcff415e040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c490 tx=0x7fcfe400c950 comp rx=0 tx=0).stop 2023-11-26T07:19:05.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 -- 172.21.15.31:0/933509287 shutdown_connections 2023-11-26T07:19:05.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfe0066620 0x7fcfe0068ae0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff40ce5d0 0x7fcff40aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff40cb9b0 0x7fcff40af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 --2- 172.21.15.31:0/933509287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff406e3f0 0x7fcff415e040 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 -- 172.21.15.31:0/933509287 >> 172.21.15.31:0/933509287 conn(0x7fcff40b0cb0 msgr2=0x7fcff40cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:05.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 -- 172.21.15.31:0/933509287 shutdown_connections 2023-11-26T07:19:05.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.187+0000 7fcffa632640 1 -- 172.21.15.31:0/933509287 wait complete. 2023-11-26T07:19:05.212 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:19:05.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f3936a46640 1 Processor -- start 2023-11-26T07:19:05.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f3936a46640 1 -- start start 2023-11-26T07:19:05.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f3936a46640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f393015d8b0 0x7f39300b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f3936a46640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f39300cecd0 0x7f393006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f3936a46640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39300b0a70 con 0x7f39300b5120 2023-11-26T07:19:05.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f3936a46640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300cddb0 con 0x7f393015d8b0 2023-11-26T07:19:05.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f3936a46640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300b4100 con 0x7f39300cecd0 2023-11-26T07:19:05.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f392ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f393015d8b0 0x7f39300b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f392ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f393015d8b0 0x7f39300b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51564/0 (socket says 172.21.15.31:51564) 2023-11-26T07:19:05.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f392ffff640 1 -- 172.21.15.31:0/890029258 learned_addr learned my addr 172.21.15.31:0/890029258 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:05.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f3934fe3640 1 --2- 172.21.15.31:0/890029258 >> v2:172.21.15.97:3300/0 conn(0x7f39300cecd0 0x7f393006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f392ffff640 1 -- 172.21.15.31:0/890029258 >> v2:172.21.15.97:3300/0 conn(0x7f39300cecd0 msgr2=0x7f393006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:05.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f392ffff640 1 --2- 172.21.15.31:0/890029258 >> v2:172.21.15.97:3300/0 conn(0x7f39300cecd0 0x7f393006a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f392ffff640 1 -- 172.21.15.31:0/890029258 >> v1:172.21.15.97:6789/0 conn(0x7f39300b5120 legacy=0x7f39300ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:05.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f392ffff640 1 -- 172.21.15.31:0/890029258 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300b43e0 con 0x7f393015d8b0 2023-11-26T07:19:05.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.351+0000 7f3934fe3640 1 --2- 172.21.15.31:0/890029258 >> v2:172.21.15.97:3300/0 conn(0x7f39300cecd0 0x7f393006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:05.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f392ffff640 1 --2- 172.21.15.31:0/890029258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 0x7f39300b4be0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f39300712f0 tx=0x7f392001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4634a4773d926 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:05.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f392effd640 1 -- 172.21.15.31:0/890029258 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200a89b0 con 0x7f393015d8b0 2023-11-26T07:19:05.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f392effd640 1 -- 172.21.15.31:0/890029258 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39200a8b10 con 0x7f393015d8b0 2023-11-26T07:19:05.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f392effd640 1 -- 172.21.15.31:0/890029258 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200a8ea0 con 0x7f393015d8b0 2023-11-26T07:19:05.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f3936a46640 1 -- 172.21.15.31:0/890029258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 msgr2=0x7f39300b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f3936a46640 1 --2- 172.21.15.31:0/890029258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 0x7f39300b4be0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f39300712f0 tx=0x7f392001ecb0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f3936a46640 1 -- 172.21.15.31:0/890029258 shutdown_connections 2023-11-26T07:19:05.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f3936a46640 1 --2- 172.21.15.31:0/890029258 >> v2:172.21.15.97:3300/0 conn(0x7f39300cecd0 0x7f393006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f3936a46640 1 --2- 172.21.15.31:0/890029258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 0x7f39300b4be0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f3936a46640 1 -- 172.21.15.31:0/890029258 >> 172.21.15.31:0/890029258 conn(0x7f39300b5570 msgr2=0x7f39300b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:05.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f3936a46640 1 -- 172.21.15.31:0/890029258 shutdown_connections 2023-11-26T07:19:05.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.355+0000 7f3936a46640 1 -- 172.21.15.31:0/890029258 wait complete. 2023-11-26T07:19:05.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3936a46640 1 Processor -- start 2023-11-26T07:19:05.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3936a46640 1 -- start start 2023-11-26T07:19:05.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3936a46640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39300b5120 0x7f3930160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3936a46640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39300cecd0 0x7f39300ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3936a46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 0x7f393014be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3936a46640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3930156fe0 con 0x7f393015d8b0 2023-11-26T07:19:05.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3936a46640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3930156cb0 con 0x7f39300cecd0 2023-11-26T07:19:05.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3936a46640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39301551a0 con 0x7f39300b5120 2023-11-26T07:19:05.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3934fe3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 0x7f393014be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3934fe3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 0x7f393014be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51568/0 (socket says 172.21.15.31:51568) 2023-11-26T07:19:05.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3934fe3640 1 -- 172.21.15.31:0/1070501354 learned_addr learned my addr 172.21.15.31:0/1070501354 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:05.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f392ffff640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39300b5120 0x7f3930160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f392f7fe640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39300cecd0 0x7f39300ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3934fe3640 1 -- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39300b5120 msgr2=0x7f3930160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3934fe3640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39300b5120 0x7f3930160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3934fe3640 1 -- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39300cecd0 msgr2=0x7f39300ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3934fe3640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39300cecd0 0x7f39300ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3934fe3640 1 -- 172.21.15.31:0/1070501354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39200094d0 con 0x7f393015d8b0 2023-11-26T07:19:05.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f392f7fe640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39300cecd0 0x7f39300ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:05.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3934fe3640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 0x7f393014be30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3928009fa0 tx=0x7f392800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:05.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f392cff9640 1 -- 172.21.15.31:0/1070501354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928003ba0 con 0x7f393015d8b0 2023-11-26T07:19:05.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f392cff9640 1 -- 172.21.15.31:0/1070501354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392800f040 con 0x7f393015d8b0 2023-11-26T07:19:05.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f392cff9640 1 -- 172.21.15.31:0/1070501354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928004030 con 0x7f393015d8b0 2023-11-26T07:19:05.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3936a46640 1 -- 172.21.15.31:0/1070501354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39300aeac0 con 0x7f393015d8b0 2023-11-26T07:19:05.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.359+0000 7f3936a46640 1 -- 172.21.15.31:0/1070501354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39300ae220 con 0x7f393015d8b0 2023-11-26T07:19:05.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.363+0000 7f3936a46640 1 -- 172.21.15.31:0/1070501354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3930157120 con 0x7f393015d8b0 2023-11-26T07:19:05.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.363+0000 7f392cff9640 1 -- 172.21.15.31:0/1070501354 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f3928004190 con 0x7f393015d8b0 2023-11-26T07:19:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.363+0000 7f392cff9640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f391c066620 0x7f391c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.363+0000 7f392cff9640 1 -- 172.21.15.31:0/1070501354 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39280845f0 con 0x7f393015d8b0 2023-11-26T07:19:05.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.363+0000 7f392ffff640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f391c066620 0x7f391c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.363+0000 7f392ffff640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f391c066620 0x7f391c068ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f39200b2000 tx=0x7f39200aa070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:05.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.367+0000 7f392cff9640 1 -- 172.21.15.31:0/1070501354 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3930157120 con 0x7f393015d8b0 2023-11-26T07:19:05.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.555+0000 7f3936a46640 1 -- 172.21.15.31:0/1070501354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3930157120 con 0x7f393015d8b0 2023-11-26T07:19:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.555+0000 7f392cff9640 1 -- 172.21.15.31:0/1070501354 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f3930157120 con 0x7f393015d8b0 2023-11-26T07:19:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:05.563 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-11-26T07:19:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 -- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f391c066620 msgr2=0x7f391c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f391c066620 0x7f391c068ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f39200b2000 tx=0x7f39200aa070 comp rx=0 tx=0).stop 2023-11-26T07:19:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 -- 172.21.15.31:0/1070501354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 msgr2=0x7f393014be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 0x7f393014be30 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3928009fa0 tx=0x7f392800d560 comp rx=0 tx=0).stop 2023-11-26T07:19:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 -- 172.21.15.31:0/1070501354 shutdown_connections 2023-11-26T07:19:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f391c066620 0x7f391c068ae0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f393015d8b0 0x7f393014be30 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39300cecd0 0x7f39300ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 --2- 172.21.15.31:0/1070501354 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f39300b5120 0x7f3930160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 -- 172.21.15.31:0/1070501354 >> 172.21.15.31:0/1070501354 conn(0x7f39300b5570 msgr2=0x7f39300d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 -- 172.21.15.31:0/1070501354 shutdown_connections 2023-11-26T07:19:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.559+0000 7f3936a46640 1 -- 172.21.15.31:0/1070501354 wait complete. 2023-11-26T07:19:05.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:05.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.719+0000 7fc3887c1640 1 Processor -- start 2023-11-26T07:19:05.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.719+0000 7fc3887c1640 1 -- start start 2023-11-26T07:19:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc3887c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc380143ae0 0x7fc3800b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc3887c1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc38006dfd0 0x7fc38006e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc3887c1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3800b2480 con 0x7fc3800adcc0 2023-11-26T07:19:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc3887c1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3800ae820 con 0x7fc38006dfd0 2023-11-26T07:19:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc3887c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38006e950 con 0x7fc380143ae0 2023-11-26T07:19:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc385d5c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc38006dfd0 0x7fc38006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc385d5c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc38006dfd0 0x7fc38006e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51580/0 (socket says 172.21.15.31:51580) 2023-11-26T07:19:05.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc38655d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc380143ae0 0x7fc3800b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc385d5c640 1 -- 172.21.15.31:0/1241293721 learned_addr learned my addr 172.21.15.31:0/1241293721 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:05.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc38655d640 1 -- 172.21.15.31:0/1241293721 >> v2:172.21.15.31:3300/0 conn(0x7fc38006dfd0 msgr2=0x7fc38006e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:05.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc38655d640 1 --2- 172.21.15.31:0/1241293721 >> v2:172.21.15.31:3300/0 conn(0x7fc38006dfd0 0x7fc38006e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc38655d640 1 -- 172.21.15.31:0/1241293721 >> v1:172.21.15.97:6789/0 conn(0x7fc3800adcc0 legacy=0x7fc3800ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.723+0000 7fc38655d640 1 -- 172.21.15.31:0/1241293721 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc380070670 con 0x7fc380143ae0 2023-11-26T07:19:05.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc38655d640 1 --2- 172.21.15.31:0/1241293721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc380143ae0 0x7fc3800b1f40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0097c0 tx=0x7fc37c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66cb212ca8b30e0c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:05.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc38555b640 1 -- 172.21.15.31:0/1241293721 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0ab650 con 0x7fc380143ae0 2023-11-26T07:19:05.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc38555b640 1 -- 172.21.15.31:0/1241293721 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37c0ab7b0 con 0x7fc380143ae0 2023-11-26T07:19:05.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc38555b640 1 -- 172.21.15.31:0/1241293721 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0abb40 con 0x7fc380143ae0 2023-11-26T07:19:05.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc3887c1640 1 -- 172.21.15.31:0/1241293721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc380143ae0 msgr2=0x7fc3800b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc3887c1640 1 --2- 172.21.15.31:0/1241293721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc380143ae0 0x7fc3800b1f40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0097c0 tx=0x7fc37c01c450 comp rx=0 tx=0).stop 2023-11-26T07:19:05.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc3887c1640 1 -- 172.21.15.31:0/1241293721 shutdown_connections 2023-11-26T07:19:05.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc3887c1640 1 --2- 172.21.15.31:0/1241293721 >> v2:172.21.15.31:3300/0 conn(0x7fc38006dfd0 0x7fc38006e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc3887c1640 1 --2- 172.21.15.31:0/1241293721 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc380143ae0 0x7fc3800b1f40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc3887c1640 1 -- 172.21.15.31:0/1241293721 >> 172.21.15.31:0/1241293721 conn(0x7fc3800b0c40 msgr2=0x7fc3800b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:05.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc3887c1640 1 -- 172.21.15.31:0/1241293721 shutdown_connections 2023-11-26T07:19:05.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc3887c1640 1 -- 172.21.15.31:0/1241293721 wait complete. 2023-11-26T07:19:05.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc3887c1640 1 Processor -- start 2023-11-26T07:19:05.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.727+0000 7fc3887c1640 1 -- start start 2023-11-26T07:19:05.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc3887c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc38006dfd0 0x7fc380138a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc3887c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3800adcc0 0x7fc38009aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc3887c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc380143ae0 0x7fc3800b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc3887c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc380099a30 con 0x7fc380143ae0 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc3887c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc380138f70 con 0x7fc3800adcc0 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc3887c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc380099c80 con 0x7fc38006dfd0 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc386d5e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc380143ae0 0x7fc3800b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc386d5e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc380143ae0 0x7fc3800b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51586/0 (socket says 172.21.15.31:51586) 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc386d5e640 1 -- 172.21.15.31:0/3186148561 learned_addr learned my addr 172.21.15.31:0/3186148561 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc38655d640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc38006dfd0 0x7fc380138a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc385d5c640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3800adcc0 0x7fc38009aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc386d5e640 1 -- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc38006dfd0 msgr2=0x7fc380138a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc386d5e640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc38006dfd0 0x7fc380138a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc386d5e640 1 -- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3800adcc0 msgr2=0x7fc38009aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc386d5e640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3800adcc0 0x7fc38009aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc386d5e640 1 -- 172.21.15.31:0/3186148561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c009420 con 0x7fc380143ae0 2023-11-26T07:19:05.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc386d5e640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc380143ae0 0x7fc3800b2650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc37800c770 tx=0x7fc37800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:05.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc376ffd640 1 -- 172.21.15.31:0/3186148561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378003b80 con 0x7fc380143ae0 2023-11-26T07:19:05.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc376ffd640 1 -- 172.21.15.31:0/3186148561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378003ce0 con 0x7fc380143ae0 2023-11-26T07:19:05.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc376ffd640 1 -- 172.21.15.31:0/3186148561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37800aa90 con 0x7fc380143ae0 2023-11-26T07:19:05.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc3887c1640 1 -- 172.21.15.31:0/3186148561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc380099f60 con 0x7fc380143ae0 2023-11-26T07:19:05.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc3887c1640 1 -- 172.21.15.31:0/3186148561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38009a520 con 0x7fc380143ae0 2023-11-26T07:19:05.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.731+0000 7fc3887c1640 1 -- 172.21.15.31:0/3186148561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38004e850 con 0x7fc380143ae0 2023-11-26T07:19:05.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.735+0000 7fc376ffd640 1 -- 172.21.15.31:0/3186148561 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fc37800abf0 con 0x7fc380143ae0 2023-11-26T07:19:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.735+0000 7fc376ffd640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc364066620 0x7fc364068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.735+0000 7fc376ffd640 1 -- 172.21.15.31:0/3186148561 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc378083ea0 con 0x7fc380143ae0 2023-11-26T07:19:05.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.735+0000 7fc38655d640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc364066620 0x7fc364068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:05.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.735+0000 7fc38655d640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc364066620 0x7fc364068ae0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0af000 tx=0x7fc37c0b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:05.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.739+0000 7fc376ffd640 1 -- 172.21.15.31:0/3186148561 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc378051140 con 0x7fc380143ae0 2023-11-26T07:19:05.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.895+0000 7fc3887c1640 1 -- 172.21.15.31:0/3186148561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc38006cc40 con 0x7fc380143ae0 2023-11-26T07:19:05.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.895+0000 7fc376ffd640 1 -- 172.21.15.31:0/3186148561 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16429 (secure 0 0 0) 0x7fc378054760 con 0x7fc380143ae0 2023-11-26T07:19:05.903 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:05.904 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":44,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:19:03.010965+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:59.966450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:00.550730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:05.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 -- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc364066620 msgr2=0x7fc364068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc364066620 0x7fc364068ae0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0af000 tx=0x7fc37c0b6040 comp rx=0 tx=0).stop 2023-11-26T07:19:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 -- 172.21.15.31:0/3186148561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc380143ae0 msgr2=0x7fc3800b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc380143ae0 0x7fc3800b2650 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc37800c770 tx=0x7fc37800cc30 comp rx=0 tx=0).stop 2023-11-26T07:19:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 -- 172.21.15.31:0/3186148561 shutdown_connections 2023-11-26T07:19:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc364066620 0x7fc364068ae0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc380143ae0 0x7fc3800b2650 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3800adcc0 0x7fc38009aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 --2- 172.21.15.31:0/3186148561 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc38006dfd0 0x7fc380138a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 -- 172.21.15.31:0/3186148561 >> 172.21.15.31:0/3186148561 conn(0x7fc3800b0c40 msgr2=0x7fc3800ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 -- 172.21.15.31:0/3186148561 shutdown_connections 2023-11-26T07:19:05.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:05.899+0000 7fc3887c1640 1 -- 172.21.15.31:0/3186148561 wait complete. 2023-11-26T07:19:05.924 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:19:05.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:19:06.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.059+0000 7f38a6481640 1 Processor -- start 2023-11-26T07:19:06.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a6481640 1 -- start start 2023-11-26T07:19:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a6481640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38a015b130 0x7f38a0070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a6481640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38a00b4b20 0x7f38a00ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a6481640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a00709d0 con 0x7f38a00cc680 2023-11-26T07:19:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a6481640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a00706a0 con 0x7f38a015b130 2023-11-26T07:19:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a6481640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00cd690 con 0x7f38a00b4b20 2023-11-26T07:19:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a4c7e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f38a00cc680 0x7f38a00b4410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52442/0 (socket says 172.21.15.31:52442) 2023-11-26T07:19:06.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a4c7e640 1 -- 172.21.15.31:0/2269203758 learned_addr learned my addr 172.21.15.31:0/2269203758 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:06.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a547f640 1 --2- 172.21.15.31:0/2269203758 >> v2:172.21.15.97:3300/0 conn(0x7f38a015b130 0x7f38a0070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a5c80640 1 --2- 172.21.15.31:0/2269203758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38a00b4b20 0x7f38a00ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f3897fff640 1 -- 172.21.15.31:0/2269203758 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000421669 0 0) 0x7f38a00709d0 con 0x7f38a00cc680 2023-11-26T07:19:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f3897fff640 1 -- 172.21.15.31:0/2269203758 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f388c003530 con 0x7f38a00cc680 2023-11-26T07:19:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a547f640 1 -- 172.21.15.31:0/2269203758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38a00b4b20 msgr2=0x7f38a00ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a547f640 1 --2- 172.21.15.31:0/2269203758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38a00b4b20 0x7f38a00ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a547f640 1 -- 172.21.15.31:0/2269203758 >> v1:172.21.15.31:6789/0 conn(0x7f38a00cc680 legacy=0x7f38a00b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.063+0000 7f38a547f640 1 -- 172.21.15.31:0/2269203758 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00cd0d0 con 0x7f38a015b130 2023-11-26T07:19:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f38a547f640 1 --2- 172.21.15.31:0/2269203758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38a015b130 0x7f38a0070e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f389c0097c0 tx=0x7f389c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337b06c75608c446 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f3897fff640 1 -- 172.21.15.31:0/2269203758 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0ab600 con 0x7f38a015b130 2023-11-26T07:19:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f3897fff640 1 -- 172.21.15.31:0/2269203758 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c0a7050 con 0x7f38a015b130 2023-11-26T07:19:06.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f3897fff640 1 -- 172.21.15.31:0/2269203758 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0ab980 con 0x7f38a015b130 2023-11-26T07:19:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f38a6481640 1 -- 172.21.15.31:0/2269203758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38a015b130 msgr2=0x7f38a0070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f38a6481640 1 --2- 172.21.15.31:0/2269203758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38a015b130 0x7f38a0070e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f389c0097c0 tx=0x7f389c01c450 comp rx=0 tx=0).stop 2023-11-26T07:19:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f38a6481640 1 -- 172.21.15.31:0/2269203758 shutdown_connections 2023-11-26T07:19:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f38a6481640 1 --2- 172.21.15.31:0/2269203758 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38a00b4b20 0x7f38a00ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f38a6481640 1 --2- 172.21.15.31:0/2269203758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38a015b130 0x7f38a0070e60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f38a6481640 1 -- 172.21.15.31:0/2269203758 >> 172.21.15.31:0/2269203758 conn(0x7f38a00b0e00 msgr2=0x7f38a00b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:06.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f38a6481640 1 -- 172.21.15.31:0/2269203758 shutdown_connections 2023-11-26T07:19:06.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.067+0000 7f38a6481640 1 -- 172.21.15.31:0/2269203758 wait complete. 2023-11-26T07:19:06.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 Processor -- start 2023-11-26T07:19:06.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 -- start start 2023-11-26T07:19:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00b4b20 0x7f38a01553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38a00cc680 0x7f38a00b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38a015b130 0x7f38a00b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a014a230 con 0x7f38a00b4b20 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a01495b0 con 0x7f38a00cc680 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a0149890 con 0x7f38a015b130 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a547f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00b4b20 0x7f38a01553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a547f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00b4b20 0x7f38a01553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51592/0 (socket says 172.21.15.31:51592) 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a547f640 1 -- 172.21.15.31:0/2386489340 learned_addr learned my addr 172.21.15.31:0/2386489340 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a547f640 1 -- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38a015b130 msgr2=0x7f38a00b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a547f640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38a015b130 0x7f38a00b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a547f640 1 -- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38a00cc680 msgr2=0x7f38a00b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a547f640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38a00cc680 0x7f38a00b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a547f640 1 -- 172.21.15.31:0/2386489340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c009420 con 0x7f38a00b4b20 2023-11-26T07:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a547f640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00b4b20 0x7f38a01553a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f389c01a080 tx=0x7f389c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f3895ffb640 1 -- 172.21.15.31:0/2386489340 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0b4410 con 0x7f38a00b4b20 2023-11-26T07:19:06.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f3895ffb640 1 -- 172.21.15.31:0/2386489340 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c0b4590 con 0x7f38a00b4b20 2023-11-26T07:19:06.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 -- 172.21.15.31:0/2386489340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a0149b70 con 0x7f38a00b4b20 2023-11-26T07:19:06.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f3895ffb640 1 -- 172.21.15.31:0/2386489340 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0b4940 con 0x7f38a00b4b20 2023-11-26T07:19:06.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 -- 172.21.15.31:0/2386489340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a01558e0 con 0x7f38a00b4b20 2023-11-26T07:19:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.071+0000 7f38a6481640 1 -- 172.21.15.31:0/2386489340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a0074a00 con 0x7f38a00b4b20 2023-11-26T07:19:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.075+0000 7f3895ffb640 1 -- 172.21.15.31:0/2386489340 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f389c0b2040 con 0x7f38a00b4b20 2023-11-26T07:19:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.075+0000 7f3895ffb640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3884066620 0x7f3884068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.075+0000 7f3895ffb640 1 -- 172.21.15.31:0/2386489340 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f389c11e190 con 0x7f38a00b4b20 2023-11-26T07:19:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.075+0000 7f38a4c7e640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3884066620 0x7f3884068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.075+0000 7f38a4c7e640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3884066620 0x7f3884068ae0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3890007fd0 tx=0x7f3890007940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.079+0000 7f3895ffb640 1 -- 172.21.15.31:0/2386489340 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f389c0eb390 con 0x7f38a00b4b20 2023-11-26T07:19:06.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.239+0000 7f38a6481640 1 -- 172.21.15.31:0/2386489340 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38a00715b0 con 0x7f3884066620 2023-11-26T07:19:06.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.247+0000 7f3895ffb640 1 -- 172.21.15.31:0/2386489340 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+276692 (secure 0 0 0) 0x7f38900058c0 con 0x7f3884066620 2023-11-26T07:19:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:06.258 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:19:06.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 -- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3884066620 msgr2=0x7f3884068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3884066620 0x7f3884068ae0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3890007fd0 tx=0x7f3890007940 comp rx=0 tx=0).stop 2023-11-26T07:19:06.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 -- 172.21.15.31:0/2386489340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00b4b20 msgr2=0x7f38a01553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00b4b20 0x7f38a01553a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f389c01a080 tx=0x7f389c0191f0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 -- 172.21.15.31:0/2386489340 shutdown_connections 2023-11-26T07:19:06.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3884066620 0x7f3884068ae0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38a015b130 0x7f38a00b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38a00cc680 0x7f38a00b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 --2- 172.21.15.31:0/2386489340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00b4b20 0x7f38a01553a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 -- 172.21.15.31:0/2386489340 >> 172.21.15.31:0/2386489340 conn(0x7f38a00b0e00 msgr2=0x7f38a015c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:06.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 -- 172.21.15.31:0/2386489340 shutdown_connections 2023-11-26T07:19:06.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.251+0000 7f38a6481640 1 -- 172.21.15.31:0/2386489340 wait complete. 2023-11-26T07:19:06.277 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2023-11-26T07:19:04.327713+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":221040,"kb_used_data":4816,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753680,"statfs":{"total":773094113280,"available":772867768320,"internally_reserved":0,"allocated":4931584,"data_stored":2661494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.173402"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.156041+0000","last_change":"2023-11-26T07:19:00.156041+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.156041+0000","last_undegraded":"2023-11-26T07:19:00.156041+0000","last_fullsized":"2023-11-26T07:19:00.156041+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.156565+0000","last_change":"2023-11-26T07:19:00.156565+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.156565+0000","last_undegraded":"2023-11-26T07:19:00.156565+0000","last_fullsized":"2023-11-26T07:19:00.156565+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.158925+0000","last_change":"2023-11-26T07:19:00.158925+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.158925+0000","last_undegraded":"2023-11-26T07:19:00.158925+0000","last_fullsized":"2023-11-26T07:19:00.158925+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.158861+0000","last_change":"2023-11-26T07:19:00.158861+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.158861+0000","last_undegraded":"2023-11-26T07:19:00.158861+0000","last_fullsized":"2023-11-26T07:19:00.158861+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.157982+0000","last_change":"2023-11-26T07:19:00.157982+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.157982+0000","last_undegraded":"2023-11-26T07:19:00.157982+0000","last_fullsized":"2023-11-26T07:19:00.157982+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.156363+0000","last_change":"2023-11-26T07:19:00.156363+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.156363+0000","last_undegraded":"2023-11-26T07:19:00.156363+0000","last_fullsized":"2023-11-26T07:19:00.156363+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.160079+0000","last_change":"2023-11-26T07:19:00.160079+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.160079+0000","last_undegraded":"2023-11-26T07:19:00.160079+0000","last_fullsized":"2023-11-26T07:19:00.160079+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.162942+0000","last_change":"2023-11-26T07:19:00.162942+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.162942+0000","last_undegraded":"2023-11-26T07:19:00.162942+0000","last_fullsized":"2023-11-26T07:19:00.162942+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.155683+0000","last_change":"2023-11-26T07:19:00.155683+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.155683+0000","last_undegraded":"2023-11-26T07:19:00.155683+0000","last_fullsized":"2023-11-26T07:19:00.155683+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.162043+0000","last_change":"2023-11-26T07:19:00.162043+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.162043+0000","last_undegraded":"2023-11-26T07:19:00.162043+0000","last_fullsized":"2023-11-26T07:19:00.162043+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.158492+0000","last_change":"2023-11-26T07:19:00.158492+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.158492+0000","last_undegraded":"2023-11-26T07:19:00.158492+0000","last_fullsized":"2023-11-26T07:19:00.158492+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.153821+0000","last_change":"2023-11-26T07:19:00.153821+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.153821+0000","last_undegraded":"2023-11-26T07:19:00.153821+0000","last_fullsized":"2023-11-26T07:19:00.153821+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.157320+0000","last_change":"2023-11-26T07:19:00.157320+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.157320+0000","last_undegraded":"2023-11-26T07:19:00.157320+0000","last_fullsized":"2023-11-26T07:19:00.157320+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.158165+0000","last_change":"2023-11-26T07:19:00.158165+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.158165+0000","last_undegraded":"2023-11-26T07:19:00.158165+0000","last_fullsized":"2023-11-26T07:19:00.158165+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.155967+0000","last_change":"2023-11-26T07:19:00.155967+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.155967+0000","last_undegraded":"2023-11-26T07:19:00.155967+0000","last_fullsized":"2023-11-26T07:19:00.155967+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.162461+0000","last_change":"2023-11-26T07:19:00.162461+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.162461+0000","last_undegraded":"2023-11-26T07:19:00.162461+0000","last_fullsized":"2023-11-26T07:19:00.162461+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.153835+0000","last_change":"2023-11-26T07:19:00.153835+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.153835+0000","last_undegraded":"2023-11-26T07:19:00.153835+0000","last_fullsized":"2023-11-26T07:19:00.153835+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.155642+0000","last_change":"2023-11-26T07:19:00.155642+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.155642+0000","last_undegraded":"2023-11-26T07:19:00.155642+0000","last_fullsized":"2023-11-26T07:19:00.155642+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.159971+0000","last_change":"2023-11-26T07:19:00.159971+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.159971+0000","last_undegraded":"2023-11-26T07:19:00.159971+0000","last_fullsized":"2023-11-26T07:19:00.159971+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.156004+0000","last_change":"2023-11-26T07:19:00.156004+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.156004+0000","last_undegraded":"2023-11-26T07:19:00.156004+0000","last_fullsized":"2023-11-26T07:19:00.156004+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.156064+0000","last_change":"2023-11-26T07:19:00.156064+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.156064+0000","last_undegraded":"2023-11-26T07:19:00.156064+0000","last_fullsized":"2023-11-26T07:19:00.156064+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.160270+0000","last_change":"2023-11-26T07:19:00.160270+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.160270+0000","last_undegraded":"2023-11-26T07:19:00.160270+0000","last_fullsized":"2023-11-26T07:19:00.160270+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.161992+0000","last_change":"2023-11-26T07:19:00.161992+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.161992+0000","last_undegraded":"2023-11-26T07:19:00.161992+0000","last_fullsized":"2023-11-26T07:19:00.161992+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.157941+0000","last_change":"2023-11-26T07:19:00.157941+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.157941+0000","last_undegraded":"2023-11-26T07:19:00.157941+0000","last_fullsized":"2023-11-26T07:19:00.157941+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.156019+0000","last_change":"2023-11-26T07:19:00.156019+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.156019+0000","last_undegraded":"2023-11-26T07:19:00.156019+0000","last_fullsized":"2023-11-26T07:19:00.156019+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.157953+0000","last_change":"2023-11-26T07:19:00.157953+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.157953+0000","last_undegraded":"2023-11-26T07:19:00.157953+0000","last_fullsized":"2023-11-26T07:19:00.157953+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.158923+0000","last_change":"2023-11-26T07:19:00.158923+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.158923+0000","last_undegraded":"2023-11-26T07:19:00.158923+0000","last_fullsized":"2023-11-26T07:19:00.158923+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.158466+0000","last_change":"2023-11-26T07:19:00.158466+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.158466+0000","last_undegraded":"2023-11-26T07:19:00.158466+0000","last_fullsized":"2023-11-26T07:19:00.158466+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.155647+0000","last_change":"2023-11-26T07:19:00.155647+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.155647+0000","last_undegraded":"2023-11-26T07:19:00.155647+0000","last_fullsized":"2023-11-26T07:19:00.155647+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.158529+0000","last_change":"2023-11-26T07:19:00.158529+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.158529+0000","last_undegraded":"2023-11-26T07:19:00.158529+0000","last_fullsized":"2023-11-26T07:19:00.158529+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.161986+0000","last_change":"2023-11-26T07:19:00.161986+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.161986+0000","last_undegraded":"2023-11-26T07:19:00.161986+0000","last_fullsized":"2023-11-26T07:19:00.161986+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-26T07:19:01.151385+0000","last_change":"2023-11-26T07:19:01.151385+0000","last_active":"2023-11-26T07:19:01.151385+0000","last_peered":"2023-11-26T07:19:01.151385+0000","last_clean":"2023-11-26T07:19:01.151385+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:01.151385+0000","last_undegraded":"2023-11-26T07:19:01.151385+0000","last_fullsized":"2023-11-26T07:19:01.151385+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2023-11-26T07:19:00.160431+0000","last_change":"2023-11-26T07:19:00.160431+0000","last_active":"2023-11-26T07:19:00.148535+0000","last_peered":"2023-11-26T07:19:00.148535+0000","last_clean":"2023-11-26T07:19:00.148535+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:19:00.160431+0000","last_undegraded":"2023-11-26T07:19:00.160431+0000","last_fullsized":"2023-11-26T07:19:00.160431+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-26T07:18:58.741763+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:18:58.741763+0000","last_peered":"2023-11-26T07:18:58.741763+0000","last_clean":"2023-11-26T07:18:58.741763+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:18:58.741763+0000","last_undegraded":"2023-11-26T07:18:58.741763+0000","last_fullsized":"2023-11-26T07:18:58.741763+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":774863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.58299999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.72099999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64600000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76800000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.755},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.624}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.80200000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.78100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":184495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.82999999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.873},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.64600000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.161}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":1.073}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.79200000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.1399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":184495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.63300000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.505},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":1.036}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":1.1140000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.755}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":1.0820000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.1890000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":184495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.58999999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.72599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.80500000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.872},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.70499999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.94999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":184495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.65900000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.37},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.61599999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.5},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70999999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.89600000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.79300000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.86599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":184495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.49099999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":8.0990000000000002,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.48099999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":4.431,"5min":4.431,"15min":4.431},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.70399999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.82199999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.78400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":184495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.253},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.51900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.441},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.47899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.56000000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.624}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1069056,"data_stored":779661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.50800000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56000000000000005}]},{"osd":3,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.624},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68999999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.92400000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.872}]},{"osd":6,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.90800000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.89000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:19:06.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:19:06.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdefa49a640 1 Processor -- start 2023-11-26T07:19:06.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdefa49a640 1 -- start start 2023-11-26T07:19:06.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdefa49a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdef4159880 0x7fdef40b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdefa49a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdef4071600 0x7fdef406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdefa49a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef40b41a0 con 0x7fdef40cb230 2023-11-26T07:19:06.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdefa49a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef40b3be0 con 0x7fdef4159880 2023-11-26T07:19:06.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdefa49a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40b3ec0 con 0x7fdef4071600 2023-11-26T07:19:06.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdef3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdef4159880 0x7fdef40b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdef37fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdef40cb230 0x7fdef4070ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52454/0 (socket says 172.21.15.31:52454) 2023-11-26T07:19:06.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdef37fe640 1 -- 172.21.15.31:0/2218242127 learned_addr learned my addr 172.21.15.31:0/2218242127 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:06.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdef8a37640 1 --2- 172.21.15.31:0/2218242127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdef4071600 0x7fdef406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdef3fff640 1 -- 172.21.15.31:0/2218242127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdef4071600 msgr2=0x7fdef406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:06.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdef3fff640 1 --2- 172.21.15.31:0/2218242127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdef4071600 0x7fdef406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdef3fff640 1 -- 172.21.15.31:0/2218242127 >> v1:172.21.15.31:6789/0 conn(0x7fdef40cb230 legacy=0x7fdef4070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.455+0000 7fdef3fff640 1 -- 172.21.15.31:0/2218242127 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef406cf60 con 0x7fdef4159880 2023-11-26T07:19:06.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdef3fff640 1 --2- 172.21.15.31:0/2218242127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4159880 0x7fdef40b3320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdee40097c0 tx=0x7fdee401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e966093b2e3b9d99 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdef2ffd640 1 -- 172.21.15.31:0/2218242127 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40ab600 con 0x7fdef4159880 2023-11-26T07:19:06.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdef2ffd640 1 -- 172.21.15.31:0/2218242127 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee40a7050 con 0x7fdef4159880 2023-11-26T07:19:06.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdef2ffd640 1 -- 172.21.15.31:0/2218242127 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40ab980 con 0x7fdef4159880 2023-11-26T07:19:06.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdefa49a640 1 -- 172.21.15.31:0/2218242127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4159880 msgr2=0x7fdef40b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdefa49a640 1 --2- 172.21.15.31:0/2218242127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4159880 0x7fdef40b3320 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdee40097c0 tx=0x7fdee401c3c0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdefa49a640 1 -- 172.21.15.31:0/2218242127 shutdown_connections 2023-11-26T07:19:06.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdefa49a640 1 --2- 172.21.15.31:0/2218242127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdef4071600 0x7fdef406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdefa49a640 1 --2- 172.21.15.31:0/2218242127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4159880 0x7fdef40b3320 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdefa49a640 1 -- 172.21.15.31:0/2218242127 >> 172.21.15.31:0/2218242127 conn(0x7fdef40b0c40 msgr2=0x7fdef40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:06.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdefa49a640 1 -- 172.21.15.31:0/2218242127 shutdown_connections 2023-11-26T07:19:06.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.459+0000 7fdefa49a640 1 -- 172.21.15.31:0/2218242127 wait complete. 2023-11-26T07:19:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdefa49a640 1 Processor -- start 2023-11-26T07:19:06.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdefa49a640 1 -- start start 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdefa49a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4071600 0x7fdef41f7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdefa49a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40cb230 0x7fdef41f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdefa49a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdef4159880 0x7fdef414a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdefa49a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef4144290 con 0x7fdef4071600 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdefa49a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef4143f90 con 0x7fdef40cb230 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdefa49a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef414b300 con 0x7fdef4159880 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4071600 0x7fdef41f7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4071600 0x7fdef41f7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51616/0 (socket says 172.21.15.31:51616) 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef3fff640 1 -- 172.21.15.31:0/3842263925 learned_addr learned my addr 172.21.15.31:0/3842263925 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef37fe640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40cb230 0x7fdef41f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef8a37640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdef4159880 0x7fdef414a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef3fff640 1 -- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdef4159880 msgr2=0x7fdef414a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef3fff640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdef4159880 0x7fdef414a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef3fff640 1 -- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40cb230 msgr2=0x7fdef41f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef3fff640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40cb230 0x7fdef41f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef3fff640 1 -- 172.21.15.31:0/3842263925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee4009420 con 0x7fdef4071600 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.463+0000 7fdef3fff640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4071600 0x7fdef41f7620 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdee4019ff0 tx=0x7fdee4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.467+0000 7fdef0ff9640 1 -- 172.21.15.31:0/3842263925 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40b4610 con 0x7fdef4071600 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.467+0000 7fdef0ff9640 1 -- 172.21.15.31:0/3842263925 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee40abe90 con 0x7fdef4071600 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.467+0000 7fdefa49a640 1 -- 172.21.15.31:0/3842263925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef4144500 con 0x7fdef4071600 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.467+0000 7fdef0ff9640 1 -- 172.21.15.31:0/3842263925 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee40b49d0 con 0x7fdef4071600 2023-11-26T07:19:06.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.467+0000 7fdefa49a640 1 -- 172.21.15.31:0/3842263925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef414b5e0 con 0x7fdef4071600 2023-11-26T07:19:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.467+0000 7fdefa49a640 1 -- 172.21.15.31:0/3842263925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef404e850 con 0x7fdef4071600 2023-11-26T07:19:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.467+0000 7fdef0ff9640 1 -- 172.21.15.31:0/3842263925 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fdee40b2040 con 0x7fdef4071600 2023-11-26T07:19:06.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.467+0000 7fdef0ff9640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdee0066620 0x7fdee0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.467+0000 7fdef0ff9640 1 -- 172.21.15.31:0/3842263925 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdee411e2c0 con 0x7fdef4071600 2023-11-26T07:19:06.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.467+0000 7fdef37fe640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdee0066620 0x7fdee0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.471+0000 7fdef37fe640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdee0066620 0x7fdee0068ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdef4166720 tx=0x7fdee8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.471+0000 7fdef0ff9640 1 -- 172.21.15.31:0/3842263925 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdee40eb460 con 0x7fdef4071600 2023-11-26T07:19:06.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.627+0000 7fdefa49a640 1 -- 172.21.15.31:0/3842263925 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdef40b3860 con 0x7fdee0066620 2023-11-26T07:19:06.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdef0ff9640 1 -- 172.21.15.31:0/3842263925 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288796 (secure 0 0 0) 0x7fdee8001510 con 0x7fdee0066620 2023-11-26T07:19:06.646 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:06.648 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:19:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdefa49a640 1 -- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdee0066620 msgr2=0x7fdee0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdefa49a640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdee0066620 0x7fdee0068ae0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdef4166720 tx=0x7fdee8009040 comp rx=0 tx=0).stop 2023-11-26T07:19:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdefa49a640 1 -- 172.21.15.31:0/3842263925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4071600 msgr2=0x7fdef41f7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdefa49a640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4071600 0x7fdef41f7620 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdee4019ff0 tx=0x7fdee4019160 comp rx=0 tx=0).stop 2023-11-26T07:19:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdefa49a640 1 -- 172.21.15.31:0/3842263925 shutdown_connections 2023-11-26T07:19:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdefa49a640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdee0066620 0x7fdee0068ae0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdefa49a640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdef4159880 0x7fdef414a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdefa49a640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40cb230 0x7fdef41f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdefa49a640 1 --2- 172.21.15.31:0/3842263925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef4071600 0x7fdef41f7620 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.639+0000 7fdefa49a640 1 -- 172.21.15.31:0/3842263925 >> 172.21.15.31:0/3842263925 conn(0x7fdef40b0c40 msgr2=0x7fdef41588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.643+0000 7fdefa49a640 1 -- 172.21.15.31:0/3842263925 shutdown_connections 2023-11-26T07:19:06.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.643+0000 7fdefa49a640 1 -- 172.21.15.31:0/3842263925 wait complete. 2023-11-26T07:19:06.667 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-11-26T07:19:06.328106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225588,"kb_used_data":5252,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749132,"statfs":{"total":773094113280,"available":772863111168,"internally_reserved":0,"allocated":5378048,"data_stored":2861769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001174"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259998+0000","last_change":"2023-11-26T07:19:02.040620+0000","last_active":"2023-11-26T07:19:03.259998+0000","last_peered":"2023-11-26T07:19:03.259998+0000","last_clean":"2023-11-26T07:19:03.259998+0000","last_became_active":"2023-11-26T07:19:02.040212+0000","last_became_peered":"2023-11-26T07:19:02.040212+0000","last_unstale":"2023-11-26T07:19:03.259998+0000","last_undegraded":"2023-11-26T07:19:03.259998+0000","last_fullsized":"2023-11-26T07:19:03.259998+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:19:21.180321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259754+0000","last_change":"2023-11-26T07:19:02.035127+0000","last_active":"2023-11-26T07:19:03.259754+0000","last_peered":"2023-11-26T07:19:03.259754+0000","last_clean":"2023-11-26T07:19:03.259754+0000","last_became_active":"2023-11-26T07:19:02.034880+0000","last_became_peered":"2023-11-26T07:19:02.034880+0000","last_unstale":"2023-11-26T07:19:03.259754+0000","last_undegraded":"2023-11-26T07:19:03.259754+0000","last_fullsized":"2023-11-26T07:19:03.259754+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:01.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016741+0000","last_change":"2023-11-26T07:19:02.028550+0000","last_active":"2023-11-26T07:19:03.016741+0000","last_peered":"2023-11-26T07:19:03.016741+0000","last_clean":"2023-11-26T07:19:03.016741+0000","last_became_active":"2023-11-26T07:19:02.028239+0000","last_became_peered":"2023-11-26T07:19:02.028239+0000","last_unstale":"2023-11-26T07:19:03.016741+0000","last_undegraded":"2023-11-26T07:19:03.016741+0000","last_fullsized":"2023-11-26T07:19:03.016741+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:13:40.745068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016329+0000","last_change":"2023-11-26T07:19:02.029858+0000","last_active":"2023-11-26T07:19:03.016329+0000","last_peered":"2023-11-26T07:19:03.016329+0000","last_clean":"2023-11-26T07:19:03.016329+0000","last_became_active":"2023-11-26T07:19:02.029564+0000","last_became_peered":"2023-11-26T07:19:02.029564+0000","last_unstale":"2023-11-26T07:19:03.016329+0000","last_undegraded":"2023-11-26T07:19:03.016329+0000","last_fullsized":"2023-11-26T07:19:03.016329+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:52:22.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259155+0000","last_change":"2023-11-26T07:19:02.036626+0000","last_active":"2023-11-26T07:19:03.259155+0000","last_peered":"2023-11-26T07:19:03.259155+0000","last_clean":"2023-11-26T07:19:03.259155+0000","last_became_active":"2023-11-26T07:19:02.036297+0000","last_became_peered":"2023-11-26T07:19:02.036297+0000","last_unstale":"2023-11-26T07:19:03.259155+0000","last_undegraded":"2023-11-26T07:19:03.259155+0000","last_fullsized":"2023-11-26T07:19:03.259155+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:04.873548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260349+0000","last_change":"2023-11-26T07:19:02.038495+0000","last_active":"2023-11-26T07:19:03.260349+0000","last_peered":"2023-11-26T07:19:03.260349+0000","last_clean":"2023-11-26T07:19:03.260349+0000","last_became_active":"2023-11-26T07:19:02.038115+0000","last_became_peered":"2023-11-26T07:19:02.038115+0000","last_unstale":"2023-11-26T07:19:03.260349+0000","last_undegraded":"2023-11-26T07:19:03.260349+0000","last_fullsized":"2023-11-26T07:19:03.260349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:59:13.586499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259427+0000","last_change":"2023-11-26T07:19:02.043211+0000","last_active":"2023-11-26T07:19:03.259427+0000","last_peered":"2023-11-26T07:19:03.259427+0000","last_clean":"2023-11-26T07:19:03.259427+0000","last_became_active":"2023-11-26T07:19:02.042795+0000","last_became_peered":"2023-11-26T07:19:02.042795+0000","last_unstale":"2023-11-26T07:19:03.259427+0000","last_undegraded":"2023-11-26T07:19:03.259427+0000","last_fullsized":"2023-11-26T07:19:03.259427+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:01.447677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.015905+0000","last_change":"2023-11-26T07:19:02.030271+0000","last_active":"2023-11-26T07:19:03.015905+0000","last_peered":"2023-11-26T07:19:03.015905+0000","last_clean":"2023-11-26T07:19:03.015905+0000","last_became_active":"2023-11-26T07:19:02.029473+0000","last_became_peered":"2023-11-26T07:19:02.029473+0000","last_unstale":"2023-11-26T07:19:03.015905+0000","last_undegraded":"2023-11-26T07:19:03.015905+0000","last_fullsized":"2023-11-26T07:19:03.015905+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:25:52.132340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016745+0000","last_change":"2023-11-26T07:19:02.039278+0000","last_active":"2023-11-26T07:19:03.016745+0000","last_peered":"2023-11-26T07:19:03.016745+0000","last_clean":"2023-11-26T07:19:03.016745+0000","last_became_active":"2023-11-26T07:19:02.035403+0000","last_became_peered":"2023-11-26T07:19:02.035403+0000","last_unstale":"2023-11-26T07:19:03.016745+0000","last_undegraded":"2023-11-26T07:19:03.016745+0000","last_fullsized":"2023-11-26T07:19:03.016745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:27.196740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016787+0000","last_change":"2023-11-26T07:19:02.030301+0000","last_active":"2023-11-26T07:19:03.016787+0000","last_peered":"2023-11-26T07:19:03.016787+0000","last_clean":"2023-11-26T07:19:03.016787+0000","last_became_active":"2023-11-26T07:19:02.029981+0000","last_became_peered":"2023-11-26T07:19:02.029981+0000","last_unstale":"2023-11-26T07:19:03.016787+0000","last_undegraded":"2023-11-26T07:19:03.016787+0000","last_fullsized":"2023-11-26T07:19:03.016787+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:19:13.650698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016612+0000","last_change":"2023-11-26T07:19:02.035998+0000","last_active":"2023-11-26T07:19:03.016612+0000","last_peered":"2023-11-26T07:19:03.016612+0000","last_clean":"2023-11-26T07:19:03.016612+0000","last_became_active":"2023-11-26T07:19:02.035723+0000","last_became_peered":"2023-11-26T07:19:02.035723+0000","last_unstale":"2023-11-26T07:19:03.016612+0000","last_undegraded":"2023-11-26T07:19:03.016612+0000","last_fullsized":"2023-11-26T07:19:03.016612+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:29:18.941647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259343+0000","last_change":"2023-11-26T07:19:02.034250+0000","last_active":"2023-11-26T07:19:03.259343+0000","last_peered":"2023-11-26T07:19:03.259343+0000","last_clean":"2023-11-26T07:19:03.259343+0000","last_became_active":"2023-11-26T07:19:02.033926+0000","last_became_peered":"2023-11-26T07:19:02.033926+0000","last_unstale":"2023-11-26T07:19:03.259343+0000","last_undegraded":"2023-11-26T07:19:03.259343+0000","last_fullsized":"2023-11-26T07:19:03.259343+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:19:13.650698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259632+0000","last_change":"2023-11-26T07:19:02.042087+0000","last_active":"2023-11-26T07:19:03.259632+0000","last_peered":"2023-11-26T07:19:03.259632+0000","last_clean":"2023-11-26T07:19:03.259632+0000","last_became_active":"2023-11-26T07:19:02.041764+0000","last_became_peered":"2023-11-26T07:19:02.041764+0000","last_unstale":"2023-11-26T07:19:03.259632+0000","last_undegraded":"2023-11-26T07:19:03.259632+0000","last_fullsized":"2023-11-26T07:19:03.259632+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:34:18.897026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016175+0000","last_change":"2023-11-26T07:19:02.031065+0000","last_active":"2023-11-26T07:19:03.016175+0000","last_peered":"2023-11-26T07:19:03.016175+0000","last_clean":"2023-11-26T07:19:03.016175+0000","last_became_active":"2023-11-26T07:19:02.030743+0000","last_became_peered":"2023-11-26T07:19:02.030743+0000","last_unstale":"2023-11-26T07:19:03.016175+0000","last_undegraded":"2023-11-26T07:19:03.016175+0000","last_fullsized":"2023-11-26T07:19:03.016175+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:32.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260082+0000","last_change":"2023-11-26T07:19:02.027519+0000","last_active":"2023-11-26T07:19:03.260082+0000","last_peered":"2023-11-26T07:19:03.260082+0000","last_clean":"2023-11-26T07:19:03.260082+0000","last_became_active":"2023-11-26T07:19:02.027249+0000","last_became_peered":"2023-11-26T07:19:02.027249+0000","last_unstale":"2023-11-26T07:19:03.260082+0000","last_undegraded":"2023-11-26T07:19:03.260082+0000","last_fullsized":"2023-11-26T07:19:03.260082+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:42:06.812332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016518+0000","last_change":"2023-11-26T07:19:02.027426+0000","last_active":"2023-11-26T07:19:03.016518+0000","last_peered":"2023-11-26T07:19:03.016518+0000","last_clean":"2023-11-26T07:19:03.016518+0000","last_became_active":"2023-11-26T07:19:02.027058+0000","last_became_peered":"2023-11-26T07:19:02.027058+0000","last_unstale":"2023-11-26T07:19:03.016518+0000","last_undegraded":"2023-11-26T07:19:03.016518+0000","last_fullsized":"2023-11-26T07:19:03.016518+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:46.728379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017181+0000","last_change":"2023-11-26T07:19:02.030606+0000","last_active":"2023-11-26T07:19:03.017181+0000","last_peered":"2023-11-26T07:19:03.017181+0000","last_clean":"2023-11-26T07:19:03.017181+0000","last_became_active":"2023-11-26T07:19:02.030274+0000","last_became_peered":"2023-11-26T07:19:02.030274+0000","last_unstale":"2023-11-26T07:19:03.017181+0000","last_undegraded":"2023-11-26T07:19:03.017181+0000","last_fullsized":"2023-11-26T07:19:03.017181+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:32.797532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017192+0000","last_change":"2023-11-26T07:19:02.038185+0000","last_active":"2023-11-26T07:19:03.017192+0000","last_peered":"2023-11-26T07:19:03.017192+0000","last_clean":"2023-11-26T07:19:03.017192+0000","last_became_active":"2023-11-26T07:19:02.037419+0000","last_became_peered":"2023-11-26T07:19:02.037419+0000","last_unstale":"2023-11-26T07:19:03.017192+0000","last_undegraded":"2023-11-26T07:19:03.017192+0000","last_fullsized":"2023-11-26T07:19:03.017192+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:15.733346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017573+0000","last_change":"2023-11-26T07:19:02.027464+0000","last_active":"2023-11-26T07:19:03.017573+0000","last_peered":"2023-11-26T07:19:03.017573+0000","last_clean":"2023-11-26T07:19:03.017573+0000","last_became_active":"2023-11-26T07:19:02.026056+0000","last_became_peered":"2023-11-26T07:19:02.026056+0000","last_unstale":"2023-11-26T07:19:03.017573+0000","last_undegraded":"2023-11-26T07:19:03.017573+0000","last_fullsized":"2023-11-26T07:19:03.017573+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:48.304523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259771+0000","last_change":"2023-11-26T07:19:02.038927+0000","last_active":"2023-11-26T07:19:03.259771+0000","last_peered":"2023-11-26T07:19:03.259771+0000","last_clean":"2023-11-26T07:19:03.259771+0000","last_became_active":"2023-11-26T07:19:02.038660+0000","last_became_peered":"2023-11-26T07:19:02.038660+0000","last_unstale":"2023-11-26T07:19:03.259771+0000","last_undegraded":"2023-11-26T07:19:03.259771+0000","last_fullsized":"2023-11-26T07:19:03.259771+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:23:40.607968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260321+0000","last_change":"2023-11-26T07:19:02.036873+0000","last_active":"2023-11-26T07:19:03.260321+0000","last_peered":"2023-11-26T07:19:03.260321+0000","last_clean":"2023-11-26T07:19:03.260321+0000","last_became_active":"2023-11-26T07:19:02.036105+0000","last_became_peered":"2023-11-26T07:19:02.036105+0000","last_unstale":"2023-11-26T07:19:03.260321+0000","last_undegraded":"2023-11-26T07:19:03.260321+0000","last_fullsized":"2023-11-26T07:19:03.260321+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:29:54.602515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016183+0000","last_change":"2023-11-26T07:19:02.036194+0000","last_active":"2023-11-26T07:19:03.016183+0000","last_peered":"2023-11-26T07:19:03.016183+0000","last_clean":"2023-11-26T07:19:03.016183+0000","last_became_active":"2023-11-26T07:19:02.035925+0000","last_became_peered":"2023-11-26T07:19:02.035925+0000","last_unstale":"2023-11-26T07:19:03.016183+0000","last_undegraded":"2023-11-26T07:19:03.016183+0000","last_fullsized":"2023-11-26T07:19:03.016183+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:15.733346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016700+0000","last_change":"2023-11-26T07:19:02.032544+0000","last_active":"2023-11-26T07:19:03.016700+0000","last_peered":"2023-11-26T07:19:03.016700+0000","last_clean":"2023-11-26T07:19:03.016700+0000","last_became_active":"2023-11-26T07:19:02.032089+0000","last_became_peered":"2023-11-26T07:19:02.032089+0000","last_unstale":"2023-11-26T07:19:03.016700+0000","last_undegraded":"2023-11-26T07:19:03.016700+0000","last_fullsized":"2023-11-26T07:19:03.016700+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:29:34.573578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016674+0000","last_change":"2023-11-26T07:19:02.035387+0000","last_active":"2023-11-26T07:19:03.016674+0000","last_peered":"2023-11-26T07:19:03.016674+0000","last_clean":"2023-11-26T07:19:03.016674+0000","last_became_active":"2023-11-26T07:19:02.035001+0000","last_became_peered":"2023-11-26T07:19:02.035001+0000","last_unstale":"2023-11-26T07:19:03.016674+0000","last_undegraded":"2023-11-26T07:19:03.016674+0000","last_fullsized":"2023-11-26T07:19:03.016674+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:19:46.614464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017081+0000","last_change":"2023-11-26T07:19:02.031206+0000","last_active":"2023-11-26T07:19:03.017081+0000","last_peered":"2023-11-26T07:19:03.017081+0000","last_clean":"2023-11-26T07:19:03.017081+0000","last_became_active":"2023-11-26T07:19:02.028267+0000","last_became_peered":"2023-11-26T07:19:02.028267+0000","last_unstale":"2023-11-26T07:19:03.017081+0000","last_undegraded":"2023-11-26T07:19:03.017081+0000","last_fullsized":"2023-11-26T07:19:03.017081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:04:57.697304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259588+0000","last_change":"2023-11-26T07:19:02.043859+0000","last_active":"2023-11-26T07:19:03.259588+0000","last_peered":"2023-11-26T07:19:03.259588+0000","last_clean":"2023-11-26T07:19:03.259588+0000","last_became_active":"2023-11-26T07:19:02.043510+0000","last_became_peered":"2023-11-26T07:19:02.043510+0000","last_unstale":"2023-11-26T07:19:03.259588+0000","last_undegraded":"2023-11-26T07:19:03.259588+0000","last_fullsized":"2023-11-26T07:19:03.259588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:49:25.962262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017643+0000","last_change":"2023-11-26T07:19:02.024202+0000","last_active":"2023-11-26T07:19:03.017643+0000","last_peered":"2023-11-26T07:19:03.017643+0000","last_clean":"2023-11-26T07:19:03.017643+0000","last_became_active":"2023-11-26T07:19:02.023902+0000","last_became_peered":"2023-11-26T07:19:02.023902+0000","last_unstale":"2023-11-26T07:19:03.017643+0000","last_undegraded":"2023-11-26T07:19:03.017643+0000","last_fullsized":"2023-11-26T07:19:03.017643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:52:22.152426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259467+0000","last_change":"2023-11-26T07:19:02.029692+0000","last_active":"2023-11-26T07:19:03.259467+0000","last_peered":"2023-11-26T07:19:03.259467+0000","last_clean":"2023-11-26T07:19:03.259467+0000","last_became_active":"2023-11-26T07:19:02.029383+0000","last_became_peered":"2023-11-26T07:19:02.029383+0000","last_unstale":"2023-11-26T07:19:03.259467+0000","last_undegraded":"2023-11-26T07:19:03.259467+0000","last_fullsized":"2023-11-26T07:19:03.259467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:52.007636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017153+0000","last_change":"2023-11-26T07:19:02.027661+0000","last_active":"2023-11-26T07:19:03.017153+0000","last_peered":"2023-11-26T07:19:03.017153+0000","last_clean":"2023-11-26T07:19:03.017153+0000","last_became_active":"2023-11-26T07:19:02.027407+0000","last_became_peered":"2023-11-26T07:19:02.027407+0000","last_unstale":"2023-11-26T07:19:03.017153+0000","last_undegraded":"2023-11-26T07:19:03.017153+0000","last_fullsized":"2023-11-26T07:19:03.017153+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:04:02.104197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016154+0000","last_change":"2023-11-26T07:19:02.034152+0000","last_active":"2023-11-26T07:19:03.016154+0000","last_peered":"2023-11-26T07:19:03.016154+0000","last_clean":"2023-11-26T07:19:03.016154+0000","last_became_active":"2023-11-26T07:19:02.033809+0000","last_became_peered":"2023-11-26T07:19:02.033809+0000","last_unstale":"2023-11-26T07:19:03.016154+0000","last_undegraded":"2023-11-26T07:19:03.016154+0000","last_fullsized":"2023-11-26T07:19:03.016154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:25:52.132340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260685+0000","last_change":"2023-11-26T07:19:02.028093+0000","last_active":"2023-11-26T07:19:03.260685+0000","last_peered":"2023-11-26T07:19:03.260685+0000","last_clean":"2023-11-26T07:19:03.260685+0000","last_became_active":"2023-11-26T07:19:02.026870+0000","last_became_peered":"2023-11-26T07:19:02.026870+0000","last_unstale":"2023-11-26T07:19:03.260685+0000","last_undegraded":"2023-11-26T07:19:03.260685+0000","last_fullsized":"2023-11-26T07:19:03.260685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:41:14.036019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259307+0000","last_change":"2023-11-26T07:19:02.030546+0000","last_active":"2023-11-26T07:19:03.259307+0000","last_peered":"2023-11-26T07:19:03.259307+0000","last_clean":"2023-11-26T07:19:03.259307+0000","last_became_active":"2023-11-26T07:19:02.030142+0000","last_became_peered":"2023-11-26T07:19:02.030142+0000","last_unstale":"2023-11-26T07:19:03.259307+0000","last_undegraded":"2023-11-26T07:19:03.259307+0000","last_fullsized":"2023-11-26T07:19:03.259307+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:39:04.246982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260046+0000","last_change":"2023-11-26T07:19:02.032138+0000","last_active":"2023-11-26T07:19:03.260046+0000","last_peered":"2023-11-26T07:19:03.260046+0000","last_clean":"2023-11-26T07:19:03.260046+0000","last_became_active":"2023-11-26T07:19:02.031890+0000","last_became_peered":"2023-11-26T07:19:02.031890+0000","last_unstale":"2023-11-26T07:19:03.260046+0000","last_undegraded":"2023-11-26T07:19:03.260046+0000","last_fullsized":"2023-11-26T07:19:03.260046+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:07:29.584444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.177624+0000","last_change":"2023-11-26T07:19:01.185997+0000","last_active":"2023-11-26T07:19:02.177624+0000","last_peered":"2023-11-26T07:19:02.177624+0000","last_clean":"2023-11-26T07:19:02.177624+0000","last_became_active":"2023-11-26T07:19:01.184511+0000","last_became_peered":"2023-11-26T07:19:01.184511+0000","last_unstale":"2023-11-26T07:19:02.177624+0000","last_undegraded":"2023-11-26T07:19:02.177624+0000","last_fullsized":"2023-11-26T07:19:02.177624+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:43:30.461349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017230+0000","last_change":"2023-11-26T07:19:02.033596+0000","last_active":"2023-11-26T07:19:03.017230+0000","last_peered":"2023-11-26T07:19:03.017230+0000","last_clean":"2023-11-26T07:19:03.017230+0000","last_became_active":"2023-11-26T07:19:02.028752+0000","last_became_peered":"2023-11-26T07:19:02.028752+0000","last_unstale":"2023-11-26T07:19:03.017230+0000","last_undegraded":"2023-11-26T07:19:03.017230+0000","last_fullsized":"2023-11-26T07:19:03.017230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:11.829132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.176438+0000","last_change":"2023-11-26T07:19:01.174676+0000","last_active":"2023-11-26T07:19:02.176438+0000","last_peered":"2023-11-26T07:19:02.176438+0000","last_clean":"2023-11-26T07:19:02.176438+0000","last_became_active":"2023-11-26T07:19:01.174090+0000","last_became_peered":"2023-11-26T07:19:01.174090+0000","last_unstale":"2023-11-26T07:19:02.176438+0000","last_undegraded":"2023-11-26T07:19:02.176438+0000","last_fullsized":"2023-11-26T07:19:02.176438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:27:59.906430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259547+0000","last_change":"2023-11-26T07:19:02.042463+0000","last_active":"2023-11-26T07:19:03.259547+0000","last_peered":"2023-11-26T07:19:03.259547+0000","last_clean":"2023-11-26T07:19:03.259547+0000","last_became_active":"2023-11-26T07:19:02.042268+0000","last_became_peered":"2023-11-26T07:19:02.042268+0000","last_unstale":"2023-11-26T07:19:03.259547+0000","last_undegraded":"2023-11-26T07:19:03.259547+0000","last_fullsized":"2023-11-26T07:19:03.259547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:17:22.619189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.178342+0000","last_change":"2023-11-26T07:19:01.182588+0000","last_active":"2023-11-26T07:19:02.178342+0000","last_peered":"2023-11-26T07:19:02.178342+0000","last_clean":"2023-11-26T07:19:02.178342+0000","last_became_active":"2023-11-26T07:19:01.182316+0000","last_became_peered":"2023-11-26T07:19:01.182316+0000","last_unstale":"2023-11-26T07:19:02.178342+0000","last_undegraded":"2023-11-26T07:19:02.178342+0000","last_fullsized":"2023-11-26T07:19:02.178342+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:04:01.101347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260106+0000","last_change":"2023-11-26T07:19:02.041864+0000","last_active":"2023-11-26T07:19:03.260106+0000","last_peered":"2023-11-26T07:19:03.260106+0000","last_clean":"2023-11-26T07:19:03.260106+0000","last_became_active":"2023-11-26T07:19:02.041590+0000","last_became_peered":"2023-11-26T07:19:02.041590+0000","last_unstale":"2023-11-26T07:19:03.260106+0000","last_undegraded":"2023-11-26T07:19:03.260106+0000","last_fullsized":"2023-11-26T07:19:03.260106+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:48.304523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.172245+0000","last_change":"2023-11-26T07:19:01.173907+0000","last_active":"2023-11-26T07:19:02.172245+0000","last_peered":"2023-11-26T07:19:02.172245+0000","last_clean":"2023-11-26T07:19:02.172245+0000","last_became_active":"2023-11-26T07:19:01.173540+0000","last_became_peered":"2023-11-26T07:19:01.173540+0000","last_unstale":"2023-11-26T07:19:02.172245+0000","last_undegraded":"2023-11-26T07:19:02.172245+0000","last_fullsized":"2023-11-26T07:19:02.172245+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:13:39.742217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016294+0000","last_change":"2023-11-26T07:19:02.028718+0000","last_active":"2023-11-26T07:19:03.016294+0000","last_peered":"2023-11-26T07:19:03.016294+0000","last_clean":"2023-11-26T07:19:03.016294+0000","last_became_active":"2023-11-26T07:19:02.027929+0000","last_became_peered":"2023-11-26T07:19:02.027929+0000","last_unstale":"2023-11-26T07:19:03.016294+0000","last_undegraded":"2023-11-26T07:19:03.016294+0000","last_fullsized":"2023-11-26T07:19:03.016294+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:51:15.180329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.175570+0000","last_change":"2023-11-26T07:19:01.179813+0000","last_active":"2023-11-26T07:19:02.175570+0000","last_peered":"2023-11-26T07:19:02.175570+0000","last_clean":"2023-11-26T07:19:02.175570+0000","last_became_active":"2023-11-26T07:19:01.179053+0000","last_became_peered":"2023-11-26T07:19:01.179053+0000","last_unstale":"2023-11-26T07:19:02.175570+0000","last_undegraded":"2023-11-26T07:19:02.175570+0000","last_fullsized":"2023-11-26T07:19:02.175570+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:51.004786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016247+0000","last_change":"2023-11-26T07:19:02.032610+0000","last_active":"2023-11-26T07:19:03.016247+0000","last_peered":"2023-11-26T07:19:03.016247+0000","last_clean":"2023-11-26T07:19:03.016247+0000","last_became_active":"2023-11-26T07:19:02.032204+0000","last_became_peered":"2023-11-26T07:19:02.032204+0000","last_unstale":"2023-11-26T07:19:03.016247+0000","last_undegraded":"2023-11-26T07:19:03.016247+0000","last_fullsized":"2023-11-26T07:19:03.016247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:39:48.888963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.176944+0000","last_change":"2023-11-26T07:19:01.172607+0000","last_active":"2023-11-26T07:19:02.176944+0000","last_peered":"2023-11-26T07:19:02.176944+0000","last_clean":"2023-11-26T07:19:02.176944+0000","last_became_active":"2023-11-26T07:19:01.172248+0000","last_became_peered":"2023-11-26T07:19:01.172248+0000","last_unstale":"2023-11-26T07:19:02.176944+0000","last_undegraded":"2023-11-26T07:19:02.176944+0000","last_fullsized":"2023-11-26T07:19:02.176944+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:19:45.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.015944+0000","last_change":"2023-11-26T07:19:02.030496+0000","last_active":"2023-11-26T07:19:03.015944+0000","last_peered":"2023-11-26T07:19:03.015944+0000","last_clean":"2023-11-26T07:19:03.015944+0000","last_became_active":"2023-11-26T07:19:02.029921+0000","last_became_peered":"2023-11-26T07:19:02.029921+0000","last_unstale":"2023-11-26T07:19:03.015944+0000","last_undegraded":"2023-11-26T07:19:03.015944+0000","last_fullsized":"2023-11-26T07:19:03.015944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:13:40.745068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.020425+0000","last_change":"2023-11-26T07:19:01.186853+0000","last_active":"2023-11-26T07:19:02.020425+0000","last_peered":"2023-11-26T07:19:02.020425+0000","last_clean":"2023-11-26T07:19:02.020425+0000","last_became_active":"2023-11-26T07:19:01.186574+0000","last_became_peered":"2023-11-26T07:19:01.186574+0000","last_unstale":"2023-11-26T07:19:02.020425+0000","last_undegraded":"2023-11-26T07:19:02.020425+0000","last_fullsized":"2023-11-26T07:19:02.020425+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:31.794682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016478+0000","last_change":"2023-11-26T07:19:02.033072+0000","last_active":"2023-11-26T07:19:03.016478+0000","last_peered":"2023-11-26T07:19:03.016478+0000","last_clean":"2023-11-26T07:19:03.016478+0000","last_became_active":"2023-11-26T07:19:02.032464+0000","last_became_peered":"2023-11-26T07:19:02.032464+0000","last_unstale":"2023-11-26T07:19:03.016478+0000","last_undegraded":"2023-11-26T07:19:03.016478+0000","last_fullsized":"2023-11-26T07:19:03.016478+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:54:50.867507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.014257+0000","last_change":"2023-11-26T07:19:01.187344+0000","last_active":"2023-11-26T07:19:02.014257+0000","last_peered":"2023-11-26T07:19:02.014257+0000","last_clean":"2023-11-26T07:19:02.014257+0000","last_became_active":"2023-11-26T07:19:01.187078+0000","last_became_peered":"2023-11-26T07:19:01.187078+0000","last_unstale":"2023-11-26T07:19:02.014257+0000","last_undegraded":"2023-11-26T07:19:02.014257+0000","last_fullsized":"2023-11-26T07:19:02.014257+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:58:20.050151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017341+0000","last_change":"2023-11-26T07:19:02.025621+0000","last_active":"2023-11-26T07:19:03.017341+0000","last_peered":"2023-11-26T07:19:03.017341+0000","last_clean":"2023-11-26T07:19:03.017341+0000","last_became_active":"2023-11-26T07:19:02.025356+0000","last_became_peered":"2023-11-26T07:19:02.025356+0000","last_unstale":"2023-11-26T07:19:03.017341+0000","last_undegraded":"2023-11-26T07:19:03.017341+0000","last_fullsized":"2023-11-26T07:19:03.017341+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:39:48.888963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.022510+0000","last_change":"2023-11-26T07:19:01.187011+0000","last_active":"2023-11-26T07:19:02.022510+0000","last_peered":"2023-11-26T07:19:02.022510+0000","last_clean":"2023-11-26T07:19:02.022510+0000","last_became_active":"2023-11-26T07:19:01.186742+0000","last_became_peered":"2023-11-26T07:19:01.186742+0000","last_unstale":"2023-11-26T07:19:02.022510+0000","last_undegraded":"2023-11-26T07:19:02.022510+0000","last_fullsized":"2023-11-26T07:19:02.022510+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:07:28.581594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259977+0000","last_change":"2023-11-26T07:19:02.041653+0000","last_active":"2023-11-26T07:19:03.259977+0000","last_peered":"2023-11-26T07:19:03.259977+0000","last_clean":"2023-11-26T07:19:03.259977+0000","last_became_active":"2023-11-26T07:19:02.041323+0000","last_became_peered":"2023-11-26T07:19:02.041323+0000","last_unstale":"2023-11-26T07:19:03.259977+0000","last_undegraded":"2023-11-26T07:19:03.259977+0000","last_fullsized":"2023-11-26T07:19:03.259977+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:41:21.349273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.014430+0000","last_change":"2023-11-26T07:19:01.182623+0000","last_active":"2023-11-26T07:19:02.014430+0000","last_peered":"2023-11-26T07:19:02.014430+0000","last_clean":"2023-11-26T07:19:02.014430+0000","last_became_active":"2023-11-26T07:19:01.182464+0000","last_became_peered":"2023-11-26T07:19:01.182464+0000","last_unstale":"2023-11-26T07:19:02.014430+0000","last_undegraded":"2023-11-26T07:19:02.014430+0000","last_fullsized":"2023-11-26T07:19:02.014430+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:19:45.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260799+0000","last_change":"2023-11-26T07:19:02.031178+0000","last_active":"2023-11-26T07:19:03.260799+0000","last_peered":"2023-11-26T07:19:03.260799+0000","last_clean":"2023-11-26T07:19:03.260799+0000","last_became_active":"2023-11-26T07:19:02.030900+0000","last_became_peered":"2023-11-26T07:19:02.030900+0000","last_unstale":"2023-11-26T07:19:03.260799+0000","last_undegraded":"2023-11-26T07:19:03.260799+0000","last_fullsized":"2023-11-26T07:19:03.260799+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:27:22.793469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.176603+0000","last_change":"2023-11-26T07:19:01.181606+0000","last_active":"2023-11-26T07:19:02.176603+0000","last_peered":"2023-11-26T07:19:02.176603+0000","last_clean":"2023-11-26T07:19:02.176603+0000","last_became_active":"2023-11-26T07:19:01.181166+0000","last_became_peered":"2023-11-26T07:19:01.181166+0000","last_unstale":"2023-11-26T07:19:02.176603+0000","last_undegraded":"2023-11-26T07:19:02.176603+0000","last_fullsized":"2023-11-26T07:19:02.176603+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:03.870698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259293+0000","last_change":"2023-11-26T07:19:02.044950+0000","last_active":"2023-11-26T07:19:03.259293+0000","last_peered":"2023-11-26T07:19:03.259293+0000","last_clean":"2023-11-26T07:19:03.259293+0000","last_became_active":"2023-11-26T07:19:02.044613+0000","last_became_peered":"2023-11-26T07:19:02.044613+0000","last_unstale":"2023-11-26T07:19:03.259293+0000","last_undegraded":"2023-11-26T07:19:03.259293+0000","last_fullsized":"2023-11-26T07:19:03.259293+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:17:22.619189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.182470+0000","last_change":"2023-11-26T07:19:01.184226+0000","last_active":"2023-11-26T07:19:02.182470+0000","last_peered":"2023-11-26T07:19:02.182470+0000","last_clean":"2023-11-26T07:19:02.182470+0000","last_became_active":"2023-11-26T07:19:01.183946+0000","last_became_peered":"2023-11-26T07:19:01.183946+0000","last_unstale":"2023-11-26T07:19:02.182470+0000","last_undegraded":"2023-11-26T07:19:02.182470+0000","last_fullsized":"2023-11-26T07:19:02.182470+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:55:56.937756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016846+0000","last_change":"2023-11-26T07:19:02.026288+0000","last_active":"2023-11-26T07:19:03.016846+0000","last_peered":"2023-11-26T07:19:03.016846+0000","last_clean":"2023-11-26T07:19:03.016846+0000","last_became_active":"2023-11-26T07:19:02.025964+0000","last_became_peered":"2023-11-26T07:19:02.025964+0000","last_unstale":"2023-11-26T07:19:03.016846+0000","last_undegraded":"2023-11-26T07:19:03.016846+0000","last_fullsized":"2023-11-26T07:19:03.016846+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:08:13.424795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.177166+0000","last_change":"2023-11-26T07:19:01.184345+0000","last_active":"2023-11-26T07:19:02.177166+0000","last_peered":"2023-11-26T07:19:02.177166+0000","last_clean":"2023-11-26T07:19:02.177166+0000","last_became_active":"2023-11-26T07:19:01.184059+0000","last_became_peered":"2023-11-26T07:19:01.184059+0000","last_unstale":"2023-11-26T07:19:02.177166+0000","last_undegraded":"2023-11-26T07:19:02.177166+0000","last_fullsized":"2023-11-26T07:19:02.177166+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:17:21.616339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016363+0000","last_change":"2023-11-26T07:19:02.036697+0000","last_active":"2023-11-26T07:19:03.016363+0000","last_peered":"2023-11-26T07:19:03.016363+0000","last_clean":"2023-11-26T07:19:03.016363+0000","last_became_active":"2023-11-26T07:19:02.036392+0000","last_became_peered":"2023-11-26T07:19:02.036392+0000","last_unstale":"2023-11-26T07:19:03.016363+0000","last_undegraded":"2023-11-26T07:19:03.016363+0000","last_fullsized":"2023-11-26T07:19:03.016363+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:38.623200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.032070+0000","last_change":"2023-11-26T07:19:01.186309+0000","last_active":"2023-11-26T07:19:02.032070+0000","last_peered":"2023-11-26T07:19:02.032070+0000","last_clean":"2023-11-26T07:19:02.032070+0000","last_became_active":"2023-11-26T07:19:01.185977+0000","last_became_peered":"2023-11-26T07:19:01.185977+0000","last_unstale":"2023-11-26T07:19:02.032070+0000","last_undegraded":"2023-11-26T07:19:02.032070+0000","last_fullsized":"2023-11-26T07:19:02.032070+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:31.794682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260052+0000","last_change":"2023-11-26T07:19:02.030552+0000","last_active":"2023-11-26T07:19:03.260052+0000","last_peered":"2023-11-26T07:19:03.260052+0000","last_clean":"2023-11-26T07:19:03.260052+0000","last_became_active":"2023-11-26T07:19:02.030344+0000","last_became_peered":"2023-11-26T07:19:02.030344+0000","last_unstale":"2023-11-26T07:19:03.260052+0000","last_undegraded":"2023-11-26T07:19:03.260052+0000","last_fullsized":"2023-11-26T07:19:03.260052+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:10:01.313109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.022028+0000","last_change":"2023-11-26T07:19:01.188066+0000","last_active":"2023-11-26T07:19:02.022028+0000","last_peered":"2023-11-26T07:19:02.022028+0000","last_clean":"2023-11-26T07:19:02.022028+0000","last_became_active":"2023-11-26T07:19:01.184271+0000","last_became_peered":"2023-11-26T07:19:01.184271+0000","last_unstale":"2023-11-26T07:19:02.022028+0000","last_undegraded":"2023-11-26T07:19:02.022028+0000","last_fullsized":"2023-11-26T07:19:02.022028+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:19:12.647848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259568+0000","last_change":"2023-11-26T07:19:02.044414+0000","last_active":"2023-11-26T07:19:03.259568+0000","last_peered":"2023-11-26T07:19:03.259568+0000","last_clean":"2023-11-26T07:19:03.259568+0000","last_became_active":"2023-11-26T07:19:02.044163+0000","last_became_peered":"2023-11-26T07:19:02.044163+0000","last_unstale":"2023-11-26T07:19:03.259568+0000","last_undegraded":"2023-11-26T07:19:03.259568+0000","last_fullsized":"2023-11-26T07:19:03.259568+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:43:31.464199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.180092+0000","last_change":"2023-11-26T07:19:01.181598+0000","last_active":"2023-11-26T07:19:02.180092+0000","last_peered":"2023-11-26T07:19:02.180092+0000","last_clean":"2023-11-26T07:19:02.180092+0000","last_became_active":"2023-11-26T07:19:01.181292+0000","last_became_peered":"2023-11-26T07:19:01.181292+0000","last_unstale":"2023-11-26T07:19:02.180092+0000","last_undegraded":"2023-11-26T07:19:02.180092+0000","last_fullsized":"2023-11-26T07:19:02.180092+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:45.725529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259733+0000","last_change":"2023-11-26T07:19:02.030053+0000","last_active":"2023-11-26T07:19:03.259733+0000","last_peered":"2023-11-26T07:19:03.259733+0000","last_clean":"2023-11-26T07:19:03.259733+0000","last_became_active":"2023-11-26T07:19:02.028564+0000","last_became_peered":"2023-11-26T07:19:02.028564+0000","last_unstale":"2023-11-26T07:19:03.259733+0000","last_undegraded":"2023-11-26T07:19:03.259733+0000","last_fullsized":"2023-11-26T07:19:03.259733+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:48.948343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.176432+0000","last_change":"2023-11-26T07:19:01.193066+0000","last_active":"2023-11-26T07:19:02.176432+0000","last_peered":"2023-11-26T07:19:02.176432+0000","last_clean":"2023-11-26T07:19:02.176432+0000","last_became_active":"2023-11-26T07:19:01.192530+0000","last_became_peered":"2023-11-26T07:19:01.192530+0000","last_unstale":"2023-11-26T07:19:02.176432+0000","last_undegraded":"2023-11-26T07:19:02.176432+0000","last_fullsized":"2023-11-26T07:19:02.176432+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:20:48.722094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016863+0000","last_change":"2023-11-26T07:19:02.032465+0000","last_active":"2023-11-26T07:19:03.016863+0000","last_peered":"2023-11-26T07:19:03.016863+0000","last_clean":"2023-11-26T07:19:03.016863+0000","last_became_active":"2023-11-26T07:19:02.031718+0000","last_became_peered":"2023-11-26T07:19:02.031718+0000","last_unstale":"2023-11-26T07:19:03.016863+0000","last_undegraded":"2023-11-26T07:19:03.016863+0000","last_fullsized":"2023-11-26T07:19:03.016863+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:52.007636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.176385+0000","last_change":"2023-11-26T07:19:01.186618+0000","last_active":"2023-11-26T07:19:02.176385+0000","last_peered":"2023-11-26T07:19:02.176385+0000","last_clean":"2023-11-26T07:19:02.176385+0000","last_became_active":"2023-11-26T07:19:01.186260+0000","last_became_peered":"2023-11-26T07:19:01.186260+0000","last_unstale":"2023-11-26T07:19:02.176385+0000","last_undegraded":"2023-11-26T07:19:02.176385+0000","last_fullsized":"2023-11-26T07:19:02.176385+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:04:01.101347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017090+0000","last_change":"2023-11-26T07:19:02.037881+0000","last_active":"2023-11-26T07:19:03.017090+0000","last_peered":"2023-11-26T07:19:03.017090+0000","last_clean":"2023-11-26T07:19:03.017090+0000","last_became_active":"2023-11-26T07:19:02.037027+0000","last_became_peered":"2023-11-26T07:19:02.037027+0000","last_unstale":"2023-11-26T07:19:03.017090+0000","last_undegraded":"2023-11-26T07:19:03.017090+0000","last_fullsized":"2023-11-26T07:19:03.017090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:00.909280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.173106+0000","last_change":"2023-11-26T07:19:01.186515+0000","last_active":"2023-11-26T07:19:02.173106+0000","last_peered":"2023-11-26T07:19:02.173106+0000","last_clean":"2023-11-26T07:19:02.173106+0000","last_became_active":"2023-11-26T07:19:01.186208+0000","last_became_peered":"2023-11-26T07:19:01.186208+0000","last_unstale":"2023-11-26T07:19:02.173106+0000","last_undegraded":"2023-11-26T07:19:02.173106+0000","last_fullsized":"2023-11-26T07:19:02.173106+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:52:21.149576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017350+0000","last_change":"2023-11-26T07:19:02.036737+0000","last_active":"2023-11-26T07:19:03.017350+0000","last_peered":"2023-11-26T07:19:03.017350+0000","last_clean":"2023-11-26T07:19:03.017350+0000","last_became_active":"2023-11-26T07:19:02.035998+0000","last_became_peered":"2023-11-26T07:19:02.035998+0000","last_unstale":"2023-11-26T07:19:03.017350+0000","last_undegraded":"2023-11-26T07:19:03.017350+0000","last_fullsized":"2023-11-26T07:19:03.017350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:07:29.584444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.023135+0000","last_change":"2023-11-26T07:19:01.185736+0000","last_active":"2023-11-26T07:19:02.023135+0000","last_peered":"2023-11-26T07:19:02.023135+0000","last_clean":"2023-11-26T07:19:02.023135+0000","last_became_active":"2023-11-26T07:19:01.185181+0000","last_became_peered":"2023-11-26T07:19:01.185181+0000","last_unstale":"2023-11-26T07:19:02.023135+0000","last_undegraded":"2023-11-26T07:19:02.023135+0000","last_fullsized":"2023-11-26T07:19:02.023135+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:41:20.346422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260489+0000","last_change":"2023-11-26T07:19:02.032265+0000","last_active":"2023-11-26T07:19:03.260489+0000","last_peered":"2023-11-26T07:19:03.260489+0000","last_clean":"2023-11-26T07:19:03.260489+0000","last_became_active":"2023-11-26T07:19:02.032002+0000","last_became_peered":"2023-11-26T07:19:02.032002+0000","last_unstale":"2023-11-26T07:19:03.260489+0000","last_undegraded":"2023-11-26T07:19:03.260489+0000","last_fullsized":"2023-11-26T07:19:03.260489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:38:54.360430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.177414+0000","last_change":"2023-11-26T07:19:01.183772+0000","last_active":"2023-11-26T07:19:02.177414+0000","last_peered":"2023-11-26T07:19:02.177414+0000","last_clean":"2023-11-26T07:19:02.177414+0000","last_became_active":"2023-11-26T07:19:01.179243+0000","last_became_peered":"2023-11-26T07:19:01.179243+0000","last_unstale":"2023-11-26T07:19:02.177414+0000","last_undegraded":"2023-11-26T07:19:02.177414+0000","last_fullsized":"2023-11-26T07:19:02.177414+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:00.444826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259839+0000","last_change":"2023-11-26T07:19:02.028811+0000","last_active":"2023-11-26T07:19:03.259839+0000","last_peered":"2023-11-26T07:19:03.259839+0000","last_clean":"2023-11-26T07:19:03.259839+0000","last_became_active":"2023-11-26T07:19:02.028467+0000","last_became_peered":"2023-11-26T07:19:02.028467+0000","last_unstale":"2023-11-26T07:19:03.259839+0000","last_undegraded":"2023-11-26T07:19:03.259839+0000","last_fullsized":"2023-11-26T07:19:03.259839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:11.829132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.021119+0000","last_change":"2023-11-26T07:19:01.175599+0000","last_active":"2023-11-26T07:19:02.021119+0000","last_peered":"2023-11-26T07:19:02.021119+0000","last_clean":"2023-11-26T07:19:02.021119+0000","last_became_active":"2023-11-26T07:19:01.175339+0000","last_became_peered":"2023-11-26T07:19:01.175339+0000","last_unstale":"2023-11-26T07:19:02.021119+0000","last_undegraded":"2023-11-26T07:19:02.021119+0000","last_fullsized":"2023-11-26T07:19:02.021119+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:06:37.620350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259377+0000","last_change":"2023-11-26T07:19:02.034056+0000","last_active":"2023-11-26T07:19:03.259377+0000","last_peered":"2023-11-26T07:19:03.259377+0000","last_clean":"2023-11-26T07:19:03.259377+0000","last_became_active":"2023-11-26T07:19:02.033786+0000","last_became_peered":"2023-11-26T07:19:02.033786+0000","last_unstale":"2023-11-26T07:19:03.259377+0000","last_undegraded":"2023-11-26T07:19:03.259377+0000","last_fullsized":"2023-11-26T07:19:03.259377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:08:55.860456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.014801+0000","last_change":"2023-11-26T07:19:01.180934+0000","last_active":"2023-11-26T07:19:02.014801+0000","last_peered":"2023-11-26T07:19:02.014801+0000","last_clean":"2023-11-26T07:19:02.014801+0000","last_became_active":"2023-11-26T07:19:01.180637+0000","last_became_peered":"2023-11-26T07:19:01.180637+0000","last_unstale":"2023-11-26T07:19:02.014801+0000","last_undegraded":"2023-11-26T07:19:02.014801+0000","last_fullsized":"2023-11-26T07:19:02.014801+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:10.826282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.015927+0000","last_change":"2023-11-26T07:19:02.027510+0000","last_active":"2023-11-26T07:19:03.015927+0000","last_peered":"2023-11-26T07:19:03.015927+0000","last_clean":"2023-11-26T07:19:03.015927+0000","last_became_active":"2023-11-26T07:19:02.027244+0000","last_became_peered":"2023-11-26T07:19:02.027244+0000","last_unstale":"2023-11-26T07:19:03.015927+0000","last_undegraded":"2023-11-26T07:19:03.015927+0000","last_fullsized":"2023-11-26T07:19:03.015927+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:51:15.180329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.032145+0000","last_change":"2023-11-26T07:19:01.185304+0000","last_active":"2023-11-26T07:19:02.032145+0000","last_peered":"2023-11-26T07:19:02.032145+0000","last_clean":"2023-11-26T07:19:02.032145+0000","last_became_active":"2023-11-26T07:19:01.185037+0000","last_became_peered":"2023-11-26T07:19:01.185037+0000","last_unstale":"2023-11-26T07:19:02.032145+0000","last_undegraded":"2023-11-26T07:19:02.032145+0000","last_fullsized":"2023-11-26T07:19:02.032145+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:14.730496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.017599+0000","last_change":"2023-11-26T07:19:02.028521+0000","last_active":"2023-11-26T07:19:03.017599+0000","last_peered":"2023-11-26T07:19:03.017599+0000","last_clean":"2023-11-26T07:19:03.017599+0000","last_became_active":"2023-11-26T07:19:02.027906+0000","last_became_peered":"2023-11-26T07:19:02.027906+0000","last_unstale":"2023-11-26T07:19:03.017599+0000","last_undegraded":"2023-11-26T07:19:03.017599+0000","last_fullsized":"2023-11-26T07:19:03.017599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:20:07.162190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.022965+0000","last_change":"2023-11-26T07:19:01.184749+0000","last_active":"2023-11-26T07:19:02.022965+0000","last_peered":"2023-11-26T07:19:02.022965+0000","last_clean":"2023-11-26T07:19:02.022965+0000","last_became_active":"2023-11-26T07:19:01.183258+0000","last_became_peered":"2023-11-26T07:19:01.183258+0000","last_unstale":"2023-11-26T07:19:02.022965+0000","last_undegraded":"2023-11-26T07:19:02.022965+0000","last_fullsized":"2023-11-26T07:19:02.022965+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:10:14.733264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016489+0000","last_change":"2023-11-26T07:19:02.025152+0000","last_active":"2023-11-26T07:19:03.016489+0000","last_peered":"2023-11-26T07:19:03.016489+0000","last_clean":"2023-11-26T07:19:03.016489+0000","last_became_active":"2023-11-26T07:19:02.024852+0000","last_became_peered":"2023-11-26T07:19:02.024852+0000","last_unstale":"2023-11-26T07:19:03.016489+0000","last_undegraded":"2023-11-26T07:19:03.016489+0000","last_fullsized":"2023-11-26T07:19:03.016489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:42:01.777951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.033472+0000","last_change":"2023-11-26T07:19:01.181491+0000","last_active":"2023-11-26T07:19:02.033472+0000","last_peered":"2023-11-26T07:19:02.033472+0000","last_clean":"2023-11-26T07:19:02.033472+0000","last_became_active":"2023-11-26T07:19:01.181171+0000","last_became_peered":"2023-11-26T07:19:01.181171+0000","last_unstale":"2023-11-26T07:19:02.033472+0000","last_undegraded":"2023-11-26T07:19:02.033472+0000","last_fullsized":"2023-11-26T07:19:02.033472+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:19:12.647848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259906+0000","last_change":"2023-11-26T07:19:02.041835+0000","last_active":"2023-11-26T07:19:03.259906+0000","last_peered":"2023-11-26T07:19:03.259906+0000","last_clean":"2023-11-26T07:19:03.259906+0000","last_became_active":"2023-11-26T07:19:02.040497+0000","last_became_peered":"2023-11-26T07:19:02.040497+0000","last_unstale":"2023-11-26T07:19:03.259906+0000","last_undegraded":"2023-11-26T07:19:03.259906+0000","last_fullsized":"2023-11-26T07:19:03.259906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:10:15.736114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.037081+0000","last_change":"2023-11-26T07:19:01.187099+0000","last_active":"2023-11-26T07:19:02.037081+0000","last_peered":"2023-11-26T07:19:02.037081+0000","last_clean":"2023-11-26T07:19:02.037081+0000","last_became_active":"2023-11-26T07:19:01.186822+0000","last_became_peered":"2023-11-26T07:19:01.186822+0000","last_unstale":"2023-11-26T07:19:02.037081+0000","last_undegraded":"2023-11-26T07:19:02.037081+0000","last_fullsized":"2023-11-26T07:19:02.037081+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:27:59.906430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260562+0000","last_change":"2023-11-26T07:19:02.028871+0000","last_active":"2023-11-26T07:19:03.260562+0000","last_peered":"2023-11-26T07:19:03.260562+0000","last_clean":"2023-11-26T07:19:03.260562+0000","last_became_active":"2023-11-26T07:19:02.028591+0000","last_became_peered":"2023-11-26T07:19:02.028591+0000","last_unstale":"2023-11-26T07:19:03.260562+0000","last_undegraded":"2023-11-26T07:19:03.260562+0000","last_fullsized":"2023-11-26T07:19:03.260562+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:44:14.539291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.032351+0000","last_change":"2023-11-26T07:19:01.179737+0000","last_active":"2023-11-26T07:19:02.032351+0000","last_peered":"2023-11-26T07:19:02.032351+0000","last_clean":"2023-11-26T07:19:02.032351+0000","last_became_active":"2023-11-26T07:19:01.179443+0000","last_became_peered":"2023-11-26T07:19:01.179443+0000","last_unstale":"2023-11-26T07:19:02.032351+0000","last_undegraded":"2023-11-26T07:19:02.032351+0000","last_fullsized":"2023-11-26T07:19:02.032351+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:52:21.149576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.260476+0000","last_change":"2023-11-26T07:19:02.037058+0000","last_active":"2023-11-26T07:19:03.260476+0000","last_peered":"2023-11-26T07:19:03.260476+0000","last_clean":"2023-11-26T07:19:03.260476+0000","last_became_active":"2023-11-26T07:19:02.036606+0000","last_became_peered":"2023-11-26T07:19:02.036606+0000","last_unstale":"2023-11-26T07:19:03.260476+0000","last_undegraded":"2023-11-26T07:19:03.260476+0000","last_fullsized":"2023-11-26T07:19:03.260476+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:28:03.358251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.018287+0000","last_change":"2023-11-26T07:19:01.191592+0000","last_active":"2023-11-26T07:19:02.018287+0000","last_peered":"2023-11-26T07:19:02.018287+0000","last_clean":"2023-11-26T07:19:02.018287+0000","last_became_active":"2023-11-26T07:19:01.191330+0000","last_became_peered":"2023-11-26T07:19:01.191330+0000","last_unstale":"2023-11-26T07:19:02.018287+0000","last_undegraded":"2023-11-26T07:19:02.018287+0000","last_fullsized":"2023-11-26T07:19:02.018287+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:29:33.570728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.259013+0000","last_change":"2023-11-26T07:19:02.029954+0000","last_active":"2023-11-26T07:19:03.259013+0000","last_peered":"2023-11-26T07:19:03.259013+0000","last_clean":"2023-11-26T07:19:03.259013+0000","last_became_active":"2023-11-26T07:19:02.029629+0000","last_became_peered":"2023-11-26T07:19:02.029629+0000","last_unstale":"2023-11-26T07:19:03.259013+0000","last_undegraded":"2023-11-26T07:19:03.259013+0000","last_fullsized":"2023-11-26T07:19:03.259013+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:27.196740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.035156+0000","last_change":"2023-11-26T07:19:01.187187+0000","last_active":"2023-11-26T07:19:02.035156+0000","last_peered":"2023-11-26T07:19:02.035156+0000","last_clean":"2023-11-26T07:19:02.035156+0000","last_became_active":"2023-11-26T07:19:01.186790+0000","last_became_peered":"2023-11-26T07:19:01.186790+0000","last_unstale":"2023-11-26T07:19:02.035156+0000","last_undegraded":"2023-11-26T07:19:02.035156+0000","last_fullsized":"2023-11-26T07:19:02.035156+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:04:56.694454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016154+0000","last_change":"2023-11-26T07:19:02.026767+0000","last_active":"2023-11-26T07:19:03.016154+0000","last_peered":"2023-11-26T07:19:03.016154+0000","last_clean":"2023-11-26T07:19:03.016154+0000","last_became_active":"2023-11-26T07:19:02.026459+0000","last_became_peered":"2023-11-26T07:19:02.026459+0000","last_unstale":"2023-11-26T07:19:03.016154+0000","last_undegraded":"2023-11-26T07:19:03.016154+0000","last_fullsized":"2023-11-26T07:19:03.016154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:20:07.162190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.014574+0000","last_change":"2023-11-26T07:19:01.184034+0000","last_active":"2023-11-26T07:19:02.014574+0000","last_peered":"2023-11-26T07:19:02.014574+0000","last_clean":"2023-11-26T07:19:02.014574+0000","last_became_active":"2023-11-26T07:19:01.183748+0000","last_became_peered":"2023-11-26T07:19:01.183748+0000","last_unstale":"2023-11-26T07:19:02.014574+0000","last_undegraded":"2023-11-26T07:19:02.014574+0000","last_fullsized":"2023-11-26T07:19:02.014574+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:04:56.694454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-26T07:19:03.016778+0000","last_change":"2023-11-26T07:19:02.035655+0000","last_active":"2023-11-26T07:19:03.016778+0000","last_peered":"2023-11-26T07:19:03.016778+0000","last_clean":"2023-11-26T07:19:03.016778+0000","last_became_active":"2023-11-26T07:19:02.035174+0000","last_became_peered":"2023-11-26T07:19:02.035174+0000","last_unstale":"2023-11-26T07:19:03.016778+0000","last_undegraded":"2023-11-26T07:19:03.016778+0000","last_fullsized":"2023-11-26T07:19:03.016778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:01.151385+0000","last_clean_scrub_stamp":"2023-11-26T07:19:01.151385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:02:04.873548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.175225+0000","last_change":"2023-11-26T07:19:01.169180+0000","last_active":"2023-11-26T07:19:02.175225+0000","last_peered":"2023-11-26T07:19:02.175225+0000","last_clean":"2023-11-26T07:19:02.175225+0000","last_became_active":"2023-11-26T07:19:01.168919+0000","last_became_peered":"2023-11-26T07:19:01.168919+0000","last_unstale":"2023-11-26T07:19:02.175225+0000","last_undegraded":"2023-11-26T07:19:02.175225+0000","last_fullsized":"2023-11-26T07:19:02.175225+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:19:00.148535+0000","last_clean_scrub_stamp":"2023-11-26T07:19:00.148535+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:14.730496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":183,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-26T07:19:02.031632+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:19:02.031632+0000","last_peered":"2023-11-26T07:19:02.031632+0000","last_clean":"2023-11-26T07:19:02.031632+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:19:02.031632+0000","last_undegraded":"2023-11-26T07:19:02.031632+0000","last_fullsized":"2023-11-26T07:19:02.031632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1114112,"data_stored":800112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.73799999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.60399999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76200000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.435},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.39300000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.64900000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.627}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":210326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":0.441},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.52000000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.751},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.72699999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.9340000000000002,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.70399999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.626},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.80700000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.74299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":209724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53200000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.50700000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.46200000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.49299999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.44400000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.59399999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.264,"5min":1.264,"15min":1.264},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.68300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":211266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.59399999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.70199999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.752}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.68300000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.5},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.46999999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.60899999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":528384,"data_stored":210292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55700000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.375},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.51100000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.58899999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.621}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":532480,"data_stored":211288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.37},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":3.9929999999999999,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.495},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.40600000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":8.0990000000000002,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":4.431,"5min":4.431,"15min":4.431},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.82799999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.68799999999999994}]},{"osd":6,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.753}]},{"osd":7,"last update":"Sun Nov 26 07:18:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.88},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.71199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":209724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.253},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39400000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.51900000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.441},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.47899999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.56000000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.624}]},{"osd":7,"last update":"Sun Nov 26 07:18:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1110016,"data_stored":799037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.45000000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.61699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55200000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.89500000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79100000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.875},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.77000000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:18:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.84899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:19:06.671 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:19:06.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:19:06.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f736d4c9640 1 Processor -- start 2023-11-26T07:19:06.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f736d4c9640 1 -- start start 2023-11-26T07:19:06.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f736d4c9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f736806e120 0x7f736815df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f736d4c9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f736815e4d0 0x7f73680ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f736d4c9640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73680cd870 con 0x7f7368154940 2023-11-26T07:19:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f736d4c9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7368154d20 con 0x7f736806e120 2023-11-26T07:19:06.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f736d4c9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736815ecc0 con 0x7f736815e4d0 2023-11-26T07:19:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f735ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f736806e120 0x7f736815df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f73677fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f736815e4d0 0x7f73680ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f735ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f736806e120 0x7f736815df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51252/0 (socket says 172.21.15.31:51252) 2023-11-26T07:19:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f73677fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f736815e4d0 0x7f73680ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:36386/0 (socket says 172.21.15.31:36386) 2023-11-26T07:19:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f735ffff640 1 -- 172.21.15.31:0/3908006382 learned_addr learned my addr 172.21.15.31:0/3908006382 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f7366ffd640 1 --1- 172.21.15.31:0/3908006382 >> v1:172.21.15.97:6789/0 conn(0x7f7368154940 0x7f736806da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:47222/0 (socket says 172.21.15.31:47222) 2023-11-26T07:19:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f735ffff640 1 -- 172.21.15.31:0/3908006382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f736815e4d0 msgr2=0x7f73680ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f735ffff640 1 --2- 172.21.15.31:0/3908006382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f736815e4d0 0x7f73680ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f735ffff640 1 -- 172.21.15.31:0/3908006382 >> v1:172.21.15.97:6789/0 conn(0x7f7368154940 legacy=0x7f736806da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f735ffff640 1 -- 172.21.15.31:0/3908006382 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73680ccf40 con 0x7f736806e120 2023-11-26T07:19:06.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.851+0000 7f73677fe640 1 --2- 172.21.15.31:0/3908006382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f736815e4d0 0x7f73680ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:06.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f735ffff640 1 --2- 172.21.15.31:0/3908006382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f736806e120 0x7f736815df90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7350009870 tx=0x7f735001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a5c66c5874d7457 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f73667fc640 1 -- 172.21.15.31:0/3908006382 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73500ab600 con 0x7f736806e120 2023-11-26T07:19:06.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f73667fc640 1 -- 172.21.15.31:0/3908006382 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73500a7050 con 0x7f736806e120 2023-11-26T07:19:06.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f73667fc640 1 -- 172.21.15.31:0/3908006382 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73500ab980 con 0x7f736806e120 2023-11-26T07:19:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f736d4c9640 1 -- 172.21.15.31:0/3908006382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f736806e120 msgr2=0x7f736815df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f736d4c9640 1 --2- 172.21.15.31:0/3908006382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f736806e120 0x7f736815df90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7350009870 tx=0x7f735001c330 comp rx=0 tx=0).stop 2023-11-26T07:19:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f736d4c9640 1 -- 172.21.15.31:0/3908006382 shutdown_connections 2023-11-26T07:19:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f736d4c9640 1 --2- 172.21.15.31:0/3908006382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f736815e4d0 0x7f73680ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f736d4c9640 1 --2- 172.21.15.31:0/3908006382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f736806e120 0x7f736815df90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f736d4c9640 1 -- 172.21.15.31:0/3908006382 >> 172.21.15.31:0/3908006382 conn(0x7f73680b0cb0 msgr2=0x7f73680b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f736d4c9640 1 -- 172.21.15.31:0/3908006382 shutdown_connections 2023-11-26T07:19:06.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.855+0000 7f736d4c9640 1 -- 172.21.15.31:0/3908006382 wait complete. 2023-11-26T07:19:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 Processor -- start 2023-11-26T07:19:06.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 -- start start 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806e120 0x7f73680717f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7368154940 0x7f7368071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f736815e4d0 0x7f736814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7368072270 con 0x7f736806e120 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736809b160 con 0x7f736815e4d0 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736809b440 con 0x7f7368154940 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f7366ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806e120 0x7f73680717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f7366ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806e120 0x7f73680717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51626/0 (socket says 172.21.15.31:51626) 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f7366ffd640 1 -- 172.21.15.31:0/2409731873 learned_addr learned my addr 172.21.15.31:0/2409731873 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f735ffff640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7368154940 0x7f7368071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f73677fe640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f736815e4d0 0x7f736814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f7366ffd640 1 -- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7368154940 msgr2=0x7f7368071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f7366ffd640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7368154940 0x7f7368071d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f7366ffd640 1 -- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f736815e4d0 msgr2=0x7f736814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f7366ffd640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f736815e4d0 0x7f736814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:06.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f7366ffd640 1 -- 172.21.15.31:0/2409731873 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360009a70 con 0x7f736806e120 2023-11-26T07:19:06.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f7366ffd640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806e120 0x7f73680717f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f735800c490 tx=0x7f735800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f735f7fe640 1 -- 172.21.15.31:0/2409731873 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73580074e0 con 0x7f736806e120 2023-11-26T07:19:06.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f735f7fe640 1 -- 172.21.15.31:0/2409731873 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358007640 con 0x7f736806e120 2023-11-26T07:19:06.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f735f7fe640 1 -- 172.21.15.31:0/2409731873 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358004a10 con 0x7f736806e120 2023-11-26T07:19:06.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 -- 172.21.15.31:0/2409731873 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73500094d0 con 0x7f736806e120 2023-11-26T07:19:06.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 -- 172.21.15.31:0/2409731873 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736809ba00 con 0x7f736806e120 2023-11-26T07:19:06.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.859+0000 7f736d4c9640 1 -- 172.21.15.31:0/2409731873 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73680671f0 con 0x7f736806e120 2023-11-26T07:19:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.863+0000 7f735f7fe640 1 -- 172.21.15.31:0/2409731873 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f7358004b70 con 0x7f736806e120 2023-11-26T07:19:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.863+0000 7f735f7fe640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7348066620 0x7f7348068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.863+0000 7f735f7fe640 1 -- 172.21.15.31:0/2409731873 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7358083af0 con 0x7f736806e120 2023-11-26T07:19:06.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.863+0000 7f735ffff640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7348066620 0x7f7348068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:06.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.863+0000 7f735ffff640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7348066620 0x7f7348068ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f73500ad000 tx=0x7f7350008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:06.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:06.867+0000 7f735f7fe640 1 -- 172.21.15.31:0/2409731873 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7358050ce0 con 0x7f736806e120 2023-11-26T07:19:07.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.079+0000 7f736d4c9640 1 -- 172.21.15.31:0/2409731873 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f736806b460 con 0x7f736806e120 2023-11-26T07:19:07.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f735f7fe640 1 -- 172.21.15.31:0/2409731873 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7358054300 con 0x7f736806e120 2023-11-26T07:19:07.089 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-11-26T07:19:07.089 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA== 2023-11-26T07:19:07.089 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2023-11-26T07:19:07.089 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2023-11-26T07:19:07.089 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:19:07.090 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-11-26T07:19:07.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 -- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7348066620 msgr2=0x7f7348068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7348066620 0x7f7348068ae0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f73500ad000 tx=0x7f7350008040 comp rx=0 tx=0).stop 2023-11-26T07:19:07.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 -- 172.21.15.31:0/2409731873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806e120 msgr2=0x7f73680717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806e120 0x7f73680717f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f735800c490 tx=0x7f735800c890 comp rx=0 tx=0).stop 2023-11-26T07:19:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 -- 172.21.15.31:0/2409731873 shutdown_connections 2023-11-26T07:19:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7348066620 0x7f7348068ae0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f736815e4d0 0x7f736814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7368154940 0x7f7368071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 --2- 172.21.15.31:0/2409731873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f736806e120 0x7f73680717f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 -- 172.21.15.31:0/2409731873 >> 172.21.15.31:0/2409731873 conn(0x7f73680b0cb0 msgr2=0x7f73680aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 -- 172.21.15.31:0/2409731873 shutdown_connections 2023-11-26T07:19:07.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.083+0000 7f736d4c9640 1 -- 172.21.15.31:0/2409731873 wait complete. 2023-11-26T07:19:07.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:07.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.247+0000 7f792d38d640 1 Processor -- start 2023-11-26T07:19:07.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.247+0000 7f792d38d640 1 -- start start 2023-11-26T07:19:07.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.247+0000 7f792d38d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7928143ae0 0x7f79280b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.247+0000 7f792d38d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f792806dfd0 0x7f792806e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.247+0000 7f792d38d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79280b2480 con 0x7f79280adcc0 2023-11-26T07:19:07.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.247+0000 7f792d38d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79280ae820 con 0x7f792806dfd0 2023-11-26T07:19:07.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.247+0000 7f792d38d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792806e950 con 0x7f7928143ae0 2023-11-26T07:19:07.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f7926ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7928143ae0 0x7f79280b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f79277fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f79280adcc0 0x7f79280ae110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52470/0 (socket says 172.21.15.31:52470) 2023-11-26T07:19:07.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f79277fe640 1 -- 172.21.15.31:0/3661638005 learned_addr learned my addr 172.21.15.31:0/3661638005 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:07.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f791ffff640 1 --2- 172.21.15.31:0/3661638005 >> v2:172.21.15.31:3300/0 conn(0x7f792806dfd0 0x7f792806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f79267fc640 1 -- 172.21.15.31:0/3661638005 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648556345 0 0) 0x7f79280b2480 con 0x7f79280adcc0 2023-11-26T07:19:07.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f79267fc640 1 -- 172.21.15.31:0/3661638005 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7914005740 con 0x7f79280adcc0 2023-11-26T07:19:07.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f791ffff640 1 -- 172.21.15.31:0/3661638005 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7928143ae0 msgr2=0x7f79280b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:07.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f791ffff640 1 --2- 172.21.15.31:0/3661638005 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7928143ae0 0x7f79280b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f791ffff640 1 -- 172.21.15.31:0/3661638005 >> v1:172.21.15.31:6789/0 conn(0x7f79280adcc0 legacy=0x7f79280ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f791ffff640 1 -- 172.21.15.31:0/3661638005 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7928070670 con 0x7f792806dfd0 2023-11-26T07:19:07.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f791ffff640 1 --2- 172.21.15.31:0/3661638005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 0x7f792806e410 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f79100097c0 tx=0x7f791001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c6574348004345 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:07.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f79267fc640 1 -- 172.21.15.31:0/3661638005 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79100ab650 con 0x7f792806dfd0 2023-11-26T07:19:07.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f79267fc640 1 -- 172.21.15.31:0/3661638005 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79100ab7b0 con 0x7f792806dfd0 2023-11-26T07:19:07.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.251+0000 7f79267fc640 1 -- 172.21.15.31:0/3661638005 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79100abb40 con 0x7f792806dfd0 2023-11-26T07:19:07.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 -- 172.21.15.31:0/3661638005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 msgr2=0x7f792806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 --2- 172.21.15.31:0/3661638005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 0x7f792806e410 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f79100097c0 tx=0x7f791001c450 comp rx=0 tx=0).stop 2023-11-26T07:19:07.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 -- 172.21.15.31:0/3661638005 shutdown_connections 2023-11-26T07:19:07.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 --2- 172.21.15.31:0/3661638005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 0x7f792806e410 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 --2- 172.21.15.31:0/3661638005 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7928143ae0 0x7f79280b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 -- 172.21.15.31:0/3661638005 >> 172.21.15.31:0/3661638005 conn(0x7f79280b0c40 msgr2=0x7f79280b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 -- 172.21.15.31:0/3661638005 shutdown_connections 2023-11-26T07:19:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 -- 172.21.15.31:0/3661638005 wait complete. 2023-11-26T07:19:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 Processor -- start 2023-11-26T07:19:07.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 -- start start 2023-11-26T07:19:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 0x7f7928139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79280adcc0 0x7f79280b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7928143ae0 0x7f79281f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79281f7500 con 0x7f792806dfd0 2023-11-26T07:19:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7928138d70 con 0x7f7928143ae0 2023-11-26T07:19:07.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f792d38d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7928139050 con 0x7f79280adcc0 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f7926ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 0x7f7928139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f7926ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 0x7f7928139670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51634/0 (socket says 172.21.15.31:51634) 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f7926ffd640 1 -- 172.21.15.31:0/3149965651 learned_addr learned my addr 172.21.15.31:0/3149965651 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f791ffff640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79280adcc0 0x7f79280b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f79277fe640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7928143ae0 0x7f79281f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f7926ffd640 1 -- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79280adcc0 msgr2=0x7f79280b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f7926ffd640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79280adcc0 0x7f79280b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f7926ffd640 1 -- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7928143ae0 msgr2=0x7f79281f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f7926ffd640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7928143ae0 0x7f79281f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f7926ffd640 1 -- 172.21.15.31:0/3149965651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7910009420 con 0x7f792806dfd0 2023-11-26T07:19:07.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.255+0000 7f79277fe640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7928143ae0 0x7f79281f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:07.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f7926ffd640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 0x7f7928139670 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f791800e870 tx=0x7f791800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:07.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f791f7fe640 1 -- 172.21.15.31:0/3149965651 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918009510 con 0x7f792806dfd0 2023-11-26T07:19:07.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f791f7fe640 1 -- 172.21.15.31:0/3149965651 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791800eec0 con 0x7f792806dfd0 2023-11-26T07:19:07.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f791f7fe640 1 -- 172.21.15.31:0/3149965651 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791800cb40 con 0x7f792806dfd0 2023-11-26T07:19:07.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f792d38d640 1 -- 172.21.15.31:0/3149965651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7928139330 con 0x7f792806dfd0 2023-11-26T07:19:07.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f792d38d640 1 -- 172.21.15.31:0/3149965651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7928139bb0 con 0x7f792806dfd0 2023-11-26T07:19:07.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f792d38d640 1 -- 172.21.15.31:0/3149965651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792804e850 con 0x7f792806dfd0 2023-11-26T07:19:07.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f791f7fe640 1 -- 172.21.15.31:0/3149965651 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f791800cca0 con 0x7f792806dfd0 2023-11-26T07:19:07.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f791f7fe640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7908066620 0x7f7908068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:07.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f791f7fe640 1 -- 172.21.15.31:0/3149965651 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79180842c0 con 0x7f792806dfd0 2023-11-26T07:19:07.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.259+0000 7f791ffff640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7908066620 0x7f7908068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:07.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.263+0000 7f791ffff640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7908066620 0x7f7908068ae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f79100af000 tx=0x7f79100a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:07.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.263+0000 7f791f7fe640 1 -- 172.21.15.31:0/3149965651 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79180514d0 con 0x7f792806dfd0 2023-11-26T07:19:07.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.455+0000 7f792d38d640 1 -- 172.21.15.31:0/3149965651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f792806cc40 con 0x7f792806dfd0 2023-11-26T07:19:07.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.455+0000 7f791f7fe640 1 -- 172.21.15.31:0/3149965651 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f7918054af0 con 0x7f792806dfd0 2023-11-26T07:19:07.463 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:07.463 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:07.463 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2023-11-26T07:19:07.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 -- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7908066620 msgr2=0x7f7908068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7908066620 0x7f7908068ae0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f79100af000 tx=0x7f79100a7000 comp rx=0 tx=0).stop 2023-11-26T07:19:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 -- 172.21.15.31:0/3149965651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 msgr2=0x7f7928139670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 0x7f7928139670 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f791800e870 tx=0x7f791800ec70 comp rx=0 tx=0).stop 2023-11-26T07:19:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 -- 172.21.15.31:0/3149965651 shutdown_connections 2023-11-26T07:19:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7908066620 0x7f7908068ae0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7928143ae0 0x7f79281f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79280adcc0 0x7f79280b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 --2- 172.21.15.31:0/3149965651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f792806dfd0 0x7f7928139670 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 -- 172.21.15.31:0/3149965651 >> 172.21.15.31:0/3149965651 conn(0x7f79280b0c40 msgr2=0x7f79280b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:07.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 -- 172.21.15.31:0/3149965651 shutdown_connections 2023-11-26T07:19:07.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:07.459+0000 7f792d38d640 1 -- 172.21.15.31:0/3149965651 wait complete. 2023-11-26T07:19:07.887 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:19:01.563598+0000', 'modified': '2023-11-26T07:19:04.566715+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5622}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5622': {'gid': 5622, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/1115502954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6834', 'nonce': 1115502954}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 1115502954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:19:07.887 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:19:07.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-26T07:19:08.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.039+0000 7f8355f55640 1 Processor -- start 2023-11-26T07:19:08.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.039+0000 7f8355f55640 1 -- start start 2023-11-26T07:19:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f8355f55640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f835006e3f0 0x7f835015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f8355f55640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8350159630 con 0x7f83500cb9b0 2023-11-26T07:19:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f8355f55640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835015f6d0 con 0x7f83500ce5d0 2023-11-26T07:19:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f8355f55640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835006e860 con 0x7f835006e3f0 2023-11-26T07:19:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f834effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f835006e3f0 0x7f835015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f834f7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f83500cb9b0 0x7f83500cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52484/0 (socket says 172.21.15.31:52484) 2023-11-26T07:19:08.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f834f7fe640 1 -- 172.21.15.31:0/703847546 learned_addr learned my addr 172.21.15.31:0/703847546 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:08.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f834effd640 1 -- 172.21.15.31:0/703847546 >> v1:172.21.15.97:6789/0 conn(0x7f83500ce5d0 legacy=0x7f835015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f834effd640 1 -- 172.21.15.31:0/703847546 >> v1:172.21.15.31:6789/0 conn(0x7f83500cb9b0 legacy=0x7f83500cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f834effd640 1 -- 172.21.15.31:0/703847546 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83500cd1d0 con 0x7f835006e3f0 2023-11-26T07:19:08.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.043+0000 7f834e7fc640 1 -- 172.21.15.31:0/703847546 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481338014 0 0) 0x7f8350159630 con 0x7f83500cb9b0 2023-11-26T07:19:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f834effd640 1 --2- 172.21.15.31:0/703847546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006e3f0 0x7f835015f190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f83440097c0 tx=0x7f834401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb929a8e1d7b1ad2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f834e7fc640 1 -- 172.21.15.31:0/703847546 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83440ab600 con 0x7f835006e3f0 2023-11-26T07:19:08.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f834e7fc640 1 -- 172.21.15.31:0/703847546 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83440a7050 con 0x7f835006e3f0 2023-11-26T07:19:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f834e7fc640 1 -- 172.21.15.31:0/703847546 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83440ab980 con 0x7f835006e3f0 2023-11-26T07:19:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f8355f55640 1 -- 172.21.15.31:0/703847546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006e3f0 msgr2=0x7f835015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f8355f55640 1 --2- 172.21.15.31:0/703847546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006e3f0 0x7f835015f190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f83440097c0 tx=0x7f834401c480 comp rx=0 tx=0).stop 2023-11-26T07:19:08.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f8355f55640 1 -- 172.21.15.31:0/703847546 shutdown_connections 2023-11-26T07:19:08.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f8355f55640 1 --2- 172.21.15.31:0/703847546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006e3f0 0x7f835015f190 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f8355f55640 1 -- 172.21.15.31:0/703847546 >> 172.21.15.31:0/703847546 conn(0x7f83500b0cb0 msgr2=0x7f83500b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:08.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f8355f55640 1 -- 172.21.15.31:0/703847546 shutdown_connections 2023-11-26T07:19:08.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f8355f55640 1 -- 172.21.15.31:0/703847546 wait complete. 2023-11-26T07:19:08.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f8355f55640 1 Processor -- start 2023-11-26T07:19:08.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.047+0000 7f8355f55640 1 -- start start 2023-11-26T07:19:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8355f55640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835006e3f0 0x7f8350153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8355f55640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83500cb9b0 0x7f83501577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8355f55640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f83500ce5d0 0x7f835015a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8355f55640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83501548a0 con 0x7f83500cb9b0 2023-11-26T07:19:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8355f55640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8350159730 con 0x7f835006e3f0 2023-11-26T07:19:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8355f55640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8350159a10 con 0x7f83500ce5d0 2023-11-26T07:19:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83500cb9b0 0x7f83501577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83500cb9b0 0x7f83501577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51642/0 (socket says 172.21.15.31:51642) 2023-11-26T07:19:08.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834effd640 1 -- 172.21.15.31:0/2658330369 learned_addr learned my addr 172.21.15.31:0/2658330369 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:08.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834f7fe640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835006e3f0 0x7f8350153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834ffff640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f83500ce5d0 0x7f835015a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834effd640 1 -- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f83500ce5d0 msgr2=0x7f835015a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834effd640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f83500ce5d0 0x7f835015a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834effd640 1 -- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835006e3f0 msgr2=0x7f8350153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834effd640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835006e3f0 0x7f8350153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834effd640 1 -- 172.21.15.31:0/2658330369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8344009420 con 0x7f83500cb9b0 2023-11-26T07:19:08.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f834effd640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83500cb9b0 0x7f83501577e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f83440ae040 tx=0x7f834401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8337fff640 1 -- 172.21.15.31:0/2658330369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83440b7650 con 0x7f83500cb9b0 2023-11-26T07:19:08.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8337fff640 1 -- 172.21.15.31:0/2658330369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83440b77b0 con 0x7f83500cb9b0 2023-11-26T07:19:08.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8337fff640 1 -- 172.21.15.31:0/2658330369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83440b7b60 con 0x7f83500cb9b0 2023-11-26T07:19:08.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8355f55640 1 -- 172.21.15.31:0/2658330369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8350159cf0 con 0x7f83500cb9b0 2023-11-26T07:19:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.051+0000 7f8355f55640 1 -- 172.21.15.31:0/2658330369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835015a2b0 con 0x7f83500cb9b0 2023-11-26T07:19:08.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f8337fff640 1 -- 172.21.15.31:0/2658330369 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f83440b7cc0 con 0x7f83500cb9b0 2023-11-26T07:19:08.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f8337fff640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f833c066620 0x7f833c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f8337fff640 1 -- 172.21.15.31:0/2658330369 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f834411eb50 con 0x7f83500cb9b0 2023-11-26T07:19:08.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f83592b7000 1 Processor -- start 2023-11-26T07:19:08.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f83592b7000 1 -- start start 2023-11-26T07:19:08.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834f7fe640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f833c066620 0x7f833c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834f7fe640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f833c066620 0x7f833c068ae0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8340000f70 tx=0x7f8340007240 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f83592b7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64aa5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f83592b7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64aa55b0 0x559c64a5ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f83592b7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64a5b320 0x559c64a5b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f83592b7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c648c5130 con 0x559c64a5b320 2023-11-26T07:19:08.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f83592b7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c648b29a0 con 0x559c64aa55b0 2023-11-26T07:19:08.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f83592b7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c648afed0 con 0x559c64aa4c90 2023-11-26T07:19:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64a5b320 0x559c64a5b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64a5b320 0x559c64a5b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51646/0 (socket says 172.21.15.31:51646) 2023-11-26T07:19:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834ffff640 1 -- 172.21.15.31:0/89604432 learned_addr learned my addr 172.21.15.31:0/89604432 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834effd640 1 --2- 172.21.15.31:0/89604432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64aa55b0 0x559c64a5ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834f7fe640 1 --2- 172.21.15.31:0/89604432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64aa5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834ffff640 1 -- 172.21.15.31:0/89604432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 msgr2=0x559c64aa5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834ffff640 1 --2- 172.21.15.31:0/89604432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64aa5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834ffff640 1 -- 172.21.15.31:0/89604432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64aa55b0 msgr2=0x559c64a5ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834ffff640 1 --2- 172.21.15.31:0/89604432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64aa55b0 0x559c64a5ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834ffff640 1 -- 172.21.15.31:0/89604432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c645a3890 con 0x559c64a5b320 2023-11-26T07:19:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.055+0000 7f834ffff640 1 --2- 172.21.15.31:0/89604432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64a5b320 0x559c64a5b700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8348001d20 tx=0x7f8348001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42016227c9cd36d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83357fa640 1 -- 172.21.15.31:0/89604432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348003b90 con 0x559c64a5b320 2023-11-26T07:19:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83357fa640 1 -- 172.21.15.31:0/89604432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8348003cf0 con 0x559c64a5b320 2023-11-26T07:19:08.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83357fa640 1 -- 172.21.15.31:0/89604432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800fb30 con 0x559c64a5b320 2023-11-26T07:19:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 -- 172.21.15.31:0/89604432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64a5b320 msgr2=0x559c64a5b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 --2- 172.21.15.31:0/89604432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64a5b320 0x559c64a5b700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8348001d20 tx=0x7f8348001610 comp rx=0 tx=0).stop 2023-11-26T07:19:08.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 -- 172.21.15.31:0/89604432 shutdown_connections 2023-11-26T07:19:08.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 --2- 172.21.15.31:0/89604432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64a5b320 0x559c64a5b700 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 --2- 172.21.15.31:0/89604432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64aa55b0 0x559c64a5ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 --2- 172.21.15.31:0/89604432 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64aa5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 -- 172.21.15.31:0/89604432 >> 172.21.15.31:0/89604432 conn(0x559c64a572d0 msgr2=0x559c64a596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:08.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 -- 172.21.15.31:0/89604432 shutdown_connections 2023-11-26T07:19:08.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 -- 172.21.15.31:0/89604432 wait complete. 2023-11-26T07:19:08.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 -- start start 2023-11-26T07:19:08.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 Processor -- start 2023-11-26T07:19:08.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64a5fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64a602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64a73910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c648ab7c0 con 0x559c64aa55b0 2023-11-26T07:19:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c64863bf0 con 0x559c64a5b320 2023-11-26T07:19:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c64869a90 con 0x559c64aa4c90 2023-11-26T07:19:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f83592b7000 10 client.0 _ll_register_callbacks cb 0x559c64a463c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:19:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f834ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64a73910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f834ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64a73910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51656/0 (socket says 172.21.15.31:51656) 2023-11-26T07:19:08.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f834ffff640 1 -- 172.21.15.31:0/4103873564 learned_addr learned my addr 172.21.15.31:0/4103873564 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f834f7fe640 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64a5fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f834effd640 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64a602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f834ffff640 1 -- 172.21.15.31:0/4103873564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 msgr2=0x559c64a602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f834ffff640 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64a602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f834ffff640 1 -- 172.21.15.31:0/4103873564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 msgr2=0x559c64a5fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f834ffff640 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64a5fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.059+0000 7f834ffff640 1 -- 172.21.15.31:0/4103873564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835015f6d0 con 0x559c64aa55b0 2023-11-26T07:19:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f834ffff640 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64a73910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8348001650 tx=0x7f8348001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9255b308a7657b9a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f832e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:19:08.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f832e7fc640 1 -- 172.21.15.31:0/4103873564 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480050d0 con 0x559c64aa55b0 2023-11-26T07:19:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f832e7fc640 1 -- 172.21.15.31:0/4103873564 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8348027ae0 con 0x559c64aa55b0 2023-11-26T07:19:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 -- 172.21.15.31:0/4103873564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c6486d470 con 0x559c64aa55b0 2023-11-26T07:19:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f832e7fc640 1 -- 172.21.15.31:0/4103873564 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8348027ca0 con 0x559c64aa55b0 2023-11-26T07:19:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 10 client.5751 fetch_fsmap learned FSMap version 27 2023-11-26T07:19:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 -- 172.21.15.31:0/4103873564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x559c6486c190 con 0x559c64aa55b0 2023-11-26T07:19:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f832e7fc640 1 -- 172.21.15.31:0/4103873564 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c6486d470 con 0x559c64aa55b0 2023-11-26T07:19:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f832e7fc640 1 -- 172.21.15.31:0/4103873564 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8348012070 con 0x559c64aa55b0 2023-11-26T07:19:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 10 client.5751 fetch_fsmap finished waiting for FSMap version 27 2023-11-26T07:19:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 10 client.5751 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:19:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 20 client.5751 populate_metadata read hostname 'smithi031' 2023-11-26T07:19:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.31:6834/1115502954,v1:172.21.15.31:6835/1115502954] conn(0x559c64a74320 0x559c64ab6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 4 client.5751 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:19:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 -- 172.21.15.31:0/4103873564 --> [v2:172.21.15.31:6834/1115502954,v1:172.21.15.31:6835/1115502954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c64870b50 con 0x559c64a74320 2023-11-26T07:19:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f834f7fe640 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.31:6834/1115502954,v1:172.21.15.31:6835/1115502954] conn(0x559c64a74320 0x559c64ab6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f834f7fe640 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.31:6834/1115502954,v1:172.21.15.31:6835/1115502954] conn(0x559c64a74320 0x559c64ab6f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb57c1340f4eabf server_cookie=c00db45a07438581 in_seq=0 out_seq=0 2023-11-26T07:19:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f832e7fc640 10 client.5751 ms_handle_connect on v2:172.21.15.31:6834/1115502954 2023-11-26T07:19:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f832e7fc640 1 -- 172.21.15.31:0/4103873564 <== mds.0 v2:172.21.15.31:6834/1115502954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c64870b50 con 0x559c64a74320 2023-11-26T07:19:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f832e7fc640 10 client.5751 handle_command_reply: tid=0 2023-11-26T07:19:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 client.5751 shutdown 2023-11-26T07:19:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 -- 172.21.15.31:0/4103873564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 msgr2=0x559c64a73910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64a73910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8348001650 tx=0x7f8348001260 comp rx=0 tx=0).stop 2023-11-26T07:19:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 -- 172.21.15.31:0/4103873564 shutdown_connections 2023-11-26T07:19:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.31:6834/1115502954,v1:172.21.15.31:6835/1115502954] conn(0x559c64a74320 0x559c64ab6f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64a73910 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64a602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 --2- 172.21.15.31:0/4103873564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64a5fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f832e7fc640 0 client.5751 ms_handle_reset on v2:172.21.15.31:6834/1115502954 2023-11-26T07:19:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 -- 172.21.15.31:0/4103873564 >> 172.21.15.31:0/4103873564 conn(0x559c64a572d0 msgr2=0x559c64a596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:08.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 -- 172.21.15.31:0/4103873564 shutdown_connections 2023-11-26T07:19:08.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 1 -- 172.21.15.31:0/4103873564 wait complete. 2023-11-26T07:19:08.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.063+0000 7f83592b7000 20 client.5751 trim_cache size 0 max 16384 2023-11-26T07:19:08.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.079+0000 7f83592b7000 1 Processor -- start 2023-11-26T07:19:08.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.079+0000 7f83592b7000 1 -- start start 2023-11-26T07:19:08.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.079+0000 7f83592b7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64aa2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.079+0000 7f83592b7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64aa3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.079+0000 7f83592b7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64aa35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.079+0000 7f83592b7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c64564940 con 0x559c64aa55b0 2023-11-26T07:19:08.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.079+0000 7f83592b7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c64a56c70 con 0x559c64a5b320 2023-11-26T07:19:08.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.079+0000 7f83592b7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c64864d60 con 0x559c64aa4c90 2023-11-26T07:19:08.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64aa35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64aa35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51666/0 (socket says 172.21.15.31:51666) 2023-11-26T07:19:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834ffff640 1 -- 172.21.15.31:0/3331773696 learned_addr learned my addr 172.21.15.31:0/3331773696 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834effd640 1 --2- 172.21.15.31:0/3331773696 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64aa3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834f7fe640 1 --2- 172.21.15.31:0/3331773696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64aa2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834ffff640 1 -- 172.21.15.31:0/3331773696 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 msgr2=0x559c64aa3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834ffff640 1 --2- 172.21.15.31:0/3331773696 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64aa3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834ffff640 1 -- 172.21.15.31:0/3331773696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 msgr2=0x559c64aa2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834ffff640 1 --2- 172.21.15.31:0/3331773696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64aa2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834ffff640 1 -- 172.21.15.31:0/3331773696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c64852490 con 0x559c64aa55b0 2023-11-26T07:19:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834ffff640 1 --2- 172.21.15.31:0/3331773696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64aa35b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8348001c40 tx=0x7f834801f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbeaeaa608177a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f8335ffb640 1 -- 172.21.15.31:0/3331773696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348010d50 con 0x559c64aa55b0 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f8335ffb640 1 -- 172.21.15.31:0/3331773696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8348021070 con 0x559c64aa55b0 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f8335ffb640 1 -- 172.21.15.31:0/3331773696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348006d60 con 0x559c64aa55b0 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 -- 172.21.15.31:0/3331773696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 msgr2=0x559c64aa35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 --2- 172.21.15.31:0/3331773696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64aa35b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8348001c40 tx=0x7f834801f070 comp rx=0 tx=0).stop 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 -- 172.21.15.31:0/3331773696 shutdown_connections 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 --2- 172.21.15.31:0/3331773696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa55b0 0x559c64aa35b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 --2- 172.21.15.31:0/3331773696 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa4c90 0x559c64aa3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 --2- 172.21.15.31:0/3331773696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64aa2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 -- 172.21.15.31:0/3331773696 >> 172.21.15.31:0/3331773696 conn(0x559c64a572d0 msgr2=0x559c64ab2280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 -- 172.21.15.31:0/3331773696 shutdown_connections 2023-11-26T07:19:08.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 -- 172.21.15.31:0/3331773696 wait complete. 2023-11-26T07:19:08.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 -- start start 2023-11-26T07:19:08.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 Processor -- start 2023-11-26T07:19:08.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64a5ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa4c90 0x559c64ab7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa55b0 0x559c64ab84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c648a8bc0 con 0x559c64aa4c90 2023-11-26T07:19:08.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c64a738c0 con 0x559c64a5b320 2023-11-26T07:19:08.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c648468a0 con 0x559c64aa55b0 2023-11-26T07:19:08.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f83592b7000 10 client.0 _ll_register_callbacks cb 0x559c648f9c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa4c90 0x559c64ab7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa4c90 0x559c64ab7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51680/0 (socket says 172.21.15.31:51680) 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64a5ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f834effd640 1 -- 172.21.15.31:0/800539281 learned_addr learned my addr 172.21.15.31:0/800539281 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.083+0000 7f834ffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa55b0 0x559c64ab84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f834effd640 1 -- 172.21.15.31:0/800539281 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa55b0 msgr2=0x559c64ab84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f834effd640 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa55b0 0x559c64ab84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f834effd640 1 -- 172.21.15.31:0/800539281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 msgr2=0x559c64a5ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f834effd640 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64a5ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f834effd640 1 -- 172.21.15.31:0/800539281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c6486c190 con 0x559c64aa4c90 2023-11-26T07:19:08.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f834effd640 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa4c90 0x559c64ab7fa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f83440ef8a0 tx=0x7f83440ef270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43becf24b928da23 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f832e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f832e7fc640 1 -- 172.21.15.31:0/800539281 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83440efd70 con 0x559c64aa4c90 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f832e7fc640 1 -- 172.21.15.31:0/800539281 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83440f39b0 con 0x559c64aa4c90 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 -- 172.21.15.31:0/800539281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c648aa110 con 0x559c64aa4c90 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f832e7fc640 1 -- 172.21.15.31:0/800539281 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83440f7b60 con 0x559c64aa4c90 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 10 client.5757 fetch_fsmap learned FSMap version 27 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 -- 172.21.15.31:0/800539281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x559c645a3cc0 con 0x559c64aa4c90 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f832e7fc640 1 -- 172.21.15.31:0/800539281 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c648aa110 con 0x559c64aa4c90 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f832e7fc640 1 -- 172.21.15.31:0/800539281 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83440eee00 con 0x559c64aa4c90 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 10 client.5757 fetch_fsmap finished waiting for FSMap version 27 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 10 client.5757 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:19:08.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 20 client.5757 populate_metadata read hostname 'smithi031' 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.31:6834/1115502954,v1:172.21.15.31:6835/1115502954] conn(0x559c64b0ae30 0x559c64b0d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 4 client.5757 mds_command: new command op to 5622 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 -- 172.21.15.31:0/800539281 --> [v2:172.21.15.31:6834/1115502954,v1:172.21.15.31:6835/1115502954] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559c648d99a0 con 0x559c64b0ae30 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f834f7fe640 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.31:6834/1115502954,v1:172.21.15.31:6835/1115502954] conn(0x559c64b0ae30 0x559c64b0d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f834f7fe640 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.31:6834/1115502954,v1:172.21.15.31:6835/1115502954] conn(0x559c64b0ae30 0x559c64b0d220 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e83ad0ed3a06cf1 server_cookie=3b6661a58f90aa91 in_seq=0 out_seq=0 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f832e7fc640 10 client.5757 ms_handle_connect on v2:172.21.15.31:6834/1115502954 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f832e7fc640 1 -- 172.21.15.31:0/800539281 <== mds.0 v2:172.21.15.31:6834/1115502954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x559c648d99a0 con 0x559c64b0ae30 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f832e7fc640 10 client.5757 handle_command_reply: tid=0 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 client.5757 shutdown 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 -- 172.21.15.31:0/800539281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa4c90 msgr2=0x559c64ab7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa4c90 0x559c64ab7fa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f83440ef8a0 tx=0x7f83440ef270 comp rx=0 tx=0).stop 2023-11-26T07:19:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 -- 172.21.15.31:0/800539281 shutdown_connections 2023-11-26T07:19:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.31:6834/1115502954,v1:172.21.15.31:6835/1115502954] conn(0x559c64b0ae30 0x559c64b0d220 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x559c64aa55b0 0x559c64ab84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x559c64aa4c90 0x559c64ab7fa0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 --2- 172.21.15.31:0/800539281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559c64a5b320 0x559c64a5ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f832e7fc640 0 client.5757 ms_handle_reset on v2:172.21.15.31:6834/1115502954 2023-11-26T07:19:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 -- 172.21.15.31:0/800539281 >> 172.21.15.31:0/800539281 conn(0x559c64a572d0 msgr2=0x559c64a596c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 -- 172.21.15.31:0/800539281 shutdown_connections 2023-11-26T07:19:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 1 -- 172.21.15.31:0/800539281 wait complete. 2023-11-26T07:19:08.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f83592b7000 20 client.5757 trim_cache size 0 max 16384 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "4c2b4df1-71b1-4841-8807-af144d2610e7", 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 5622, 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 5.9318516929999996, 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 27, 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 44, 2023-11-26T07:19:08.100 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 44, 2023-11-26T07:19:08.101 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 10.811613862 2023-11-26T07:19:08.101 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:19:08.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 -- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f833c066620 msgr2=0x7f833c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f833c066620 0x7f833c068ae0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8340000f70 tx=0x7f8340007240 comp rx=0 tx=0).stop 2023-11-26T07:19:08.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 -- 172.21.15.31:0/2658330369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83500cb9b0 msgr2=0x7f83501577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83500cb9b0 0x7f83501577e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f83440ae040 tx=0x7f834401acc0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 -- 172.21.15.31:0/2658330369 shutdown_connections 2023-11-26T07:19:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f833c066620 0x7f833c068ae0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f83500ce5d0 0x7f835015a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f83500cb9b0 0x7f83501577e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 --2- 172.21.15.31:0/2658330369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f835006e3f0 0x7f8350153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 -- 172.21.15.31:0/2658330369 >> 172.21.15.31:0/2658330369 conn(0x7f83500b0cb0 msgr2=0x7f835015bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 -- 172.21.15.31:0/2658330369 shutdown_connections 2023-11-26T07:19:08.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.087+0000 7f8355f55640 1 -- 172.21.15.31:0/2658330369 wait complete. 2023-11-26T07:19:08.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:19:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.251+0000 7efdac78d640 1 Processor -- start 2023-11-26T07:19:08.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.251+0000 7efdac78d640 1 -- start start 2023-11-26T07:19:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdac78d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efda4144c80 0x7efda40abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdac78d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4070df0 0x7efda406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdac78d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda4145060 con 0x7efda40ac460 2023-11-26T07:19:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdac78d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda4071fd0 con 0x7efda4144c80 2023-11-26T07:19:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdac78d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda40ac8c0 con 0x7efda4070df0 2023-11-26T07:19:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdaa529640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efda4144c80 0x7efda40abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdaa529640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efda4144c80 0x7efda40abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51692/0 (socket says 172.21.15.31:51692) 2023-11-26T07:19:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdaa529640 1 -- 172.21.15.31:0/3642059530 learned_addr learned my addr 172.21.15.31:0/3642059530 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdaad2a640 1 --2- 172.21.15.31:0/3642059530 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4070df0 0x7efda406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdaa529640 1 -- 172.21.15.31:0/3642059530 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4070df0 msgr2=0x7efda406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdaa529640 1 --2- 172.21.15.31:0/3642059530 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4070df0 0x7efda406c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdaa529640 1 -- 172.21.15.31:0/3642059530 >> v1:172.21.15.97:6789/0 conn(0x7efda40ac460 legacy=0x7efda40706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdaa529640 1 -- 172.21.15.31:0/3642059530 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda4071450 con 0x7efda4144c80 2023-11-26T07:19:08.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.255+0000 7efdaad2a640 1 --2- 172.21.15.31:0/3642059530 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4070df0 0x7efda406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:08.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdaa529640 1 --2- 172.21.15.31:0/3642059530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4144c80 0x7efda40abf20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efda0009870 tx=0x7efda001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87864112e055018 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efda9527640 1 -- 172.21.15.31:0/3642059530 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda00a7070 con 0x7efda4144c80 2023-11-26T07:19:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efda9527640 1 -- 172.21.15.31:0/3642059530 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda00ab430 con 0x7efda4144c80 2023-11-26T07:19:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efda9527640 1 -- 172.21.15.31:0/3642059530 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda00ab7c0 con 0x7efda4144c80 2023-11-26T07:19:08.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdac78d640 1 -- 172.21.15.31:0/3642059530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4144c80 msgr2=0x7efda40abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdac78d640 1 --2- 172.21.15.31:0/3642059530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4144c80 0x7efda40abf20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efda0009870 tx=0x7efda001c540 comp rx=0 tx=0).stop 2023-11-26T07:19:08.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdac78d640 1 -- 172.21.15.31:0/3642059530 shutdown_connections 2023-11-26T07:19:08.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdac78d640 1 --2- 172.21.15.31:0/3642059530 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4070df0 0x7efda406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdac78d640 1 --2- 172.21.15.31:0/3642059530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda4144c80 0x7efda40abf20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdac78d640 1 -- 172.21.15.31:0/3642059530 >> 172.21.15.31:0/3642059530 conn(0x7efda40b0c40 msgr2=0x7efda40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:08.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdac78d640 1 -- 172.21.15.31:0/3642059530 shutdown_connections 2023-11-26T07:19:08.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdac78d640 1 -- 172.21.15.31:0/3642059530 wait complete. 2023-11-26T07:19:08.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdac78d640 1 Processor -- start 2023-11-26T07:19:08.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.259+0000 7efdac78d640 1 -- start start 2023-11-26T07:19:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdac78d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4070df0 0x7efda40b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdac78d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda40ac460 0x7efda413bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdac78d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4144c80 0x7efda40cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdac78d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda409a990 con 0x7efda40ac460 2023-11-26T07:19:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdac78d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40b3bd0 con 0x7efda4070df0 2023-11-26T07:19:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdac78d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda413c0a0 con 0x7efda4144c80 2023-11-26T07:19:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efda9d28640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda40ac460 0x7efda413bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efda9d28640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda40ac460 0x7efda413bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51702/0 (socket says 172.21.15.31:51702) 2023-11-26T07:19:08.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efda9d28640 1 -- 172.21.15.31:0/3284482029 learned_addr learned my addr 172.21.15.31:0/3284482029 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdaa529640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4070df0 0x7efda40b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdaad2a640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4144c80 0x7efda40cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efda9d28640 1 -- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4144c80 msgr2=0x7efda40cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efda9d28640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4144c80 0x7efda40cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efda9d28640 1 -- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4070df0 msgr2=0x7efda40b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efda9d28640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4070df0 0x7efda40b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efda9d28640 1 -- 172.21.15.31:0/3284482029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda00094d0 con 0x7efda40ac460 2023-11-26T07:19:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efda9d28640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda40ac460 0x7efda413bb60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efd9400c500 tx=0x7efd9400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efd9affd640 1 -- 172.21.15.31:0/3284482029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94005220 con 0x7efda40ac460 2023-11-26T07:19:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efd9affd640 1 -- 172.21.15.31:0/3284482029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd94005380 con 0x7efda40ac460 2023-11-26T07:19:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efd9affd640 1 -- 172.21.15.31:0/3284482029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94007270 con 0x7efda40ac460 2023-11-26T07:19:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdac78d640 1 -- 172.21.15.31:0/3284482029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda4099a30 con 0x7efda40ac460 2023-11-26T07:19:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdac78d640 1 -- 172.21.15.31:0/3284482029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda4099ff0 con 0x7efda40ac460 2023-11-26T07:19:08.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.263+0000 7efdac78d640 1 -- 172.21.15.31:0/3284482029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda40b1ba0 con 0x7efda40ac460 2023-11-26T07:19:08.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.267+0000 7efd9affd640 1 -- 172.21.15.31:0/3284482029 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7efd940073d0 con 0x7efda40ac460 2023-11-26T07:19:08.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.267+0000 7efd9affd640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efd88066620 0x7efd88068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:08.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.267+0000 7efd9affd640 1 -- 172.21.15.31:0/3284482029 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd940833c0 con 0x7efda40ac460 2023-11-26T07:19:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.267+0000 7efdaa529640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efd88066620 0x7efd88068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:08.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.267+0000 7efdaa529640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efd88066620 0x7efd88068ae0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efda00af000 tx=0x7efda0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:08.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.271+0000 7efd9affd640 1 -- 172.21.15.31:0/3284482029 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efd94050730 con 0x7efda40ac460 2023-11-26T07:19:08.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.463+0000 7efdac78d640 1 -- 172.21.15.31:0/3284482029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efda409aad0 con 0x7efda40ac460 2023-11-26T07:19:08.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.463+0000 7efd9affd640 1 -- 172.21.15.31:0/3284482029 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7efda409aad0 con 0x7efda40ac460 2023-11-26T07:19:08.472 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:19:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 -- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efd88066620 msgr2=0x7efd88068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efd88066620 0x7efd88068ae0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efda00af000 tx=0x7efda0008040 comp rx=0 tx=0).stop 2023-11-26T07:19:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 -- 172.21.15.31:0/3284482029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda40ac460 msgr2=0x7efda413bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda40ac460 0x7efda413bb60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efd9400c500 tx=0x7efd9400c9c0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 -- 172.21.15.31:0/3284482029 shutdown_connections 2023-11-26T07:19:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efd88066620 0x7efd88068ae0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efda4144c80 0x7efda40cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efda40ac460 0x7efda413bb60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 --2- 172.21.15.31:0/3284482029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efda4070df0 0x7efda40b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 -- 172.21.15.31:0/3284482029 >> 172.21.15.31:0/3284482029 conn(0x7efda40b0c40 msgr2=0x7efda41655f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 -- 172.21.15.31:0/3284482029 shutdown_connections 2023-11-26T07:19:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:08.467+0000 7efdac78d640 1 -- 172.21.15.31:0/3284482029 wait complete. 2023-11-26T07:19:08.492 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:19:08.492 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:19:08.492 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-11-26T07:19:08.492 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:19:08.493 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:08.493 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:19:08.493 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:19:08.493 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:19:08.493 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:19:08.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:08.493 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-11-26T07:19:08.504 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:19:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:19:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:19:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-11-26T07:19:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:08.504 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:08.504 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-11-26T07:19:08.504 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42291sec preferred_lft 42291sec 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:08.505 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 02:42:1b:e4:a9:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:19:08.506 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:08.507 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:19:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:19:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:08.507 INFO:teuthology.orchestra.run.smithi031.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:19:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:19:08.507 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:08.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:08.508 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:19:08.570 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-11-26T07:19:08.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:08.572 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-11-26T07:19:08.624 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-11-26T07:19:08.625 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:19:08.625 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-26T07:19:08.625 INFO:teuthology.orchestra.run.smithi031.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:19:08.625 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-26T07:19:08.625 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:19:08.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:08.625 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:19:08.625 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:19:08.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-26T07:19:08.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-26T07:19:08.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-26T07:19:08.626 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:19:08.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:08.719 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:19:08.728 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:19:08.729 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:19:08.729 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:19:08.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:08.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-11-26T07:19:08.796 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:08.796 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:08.851 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:19:08.852 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:08.852 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:08.903 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:19:08.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:08.903 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:19:08.998 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:19:08.998 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:19:09.011 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:19:09.012 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:19:09.012 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:19:09.012 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:19:09.012 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:19:09.015 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:19:09.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:09.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:09.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:09.037 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:19:09.100 INFO:teuthology.orchestra.run.smithi031.stdout:client5766 2023-11-26T07:19:09.100 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:19:09.100 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:09.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:19:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.331+0000 7f5a631e7640 1 Processor -- start 2023-11-26T07:19:09.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.331+0000 7f5a631e7640 1 -- start start 2023-11-26T07:19:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a631e7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5a5c146c60 0x7f5a5c070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a631e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c06c220 0x7f5a5c070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a631e7640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5c15dba0 con 0x7f5a5c147040 2023-11-26T07:19:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a631e7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5c06c670 con 0x7f5a5c146c60 2023-11-26T07:19:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a631e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c06d100 con 0x7f5a5c06c220 2023-11-26T07:19:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5a5c147040 0x7f5a5c06bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52494/0 (socket says 172.21.15.31:52494) 2023-11-26T07:19:09.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5bfff640 1 -- 172.21.15.31:0/339290526 learned_addr learned my addr 172.21.15.31:0/339290526 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:09.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a61784640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c06c220 0x7f5a5c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:09.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a60f83640 1 --2- 172.21.15.31:0/339290526 >> v2:172.21.15.97:3300/0 conn(0x7f5a5c146c60 0x7f5a5c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:09.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238539554 0 0) 0x7f5a5c15dba0 con 0x7f5a5c147040 2023-11-26T07:19:09.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a4c003530 con 0x7f5a5c147040 2023-11-26T07:19:09.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 212578366 0 0) 0x7f5a4c003530 con 0x7f5a5c147040 2023-11-26T07:19:09.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c06c220 msgr2=0x7f5a5c070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:09.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c06c220 0x7f5a5c070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:09.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 >> v2:172.21.15.97:3300/0 conn(0x7f5a5c146c60 msgr2=0x7f5a5c070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 --2- 172.21.15.31:0/339290526 >> v2:172.21.15.97:3300/0 conn(0x7f5a5c146c60 0x7f5a5c070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c06c860 con 0x7f5a5c147040 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f5a480016a0 con 0x7f5a5c147040 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 >> v1:172.21.15.31:6789/0 conn(0x7f5a5c147040 legacy=0x7f5a5c06bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c06c220 0x7f5a4c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c003f50 0x7f5a4c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c004d10 0x7f5a4c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c06d100 con 0x7f5a4c004d10 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c00b990 con 0x7f5a4c003f50 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c00bad0 con 0x7f5a5c06c220 2023-11-26T07:19:09.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a5bfff640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c003f50 0x7f5a4c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a61784640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c004d10 0x7f5a4c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a60f83640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c06c220 0x7f5a4c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a61784640 1 -- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c06c220 msgr2=0x7f5a4c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a61784640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c06c220 0x7f5a4c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a61784640 1 -- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c003f50 msgr2=0x7f5a4c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a61784640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c003f50 0x7f5a4c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:09.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.335+0000 7f5a61784640 1 -- 172.21.15.31:0/339290526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c003530 con 0x7f5a4c004d10 2023-11-26T07:19:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a61784640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c004d10 0x7f5a4c005190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e170 tx=0x7f5a5401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee08b4f34351687 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540a5c50 con 0x7f5a4c004d10 2023-11-26T07:19:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5400f040 con 0x7f5a4c004d10 2023-11-26T07:19:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a5b7fe640 1 -- 172.21.15.31:0/339290526 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540ad440 con 0x7f5a4c004d10 2023-11-26T07:19:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a631e7640 1 -- 172.21.15.31:0/339290526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c004d10 msgr2=0x7f5a4c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:09.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a631e7640 1 --2- 172.21.15.31:0/339290526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c004d10 0x7f5a4c005190 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e170 tx=0x7f5a5401cc40 comp rx=0 tx=0).stop 2023-11-26T07:19:09.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a61784640 1 -- 172.21.15.31:0/339290526 reap_dead start 2023-11-26T07:19:09.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a631e7640 1 -- 172.21.15.31:0/339290526 shutdown_connections 2023-11-26T07:19:09.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a631e7640 1 -- 172.21.15.31:0/339290526 >> 172.21.15.31:0/339290526 conn(0x7f5a5c0b2ea0 msgr2=0x7f5a5c0b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:09.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a631e7640 1 -- 172.21.15.31:0/339290526 shutdown_connections 2023-11-26T07:19:09.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a631e7640 1 -- 172.21.15.31:0/339290526 wait complete. 2023-11-26T07:19:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a631e7640 1 Processor -- start 2023-11-26T07:19:09.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.339+0000 7f5a631e7640 1 -- start start 2023-11-26T07:19:09.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a631e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c004d10 0x7f5a5c15c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a631e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c1e76d0 0x7f5a5c1e7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a631e7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5c13d4d0 0x7f5a5c13d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a631e7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c13cff0 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a631e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c1e8430 con 0x7f5a4c004d10 2023-11-26T07:19:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a631e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c1e8710 con 0x7f5a5c1e76d0 2023-11-26T07:19:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a61784640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5c13d4d0 0x7f5a5c13d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a61784640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5c13d4d0 0x7f5a5c13d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51712/0 (socket says 172.21.15.31:51712) 2023-11-26T07:19:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a61784640 1 -- 172.21.15.31:0/940438262 learned_addr learned my addr 172.21.15.31:0/940438262 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a5bfff640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c1e76d0 0x7f5a5c1e7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:09.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a60f83640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c004d10 0x7f5a5c15c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a61784640 1 -- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c1e76d0 msgr2=0x7f5a5c1e7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a61784640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c1e76d0 0x7f5a5c1e7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a61784640 1 -- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c004d10 msgr2=0x7f5a5c15c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a61784640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c004d10 0x7f5a5c15c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a61784640 1 -- 172.21.15.31:0/940438262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c1e89f0 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a60f83640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c004d10 0x7f5a5c15c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a61784640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5c13d4d0 0x7f5a5c13d950 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e170 tx=0x7f5a54009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a597fa640 1 -- 172.21.15.31:0/940438262 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540bc070 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a597fa640 1 -- 172.21.15.31:0/940438262 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a540039a0 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a597fa640 1 -- 172.21.15.31:0/940438262 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a54003b30 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a631e7640 1 -- 172.21.15.31:0/940438262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c1e8cd0 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a631e7640 1 -- 172.21.15.31:0/940438262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c13de90 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.343+0000 7f5a631e7640 1 -- 172.21.15.31:0/940438262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c067a40 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.347+0000 7f5a597fa640 1 -- 172.21.15.31:0/940438262 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f5a540b4040 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.347+0000 7f5a597fa640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a40066650 0x7f5a40068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:09.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.347+0000 7f5a597fa640 1 -- 172.21.15.31:0/940438262 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a5411e3c0 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.347+0000 7f5a60f83640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a40066650 0x7f5a40068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:09.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.347+0000 7f5a60f83640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a40066650 0x7f5a40068b10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009c60 tx=0x7f5a50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:09.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.351+0000 7f5a597fa640 1 -- 172.21.15.31:0/940438262 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5a540eb5d0 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 -- 172.21.15.31:0/940438262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5a5c04e850 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a597fa640 1 -- 172.21.15.31:0/940438262 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5a540b2060 con 0x7f5a5c13d4d0 2023-11-26T07:19:09.548 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:19:09.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 -- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a40066650 msgr2=0x7f5a40068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:09.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a40066650 0x7f5a40068b10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009c60 tx=0x7f5a50008040 comp rx=0 tx=0).stop 2023-11-26T07:19:09.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 -- 172.21.15.31:0/940438262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5c13d4d0 msgr2=0x7f5a5c13d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:09.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5c13d4d0 0x7f5a5c13d950 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400e170 tx=0x7f5a54009750 comp rx=0 tx=0).stop 2023-11-26T07:19:09.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 -- 172.21.15.31:0/940438262 shutdown_connections 2023-11-26T07:19:09.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5a40066650 0x7f5a40068b10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:09.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a5c13d4d0 0x7f5a5c13d950 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:09.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a5c1e76d0 0x7f5a5c1e7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:09.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 --2- 172.21.15.31:0/940438262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c004d10 0x7f5a5c15c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:09.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 -- 172.21.15.31:0/940438262 >> 172.21.15.31:0/940438262 conn(0x7f5a5c0b2ea0 msgr2=0x7f5a5c160230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:09.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 -- 172.21.15.31:0/940438262 shutdown_connections 2023-11-26T07:19:09.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:09.543+0000 7f5a631e7640 1 -- 172.21.15.31:0/940438262 wait complete. 2023-11-26T07:19:09.569 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:19:09.569 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:19:09.569 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-26T07:19:09.569 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:19:09.569 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:09.570 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:19:09.570 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:19:09.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:19:09.570 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:19:09.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:09.570 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-26T07:19:09.580 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:19:09.580 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-11-26T07:19:09.581 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:19:09.582 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:09.582 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:19:09.582 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:09.582 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:19:09.582 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42313sec preferred_lft 42313sec 2023-11-26T07:19:09.582 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-11-26T07:19:09.582 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:09.582 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:19:09.582 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:61:bc:5b:3d brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:09.582 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:19:09.583 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:09.584 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:19:09.584 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:19:09.584 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:19:09.584 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:19:09.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:09.585 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:19:09.647 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-26T07:19:09.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:09.650 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-26T07:19:09.702 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-26T07:19:09.702 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:19:09.702 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:19:09.702 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:19:09.702 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:19:09.703 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:19:09.703 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:09.703 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:09.703 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:19:09.703 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:19:09.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:09.703 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:19:09.703 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:19:09.703 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:19:09.703 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:19:09.704 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:19:09.704 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:19:09.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:09.797 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:19:09.806 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:19:09.806 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:19:09.807 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:19:09.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:09.807 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-26T07:19:09.867 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:09.868 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:09.919 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:19:09.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:19:09.920 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:09.970 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:19:09.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:09.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:19:10.062 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:19:10.062 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:19:10.076 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:19:10.076 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:19:10.076 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:19:10.076 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 1@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:19:10.076 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:19:10.079 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:19:10.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:10.079 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:10.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:10.099 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:19:10.154 INFO:teuthology.orchestra.run.smithi097.stdout:client5775 2023-11-26T07:19:10.154 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:19:10.154 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:10.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:19:10.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.351+0000 7f7bbdb80640 1 Processor -- start 2023-11-26T07:19:10.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.351+0000 7f7bbdb80640 1 -- start start 2023-11-26T07:19:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.351+0000 7f7bbdb80640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bb8146820 0x7f7bb8070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.351+0000 7f7bbdb80640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bb80b4210 0x7f7bb80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.351+0000 7f7bbdb80640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb8147190 con 0x7f7bb8161180 2023-11-26T07:19:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.351+0000 7f7bbdb80640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb8146c00 con 0x7f7bb8146820 2023-11-26T07:19:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bbdb80640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb806bf90 con 0x7f7bb80b4210 2023-11-26T07:19:10.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb77fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bb8146820 0x7f7bb8070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb7fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7bb8161180 0x7f7bb81615d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52496/0 (socket says 172.21.15.31:52496) 2023-11-26T07:19:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb77fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7bb8146820 0x7f7bb8070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51714/0 (socket says 172.21.15.31:51714) 2023-11-26T07:19:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb7fff640 1 -- 172.21.15.31:0/2249424349 learned_addr learned my addr 172.21.15.31:0/2249424349 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb6ffd640 1 --2- 172.21.15.31:0/2249424349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bb80b4210 0x7f7bb80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb77fe640 1 -- 172.21.15.31:0/2249424349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bb80b4210 msgr2=0x7f7bb80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb77fe640 1 --2- 172.21.15.31:0/2249424349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bb80b4210 0x7f7bb80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb77fe640 1 -- 172.21.15.31:0/2249424349 >> v1:172.21.15.31:6789/0 conn(0x7f7bb8161180 legacy=0x7f7bb81615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:10.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb77fe640 1 -- 172.21.15.31:0/2249424349 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80b4bb0 con 0x7f7bb8146820 2023-11-26T07:19:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb77fe640 1 --2- 172.21.15.31:0/2249424349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb8146820 0x7f7bb8070a50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80097c0 tx=0x7f7ba801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9806319b1dcdcc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.355+0000 7f7bb67fc640 1 -- 172.21.15.31:0/2249424349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80ab650 con 0x7f7bb8146820 2023-11-26T07:19:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bb67fc640 1 -- 172.21.15.31:0/2249424349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba80ab7b0 con 0x7f7bb8146820 2023-11-26T07:19:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bb67fc640 1 -- 172.21.15.31:0/2249424349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80abb40 con 0x7f7bb8146820 2023-11-26T07:19:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 -- 172.21.15.31:0/2249424349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb8146820 msgr2=0x7f7bb8070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:10.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 --2- 172.21.15.31:0/2249424349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb8146820 0x7f7bb8070a50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80097c0 tx=0x7f7ba801c450 comp rx=0 tx=0).stop 2023-11-26T07:19:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 -- 172.21.15.31:0/2249424349 shutdown_connections 2023-11-26T07:19:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 --2- 172.21.15.31:0/2249424349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bb80b4210 0x7f7bb80b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 --2- 172.21.15.31:0/2249424349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb8146820 0x7f7bb8070a50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 -- 172.21.15.31:0/2249424349 >> 172.21.15.31:0/2249424349 conn(0x7f7bb80b2f10 msgr2=0x7f7bb80b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 -- 172.21.15.31:0/2249424349 shutdown_connections 2023-11-26T07:19:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 -- 172.21.15.31:0/2249424349 wait complete. 2023-11-26T07:19:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 Processor -- start 2023-11-26T07:19:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 -- start start 2023-11-26T07:19:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb80b4210 0x7f7bb815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bb8146820 0x7f7bb8138b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bb8161180 0x7f7bb813b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb815b980 con 0x7f7bb80b4210 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb815bc60 con 0x7f7bb8161180 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bbdb80640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb815bf40 con 0x7f7bb8146820 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb80b4210 0x7f7bb815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb80b4210 0x7f7bb815c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51718/0 (socket says 172.21.15.31:51718) 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bb77fe640 1 -- 172.21.15.31:0/957489817 learned_addr learned my addr 172.21.15.31:0/957489817 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bb6ffd640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bb8146820 0x7f7bb8138b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.359+0000 7f7bb7fff640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bb8161180 0x7f7bb813b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7bb77fe640 1 -- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bb8146820 msgr2=0x7f7bb8138b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7bb77fe640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bb8146820 0x7f7bb8138b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7bb77fe640 1 -- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bb8161180 msgr2=0x7f7bb813b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7bb77fe640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bb8161180 0x7f7bb813b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:10.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7bb77fe640 1 -- 172.21.15.31:0/957489817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba8009420 con 0x7f7bb80b4210 2023-11-26T07:19:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7bb77fe640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb80b4210 0x7f7bb815c9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80ad040 tx=0x7f7ba801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7b9ffff640 1 -- 172.21.15.31:0/957489817 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80b5810 con 0x7f7bb80b4210 2023-11-26T07:19:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7b9ffff640 1 -- 172.21.15.31:0/957489817 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba80b5990 con 0x7f7bb80b4210 2023-11-26T07:19:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7b9ffff640 1 -- 172.21.15.31:0/957489817 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80b5d40 con 0x7f7bb80b4210 2023-11-26T07:19:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7bbdb80640 1 -- 172.21.15.31:0/957489817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb806d030 con 0x7f7bb80b4210 2023-11-26T07:19:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7bbdb80640 1 -- 172.21.15.31:0/957489817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb815c500 con 0x7f7bb80b4210 2023-11-26T07:19:10.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7bbdb80640 1 -- 172.21.15.31:0/957489817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb815c6f0 con 0x7f7bb80b4210 2023-11-26T07:19:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7b9ffff640 1 -- 172.21.15.31:0/957489817 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f7ba80b4390 con 0x7f7bb80b4210 2023-11-26T07:19:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7b9ffff640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066650 0x7f7ba4068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.363+0000 7f7b9ffff640 1 -- 172.21.15.31:0/957489817 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ba811eb10 con 0x7f7bb80b4210 2023-11-26T07:19:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.367+0000 7f7bb6ffd640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066650 0x7f7ba4068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:10.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.367+0000 7f7bb6ffd640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066650 0x7f7ba4068b10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80b3c40 tx=0x7f7bac009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:10.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.367+0000 7f7b9ffff640 1 -- 172.21.15.31:0/957489817 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7ba80ebdd0 con 0x7f7bb80b4210 2023-11-26T07:19:10.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:10.559+0000 7f7bbdb80640 1 -- 172.21.15.31:0/957489817 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7bb815c080 con 0x7f7bb80b4210 2023-11-26T07:19:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7b9ffff640 1 -- 172.21.15.31:0/957489817 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 136+0+0 (secure 0 0 0) 0x7f7bb815c080 con 0x7f7bb80b4210 2023-11-26T07:19:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 -- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066650 msgr2=0x7f7ba4068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066650 0x7f7ba4068b10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80b3c40 tx=0x7f7bac009190 comp rx=0 tx=0).stop 2023-11-26T07:19:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 -- 172.21.15.31:0/957489817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb80b4210 msgr2=0x7f7bb815c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb80b4210 0x7f7bb815c9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80ad040 tx=0x7f7ba801ac90 comp rx=0 tx=0).stop 2023-11-26T07:19:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 -- 172.21.15.31:0/957489817 shutdown_connections 2023-11-26T07:19:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7ba4066650 0x7f7ba4068b10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7bb8161180 0x7f7bb813b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7bb8146820 0x7f7bb8138b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 --2- 172.21.15.31:0/957489817 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb80b4210 0x7f7bb815c9c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 -- 172.21.15.31:0/957489817 >> 172.21.15.31:0/957489817 conn(0x7f7bb80b2f10 msgr2=0x7f7bb806d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 -- 172.21.15.31:0/957489817 shutdown_connections 2023-11-26T07:19:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.211+0000 7f7bbdb80640 1 -- 172.21.15.31:0/957489817 wait complete. 2023-11-26T07:19:11.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:19:11.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.383+0000 7ffab578d640 1 Processor -- start 2023-11-26T07:19:11.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.383+0000 7ffab578d640 1 -- start start 2023-11-26T07:19:11.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffab578d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffab00b46a0 0x7ffab015e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffab578d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab0147190 con 0x7ffab0146820 2023-11-26T07:19:11.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffab578d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab006d030 con 0x7ffab00b4210 2023-11-26T07:19:11.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffab578d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab0146c00 con 0x7ffab00b46a0 2023-11-26T07:19:11.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaaeffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffab0146820 0x7ffab0070a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52498/0 (socket says 172.21.15.31:52498) 2023-11-26T07:19:11.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaaeffd640 1 -- 172.21.15.31:0/3249584269 learned_addr learned my addr 172.21.15.31:0/3249584269 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:11.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaaf7fe640 1 --2- 172.21.15.31:0/3249584269 >> v2:172.21.15.31:3300/0 conn(0x7ffab00b46a0 0x7ffab015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaae7fc640 1 -- 172.21.15.31:0/3249584269 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711328055 0 0) 0x7ffab0147190 con 0x7ffab0146820 2023-11-26T07:19:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaae7fc640 1 -- 172.21.15.31:0/3249584269 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa9c003530 con 0x7ffab0146820 2023-11-26T07:19:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaaf7fe640 1 -- 172.21.15.31:0/3249584269 >> v1:172.21.15.97:6789/0 conn(0x7ffab00b4210 legacy=0x7ffab015db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaaf7fe640 1 -- 172.21.15.31:0/3249584269 >> v1:172.21.15.31:6789/0 conn(0x7ffab0146820 legacy=0x7ffab0070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaaf7fe640 1 -- 172.21.15.31:0/3249584269 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab006bf90 con 0x7ffab00b46a0 2023-11-26T07:19:11.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaaf7fe640 1 --2- 172.21.15.31:0/3249584269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab00b46a0 0x7ffab015e250 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009d90 tx=0x7ffaa801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5c51840e6a35e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaae7fc640 1 -- 172.21.15.31:0/3249584269 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa80a59e0 con 0x7ffab00b46a0 2023-11-26T07:19:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaae7fc640 1 -- 172.21.15.31:0/3249584269 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa80a5b40 con 0x7ffab00b46a0 2023-11-26T07:19:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.387+0000 7ffaae7fc640 1 -- 172.21.15.31:0/3249584269 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa80ad440 con 0x7ffab00b46a0 2023-11-26T07:19:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 -- 172.21.15.31:0/3249584269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab00b46a0 msgr2=0x7ffab015e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 --2- 172.21.15.31:0/3249584269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab00b46a0 0x7ffab015e250 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009d90 tx=0x7ffaa801c920 comp rx=0 tx=0).stop 2023-11-26T07:19:11.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 -- 172.21.15.31:0/3249584269 shutdown_connections 2023-11-26T07:19:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 --2- 172.21.15.31:0/3249584269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab00b46a0 0x7ffab015e250 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 -- 172.21.15.31:0/3249584269 >> 172.21.15.31:0/3249584269 conn(0x7ffab00b2f10 msgr2=0x7ffab00b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 -- 172.21.15.31:0/3249584269 shutdown_connections 2023-11-26T07:19:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 -- 172.21.15.31:0/3249584269 wait complete. 2023-11-26T07:19:11.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 Processor -- start 2023-11-26T07:19:11.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 -- start start 2023-11-26T07:19:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffab00b4210 0x7ffab01f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab00b46a0 0x7ffab013f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab0146820 0x7ffab013a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab01f68a0 con 0x7ffab0146820 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab01f6b80 con 0x7ffab00b46a0 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffab578d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab01f6e60 con 0x7ffab00b4210 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffaaf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab0146820 0x7ffab013a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffaaf7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab0146820 0x7ffab013a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51742/0 (socket says 172.21.15.31:51742) 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffaaf7fe640 1 -- 172.21.15.31:0/2145727038 learned_addr learned my addr 172.21.15.31:0/2145727038 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffaaeffd640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffab00b4210 0x7ffab01f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffaa7fff640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab00b46a0 0x7ffab013f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.391+0000 7ffaaf7fe640 1 -- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffab00b4210 msgr2=0x7ffab01f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaaf7fe640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffab00b4210 0x7ffab01f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaaf7fe640 1 -- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab00b46a0 msgr2=0x7ffab013f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaaf7fe640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab00b46a0 0x7ffab013f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaaf7fe640 1 -- 172.21.15.31:0/2145727038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8009a70 con 0x7ffab0146820 2023-11-26T07:19:11.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaa7fff640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab00b46a0 0x7ffab013f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:11.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaaeffd640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffab00b4210 0x7ffab01f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:11.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaaf7fe640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab0146820 0x7ffab013a240 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8019530 tx=0x7ffaa80196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaa77fe640 1 -- 172.21.15.31:0/2145727038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa80aee00 con 0x7ffab0146820 2023-11-26T07:19:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaa77fe640 1 -- 172.21.15.31:0/2145727038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa8009450 con 0x7ffab0146820 2023-11-26T07:19:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaa77fe640 1 -- 172.21.15.31:0/2145727038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8003bc0 con 0x7ffab0146820 2023-11-26T07:19:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffab578d640 1 -- 172.21.15.31:0/2145727038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab0139a40 con 0x7ffab0146820 2023-11-26T07:19:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffab578d640 1 -- 172.21.15.31:0/2145727038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab01f6560 con 0x7ffab0146820 2023-11-26T07:19:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffab578d640 1 -- 172.21.15.31:0/2145727038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab006bb10 con 0x7ffab0146820 2023-11-26T07:19:11.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaa77fe640 1 -- 172.21.15.31:0/2145727038 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ffaa80b4050 con 0x7ffab0146820 2023-11-26T07:19:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaa77fe640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa90066620 0x7ffa90068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaa77fe640 1 -- 172.21.15.31:0/2145727038 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffaa811e890 con 0x7ffab0146820 2023-11-26T07:19:11.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.395+0000 7ffaaeffd640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa90066620 0x7ffa90068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.399+0000 7ffaaeffd640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa90066620 0x7ffa90068ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00011f0 tx=0x7ffaa0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.399+0000 7ffaa77fe640 1 -- 172.21.15.31:0/2145727038 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffaa80eba90 con 0x7ffab0146820 2023-11-26T07:19:11.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.579+0000 7ffab578d640 1 -- 172.21.15.31:0/2145727038 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffab00b4ab0 con 0x7ffa90066620 2023-11-26T07:19:11.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.579+0000 7ffaa77fe640 1 -- 172.21.15.31:0/2145727038 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ffaa00013e0 con 0x7ffa90066620 2023-11-26T07:19:11.587 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:19:11.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:19:11.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-11-26T07:19:11.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:19:11.587 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:19:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 -- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa90066620 msgr2=0x7ffa90068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa90066620 0x7ffa90068ae0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00011f0 tx=0x7ffaa0009040 comp rx=0 tx=0).stop 2023-11-26T07:19:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 -- 172.21.15.31:0/2145727038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab0146820 msgr2=0x7ffab013a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab0146820 0x7ffab013a240 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8019530 tx=0x7ffaa80196c0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 -- 172.21.15.31:0/2145727038 shutdown_connections 2023-11-26T07:19:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ffa90066620 0x7ffa90068ae0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffab0146820 0x7ffab013a240 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffab00b46a0 0x7ffab013f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 --2- 172.21.15.31:0/2145727038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffab00b4210 0x7ffab01f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 -- 172.21.15.31:0/2145727038 >> 172.21.15.31:0/2145727038 conn(0x7ffab00b2f10 msgr2=0x7ffab0143e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:11.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 -- 172.21.15.31:0/2145727038 shutdown_connections 2023-11-26T07:19:11.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.583+0000 7ffab578d640 1 -- 172.21.15.31:0/2145727038 wait complete. 2023-11-26T07:19:11.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:19:11.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.747+0000 7f38e009a640 1 Processor -- start 2023-11-26T07:19:11.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.747+0000 7f38e009a640 1 -- start start 2023-11-26T07:19:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38e009a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f38d80cb9b0 0x7f38d806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38e009a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d80cbd90 0x7f38d815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38e009a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d8159630 con 0x7f38d80ce5d0 2023-11-26T07:19:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38e009a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d815bba0 con 0x7f38d80cb9b0 2023-11-26T07:19:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38e009a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80cdd50 con 0x7f38d80cbd90 2023-11-26T07:19:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38de637640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f38d80ce5d0 0x7f38d815bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52500/0 (socket says 172.21.15.31:52500) 2023-11-26T07:19:11.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38de637640 1 -- 172.21.15.31:0/1990154096 learned_addr learned my addr 172.21.15.31:0/1990154096 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dd635640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d80cbd90 0x7f38d815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dde36640 1 --2- 172.21.15.31:0/1990154096 >> v2:172.21.15.97:3300/0 conn(0x7f38d80cb9b0 0x7f38d806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615842372 0 0) 0x7f38d8159630 con 0x7f38d80ce5d0 2023-11-26T07:19:11.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38c4003530 con 0x7f38d80ce5d0 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1610880851 0 0) 0x7f38c4003530 con 0x7f38d80ce5d0 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d80cbd90 msgr2=0x7f38d815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d80cbd90 0x7f38d815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 >> v2:172.21.15.97:3300/0 conn(0x7f38d80cb9b0 msgr2=0x7f38d806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 --2- 172.21.15.31:0/1990154096 >> v2:172.21.15.97:3300/0 conn(0x7f38d80cb9b0 0x7f38d806e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d815f6d0 con 0x7f38d80ce5d0 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f38d0001a00 con 0x7f38d80ce5d0 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 >> v1:172.21.15.31:6789/0 conn(0x7f38d80ce5d0 legacy=0x7f38d815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 0x7f38c4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38c40047d0 0x7f38c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c4005190 0x7f38c400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80cdd50 con 0x7f38c4003980 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c400bed0 con 0x7f38c4005190 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c400c010 con 0x7f38c40047d0 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dd635640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d80cbd90 0x7f38d815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dde36640 1 --2- 172.21.15.31:0/1990154096 >> v2:172.21.15.97:3300/0 conn(0x7f38d80cb9b0 0x7f38d806e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dde36640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 0x7f38c4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38de637640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c4005190 0x7f38c400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dd635640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38c40047d0 0x7f38c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dde36640 1 -- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38c40047d0 msgr2=0x7f38c4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dde36640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38c40047d0 0x7f38c4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dde36640 1 -- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c4005190 msgr2=0x7f38c400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:11.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dde36640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38c4005190 0x7f38c400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.751+0000 7f38dde36640 1 -- 172.21.15.31:0/1990154096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c4003530 con 0x7f38c4003980 2023-11-26T07:19:11.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38dde36640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 0x7f38c4003da0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009cc0 tx=0x7f38d401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7710a459ef1f611 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4004020 con 0x7f38c4003980 2023-11-26T07:19:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d4004180 con 0x7f38c4003980 2023-11-26T07:19:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38dce34640 1 -- 172.21.15.31:0/1990154096 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40a58a0 con 0x7f38c4003980 2023-11-26T07:19:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38e009a640 1 -- 172.21.15.31:0/1990154096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 msgr2=0x7f38c4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38e009a640 1 --2- 172.21.15.31:0/1990154096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 0x7f38c4003da0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009cc0 tx=0x7f38d401cc00 comp rx=0 tx=0).stop 2023-11-26T07:19:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38de637640 1 -- 172.21.15.31:0/1990154096 reap_dead start 2023-11-26T07:19:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38e009a640 1 -- 172.21.15.31:0/1990154096 shutdown_connections 2023-11-26T07:19:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38e009a640 1 -- 172.21.15.31:0/1990154096 >> 172.21.15.31:0/1990154096 conn(0x7f38d80b0cb0 msgr2=0x7f38d80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38e009a640 1 -- 172.21.15.31:0/1990154096 shutdown_connections 2023-11-26T07:19:11.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.755+0000 7f38e009a640 1 -- 172.21.15.31:0/1990154096 wait complete. 2023-11-26T07:19:11.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 Processor -- start 2023-11-26T07:19:11.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 -- start start 2023-11-26T07:19:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 0x7f38d814b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d8156980 0x7f38d815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38d8161740 0x7f38d8161bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d8162100 con 0x7f38c4003980 2023-11-26T07:19:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80b2b30 con 0x7f38d8161740 2023-11-26T07:19:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80b33d0 con 0x7f38d8156980 2023-11-26T07:19:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dde36640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 0x7f38d814b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dde36640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 0x7f38d814b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51760/0 (socket says 172.21.15.31:51760) 2023-11-26T07:19:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dde36640 1 -- 172.21.15.31:0/4285052264 learned_addr learned my addr 172.21.15.31:0/4285052264 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dd635640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d8156980 0x7f38d815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38de637640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38d8161740 0x7f38d8161bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dde36640 1 -- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d8156980 msgr2=0x7f38d815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dde36640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d8156980 0x7f38d815d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dde36640 1 -- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38d8161740 msgr2=0x7f38d8161bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dde36640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38d8161740 0x7f38d8161bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dde36640 1 -- 172.21.15.31:0/4285052264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d815dc80 con 0x7f38c4003980 2023-11-26T07:19:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dd635640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d8156980 0x7f38d815d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:11.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38dde36640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 0x7f38d814b8b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f38d40af040 tx=0x7f38c4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38ce7fc640 1 -- 172.21.15.31:0/4285052264 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40044f0 con 0x7f38c4003980 2023-11-26T07:19:11.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38ce7fc640 1 -- 172.21.15.31:0/4285052264 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d400f080 con 0x7f38c4003980 2023-11-26T07:19:11.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 -- 172.21.15.31:0/4285052264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d80b2d80 con 0x7f38c4003980 2023-11-26T07:19:11.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38ce7fc640 1 -- 172.21.15.31:0/4285052264 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d40a5aa0 con 0x7f38c4003980 2023-11-26T07:19:11.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 -- 172.21.15.31:0/4285052264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d814bdf0 con 0x7f38c4003980 2023-11-26T07:19:11.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.759+0000 7f38e009a640 1 -- 172.21.15.31:0/4285052264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d80671f0 con 0x7f38c4003980 2023-11-26T07:19:11.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.763+0000 7f38ce7fc640 1 -- 172.21.15.31:0/4285052264 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f38d40a5c00 con 0x7f38c4003980 2023-11-26T07:19:11.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.763+0000 7f38ce7fc640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f38bc066670 0x7f38bc068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:11.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.763+0000 7f38ce7fc640 1 -- 172.21.15.31:0/4285052264 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38d411eb10 con 0x7f38c4003980 2023-11-26T07:19:11.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.763+0000 7f38dd635640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f38bc066670 0x7f38bc068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:11.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.763+0000 7f38dd635640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f38bc066670 0x7f38bc068b30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f38d80cb7f0 tx=0x7f38c8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:11.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.767+0000 7f38ce7fc640 1 -- 172.21.15.31:0/4285052264 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f38d40ebcf0 con 0x7f38c4003980 2023-11-26T07:19:11.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.923+0000 7f38e009a640 1 -- 172.21.15.31:0/4285052264 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f38d80671f0 con 0x7f38c4003980 2023-11-26T07:19:11.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38ce7fc640 1 -- 172.21.15.31:0/4285052264 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f38d40ef310 con 0x7f38c4003980 2023-11-26T07:19:11.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 -- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f38bc066670 msgr2=0x7f38bc068b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f38bc066670 0x7f38bc068b30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f38d80cb7f0 tx=0x7f38c8008040 comp rx=0 tx=0).stop 2023-11-26T07:19:11.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 -- 172.21.15.31:0/4285052264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 msgr2=0x7f38d814b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:11.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 0x7f38d814b8b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f38d40af040 tx=0x7f38c4005800 comp rx=0 tx=0).stop 2023-11-26T07:19:11.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 -- 172.21.15.31:0/4285052264 shutdown_connections 2023-11-26T07:19:11.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f38bc066670 0x7f38bc068b30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38d8161740 0x7f38d8161bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f38d8156980 0x7f38d815d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 --2- 172.21.15.31:0/4285052264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38c4003980 0x7f38d814b8b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:11.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 -- 172.21.15.31:0/4285052264 >> 172.21.15.31:0/4285052264 conn(0x7f38d80b0cb0 msgr2=0x7f38d8157370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:11.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 -- 172.21.15.31:0/4285052264 shutdown_connections 2023-11-26T07:19:11.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:11.927+0000 7f38e009a640 1 -- 172.21.15.31:0/4285052264 wait complete. 2023-11-26T07:19:11.953 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:19:11.953 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:11.962 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:19:11.962 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:12.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:12.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba4053e640 1 Processor -- start 2023-11-26T07:19:12.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba4053e640 1 -- start start 2023-11-26T07:19:12.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba4053e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba3815f540 0x7fba380b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba4053e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba380b1f40 0x7fba3806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba4053e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba380b2920 con 0x7fba380b23c0 2023-11-26T07:19:12.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba4053e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba380b2c00 con 0x7fba380b1f40 2023-11-26T07:19:12.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba4053e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba380b2ee0 con 0x7fba3815f540 2023-11-26T07:19:12.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba3eadb640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fba380b23c0 0x7fba38070880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52504/0 (socket says 172.21.15.31:52504) 2023-11-26T07:19:12.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba3eadb640 1 -- 172.21.15.31:0/3392617444 learned_addr learned my addr 172.21.15.31:0/3392617444 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:12.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba3dad9640 1 --2- 172.21.15.31:0/3392617444 >> v2:172.21.15.97:3300/0 conn(0x7fba380b1f40 0x7fba3806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba3e2da640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba3815f540 0x7fba380b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725522117 0 0) 0x7fba380b2920 con 0x7fba380b23c0 2023-11-26T07:19:12.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.191+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba24003530 con 0x7fba380b23c0 2023-11-26T07:19:12.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 178063926 0 0) 0x7fba24003530 con 0x7fba380b23c0 2023-11-26T07:19:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba3815f540 msgr2=0x7fba380b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba3815f540 0x7fba380b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 >> v2:172.21.15.97:3300/0 conn(0x7fba380b1f40 msgr2=0x7fba3806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 --2- 172.21.15.31:0/3392617444 >> v2:172.21.15.97:3300/0 conn(0x7fba380b1f40 0x7fba3806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba380b31c0 con 0x7fba380b23c0 2023-11-26T07:19:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fba30001a00 con 0x7fba380b23c0 2023-11-26T07:19:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 >> v1:172.21.15.31:6789/0 conn(0x7fba380b23c0 legacy=0x7fba38070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba24003980 0x7fba24003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba240047d0 0x7fba24004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba24005190 0x7fba2400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba380b2c00 con 0x7fba240047d0 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba380b2ee0 con 0x7fba24003980 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2400bed0 con 0x7fba24005190 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3e2da640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba3815f540 0x7fba380b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3dad9640 1 --2- 172.21.15.31:0/3392617444 >> v2:172.21.15.97:3300/0 conn(0x7fba380b1f40 0x7fba3806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3dad9640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba240047d0 0x7fba24004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3eadb640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba24005190 0x7fba2400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3e2da640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba24003980 0x7fba24003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3dad9640 1 -- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba24005190 msgr2=0x7fba2400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3dad9640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba24005190 0x7fba2400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3dad9640 1 -- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba24003980 msgr2=0x7fba24003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3dad9640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba24003980 0x7fba24003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3dad9640 1 -- 172.21.15.31:0/3392617444 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24003530 con 0x7fba240047d0 2023-11-26T07:19:12.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3e2da640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba24003980 0x7fba24003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.195+0000 7fba3dad9640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba240047d0 0x7fba24004c50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fba28009cc0 tx=0x7fba2801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c9d2c8eb3730c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28004020 con 0x7fba240047d0 2023-11-26T07:19:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba28004180 con 0x7fba240047d0 2023-11-26T07:19:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba3d2d8640 1 -- 172.21.15.31:0/3392617444 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba280a58f0 con 0x7fba240047d0 2023-11-26T07:19:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 -- 172.21.15.31:0/3392617444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba240047d0 msgr2=0x7fba24004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 --2- 172.21.15.31:0/3392617444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba240047d0 0x7fba24004c50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fba28009cc0 tx=0x7fba2801cc30 comp rx=0 tx=0).stop 2023-11-26T07:19:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba3eadb640 1 -- 172.21.15.31:0/3392617444 reap_dead start 2023-11-26T07:19:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 -- 172.21.15.31:0/3392617444 shutdown_connections 2023-11-26T07:19:12.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 -- 172.21.15.31:0/3392617444 >> 172.21.15.31:0/3392617444 conn(0x7fba380b0c40 msgr2=0x7fba380b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 -- 172.21.15.31:0/3392617444 shutdown_connections 2023-11-26T07:19:12.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 -- 172.21.15.31:0/3392617444 wait complete. 2023-11-26T07:19:12.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 Processor -- start 2023-11-26T07:19:12.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 -- start start 2023-11-26T07:19:12.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba240047d0 0x7fba3814cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba3814d6c0 0x7fba3809be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3809c3c0 0x7fba3814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3809c840 con 0x7fba3809c3c0 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3809ca40 con 0x7fba240047d0 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.199+0000 7fba4053e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba380cb820 con 0x7fba3814d6c0 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3eadb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3809c3c0 0x7fba3814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3eadb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3809c3c0 0x7fba3814f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51766/0 (socket says 172.21.15.31:51766) 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3eadb640 1 -- 172.21.15.31:0/3320979435 learned_addr learned my addr 172.21.15.31:0/3320979435 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3e2da640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba240047d0 0x7fba3814cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3dad9640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba3814d6c0 0x7fba3809be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3eadb640 1 -- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba3814d6c0 msgr2=0x7fba3809be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3eadb640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba3814d6c0 0x7fba3809be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3eadb640 1 -- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba240047d0 msgr2=0x7fba3814cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3eadb640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba240047d0 0x7fba3814cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3eadb640 1 -- 172.21.15.31:0/3320979435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba380cbb00 con 0x7fba3809c3c0 2023-11-26T07:19:12.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba3eadb640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3809c3c0 0x7fba3814f720 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fba30002240 tx=0x7fba30012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba2effd640 1 -- 172.21.15.31:0/3320979435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3001b070 con 0x7fba3809c3c0 2023-11-26T07:19:12.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba2effd640 1 -- 172.21.15.31:0/3320979435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba30007110 con 0x7fba3809c3c0 2023-11-26T07:19:12.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba2effd640 1 -- 172.21.15.31:0/3320979435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba30016680 con 0x7fba3809c3c0 2023-11-26T07:19:12.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba4053e640 1 -- 172.21.15.31:0/3320979435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba380cbde0 con 0x7fba3809c3c0 2023-11-26T07:19:12.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba4053e640 1 -- 172.21.15.31:0/3320979435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba38150520 con 0x7fba3809c3c0 2023-11-26T07:19:12.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba4053e640 1 -- 172.21.15.31:0/3320979435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba3804e850 con 0x7fba3809c3c0 2023-11-26T07:19:12.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.203+0000 7fba2effd640 1 -- 172.21.15.31:0/3320979435 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fba300167e0 con 0x7fba3809c3c0 2023-11-26T07:19:12.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.207+0000 7fba2effd640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba1c066620 0x7fba1c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.207+0000 7fba2effd640 1 -- 172.21.15.31:0/3320979435 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba30087ac0 con 0x7fba3809c3c0 2023-11-26T07:19:12.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.207+0000 7fba3e2da640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba1c066620 0x7fba1c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.207+0000 7fba3e2da640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba1c066620 0x7fba1c068ae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba340093f0 tx=0x7fba34008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.211+0000 7fba2effd640 1 -- 172.21.15.31:0/3320979435 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fba30054e10 con 0x7fba3809c3c0 2023-11-26T07:19:12.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.367+0000 7fba4053e640 1 -- 172.21.15.31:0/3320979435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba380cc290 con 0x7fba3809c3c0 2023-11-26T07:19:12.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba2effd640 1 -- 172.21.15.31:0/3320979435 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16429 (secure 0 0 0) 0x7fba30058430 con 0x7fba3809c3c0 2023-11-26T07:19:12.376 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:12.376 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":44,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:19:03.010965+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:59.966450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:00.550730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 -- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba1c066620 msgr2=0x7fba1c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba1c066620 0x7fba1c068ae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba340093f0 tx=0x7fba34008040 comp rx=0 tx=0).stop 2023-11-26T07:19:12.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 -- 172.21.15.31:0/3320979435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3809c3c0 msgr2=0x7fba3814f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3809c3c0 0x7fba3814f720 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fba30002240 tx=0x7fba30012630 comp rx=0 tx=0).stop 2023-11-26T07:19:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 -- 172.21.15.31:0/3320979435 shutdown_connections 2023-11-26T07:19:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba1c066620 0x7fba1c068ae0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3809c3c0 0x7fba3814f720 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba3814d6c0 0x7fba3809be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 --2- 172.21.15.31:0/3320979435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba240047d0 0x7fba3814cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 -- 172.21.15.31:0/3320979435 >> 172.21.15.31:0/3320979435 conn(0x7fba380b0c40 msgr2=0x7fba380cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 -- 172.21.15.31:0/3320979435 shutdown_connections 2023-11-26T07:19:12.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.371+0000 7fba4053e640 1 -- 172.21.15.31:0/3320979435 wait complete. 2023-11-26T07:19:12.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:18:59.966450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:19:00.550730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:12.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:12.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9b1d72640 1 Processor -- start 2023-11-26T07:19:12.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9b1d72640 1 -- start start 2023-11-26T07:19:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9b1d72640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9ac0cdfb0 0x7ff9ac0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9b1d72640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac0cbd50 0x7ff9ac0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9b1d72640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac0b2710 con 0x7ff9ac15a710 2023-11-26T07:19:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9b1d72640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac0cca80 con 0x7ff9ac0cdfb0 2023-11-26T07:19:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9b1d72640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0b29f0 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9ab7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9ac0cdfb0 0x7ff9ac0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9ab7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff9ac0cdfb0 0x7ff9ac0abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51768/0 (socket says 172.21.15.31:51768) 2023-11-26T07:19:12.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9ab7fe640 1 -- 172.21.15.31:0/3937713790 learned_addr learned my addr 172.21.15.31:0/3937713790 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9abfff640 1 --2- 172.21.15.31:0/3937713790 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac0cbd50 0x7ff9ac0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9ab7fe640 1 -- 172.21.15.31:0/3937713790 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac0cbd50 msgr2=0x7ff9ac0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9ab7fe640 1 --2- 172.21.15.31:0/3937713790 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac0cbd50 0x7ff9ac0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9ab7fe640 1 -- 172.21.15.31:0/3937713790 >> v1:172.21.15.97:6789/0 conn(0x7ff9ac15a710 legacy=0x7ff9ac0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9ab7fe640 1 -- 172.21.15.31:0/3937713790 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0b2cd0 con 0x7ff9ac0cdfb0 2023-11-26T07:19:12.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.539+0000 7ff9abfff640 1 --2- 172.21.15.31:0/3937713790 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac0cbd50 0x7ff9ac0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9ab7fe640 1 --2- 172.21.15.31:0/3937713790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cdfb0 0x7ff9ac0abf90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009870 tx=0x7ff99c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e795064233bd73 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9aa7fc640 1 -- 172.21.15.31:0/3937713790 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0a95f0 con 0x7ff9ac0cdfb0 2023-11-26T07:19:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9aa7fc640 1 -- 172.21.15.31:0/3937713790 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c0a9750 con 0x7ff9ac0cdfb0 2023-11-26T07:19:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9aa7fc640 1 -- 172.21.15.31:0/3937713790 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0a9ae0 con 0x7ff9ac0cdfb0 2023-11-26T07:19:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9b1d72640 1 -- 172.21.15.31:0/3937713790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cdfb0 msgr2=0x7ff9ac0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9b1d72640 1 --2- 172.21.15.31:0/3937713790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cdfb0 0x7ff9ac0abf90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009870 tx=0x7ff99c01c510 comp rx=0 tx=0).stop 2023-11-26T07:19:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9b1d72640 1 -- 172.21.15.31:0/3937713790 shutdown_connections 2023-11-26T07:19:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9b1d72640 1 --2- 172.21.15.31:0/3937713790 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac0cbd50 0x7ff9ac0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9b1d72640 1 --2- 172.21.15.31:0/3937713790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cdfb0 0x7ff9ac0abf90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9b1d72640 1 -- 172.21.15.31:0/3937713790 >> 172.21.15.31:0/3937713790 conn(0x7ff9ac0b0cb0 msgr2=0x7ff9ac0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9b1d72640 1 -- 172.21.15.31:0/3937713790 shutdown_connections 2023-11-26T07:19:12.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.543+0000 7ff9b1d72640 1 -- 172.21.15.31:0/3937713790 wait complete. 2023-11-26T07:19:12.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 Processor -- start 2023-11-26T07:19:12.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 -- start start 2023-11-26T07:19:12.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cbd50 0x7ff9ac158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0cdfb0 0x7ff9ac0cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac15a710 0x7ff9ac163490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac157950 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac157bd0 con 0x7ff9ac0cdfb0 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac1f7b40 con 0x7ff9ac15a710 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9ab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cbd50 0x7ff9ac158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9ab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cbd50 0x7ff9ac158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51780/0 (socket says 172.21.15.31:51780) 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9ab7fe640 1 -- 172.21.15.31:0/1440307787 learned_addr learned my addr 172.21.15.31:0/1440307787 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9aaffd640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0cdfb0 0x7ff9ac0cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9abfff640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac15a710 0x7ff9ac163490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9ab7fe640 1 -- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac15a710 msgr2=0x7ff9ac163490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9ab7fe640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac15a710 0x7ff9ac163490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9ab7fe640 1 -- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0cdfb0 msgr2=0x7ff9ac0cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9ab7fe640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0cdfb0 0x7ff9ac0cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9ab7fe640 1 -- 172.21.15.31:0/1440307787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c0094d0 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9ab7fe640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cbd50 0x7ff9ac158690 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0ab040 tx=0x7ff99c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff993fff640 1 -- 172.21.15.31:0/1440307787 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0b1800 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff993fff640 1 -- 172.21.15.31:0/1440307787 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c0b1980 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 -- 172.21.15.31:0/1440307787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac157e90 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff993fff640 1 -- 172.21.15.31:0/1440307787 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0b1d30 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 -- 172.21.15.31:0/1440307787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac158bd0 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.547+0000 7ff9b1d72640 1 -- 172.21.15.31:0/1440307787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac0671f0 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.551+0000 7ff993fff640 1 -- 172.21.15.31:0/1440307787 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff99c0bc450 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.551+0000 7ff993fff640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff998066620 0x7ff998068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.551+0000 7ff993fff640 1 -- 172.21.15.31:0/1440307787 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff99c11ef30 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.551+0000 7ff9aaffd640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff998066620 0x7ff998068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.551+0000 7ff9aaffd640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff998066620 0x7ff998068ae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac15dc60 tx=0x7ff9a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.555+0000 7ff993fff640 1 -- 172.21.15.31:0/1440307787 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff99c0ebfe0 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.711+0000 7ff9b1d72640 1 -- 172.21.15.31:0/1440307787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9ac06b460 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.711+0000 7ff993fff640 1 -- 172.21.15.31:0/1440307787 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff99c0ef600 con 0x7ff9ac0cbd50 2023-11-26T07:19:12.719 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:19:12.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 -- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff998066620 msgr2=0x7ff998068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff998066620 0x7ff998068ae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac15dc60 tx=0x7ff9a0009040 comp rx=0 tx=0).stop 2023-11-26T07:19:12.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 -- 172.21.15.31:0/1440307787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cbd50 msgr2=0x7ff9ac158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cbd50 0x7ff9ac158690 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0ab040 tx=0x7ff99c01ad50 comp rx=0 tx=0).stop 2023-11-26T07:19:12.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 -- 172.21.15.31:0/1440307787 shutdown_connections 2023-11-26T07:19:12.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff998066620 0x7ff998068ae0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff9ac15a710 0x7ff9ac163490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9ac0cdfb0 0x7ff9ac0cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 --2- 172.21.15.31:0/1440307787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9ac0cbd50 0x7ff9ac158690 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 -- 172.21.15.31:0/1440307787 >> 172.21.15.31:0/1440307787 conn(0x7ff9ac0b0cb0 msgr2=0x7ff9ac0ad170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 -- 172.21.15.31:0/1440307787 shutdown_connections 2023-11-26T07:19:12.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.715+0000 7ff9b1d72640 1 -- 172.21.15.31:0/1440307787 wait complete. 2023-11-26T07:19:12.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:19:12.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd57409a640 1 Processor -- start 2023-11-26T07:19:12.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd57409a640 1 -- start start 2023-11-26T07:19:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd57409a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0b5830 0x7fd56c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd57409a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd56c0ad8e0 0x7fd56c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd57409a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd56c0af630 con 0x7fd56c0acd70 2023-11-26T07:19:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd57409a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56c0acb10 con 0x7fd56c0ad8e0 2023-11-26T07:19:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd57409a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0ae5b0 con 0x7fd56c0b5830 2023-11-26T07:19:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd572637640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd56c0ad8e0 0x7fd56c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd571635640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd56c0acd70 0x7fd56c0ad1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52516/0 (socket says 172.21.15.31:52516) 2023-11-26T07:19:12.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd571635640 1 -- 172.21.15.31:0/116453755 learned_addr learned my addr 172.21.15.31:0/116453755 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd571e36640 1 --2- 172.21.15.31:0/116453755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0b5830 0x7fd56c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd572637640 1 -- 172.21.15.31:0/116453755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0b5830 msgr2=0x7fd56c15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd572637640 1 --2- 172.21.15.31:0/116453755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0b5830 0x7fd56c15e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd572637640 1 -- 172.21.15.31:0/116453755 >> v1:172.21.15.31:6789/0 conn(0x7fd56c0acd70 legacy=0x7fd56c0ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd572637640 1 -- 172.21.15.31:0/116453755 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c0ae870 con 0x7fd56c0ad8e0 2023-11-26T07:19:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.879+0000 7fd570e34640 1 -- 172.21.15.31:0/116453755 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917623517 0 0) 0x7fd55c001120 con 0x7fd56c0acd70 2023-11-26T07:19:12.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd571e36640 1 --2- 172.21.15.31:0/116453755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0b5830 0x7fd56c15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd572637640 1 --2- 172.21.15.31:0/116453755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0ad8e0 0x7fd56c06d190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd564009d90 tx=0x7fd56401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9cbad576eafcf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd570e34640 1 -- 172.21.15.31:0/116453755 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640a5a10 con 0x7fd56c0ad8e0 2023-11-26T07:19:12.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd570e34640 1 -- 172.21.15.31:0/116453755 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5640a5b70 con 0x7fd56c0ad8e0 2023-11-26T07:19:12.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd570e34640 1 -- 172.21.15.31:0/116453755 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640ad440 con 0x7fd56c0ad8e0 2023-11-26T07:19:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd57409a640 1 -- 172.21.15.31:0/116453755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0ad8e0 msgr2=0x7fd56c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd57409a640 1 --2- 172.21.15.31:0/116453755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0ad8e0 0x7fd56c06d190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd564009d90 tx=0x7fd56401c950 comp rx=0 tx=0).stop 2023-11-26T07:19:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd57409a640 1 -- 172.21.15.31:0/116453755 shutdown_connections 2023-11-26T07:19:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd57409a640 1 --2- 172.21.15.31:0/116453755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0ad8e0 0x7fd56c06d190 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd57409a640 1 --2- 172.21.15.31:0/116453755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0b5830 0x7fd56c15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd57409a640 1 -- 172.21.15.31:0/116453755 >> 172.21.15.31:0/116453755 conn(0x7fd56c071d80 msgr2=0x7fd56c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd57409a640 1 -- 172.21.15.31:0/116453755 shutdown_connections 2023-11-26T07:19:12.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.883+0000 7fd57409a640 1 -- 172.21.15.31:0/116453755 wait complete. 2023-11-26T07:19:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd57409a640 1 Processor -- start 2023-11-26T07:19:12.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd57409a640 1 -- start start 2023-11-26T07:19:12.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd57409a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0acd70 0x7fd56c158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd57409a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c0ad8e0 0x7fd56c1527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd57409a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0b5830 0x7fd56c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd57409a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c153600 con 0x7fd56c0b5830 2023-11-26T07:19:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd57409a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c152cf0 con 0x7fd56c0ad8e0 2023-11-26T07:19:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd57409a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c152f80 con 0x7fd56c0acd70 2023-11-26T07:19:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd572637640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0b5830 0x7fd56c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd572637640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0b5830 0x7fd56c153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51792/0 (socket says 172.21.15.31:51792) 2023-11-26T07:19:12.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd572637640 1 -- 172.21.15.31:0/1146680099 learned_addr learned my addr 172.21.15.31:0/1146680099 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd571635640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c0ad8e0 0x7fd56c1527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd571e36640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0acd70 0x7fd56c158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd572637640 1 -- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0acd70 msgr2=0x7fd56c158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd572637640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0acd70 0x7fd56c158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd572637640 1 -- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c0ad8e0 msgr2=0x7fd56c1527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd572637640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c0ad8e0 0x7fd56c1527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd572637640 1 -- 172.21.15.31:0/1146680099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564009a70 con 0x7fd56c0b5830 2023-11-26T07:19:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd572637640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0b5830 0x7fd56c153a20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd564009d90 tx=0x7fd5640196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd5627fc640 1 -- 172.21.15.31:0/1146680099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640ade40 con 0x7fd56c0b5830 2023-11-26T07:19:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd5627fc640 1 -- 172.21.15.31:0/1146680099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5640b9450 con 0x7fd56c0b5830 2023-11-26T07:19:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd5627fc640 1 -- 172.21.15.31:0/1146680099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640b9800 con 0x7fd56c0b5830 2023-11-26T07:19:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd57409a640 1 -- 172.21.15.31:0/1146680099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c159370 con 0x7fd56c0b5830 2023-11-26T07:19:12.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.887+0000 7fd57409a640 1 -- 172.21.15.31:0/1146680099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c154b50 con 0x7fd56c0b5830 2023-11-26T07:19:12.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.891+0000 7fd57409a640 1 -- 172.21.15.31:0/1146680099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56c153740 con 0x7fd56c0b5830 2023-11-26T07:19:12.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.891+0000 7fd5627fc640 1 -- 172.21.15.31:0/1146680099 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd5640b9960 con 0x7fd56c0b5830 2023-11-26T07:19:12.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.891+0000 7fd5627fc640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd550066620 0x7fd550068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.891+0000 7fd5627fc640 1 -- 172.21.15.31:0/1146680099 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd56411fa80 con 0x7fd56c0b5830 2023-11-26T07:19:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.891+0000 7fd571e36640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd550066620 0x7fd550068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:12.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.891+0000 7fd571e36640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd550066620 0x7fd550068ae0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd56c156db0 tx=0x7fd568008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:12.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:12.895+0000 7fd5627fc640 1 -- 172.21.15.31:0/1146680099 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd56c153740 con 0x7fd56c0b5830 2023-11-26T07:19:13.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 -- 172.21.15.31:0/1146680099 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd56c153740 con 0x7fd56c0b5830 2023-11-26T07:19:13.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd5627fc640 1 -- 172.21.15.31:0/1146680099 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd56c153740 con 0x7fd56c0b5830 2023-11-26T07:19:13.061 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:19:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 -- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd550066620 msgr2=0x7fd550068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd550066620 0x7fd550068ae0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd56c156db0 tx=0x7fd568008040 comp rx=0 tx=0).stop 2023-11-26T07:19:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 -- 172.21.15.31:0/1146680099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0b5830 msgr2=0x7fd56c153a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0b5830 0x7fd56c153a20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd564009d90 tx=0x7fd5640196f0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 -- 172.21.15.31:0/1146680099 shutdown_connections 2023-11-26T07:19:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd550066620 0x7fd550068ae0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd56c0b5830 0x7fd56c153a20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd56c0ad8e0 0x7fd56c1527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 --2- 172.21.15.31:0/1146680099 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd56c0acd70 0x7fd56c158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 -- 172.21.15.31:0/1146680099 >> 172.21.15.31:0/1146680099 conn(0x7fd56c071d80 msgr2=0x7fd56c15fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 -- 172.21.15.31:0/1146680099 shutdown_connections 2023-11-26T07:19:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.055+0000 7fd57409a640 1 -- 172.21.15.31:0/1146680099 wait complete. 2023-11-26T07:19:13.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:19:13.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2359e640 1 Processor -- start 2023-11-26T07:19:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2359e640 1 -- start start 2023-11-26T07:19:13.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2359e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a2415d550 0x7f8a24161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2359e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a240d0190 0x7f8a240cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2359e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a24161ff0 con 0x7f8a240d3a00 2023-11-26T07:19:13.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2359e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a240d1210 con 0x7f8a240d0190 2023-11-26T07:19:13.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2359e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240d4230 con 0x7f8a2415d550 2023-11-26T07:19:13.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2259c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a2415d550 0x7f8a24161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a22d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a240d0190 0x7f8a240cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a21d9b640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f8a240d3a00 0x7f8a240cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:47274/0 (socket says 172.21.15.31:47274) 2023-11-26T07:19:13.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a22d9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8a240d0190 0x7f8a240cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51460/0 (socket says 172.21.15.31:51460) 2023-11-26T07:19:13.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a21d9b640 1 -- 172.21.15.31:0/1294181548 learned_addr learned my addr 172.21.15.31:0/1294181548 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:13.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2159a640 1 -- 172.21.15.31:0/1294181548 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757637451 0 0) 0x7f8a24161ff0 con 0x7f8a240d3a00 2023-11-26T07:19:13.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2159a640 1 -- 172.21.15.31:0/1294181548 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a0c003530 con 0x7f8a240d3a00 2023-11-26T07:19:13.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a22d9d640 1 -- 172.21.15.31:0/1294181548 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a2415d550 msgr2=0x7f8a24161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:13.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a22d9d640 1 --2- 172.21.15.31:0/1294181548 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a2415d550 0x7f8a24161ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a22d9d640 1 -- 172.21.15.31:0/1294181548 >> v1:172.21.15.97:6789/0 conn(0x7f8a240d3a00 legacy=0x7f8a240cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a22d9d640 1 -- 172.21.15.31:0/1294181548 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2415dd00 con 0x7f8a240d0190 2023-11-26T07:19:13.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.223+0000 7f8a2259c640 1 --2- 172.21.15.31:0/1294181548 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a2415d550 0x7f8a24161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:13.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a22d9d640 1 --2- 172.21.15.31:0/1294181548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a240d0190 0x7f8a240cddc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009dc0 tx=0x7f8a1c01f120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316566872ac76dfd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2159a640 1 -- 172.21.15.31:0/1294181548 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0af600 con 0x7f8a240d0190 2023-11-26T07:19:13.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2159a640 1 -- 172.21.15.31:0/1294181548 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0ab050 con 0x7f8a240d0190 2023-11-26T07:19:13.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2159a640 1 -- 172.21.15.31:0/1294181548 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0af980 con 0x7f8a240d0190 2023-11-26T07:19:13.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2359e640 1 -- 172.21.15.31:0/1294181548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a240d0190 msgr2=0x7f8a240cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2359e640 1 --2- 172.21.15.31:0/1294181548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a240d0190 0x7f8a240cddc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c009dc0 tx=0x7f8a1c01f120 comp rx=0 tx=0).stop 2023-11-26T07:19:13.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2359e640 1 -- 172.21.15.31:0/1294181548 shutdown_connections 2023-11-26T07:19:13.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2359e640 1 --2- 172.21.15.31:0/1294181548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a240d0190 0x7f8a240cddc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2359e640 1 --2- 172.21.15.31:0/1294181548 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a2415d550 0x7f8a24161ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2359e640 1 -- 172.21.15.31:0/1294181548 >> 172.21.15.31:0/1294181548 conn(0x7f8a2409c670 msgr2=0x7f8a240d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:13.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2359e640 1 -- 172.21.15.31:0/1294181548 shutdown_connections 2023-11-26T07:19:13.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.227+0000 7f8a2359e640 1 -- 172.21.15.31:0/1294181548 wait complete. 2023-11-26T07:19:13.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2359e640 1 Processor -- start 2023-11-26T07:19:13.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2359e640 1 -- start start 2023-11-26T07:19:13.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a240d0190 0x7f8a2415a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2359e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a240d3a00 0x7f8a240cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a2415d550 0x7f8a240b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2359e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240b7940 con 0x7f8a240d0190 2023-11-26T07:19:13.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2359e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240b8980 con 0x7f8a2415d550 2023-11-26T07:19:13.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2359e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2415ac30 con 0x7f8a240d3a00 2023-11-26T07:19:13.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a240d0190 0x7f8a2415a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a240d0190 0x7f8a2415a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51794/0 (socket says 172.21.15.31:51794) 2023-11-26T07:19:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2259c640 1 -- 172.21.15.31:0/1243443194 learned_addr learned my addr 172.21.15.31:0/1243443194 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a21d9b640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a240d3a00 0x7f8a240cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a22d9d640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a2415d550 0x7f8a240b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2259c640 1 -- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a240d3a00 msgr2=0x7f8a240cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2259c640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a240d3a00 0x7f8a240cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2259c640 1 -- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a2415d550 msgr2=0x7f8a240b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2259c640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a2415d550 0x7f8a240b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2259c640 1 -- 172.21.15.31:0/1243443194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a14009420 con 0x7f8a240d0190 2023-11-26T07:19:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a2259c640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a240d0190 0x7f8a2415a380 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e980 tx=0x7f8a1400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a12ffd640 1 -- 172.21.15.31:0/1243443194 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400ca00 con 0x7f8a240d0190 2023-11-26T07:19:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a12ffd640 1 -- 172.21.15.31:0/1243443194 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1400cb60 con 0x7f8a240d0190 2023-11-26T07:19:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.235+0000 7f8a2359e640 1 -- 172.21.15.31:0/1243443194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1c009a70 con 0x7f8a240d0190 2023-11-26T07:19:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.231+0000 7f8a12ffd640 1 -- 172.21.15.31:0/1243443194 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14013470 con 0x7f8a240d0190 2023-11-26T07:19:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.235+0000 7f8a2359e640 1 -- 172.21.15.31:0/1243443194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a240b8750 con 0x7f8a240d0190 2023-11-26T07:19:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.235+0000 7f8a2359e640 1 -- 172.21.15.31:0/1243443194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a24074990 con 0x7f8a240d0190 2023-11-26T07:19:13.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.235+0000 7f8a12ffd640 1 -- 172.21.15.31:0/1243443194 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f8a140135d0 con 0x7f8a240d0190 2023-11-26T07:19:13.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.235+0000 7f8a12ffd640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a04066620 0x7f8a04068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.235+0000 7f8a12ffd640 1 -- 172.21.15.31:0/1243443194 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a14084360 con 0x7f8a240d0190 2023-11-26T07:19:13.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.235+0000 7f8a21d9b640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a04066620 0x7f8a04068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.239+0000 7f8a21d9b640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a04066620 0x7f8a04068ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8a2406b7b0 tx=0x7f8a18007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.239+0000 7f8a12ffd640 1 -- 172.21.15.31:0/1243443194 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a14051630 con 0x7f8a240d0190 2023-11-26T07:19:13.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.399+0000 7f8a2359e640 1 -- 172.21.15.31:0/1243443194 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a240cd720 con 0x7f8a240d0190 2023-11-26T07:19:13.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.399+0000 7f8a12ffd640 1 -- 172.21.15.31:0/1243443194 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a14054c50 con 0x7f8a240d0190 2023-11-26T07:19:13.406 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:19:13.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 -- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a04066620 msgr2=0x7f8a04068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a04066620 0x7f8a04068ae0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8a2406b7b0 tx=0x7f8a18007d40 comp rx=0 tx=0).stop 2023-11-26T07:19:13.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 -- 172.21.15.31:0/1243443194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a240d0190 msgr2=0x7f8a2415a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a240d0190 0x7f8a2415a380 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400e980 tx=0x7f8a1400ee40 comp rx=0 tx=0).stop 2023-11-26T07:19:13.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 -- 172.21.15.31:0/1243443194 shutdown_connections 2023-11-26T07:19:13.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a04066620 0x7f8a04068ae0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a2415d550 0x7f8a240b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a240d3a00 0x7f8a240cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 --2- 172.21.15.31:0/1243443194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a240d0190 0x7f8a2415a380 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 -- 172.21.15.31:0/1243443194 >> 172.21.15.31:0/1243443194 conn(0x7f8a2409c670 msgr2=0x7f8a2406cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:13.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 -- 172.21.15.31:0/1243443194 shutdown_connections 2023-11-26T07:19:13.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.403+0000 7f8a2359e640 1 -- 172.21.15.31:0/1243443194 wait complete. 2023-11-26T07:19:13.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:19:13.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0bb0faa640 1 Processor -- start 2023-11-26T07:19:13.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0bb0faa640 1 -- start start 2023-11-26T07:19:13.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0bb0faa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0bac0cc460 0x7f0bac0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0bb0faa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bac0b2710 con 0x7f0bac0cdfb0 2023-11-26T07:19:13.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0bb0faa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bac0b29f0 con 0x7f0bac15a710 2023-11-26T07:19:13.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0bb0faa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bac0cca80 con 0x7f0bac0cc460 2023-11-26T07:19:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0baa59c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0bac0cdfb0 0x7f0bac0ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52528/0 (socket says 172.21.15.31:52528) 2023-11-26T07:19:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0baa59c640 1 -- 172.21.15.31:0/2923335560 learned_addr learned my addr 172.21.15.31:0/2923335560 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:13.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0baad9d640 1 --2- 172.21.15.31:0/2923335560 >> v2:172.21.15.97:3300/0 conn(0x7f0bac0cc460 0x7f0bac0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0ba9d9b640 1 -- 172.21.15.31:0/2923335560 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968455064 0 0) 0x7f0bac0b2710 con 0x7f0bac0cdfb0 2023-11-26T07:19:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0ba9d9b640 1 -- 172.21.15.31:0/2923335560 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b98003530 con 0x7f0bac0cdfb0 2023-11-26T07:19:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0baad9d640 1 -- 172.21.15.31:0/2923335560 >> v1:172.21.15.97:6789/0 conn(0x7f0bac15a710 legacy=0x7f0bac0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0baad9d640 1 -- 172.21.15.31:0/2923335560 >> v1:172.21.15.31:6789/0 conn(0x7f0bac0cdfb0 legacy=0x7f0bac0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.567+0000 7f0baad9d640 1 -- 172.21.15.31:0/2923335560 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac0b2cd0 con 0x7f0bac0cc460 2023-11-26T07:19:13.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0baad9d640 1 --2- 172.21.15.31:0/2923335560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac0cc460 0x7f0bac0abf90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009d90 tx=0x7f0ba401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=624879a47f34041c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0ba9d9b640 1 -- 172.21.15.31:0/2923335560 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40a5a00 con 0x7f0bac0cc460 2023-11-26T07:19:13.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0ba9d9b640 1 -- 172.21.15.31:0/2923335560 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba40a5b60 con 0x7f0bac0cc460 2023-11-26T07:19:13.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0ba9d9b640 1 -- 172.21.15.31:0/2923335560 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40ad440 con 0x7f0bac0cc460 2023-11-26T07:19:13.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0bb0faa640 1 -- 172.21.15.31:0/2923335560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac0cc460 msgr2=0x7f0bac0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0bb0faa640 1 --2- 172.21.15.31:0/2923335560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac0cc460 0x7f0bac0abf90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009d90 tx=0x7f0ba401c920 comp rx=0 tx=0).stop 2023-11-26T07:19:13.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0bb0faa640 1 -- 172.21.15.31:0/2923335560 shutdown_connections 2023-11-26T07:19:13.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0bb0faa640 1 --2- 172.21.15.31:0/2923335560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac0cc460 0x7f0bac0abf90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0bb0faa640 1 -- 172.21.15.31:0/2923335560 >> 172.21.15.31:0/2923335560 conn(0x7f0bac0b0cb0 msgr2=0x7f0bac0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:13.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0bb0faa640 1 -- 172.21.15.31:0/2923335560 shutdown_connections 2023-11-26T07:19:13.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.571+0000 7f0bb0faa640 1 -- 172.21.15.31:0/2923335560 wait complete. 2023-11-26T07:19:13.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 Processor -- start 2023-11-26T07:19:13.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 -- start start 2023-11-26T07:19:13.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac0cc460 0x7f0bac15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bac0cdfb0 0x7f0bac09c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac15a710 0x7f0bac15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac09ced0 con 0x7f0bac15a710 2023-11-26T07:19:13.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac1f6630 con 0x7f0bac0cc460 2023-11-26T07:19:13.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac1f5a40 con 0x7f0bac0cdfb0 2023-11-26T07:19:13.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0baad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac15a710 0x7f0bac15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0baad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac15a710 0x7f0bac15cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51796/0 (socket says 172.21.15.31:51796) 2023-11-26T07:19:13.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0baad9d640 1 -- 172.21.15.31:0/1354491004 learned_addr learned my addr 172.21.15.31:0/1354491004 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:13.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0baa59c640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac0cc460 0x7f0bac15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0ba3fff640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bac0cdfb0 0x7f0bac09c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0baad9d640 1 -- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bac0cdfb0 msgr2=0x7f0bac09c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0baad9d640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bac0cdfb0 0x7f0bac09c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0baad9d640 1 -- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac0cc460 msgr2=0x7f0bac15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0baad9d640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac0cc460 0x7f0bac15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0baad9d640 1 -- 172.21.15.31:0/1354491004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba4009a70 con 0x7f0bac15a710 2023-11-26T07:19:13.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0ba3fff640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bac0cdfb0 0x7f0bac09c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:13.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0baad9d640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac15a710 0x7f0bac15cfd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40b0040 tx=0x7f0ba40196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0ba2ffd640 1 -- 172.21.15.31:0/1354491004 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40a5a00 con 0x7f0bac15a710 2023-11-26T07:19:13.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0ba2ffd640 1 -- 172.21.15.31:0/1354491004 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4009450 con 0x7f0bac15a710 2023-11-26T07:19:13.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0ba2ffd640 1 -- 172.21.15.31:0/1354491004 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40b9690 con 0x7f0bac15a710 2023-11-26T07:19:13.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 -- 172.21.15.31:0/1354491004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac1f5d00 con 0x7f0bac15a710 2023-11-26T07:19:13.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 -- 172.21.15.31:0/1354491004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac1f62c0 con 0x7f0bac15a710 2023-11-26T07:19:13.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.575+0000 7f0bb0faa640 1 -- 172.21.15.31:0/1354491004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bac0671f0 con 0x7f0bac15a710 2023-11-26T07:19:13.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.579+0000 7f0ba2ffd640 1 -- 172.21.15.31:0/1354491004 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f0ba40b97f0 con 0x7f0bac15a710 2023-11-26T07:19:13.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.579+0000 7f0ba2ffd640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b8c066650 0x7f0b8c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:13.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.579+0000 7f0ba2ffd640 1 -- 172.21.15.31:0/1354491004 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ba411fb30 con 0x7f0bac15a710 2023-11-26T07:19:13.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.579+0000 7f0baa59c640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b8c066650 0x7f0b8c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:13.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.579+0000 7f0baa59c640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b8c066650 0x7f0b8c068b10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0011f0 tx=0x7f0b9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:13.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.583+0000 7f0ba2ffd640 1 -- 172.21.15.31:0/1354491004 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0ba40ecc90 con 0x7f0bac15a710 2023-11-26T07:19:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 -- 172.21.15.31:0/1354491004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0bac1f67c0 con 0x7f0bac15a710 2023-11-26T07:19:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0ba2ffd640 1 -- 172.21.15.31:0/1354491004 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v44) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0ba40b2080 con 0x7f0bac15a710 2023-11-26T07:19:13.757 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:19:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 -- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b8c066650 msgr2=0x7f0b8c068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b8c066650 0x7f0b8c068b10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0011f0 tx=0x7f0b9c009040 comp rx=0 tx=0).stop 2023-11-26T07:19:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 -- 172.21.15.31:0/1354491004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac15a710 msgr2=0x7f0bac15cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:13.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac15a710 0x7f0bac15cfd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40b0040 tx=0x7f0ba40196c0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 -- 172.21.15.31:0/1354491004 shutdown_connections 2023-11-26T07:19:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0b8c066650 0x7f0b8c068b10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0bac15a710 0x7f0bac15cfd0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0bac0cdfb0 0x7f0bac09c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 --2- 172.21.15.31:0/1354491004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0bac0cc460 0x7f0bac15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 -- 172.21.15.31:0/1354491004 >> 172.21.15.31:0/1354491004 conn(0x7f0bac0b0cb0 msgr2=0x7f0bac0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 -- 172.21.15.31:0/1354491004 shutdown_connections 2023-11-26T07:19:13.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:13.751+0000 7f0bb0faa640 1 -- 172.21.15.31:0/1354491004 wait complete. 2023-11-26T07:19:13.779 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:19:13.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:13.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:19:13.806 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:19:13.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:13.807 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:19:13.856 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:19:13.856 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:13.866 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:19:13.867 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:19:13.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:19:14.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40ba4ea640 1 Processor -- start 2023-11-26T07:19:14.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40ba4ea640 1 -- start start 2023-11-26T07:19:14.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40ba4ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cead0 0x7f40b40d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40ba4ea640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40b40d1eb0 0x7f40b406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40ba4ea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b40cefd0 con 0x7f40b415d7b0 2023-11-26T07:19:14.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40ba4ea640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b40d17e0 con 0x7f40b40d1eb0 2023-11-26T07:19:14.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40ba4ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40d2340 con 0x7f40b40cead0 2023-11-26T07:19:14.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40b8a87640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40b40d1eb0 0x7f40b406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40b8a87640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f40b40d1eb0 0x7f40b406bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40724/0 (socket says 172.21.15.31:40724) 2023-11-26T07:19:14.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40b8a87640 1 -- 172.21.15.31:0/3863210484 learned_addr learned my addr 172.21.15.31:0/3863210484 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:14.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40b37fe640 1 --2- 172.21.15.31:0/3863210484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cead0 0x7f40b40d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40b8a87640 1 -- 172.21.15.31:0/3863210484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cead0 msgr2=0x7f40b40d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:14.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40b8a87640 1 --2- 172.21.15.31:0/3863210484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cead0 0x7f40b40d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40b8a87640 1 -- 172.21.15.31:0/3863210484 >> v1:172.21.15.97:6789/0 conn(0x7f40b415d7b0 legacy=0x7f40b40d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.051+0000 7f40b8a87640 1 -- 172.21.15.31:0/3863210484 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b40cf2b0 con 0x7f40b40d1eb0 2023-11-26T07:19:14.061 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40b8a87640 1 --2- 172.21.15.31:0/3863210484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 0x7f40b406bed0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009e50 tx=0x7f40ac01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfbc8d9ce92bd18 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40b2ffd640 1 -- 172.21.15.31:0/3863210484 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0a5780 con 0x7f40b40d1eb0 2023-11-26T07:19:14.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40b2ffd640 1 -- 172.21.15.31:0/3863210484 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac0a58e0 con 0x7f40b40d1eb0 2023-11-26T07:19:14.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40b2ffd640 1 -- 172.21.15.31:0/3863210484 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0a5c70 con 0x7f40b40d1eb0 2023-11-26T07:19:14.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40ba4ea640 1 -- 172.21.15.31:0/3863210484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 msgr2=0x7f40b406bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40ba4ea640 1 --2- 172.21.15.31:0/3863210484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 0x7f40b406bed0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009e50 tx=0x7f40ac01c8c0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40ba4ea640 1 -- 172.21.15.31:0/3863210484 shutdown_connections 2023-11-26T07:19:14.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40ba4ea640 1 --2- 172.21.15.31:0/3863210484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 0x7f40b406bed0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40ba4ea640 1 --2- 172.21.15.31:0/3863210484 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cead0 0x7f40b40d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40ba4ea640 1 -- 172.21.15.31:0/3863210484 >> 172.21.15.31:0/3863210484 conn(0x7f40b40d33b0 msgr2=0x7f40b40d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40ba4ea640 1 -- 172.21.15.31:0/3863210484 shutdown_connections 2023-11-26T07:19:14.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.055+0000 7f40ba4ea640 1 -- 172.21.15.31:0/3863210484 wait complete. 2023-11-26T07:19:14.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40ba4ea640 1 Processor -- start 2023-11-26T07:19:14.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40ba4ea640 1 -- start start 2023-11-26T07:19:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40ba4ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cead0 0x7f40b40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40ba4ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 0x7f40b41539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40ba4ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b415d7b0 0x7f40b40c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40ba4ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40b66e0 con 0x7f40b40d1eb0 2023-11-26T07:19:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40ba4ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b415a830 con 0x7f40b415d7b0 2023-11-26T07:19:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40ba4ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b415aaf0 con 0x7f40b40cead0 2023-11-26T07:19:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 0x7f40b41539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 0x7f40b41539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40738/0 (socket says 172.21.15.31:40738) 2023-11-26T07:19:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b37fe640 1 -- 172.21.15.31:0/1523000381 learned_addr learned my addr 172.21.15.31:0/1523000381 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b3fff640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cead0 0x7f40b40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b8a87640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b415d7b0 0x7f40b40c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b37fe640 1 -- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cead0 msgr2=0x7f40b40b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b37fe640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cead0 0x7f40b40b61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b37fe640 1 -- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b415d7b0 msgr2=0x7f40b40c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b37fe640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b415d7b0 0x7f40b40c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b37fe640 1 -- 172.21.15.31:0/1523000381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac009b30 con 0x7f40b40d1eb0 2023-11-26T07:19:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b37fe640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 0x7f40b41539a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f40a800d800 tx=0x7f40a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b0ff9640 1 -- 172.21.15.31:0/1523000381 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a800b6c0 con 0x7f40b40d1eb0 2023-11-26T07:19:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b0ff9640 1 -- 172.21.15.31:0/1523000381 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a800b860 con 0x7f40b40d1eb0 2023-11-26T07:19:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40b0ff9640 1 -- 172.21.15.31:0/1523000381 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a800bc10 con 0x7f40b40d1eb0 2023-11-26T07:19:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40ba4ea640 1 -- 172.21.15.31:0/1523000381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b415add0 con 0x7f40b40d1eb0 2023-11-26T07:19:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.059+0000 7f40ba4ea640 1 -- 172.21.15.31:0/1523000381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b4153ee0 con 0x7f40b40d1eb0 2023-11-26T07:19:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.063+0000 7f40ba4ea640 1 -- 172.21.15.31:0/1523000381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b40d2bf0 con 0x7f40b40d1eb0 2023-11-26T07:19:14.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.063+0000 7f40b0ff9640 1 -- 172.21.15.31:0/1523000381 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f40a801e460 con 0x7f40b40d1eb0 2023-11-26T07:19:14.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.063+0000 7f40b0ff9640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f40a0066620 0x7f40a0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.063+0000 7f40b0ff9640 1 -- 172.21.15.31:0/1523000381 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40a8084630 con 0x7f40b40d1eb0 2023-11-26T07:19:14.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.063+0000 7f40b3fff640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f40a0066620 0x7f40a0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.063+0000 7f40b3fff640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f40a0066620 0x7f40a0068ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40a4007fd0 tx=0x7f40a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.067+0000 7f40b0ff9640 1 -- 172.21.15.31:0/1523000381 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f40a80518d0 con 0x7f40b40d1eb0 2023-11-26T07:19:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.227+0000 7f40ba4ea640 1 -- 172.21.15.31:0/1523000381 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40b404e850 con 0x7f40b40d1eb0 2023-11-26T07:19:14.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40b0ff9640 1 -- 172.21.15.31:0/1523000381 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f40a8054ef0 con 0x7f40b40d1eb0 2023-11-26T07:19:14.237 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:14.237 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":45,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:19:14.223374+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:59.966450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:00.550730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.18","mappings":[{"from":0,"to":5}]},{"pgid":"7.2d","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:19:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 -- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f40a0066620 msgr2=0x7f40a0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f40a0066620 0x7f40a0068ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40a4007fd0 tx=0x7f40a4009040 comp rx=0 tx=0).stop 2023-11-26T07:19:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 -- 172.21.15.31:0/1523000381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 msgr2=0x7f40b41539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 0x7f40b41539a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f40a800d800 tx=0x7f40a800dcc0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 -- 172.21.15.31:0/1523000381 shutdown_connections 2023-11-26T07:19:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f40a0066620 0x7f40a0068ae0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40b415d7b0 0x7f40b40c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40b40d1eb0 0x7f40b41539a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 --2- 172.21.15.31:0/1523000381 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40b40cead0 0x7f40b40b61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 -- 172.21.15.31:0/1523000381 >> 172.21.15.31:0/1523000381 conn(0x7f40b40d33b0 msgr2=0x7f40b40cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 -- 172.21.15.31:0/1523000381 shutdown_connections 2023-11-26T07:19:14.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.231+0000 7f40ba4ea640 1 -- 172.21.15.31:0/1523000381 wait complete. 2023-11-26T07:19:14.258 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:18:59.966450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:19:00.550730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:19:14.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:19:14.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6ec31c640 1 Processor -- start 2023-11-26T07:19:14.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6ec31c640 1 -- start start 2023-11-26T07:19:14.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6ec31c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6e40d0510 0x7ff6e40d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6ec31c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e40cdfb0 con 0x7ff6e40ced30 2023-11-26T07:19:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6ec31c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e40d4140 con 0x7ff6e415d8f0 2023-11-26T07:19:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6ec31c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e40d4420 con 0x7ff6e40d0510 2023-11-26T07:19:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6ea8b9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6e40d0510 0x7ff6e40d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6e98b7640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff6e40ced30 0x7ff6e40cfe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36226/0 (socket says 172.21.15.31:36226) 2023-11-26T07:19:14.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6e98b7640 1 -- 172.21.15.31:0/3821982128 learned_addr learned my addr 172.21.15.31:0/3821982128 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6ea8b9640 1 -- 172.21.15.31:0/3821982128 >> v1:172.21.15.97:6789/0 conn(0x7ff6e415d8f0 legacy=0x7ff6e415e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6ea8b9640 1 -- 172.21.15.31:0/3821982128 >> v1:172.21.15.31:6789/0 conn(0x7ff6e40ced30 legacy=0x7ff6e40cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.399+0000 7ff6ea8b9640 1 -- 172.21.15.31:0/3821982128 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40ce250 con 0x7ff6e40d0510 2023-11-26T07:19:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6ea8b9640 1 --2- 172.21.15.31:0/3821982128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40d0510 0x7ff6e40d1000 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009d90 tx=0x7ff6dc01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee52b8823857a22c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6e90b6640 1 -- 172.21.15.31:0/3821982128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0a58f0 con 0x7ff6e40d0510 2023-11-26T07:19:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6e90b6640 1 -- 172.21.15.31:0/3821982128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc0a5a50 con 0x7ff6e40d0510 2023-11-26T07:19:14.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6e90b6640 1 -- 172.21.15.31:0/3821982128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0a5de0 con 0x7ff6e40d0510 2023-11-26T07:19:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6ec31c640 1 -- 172.21.15.31:0/3821982128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40d0510 msgr2=0x7ff6e40d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6ec31c640 1 --2- 172.21.15.31:0/3821982128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40d0510 0x7ff6e40d1000 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009d90 tx=0x7ff6dc01c920 comp rx=0 tx=0).stop 2023-11-26T07:19:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6ec31c640 1 -- 172.21.15.31:0/3821982128 shutdown_connections 2023-11-26T07:19:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6ec31c640 1 --2- 172.21.15.31:0/3821982128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40d0510 0x7ff6e40d1000 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6ec31c640 1 -- 172.21.15.31:0/3821982128 >> 172.21.15.31:0/3821982128 conn(0x7ff6e409c750 msgr2=0x7ff6e40d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6ec31c640 1 -- 172.21.15.31:0/3821982128 shutdown_connections 2023-11-26T07:19:14.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.403+0000 7ff6ec31c640 1 -- 172.21.15.31:0/3821982128 wait complete. 2023-11-26T07:19:14.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ec31c640 1 Processor -- start 2023-11-26T07:19:14.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ec31c640 1 -- start start 2023-11-26T07:19:14.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ec31c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40ced30 0x7ff6e4160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ec31c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e40d0510 0x7ff6e4152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ec31c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6e415d8f0 0x7ff6e4152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ec31c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e415e740 con 0x7ff6e40ced30 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ec31c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4158ff0 con 0x7ff6e40d0510 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ec31c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e41592d0 con 0x7ff6e415d8f0 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea0b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40ced30 0x7ff6e4160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea0b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40ced30 0x7ff6e4160e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40766/0 (socket says 172.21.15.31:40766) 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea0b8640 1 -- 172.21.15.31:0/1969283089 learned_addr learned my addr 172.21.15.31:0/1969283089 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6e98b7640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e40d0510 0x7ff6e4152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea8b9640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6e415d8f0 0x7ff6e4152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea0b8640 1 -- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6e415d8f0 msgr2=0x7ff6e4152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea0b8640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6e415d8f0 0x7ff6e4152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea0b8640 1 -- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e40d0510 msgr2=0x7ff6e4152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea0b8640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e40d0510 0x7ff6e4152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea0b8640 1 -- 172.21.15.31:0/1969283089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc009a70 con 0x7ff6e40ced30 2023-11-26T07:19:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6e98b7640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e40d0510 0x7ff6e4152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea8b9640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6e415d8f0 0x7ff6e4152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:14.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ea0b8640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40ced30 0x7ff6e4160e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000c3c0 tx=0x7ff6e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6daffd640 1 -- 172.21.15.31:0/1969283089 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00052d0 con 0x7ff6e40ced30 2023-11-26T07:19:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6daffd640 1 -- 172.21.15.31:0/1969283089 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e0005430 con 0x7ff6e40ced30 2023-11-26T07:19:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6daffd640 1 -- 172.21.15.31:0/1969283089 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0003840 con 0x7ff6e40ced30 2023-11-26T07:19:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ec31c640 1 -- 172.21.15.31:0/1969283089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e41595b0 con 0x7ff6e40ced30 2023-11-26T07:19:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.407+0000 7ff6ec31c640 1 -- 172.21.15.31:0/1969283089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e4161360 con 0x7ff6e40ced30 2023-11-26T07:19:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.411+0000 7ff6ec31c640 1 -- 172.21.15.31:0/1969283089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e404e850 con 0x7ff6e40ced30 2023-11-26T07:19:14.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.411+0000 7ff6daffd640 1 -- 172.21.15.31:0/1969283089 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff6e00039a0 con 0x7ff6e40ced30 2023-11-26T07:19:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.411+0000 7ff6daffd640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff6c8066620 0x7ff6c8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.411+0000 7ff6daffd640 1 -- 172.21.15.31:0/1969283089 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff6e00837c0 con 0x7ff6e40ced30 2023-11-26T07:19:14.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.411+0000 7ff6e98b7640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff6c8066620 0x7ff6c8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.411+0000 7ff6e98b7640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff6c8066620 0x7ff6c8068ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40b94e0 tx=0x7ff6d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.415+0000 7ff6daffd640 1 -- 172.21.15.31:0/1969283089 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6e00508e0 con 0x7ff6e40ced30 2023-11-26T07:19:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.579+0000 7ff6ec31c640 1 -- 172.21.15.31:0/1969283089 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6e40cf9d0 con 0x7ff6e40ced30 2023-11-26T07:19:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.579+0000 7ff6daffd640 1 -- 172.21.15.31:0/1969283089 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6e0017020 con 0x7ff6e40ced30 2023-11-26T07:19:14.586 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:19:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 -- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff6c8066620 msgr2=0x7ff6c8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff6c8066620 0x7ff6c8068ae0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40b94e0 tx=0x7ff6d4009040 comp rx=0 tx=0).stop 2023-11-26T07:19:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 -- 172.21.15.31:0/1969283089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40ced30 msgr2=0x7ff6e4160e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40ced30 0x7ff6e4160e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000c3c0 tx=0x7ff6e000c880 comp rx=0 tx=0).stop 2023-11-26T07:19:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 -- 172.21.15.31:0/1969283089 shutdown_connections 2023-11-26T07:19:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff6c8066620 0x7ff6c8068ae0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6e415d8f0 0x7ff6e4152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6e40d0510 0x7ff6e4152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 --2- 172.21.15.31:0/1969283089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6e40ced30 0x7ff6e4160e20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 -- 172.21.15.31:0/1969283089 >> 172.21.15.31:0/1969283089 conn(0x7ff6e409c750 msgr2=0x7ff6e40b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 -- 172.21.15.31:0/1969283089 shutdown_connections 2023-11-26T07:19:14.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.583+0000 7ff6ec31c640 1 -- 172.21.15.31:0/1969283089 wait complete. 2023-11-26T07:19:14.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:19:14.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc8271c4640 1 Processor -- start 2023-11-26T07:19:14.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc8271c4640 1 -- start start 2023-11-26T07:19:14.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc8271c4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc82015d8f0 0x7fc82015e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc8271c4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200d0510 0x7fc8200d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc8271c4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82015ecc0 con 0x7fc8200ced30 2023-11-26T07:19:14.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc8271c4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8200d4420 con 0x7fc82015d8f0 2023-11-26T07:19:14.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc8271c4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200cdfb0 con 0x7fc8200d0510 2023-11-26T07:19:14.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc8200ced30 0x7fc8200cfe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36228/0 (socket says 172.21.15.31:36228) 2023-11-26T07:19:14.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81ffff640 1 -- 172.21.15.31:0/3956936004 learned_addr learned my addr 172.21.15.31:0/3956936004 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:14.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc824f60640 1 --2- 172.21.15.31:0/3956936004 >> v2:172.21.15.97:3300/0 conn(0x7fc82015d8f0 0x7fc82015e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc825761640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200d0510 0x7fc8200d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204100950 0 0) 0x7fc82015ecc0 con 0x7fc8200ced30 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc810003530 con 0x7fc8200ced30 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3973396536 0 0) 0x7fc810003530 con 0x7fc8200ced30 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200d0510 msgr2=0x7fc8200d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200d0510 0x7fc8200d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 >> v2:172.21.15.97:3300/0 conn(0x7fc82015d8f0 msgr2=0x7fc82015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 --2- 172.21.15.31:0/3956936004 >> v2:172.21.15.97:3300/0 conn(0x7fc82015d8f0 0x7fc82015e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8200d4140 con 0x7fc8200ced30 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fc80c001680 con 0x7fc8200ced30 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 >> v1:172.21.15.31:6789/0 conn(0x7fc8200ced30 legacy=0x7fc8200cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810003980 0x7fc810003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8100047d0 0x7fc810004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.747+0000 7fc81f7fe640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200d0510 0x7fc810005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200d4420 con 0x7fc8100047d0 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200cdfb0 con 0x7fc810003980 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81000b990 con 0x7fc8200d0510 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81ffff640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8100047d0 0x7fc810004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc824f60640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810003980 0x7fc810003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc825761640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200d0510 0x7fc810005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81ffff640 1 -- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200d0510 msgr2=0x7fc810005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81ffff640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200d0510 0x7fc810005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81ffff640 1 -- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810003980 msgr2=0x7fc810003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:14.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81ffff640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc810003980 0x7fc810003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81ffff640 1 -- 172.21.15.31:0/3956936004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc810003530 con 0x7fc8100047d0 2023-11-26T07:19:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81ffff640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8100047d0 0x7fc810004c50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00c0e0 tx=0x7fc80c01f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0d0898912d33de server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0a8df0 con 0x7fc8100047d0 2023-11-26T07:19:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c0058e0 con 0x7fc8100047d0 2023-11-26T07:19:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.751+0000 7fc81f7fe640 1 -- 172.21.15.31:0/3956936004 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c005c90 con 0x7fc8100047d0 2023-11-26T07:19:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 -- 172.21.15.31:0/3956936004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8100047d0 msgr2=0x7fc810004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 --2- 172.21.15.31:0/3956936004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8100047d0 0x7fc810004c50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00c0e0 tx=0x7fc80c01f2d0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc825761640 1 -- 172.21.15.31:0/3956936004 reap_dead start 2023-11-26T07:19:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 -- 172.21.15.31:0/3956936004 shutdown_connections 2023-11-26T07:19:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 -- 172.21.15.31:0/3956936004 >> 172.21.15.31:0/3956936004 conn(0x7fc82009c750 msgr2=0x7fc8200d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 -- 172.21.15.31:0/3956936004 shutdown_connections 2023-11-26T07:19:14.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 -- 172.21.15.31:0/3956936004 wait complete. 2023-11-26T07:19:14.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 Processor -- start 2023-11-26T07:19:14.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 -- start start 2023-11-26T07:19:14.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8100047d0 0x7fc8201507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82009ea90 0x7fc82014d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b8d60 0x7fc82006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200b9300 con 0x7fc82009ea90 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc820150ce0 con 0x7fc8100047d0 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc8271c4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82014d9e0 con 0x7fc8200b8d60 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc81ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82009ea90 0x7fc82014d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc81ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82009ea90 0x7fc82014d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40794/0 (socket says 172.21.15.31:40794) 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc81ffff640 1 -- 172.21.15.31:0/253107010 learned_addr learned my addr 172.21.15.31:0/253107010 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc824f60640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8100047d0 0x7fc8201507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc825761640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b8d60 0x7fc82006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc81ffff640 1 -- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b8d60 msgr2=0x7fc82006b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc81ffff640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b8d60 0x7fc82006b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc81ffff640 1 -- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8100047d0 msgr2=0x7fc8201507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc81ffff640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8100047d0 0x7fc8201507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.755+0000 7fc81ffff640 1 -- 172.21.15.31:0/253107010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82014de30 con 0x7fc82009ea90 2023-11-26T07:19:14.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc81ffff640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82009ea90 0x7fc82014d4a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00c0e0 tx=0x7fc810005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc81d7fa640 1 -- 172.21.15.31:0/253107010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0b7070 con 0x7fc82009ea90 2023-11-26T07:19:14.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc81d7fa640 1 -- 172.21.15.31:0/253107010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c005ad0 con 0x7fc82009ea90 2023-11-26T07:19:14.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc81d7fa640 1 -- 172.21.15.31:0/253107010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c005e40 con 0x7fc82009ea90 2023-11-26T07:19:14.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc8271c4640 1 -- 172.21.15.31:0/253107010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82014e110 con 0x7fc82009ea90 2023-11-26T07:19:14.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc8271c4640 1 -- 172.21.15.31:0/253107010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82014e6d0 con 0x7fc82009ea90 2023-11-26T07:19:14.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc8271c4640 1 -- 172.21.15.31:0/253107010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82004e850 con 0x7fc82009ea90 2023-11-26T07:19:14.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc81d7fa640 1 -- 172.21.15.31:0/253107010 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fc80c007090 con 0x7fc82009ea90 2023-11-26T07:19:14.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc81d7fa640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066650 0x7fc804068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:14.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc81d7fa640 1 -- 172.21.15.31:0/253107010 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc80c1211a0 con 0x7fc82009ea90 2023-11-26T07:19:14.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.759+0000 7fc824f60640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066650 0x7fc804068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:14.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.763+0000 7fc824f60640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066650 0x7fc804068b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc814009be0 tx=0x7fc814009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:14.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.763+0000 7fc81d7fa640 1 -- 172.21.15.31:0/253107010 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc80c0ee350 con 0x7fc82009ea90 2023-11-26T07:19:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.919+0000 7fc8271c4640 1 -- 172.21.15.31:0/253107010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8200cf9d0 con 0x7fc82009ea90 2023-11-26T07:19:14.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.919+0000 7fc81d7fa640 1 -- 172.21.15.31:0/253107010 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc80c0f1970 con 0x7fc82009ea90 2023-11-26T07:19:14.927 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 -- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066650 msgr2=0x7fc804068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066650 0x7fc804068b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc814009be0 tx=0x7fc814009190 comp rx=0 tx=0).stop 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 -- 172.21.15.31:0/253107010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82009ea90 msgr2=0x7fc82014d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82009ea90 0x7fc82014d4a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00c0e0 tx=0x7fc810005800 comp rx=0 tx=0).stop 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 -- 172.21.15.31:0/253107010 shutdown_connections 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc804066650 0x7fc804068b10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8200b8d60 0x7fc82006b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc82009ea90 0x7fc82014d4a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 --2- 172.21.15.31:0/253107010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8100047d0 0x7fc8201507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 -- 172.21.15.31:0/253107010 >> 172.21.15.31:0/253107010 conn(0x7fc82009c750 msgr2=0x7fc82015ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 -- 172.21.15.31:0/253107010 shutdown_connections 2023-11-26T07:19:14.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:14.923+0000 7fc8271c4640 1 -- 172.21.15.31:0/253107010 wait complete. 2023-11-26T07:19:14.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:19:15.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.087+0000 7f8739c32640 1 Processor -- start 2023-11-26T07:19:15.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.087+0000 7f8739c32640 1 -- start start 2023-11-26T07:19:15.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8739c32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87340ae680 0x7f87340cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8739c32640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8734070890 0x7f8734070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8739c32640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8734072950 con 0x7f8734159f00 2023-11-26T07:19:15.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8739c32640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873406e2b0 con 0x7f8734070890 2023-11-26T07:19:15.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8739c32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873406c6f0 con 0x7f87340ae680 2023-11-26T07:19:15.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87337fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8734159f00 0x7f87340ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36244/0 (socket says 172.21.15.31:36244) 2023-11-26T07:19:15.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87337fe640 1 -- 172.21.15.31:0/3955852719 learned_addr learned my addr 172.21.15.31:0/3955852719 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194869020 0 0) 0x7f8734072950 con 0x7f8734159f00 2023-11-26T07:19:15.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f871c003450 con 0x7f8734159f00 2023-11-26T07:19:15.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8732ffd640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87340ae680 0x7f87340cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8733fff640 1 --2- 172.21.15.31:0/3955852719 >> v2:172.21.15.97:3300/0 conn(0x7f8734070890 0x7f8734070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 132654043 0 0) 0x7f871c003450 con 0x7f8734159f00 2023-11-26T07:19:15.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87340ae680 msgr2=0x7f87340cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:15.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87340ae680 0x7f87340cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 >> v2:172.21.15.97:3300/0 conn(0x7f8734070890 msgr2=0x7f8734070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:15.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 --2- 172.21.15.31:0/3955852719 >> v2:172.21.15.97:3300/0 conn(0x7f8734070890 0x7f8734070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873406e550 con 0x7f8734159f00 2023-11-26T07:19:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f8724001680 con 0x7f8734159f00 2023-11-26T07:19:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8732ffd640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87340ae680 0x7f87340cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 >> v1:172.21.15.31:6789/0 conn(0x7f8734159f00 legacy=0x7f87340ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c003a70 0x7f871c003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 0x7f871c00acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87340ae680 0x7f871c00d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873406e2b0 con 0x7f871c008860 2023-11-26T07:19:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873406c6f0 con 0x7f871c003a70 2023-11-26T07:19:15.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c00dc00 con 0x7f87340ae680 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8733fff640 1 --2- 172.21.15.31:0/3955852719 >> v2:172.21.15.97:3300/0 conn(0x7f8734070890 0x7f8734070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8732ffd640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 0x7f871c00acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87337fe640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c003a70 0x7f871c003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8733fff640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87340ae680 0x7f871c00d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8732ffd640 1 -- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87340ae680 msgr2=0x7f871c00d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8732ffd640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87340ae680 0x7f871c00d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8732ffd640 1 -- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c003a70 msgr2=0x7f871c003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8732ffd640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c003a70 0x7f871c003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f8732ffd640 1 -- 172.21.15.31:0/3955852719 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c003450 con 0x7f871c008860 2023-11-26T07:19:15.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.091+0000 7f87337fe640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f871c003a70 0x7f871c003e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f8732ffd640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 0x7f871c00acf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f87280098d0 tx=0x7f872801c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668b403d6621bbb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87280a8620 con 0x7f871c008860 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87280a8780 con 0x7f871c008860 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f87327fc640 1 -- 172.21.15.31:0/3955852719 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87280a8af0 con 0x7f871c008860 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f8739c32640 1 -- 172.21.15.31:0/3955852719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 msgr2=0x7f871c00acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f8739c32640 1 --2- 172.21.15.31:0/3955852719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 0x7f871c00acf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f87280098d0 tx=0x7f872801c5b0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f8733fff640 1 -- 172.21.15.31:0/3955852719 reap_dead start 2023-11-26T07:19:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f8739c32640 1 -- 172.21.15.31:0/3955852719 shutdown_connections 2023-11-26T07:19:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f8739c32640 1 -- 172.21.15.31:0/3955852719 >> 172.21.15.31:0/3955852719 conn(0x7f87340b0cb0 msgr2=0x7f87340b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f8739c32640 1 -- 172.21.15.31:0/3955852719 shutdown_connections 2023-11-26T07:19:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f8739c32640 1 -- 172.21.15.31:0/3955852719 wait complete. 2023-11-26T07:19:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.095+0000 7f8739c32640 1 Processor -- start 2023-11-26T07:19:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8739c32640 1 -- start start 2023-11-26T07:19:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8739c32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 0x7f87341549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8739c32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87340b1da0 0x7f873414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8739c32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87341535d0 0x7f87340acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8739c32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873414e700 con 0x7f871c008860 2023-11-26T07:19:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8739c32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8734153b00 con 0x7f87340b1da0 2023-11-26T07:19:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8739c32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8734153de0 con 0x7f87341535d0 2023-11-26T07:19:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f87337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 0x7f87341549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f87337fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 0x7f87341549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40806/0 (socket says 172.21.15.31:40806) 2023-11-26T07:19:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f87337fe640 1 -- 172.21.15.31:0/559885906 learned_addr learned my addr 172.21.15.31:0/559885906 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8733fff640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87341535d0 0x7f87340acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8732ffd640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87340b1da0 0x7f873414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f87337fe640 1 -- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87341535d0 msgr2=0x7f87340acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f87337fe640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87341535d0 0x7f87340acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f87337fe640 1 -- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87340b1da0 msgr2=0x7f873414e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f87337fe640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87340b1da0 0x7f873414e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f87337fe640 1 -- 172.21.15.31:0/559885906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87341540c0 con 0x7f871c008860 2023-11-26T07:19:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8732ffd640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87340b1da0 0x7f873414e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8733fff640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87341535d0 0x7f87340acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f87337fe640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 0x7f87341549b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8724011980 tx=0x7f8724011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f871bfff640 1 -- 172.21.15.31:0/559885906 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872400fbc0 con 0x7f871c008860 2023-11-26T07:19:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f871bfff640 1 -- 172.21.15.31:0/559885906 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872400fd20 con 0x7f871c008860 2023-11-26T07:19:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f871bfff640 1 -- 172.21.15.31:0/559885906 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724015650 con 0x7f871c008860 2023-11-26T07:19:15.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8739c32640 1 -- 172.21.15.31:0/559885906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873415b710 con 0x7f871c008860 2023-11-26T07:19:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8739c32640 1 -- 172.21.15.31:0/559885906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873414eb60 con 0x7f871c008860 2023-11-26T07:19:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.099+0000 7f8739c32640 1 -- 172.21.15.31:0/559885906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87340671f0 con 0x7f871c008860 2023-11-26T07:19:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.103+0000 7f871bfff640 1 -- 172.21.15.31:0/559885906 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f87240157b0 con 0x7f871c008860 2023-11-26T07:19:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.103+0000 7f871bfff640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8720066650 0x7f8720068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.103+0000 7f871bfff640 1 -- 172.21.15.31:0/559885906 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8724087640 con 0x7f871c008860 2023-11-26T07:19:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.103+0000 7f8732ffd640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8720066650 0x7f8720068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.103+0000 7f8732ffd640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8720066650 0x7f8720068b10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f87280af000 tx=0x7f87280aa040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.107+0000 7f871bfff640 1 -- 172.21.15.31:0/559885906 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8724054760 con 0x7f871c008860 2023-11-26T07:19:15.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 -- 172.21.15.31:0/559885906 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f873415b570 con 0x7f871c008860 2023-11-26T07:19:15.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f871bfff640 1 -- 172.21.15.31:0/559885906 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7f873415b570 con 0x7f871c008860 2023-11-26T07:19:15.273 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:19:15.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 -- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8720066650 msgr2=0x7f8720068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8720066650 0x7f8720068b10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f87280af000 tx=0x7f87280aa040 comp rx=0 tx=0).stop 2023-11-26T07:19:15.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 -- 172.21.15.31:0/559885906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 msgr2=0x7f87341549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 0x7f87341549b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8724011980 tx=0x7f8724011e40 comp rx=0 tx=0).stop 2023-11-26T07:19:15.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 -- 172.21.15.31:0/559885906 shutdown_connections 2023-11-26T07:19:15.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8720066650 0x7f8720068b10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87341535d0 0x7f87340acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87340b1da0 0x7f873414e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 --2- 172.21.15.31:0/559885906 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f871c008860 0x7f87341549b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 -- 172.21.15.31:0/559885906 >> 172.21.15.31:0/559885906 conn(0x7f87340b0cb0 msgr2=0x7f8734156320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:15.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 -- 172.21.15.31:0/559885906 shutdown_connections 2023-11-26T07:19:15.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.267+0000 7f8739c32640 1 -- 172.21.15.31:0/559885906 wait complete. 2023-11-26T07:19:15.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:19:15.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.431+0000 7fa85a65a640 1 Processor -- start 2023-11-26T07:19:15.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.431+0000 7fa85a65a640 1 -- start start 2023-11-26T07:19:15.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa85a65a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 0x7fa8540abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa85a65a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa85415a710 0x7fa8540ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa85a65a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8540ce8d0 con 0x7fa8540cbd50 2023-11-26T07:19:15.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa85a65a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8540cca80 con 0x7fa85415a710 2023-11-26T07:19:15.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa85a65a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540b2710 con 0x7fa8540cdfb0 2023-11-26T07:19:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa8537fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa85415a710 0x7fa8540ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa858bf7640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa8540cbd50 0x7fa8540cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36258/0 (socket says 172.21.15.31:36258) 2023-11-26T07:19:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa858bf7640 1 -- 172.21.15.31:0/439038874 learned_addr learned my addr 172.21.15.31:0/439038874 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:15.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa853fff640 1 --2- 172.21.15.31:0/439038874 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 0x7fa8540abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa852ffd640 1 -- 172.21.15.31:0/439038874 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170002603 0 0) 0x7fa8540ce8d0 con 0x7fa8540cbd50 2023-11-26T07:19:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa8537fe640 1 -- 172.21.15.31:0/439038874 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 msgr2=0x7fa8540abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa8537fe640 1 --2- 172.21.15.31:0/439038874 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 0x7fa8540abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa8537fe640 1 -- 172.21.15.31:0/439038874 >> v1:172.21.15.31:6789/0 conn(0x7fa8540cbd50 legacy=0x7fa8540cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa8537fe640 1 -- 172.21.15.31:0/439038874 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540b29f0 con 0x7fa85415a710 2023-11-26T07:19:15.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.435+0000 7fa853fff640 1 --2- 172.21.15.31:0/439038874 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 0x7fa8540abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa8537fe640 1 --2- 172.21.15.31:0/439038874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa85415a710 0x7fa8540ce390 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa8480097c0 tx=0x7fa84801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbae3a54684f38 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa852ffd640 1 -- 172.21.15.31:0/439038874 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480ab650 con 0x7fa85415a710 2023-11-26T07:19:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa852ffd640 1 -- 172.21.15.31:0/439038874 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8480ab7b0 con 0x7fa85415a710 2023-11-26T07:19:15.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa852ffd640 1 -- 172.21.15.31:0/439038874 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480abb40 con 0x7fa85415a710 2023-11-26T07:19:15.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 -- 172.21.15.31:0/439038874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa85415a710 msgr2=0x7fa8540ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 --2- 172.21.15.31:0/439038874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa85415a710 0x7fa8540ce390 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa8480097c0 tx=0x7fa84801c450 comp rx=0 tx=0).stop 2023-11-26T07:19:15.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 -- 172.21.15.31:0/439038874 shutdown_connections 2023-11-26T07:19:15.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 --2- 172.21.15.31:0/439038874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa85415a710 0x7fa8540ce390 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 --2- 172.21.15.31:0/439038874 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 0x7fa8540abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 -- 172.21.15.31:0/439038874 >> 172.21.15.31:0/439038874 conn(0x7fa8540b0cb0 msgr2=0x7fa8540b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:15.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 -- 172.21.15.31:0/439038874 shutdown_connections 2023-11-26T07:19:15.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 -- 172.21.15.31:0/439038874 wait complete. 2023-11-26T07:19:15.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 Processor -- start 2023-11-26T07:19:15.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 -- start start 2023-11-26T07:19:15.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540cbd50 0x7fa854150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.439+0000 7fa85a65a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 0x7fa854071150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa85a65a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa85415a710 0x7fa854071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa85a65a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa854151dc0 con 0x7fa8540cbd50 2023-11-26T07:19:15.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa85a65a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa854072860 con 0x7fa85415a710 2023-11-26T07:19:15.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa85a65a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa854071ce0 con 0x7fa8540cdfb0 2023-11-26T07:19:15.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa853fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540cbd50 0x7fa854150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa853fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540cbd50 0x7fa854150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40816/0 (socket says 172.21.15.31:40816) 2023-11-26T07:19:15.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa853fff640 1 -- 172.21.15.31:0/1948129713 learned_addr learned my addr 172.21.15.31:0/1948129713 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:15.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa858bf7640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa85415a710 0x7fa854071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa8537fe640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 0x7fa854071150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa853fff640 1 -- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 msgr2=0x7fa854071150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa853fff640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 0x7fa854071150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa853fff640 1 -- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa85415a710 msgr2=0x7fa854071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa853fff640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa85415a710 0x7fa854071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa853fff640 1 -- 172.21.15.31:0/1948129713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa848009420 con 0x7fa8540cbd50 2023-11-26T07:19:15.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa858bf7640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa85415a710 0x7fa854071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:15.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa853fff640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540cbd50 0x7fa854150ce0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa84400c7c0 tx=0x7fa84400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa850ff9640 1 -- 172.21.15.31:0/1948129713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844007a30 con 0x7fa8540cbd50 2023-11-26T07:19:15.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa850ff9640 1 -- 172.21.15.31:0/1948129713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa844007b90 con 0x7fa8540cbd50 2023-11-26T07:19:15.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa85a65a640 1 -- 172.21.15.31:0/1948129713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa854071fc0 con 0x7fa8540cbd50 2023-11-26T07:19:15.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa850ff9640 1 -- 172.21.15.31:0/1948129713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84400ce70 con 0x7fa8540cbd50 2023-11-26T07:19:15.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa85a65a640 1 -- 172.21.15.31:0/1948129713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa854072580 con 0x7fa8540cbd50 2023-11-26T07:19:15.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa85a65a640 1 -- 172.21.15.31:0/1948129713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8540671f0 con 0x7fa8540cbd50 2023-11-26T07:19:15.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.443+0000 7fa850ff9640 1 -- 172.21.15.31:0/1948129713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fa8440113f0 con 0x7fa8540cbd50 2023-11-26T07:19:15.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.447+0000 7fa850ff9640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa840066620 0x7fa840068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.447+0000 7fa850ff9640 1 -- 172.21.15.31:0/1948129713 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa8440846d0 con 0x7fa8540cbd50 2023-11-26T07:19:15.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.447+0000 7fa8537fe640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa840066620 0x7fa840068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.447+0000 7fa8537fe640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa840066620 0x7fa840068ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa8480af000 tx=0x7fa8480a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.451+0000 7fa850ff9640 1 -- 172.21.15.31:0/1948129713 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa844051820 con 0x7fa8540cbd50 2023-11-26T07:19:15.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 -- 172.21.15.31:0/1948129713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa85406b460 con 0x7fa8540cbd50 2023-11-26T07:19:15.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa850ff9640 1 -- 172.21.15.31:0/1948129713 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa844054e40 con 0x7fa8540cbd50 2023-11-26T07:19:15.629 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:19:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 -- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa840066620 msgr2=0x7fa840068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa840066620 0x7fa840068ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa8480af000 tx=0x7fa8480a7000 comp rx=0 tx=0).stop 2023-11-26T07:19:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 -- 172.21.15.31:0/1948129713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540cbd50 msgr2=0x7fa854150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540cbd50 0x7fa854150ce0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa84400c7c0 tx=0x7fa84400cc80 comp rx=0 tx=0).stop 2023-11-26T07:19:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 -- 172.21.15.31:0/1948129713 shutdown_connections 2023-11-26T07:19:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa840066620 0x7fa840068ae0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa85415a710 0x7fa854071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8540cdfb0 0x7fa854071150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 --2- 172.21.15.31:0/1948129713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8540cbd50 0x7fa854150ce0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 -- 172.21.15.31:0/1948129713 >> 172.21.15.31:0/1948129713 conn(0x7fa8540b0cb0 msgr2=0x7fa8540b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 -- 172.21.15.31:0/1948129713 shutdown_connections 2023-11-26T07:19:15.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.623+0000 7fa85a65a640 1 -- 172.21.15.31:0/1948129713 wait complete. 2023-11-26T07:19:15.649 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:19:15.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:15.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:19:15.684 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:19:15.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:19:15.685 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:19:15.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:15.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4e21f5640 1 Processor -- start 2023-11-26T07:19:15.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4e21f5640 1 -- start start 2023-11-26T07:19:15.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4e21f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc15d530 0x7fd4dc0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4e21f5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd4dc0d04c0 0x7fd4dc1674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4e21f5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc0d12e0 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4e21f5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc0d15c0 con 0x7fd4dc0d04c0 2023-11-26T07:19:15.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4e21f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0d18a0 con 0x7fd4dc15d530 2023-11-26T07:19:15.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4dbfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd4dc0d04c0 0x7fd4dc1674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4daffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd4dc0cf1b0 0x7fd4dc0cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36270/0 (socket says 172.21.15.31:36270) 2023-11-26T07:19:15.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4daffd640 1 -- 172.21.15.31:0/1017580778 learned_addr learned my addr 172.21.15.31:0/1017580778 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:15.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4dbfff640 1 -- 172.21.15.31:0/1017580778 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc15d530 msgr2=0x7fd4dc0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:15.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4db7fe640 1 --2- 172.21.15.31:0/1017580778 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc15d530 0x7fd4dc0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4dbfff640 1 --2- 172.21.15.31:0/1017580778 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc15d530 0x7fd4dc0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4dbfff640 1 -- 172.21.15.31:0/1017580778 >> v1:172.21.15.31:6789/0 conn(0x7fd4dc0cf1b0 legacy=0x7fd4dc0cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4dbfff640 1 -- 172.21.15.31:0/1017580778 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0d1b80 con 0x7fd4dc0d04c0 2023-11-26T07:19:15.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.875+0000 7fd4db7fe640 1 --2- 172.21.15.31:0/1017580778 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc15d530 0x7fd4dc0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:15.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4dbfff640 1 --2- 172.21.15.31:0/1017580778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0d04c0 0x7fd4dc1674d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009d90 tx=0x7fd4d401f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbdb8cee46abe90 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4da7fc640 1 -- 172.21.15.31:0/1017580778 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d40af600 con 0x7fd4dc0d04c0 2023-11-26T07:19:15.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4da7fc640 1 -- 172.21.15.31:0/1017580778 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d40ab050 con 0x7fd4dc0d04c0 2023-11-26T07:19:15.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4da7fc640 1 -- 172.21.15.31:0/1017580778 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d40af980 con 0x7fd4dc0d04c0 2023-11-26T07:19:15.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1017580778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0d04c0 msgr2=0x7fd4dc1674d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4e21f5640 1 --2- 172.21.15.31:0/1017580778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0d04c0 0x7fd4dc1674d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009d90 tx=0x7fd4d401f130 comp rx=0 tx=0).stop 2023-11-26T07:19:15.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1017580778 shutdown_connections 2023-11-26T07:19:15.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4e21f5640 1 --2- 172.21.15.31:0/1017580778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0d04c0 0x7fd4dc1674d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4e21f5640 1 --2- 172.21.15.31:0/1017580778 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc15d530 0x7fd4dc0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1017580778 >> 172.21.15.31:0/1017580778 conn(0x7fd4dc09c6d0 msgr2=0x7fd4dc0d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:15.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1017580778 shutdown_connections 2023-11-26T07:19:15.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.879+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1017580778 wait complete. 2023-11-26T07:19:15.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 Processor -- start 2023-11-26T07:19:15.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 -- start start 2023-11-26T07:19:15.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0cf1b0 0x7fd4dc15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc0d04c0 0x7fd4dc0b8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4dc15d530 0x7fd4dc0b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc1f7c10 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc15c790 con 0x7fd4dc15d530 2023-11-26T07:19:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc1f7930 con 0x7fd4dc0d04c0 2023-11-26T07:19:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0cf1b0 0x7fd4dc15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4db7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0cf1b0 0x7fd4dc15bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40830/0 (socket says 172.21.15.31:40830) 2023-11-26T07:19:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4db7fe640 1 -- 172.21.15.31:0/1030887802 learned_addr learned my addr 172.21.15.31:0/1030887802 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4daffd640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc0d04c0 0x7fd4dc0b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4dbfff640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4dc15d530 0x7fd4dc0b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4db7fe640 1 -- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc0d04c0 msgr2=0x7fd4dc0b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4db7fe640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc0d04c0 0x7fd4dc0b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4db7fe640 1 -- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4dc15d530 msgr2=0x7fd4dc0b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:15.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4db7fe640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4dc15d530 0x7fd4dc0b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:15.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4db7fe640 1 -- 172.21.15.31:0/1030887802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d4009a70 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4daffd640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc0d04c0 0x7fd4dc0b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:15.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4dbfff640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4dc15d530 0x7fd4dc0b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:15.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4db7fe640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0cf1b0 0x7fd4dc15bc60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c7f0 tx=0x7fd4cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4c3fff640 1 -- 172.21.15.31:0/1030887802 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc007820 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4c3fff640 1 -- 172.21.15.31:0/1030887802 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc007980 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4c3fff640 1 -- 172.21.15.31:0/1030887802 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc007d30 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1030887802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc0b6110 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1030887802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc0b66d0 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.883+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1030887802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc0b68f0 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.887+0000 7fd4c3fff640 1 -- 172.21.15.31:0/1030887802 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd4cc0113f0 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.887+0000 7fd4c3fff640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd4c8066620 0x7fd4c8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:15.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.887+0000 7fd4c3fff640 1 -- 172.21.15.31:0/1030887802 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd4cc084660 con 0x7fd4dc0cf1b0 2023-11-26T07:19:15.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.887+0000 7fd4daffd640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd4c8066620 0x7fd4c8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:15.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.887+0000 7fd4daffd640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd4c8066620 0x7fd4c8068ae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0ccbb0 tx=0x7fd4d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:15.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:15.891+0000 7fd4c3fff640 1 -- 172.21.15.31:0/1030887802 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd4cc051700 con 0x7fd4dc0cf1b0 2023-11-26T07:19:16.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.079+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1030887802 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4dc04e850 con 0x7fd4dc0cf1b0 2023-11-26T07:19:16.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.079+0000 7fd4c3fff640 1 -- 172.21.15.31:0/1030887802 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd4cc054d20 con 0x7fd4dc0cf1b0 2023-11-26T07:19:16.087 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:16.087 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:16.087 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:19:16.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd4c8066620 msgr2=0x7fd4c8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd4c8066620 0x7fd4c8068ae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0ccbb0 tx=0x7fd4d0009040 comp rx=0 tx=0).stop 2023-11-26T07:19:16.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1030887802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0cf1b0 msgr2=0x7fd4dc15bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0cf1b0 0x7fd4dc15bc60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00c7f0 tx=0x7fd4cc00ccb0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1030887802 shutdown_connections 2023-11-26T07:19:16.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd4c8066620 0x7fd4c8068ae0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4dc15d530 0x7fd4dc0b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd4dc0d04c0 0x7fd4dc0b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 --2- 172.21.15.31:0/1030887802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4dc0cf1b0 0x7fd4dc15bc60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1030887802 >> 172.21.15.31:0/1030887802 conn(0x7fd4dc09c6d0 msgr2=0x7fd4dc0b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:16.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1030887802 shutdown_connections 2023-11-26T07:19:16.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.083+0000 7fd4e21f5640 1 -- 172.21.15.31:0/1030887802 wait complete. 2023-11-26T07:19:16.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:19:16.273 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-11-26T07:19:16.273 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-26T07:19:16.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol87490 2023-11-26T07:19:16.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb835abb640 1 Processor -- start 2023-11-26T07:19:16.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb835abb640 1 -- start start 2023-11-26T07:19:16.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb835abb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb83015d550 0x7fb830161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb835abb640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb830161ff0 con 0x7fb8300d0190 2023-11-26T07:19:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb835abb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8300d4230 con 0x7fb8300d3a00 2023-11-26T07:19:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb835abb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8300d1210 con 0x7fb83015d550 2023-11-26T07:19:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb8352ba640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb8300d0190 0x7fb8300cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36286/0 (socket says 172.21.15.31:36286) 2023-11-26T07:19:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb8352ba640 1 -- 172.21.15.31:0/3371191159 learned_addr learned my addr 172.21.15.31:0/3371191159 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb834ab9640 1 --2- 172.21.15.31:0/3371191159 >> v2:172.21.15.97:3300/0 conn(0x7fb83015d550 0x7fb830161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864631762 0 0) 0x7fb830161ff0 con 0x7fb8300d0190 2023-11-26T07:19:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb820003530 con 0x7fb8300d0190 2023-11-26T07:19:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4066313019 0 0) 0x7fb820003530 con 0x7fb8300d0190 2023-11-26T07:19:16.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 >> v2:172.21.15.97:3300/0 conn(0x7fb83015d550 msgr2=0x7fb830161ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:16.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 --2- 172.21.15.31:0/3371191159 >> v2:172.21.15.97:3300/0 conn(0x7fb83015d550 0x7fb830161ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 >> v1:172.21.15.97:6789/0 conn(0x7fb8300d3a00 legacy=0x7fb8300cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83015dd00 con 0x7fb8300d0190 2023-11-26T07:19:16.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fb828001a00 con 0x7fb8300d0190 2023-11-26T07:19:16.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 >> v1:172.21.15.31:6789/0 conn(0x7fb8300d0190 legacy=0x7fb8300cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb820003980 0x7fb820003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.415+0000 7fb82f7fe640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8200047d0 0x7fb820004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82f7fe640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb820005190 0x7fb82000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300d1210 con 0x7fb8200047d0 2023-11-26T07:19:16.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82000c040 con 0x7fb820005190 2023-11-26T07:19:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82000c200 con 0x7fb820003980 2023-11-26T07:19:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb834ab9640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb820003980 0x7fb820003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82ffff640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8200047d0 0x7fb820004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb8352ba640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb820005190 0x7fb82000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82ffff640 1 -- 172.21.15.31:0/3371191159 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb820003980 msgr2=0x7fb820003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82ffff640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb820003980 0x7fb820003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82ffff640 1 -- 172.21.15.31:0/3371191159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb820005190 msgr2=0x7fb82000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:16.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82ffff640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb820005190 0x7fb82000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82ffff640 1 -- 172.21.15.31:0/3371191159 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820003530 con 0x7fb8200047d0 2023-11-26T07:19:16.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb834ab9640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb820003980 0x7fb820003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:16.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb8352ba640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb820005190 0x7fb82000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:16.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb8352ba640 1 -- 172.21.15.31:0/3371191159 reap_dead start 2023-11-26T07:19:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82ffff640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8200047d0 0x7fb820004c50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb81c001780 tx=0x7fb81c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bff4797796df922 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0052d0 con 0x7fb8200047d0 2023-11-26T07:19:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c0a7050 con 0x7fb8200047d0 2023-11-26T07:19:16.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.419+0000 7fb82f7fe640 1 -- 172.21.15.31:0/3371191159 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c003600 con 0x7fb8200047d0 2023-11-26T07:19:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 -- 172.21.15.31:0/3371191159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8200047d0 msgr2=0x7fb820004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8200047d0 0x7fb820004c50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb81c001780 tx=0x7fb81c01c3e0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 -- 172.21.15.31:0/3371191159 shutdown_connections 2023-11-26T07:19:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 --2- 172.21.15.31:0/3371191159 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8200047d0 0x7fb820004c50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 -- 172.21.15.31:0/3371191159 >> 172.21.15.31:0/3371191159 conn(0x7fb83009c670 msgr2=0x7fb8300d32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 -- 172.21.15.31:0/3371191159 shutdown_connections 2023-11-26T07:19:16.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 -- 172.21.15.31:0/3371191159 wait complete. 2023-11-26T07:19:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 Processor -- start 2023-11-26T07:19:16.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 -- start start 2023-11-26T07:19:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8200047d0 0x7fb8301f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83015a380 0x7fb83015a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8300ca5f0 0x7fb8300caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83015ad40 con 0x7fb83015a380 2023-11-26T07:19:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300cbf30 con 0x7fb8300ca5f0 2023-11-26T07:19:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb835abb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8300cb320 con 0x7fb8200047d0 2023-11-26T07:19:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb82ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83015a380 0x7fb83015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb82ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83015a380 0x7fb83015a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40840/0 (socket says 172.21.15.31:40840) 2023-11-26T07:19:16.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb82ffff640 1 -- 172.21.15.31:0/2084040976 learned_addr learned my addr 172.21.15.31:0/2084040976 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb834ab9640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8200047d0 0x7fb8301f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.423+0000 7fb8352ba640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8300ca5f0 0x7fb8300caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82ffff640 1 -- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8200047d0 msgr2=0x7fb8301f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82ffff640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8200047d0 0x7fb8301f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82ffff640 1 -- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8300ca5f0 msgr2=0x7fb8300caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82ffff640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8300ca5f0 0x7fb8300caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82ffff640 1 -- 172.21.15.31:0/2084040976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8300cb600 con 0x7fb83015a380 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82ffff640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83015a380 0x7fb83015a800 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb81c01c210 tx=0x7fb820005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82d7fa640 1 -- 172.21.15.31:0/2084040976 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c007440 con 0x7fb83015a380 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82d7fa640 1 -- 172.21.15.31:0/2084040976 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c0075c0 con 0x7fb83015a380 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82d7fa640 1 -- 172.21.15.31:0/2084040976 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c006a20 con 0x7fb83015a380 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb835abb640 1 -- 172.21.15.31:0/2084040976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8300cb8e0 con 0x7fb83015a380 2023-11-26T07:19:16.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb835abb640 1 -- 172.21.15.31:0/2084040976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8300caf90 con 0x7fb83015a380 2023-11-26T07:19:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb835abb640 1 -- 172.21.15.31:0/2084040976 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb830074990 con 0x7fb83015a380 2023-11-26T07:19:16.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82d7fa640 1 -- 172.21.15.31:0/2084040976 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fb81c006b80 con 0x7fb83015a380 2023-11-26T07:19:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82d7fa640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066620 0x7fb814068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb82d7fa640 1 -- 172.21.15.31:0/2084040976 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb81c11e630 con 0x7fb83015a380 2023-11-26T07:19:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.427+0000 7fb834ab9640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066620 0x7fb814068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.431+0000 7fb834ab9640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066620 0x7fb814068ae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb830064280 tx=0x7fb824008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.431+0000 7fb82d7fa640 1 -- 172.21.15.31:0/2084040976 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb81c0eb640 con 0x7fb83015a380 2023-11-26T07:19:16.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.615+0000 7fb835abb640 1 -- 172.21.15.31:0/2084040976 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol87490", "target": ["mon-mgr", ""]}) v1 -- 0x7fb83015dfa0 con 0x7fb814066620 2023-11-26T07:19:16.630 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:19:16.622+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:16.630 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:19:16.622+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:16.630 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:19:16.622+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:16.630 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:19:16.622+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:16.630 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:19:16.622+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:19:16.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb82d7fa640 1 -- 172.21.15.31:0/2084040976 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb824014ed0 con 0x7fb814066620 2023-11-26T07:19:16.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 -- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066620 msgr2=0x7fb814068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066620 0x7fb814068ae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb830064280 tx=0x7fb824008040 comp rx=0 tx=0).stop 2023-11-26T07:19:16.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 -- 172.21.15.31:0/2084040976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83015a380 msgr2=0x7fb83015a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83015a380 0x7fb83015a800 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb81c01c210 tx=0x7fb820005800 comp rx=0 tx=0).stop 2023-11-26T07:19:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 -- 172.21.15.31:0/2084040976 shutdown_connections 2023-11-26T07:19:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb814066620 0x7fb814068ae0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8300ca5f0 0x7fb8300caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb83015a380 0x7fb83015a800 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 --2- 172.21.15.31:0/2084040976 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb8200047d0 0x7fb8301f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 -- 172.21.15.31:0/2084040976 >> 172.21.15.31:0/2084040976 conn(0x7fb83009c670 msgr2=0x7fb8300cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 -- 172.21.15.31:0/2084040976 shutdown_connections 2023-11-26T07:19:16.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.659+0000 7fb835abb640 1 -- 172.21.15.31:0/2084040976 wait complete. 2023-11-26T07:19:16.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:16.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2c5c1640 1 Processor -- start 2023-11-26T07:19:16.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2c5c1640 1 -- start start 2023-11-26T07:19:16.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2c5c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2406dfd0 0x7fee2406e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2c5c1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee2406e950 con 0x7fee240adcc0 2023-11-26T07:19:16.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2c5c1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee24070670 con 0x7fee24143ae0 2023-11-26T07:19:16.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2c5c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee240ae820 con 0x7fee2406dfd0 2023-11-26T07:19:16.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2ab5e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fee240adcc0 0x7fee240ae110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36288/0 (socket says 172.21.15.31:36288) 2023-11-26T07:19:16.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2ab5e640 1 -- 172.21.15.31:0/2832410812 learned_addr learned my addr 172.21.15.31:0/2832410812 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee29b5c640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2406dfd0 0x7fee2406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351154003 0 0) 0x7fee2406e950 con 0x7fee240adcc0 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee10003530 con 0x7fee240adcc0 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945745495 0 0) 0x7fee24070670 con 0x7fee24143ae0 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.823+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee2406e950 con 0x7fee24143ae0 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3830924529 0 0) 0x7fee10003530 con 0x7fee240adcc0 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2406dfd0 msgr2=0x7fee2406e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2406dfd0 0x7fee2406e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 >> v1:172.21.15.97:6789/0 conn(0x7fee24143ae0 legacy=0x7fee240b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24070950 con 0x7fee240adcc0 2023-11-26T07:19:16.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fee1c001a00 con 0x7fee240adcc0 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee29b5c640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2406dfd0 0x7fee2406e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 >> v1:172.21.15.31:6789/0 conn(0x7fee240adcc0 legacy=0x7fee240ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 0x7fee10003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2406dfd0 0x7fee100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee10004d10 0x7fee10005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee240ae820 con 0x7fee10003b40 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1000dc50 con 0x7fee10004d10 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1000ddc0 con 0x7fee2406dfd0 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2a35d640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 0x7fee10003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee29b5c640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2406dfd0 0x7fee100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2ab5e640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee10004d10 0x7fee10005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2a35d640 1 -- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2406dfd0 msgr2=0x7fee100047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2a35d640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2406dfd0 0x7fee100047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2a35d640 1 -- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee10004d10 msgr2=0x7fee10005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2a35d640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee10004d10 0x7fee10005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2a35d640 1 -- 172.21.15.31:0/2832410812 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee14009420 con 0x7fee10003b40 2023-11-26T07:19:16.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.827+0000 7fee2a35d640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 0x7fee10003f40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fee200018d0 tx=0x7fee2001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8004381d29f15e3d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee200052d0 con 0x7fee10003b40 2023-11-26T07:19:16.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee20005450 con 0x7fee10003b40 2023-11-26T07:19:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2935b640 1 -- 172.21.15.31:0/2832410812 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20003840 con 0x7fee10003b40 2023-11-26T07:19:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 -- 172.21.15.31:0/2832410812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 msgr2=0x7fee10003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 --2- 172.21.15.31:0/2832410812 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 0x7fee10003f40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fee200018d0 tx=0x7fee2001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2ab5e640 1 -- 172.21.15.31:0/2832410812 reap_dead start 2023-11-26T07:19:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 -- 172.21.15.31:0/2832410812 shutdown_connections 2023-11-26T07:19:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 -- 172.21.15.31:0/2832410812 >> 172.21.15.31:0/2832410812 conn(0x7fee240b0c40 msgr2=0x7fee240b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 -- 172.21.15.31:0/2832410812 shutdown_connections 2023-11-26T07:19:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 -- 172.21.15.31:0/2832410812 wait complete. 2023-11-26T07:19:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 Processor -- start 2023-11-26T07:19:16.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 -- start start 2023-11-26T07:19:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 0x7fee2409c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee2415c0f0 0x7fee2415c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2415dd00 0x7fee2415e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2409a270 con 0x7fee10003b40 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2415e6c0 con 0x7fee2415c0f0 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2c5c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee24137520 con 0x7fee2415dd00 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.831+0000 7fee2a35d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 0x7fee2409c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2a35d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 0x7fee2409c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40858/0 (socket says 172.21.15.31:40858) 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2a35d640 1 -- 172.21.15.31:0/1991355770 learned_addr learned my addr 172.21.15.31:0/1991355770 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2ab5e640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2415dd00 0x7fee2415e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee29b5c640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee2415c0f0 0x7fee2415c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2a35d640 1 -- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2415dd00 msgr2=0x7fee2415e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2a35d640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2415dd00 0x7fee2415e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2a35d640 1 -- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee2415c0f0 msgr2=0x7fee2415c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:16.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2a35d640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee2415c0f0 0x7fee2415c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2a35d640 1 -- 172.21.15.31:0/1991355770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2409cb20 con 0x7fee10003b40 2023-11-26T07:19:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2ab5e640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2415dd00 0x7fee2415e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2a35d640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 0x7fee2409c5e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee200018d0 tx=0x7fee10005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee1affd640 1 -- 172.21.15.31:0/1991355770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee200050d0 con 0x7fee10003b40 2023-11-26T07:19:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee1affd640 1 -- 172.21.15.31:0/1991355770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee20005230 con 0x7fee10003b40 2023-11-26T07:19:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee1affd640 1 -- 172.21.15.31:0/1991355770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20003e20 con 0x7fee10003b40 2023-11-26T07:19:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2c5c1640 1 -- 172.21.15.31:0/1991355770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2415cb90 con 0x7fee10003b40 2023-11-26T07:19:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2c5c1640 1 -- 172.21.15.31:0/1991355770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2409a480 con 0x7fee10003b40 2023-11-26T07:19:16.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee2c5c1640 1 -- 172.21.15.31:0/1991355770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2404e850 con 0x7fee10003b40 2023-11-26T07:19:16.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee1affd640 1 -- 172.21.15.31:0/1991355770 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fee200067e0 con 0x7fee10003b40 2023-11-26T07:19:16.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee1affd640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fee08066650 0x7fee08068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:16.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.835+0000 7fee1affd640 1 -- 172.21.15.31:0/1991355770 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fee2011d850 con 0x7fee10003b40 2023-11-26T07:19:16.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.839+0000 7fee29b5c640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fee08066650 0x7fee08068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:16.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.839+0000 7fee29b5c640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fee08066650 0x7fee08068b10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fee240b3bc0 tx=0x7fee14009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:16.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:16.839+0000 7fee1affd640 1 -- 172.21.15.31:0/1991355770 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee200eaa30 con 0x7fee10003b40 2023-11-26T07:19:17.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.031+0000 7fee2c5c1640 1 -- 172.21.15.31:0/1991355770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee240abf50 con 0x7fee10003b40 2023-11-26T07:19:17.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee1affd640 1 -- 172.21.15.31:0/1991355770 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fee200ee050 con 0x7fee10003b40 2023-11-26T07:19:17.040 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:17.040 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:17.040 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:19:17.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 -- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fee08066650 msgr2=0x7fee08068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fee08066650 0x7fee08068b10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fee240b3bc0 tx=0x7fee14009190 comp rx=0 tx=0).stop 2023-11-26T07:19:17.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 -- 172.21.15.31:0/1991355770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 msgr2=0x7fee2409c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 0x7fee2409c5e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee200018d0 tx=0x7fee10005800 comp rx=0 tx=0).stop 2023-11-26T07:19:17.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 -- 172.21.15.31:0/1991355770 shutdown_connections 2023-11-26T07:19:17.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fee08066650 0x7fee08068b10 secure :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fee240b3bc0 tx=0x7fee14009190 comp rx=0 tx=0).stop 2023-11-26T07:19:17.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fee2415dd00 0x7fee2415e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee2415c0f0 0x7fee2415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 --2- 172.21.15.31:0/1991355770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee10003b40 0x7fee2409c5e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 -- 172.21.15.31:0/1991355770 >> 172.21.15.31:0/1991355770 conn(0x7fee240b0c40 msgr2=0x7fee2406d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:17.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 -- 172.21.15.31:0/1991355770 shutdown_connections 2023-11-26T07:19:17.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.035+0000 7fee2c5c1640 1 -- 172.21.15.31:0/1991355770 wait complete. 2023-11-26T07:19:17.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:19:17.226 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.57599388100000004,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.971378100733524,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:19:17.227 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.971378100733524 }, "state": "open", "uptime": 0.575993881 } ] 2023-11-26T07:19:17.227 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:17.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb9157640 1 Processor -- start 2023-11-26T07:19:17.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb9157640 1 -- start start 2023-11-26T07:19:17.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb9157640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0cb406dfd0 0x7f0cb406e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb9157640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0cb40adcc0 0x7f0cb40ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb9157640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb406e950 con 0x7f0cb4143ae0 2023-11-26T07:19:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb9157640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb4070670 con 0x7f0cb40adcc0 2023-11-26T07:19:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb9157640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4070950 con 0x7f0cb406dfd0 2023-11-26T07:19:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb359e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0cb40adcc0 0x7f0cb40ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb359e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0cb40adcc0 0x7f0cb40ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40860/0 (socket says 172.21.15.31:40860) 2023-11-26T07:19:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb359e640 1 -- 172.21.15.31:0/4290845613 learned_addr learned my addr 172.21.15.31:0/4290845613 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cabfff640 1 --2- 172.21.15.31:0/4290845613 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0cb406dfd0 0x7f0cb406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb359e640 1 -- 172.21.15.31:0/4290845613 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0cb406dfd0 msgr2=0x7f0cb406e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb359e640 1 --2- 172.21.15.31:0/4290845613 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0cb406dfd0 0x7f0cb406e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb359e640 1 -- 172.21.15.31:0/4290845613 >> v1:172.21.15.97:6789/0 conn(0x7f0cb4143ae0 legacy=0x7f0cb40b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:17.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.367+0000 7f0cb359e640 1 -- 172.21.15.31:0/4290845613 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb4070c30 con 0x7f0cb40adcc0 2023-11-26T07:19:17.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb359e640 1 --2- 172.21.15.31:0/4290845613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb40adcc0 0x7f0cb40ae110 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009e50 tx=0x7f0cac01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba2beb104a1793a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb259c640 1 -- 172.21.15.31:0/4290845613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0a58a0 con 0x7f0cb40adcc0 2023-11-26T07:19:17.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb259c640 1 -- 172.21.15.31:0/4290845613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac0a5a00 con 0x7f0cb40adcc0 2023-11-26T07:19:17.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb259c640 1 -- 172.21.15.31:0/4290845613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0a5d90 con 0x7f0cb40adcc0 2023-11-26T07:19:17.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb9157640 1 -- 172.21.15.31:0/4290845613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb40adcc0 msgr2=0x7f0cb40ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb9157640 1 --2- 172.21.15.31:0/4290845613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb40adcc0 0x7f0cb40ae110 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009e50 tx=0x7f0cac01c9e0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb9157640 1 -- 172.21.15.31:0/4290845613 shutdown_connections 2023-11-26T07:19:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb9157640 1 --2- 172.21.15.31:0/4290845613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb40adcc0 0x7f0cb40ae110 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb9157640 1 --2- 172.21.15.31:0/4290845613 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0cb406dfd0 0x7f0cb406e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb9157640 1 -- 172.21.15.31:0/4290845613 >> 172.21.15.31:0/4290845613 conn(0x7f0cb40b0c40 msgr2=0x7f0cb40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb9157640 1 -- 172.21.15.31:0/4290845613 shutdown_connections 2023-11-26T07:19:17.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.371+0000 7f0cb9157640 1 -- 172.21.15.31:0/4290845613 wait complete. 2023-11-26T07:19:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 Processor -- start 2023-11-26T07:19:17.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 -- start start 2023-11-26T07:19:17.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0cb406dfd0 0x7f0cb415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40adcc0 0x7f0cb415a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4143ae0 0x7f0cb41581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb415b4e0 con 0x7f0cb4143ae0 2023-11-26T07:19:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb415af20 con 0x7f0cb40adcc0 2023-11-26T07:19:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb415b200 con 0x7f0cb406dfd0 2023-11-26T07:19:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4143ae0 0x7f0cb41581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4143ae0 0x7f0cb41581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40862/0 (socket says 172.21.15.31:40862) 2023-11-26T07:19:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb359e640 1 -- 172.21.15.31:0/3935792040 learned_addr learned my addr 172.21.15.31:0/3935792040 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:17.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cabfff640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40adcc0 0x7f0cb415a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0cb406dfd0 0x7f0cb415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb359e640 1 -- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0cb406dfd0 msgr2=0x7f0cb415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb359e640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0cb406dfd0 0x7f0cb415d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb359e640 1 -- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40adcc0 msgr2=0x7f0cb415a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb359e640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40adcc0 0x7f0cb415a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb359e640 1 -- 172.21.15.31:0/3935792040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac009b30 con 0x7f0cb4143ae0 2023-11-26T07:19:17.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb359e640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4143ae0 0x7f0cb41581c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0cac01a610 tx=0x7f0cac019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cab7fe640 1 -- 172.21.15.31:0/3935792040 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0b8670 con 0x7f0cb4143ae0 2023-11-26T07:19:17.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cab7fe640 1 -- 172.21.15.31:0/3935792040 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac0b87f0 con 0x7f0cb4143ae0 2023-11-26T07:19:17.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cab7fe640 1 -- 172.21.15.31:0/3935792040 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0b8ba0 con 0x7f0cb4143ae0 2023-11-26T07:19:17.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 -- 172.21.15.31:0/3935792040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb415cbc0 con 0x7f0cb4143ae0 2023-11-26T07:19:17.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 -- 172.21.15.31:0/3935792040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb415c8b0 con 0x7f0cb4143ae0 2023-11-26T07:19:17.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.375+0000 7f0cb9157640 1 -- 172.21.15.31:0/3935792040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb415d0e0 con 0x7f0cb4143ae0 2023-11-26T07:19:17.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.379+0000 7f0cab7fe640 1 -- 172.21.15.31:0/3935792040 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 6) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f0cac0b8d00 con 0x7f0cb4143ae0 2023-11-26T07:19:17.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.379+0000 7f0cab7fe640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c94066620 0x7f0c94068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:17.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.379+0000 7f0cab7fe640 1 -- 172.21.15.31:0/3935792040 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0cac11f850 con 0x7f0cb4143ae0 2023-11-26T07:19:17.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.379+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c94066620 0x7f0c94068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:17.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.379+0000 7f0cb2d9d640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c94066620 0x7f0c94068ae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4007fd0 tx=0x7f0ca4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:17.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.383+0000 7f0cab7fe640 1 -- 172.21.15.31:0/3935792040 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0cb415d0e0 con 0x7f0cb4143ae0 2023-11-26T07:19:17.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 -- 172.21.15.31:0/3935792040 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cb415d0e0 con 0x7f0cb4143ae0 2023-11-26T07:19:17.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cab7fe640 1 -- 172.21.15.31:0/3935792040 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f0cb415d0e0 con 0x7f0cb4143ae0 2023-11-26T07:19:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:17.582 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:19:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 -- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c94066620 msgr2=0x7f0c94068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c94066620 0x7f0c94068ae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4007fd0 tx=0x7f0ca4009040 comp rx=0 tx=0).stop 2023-11-26T07:19:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 -- 172.21.15.31:0/3935792040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4143ae0 msgr2=0x7f0cb41581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4143ae0 0x7f0cb41581c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0cac01a610 tx=0x7f0cac019780 comp rx=0 tx=0).stop 2023-11-26T07:19:17.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 -- 172.21.15.31:0/3935792040 shutdown_connections 2023-11-26T07:19:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c94066620 0x7f0c94068ae0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb4143ae0 0x7f0cb41581c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40adcc0 0x7f0cb415a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 --2- 172.21.15.31:0/3935792040 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0cb406dfd0 0x7f0cb415d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 -- 172.21.15.31:0/3935792040 >> 172.21.15.31:0/3935792040 conn(0x7f0cb40b0c40 msgr2=0x7f0cb40b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 -- 172.21.15.31:0/3935792040 shutdown_connections 2023-11-26T07:19:17.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:17.575+0000 7f0cb9157640 1 -- 172.21.15.31:0/3935792040 wait complete. 2023-11-26T07:19:17.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:19:17.765 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":1.1159881469999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.943982567602887,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:19:17.766 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.943982567602887 }, "state": "open", "uptime": 1.115988147 } ] 2023-11-26T07:19:17.766 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:19:22.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b2d1c640 1 Processor -- start 2023-11-26T07:19:22.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b2d1c640 1 -- start start 2023-11-26T07:19:22.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b2d1c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac15d7b0 0x7f16ac0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b2d1c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16ac0d2080 0x7f16ac06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b2d1c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac0cefd0 con 0x7f16ac0cead0 2023-11-26T07:19:22.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b2d1c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ac0cf2b0 con 0x7f16ac0d2080 2023-11-26T07:19:22.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b2d1c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac06b3f0 con 0x7f16ac15d7b0 2023-11-26T07:19:22.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b12b9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16ac0d2080 0x7f16ac06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16abfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f16ac0cead0 0x7f16ac0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36298/0 (socket says 172.21.15.31:36298) 2023-11-26T07:19:22.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16abfff640 1 -- 172.21.15.31:0/1165399565 learned_addr learned my addr 172.21.15.31:0/1165399565 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:22.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b0ab8640 1 --2- 172.21.15.31:0/1165399565 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac15d7b0 0x7f16ac0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b12b9640 1 -- 172.21.15.31:0/1165399565 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac15d7b0 msgr2=0x7f16ac0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:22.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b12b9640 1 --2- 172.21.15.31:0/1165399565 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac15d7b0 0x7f16ac0d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b12b9640 1 -- 172.21.15.31:0/1165399565 >> v1:172.21.15.31:6789/0 conn(0x7f16ac0cead0 legacy=0x7f16ac0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.915+0000 7f16b12b9640 1 -- 172.21.15.31:0/1165399565 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac06b6d0 con 0x7f16ac0d2080 2023-11-26T07:19:22.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16b12b9640 1 --2- 172.21.15.31:0/1165399565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac0d2080 0x7f16ac06bed0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009d90 tx=0x7f16a401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d873728aa4ffabb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:22.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16ab7fe640 1 -- 172.21.15.31:0/1165399565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40a4c90 con 0x7f16ac0d2080 2023-11-26T07:19:22.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16ab7fe640 1 -- 172.21.15.31:0/1165399565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a40a4df0 con 0x7f16ac0d2080 2023-11-26T07:19:22.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16ab7fe640 1 -- 172.21.15.31:0/1165399565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40ac660 con 0x7f16ac0d2080 2023-11-26T07:19:22.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16b2d1c640 1 -- 172.21.15.31:0/1165399565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac0d2080 msgr2=0x7f16ac06bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16b2d1c640 1 --2- 172.21.15.31:0/1165399565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac0d2080 0x7f16ac06bed0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009d90 tx=0x7f16a401c400 comp rx=0 tx=0).stop 2023-11-26T07:19:22.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16b2d1c640 1 -- 172.21.15.31:0/1165399565 shutdown_connections 2023-11-26T07:19:22.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16b2d1c640 1 --2- 172.21.15.31:0/1165399565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac0d2080 0x7f16ac06bed0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16b2d1c640 1 --2- 172.21.15.31:0/1165399565 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac15d7b0 0x7f16ac0d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16b2d1c640 1 -- 172.21.15.31:0/1165399565 >> 172.21.15.31:0/1165399565 conn(0x7f16ac0d33b0 msgr2=0x7f16ac0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:22.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16b2d1c640 1 -- 172.21.15.31:0/1165399565 shutdown_connections 2023-11-26T07:19:22.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.919+0000 7f16b2d1c640 1 -- 172.21.15.31:0/1165399565 wait complete. 2023-11-26T07:19:22.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b2d1c640 1 Processor -- start 2023-11-26T07:19:22.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b2d1c640 1 -- start start 2023-11-26T07:19:22.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b2d1c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0cead0 0x7f16ac0c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b2d1c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac0d2080 0x7f16ac0c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b2d1c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac15d7b0 0x7f16ac153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b2d1c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0c8aa0 con 0x7f16ac15d7b0 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b2d1c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac153e80 con 0x7f16ac0d2080 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b2d1c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0c74c0 con 0x7f16ac0cead0 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b12b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac15d7b0 0x7f16ac153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b12b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac15d7b0 0x7f16ac153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40878/0 (socket says 172.21.15.31:40878) 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b12b9640 1 -- 172.21.15.31:0/470947588 learned_addr learned my addr 172.21.15.31:0/470947588 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b0ab8640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0cead0 0x7f16ac0c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16abfff640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac0d2080 0x7f16ac0c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b12b9640 1 -- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0cead0 msgr2=0x7f16ac0c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b12b9640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0cead0 0x7f16ac0c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b12b9640 1 -- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac0d2080 msgr2=0x7f16ac0c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:22.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b12b9640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac0d2080 0x7f16ac0c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:22.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b12b9640 1 -- 172.21.15.31:0/470947588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a4009a70 con 0x7f16ac15d7b0 2023-11-26T07:19:22.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16abfff640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac0d2080 0x7f16ac0c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:22.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b12b9640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac15d7b0 0x7f16ac153940 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009d90 tx=0x7f16a40194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:22.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16a97fa640 1 -- 172.21.15.31:0/470947588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4007000 con 0x7f16ac15d7b0 2023-11-26T07:19:22.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16a97fa640 1 -- 172.21.15.31:0/470947588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a4009450 con 0x7f16ac15d7b0 2023-11-26T07:19:22.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b2d1c640 1 -- 172.21.15.31:0/470947588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac0c77a0 con 0x7f16ac15d7b0 2023-11-26T07:19:22.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16a97fa640 1 -- 172.21.15.31:0/470947588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a40a4c90 con 0x7f16ac15d7b0 2023-11-26T07:19:22.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.923+0000 7f16b2d1c640 1 -- 172.21.15.31:0/470947588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac0c7d60 con 0x7f16ac15d7b0 2023-11-26T07:19:22.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.927+0000 7f16b2d1c640 1 -- 172.21.15.31:0/470947588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ac0b4500 con 0x7f16ac15d7b0 2023-11-26T07:19:22.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.927+0000 7f16a97fa640 1 -- 172.21.15.31:0/470947588 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f16a40b3050 con 0x7f16ac15d7b0 2023-11-26T07:19:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.927+0000 7f16a97fa640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16900666b0 0x7f1690068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.927+0000 7f16a97fa640 1 -- 172.21.15.31:0/470947588 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f16a411e690 con 0x7f16ac15d7b0 2023-11-26T07:19:22.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.927+0000 7f16b0ab8640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16900666b0 0x7f1690068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:22.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.931+0000 7f16b0ab8640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16900666b0 0x7f1690068b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b9d0 tx=0x7f16a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:22.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:22.931+0000 7f16a97fa640 1 -- 172.21.15.31:0/470947588 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f16a40eb800 con 0x7f16ac15d7b0 2023-11-26T07:19:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 -- 172.21.15.31:0/470947588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16ac04e850 con 0x7f16ac15d7b0 2023-11-26T07:19:23.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16a97fa640 1 -- 172.21.15.31:0/470947588 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f16a40eee20 con 0x7f16ac15d7b0 2023-11-26T07:19:23.129 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:23.130 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:23.130 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:19:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 -- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16900666b0 msgr2=0x7f1690068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16900666b0 0x7f1690068b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f16a000b9d0 tx=0x7f16a0008040 comp rx=0 tx=0).stop 2023-11-26T07:19:23.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 -- 172.21.15.31:0/470947588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac15d7b0 msgr2=0x7f16ac153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac15d7b0 0x7f16ac153940 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009d90 tx=0x7f16a40194c0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 -- 172.21.15.31:0/470947588 shutdown_connections 2023-11-26T07:19:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f16900666b0 0x7f1690068b70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16ac15d7b0 0x7f16ac153940 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16ac0d2080 0x7f16ac0c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 --2- 172.21.15.31:0/470947588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16ac0cead0 0x7f16ac0c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.123+0000 7f16b2d1c640 1 -- 172.21.15.31:0/470947588 >> 172.21.15.31:0/470947588 conn(0x7f16ac0d33b0 msgr2=0x7f16ac0d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.127+0000 7f16b2d1c640 1 -- 172.21.15.31:0/470947588 shutdown_connections 2023-11-26T07:19:23.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:23.127+0000 7f16b2d1c640 1 -- 172.21.15.31:0/470947588 wait complete. 2023-11-26T07:19:23.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:19:23.314 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":6.6639293390000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.664489536485842,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:19:23.315 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.664489536485842 }, "state": "open", "uptime": 6.663929339 } ] 2023-11-26T07:19:23.315 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:19:28.316 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:28.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.463+0000 7faa625df640 1 Processor -- start 2023-11-26T07:19:28.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.463+0000 7faa625df640 1 -- start start 2023-11-26T07:19:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa625df640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa5c06d920 0x7faa5c070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa625df640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5c145060 con 0x7faa5c0ac630 2023-11-26T07:19:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa625df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5c071fd0 con 0x7faa5c144c80 2023-11-26T07:19:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa625df640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5c071450 con 0x7faa5c06d920 2023-11-26T07:19:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faa5c0ac630 0x7faa5c0706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:44366/0 (socket says 172.21.15.31:44366) 2023-11-26T07:19:28.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5b7fe640 1 -- 172.21.15.31:0/308004262 learned_addr learned my addr 172.21.15.31:0/308004262 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa60b7c640 1 --2- 172.21.15.31:0/308004262 >> v2:172.21.15.97:3300/0 conn(0x7faa5c06d920 0x7faa5c070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502288508 0 0) 0x7faa5c145060 con 0x7faa5c0ac630 2023-11-26T07:19:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa44003530 con 0x7faa5c0ac630 2023-11-26T07:19:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166523984 0 0) 0x7faa5c071fd0 con 0x7faa5c144c80 2023-11-26T07:19:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa5c145060 con 0x7faa5c144c80 2023-11-26T07:19:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1866342658 0 0) 0x7faa44003530 con 0x7faa5c0ac630 2023-11-26T07:19:28.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 >> v2:172.21.15.97:3300/0 conn(0x7faa5c06d920 msgr2=0x7faa5c070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 --2- 172.21.15.31:0/308004262 >> v2:172.21.15.97:3300/0 conn(0x7faa5c06d920 0x7faa5c070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 >> v1:172.21.15.97:6789/0 conn(0x7faa5c144c80 legacy=0x7faa5c0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c071730 con 0x7faa5c0ac630 2023-11-26T07:19:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7faa50001680 con 0x7faa5c0ac630 2023-11-26T07:19:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 >> v1:172.21.15.31:6789/0 conn(0x7faa5c0ac630 legacy=0x7faa5c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa44003b60 0x7faa44003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa440047d0 0x7faa44004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa44005190 0x7faa4400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c071450 con 0x7faa44003b60 2023-11-26T07:19:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4400c060 con 0x7faa44005190 2023-11-26T07:19:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4400c220 con 0x7faa440047d0 2023-11-26T07:19:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa60b7c640 1 --2- 172.21.15.31:0/308004262 >> v2:172.21.15.97:3300/0 conn(0x7faa5c06d920 0x7faa5c070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5bfff640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa44003b60 0x7faa44003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa60b7c640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa44005190 0x7faa4400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5b7fe640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa440047d0 0x7faa44004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5bfff640 1 -- 172.21.15.31:0/308004262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa440047d0 msgr2=0x7faa44004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5bfff640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa440047d0 0x7faa44004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5bfff640 1 -- 172.21.15.31:0/308004262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa44005190 msgr2=0x7faa4400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5bfff640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa44005190 0x7faa4400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa5bfff640 1 -- 172.21.15.31:0/308004262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa44003530 con 0x7faa44003b60 2023-11-26T07:19:28.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.467+0000 7faa60b7c640 1 -- 172.21.15.31:0/308004262 reap_dead start 2023-11-26T07:19:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa5bfff640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa44003b60 0x7faa44003f60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0018c0 tx=0x7faa4c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998cc788eccf31a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c0052d0 con 0x7faa44003b60 2023-11-26T07:19:28.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c0a7050 con 0x7faa44003b60 2023-11-26T07:19:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa5affd640 1 -- 172.21.15.31:0/308004262 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c003820 con 0x7faa44003b60 2023-11-26T07:19:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa625df640 1 -- 172.21.15.31:0/308004262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa44003b60 msgr2=0x7faa44003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa625df640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa44003b60 0x7faa44003f60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0018c0 tx=0x7faa4c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa625df640 1 -- 172.21.15.31:0/308004262 shutdown_connections 2023-11-26T07:19:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa625df640 1 --2- 172.21.15.31:0/308004262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa44003b60 0x7faa44003f60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa625df640 1 -- 172.21.15.31:0/308004262 >> 172.21.15.31:0/308004262 conn(0x7faa5c0b0c40 msgr2=0x7faa5c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa625df640 1 -- 172.21.15.31:0/308004262 shutdown_connections 2023-11-26T07:19:28.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.471+0000 7faa625df640 1 -- 172.21.15.31:0/308004262 wait complete. 2023-11-26T07:19:28.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 Processor -- start 2023-11-26T07:19:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 -- start start 2023-11-26T07:19:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa44003b60 0x7faa5c149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5c0af1a0 0x7faa5c13a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa5c099a30 0x7faa5c099eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c09a3f0 con 0x7faa5c099a30 2023-11-26T07:19:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c15af40 con 0x7faa5c0af1a0 2023-11-26T07:19:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5c1476f0 con 0x7faa44003b60 2023-11-26T07:19:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa60b7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa5c099a30 0x7faa5c099eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa60b7c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa5c099a30 0x7faa5c099eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55182/0 (socket says 172.21.15.31:55182) 2023-11-26T07:19:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa60b7c640 1 -- 172.21.15.31:0/2249443854 learned_addr learned my addr 172.21.15.31:0/2249443854 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa5b7fe640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5c0af1a0 0x7faa5c13a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa5bfff640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa44003b60 0x7faa5c149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa60b7c640 1 -- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa44003b60 msgr2=0x7faa5c149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa60b7c640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa44003b60 0x7faa5c149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa60b7c640 1 -- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5c0af1a0 msgr2=0x7faa5c13a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa60b7c640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5c0af1a0 0x7faa5c13a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa60b7c640 1 -- 172.21.15.31:0/2249443854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c0af620 con 0x7faa5c099a30 2023-11-26T07:19:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa60b7c640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa5c099a30 0x7faa5c099eb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faa5c06ddb0 tx=0x7faa540115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa58ff9640 1 -- 172.21.15.31:0/2249443854 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5400e070 con 0x7faa5c099a30 2023-11-26T07:19:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa58ff9640 1 -- 172.21.15.31:0/2249443854 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa540039a0 con 0x7faa5c099a30 2023-11-26T07:19:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 -- 172.21.15.31:0/2249443854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5c149c10 con 0x7faa5c099a30 2023-11-26T07:19:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa58ff9640 1 -- 172.21.15.31:0/2249443854 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54003b50 con 0x7faa5c099a30 2023-11-26T07:19:28.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 -- 172.21.15.31:0/2249443854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5c14a1d0 con 0x7faa5c099a30 2023-11-26T07:19:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.475+0000 7faa625df640 1 -- 172.21.15.31:0/2249443854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5c0b1ba0 con 0x7faa5c099a30 2023-11-26T07:19:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.479+0000 7faa58ff9640 1 -- 172.21.15.31:0/2249443854 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7faa54003cb0 con 0x7faa5c099a30 2023-11-26T07:19:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.479+0000 7faa58ff9640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faa480666b0 0x7faa48068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.479+0000 7faa58ff9640 1 -- 172.21.15.31:0/2249443854 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7faa54084d80 con 0x7faa5c099a30 2023-11-26T07:19:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.479+0000 7faa5bfff640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faa480666b0 0x7faa48068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.479+0000 7faa5bfff640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faa480666b0 0x7faa48068b70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0ac000 tx=0x7faa4c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.483+0000 7faa58ff9640 1 -- 172.21.15.31:0/2249443854 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faa54051d60 con 0x7faa5c099a30 2023-11-26T07:19:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.671+0000 7faa625df640 1 -- 172.21.15.31:0/2249443854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa5c147860 con 0x7faa5c099a30 2023-11-26T07:19:28.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.671+0000 7faa58ff9640 1 -- 172.21.15.31:0/2249443854 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7faa54055380 con 0x7faa5c099a30 2023-11-26T07:19:28.678 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:28.679 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:28.679 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:19:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 -- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faa480666b0 msgr2=0x7faa48068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faa480666b0 0x7faa48068b70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0ac000 tx=0x7faa4c009040 comp rx=0 tx=0).stop 2023-11-26T07:19:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 -- 172.21.15.31:0/2249443854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa5c099a30 msgr2=0x7faa5c099eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa5c099a30 0x7faa5c099eb0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faa5c06ddb0 tx=0x7faa540115a0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 -- 172.21.15.31:0/2249443854 shutdown_connections 2023-11-26T07:19:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faa480666b0 0x7faa48068b70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa5c099a30 0x7faa5c099eb0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa5c0af1a0 0x7faa5c13a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 --2- 172.21.15.31:0/2249443854 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa44003b60 0x7faa5c149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 -- 172.21.15.31:0/2249443854 >> 172.21.15.31:0/2249443854 conn(0x7faa5c0b0c40 msgr2=0x7faa5c0cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 -- 172.21.15.31:0/2249443854 shutdown_connections 2023-11-26T07:19:28.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:28.675+0000 7faa625df640 1 -- 172.21.15.31:0/2249443854 wait complete. 2023-11-26T07:19:28.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:19:28.863 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.211870712,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.388556303956893,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:19:28.863 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.388556303956893 }, "state": "open", "uptime": 12.211870712 } ] 2023-11-26T07:19:28.863 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:19:30.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:19:30.289 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:19:33.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:34.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ecbb640 1 Processor -- start 2023-11-26T07:19:34.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ecbb640 1 -- start start 2023-11-26T07:19:34.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ecbb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9938143ae0 0x7f99380b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ecbb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f993806dfd0 0x7f993806e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ecbb640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99380b2480 con 0x7f99380adcc0 2023-11-26T07:19:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ecbb640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99380ae820 con 0x7f9938143ae0 2023-11-26T07:19:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ecbb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993806e950 con 0x7f993806dfd0 2023-11-26T07:19:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993d258640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f99380adcc0 0x7f99380ae110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60788/0 (socket says 172.21.15.31:60788) 2023-11-26T07:19:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ca57640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9938143ae0 0x7f99380b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993d258640 1 -- 172.21.15.31:0/856405019 learned_addr learned my addr 172.21.15.31:0/856405019 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:34.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f9937fff640 1 --2- 172.21.15.31:0/856405019 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f993806dfd0 0x7f993806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ca57640 1 -- 172.21.15.31:0/856405019 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f993806dfd0 msgr2=0x7f993806e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:34.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ca57640 1 --2- 172.21.15.31:0/856405019 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f993806dfd0 0x7f993806e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ca57640 1 -- 172.21.15.31:0/856405019 >> v1:172.21.15.31:6789/0 conn(0x7f99380adcc0 legacy=0x7f99380ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f993ca57640 1 -- 172.21.15.31:0/856405019 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9938070670 con 0x7f9938143ae0 2023-11-26T07:19:34.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.019+0000 7f99377fe640 1 -- 172.21.15.31:0/856405019 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374765654 0 0) 0x7f99380b2480 con 0x7f99380adcc0 2023-11-26T07:19:34.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.023+0000 7f993ca57640 1 --2- 172.21.15.31:0/856405019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9938143ae0 0x7f99380b1f40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f992c0097c0 tx=0x7f992c01e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c3ed388d09e797 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:34.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.023+0000 7f99377fe640 1 -- 172.21.15.31:0/856405019 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0a8630 con 0x7f9938143ae0 2023-11-26T07:19:34.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.023+0000 7f99377fe640 1 -- 172.21.15.31:0/856405019 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c0a8790 con 0x7f9938143ae0 2023-11-26T07:19:34.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.023+0000 7f99377fe640 1 -- 172.21.15.31:0/856405019 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0a8b20 con 0x7f9938143ae0 2023-11-26T07:19:34.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 -- 172.21.15.31:0/856405019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9938143ae0 msgr2=0x7f99380b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 --2- 172.21.15.31:0/856405019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9938143ae0 0x7f99380b1f40 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f992c0097c0 tx=0x7f992c01e660 comp rx=0 tx=0).stop 2023-11-26T07:19:34.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 -- 172.21.15.31:0/856405019 shutdown_connections 2023-11-26T07:19:34.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 --2- 172.21.15.31:0/856405019 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f993806dfd0 0x7f993806e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 --2- 172.21.15.31:0/856405019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9938143ae0 0x7f99380b1f40 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 -- 172.21.15.31:0/856405019 >> 172.21.15.31:0/856405019 conn(0x7f99380b0c40 msgr2=0x7f99380b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:34.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 -- 172.21.15.31:0/856405019 shutdown_connections 2023-11-26T07:19:34.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 -- 172.21.15.31:0/856405019 wait complete. 2023-11-26T07:19:34.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 Processor -- start 2023-11-26T07:19:34.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 -- start start 2023-11-26T07:19:34.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f993806dfd0 0x7f993813df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99380adcc0 0x7f99381317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9938143ae0 0x7f99380b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993815a690 con 0x7f993806dfd0 2023-11-26T07:19:34.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9938131d00 con 0x7f99380adcc0 2023-11-26T07:19:34.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ecbb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99380b3be0 con 0x7f9938143ae0 2023-11-26T07:19:34.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ca57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f993806dfd0 0x7f993813df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ca57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f993806dfd0 0x7f993813df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43062/0 (socket says 172.21.15.31:43062) 2023-11-26T07:19:34.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f993ca57640 1 -- 172.21.15.31:0/3291903333 learned_addr learned my addr 172.21.15.31:0/3291903333 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:34.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.027+0000 7f9937fff640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99380adcc0 0x7f99381317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f993d258640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9938143ae0 0x7f99380b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f993ca57640 1 -- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9938143ae0 msgr2=0x7f99380b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f993ca57640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9938143ae0 0x7f99380b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f993ca57640 1 -- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99380adcc0 msgr2=0x7f99381317c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f993ca57640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99380adcc0 0x7f99381317c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f993ca57640 1 -- 172.21.15.31:0/3291903333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c009420 con 0x7f993806dfd0 2023-11-26T07:19:34.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f993ca57640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f993806dfd0 0x7f993813df10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f992c01cd10 tx=0x7f992c01b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:34.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f99357fa640 1 -- 172.21.15.31:0/3291903333 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0b3650 con 0x7f993806dfd0 2023-11-26T07:19:34.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f99357fa640 1 -- 172.21.15.31:0/3291903333 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c0b37b0 con 0x7f993806dfd0 2023-11-26T07:19:34.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f99357fa640 1 -- 172.21.15.31:0/3291903333 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0b3b40 con 0x7f993806dfd0 2023-11-26T07:19:34.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f993ecbb640 1 -- 172.21.15.31:0/3291903333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993815a8e0 con 0x7f993806dfd0 2023-11-26T07:19:34.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f993ecbb640 1 -- 172.21.15.31:0/3291903333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993815aea0 con 0x7f993806dfd0 2023-11-26T07:19:34.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f993ecbb640 1 -- 172.21.15.31:0/3291903333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993815b090 con 0x7f993806dfd0 2023-11-26T07:19:34.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.031+0000 7f99357fa640 1 -- 172.21.15.31:0/3291903333 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f992c0b3ca0 con 0x7f993806dfd0 2023-11-26T07:19:34.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.035+0000 7f99357fa640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f991c0666b0 0x7f991c068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:34.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.035+0000 7f99357fa640 1 -- 172.21.15.31:0/3291903333 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f992c11e640 con 0x7f993806dfd0 2023-11-26T07:19:34.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.035+0000 7f9937fff640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f991c0666b0 0x7f991c068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:34.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.035+0000 7f9937fff640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f991c0666b0 0x7f991c068b70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f993815d0a0 tx=0x7f9924009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:34.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.039+0000 7f99357fa640 1 -- 172.21.15.31:0/3291903333 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f992c0eb730 con 0x7f993806dfd0 2023-11-26T07:19:34.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.227+0000 7f993ecbb640 1 -- 172.21.15.31:0/3291903333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f993804e850 con 0x7f993806dfd0 2023-11-26T07:19:34.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.227+0000 7f99357fa640 1 -- 172.21.15.31:0/3291903333 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f992c0eed50 con 0x7f993806dfd0 2023-11-26T07:19:34.236 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:34.236 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:34.237 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:19:34.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 -- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f991c0666b0 msgr2=0x7f991c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f991c0666b0 0x7f991c068b70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f993815d0a0 tx=0x7f9924009190 comp rx=0 tx=0).stop 2023-11-26T07:19:34.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 -- 172.21.15.31:0/3291903333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f993806dfd0 msgr2=0x7f993813df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:34.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f993806dfd0 0x7f993813df10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f992c01cd10 tx=0x7f992c01b400 comp rx=0 tx=0).stop 2023-11-26T07:19:34.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 -- 172.21.15.31:0/3291903333 shutdown_connections 2023-11-26T07:19:34.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f991c0666b0 0x7f991c068b70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9938143ae0 0x7f99380b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99380adcc0 0x7f99381317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 --2- 172.21.15.31:0/3291903333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f993806dfd0 0x7f993813df10 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:34.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 -- 172.21.15.31:0/3291903333 >> 172.21.15.31:0/3291903333 conn(0x7f99380b0c40 msgr2=0x7f9938141410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:34.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 -- 172.21.15.31:0/3291903333 shutdown_connections 2023-11-26T07:19:34.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:34.231+0000 7f993ecbb640 1 -- 172.21.15.31:0/3291903333 wait complete. 2023-11-26T07:19:34.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:19:34.423 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":17.771812139000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.115552080966278,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:19:34.424 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.11555208096628 }, "state": "open", "uptime": 17.771812139 } ] 2023-11-26T07:19:34.424 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:19:39.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f5500971640 1 Processor -- start 2023-11-26T07:19:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f5500971640 1 -- start start 2023-11-26T07:19:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f5500971640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54fc15d880 0x7f54fc0b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f5500971640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0b6c70 0x7f54fc0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f5500971640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc0d15f0 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f5500971640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc0d4110 con 0x7f54fc15d880 2023-11-26T07:19:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f5500971640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0ce000 con 0x7f54fc0b6c70 2023-11-26T07:19:39.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f54fa59c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0b6c70 0x7f54fc0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:39.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f54fad9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f54fc15d880 0x7f54fc0b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:39.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f54fa59c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0b6c70 0x7f54fc0d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:53548/0 (socket says 172.21.15.31:53548) 2023-11-26T07:19:39.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f54fa59c640 1 -- 172.21.15.31:0/318890343 learned_addr learned my addr 172.21.15.31:0/318890343 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:39.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f54fa59c640 1 -- 172.21.15.31:0/318890343 >> v2:172.21.15.97:3300/0 conn(0x7f54fc15d880 msgr2=0x7f54fc0b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:39.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f54fa59c640 1 --2- 172.21.15.31:0/318890343 >> v2:172.21.15.97:3300/0 conn(0x7f54fc15d880 0x7f54fc0b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:39.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f54fa59c640 1 -- 172.21.15.31:0/318890343 >> v1:172.21.15.97:6789/0 conn(0x7f54fc0d3ca0 legacy=0x7f54fc0cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:39.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.575+0000 7f54fa59c640 1 -- 172.21.15.31:0/318890343 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0d0630 con 0x7f54fc0b6c70 2023-11-26T07:19:39.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f54fa59c640 1 --2- 172.21.15.31:0/318890343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0b6c70 0x7f54fc0d3760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009870 tx=0x7f54f001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88c5830b9b291ba9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:39.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f54f9d9b640 1 -- 172.21.15.31:0/318890343 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00a9400 con 0x7f54fc0b6c70 2023-11-26T07:19:39.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f54f9d9b640 1 -- 172.21.15.31:0/318890343 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f00a9560 con 0x7f54fc0b6c70 2023-11-26T07:19:39.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f54f9d9b640 1 -- 172.21.15.31:0/318890343 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00a98f0 con 0x7f54fc0b6c70 2023-11-26T07:19:39.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f5500971640 1 -- 172.21.15.31:0/318890343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0b6c70 msgr2=0x7f54fc0d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:39.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f5500971640 1 --2- 172.21.15.31:0/318890343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0b6c70 0x7f54fc0d3760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009870 tx=0x7f54f001c540 comp rx=0 tx=0).stop 2023-11-26T07:19:39.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f5500971640 1 -- 172.21.15.31:0/318890343 shutdown_connections 2023-11-26T07:19:39.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f5500971640 1 --2- 172.21.15.31:0/318890343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0b6c70 0x7f54fc0d3760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:39.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f5500971640 1 --2- 172.21.15.31:0/318890343 >> v2:172.21.15.97:3300/0 conn(0x7f54fc15d880 0x7f54fc0b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:39.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f5500971640 1 -- 172.21.15.31:0/318890343 >> 172.21.15.31:0/318890343 conn(0x7f54fc06b9f0 msgr2=0x7f54fc06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:39.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f5500971640 1 -- 172.21.15.31:0/318890343 shutdown_connections 2023-11-26T07:19:39.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.579+0000 7f5500971640 1 -- 172.21.15.31:0/318890343 wait complete. 2023-11-26T07:19:39.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f5500971640 1 Processor -- start 2023-11-26T07:19:39.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f5500971640 1 -- start start 2023-11-26T07:19:39.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f5500971640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0b6c70 0x7f54fc0c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:39.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f5500971640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0d3ca0 0x7f54fc0cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:39.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f5500971640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc15d880 0x7f54fc15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:39.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f5500971640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0cbaf0 con 0x7f54fc15d880 2023-11-26T07:19:39.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f5500971640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0cb810 con 0x7f54fc0b6c70 2023-11-26T07:19:39.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f5500971640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc159110 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc15d880 0x7f54fc15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:39.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fa59c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0d3ca0 0x7f54fc0cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:39.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0b6c70 0x7f54fc0c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:39.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc15d880 0x7f54fc15bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43074/0 (socket says 172.21.15.31:43074) 2023-11-26T07:19:39.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fa59c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0d3ca0 0x7f54fc0cad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:53550/0 (socket says 172.21.15.31:53550) 2023-11-26T07:19:39.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fb59e640 1 -- 172.21.15.31:0/1790502567 learned_addr learned my addr 172.21.15.31:0/1790502567 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:39.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fa59c640 1 -- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0b6c70 msgr2=0x7f54fc0c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:39.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fa59c640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0b6c70 0x7f54fc0c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:39.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fa59c640 1 -- 172.21.15.31:0/1790502567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc15d880 msgr2=0x7f54fc15bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:39.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fa59c640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc15d880 0x7f54fc15bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:39.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fa59c640 1 -- 172.21.15.31:0/1790502567 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f00094d0 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fb59e640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc15d880 0x7f54fc15bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:19:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54fa59c640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0d3ca0 0x7f54fc0cad50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54f00aa000 tx=0x7f54f001ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54e37fe640 1 -- 172.21.15.31:0/1790502567 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00a9400 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54e37fe640 1 -- 172.21.15.31:0/1790502567 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f00b1820 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f54e37fe640 1 -- 172.21.15.31:0/1790502567 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f00b1bd0 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f5500971640 1 -- 172.21.15.31:0/1790502567 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc1593f0 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.583+0000 7f5500971640 1 -- 172.21.15.31:0/1790502567 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc0c9370 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.587+0000 7f5500971640 1 -- 172.21.15.31:0/1790502567 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc158f70 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.587+0000 7f54e37fe640 1 -- 172.21.15.31:0/1790502567 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f54f00ba3a0 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.587+0000 7f54e37fe640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f54e80666b0 0x7f54e8068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:39.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.587+0000 7f54e37fe640 1 -- 172.21.15.31:0/1790502567 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f54f011e610 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.587+0000 7f54fad9d640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f54e80666b0 0x7f54e8068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:39.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.587+0000 7f54fad9d640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f54e80666b0 0x7f54e8068b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009ac0 tx=0x7f54ec009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:39.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.591+0000 7f54e37fe640 1 -- 172.21.15.31:0/1790502567 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f54fc158f70 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.779+0000 7f5500971640 1 -- 172.21.15.31:0/1790502567 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54fc158f70 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.779+0000 7f54e37fe640 1 -- 172.21.15.31:0/1790502567 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f54fc158f70 con 0x7f54fc0d3ca0 2023-11-26T07:19:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:39.787 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:39.787 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:19:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 -- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f54e80666b0 msgr2=0x7f54e8068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f54e80666b0 0x7f54e8068b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009ac0 tx=0x7f54ec009190 comp rx=0 tx=0).stop 2023-11-26T07:19:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 -- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0d3ca0 msgr2=0x7f54fc0cad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0d3ca0 0x7f54fc0cad50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54f00aa000 tx=0x7f54f001ad80 comp rx=0 tx=0).stop 2023-11-26T07:19:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 -- 172.21.15.31:0/1790502567 shutdown_connections 2023-11-26T07:19:39.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f54e80666b0 0x7f54e8068b70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc15d880 0x7f54fc15bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc0d3ca0 0x7f54fc0cad50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 --2- 172.21.15.31:0/1790502567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc0b6c70 0x7f54fc0c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 -- 172.21.15.31:0/1790502567 >> 172.21.15.31:0/1790502567 conn(0x7f54fc06b9f0 msgr2=0x7f54fc1605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 -- 172.21.15.31:0/1790502567 shutdown_connections 2023-11-26T07:19:39.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:39.783+0000 7f5500971640 1 -- 172.21.15.31:0/1790502567 wait complete. 2023-11-26T07:19:39.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:19:39.972 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":23.323753814,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.846417800802634,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:19:39.972 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.846417800802634 }, "state": "open", "uptime": 23.323753814 } ] 2023-11-26T07:19:39.972 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:19:44.973 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:45.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd04c8ea640 1 Processor -- start 2023-11-26T07:19:45.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd04c8ea640 1 -- start start 2023-11-26T07:19:45.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd04c8ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 0x7fd0480d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd04c8ea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0480cead0 0x7fd0480d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd04c8ea640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0480d1eb0 0x7fd0480cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd04c8ea640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0480d2340 con 0x7fd0480d1eb0 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd04c8ea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0480cefd0 con 0x7fd0480cead0 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd04c8ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480cf2b0 con 0x7fd04815d7b0 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd046d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0480d1eb0 0x7fd0480cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd046d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0480d1eb0 0x7fd0480cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44180/0 (socket says 172.21.15.31:44180) 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd046d9d640 1 -- 172.21.15.31:0/1077572474 learned_addr learned my addr 172.21.15.31:0/1077572474 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd04659c640 1 --2- 172.21.15.31:0/1077572474 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 0x7fd0480d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd03ffff640 1 --2- 172.21.15.31:0/1077572474 >> v2:172.21.15.97:3300/0 conn(0x7fd0480cead0 0x7fd0480d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd046d9d640 1 -- 172.21.15.31:0/1077572474 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 msgr2=0x7fd0480d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:45.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd046d9d640 1 --2- 172.21.15.31:0/1077572474 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 0x7fd0480d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd046d9d640 1 -- 172.21.15.31:0/1077572474 >> v2:172.21.15.97:3300/0 conn(0x7fd0480cead0 msgr2=0x7fd0480d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:45.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd046d9d640 1 --2- 172.21.15.31:0/1077572474 >> v2:172.21.15.97:3300/0 conn(0x7fd0480cead0 0x7fd0480d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.123+0000 7fd046d9d640 1 -- 172.21.15.31:0/1077572474 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04806b3f0 con 0x7fd0480d1eb0 2023-11-26T07:19:45.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd046d9d640 1 --2- 172.21.15.31:0/1077572474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 0x7fd0480cfc80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd04808c310 tx=0x7fd04001c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ac5669520b5f25 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:45.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd045d9b640 1 -- 172.21.15.31:0/1077572474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400a6450 con 0x7fd0480d1eb0 2023-11-26T07:19:45.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd045d9b640 1 -- 172.21.15.31:0/1077572474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400a65b0 con 0x7fd0480d1eb0 2023-11-26T07:19:45.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd045d9b640 1 -- 172.21.15.31:0/1077572474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400a6940 con 0x7fd0480d1eb0 2023-11-26T07:19:45.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd04c8ea640 1 -- 172.21.15.31:0/1077572474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 msgr2=0x7fd0480cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:45.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd04c8ea640 1 --2- 172.21.15.31:0/1077572474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 0x7fd0480cfc80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd04808c310 tx=0x7fd04001c5c0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd04c8ea640 1 -- 172.21.15.31:0/1077572474 shutdown_connections 2023-11-26T07:19:45.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd04c8ea640 1 --2- 172.21.15.31:0/1077572474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 0x7fd0480cfc80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd04c8ea640 1 --2- 172.21.15.31:0/1077572474 >> v2:172.21.15.97:3300/0 conn(0x7fd0480cead0 0x7fd0480d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd04c8ea640 1 --2- 172.21.15.31:0/1077572474 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 0x7fd0480d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd04c8ea640 1 -- 172.21.15.31:0/1077572474 >> 172.21.15.31:0/1077572474 conn(0x7fd0480d33b0 msgr2=0x7fd0480d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:45.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd04c8ea640 1 -- 172.21.15.31:0/1077572474 shutdown_connections 2023-11-26T07:19:45.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.127+0000 7fd04c8ea640 1 -- 172.21.15.31:0/1077572474 wait complete. 2023-11-26T07:19:45.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04c8ea640 1 Processor -- start 2023-11-26T07:19:45.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04c8ea640 1 -- start start 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04c8ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0480cead0 0x7fd0480b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04c8ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 0x7fd0480cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04c8ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 0x7fd0481554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04c8ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480cbec0 con 0x7fd0480d1eb0 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04c8ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480cc1a0 con 0x7fd0480cead0 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04c8ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480cc480 con 0x7fd04815d7b0 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 0x7fd0480cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 0x7fd0480cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44182/0 (socket says 172.21.15.31:44182) 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03ffff640 1 -- 172.21.15.31:0/4019310137 learned_addr learned my addr 172.21.15.31:0/4019310137 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd046d9d640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 0x7fd0481554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04659c640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0480cead0 0x7fd0480b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03ffff640 1 -- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 msgr2=0x7fd0481554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03ffff640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 0x7fd0481554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03ffff640 1 -- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0480cead0 msgr2=0x7fd0480b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03ffff640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0480cead0 0x7fd0480b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03ffff640 1 -- 172.21.15.31:0/4019310137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd040009b30 con 0x7fd0480d1eb0 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd046d9d640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 0x7fd0481554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04659c640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0480cead0 0x7fd0480b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03ffff640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 0x7fd0480cb470 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd03000d800 tx=0x7fd03000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03effd640 1 -- 172.21.15.31:0/4019310137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000b6c0 con 0x7fd0480d1eb0 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03effd640 1 -- 172.21.15.31:0/4019310137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03000b820 con 0x7fd0480d1eb0 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd03effd640 1 -- 172.21.15.31:0/4019310137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000bb90 con 0x7fd0480d1eb0 2023-11-26T07:19:45.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04c8ea640 1 -- 172.21.15.31:0/4019310137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0480cc760 con 0x7fd0480d1eb0 2023-11-26T07:19:45.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.131+0000 7fd04c8ea640 1 -- 172.21.15.31:0/4019310137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0480b8ff0 con 0x7fd0480d1eb0 2023-11-26T07:19:45.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.135+0000 7fd04c8ea640 1 -- 172.21.15.31:0/4019310137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04804e850 con 0x7fd0480d1eb0 2023-11-26T07:19:45.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.135+0000 7fd03effd640 1 -- 172.21.15.31:0/4019310137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fd03000bcf0 con 0x7fd0480d1eb0 2023-11-26T07:19:45.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.135+0000 7fd03effd640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0280666b0 0x7fd028068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:45.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.135+0000 7fd03effd640 1 -- 172.21.15.31:0/4019310137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd0300849b0 con 0x7fd0480d1eb0 2023-11-26T07:19:45.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.135+0000 7fd04659c640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0280666b0 0x7fd028068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:45.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.135+0000 7fd04659c640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0280666b0 0x7fd028068b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd03800b970 tx=0x7fd038008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:45.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.139+0000 7fd03effd640 1 -- 172.21.15.31:0/4019310137 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd030051960 con 0x7fd0480d1eb0 2023-11-26T07:19:45.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 -- 172.21.15.31:0/4019310137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0480b2210 con 0x7fd0480d1eb0 2023-11-26T07:19:45.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd03effd640 1 -- 172.21.15.31:0/4019310137 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd030011340 con 0x7fd0480d1eb0 2023-11-26T07:19:45.337 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:45.337 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:45.338 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:19:45.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 -- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0280666b0 msgr2=0x7fd028068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:45.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0280666b0 0x7fd028068b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd03800b970 tx=0x7fd038008040 comp rx=0 tx=0).stop 2023-11-26T07:19:45.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 -- 172.21.15.31:0/4019310137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 msgr2=0x7fd0480cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:45.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 0x7fd0480cb470 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd03000d800 tx=0x7fd03000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 -- 172.21.15.31:0/4019310137 shutdown_connections 2023-11-26T07:19:45.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0280666b0 0x7fd028068b70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd04815d7b0 0x7fd0481554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0480d1eb0 0x7fd0480cb470 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 --2- 172.21.15.31:0/4019310137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0480cead0 0x7fd0480b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:45.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 -- 172.21.15.31:0/4019310137 >> 172.21.15.31:0/4019310137 conn(0x7fd0480d33b0 msgr2=0x7fd04815c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:45.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 -- 172.21.15.31:0/4019310137 shutdown_connections 2023-11-26T07:19:45.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:45.331+0000 7fd04c8ea640 1 -- 172.21.15.31:0/4019310137 wait complete. 2023-11-26T07:19:45.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:19:45.522 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":28.871695715000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.580904044539089,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:19:45.522 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.58090404453909 }, "state": "open", "uptime": 28.871695715 } ] 2023-11-26T07:19:45.522 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:19:50.523 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:50.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f8661580640 1 Processor -- start 2023-11-26T07:19:50.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f8661580640 1 -- start start 2023-11-26T07:19:50.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f8661580640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f865c143a30 0x7f865c0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f8661580640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f865c06cd20 0x7f865c070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f8661580640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c070820 con 0x7f865c0ae990 2023-11-26T07:19:50.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f8661580640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c0ce500 con 0x7f865c143a30 2023-11-26T07:19:50.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f8661580640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c0ce7e0 con 0x7f865c06cd20 2023-11-26T07:19:50.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f865a7fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f865c0ae990 0x7f865c06c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:51722/0 (socket says 172.21.15.31:51722) 2023-11-26T07:19:50.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f865a7fc640 1 -- 172.21.15.31:0/1583008196 learned_addr learned my addr 172.21.15.31:0/1583008196 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:19:50.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f865affd640 1 --2- 172.21.15.31:0/1583008196 >> v2:172.21.15.31:3300/0 conn(0x7f865c143a30 0x7f865c0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f865b7fe640 1 --2- 172.21.15.31:0/1583008196 >> v2:172.21.15.97:3300/0 conn(0x7f865c06cd20 0x7f865c070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f865affd640 1 -- 172.21.15.31:0/1583008196 >> v2:172.21.15.97:3300/0 conn(0x7f865c06cd20 msgr2=0x7f865c070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:50.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f865affd640 1 --2- 172.21.15.31:0/1583008196 >> v2:172.21.15.97:3300/0 conn(0x7f865c06cd20 0x7f865c070cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f865affd640 1 -- 172.21.15.31:0/1583008196 >> v1:172.21.15.31:6789/0 conn(0x7f865c0ae990 legacy=0x7f865c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f865affd640 1 -- 172.21.15.31:0/1583008196 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c0ceac0 con 0x7f865c143a30 2023-11-26T07:19:50.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.663+0000 7f8659ffb640 1 -- 172.21.15.31:0/1583008196 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606615161 0 0) 0x7f865c070820 con 0x7f865c0ae990 2023-11-26T07:19:50.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f865affd640 1 --2- 172.21.15.31:0/1583008196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c143a30 0x7f865c0cda40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f865c15ee30 tx=0x7f864c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03464c4beb976d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:50.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8659ffb640 1 -- 172.21.15.31:0/1583008196 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0a8050 con 0x7f865c143a30 2023-11-26T07:19:50.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8659ffb640 1 -- 172.21.15.31:0/1583008196 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c0ac5f0 con 0x7f865c143a30 2023-11-26T07:19:50.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8659ffb640 1 -- 172.21.15.31:0/1583008196 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0ac980 con 0x7f865c143a30 2023-11-26T07:19:50.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8661580640 1 -- 172.21.15.31:0/1583008196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c143a30 msgr2=0x7f865c0cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8661580640 1 --2- 172.21.15.31:0/1583008196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c143a30 0x7f865c0cda40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f865c15ee30 tx=0x7f864c01c460 comp rx=0 tx=0).stop 2023-11-26T07:19:50.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8661580640 1 -- 172.21.15.31:0/1583008196 shutdown_connections 2023-11-26T07:19:50.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8661580640 1 --2- 172.21.15.31:0/1583008196 >> v2:172.21.15.97:3300/0 conn(0x7f865c06cd20 0x7f865c070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8661580640 1 --2- 172.21.15.31:0/1583008196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c143a30 0x7f865c0cda40 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8661580640 1 -- 172.21.15.31:0/1583008196 >> 172.21.15.31:0/1583008196 conn(0x7f865c06a5b0 msgr2=0x7f865c06a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:50.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8661580640 1 -- 172.21.15.31:0/1583008196 shutdown_connections 2023-11-26T07:19:50.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.667+0000 7f8661580640 1 -- 172.21.15.31:0/1583008196 wait complete. 2023-11-26T07:19:50.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 Processor -- start 2023-11-26T07:19:50.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 -- start start 2023-11-26T07:19:50.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c06cd20 0x7f865c13ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0ae990 0x7f865c154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c143a30 0x7f865c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c152a70 con 0x7f865c0ae990 2023-11-26T07:19:50.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c152d50 con 0x7f865c143a30 2023-11-26T07:19:50.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c153030 con 0x7f865c06cd20 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0ae990 0x7f865c154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0ae990 0x7f865c154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44196/0 (socket says 172.21.15.31:44196) 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865a7fc640 1 -- 172.21.15.31:0/949888665 learned_addr learned my addr 172.21.15.31:0/949888665 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865b7fe640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c143a30 0x7f865c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865affd640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c06cd20 0x7f865c13ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865a7fc640 1 -- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c06cd20 msgr2=0x7f865c13ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865a7fc640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c06cd20 0x7f865c13ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865a7fc640 1 -- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c143a30 msgr2=0x7f865c158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865a7fc640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c143a30 0x7f865c158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865a7fc640 1 -- 172.21.15.31:0/949888665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c009420 con 0x7f865c0ae990 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865b7fe640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c143a30 0x7f865c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:50.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865affd640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c06cd20 0x7f865c13ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:19:50.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f865a7fc640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0ae990 0x7f865c154d70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f86500011f0 tx=0x7f865000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:50.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f86437fe640 1 -- 172.21.15.31:0/949888665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86500074e0 con 0x7f865c0ae990 2023-11-26T07:19:50.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f86437fe640 1 -- 172.21.15.31:0/949888665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8650007640 con 0x7f865c0ae990 2023-11-26T07:19:50.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 -- 172.21.15.31:0/949888665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c153310 con 0x7f865c0ae990 2023-11-26T07:19:50.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f86437fe640 1 -- 172.21.15.31:0/949888665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650004a10 con 0x7f865c0ae990 2023-11-26T07:19:50.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 -- 172.21.15.31:0/949888665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c13b240 con 0x7f865c0ae990 2023-11-26T07:19:50.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.671+0000 7f8661580640 1 -- 172.21.15.31:0/949888665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c074a00 con 0x7f865c0ae990 2023-11-26T07:19:50.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.675+0000 7f86437fe640 1 -- 172.21.15.31:0/949888665 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f8650004b70 con 0x7f865c0ae990 2023-11-26T07:19:50.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.675+0000 7f86437fe640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86480666e0 0x7f8648068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:50.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.675+0000 7f86437fe640 1 -- 172.21.15.31:0/949888665 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f86500838d0 con 0x7f865c0ae990 2023-11-26T07:19:50.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.675+0000 7f865affd640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86480666e0 0x7f8648068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:50.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.675+0000 7f865affd640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86480666e0 0x7f8648068ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f864c0ae000 tx=0x7f864c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:50.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.679+0000 7f86437fe640 1 -- 172.21.15.31:0/949888665 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8650050910 con 0x7f865c0ae990 2023-11-26T07:19:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.871+0000 7f8661580640 1 -- 172.21.15.31:0/949888665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f865c1537c0 con 0x7f865c0ae990 2023-11-26T07:19:50.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.871+0000 7f86437fe640 1 -- 172.21.15.31:0/949888665 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f8650018020 con 0x7f865c0ae990 2023-11-26T07:19:50.879 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:50.879 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:19:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 -- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86480666e0 msgr2=0x7f8648068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86480666e0 0x7f8648068ba0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f864c0ae000 tx=0x7f864c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:19:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 -- 172.21.15.31:0/949888665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0ae990 msgr2=0x7f865c154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0ae990 0x7f865c154d70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f86500011f0 tx=0x7f865000c9b0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 -- 172.21.15.31:0/949888665 shutdown_connections 2023-11-26T07:19:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86480666e0 0x7f8648068ba0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f865c143a30 0x7f865c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f865c0ae990 0x7f865c154d70 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 --2- 172.21.15.31:0/949888665 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f865c06cd20 0x7f865c13ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 -- 172.21.15.31:0/949888665 >> 172.21.15.31:0/949888665 conn(0x7f865c06a5b0 msgr2=0x7f865c0b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 -- 172.21.15.31:0/949888665 shutdown_connections 2023-11-26T07:19:50.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:50.875+0000 7f8661580640 1 -- 172.21.15.31:0/949888665 wait complete. 2023-11-26T07:19:50.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:19:51.063 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":34.415637826999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.318959823265129,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:19:51.064 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.31895982326513 }, "state": "open", "uptime": 34.415637827 } ] 2023-11-26T07:19:51.064 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:19:56.064 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:19:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb129279640 1 Processor -- start 2023-11-26T07:19:56.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb129279640 1 -- start start 2023-11-26T07:19:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb129279640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb12415d7a0 0x7fb1240b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb129279640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1240d3740 0x7fb1240d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb129279640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1240b6cb0 con 0x7fb124161310 2023-11-26T07:19:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb129279640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1240b9290 con 0x7fb1240d3740 2023-11-26T07:19:56.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb129279640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1240d4110 con 0x7fb12415d7a0 2023-11-26T07:19:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb1237fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1240d3740 0x7fb1240d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb1237fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb1240d3740 0x7fb1240d3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38466/0 (socket says 172.21.15.31:38466) 2023-11-26T07:19:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb1237fe640 1 -- 172.21.15.31:0/101017798 learned_addr learned my addr 172.21.15.31:0/101017798 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb122ffd640 1 --2- 172.21.15.31:0/101017798 >> v2:172.21.15.97:3300/0 conn(0x7fb12415d7a0 0x7fb1240b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb1237fe640 1 -- 172.21.15.31:0/101017798 >> v2:172.21.15.97:3300/0 conn(0x7fb12415d7a0 msgr2=0x7fb1240b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:19:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb1237fe640 1 --2- 172.21.15.31:0/101017798 >> v2:172.21.15.97:3300/0 conn(0x7fb12415d7a0 0x7fb1240b6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb1237fe640 1 -- 172.21.15.31:0/101017798 >> v1:172.21.15.97:6789/0 conn(0x7fb124161310 legacy=0x7fb1240ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.215+0000 7fb1237fe640 1 -- 172.21.15.31:0/101017798 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124161890 con 0x7fb1240d3740 2023-11-26T07:19:56.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb1237fe640 1 --2- 172.21.15.31:0/101017798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1240d3740 0x7fb1240d3bd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb1240b5e90 tx=0x7fb11c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b3d524d60e471b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:56.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb121ffb640 1 -- 172.21.15.31:0/101017798 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c0a5860 con 0x7fb1240d3740 2023-11-26T07:19:56.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb121ffb640 1 -- 172.21.15.31:0/101017798 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c0a59c0 con 0x7fb1240d3740 2023-11-26T07:19:56.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb121ffb640 1 -- 172.21.15.31:0/101017798 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c0a5d50 con 0x7fb1240d3740 2023-11-26T07:19:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb129279640 1 -- 172.21.15.31:0/101017798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1240d3740 msgr2=0x7fb1240d3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb129279640 1 --2- 172.21.15.31:0/101017798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1240d3740 0x7fb1240d3bd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb1240b5e90 tx=0x7fb11c01c9a0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb129279640 1 -- 172.21.15.31:0/101017798 shutdown_connections 2023-11-26T07:19:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb129279640 1 --2- 172.21.15.31:0/101017798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1240d3740 0x7fb1240d3bd0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb129279640 1 --2- 172.21.15.31:0/101017798 >> v2:172.21.15.97:3300/0 conn(0x7fb12415d7a0 0x7fb1240b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb129279640 1 -- 172.21.15.31:0/101017798 >> 172.21.15.31:0/101017798 conn(0x7fb12406b9f0 msgr2=0x7fb12406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb129279640 1 -- 172.21.15.31:0/101017798 shutdown_connections 2023-11-26T07:19:56.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.219+0000 7fb129279640 1 -- 172.21.15.31:0/101017798 wait complete. 2023-11-26T07:19:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 Processor -- start 2023-11-26T07:19:56.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 -- start start 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1240d3740 0x7fb1240b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12415d7a0 0x7fb1241f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb124161310 0x7fb1241f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240b5cc0 con 0x7fb12415d7a0 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12415a4d0 con 0x7fb124161310 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240b3de0 con 0x7fb1240d3740 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb1227fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12415d7a0 0x7fb1241f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb1227fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12415d7a0 0x7fb1241f97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38476/0 (socket says 172.21.15.31:38476) 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb1227fc640 1 -- 172.21.15.31:0/2205886984 learned_addr learned my addr 172.21.15.31:0/2205886984 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb122ffd640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1240d3740 0x7fb1240b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb1237fe640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb124161310 0x7fb1241f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb1227fc640 1 -- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1240d3740 msgr2=0x7fb1240b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb1227fc640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1240d3740 0x7fb1240b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb1227fc640 1 -- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb124161310 msgr2=0x7fb1241f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb1227fc640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb124161310 0x7fb1241f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb1227fc640 1 -- 172.21.15.31:0/2205886984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c009b30 con 0x7fb12415d7a0 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb1227fc640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12415d7a0 0x7fb1241f97f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb11800c370 tx=0x7fb11800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb10b7fe640 1 -- 172.21.15.31:0/2205886984 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1180052d0 con 0x7fb12415d7a0 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb10b7fe640 1 -- 172.21.15.31:0/2205886984 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb118005430 con 0x7fb12415d7a0 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb10b7fe640 1 -- 172.21.15.31:0/2205886984 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb118003820 con 0x7fb12415d7a0 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 -- 172.21.15.31:0/2205886984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12415aa00 con 0x7fb12415d7a0 2023-11-26T07:19:56.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 -- 172.21.15.31:0/2205886984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12415a720 con 0x7fb12415d7a0 2023-11-26T07:19:56.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.223+0000 7fb129279640 1 -- 172.21.15.31:0/2205886984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1240b3f50 con 0x7fb12415d7a0 2023-11-26T07:19:56.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.227+0000 7fb10b7fe640 1 -- 172.21.15.31:0/2205886984 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fb118003980 con 0x7fb12415d7a0 2023-11-26T07:19:56.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.227+0000 7fb10b7fe640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb110066700 0x7fb110068bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:19:56.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.227+0000 7fb10b7fe640 1 -- 172.21.15.31:0/2205886984 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb118083620 con 0x7fb12415d7a0 2023-11-26T07:19:56.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.227+0000 7fb122ffd640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb110066700 0x7fb110068bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:19:56.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.227+0000 7fb122ffd640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb110066700 0x7fb110068bc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb114009b60 tx=0x7fb114009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:19:56.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.231+0000 7fb10b7fe640 1 -- 172.21.15.31:0/2205886984 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb118050960 con 0x7fb12415d7a0 2023-11-26T07:19:56.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 -- 172.21.15.31:0/2205886984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb124074a00 con 0x7fb12415d7a0 2023-11-26T07:19:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb10b7fe640 1 -- 172.21.15.31:0/2205886984 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fb118053f80 con 0x7fb12415d7a0 2023-11-26T07:19:56.429 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:19:56.429 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:19:56.429 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:19:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 -- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb110066700 msgr2=0x7fb110068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb110066700 0x7fb110068bc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb114009b60 tx=0x7fb114009190 comp rx=0 tx=0).stop 2023-11-26T07:19:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 -- 172.21.15.31:0/2205886984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12415d7a0 msgr2=0x7fb1241f97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:19:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12415d7a0 0x7fb1241f97f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb11800c370 tx=0x7fb11800c830 comp rx=0 tx=0).stop 2023-11-26T07:19:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 -- 172.21.15.31:0/2205886984 shutdown_connections 2023-11-26T07:19:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb110066700 0x7fb110068bc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb124161310 0x7fb1241f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12415d7a0 0x7fb1241f97f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 --2- 172.21.15.31:0/2205886984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1240d3740 0x7fb1240b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:19:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 -- 172.21.15.31:0/2205886984 >> 172.21.15.31:0/2205886984 conn(0x7fb12406b9f0 msgr2=0x7fb1240c7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:19:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 -- 172.21.15.31:0/2205886984 shutdown_connections 2023-11-26T07:19:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:19:56.423+0000 7fb129279640 1 -- 172.21.15.31:0/2205886984 wait complete. 2023-11-26T07:19:56.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:19:56.612 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":39.963580059000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.060164105103432,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:19:56.613 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.060164105103432 }, "state": "open", "uptime": 39.963580059 } ] 2023-11-26T07:19:56.613 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:00.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:20:00.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:20:01.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:01.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27ada25640 1 Processor -- start 2023-11-26T07:20:01.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27ada25640 1 -- start start 2023-11-26T07:20:01.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27ada25640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a8144c80 0x7f27a80abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:01.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27ada25640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f27a80ac460 0x7f27a80706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:01.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27ada25640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a8145060 con 0x7f27a8070c20 2023-11-26T07:20:01.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27ada25640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a8071fd0 con 0x7f27a80ac460 2023-11-26T07:20:01.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27ada25640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a80ac8c0 con 0x7f27a8144c80 2023-11-26T07:20:01.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27a77fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f27a8070c20 0x7f27a815eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53178/0 (socket says 172.21.15.31:53178) 2023-11-26T07:20:01.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27a77fe640 1 -- 172.21.15.31:0/3705375946 learned_addr learned my addr 172.21.15.31:0/3705375946 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:01.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27a6ffd640 1 --2- 172.21.15.31:0/3705375946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a8144c80 0x7f27a80abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:01.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f279ffff640 1 --2- 172.21.15.31:0/3705375946 >> v2:172.21.15.97:3300/0 conn(0x7f27a80ac460 0x7f27a80706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:01.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27a67fc640 1 -- 172.21.15.31:0/3705375946 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201475444 0 0) 0x7f27a8145060 con 0x7f27a8070c20 2023-11-26T07:20:01.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27a67fc640 1 -- 172.21.15.31:0/3705375946 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2794003530 con 0x7f27a8070c20 2023-11-26T07:20:01.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27a6ffd640 1 -- 172.21.15.31:0/3705375946 >> v2:172.21.15.97:3300/0 conn(0x7f27a80ac460 msgr2=0x7f27a80706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:01.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27a6ffd640 1 --2- 172.21.15.31:0/3705375946 >> v2:172.21.15.97:3300/0 conn(0x7f27a80ac460 0x7f27a80706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:01.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27a6ffd640 1 -- 172.21.15.31:0/3705375946 >> v1:172.21.15.31:6789/0 conn(0x7f27a8070c20 legacy=0x7f27a815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:01.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f27a6ffd640 1 -- 172.21.15.31:0/3705375946 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a8071450 con 0x7f27a8144c80 2023-11-26T07:20:01.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.763+0000 7f279ffff640 1 --2- 172.21.15.31:0/3705375946 >> v2:172.21.15.97:3300/0 conn(0x7f27a80ac460 0x7f27a80706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:01.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27a6ffd640 1 --2- 172.21.15.31:0/3705375946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a8144c80 0x7f27a80abf20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f27980097c0 tx=0x7f279801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb12b54a063108a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:01.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27a67fc640 1 -- 172.21.15.31:0/3705375946 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27980ab650 con 0x7f27a8144c80 2023-11-26T07:20:01.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27a67fc640 1 -- 172.21.15.31:0/3705375946 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27980ab7b0 con 0x7f27a8144c80 2023-11-26T07:20:01.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27a67fc640 1 -- 172.21.15.31:0/3705375946 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27980abb40 con 0x7f27a8144c80 2023-11-26T07:20:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27ada25640 1 -- 172.21.15.31:0/3705375946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a8144c80 msgr2=0x7f27a80abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27ada25640 1 --2- 172.21.15.31:0/3705375946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a8144c80 0x7f27a80abf20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f27980097c0 tx=0x7f279801c450 comp rx=0 tx=0).stop 2023-11-26T07:20:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27ada25640 1 -- 172.21.15.31:0/3705375946 shutdown_connections 2023-11-26T07:20:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27ada25640 1 --2- 172.21.15.31:0/3705375946 >> v2:172.21.15.97:3300/0 conn(0x7f27a80ac460 0x7f27a80706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:01.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27ada25640 1 --2- 172.21.15.31:0/3705375946 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a8144c80 0x7f27a80abf20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27ada25640 1 -- 172.21.15.31:0/3705375946 >> 172.21.15.31:0/3705375946 conn(0x7f27a80b0c40 msgr2=0x7f27a80b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27ada25640 1 -- 172.21.15.31:0/3705375946 shutdown_connections 2023-11-26T07:20:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.767+0000 7f27ada25640 1 -- 172.21.15.31:0/3705375946 wait complete. 2023-11-26T07:20:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27ada25640 1 Processor -- start 2023-11-26T07:20:01.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27ada25640 1 -- start start 2023-11-26T07:20:01.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27ada25640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a8070c20 0x7f27a813af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:01.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27ada25640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a80ac460 0x7f27a8165680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:01.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27ada25640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27a8144c80 0x7f27a815a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:01.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27ada25640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a80b30f0 con 0x7f27a8144c80 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27ada25640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a80b33d0 con 0x7f27a8070c20 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27ada25640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a8164e80 con 0x7f27a80ac460 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27a8144c80 0x7f27a815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a8070c20 0x7f27a813af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27a8144c80 0x7f27a815a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38482/0 (socket says 172.21.15.31:38482) 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a77fe640 1 -- 172.21.15.31:0/947836765 learned_addr learned my addr 172.21.15.31:0/947836765 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f279ffff640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a80ac460 0x7f27a8165680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a77fe640 1 -- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a80ac460 msgr2=0x7f27a8165680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a77fe640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a80ac460 0x7f27a8165680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a77fe640 1 -- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a8070c20 msgr2=0x7f27a813af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a77fe640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a8070c20 0x7f27a813af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:01.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a77fe640 1 -- 172.21.15.31:0/947836765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2790009420 con 0x7f27a8144c80 2023-11-26T07:20:01.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a77fe640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27a8144c80 0x7f27a815a170 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f27a000cb40 tx=0x7f27a000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:01.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a4ff9640 1 -- 172.21.15.31:0/947836765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a000aca0 con 0x7f27a8144c80 2023-11-26T07:20:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a4ff9640 1 -- 172.21.15.31:0/947836765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a000ae00 con 0x7f27a8144c80 2023-11-26T07:20:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27a4ff9640 1 -- 172.21.15.31:0/947836765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0012620 con 0x7f27a8144c80 2023-11-26T07:20:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27ada25640 1 -- 172.21.15.31:0/947836765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2798009420 con 0x7f27a8144c80 2023-11-26T07:20:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.771+0000 7f27ada25640 1 -- 172.21.15.31:0/947836765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a815b220 con 0x7f27a8144c80 2023-11-26T07:20:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.775+0000 7f27ada25640 1 -- 172.21.15.31:0/947836765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a80b1ba0 con 0x7f27a8144c80 2023-11-26T07:20:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.775+0000 7f27a4ff9640 1 -- 172.21.15.31:0/947836765 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f27a0007070 con 0x7f27a8144c80 2023-11-26T07:20:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.775+0000 7f27a4ff9640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f27880666b0 0x7f2788068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.775+0000 7f27a4ff9640 1 -- 172.21.15.31:0/947836765 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f27a00842b0 con 0x7f27a8144c80 2023-11-26T07:20:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.775+0000 7f27a6ffd640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f27880666b0 0x7f2788068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:01.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.775+0000 7f27a6ffd640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f27880666b0 0x7f2788068b70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f27980af000 tx=0x7f27980b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:01.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.779+0000 7f27a4ff9640 1 -- 172.21.15.31:0/947836765 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f27a0051430 con 0x7f27a8144c80 2023-11-26T07:20:01.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.967+0000 7f27ada25640 1 -- 172.21.15.31:0/947836765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27a81653a0 con 0x7f27a8144c80 2023-11-26T07:20:01.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.967+0000 7f27a4ff9640 1 -- 172.21.15.31:0/947836765 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f27a81653a0 con 0x7f27a8144c80 2023-11-26T07:20:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:01.975 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:01.975 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:01.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 -- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f27880666b0 msgr2=0x7f2788068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f27880666b0 0x7f2788068b70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f27980af000 tx=0x7f27980b6040 comp rx=0 tx=0).stop 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 -- 172.21.15.31:0/947836765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27a8144c80 msgr2=0x7f27a815a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27a8144c80 0x7f27a815a170 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f27a000cb40 tx=0x7f27a000a060 comp rx=0 tx=0).stop 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 -- 172.21.15.31:0/947836765 shutdown_connections 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f27880666b0 0x7f2788068b70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27a8144c80 0x7f27a815a170 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f27a80ac460 0x7f27a8165680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 --2- 172.21.15.31:0/947836765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27a8070c20 0x7f27a813af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 -- 172.21.15.31:0/947836765 >> 172.21.15.31:0/947836765 conn(0x7f27a80b0c40 msgr2=0x7f27a80aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 -- 172.21.15.31:0/947836765 shutdown_connections 2023-11-26T07:20:01.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:01.971+0000 7f27ada25640 1 -- 172.21.15.31:0/947836765 wait complete. 2023-11-26T07:20:01.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:02.162 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":45.511522452999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.804664573050637,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:20:02.162 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.804664573050637 }, "state": "open", "uptime": 45.511522453 } ] 2023-11-26T07:20:02.162 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:07.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c5da16640 1 Processor -- start 2023-11-26T07:20:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c5da16640 1 -- start start 2023-11-26T07:20:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c5da16640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5c5806dfd0 0x7f5c5806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c5da16640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c580adcc0 0x7f5c580ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c5da16640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5806e980 con 0x7f5c58143ae0 2023-11-26T07:20:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c5da16640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c58070670 con 0x7f5c580adcc0 2023-11-26T07:20:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c5da16640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c58070950 con 0x7f5c5806dfd0 2023-11-26T07:20:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c56ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5c58143ae0 0x7f5c580b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:33050/0 (socket says 172.21.15.31:33050) 2023-11-26T07:20:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c56ffd640 1 -- 172.21.15.31:0/52184904 learned_addr learned my addr 172.21.15.31:0/52184904 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c577fe640 1 --2- 172.21.15.31:0/52184904 >> v2:172.21.15.31:3300/0 conn(0x7f5c580adcc0 0x7f5c580ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c567fc640 1 --2- 172.21.15.31:0/52184904 >> v2:172.21.15.97:3300/0 conn(0x7f5c5806dfd0 0x7f5c5806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c55ffb640 1 -- 172.21.15.31:0/52184904 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563921408 0 0) 0x7f5c5806e980 con 0x7f5c58143ae0 2023-11-26T07:20:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c55ffb640 1 -- 172.21.15.31:0/52184904 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c40003530 con 0x7f5c58143ae0 2023-11-26T07:20:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c577fe640 1 -- 172.21.15.31:0/52184904 >> v2:172.21.15.97:3300/0 conn(0x7f5c5806dfd0 msgr2=0x7f5c5806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:07.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c577fe640 1 --2- 172.21.15.31:0/52184904 >> v2:172.21.15.97:3300/0 conn(0x7f5c5806dfd0 0x7f5c5806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c577fe640 1 -- 172.21.15.31:0/52184904 >> v1:172.21.15.31:6789/0 conn(0x7f5c58143ae0 legacy=0x7f5c580b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c577fe640 1 -- 172.21.15.31:0/52184904 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c58070c30 con 0x7f5c580adcc0 2023-11-26T07:20:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.315+0000 7f5c567fc640 1 --2- 172.21.15.31:0/52184904 >> v2:172.21.15.97:3300/0 conn(0x7f5c5806dfd0 0x7f5c5806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c577fe640 1 --2- 172.21.15.31:0/52184904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c580adcc0 0x7f5c580ae0e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5c5806cbe0 tx=0x7f5c5001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c3c018cde4b34a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c55ffb640 1 -- 172.21.15.31:0/52184904 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500a59a0 con 0x7f5c580adcc0 2023-11-26T07:20:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c55ffb640 1 -- 172.21.15.31:0/52184904 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c500a5b00 con 0x7f5c580adcc0 2023-11-26T07:20:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c55ffb640 1 -- 172.21.15.31:0/52184904 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500a5e70 con 0x7f5c580adcc0 2023-11-26T07:20:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c5da16640 1 -- 172.21.15.31:0/52184904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c580adcc0 msgr2=0x7f5c580ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c5da16640 1 --2- 172.21.15.31:0/52184904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c580adcc0 0x7f5c580ae0e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5c5806cbe0 tx=0x7f5c5001c8e0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c5da16640 1 -- 172.21.15.31:0/52184904 shutdown_connections 2023-11-26T07:20:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c5da16640 1 --2- 172.21.15.31:0/52184904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c580adcc0 0x7f5c580ae0e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c5da16640 1 --2- 172.21.15.31:0/52184904 >> v2:172.21.15.97:3300/0 conn(0x7f5c5806dfd0 0x7f5c5806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c5da16640 1 -- 172.21.15.31:0/52184904 >> 172.21.15.31:0/52184904 conn(0x7f5c580b0c40 msgr2=0x7f5c580b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c5da16640 1 -- 172.21.15.31:0/52184904 shutdown_connections 2023-11-26T07:20:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c5da16640 1 -- 172.21.15.31:0/52184904 wait complete. 2023-11-26T07:20:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.319+0000 7f5c5da16640 1 Processor -- start 2023-11-26T07:20:07.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c5da16640 1 -- start start 2023-11-26T07:20:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c5da16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c5806dfd0 0x7f5c580b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c5da16640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5c580adcc0 0x7f5c5815b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c5da16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c58143ae0 0x7f5c5815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c5da16640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b2c90 con 0x7f5c58143ae0 2023-11-26T07:20:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c5da16640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b2080 con 0x7f5c5806dfd0 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c5da16640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580b2360 con 0x7f5c580adcc0 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c58143ae0 0x7f5c5815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c58143ae0 0x7f5c5815bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57760/0 (socket says 172.21.15.31:57760) 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c577fe640 1 -- 172.21.15.31:0/3531716524 learned_addr learned my addr 172.21.15.31:0/3531716524 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c56ffd640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c5806dfd0 0x7f5c580b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c567fc640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5c580adcc0 0x7f5c5815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c577fe640 1 -- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5c580adcc0 msgr2=0x7f5c5815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c577fe640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5c580adcc0 0x7f5c5815b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c577fe640 1 -- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c5806dfd0 msgr2=0x7f5c580b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c577fe640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c5806dfd0 0x7f5c580b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c577fe640 1 -- 172.21.15.31:0/3531716524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c50009a70 con 0x7f5c58143ae0 2023-11-26T07:20:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c577fe640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c58143ae0 0x7f5c5815bd40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5c500194f0 tx=0x7f5c50019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c3ffff640 1 -- 172.21.15.31:0/3531716524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500aede0 con 0x7f5c58143ae0 2023-11-26T07:20:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c3ffff640 1 -- 172.21.15.31:0/3531716524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c50009450 con 0x7f5c58143ae0 2023-11-26T07:20:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c5da16640 1 -- 172.21.15.31:0/3531716524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c580b2640 con 0x7f5c58143ae0 2023-11-26T07:20:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c3ffff640 1 -- 172.21.15.31:0/3531716524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c500b7610 con 0x7f5c58143ae0 2023-11-26T07:20:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c5da16640 1 -- 172.21.15.31:0/3531716524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c580b3660 con 0x7f5c58143ae0 2023-11-26T07:20:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.323+0000 7f5c5da16640 1 -- 172.21.15.31:0/3531716524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c580b2e40 con 0x7f5c58143ae0 2023-11-26T07:20:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.331+0000 7f5c3ffff640 1 -- 172.21.15.31:0/3531716524 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f5c500b4050 con 0x7f5c58143ae0 2023-11-26T07:20:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.331+0000 7f5c3ffff640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c440666b0 0x7f5c44068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.331+0000 7f5c3ffff640 1 -- 172.21.15.31:0/3531716524 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5c5011f640 con 0x7f5c58143ae0 2023-11-26T07:20:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.331+0000 7f5c56ffd640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c440666b0 0x7f5c44068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:07.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.331+0000 7f5c56ffd640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c440666b0 0x7f5c44068b70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005890 tx=0x7f5c48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:07.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.335+0000 7f5c3ffff640 1 -- 172.21.15.31:0/3531716524 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c580b2e40 con 0x7f5c58143ae0 2023-11-26T07:20:07.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 -- 172.21.15.31:0/3531716524 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c5804e850 con 0x7f5c58143ae0 2023-11-26T07:20:07.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c3ffff640 1 -- 172.21.15.31:0/3531716524 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f5c500ec830 con 0x7f5c58143ae0 2023-11-26T07:20:07.528 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:07.528 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:07.528 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 -- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c440666b0 msgr2=0x7f5c44068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c440666b0 0x7f5c44068b70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c48005890 tx=0x7f5c48009040 comp rx=0 tx=0).stop 2023-11-26T07:20:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 -- 172.21.15.31:0/3531716524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c58143ae0 msgr2=0x7f5c5815bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c58143ae0 0x7f5c5815bd40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5c500194f0 tx=0x7f5c50019680 comp rx=0 tx=0).stop 2023-11-26T07:20:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 -- 172.21.15.31:0/3531716524 shutdown_connections 2023-11-26T07:20:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c440666b0 0x7f5c44068b70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c58143ae0 0x7f5c5815bd40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5c580adcc0 0x7f5c5815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 --2- 172.21.15.31:0/3531716524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c5806dfd0 0x7f5c580b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:07.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 -- 172.21.15.31:0/3531716524 >> 172.21.15.31:0/3531716524 conn(0x7f5c580b0c40 msgr2=0x7f5c580aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:07.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 -- 172.21.15.31:0/3531716524 shutdown_connections 2023-11-26T07:20:07.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:07.523+0000 7f5c5da16640 1 -- 172.21.15.31:0/3531716524 wait complete. 2023-11-26T07:20:07.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:07.713 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":51.063464959000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.552238545258234,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:20:07.713 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.552238545258234 }, "state": "open", "uptime": 51.063464959 } ] 2023-11-26T07:20:07.713 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:12.715 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:12.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.866+0000 7fb90a11c640 1 Processor -- start 2023-11-26T07:20:12.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.866+0000 7fb90a11c640 1 -- start start 2023-11-26T07:20:12.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.866+0000 7fb90a11c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb904143ae0 0x7fb9040b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.866+0000 7fb90a11c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb90406dfd0 0x7fb90406e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.866+0000 7fb90a11c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9040b2480 con 0x7fb9040adcc0 2023-11-26T07:20:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.866+0000 7fb90a11c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9040ae820 con 0x7fb904143ae0 2023-11-26T07:20:12.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.866+0000 7fb90a11c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90406e950 con 0x7fb90406dfd0 2023-11-26T07:20:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb90406dfd0 0x7fb90406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb9037fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb904143ae0 0x7fb9040b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb90406dfd0 0x7fb90406e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:35694/0 (socket says 172.21.15.31:35694) 2023-11-26T07:20:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb902ffd640 1 -- 172.21.15.31:0/289846486 learned_addr learned my addr 172.21.15.31:0/289846486 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb9027fc640 1 -- 172.21.15.31:0/289846486 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484496625 0 0) 0x7fb9040b2480 con 0x7fb9040adcc0 2023-11-26T07:20:12.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb9027fc640 1 -- 172.21.15.31:0/289846486 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8ec003530 con 0x7fb9040adcc0 2023-11-26T07:20:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb9037fe640 1 -- 172.21.15.31:0/289846486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb90406dfd0 msgr2=0x7fb90406e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb9037fe640 1 --2- 172.21.15.31:0/289846486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb90406dfd0 0x7fb90406e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb9037fe640 1 -- 172.21.15.31:0/289846486 >> v1:172.21.15.97:6789/0 conn(0x7fb9040adcc0 legacy=0x7fb9040ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb9037fe640 1 -- 172.21.15.31:0/289846486 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb904070670 con 0x7fb904143ae0 2023-11-26T07:20:12.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb902ffd640 1 --2- 172.21.15.31:0/289846486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb90406dfd0 0x7fb90406e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:12.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.870+0000 7fb9037fe640 1 --2- 172.21.15.31:0/289846486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb904143ae0 0x7fb9040b1f40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40097c0 tx=0x7fb8f401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf583f6117f45a36 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb9027fc640 1 -- 172.21.15.31:0/289846486 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40ab650 con 0x7fb904143ae0 2023-11-26T07:20:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb9027fc640 1 -- 172.21.15.31:0/289846486 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f40ab7b0 con 0x7fb904143ae0 2023-11-26T07:20:12.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb9027fc640 1 -- 172.21.15.31:0/289846486 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40abb40 con 0x7fb904143ae0 2023-11-26T07:20:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 -- 172.21.15.31:0/289846486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb904143ae0 msgr2=0x7fb9040b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 --2- 172.21.15.31:0/289846486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb904143ae0 0x7fb9040b1f40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40097c0 tx=0x7fb8f401c450 comp rx=0 tx=0).stop 2023-11-26T07:20:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 -- 172.21.15.31:0/289846486 shutdown_connections 2023-11-26T07:20:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 --2- 172.21.15.31:0/289846486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb90406dfd0 0x7fb90406e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 --2- 172.21.15.31:0/289846486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb904143ae0 0x7fb9040b1f40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 -- 172.21.15.31:0/289846486 >> 172.21.15.31:0/289846486 conn(0x7fb9040b0c40 msgr2=0x7fb9040b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 -- 172.21.15.31:0/289846486 shutdown_connections 2023-11-26T07:20:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 -- 172.21.15.31:0/289846486 wait complete. 2023-11-26T07:20:12.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 Processor -- start 2023-11-26T07:20:12.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 -- start start 2023-11-26T07:20:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb90406dfd0 0x7fb9041f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9040adcc0 0x7fb904163d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:12.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb904143ae0 0x7fb904141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90413ed10 con 0x7fb9040adcc0 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90415a330 con 0x7fb90406dfd0 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.874+0000 7fb90a11c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9040ce8b0 con 0x7fb904143ae0 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9040adcc0 0x7fb904163d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9040adcc0 0x7fb904163d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57764/0 (socket says 172.21.15.31:57764) 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb902ffd640 1 -- 172.21.15.31:0/4096317023 learned_addr learned my addr 172.21.15.31:0/4096317023 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb903fff640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb904143ae0 0x7fb904141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb9037fe640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb90406dfd0 0x7fb9041f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb902ffd640 1 -- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb904143ae0 msgr2=0x7fb904141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb902ffd640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb904143ae0 0x7fb904141b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb902ffd640 1 -- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb90406dfd0 msgr2=0x7fb9041f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:12.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb902ffd640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb90406dfd0 0x7fb9041f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:12.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb902ffd640 1 -- 172.21.15.31:0/4096317023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80094d0 con 0x7fb9040adcc0 2023-11-26T07:20:12.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb902ffd640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9040adcc0 0x7fb904163d80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800ea00 tx=0x7fb8f800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:12.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb8ebfff640 1 -- 172.21.15.31:0/4096317023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800ca60 con 0x7fb9040adcc0 2023-11-26T07:20:12.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb8ebfff640 1 -- 172.21.15.31:0/4096317023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f800cbc0 con 0x7fb9040adcc0 2023-11-26T07:20:12.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb90a11c640 1 -- 172.21.15.31:0/4096317023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f4009420 con 0x7fb9040adcc0 2023-11-26T07:20:12.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb8ebfff640 1 -- 172.21.15.31:0/4096317023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8013470 con 0x7fb9040adcc0 2023-11-26T07:20:12.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb90a11c640 1 -- 172.21.15.31:0/4096317023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9041648d0 con 0x7fb9040adcc0 2023-11-26T07:20:12.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb90a11c640 1 -- 172.21.15.31:0/4096317023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90404e850 con 0x7fb9040adcc0 2023-11-26T07:20:12.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.878+0000 7fb8ebfff640 1 -- 172.21.15.31:0/4096317023 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fb8f80135d0 con 0x7fb9040adcc0 2023-11-26T07:20:12.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.882+0000 7fb8ebfff640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb8f00666b0 0x7fb8f0068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:12.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.882+0000 7fb8ebfff640 1 -- 172.21.15.31:0/4096317023 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb8f80844d0 con 0x7fb9040adcc0 2023-11-26T07:20:12.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.882+0000 7fb9037fe640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb8f00666b0 0x7fb8f0068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:12.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.882+0000 7fb9037fe640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb8f00666b0 0x7fb8f0068b70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40af000 tx=0x7fb8f40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:12.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:12.886+0000 7fb8ebfff640 1 -- 172.21.15.31:0/4096317023 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb8f80516d0 con 0x7fb9040adcc0 2023-11-26T07:20:13.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.074+0000 7fb90a11c640 1 -- 172.21.15.31:0/4096317023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb90406cc40 con 0x7fb9040adcc0 2023-11-26T07:20:13.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.074+0000 7fb8ebfff640 1 -- 172.21.15.31:0/4096317023 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fb8f805b030 con 0x7fb9040adcc0 2023-11-26T07:20:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:13.078 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:13.078 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:13.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 -- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb8f00666b0 msgr2=0x7fb8f0068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:13.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb8f00666b0 0x7fb8f0068b70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40af000 tx=0x7fb8f40a7000 comp rx=0 tx=0).stop 2023-11-26T07:20:13.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 -- 172.21.15.31:0/4096317023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9040adcc0 msgr2=0x7fb904163d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:13.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9040adcc0 0x7fb904163d80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800ea00 tx=0x7fb8f800eec0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 -- 172.21.15.31:0/4096317023 shutdown_connections 2023-11-26T07:20:13.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb8f00666b0 0x7fb8f0068b70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb904143ae0 0x7fb904141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9040adcc0 0x7fb904163d80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 --2- 172.21.15.31:0/4096317023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb90406dfd0 0x7fb9041f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:13.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 -- 172.21.15.31:0/4096317023 >> 172.21.15.31:0/4096317023 conn(0x7fb9040b0c40 msgr2=0x7fb9040aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:13.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 -- 172.21.15.31:0/4096317023 shutdown_connections 2023-11-26T07:20:13.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:13.078+0000 7fb90a11c640 1 -- 172.21.15.31:0/4096317023 wait complete. 2023-11-26T07:20:13.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:13.264 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":56.61940757,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.302851484006393,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:20:13.265 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.302851484006393 }, "state": "open", "uptime": 56.61940757 } ] 2023-11-26T07:20:13.265 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:18.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48aef8d640 1 Processor -- start 2023-11-26T07:20:18.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48aef8d640 1 -- start start 2023-11-26T07:20:18.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48aef8d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f48a8144c80 0x7f48a80abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48aef8d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48a80ac460 0x7f48a80706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48aef8d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f48a8070c20 0x7f48a815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48aef8d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a8145060 con 0x7f48a8070c20 2023-11-26T07:20:18.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48aef8d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a8071fd0 con 0x7f48a8144c80 2023-11-26T07:20:18.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48aef8d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a80ac8c0 con 0x7f48a80ac460 2023-11-26T07:20:18.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48ad52a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f48a8070c20 0x7f48a815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48ad52a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f48a8070c20 0x7f48a815eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58328/0 (socket says 172.21.15.31:58328) 2023-11-26T07:20:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48ad52a640 1 -- 172.21.15.31:0/1219538715 learned_addr learned my addr 172.21.15.31:0/1219538715 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48acd29640 1 --2- 172.21.15.31:0/1219538715 >> v2:172.21.15.97:3300/0 conn(0x7f48a8144c80 0x7f48a80abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48a7fff640 1 --2- 172.21.15.31:0/1219538715 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48a80ac460 0x7f48a80706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48ad52a640 1 -- 172.21.15.31:0/1219538715 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48a80ac460 msgr2=0x7f48a80706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48ad52a640 1 --2- 172.21.15.31:0/1219538715 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48a80ac460 0x7f48a80706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48ad52a640 1 -- 172.21.15.31:0/1219538715 >> v2:172.21.15.97:3300/0 conn(0x7f48a8144c80 msgr2=0x7f48a80abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48ad52a640 1 --2- 172.21.15.31:0/1219538715 >> v2:172.21.15.97:3300/0 conn(0x7f48a8144c80 0x7f48a80abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.418+0000 7f48ad52a640 1 -- 172.21.15.31:0/1219538715 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a8071450 con 0x7f48a8070c20 2023-11-26T07:20:18.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48ad52a640 1 --2- 172.21.15.31:0/1219538715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 0x7f48a815eed0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f48a806eb80 tx=0x7f48a001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedc11622884ece1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48a77fe640 1 -- 172.21.15.31:0/1219538715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a00a5560 con 0x7f48a8070c20 2023-11-26T07:20:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48a77fe640 1 -- 172.21.15.31:0/1219538715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a00a56c0 con 0x7f48a8070c20 2023-11-26T07:20:18.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48a77fe640 1 -- 172.21.15.31:0/1219538715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a00a5a50 con 0x7f48a8070c20 2023-11-26T07:20:18.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48aef8d640 1 -- 172.21.15.31:0/1219538715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 msgr2=0x7f48a815eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:18.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48aef8d640 1 --2- 172.21.15.31:0/1219538715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 0x7f48a815eed0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f48a806eb80 tx=0x7f48a001c6e0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48aef8d640 1 -- 172.21.15.31:0/1219538715 shutdown_connections 2023-11-26T07:20:18.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48aef8d640 1 --2- 172.21.15.31:0/1219538715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 0x7f48a815eed0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48aef8d640 1 --2- 172.21.15.31:0/1219538715 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48a80ac460 0x7f48a80706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48aef8d640 1 --2- 172.21.15.31:0/1219538715 >> v2:172.21.15.97:3300/0 conn(0x7f48a8144c80 0x7f48a80abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48aef8d640 1 -- 172.21.15.31:0/1219538715 >> 172.21.15.31:0/1219538715 conn(0x7f48a80b0c40 msgr2=0x7f48a80b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:18.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48aef8d640 1 -- 172.21.15.31:0/1219538715 shutdown_connections 2023-11-26T07:20:18.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.422+0000 7f48aef8d640 1 -- 172.21.15.31:0/1219538715 wait complete. 2023-11-26T07:20:18.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48aef8d640 1 Processor -- start 2023-11-26T07:20:18.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48aef8d640 1 -- start start 2023-11-26T07:20:18.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48aef8d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 0x7f48a815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48aef8d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48a80ac460 0x7f48a815a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48aef8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a8144c80 0x7f48a815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48aef8d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a815a5c0 con 0x7f48a8070c20 2023-11-26T07:20:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48aef8d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8141c50 con 0x7f48a8144c80 2023-11-26T07:20:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48aef8d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8141f30 con 0x7f48a80ac460 2023-11-26T07:20:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48acd29640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 0x7f48a815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48acd29640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 0x7f48a815a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58334/0 (socket says 172.21.15.31:58334) 2023-11-26T07:20:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48acd29640 1 -- 172.21.15.31:0/211349527 learned_addr learned my addr 172.21.15.31:0/211349527 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:18.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48a7fff640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48a80ac460 0x7f48a815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48ad52a640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a8144c80 0x7f48a815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48acd29640 1 -- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48a80ac460 msgr2=0x7f48a815a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48acd29640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48a80ac460 0x7f48a815a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48acd29640 1 -- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a8144c80 msgr2=0x7f48a815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48acd29640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a8144c80 0x7f48a815ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48acd29640 1 -- 172.21.15.31:0/211349527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a0009b30 con 0x7f48a8070c20 2023-11-26T07:20:18.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48acd29640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 0x7f48a815a080 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f489c00d5f0 tx=0x7f489c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:18.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.426+0000 7f48a57fa640 1 -- 172.21.15.31:0/211349527 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0125f0 con 0x7f48a8070c20 2023-11-26T07:20:18.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.430+0000 7f48a57fa640 1 -- 172.21.15.31:0/211349527 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c012750 con 0x7f48a8070c20 2023-11-26T07:20:18.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.430+0000 7f48aef8d640 1 -- 172.21.15.31:0/211349527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a81437b0 con 0x7f48a8070c20 2023-11-26T07:20:18.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.430+0000 7f48a57fa640 1 -- 172.21.15.31:0/211349527 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c012ae0 con 0x7f48a8070c20 2023-11-26T07:20:18.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.430+0000 7f48aef8d640 1 -- 172.21.15.31:0/211349527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a8143d70 con 0x7f48a8070c20 2023-11-26T07:20:18.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.430+0000 7f48aef8d640 1 -- 172.21.15.31:0/211349527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a80b1ba0 con 0x7f48a8070c20 2023-11-26T07:20:18.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.430+0000 7f48a57fa640 1 -- 172.21.15.31:0/211349527 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f489c012c40 con 0x7f48a8070c20 2023-11-26T07:20:18.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.430+0000 7f48a57fa640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f488c0666b0 0x7f488c068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.430+0000 7f48a57fa640 1 -- 172.21.15.31:0/211349527 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f489c0846d0 con 0x7f48a8070c20 2023-11-26T07:20:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.430+0000 7f48a7fff640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f488c0666b0 0x7f488c068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:18.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.434+0000 7f48a7fff640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f488c0666b0 0x7f488c068b70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48a80b3780 tx=0x7f4894008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:18.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.434+0000 7f48a57fa640 1 -- 172.21.15.31:0/211349527 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f489c0517c0 con 0x7f48a8070c20 2023-11-26T07:20:18.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.626+0000 7f48aef8d640 1 -- 172.21.15.31:0/211349527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48a804e850 con 0x7f48a8070c20 2023-11-26T07:20:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.626+0000 7f48a57fa640 1 -- 172.21.15.31:0/211349527 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f489c007380 con 0x7f48a8070c20 2023-11-26T07:20:18.631 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:18.631 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:18.631 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 -- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f488c0666b0 msgr2=0x7f488c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f488c0666b0 0x7f488c068b70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f48a80b3780 tx=0x7f4894008040 comp rx=0 tx=0).stop 2023-11-26T07:20:18.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 -- 172.21.15.31:0/211349527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 msgr2=0x7f48a815a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 0x7f48a815a080 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f489c00d5f0 tx=0x7f489c00dab0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 -- 172.21.15.31:0/211349527 shutdown_connections 2023-11-26T07:20:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f488c0666b0 0x7f488c068b70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48a8144c80 0x7f48a815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f48a80ac460 0x7f48a815a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 --2- 172.21.15.31:0/211349527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a8070c20 0x7f48a815a080 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 -- 172.21.15.31:0/211349527 >> 172.21.15.31:0/211349527 conn(0x7f48a80b0c40 msgr2=0x7f48a80afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 -- 172.21.15.31:0/211349527 shutdown_connections 2023-11-26T07:20:18.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:18.630+0000 7f48aef8d640 1 -- 172.21.15.31:0/211349527 wait complete. 2023-11-26T07:20:18.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:18.815 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":62.167350425000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.056997566620911,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:20:18.815 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.05699756662091 }, "state": "open", "uptime": 62.167350425 } ] 2023-11-26T07:20:18.815 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:23.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f897598d640 1 Processor -- start 2023-11-26T07:20:23.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f897598d640 1 -- start start 2023-11-26T07:20:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f897598d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8970143ae0 0x7f89700b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f897598d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89700ae7f0 con 0x7f897006dfd0 2023-11-26T07:20:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f897598d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89700b2480 con 0x7f89700ae3d0 2023-11-26T07:20:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f897598d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8970070670 con 0x7f8970143ae0 2023-11-26T07:20:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f8967fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f897006dfd0 0x7f89700adcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57928/0 (socket says 172.21.15.31:57928) 2023-11-26T07:20:23.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f8967fff640 1 -- 172.21.15.31:0/3519784499 learned_addr learned my addr 172.21.15.31:0/3519784499 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896effd640 1 --2- 172.21.15.31:0/3519784499 >> v2:172.21.15.97:3300/0 conn(0x7f8970143ae0 0x7f89700b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447144367 0 0) 0x7f89700ae7f0 con 0x7f897006dfd0 2023-11-26T07:20:23.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f895c003530 con 0x7f897006dfd0 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2635062776 0 0) 0x7f895c003530 con 0x7f897006dfd0 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 >> v2:172.21.15.97:3300/0 conn(0x7f8970143ae0 msgr2=0x7f89700b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 --2- 172.21.15.31:0/3519784499 >> v2:172.21.15.97:3300/0 conn(0x7f8970143ae0 0x7f89700b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 >> v1:172.21.15.97:6789/0 conn(0x7f89700ae3d0 legacy=0x7f8970071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8970070950 con 0x7f897006dfd0 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f8958001680 con 0x7f897006dfd0 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 >> v1:172.21.15.31:6789/0 conn(0x7f897006dfd0 legacy=0x7f89700adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f895c003980 0x7f895c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f895c0047d0 0x7f895c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f895c005190 0x7f895c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:23.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8970070670 con 0x7f895c0047d0 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895c00c030 con 0x7f895c003980 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.970+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895c00c1f0 con 0x7f895c005190 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f8967fff640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f895c0047d0 0x7f895c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f896effd640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f895c003980 0x7f895c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f896f7fe640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f895c005190 0x7f895c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f8967fff640 1 -- 172.21.15.31:0/3519784499 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f895c005190 msgr2=0x7f895c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f8967fff640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f895c005190 0x7f895c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f8967fff640 1 -- 172.21.15.31:0/3519784499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f895c003980 msgr2=0x7f895c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f8967fff640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f895c003980 0x7f895c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f8967fff640 1 -- 172.21.15.31:0/3519784499 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c003530 con 0x7f895c0047d0 2023-11-26T07:20:23.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f896f7fe640 1 -- 172.21.15.31:0/3519784499 reap_dead start 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f8967fff640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f895c0047d0 0x7f895c004c50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8958007cb0 tx=0x7f8958021580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd3749169c8828e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958005ae0 con 0x7f895c0047d0 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89580ad070 con 0x7f895c0047d0 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.974+0000 7f896e7fc640 1 -- 172.21.15.31:0/3519784499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958007070 con 0x7f895c0047d0 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 -- 172.21.15.31:0/3519784499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f895c0047d0 msgr2=0x7f895c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f895c0047d0 0x7f895c004c50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8958007cb0 tx=0x7f8958021580 comp rx=0 tx=0).stop 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 -- 172.21.15.31:0/3519784499 shutdown_connections 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 --2- 172.21.15.31:0/3519784499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f895c0047d0 0x7f895c004c50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 -- 172.21.15.31:0/3519784499 >> 172.21.15.31:0/3519784499 conn(0x7f89700b0c40 msgr2=0x7f89700b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 -- 172.21.15.31:0/3519784499 shutdown_connections 2023-11-26T07:20:23.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 -- 172.21.15.31:0/3519784499 wait complete. 2023-11-26T07:20:23.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 Processor -- start 2023-11-26T07:20:23.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 -- start start 2023-11-26T07:20:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f895c0047d0 0x7f897015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897015bef0 0x7f897009a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897013d980 0x7f897009b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897015bad0 con 0x7f897013d980 2023-11-26T07:20:23.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897009adb0 con 0x7f897015bef0 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f897598d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897015ae50 con 0x7f895c0047d0 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f896f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897013d980 0x7f897009b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f896f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897013d980 0x7f897009b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40644/0 (socket says 172.21.15.31:40644) 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f896f7fe640 1 -- 172.21.15.31:0/2436838512 learned_addr learned my addr 172.21.15.31:0/2436838512 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f8967fff640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897015bef0 0x7f897009a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.978+0000 7f896effd640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f895c0047d0 0x7f897015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896f7fe640 1 -- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f895c0047d0 msgr2=0x7f897015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896f7fe640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f895c0047d0 0x7f897015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896f7fe640 1 -- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897015bef0 msgr2=0x7f897009a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896f7fe640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897015bef0 0x7f897009a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896f7fe640 1 -- 172.21.15.31:0/2436838512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897015b130 con 0x7f897013d980 2023-11-26T07:20:23.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896f7fe640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897013d980 0x7f897009b450 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f89700ac510 tx=0x7f896800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896cff9640 1 -- 172.21.15.31:0/2436838512 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968003980 con 0x7f897013d980 2023-11-26T07:20:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896cff9640 1 -- 172.21.15.31:0/2436838512 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8968003b00 con 0x7f897013d980 2023-11-26T07:20:23.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896cff9640 1 -- 172.21.15.31:0/2436838512 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89680050d0 con 0x7f897013d980 2023-11-26T07:20:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f897598d640 1 -- 172.21.15.31:0/2436838512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897015b410 con 0x7f897013d980 2023-11-26T07:20:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f897598d640 1 -- 172.21.15.31:0/2436838512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897013de00 con 0x7f897013d980 2023-11-26T07:20:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f897598d640 1 -- 172.21.15.31:0/2436838512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897015bc10 con 0x7f897013d980 2023-11-26T07:20:23.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896cff9640 1 -- 172.21.15.31:0/2436838512 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f8968005230 con 0x7f897013d980 2023-11-26T07:20:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896cff9640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89500666e0 0x7f8950068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.982+0000 7f896cff9640 1 -- 172.21.15.31:0/2436838512 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8968084010 con 0x7f897013d980 2023-11-26T07:20:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.986+0000 7f896effd640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89500666e0 0x7f8950068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:23.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.986+0000 7f896effd640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89500666e0 0x7f8950068ba0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8960009be0 tx=0x7f8960009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:23.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:23.986+0000 7f896cff9640 1 -- 172.21.15.31:0/2436838512 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f897015bc10 con 0x7f897013d980 2023-11-26T07:20:24.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.174+0000 7f897598d640 1 -- 172.21.15.31:0/2436838512 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f897015bc10 con 0x7f897013d980 2023-11-26T07:20:24.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f896cff9640 1 -- 172.21.15.31:0/2436838512 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f8968058350 con 0x7f897013d980 2023-11-26T07:20:24.179 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:24.179 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:24.180 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 -- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89500666e0 msgr2=0x7f8950068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89500666e0 0x7f8950068ba0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8960009be0 tx=0x7f8960009190 comp rx=0 tx=0).stop 2023-11-26T07:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 -- 172.21.15.31:0/2436838512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897013d980 msgr2=0x7f897009b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897013d980 0x7f897009b450 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f89700ac510 tx=0x7f896800fbb0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 -- 172.21.15.31:0/2436838512 shutdown_connections 2023-11-26T07:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f89500666e0 0x7f8950068ba0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f897013d980 0x7f897009b450 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f897015bef0 0x7f897009a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 --2- 172.21.15.31:0/2436838512 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f895c0047d0 0x7f897015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:24.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 -- 172.21.15.31:0/2436838512 >> 172.21.15.31:0/2436838512 conn(0x7f89700b0c40 msgr2=0x7f89700cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:24.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 -- 172.21.15.31:0/2436838512 shutdown_connections 2023-11-26T07:20:24.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:24.178+0000 7f897598d640 1 -- 172.21.15.31:0/2436838512 wait complete. 2023-11-26T07:20:24.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:24.364 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":67.719293386999993,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.814101123097696,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:20:24.365 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.814101123097696 }, "state": "open", "uptime": 67.719293387 } ] 2023-11-26T07:20:24.365 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:29.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:29.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0cc78d640 1 Processor -- start 2023-11-26T07:20:29.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0cc78d640 1 -- start start 2023-11-26T07:20:29.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0cc78d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0c4160d00 0x7fd0c40d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:29.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0cc78d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c41613a0 0x7fd0c4161810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:29.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0cc78d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c40ce1f0 con 0x7fd0c40d3e40 2023-11-26T07:20:29.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0cc78d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c4161d50 con 0x7fd0c4160d00 2023-11-26T07:20:29.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0cc78d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c40cea70 con 0x7fd0c41613a0 2023-11-26T07:20:29.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0ca529640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0c4160d00 0x7fd0c40d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:29.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0ca529640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0c4160d00 0x7fd0c40d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40654/0 (socket says 172.21.15.31:40654) 2023-11-26T07:20:29.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0ca529640 1 -- 172.21.15.31:0/3413634638 learned_addr learned my addr 172.21.15.31:0/3413634638 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:29.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0cad2a640 1 --2- 172.21.15.31:0/3413634638 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c41613a0 0x7fd0c4161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:29.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0ca529640 1 -- 172.21.15.31:0/3413634638 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c41613a0 msgr2=0x7fd0c4161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0ca529640 1 --2- 172.21.15.31:0/3413634638 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c41613a0 0x7fd0c4161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0ca529640 1 -- 172.21.15.31:0/3413634638 >> v1:172.21.15.97:6789/0 conn(0x7fd0c40d3e40 legacy=0x7fd0c40cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0ca529640 1 -- 172.21.15.31:0/3413634638 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c40cf610 con 0x7fd0c4160d00 2023-11-26T07:20:29.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.518+0000 7fd0cad2a640 1 --2- 172.21.15.31:0/3413634638 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c41613a0 0x7fd0c4161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:29.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0ca529640 1 --2- 172.21.15.31:0/3413634638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c4160d00 0x7fd0c40d1da0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009870 tx=0x7fd0c001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3ca036db8333ef server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:29.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0c9527640 1 -- 172.21.15.31:0/3413634638 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00a8a30 con 0x7fd0c4160d00 2023-11-26T07:20:29.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0c9527640 1 -- 172.21.15.31:0/3413634638 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c00a8b90 con 0x7fd0c4160d00 2023-11-26T07:20:29.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0c9527640 1 -- 172.21.15.31:0/3413634638 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00b0440 con 0x7fd0c4160d00 2023-11-26T07:20:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0cc78d640 1 -- 172.21.15.31:0/3413634638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c4160d00 msgr2=0x7fd0c40d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0cc78d640 1 --2- 172.21.15.31:0/3413634638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c4160d00 0x7fd0c40d1da0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0009870 tx=0x7fd0c001ed30 comp rx=0 tx=0).stop 2023-11-26T07:20:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0cc78d640 1 -- 172.21.15.31:0/3413634638 shutdown_connections 2023-11-26T07:20:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0cc78d640 1 --2- 172.21.15.31:0/3413634638 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c41613a0 0x7fd0c4161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0cc78d640 1 --2- 172.21.15.31:0/3413634638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c4160d00 0x7fd0c40d1da0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0cc78d640 1 -- 172.21.15.31:0/3413634638 >> 172.21.15.31:0/3413634638 conn(0x7fd0c408afc0 msgr2=0x7fd0c40d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0cc78d640 1 -- 172.21.15.31:0/3413634638 shutdown_connections 2023-11-26T07:20:29.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.522+0000 7fd0cc78d640 1 -- 172.21.15.31:0/3413634638 wait complete. 2023-11-26T07:20:29.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 Processor -- start 2023-11-26T07:20:29.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 -- start start 2023-11-26T07:20:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c40d3e40 0x7fd0c41f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4160d00 0x7fd0c40cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c41613a0 0x7fd0c40ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c40b62e0 con 0x7fd0c41613a0 2023-11-26T07:20:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c40b65c0 con 0x7fd0c4160d00 2023-11-26T07:20:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c40b68a0 con 0x7fd0c40d3e40 2023-11-26T07:20:29.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cad2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c41613a0 0x7fd0c40ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cad2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c41613a0 0x7fd0c40ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40670/0 (socket says 172.21.15.31:40670) 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cad2a640 1 -- 172.21.15.31:0/50355542 learned_addr learned my addr 172.21.15.31:0/50355542 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0ca529640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c40d3e40 0x7fd0c41f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0c9d28640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4160d00 0x7fd0c40cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cad2a640 1 -- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c40d3e40 msgr2=0x7fd0c41f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cad2a640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c40d3e40 0x7fd0c41f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cad2a640 1 -- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4160d00 msgr2=0x7fd0c40cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cad2a640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4160d00 0x7fd0c40cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cad2a640 1 -- 172.21.15.31:0/50355542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c00094d0 con 0x7fd0c41613a0 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0c9d28640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4160d00 0x7fd0c40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cad2a640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c41613a0 0x7fd0c40ccb00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc007d40 tx=0x7fd0bc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:29.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0baffd640 1 -- 172.21.15.31:0/50355542 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0039a0 con 0x7fd0c41613a0 2023-11-26T07:20:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0baffd640 1 -- 172.21.15.31:0/50355542 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc00f040 con 0x7fd0c41613a0 2023-11-26T07:20:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0baffd640 1 -- 172.21.15.31:0/50355542 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc003d40 con 0x7fd0c41613a0 2023-11-26T07:20:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 -- 172.21.15.31:0/50355542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c40b6b80 con 0x7fd0c41613a0 2023-11-26T07:20:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 -- 172.21.15.31:0/50355542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c40b7140 con 0x7fd0c41613a0 2023-11-26T07:20:29.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.526+0000 7fd0cc78d640 1 -- 172.21.15.31:0/50355542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c404e850 con 0x7fd0c41613a0 2023-11-26T07:20:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.530+0000 7fd0baffd640 1 -- 172.21.15.31:0/50355542 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fd0bc004030 con 0x7fd0c41613a0 2023-11-26T07:20:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.530+0000 7fd0baffd640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0a80666b0 0x7fd0a8068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.530+0000 7fd0baffd640 1 -- 172.21.15.31:0/50355542 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd0bc0846f0 con 0x7fd0c41613a0 2023-11-26T07:20:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.530+0000 7fd0ca529640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0a80666b0 0x7fd0a8068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.530+0000 7fd0ca529640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0a80666b0 0x7fd0a8068b70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00b2000 tx=0x7fd0c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:29.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.534+0000 7fd0baffd640 1 -- 172.21.15.31:0/50355542 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd0bc051620 con 0x7fd0c41613a0 2023-11-26T07:20:29.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.722+0000 7fd0cc78d640 1 -- 172.21.15.31:0/50355542 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0c408b4a0 con 0x7fd0c41613a0 2023-11-26T07:20:29.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.722+0000 7fd0baffd640 1 -- 172.21.15.31:0/50355542 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd0bc05b020 con 0x7fd0c41613a0 2023-11-26T07:20:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:29.726 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:29.726 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 -- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0a80666b0 msgr2=0x7fd0a8068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:29.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0a80666b0 0x7fd0a8068b70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00b2000 tx=0x7fd0c0008040 comp rx=0 tx=0).stop 2023-11-26T07:20:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 -- 172.21.15.31:0/50355542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c41613a0 msgr2=0x7fd0c40ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c41613a0 0x7fd0c40ccb00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc007d40 tx=0x7fd0bc00d4a0 comp rx=0 tx=0).stop 2023-11-26T07:20:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 -- 172.21.15.31:0/50355542 shutdown_connections 2023-11-26T07:20:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd0a80666b0 0x7fd0a8068b70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0c41613a0 0x7fd0c40ccb00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4160d00 0x7fd0c40cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 --2- 172.21.15.31:0/50355542 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0c40d3e40 0x7fd0c41f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 -- 172.21.15.31:0/50355542 >> 172.21.15.31:0/50355542 conn(0x7fd0c408afc0 msgr2=0x7fd0c40cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 -- 172.21.15.31:0/50355542 shutdown_connections 2023-11-26T07:20:29.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:29.726+0000 7fd0cc78d640 1 -- 172.21.15.31:0/50355542 wait complete. 2023-11-26T07:20:29.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:29.909 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":73.263236578000004,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.574643905376206,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:20:29.909 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.574643905376206 }, "state": "open", "uptime": 73.263236578 } ] 2023-11-26T07:20:29.910 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:30.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:20:30.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:20:34.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:35.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4fc17640 1 Processor -- start 2023-11-26T07:20:35.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4fc17640 1 -- start start 2023-11-26T07:20:35.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4fc17640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba4815d7b0 0x7fba480d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4fc17640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba480d2080 0x7fba4806bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4fc17640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba480cefd0 con 0x7fba480cead0 2023-11-26T07:20:35.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4fc17640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba480cf2b0 con 0x7fba4815d7b0 2023-11-26T07:20:35.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4fc17640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4806b3f0 con 0x7fba480d2080 2023-11-26T07:20:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4e1b4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba480d2080 0x7fba4806bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4d9b3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba4815d7b0 0x7fba480d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4d1b2640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fba480cead0 0x7fba480d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:32980/0 (socket says 172.21.15.31:32980) 2023-11-26T07:20:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.062+0000 7fba4d1b2640 1 -- 172.21.15.31:0/781326864 learned_addr learned my addr 172.21.15.31:0/781326864 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4d9b3640 1 -- 172.21.15.31:0/781326864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba480d2080 msgr2=0x7fba4806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4d9b3640 1 --2- 172.21.15.31:0/781326864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba480d2080 0x7fba4806bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4d9b3640 1 -- 172.21.15.31:0/781326864 >> v1:172.21.15.97:6789/0 conn(0x7fba480cead0 legacy=0x7fba480d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4d9b3640 1 -- 172.21.15.31:0/781326864 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4806b6d0 con 0x7fba4815d7b0 2023-11-26T07:20:35.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4c9b1640 1 -- 172.21.15.31:0/781326864 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132377706 0 0) 0x7fba480cefd0 con 0x7fba480cead0 2023-11-26T07:20:35.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4d9b3640 1 --2- 172.21.15.31:0/781326864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba4815d7b0 0x7fba480d10d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fba440097c0 tx=0x7fba4401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec5d8ec5b31e1a2d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:35.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4c9b1640 1 -- 172.21.15.31:0/781326864 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44004230 con 0x7fba4815d7b0 2023-11-26T07:20:35.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4c9b1640 1 -- 172.21.15.31:0/781326864 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba440a7050 con 0x7fba4815d7b0 2023-11-26T07:20:35.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4c9b1640 1 -- 172.21.15.31:0/781326864 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba44003620 con 0x7fba4815d7b0 2023-11-26T07:20:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4fc17640 1 -- 172.21.15.31:0/781326864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba4815d7b0 msgr2=0x7fba480d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.066+0000 7fba4fc17640 1 --2- 172.21.15.31:0/781326864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba4815d7b0 0x7fba480d10d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fba440097c0 tx=0x7fba4401c330 comp rx=0 tx=0).stop 2023-11-26T07:20:35.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 -- 172.21.15.31:0/781326864 shutdown_connections 2023-11-26T07:20:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 --2- 172.21.15.31:0/781326864 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba480d2080 0x7fba4806bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 --2- 172.21.15.31:0/781326864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba4815d7b0 0x7fba480d10d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 -- 172.21.15.31:0/781326864 >> 172.21.15.31:0/781326864 conn(0x7fba480d33b0 msgr2=0x7fba480d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 -- 172.21.15.31:0/781326864 shutdown_connections 2023-11-26T07:20:35.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 -- 172.21.15.31:0/781326864 wait complete. 2023-11-26T07:20:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 Processor -- start 2023-11-26T07:20:35.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 -- start start 2023-11-26T07:20:35.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba480cead0 0x7fba48157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480d2080 0x7fba48153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba4815d7b0 0x7fba48154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba481555f0 con 0x7fba480d2080 2023-11-26T07:20:35.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba481540e0 con 0x7fba480cead0 2023-11-26T07:20:35.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4fc17640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba48159a90 con 0x7fba4815d7b0 2023-11-26T07:20:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4d1b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480d2080 0x7fba48153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4d1b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480d2080 0x7fba48153ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35832/0 (socket says 172.21.15.31:35832) 2023-11-26T07:20:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4d1b2640 1 -- 172.21.15.31:0/73719466 learned_addr learned my addr 172.21.15.31:0/73719466 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4e1b4640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba4815d7b0 0x7fba48154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4d9b3640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba480cead0 0x7fba48157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4d1b2640 1 -- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba4815d7b0 msgr2=0x7fba48154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4d1b2640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba4815d7b0 0x7fba48154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4d1b2640 1 -- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba480cead0 msgr2=0x7fba48157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4d1b2640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba480cead0 0x7fba48157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.070+0000 7fba4d1b2640 1 -- 172.21.15.31:0/73719466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44009420 con 0x7fba480d2080 2023-11-26T07:20:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba4d1b2640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480d2080 0x7fba48153ba0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fba3800c450 tx=0x7fba3800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba3e7fc640 1 -- 172.21.15.31:0/73719466 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba380074e0 con 0x7fba480d2080 2023-11-26T07:20:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba3e7fc640 1 -- 172.21.15.31:0/73719466 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba38007640 con 0x7fba480d2080 2023-11-26T07:20:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba4fc17640 1 -- 172.21.15.31:0/73719466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba48157960 con 0x7fba480d2080 2023-11-26T07:20:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba3e7fc640 1 -- 172.21.15.31:0/73719466 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba380049f0 con 0x7fba480d2080 2023-11-26T07:20:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba4fc17640 1 -- 172.21.15.31:0/73719466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4815a030 con 0x7fba480d2080 2023-11-26T07:20:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba4fc17640 1 -- 172.21.15.31:0/73719466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4804e850 con 0x7fba480d2080 2023-11-26T07:20:35.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba3e7fc640 1 -- 172.21.15.31:0/73719466 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fba38004b50 con 0x7fba480d2080 2023-11-26T07:20:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba3e7fc640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba2c0666b0 0x7fba2c068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba3e7fc640 1 -- 172.21.15.31:0/73719466 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fba38083890 con 0x7fba480d2080 2023-11-26T07:20:35.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.074+0000 7fba4d9b3640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba2c0666b0 0x7fba2c068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:35.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.078+0000 7fba4d9b3640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba2c0666b0 0x7fba2c068b70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fba440ad000 tx=0x7fba440a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:35.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.078+0000 7fba3e7fc640 1 -- 172.21.15.31:0/73719466 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fba38050980 con 0x7fba480d2080 2023-11-26T07:20:35.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.270+0000 7fba4fc17640 1 -- 172.21.15.31:0/73719466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba480d12a0 con 0x7fba480d2080 2023-11-26T07:20:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.270+0000 7fba3e7fc640 1 -- 172.21.15.31:0/73719466 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fba38018070 con 0x7fba480d2080 2023-11-26T07:20:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:35.275 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:35.275 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 -- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba2c0666b0 msgr2=0x7fba2c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba2c0666b0 0x7fba2c068b70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fba440ad000 tx=0x7fba440a6000 comp rx=0 tx=0).stop 2023-11-26T07:20:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 -- 172.21.15.31:0/73719466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480d2080 msgr2=0x7fba48153ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480d2080 0x7fba48153ba0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fba3800c450 tx=0x7fba3800c910 comp rx=0 tx=0).stop 2023-11-26T07:20:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 -- 172.21.15.31:0/73719466 shutdown_connections 2023-11-26T07:20:35.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fba2c0666b0 0x7fba2c068b70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba4815d7b0 0x7fba48154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba480d2080 0x7fba48153ba0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 --2- 172.21.15.31:0/73719466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba480cead0 0x7fba48157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 -- 172.21.15.31:0/73719466 >> 172.21.15.31:0/73719466 conn(0x7fba480d33b0 msgr2=0x7fba480d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 -- 172.21.15.31:0/73719466 shutdown_connections 2023-11-26T07:20:35.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:35.274+0000 7fba4fc17640 1 -- 172.21.15.31:0/73719466 wait complete. 2023-11-26T07:20:35.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:35.462 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":78.815179819999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.337895427310038,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:20:35.463 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.337895427310038 }, "state": "open", "uptime": 78.81517982 } ] 2023-11-26T07:20:35.463 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:40.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.614+0000 7fabdf9f8640 1 Processor -- start 2023-11-26T07:20:40.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.614+0000 7fabdf9f8640 1 -- start start 2023-11-26T07:20:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdf9f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 0x7fabd80b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdf9f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fabd80adcc0 0x7fabd80ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdf9f8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd80b2480 con 0x7fabd806dfd0 2023-11-26T07:20:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdf9f8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd8070670 con 0x7fabd80adcc0 2023-11-26T07:20:40.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdf9f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8070950 con 0x7fabd8143ae0 2023-11-26T07:20:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdd794640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 0x7fabd80b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:40.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabddf95640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fabd80adcc0 0x7fabd80ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdd794640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 0x7fabd80b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:53048/0 (socket says 172.21.15.31:53048) 2023-11-26T07:20:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdd794640 1 -- 172.21.15.31:0/1894462949 learned_addr learned my addr 172.21.15.31:0/1894462949 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabcffff640 1 -- 172.21.15.31:0/1894462949 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983050593 0 0) 0x7fabd80b2480 con 0x7fabd806dfd0 2023-11-26T07:20:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabcffff640 1 -- 172.21.15.31:0/1894462949 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc4003530 con 0x7fabd806dfd0 2023-11-26T07:20:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdd794640 1 -- 172.21.15.31:0/1894462949 >> v2:172.21.15.97:3300/0 conn(0x7fabd80adcc0 msgr2=0x7fabd80ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdd794640 1 --2- 172.21.15.31:0/1894462949 >> v2:172.21.15.97:3300/0 conn(0x7fabd80adcc0 0x7fabd80ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdd794640 1 -- 172.21.15.31:0/1894462949 >> v1:172.21.15.97:6789/0 conn(0x7fabd806dfd0 legacy=0x7fabd806e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabdd794640 1 -- 172.21.15.31:0/1894462949 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8070c30 con 0x7fabd8143ae0 2023-11-26T07:20:40.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.618+0000 7fabddf95640 1 --2- 172.21.15.31:0/1894462949 >> v2:172.21.15.97:3300/0 conn(0x7fabd80adcc0 0x7fabd80ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:40.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabdd794640 1 --2- 172.21.15.31:0/1894462949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 0x7fabd80b1f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fabd400bb10 tx=0x7fabd401ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb4031b69a36b53 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:40.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabcffff640 1 -- 172.21.15.31:0/1894462949 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40a7cb0 con 0x7fabd8143ae0 2023-11-26T07:20:40.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabcffff640 1 -- 172.21.15.31:0/1894462949 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40a7e10 con 0x7fabd8143ae0 2023-11-26T07:20:40.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabcffff640 1 -- 172.21.15.31:0/1894462949 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40af660 con 0x7fabd8143ae0 2023-11-26T07:20:40.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabdf9f8640 1 -- 172.21.15.31:0/1894462949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 msgr2=0x7fabd80b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:40.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabdf9f8640 1 --2- 172.21.15.31:0/1894462949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 0x7fabd80b1f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fabd400bb10 tx=0x7fabd401ebe0 comp rx=0 tx=0).stop 2023-11-26T07:20:40.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabdf9f8640 1 -- 172.21.15.31:0/1894462949 shutdown_connections 2023-11-26T07:20:40.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabdf9f8640 1 --2- 172.21.15.31:0/1894462949 >> v2:172.21.15.97:3300/0 conn(0x7fabd80adcc0 0x7fabd80ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:40.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabdf9f8640 1 --2- 172.21.15.31:0/1894462949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 0x7fabd80b1f40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:40.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabdf9f8640 1 -- 172.21.15.31:0/1894462949 >> 172.21.15.31:0/1894462949 conn(0x7fabd80b0c40 msgr2=0x7fabd80b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:40.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabdf9f8640 1 -- 172.21.15.31:0/1894462949 shutdown_connections 2023-11-26T07:20:40.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabdf9f8640 1 -- 172.21.15.31:0/1894462949 wait complete. 2023-11-26T07:20:40.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.622+0000 7fabdf9f8640 1 Processor -- start 2023-11-26T07:20:40.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdf9f8640 1 -- start start 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdf9f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd806dfd0 0x7fabd8140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdf9f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd80adcc0 0x7fabd8140690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdf9f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 0x7fabd81f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdf9f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8140bd0 con 0x7fabd806dfd0 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdf9f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd81f63c0 con 0x7fabd80adcc0 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdf9f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd81f66a0 con 0x7fabd8143ae0 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdd794640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd806dfd0 0x7fabd8140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdd794640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd806dfd0 0x7fabd8140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35844/0 (socket says 172.21.15.31:35844) 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdd794640 1 -- 172.21.15.31:0/3268121815 learned_addr learned my addr 172.21.15.31:0/3268121815 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabddf95640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 0x7fabd81f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdcf93640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd80adcc0 0x7fabd8140690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:40.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdd794640 1 -- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 msgr2=0x7fabd81f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:40.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdd794640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 0x7fabd81f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:40.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdd794640 1 -- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd80adcc0 msgr2=0x7fabd8140690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:40.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdd794640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd80adcc0 0x7fabd8140690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:40.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdd794640 1 -- 172.21.15.31:0/3268121815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0009a70 con 0x7fabd806dfd0 2023-11-26T07:20:40.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdcf93640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd80adcc0 0x7fabd8140690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:40.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdd794640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd806dfd0 0x7fabd8140150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fabd400b2c0 tx=0x7fabd401b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabcdffb640 1 -- 172.21.15.31:0/3268121815 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40afd00 con 0x7fabd806dfd0 2023-11-26T07:20:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabcdffb640 1 -- 172.21.15.31:0/3268121815 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40afe80 con 0x7fabd806dfd0 2023-11-26T07:20:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdf9f8640 1 -- 172.21.15.31:0/3268121815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd400b770 con 0x7fabd806dfd0 2023-11-26T07:20:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabcdffb640 1 -- 172.21.15.31:0/3268121815 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd40ba680 con 0x7fabd806dfd0 2023-11-26T07:20:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdf9f8640 1 -- 172.21.15.31:0/3268121815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd81f6c60 con 0x7fabd806dfd0 2023-11-26T07:20:40.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.626+0000 7fabdf9f8640 1 -- 172.21.15.31:0/3268121815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd804e850 con 0x7fabd806dfd0 2023-11-26T07:20:40.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.630+0000 7fabcdffb640 1 -- 172.21.15.31:0/3268121815 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fabd40b93e0 con 0x7fabd806dfd0 2023-11-26T07:20:40.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.630+0000 7fabcdffb640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fabbc0666b0 0x7fabbc068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:40.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.630+0000 7fabcdffb640 1 -- 172.21.15.31:0/3268121815 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fabd411e900 con 0x7fabd806dfd0 2023-11-26T07:20:40.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.630+0000 7fabdcf93640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fabbc0666b0 0x7fabbc068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:40.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.630+0000 7fabdcf93640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fabbc0666b0 0x7fabbc068b70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fabd80ae280 tx=0x7fabc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:40.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.634+0000 7fabcdffb640 1 -- 172.21.15.31:0/3268121815 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fabd40eb940 con 0x7fabd806dfd0 2023-11-26T07:20:40.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 -- 172.21.15.31:0/3268121815 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabd80acdf0 con 0x7fabd806dfd0 2023-11-26T07:20:40.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabcdffb640 1 -- 172.21.15.31:0/3268121815 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fabd40bf090 con 0x7fabd806dfd0 2023-11-26T07:20:40.825 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:40.825 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:40.825 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:40.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 -- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fabbc0666b0 msgr2=0x7fabbc068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:40.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fabbc0666b0 0x7fabbc068b70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fabd80ae280 tx=0x7fabc8009040 comp rx=0 tx=0).stop 2023-11-26T07:20:40.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 -- 172.21.15.31:0/3268121815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd806dfd0 msgr2=0x7fabd8140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:40.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd806dfd0 0x7fabd8140150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fabd400b2c0 tx=0x7fabd401b980 comp rx=0 tx=0).stop 2023-11-26T07:20:40.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 -- 172.21.15.31:0/3268121815 shutdown_connections 2023-11-26T07:20:40.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fabbc0666b0 0x7fabbc068b70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:40.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabd8143ae0 0x7fabd81f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:40.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabd80adcc0 0x7fabd8140690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:40.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 --2- 172.21.15.31:0/3268121815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabd806dfd0 0x7fabd8140150 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:40.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.822+0000 7fabdf9f8640 1 -- 172.21.15.31:0/3268121815 >> 172.21.15.31:0/3268121815 conn(0x7fabd80b0c40 msgr2=0x7fabd8142640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:40.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.826+0000 7fabdf9f8640 1 -- 172.21.15.31:0/3268121815 shutdown_connections 2023-11-26T07:20:40.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:40.826+0000 7fabdf9f8640 1 -- 172.21.15.31:0/3268121815 wait complete. 2023-11-26T07:20:40.846 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:41.009 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":5297,"entity":{"name":{"type":"client","num":5297},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":553237103}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":5,"uptime":84.363123252999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.104331806005529,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5297 172.21.15.97:0/553237103","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:20:41.009 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 553237103, "type": "any" }, "name": { "num": 5297, "type": "client" } }, "id": 5297, "inst": "client.5297 172.21.15.97:0/553237103", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 5, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.10433180600553 }, "state": "open", "uptime": 84.363123253 } ] 2023-11-26T07:20:41.009 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2023-11-26T07:20:46.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:46.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b3aaa640 1 Processor -- start 2023-11-26T07:20:46.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b3aaa640 1 -- start start 2023-11-26T07:20:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b3aaa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f21ac0b4760 0x7f21ac15a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b3aaa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac0b3ab0 con 0x7f21ac157560 2023-11-26T07:20:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b3aaa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac157940 con 0x7f21ac0715b0 2023-11-26T07:20:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b3aaa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac15af00 con 0x7f21ac0b4760 2023-11-26T07:20:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b1846640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f21ac157560 0x7f21ac0662a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49502/0 (socket says 172.21.15.31:49502) 2023-11-26T07:20:46.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b1846640 1 -- 172.21.15.31:0/3368806857 learned_addr learned my addr 172.21.15.31:0/3368806857 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:46.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b1045640 1 --2- 172.21.15.31:0/3368806857 >> v2:172.21.15.97:3300/0 conn(0x7f21ac0b4760 0x7f21ac15a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b0844640 1 -- 172.21.15.31:0/3368806857 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888963036 0 0) 0x7f21ac0b3ab0 con 0x7f21ac157560 2023-11-26T07:20:46.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b0844640 1 -- 172.21.15.31:0/3368806857 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2198003530 con 0x7f21ac157560 2023-11-26T07:20:46.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b1045640 1 -- 172.21.15.31:0/3368806857 >> v1:172.21.15.97:6789/0 conn(0x7f21ac0715b0 legacy=0x7f21ac071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b1045640 1 -- 172.21.15.31:0/3368806857 >> v1:172.21.15.31:6789/0 conn(0x7f21ac157560 legacy=0x7f21ac0662a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b0844640 1 -- 172.21.15.31:0/3368806857 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2882755352 0 0) 0x7f2198003530 con 0x7f21ac157560 2023-11-26T07:20:46.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.166+0000 7f21b1045640 1 -- 172.21.15.31:0/3368806857 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0b4e90 con 0x7f21ac0b4760 2023-11-26T07:20:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b1045640 1 --2- 172.21.15.31:0/3368806857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21ac0b4760 0x7f21ac15a9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f219c0097c0 tx=0x7f219c01c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e0ba31203ea131 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b0844640 1 -- 172.21.15.31:0/3368806857 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8001680 con 0x7f21ac0b4760 2023-11-26T07:20:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b0844640 1 -- 172.21.15.31:0/3368806857 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c0a7050 con 0x7f21ac0b4760 2023-11-26T07:20:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b0844640 1 -- 172.21.15.31:0/3368806857 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c0ab620 con 0x7f21ac0b4760 2023-11-26T07:20:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3368806857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21ac0b4760 msgr2=0x7f21ac15a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b3aaa640 1 --2- 172.21.15.31:0/3368806857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21ac0b4760 0x7f21ac15a9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f219c0097c0 tx=0x7f219c01c310 comp rx=0 tx=0).stop 2023-11-26T07:20:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3368806857 shutdown_connections 2023-11-26T07:20:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b3aaa640 1 --2- 172.21.15.31:0/3368806857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21ac0b4760 0x7f21ac15a9c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3368806857 >> 172.21.15.31:0/3368806857 conn(0x7f21ac0b1e40 msgr2=0x7f21ac070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3368806857 shutdown_connections 2023-11-26T07:20:46.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.170+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3368806857 wait complete. 2023-11-26T07:20:46.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b3aaa640 1 Processor -- start 2023-11-26T07:20:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b3aaa640 1 -- start start 2023-11-26T07:20:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b3aaa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f21ac0715b0 0x7f21ac151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b3aaa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21ac0b4760 0x7f21ac164950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b3aaa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac157560 0x7f21ac0c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b3aaa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac162c00 con 0x7f21ac157560 2023-11-26T07:20:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b3aaa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0c5ea0 con 0x7f21ac0b4760 2023-11-26T07:20:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b3aaa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0c5320 con 0x7f21ac0715b0 2023-11-26T07:20:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b2047640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac157560 0x7f21ac0c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b1045640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21ac0b4760 0x7f21ac164950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b2047640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac157560 0x7f21ac0c65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35518/0 (socket says 172.21.15.31:35518) 2023-11-26T07:20:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b1846640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f21ac0715b0 0x7f21ac151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b2047640 1 -- 172.21.15.31:0/3283627840 learned_addr learned my addr 172.21.15.31:0/3283627840 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b1045640 1 -- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f21ac0715b0 msgr2=0x7f21ac151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b1045640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f21ac0715b0 0x7f21ac151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b1045640 1 -- 172.21.15.31:0/3283627840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac157560 msgr2=0x7f21ac0c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b1045640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac157560 0x7f21ac0c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b1045640 1 -- 172.21.15.31:0/3283627840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c009420 con 0x7f21ac0b4760 2023-11-26T07:20:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b2047640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac157560 0x7f21ac0c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b1045640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21ac0b4760 0x7f21ac164950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f219c0ab000 tx=0x7f219c0190b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21a27fc640 1 -- 172.21.15.31:0/3283627840 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c0b4410 con 0x7f21ac0b4760 2023-11-26T07:20:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21a27fc640 1 -- 172.21.15.31:0/3283627840 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c0b4570 con 0x7f21ac0b4760 2023-11-26T07:20:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3283627840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac0c5600 con 0x7f21ac0b4760 2023-11-26T07:20:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21a27fc640 1 -- 172.21.15.31:0/3283627840 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c0b4900 con 0x7f21ac0b4760 2023-11-26T07:20:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.174+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3283627840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac0c5bc0 con 0x7f21ac0b4760 2023-11-26T07:20:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.178+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3283627840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac04e850 con 0x7f21ac0b4760 2023-11-26T07:20:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.178+0000 7f21a27fc640 1 -- 172.21.15.31:0/3283627840 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f219c0b2040 con 0x7f21ac0b4760 2023-11-26T07:20:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.178+0000 7f21a27fc640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2190066620 0x7f2190068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.178+0000 7f21a27fc640 1 -- 172.21.15.31:0/3283627840 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f219c11e6c0 con 0x7f21ac0b4760 2023-11-26T07:20:46.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.178+0000 7f21b1846640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2190066620 0x7f2190068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.178+0000 7f21b1846640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2190066620 0x7f2190068ae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21a8000fc0 tx=0x7f21a80019c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:46.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.182+0000 7f21a27fc640 1 -- 172.21.15.31:0/3283627840 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f219c0eb890 con 0x7f21ac0b4760 2023-11-26T07:20:46.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.370+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3283627840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21ac0ce640 con 0x7f21ac0b4760 2023-11-26T07:20:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.370+0000 7f21a27fc640 1 -- 172.21.15.31:0/3283627840 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f219c0eeeb0 con 0x7f21ac0b4760 2023-11-26T07:20:46.375 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:46.375 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:46.375 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:46.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2190066620 msgr2=0x7f2190068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2190066620 0x7f2190068ae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21a8000fc0 tx=0x7f21a80019c0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21ac0b4760 msgr2=0x7f21ac164950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21ac0b4760 0x7f21ac164950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f219c0ab000 tx=0x7f219c0190b0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3283627840 shutdown_connections 2023-11-26T07:20:46.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2190066620 0x7f2190068ae0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac157560 0x7f21ac0c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f21ac0b4760 0x7f21ac164950 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 --2- 172.21.15.31:0/3283627840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f21ac0715b0 0x7f21ac151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3283627840 >> 172.21.15.31:0/3283627840 conn(0x7f21ac0b1e40 msgr2=0x7f21ac0cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:46.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3283627840 shutdown_connections 2023-11-26T07:20:46.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.374+0000 7f21b3aaa640 1 -- 172.21.15.31:0/3283627840 wait complete. 2023-11-26T07:20:46.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2023-11-26T07:20:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-11-26T07:20:46.561 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-26T07:20:46.561 DEBUG:tasks.ceph_test_case:wait_until_true: success in 80s and 0 retries 2023-11-26T07:20:46.562 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:20:46.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:20:46.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:20:46.721 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:20:46.721 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:20:46.721 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:20:46.730 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:20:46.731 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:20:46.731 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:20:46.740 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:20:46.740 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:20:46.741 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:20:46.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:20:46.787 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:20:46.788 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:20:46.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:20:46.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:46.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.970+0000 7fc7b8e79640 1 Processor -- start 2023-11-26T07:20:46.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.970+0000 7fc7b8e79640 1 -- start start 2023-11-26T07:20:46.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b8e79640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b415d3e0 0x7fc7b40716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b8e79640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc7b406c630 0x7fc7b406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b8e79640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b4070920 con 0x7fc7b4071f30 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b8e79640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b4071be0 con 0x7fc7b406c630 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b8e79640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b415d7c0 con 0x7fc7b415d3e0 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b2d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc7b4071f30 0x7fc7b40b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49518/0 (socket says 172.21.15.31:49518) 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b2d9d640 1 -- 172.21.15.31:0/3505957466 learned_addr learned my addr 172.21.15.31:0/3505957466 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b259c640 1 --2- 172.21.15.31:0/3505957466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b415d3e0 0x7fc7b40716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7abfff640 1 --2- 172.21.15.31:0/3505957466 >> v2:172.21.15.97:3300/0 conn(0x7fc7b406c630 0x7fc7b406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b1d9b640 1 -- 172.21.15.31:0/3505957466 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125452632 0 0) 0x7fc7b4070920 con 0x7fc7b4071f30 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b1d9b640 1 -- 172.21.15.31:0/3505957466 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a0003530 con 0x7fc7b4071f30 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b259c640 1 -- 172.21.15.31:0/3505957466 >> v2:172.21.15.97:3300/0 conn(0x7fc7b406c630 msgr2=0x7fc7b406ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b259c640 1 --2- 172.21.15.31:0/3505957466 >> v2:172.21.15.97:3300/0 conn(0x7fc7b406c630 0x7fc7b406ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b259c640 1 -- 172.21.15.31:0/3505957466 >> v1:172.21.15.31:6789/0 conn(0x7fc7b4071f30 legacy=0x7fc7b40b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7b259c640 1 -- 172.21.15.31:0/3505957466 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b4070bc0 con 0x7fc7b415d3e0 2023-11-26T07:20:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.974+0000 7fc7abfff640 1 --2- 172.21.15.31:0/3505957466 >> v2:172.21.15.97:3300/0 conn(0x7fc7b406c630 0x7fc7b406ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:46.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b259c640 1 --2- 172.21.15.31:0/3505957466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b415d3e0 0x7fc7b40716a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40097f0 tx=0x7fc7a401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=170117d1b581b1f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b1d9b640 1 -- 172.21.15.31:0/3505957466 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40ab650 con 0x7fc7b415d3e0 2023-11-26T07:20:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b1d9b640 1 -- 172.21.15.31:0/3505957466 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a40ab7b0 con 0x7fc7b415d3e0 2023-11-26T07:20:46.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b1d9b640 1 -- 172.21.15.31:0/3505957466 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40abb40 con 0x7fc7b415d3e0 2023-11-26T07:20:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b8e79640 1 -- 172.21.15.31:0/3505957466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b415d3e0 msgr2=0x7fc7b40716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b8e79640 1 --2- 172.21.15.31:0/3505957466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b415d3e0 0x7fc7b40716a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40097f0 tx=0x7fc7a401c450 comp rx=0 tx=0).stop 2023-11-26T07:20:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b8e79640 1 -- 172.21.15.31:0/3505957466 shutdown_connections 2023-11-26T07:20:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b8e79640 1 --2- 172.21.15.31:0/3505957466 >> v2:172.21.15.97:3300/0 conn(0x7fc7b406c630 0x7fc7b406ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b8e79640 1 --2- 172.21.15.31:0/3505957466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b415d3e0 0x7fc7b40716a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b8e79640 1 -- 172.21.15.31:0/3505957466 >> 172.21.15.31:0/3505957466 conn(0x7fc7b40b0c40 msgr2=0x7fc7b40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b8e79640 1 -- 172.21.15.31:0/3505957466 shutdown_connections 2023-11-26T07:20:46.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b8e79640 1 -- 172.21.15.31:0/3505957466 wait complete. 2023-11-26T07:20:46.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b8e79640 1 Processor -- start 2023-11-26T07:20:46.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.978+0000 7fc7b8e79640 1 -- start start 2023-11-26T07:20:46.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b8e79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b406c630 0x7fc7b4154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b8e79640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b4071f30 0x7fc7b4156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b8e79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7b415d3e0 0x7fc7b40ac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b8e79640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b4155190 con 0x7fc7b406c630 2023-11-26T07:20:46.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b8e79640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b4155470 con 0x7fc7b415d3e0 2023-11-26T07:20:46.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b8e79640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b4155750 con 0x7fc7b4071f30 2023-11-26T07:20:46.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b406c630 0x7fc7b4154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b406c630 0x7fc7b4154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35520/0 (socket says 172.21.15.31:35520) 2023-11-26T07:20:46.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b259c640 1 -- 172.21.15.31:0/1308519983 learned_addr learned my addr 172.21.15.31:0/1308519983 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:46.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7abfff640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b4071f30 0x7fc7b4156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b2d9d640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7b415d3e0 0x7fc7b40ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b259c640 1 -- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b4071f30 msgr2=0x7fc7b4156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b259c640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b4071f30 0x7fc7b4156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b259c640 1 -- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7b415d3e0 msgr2=0x7fc7b40ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:46.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b259c640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7b415d3e0 0x7fc7b40ac380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:46.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b259c640 1 -- 172.21.15.31:0/1308519983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c009420 con 0x7fc7b406c630 2023-11-26T07:20:46.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7abfff640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b4071f30 0x7fc7b4156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:46.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b259c640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b406c630 0x7fc7b4154bb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc7a401ab00 tx=0x7fc7a40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:46.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7aaffd640 1 -- 172.21.15.31:0/1308519983 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40abdf0 con 0x7fc7b406c630 2023-11-26T07:20:46.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7aaffd640 1 -- 172.21.15.31:0/1308519983 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a40bf450 con 0x7fc7b406c630 2023-11-26T07:20:46.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7aaffd640 1 -- 172.21.15.31:0/1308519983 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40bf800 con 0x7fc7b406c630 2023-11-26T07:20:46.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b8e79640 1 -- 172.21.15.31:0/1308519983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a4009420 con 0x7fc7b406c630 2023-11-26T07:20:46.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b8e79640 1 -- 172.21.15.31:0/1308519983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b40728d0 con 0x7fc7b406c630 2023-11-26T07:20:46.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.982+0000 7fc7b8e79640 1 -- 172.21.15.31:0/1308519983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b404e850 con 0x7fc7b406c630 2023-11-26T07:20:46.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.986+0000 7fc7aaffd640 1 -- 172.21.15.31:0/1308519983 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fc7a40bf960 con 0x7fc7b406c630 2023-11-26T07:20:46.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.986+0000 7fc7aaffd640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc794066650 0x7fc794068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:46.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.986+0000 7fc7aaffd640 1 -- 172.21.15.31:0/1308519983 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc7a411ee80 con 0x7fc7b406c630 2023-11-26T07:20:46.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.986+0000 7fc7abfff640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc794066650 0x7fc794068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:46.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.986+0000 7fc7abfff640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc794066650 0x7fc794068b10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc7b415d150 tx=0x7fc79c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:46.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:46.990+0000 7fc7aaffd640 1 -- 172.21.15.31:0/1308519983 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc7a40ec200 con 0x7fc7b406c630 2023-11-26T07:20:47.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.182+0000 7fc7b8e79640 1 -- 172.21.15.31:0/1308519983 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7b40b12c0 con 0x7fc7b406c630 2023-11-26T07:20:47.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.182+0000 7fc7aaffd640 1 -- 172.21.15.31:0/1308519983 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fc7a40c4090 con 0x7fc7b406c630 2023-11-26T07:20:47.186 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:47.186 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:47.186 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:47.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 -- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc794066650 msgr2=0x7fc794068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc794066650 0x7fc794068b10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc7b415d150 tx=0x7fc79c009190 comp rx=0 tx=0).stop 2023-11-26T07:20:47.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 -- 172.21.15.31:0/1308519983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b406c630 msgr2=0x7fc7b4154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b406c630 0x7fc7b4154bb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc7a401ab00 tx=0x7fc7a40191f0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 -- 172.21.15.31:0/1308519983 shutdown_connections 2023-11-26T07:20:47.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc794066650 0x7fc794068b10 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7b415d3e0 0x7fc7b40ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc7b4071f30 0x7fc7b4156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 --2- 172.21.15.31:0/1308519983 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7b406c630 0x7fc7b4154bb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 -- 172.21.15.31:0/1308519983 >> 172.21.15.31:0/1308519983 conn(0x7fc7b40b0c40 msgr2=0x7fc7b40ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:47.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 -- 172.21.15.31:0/1308519983 shutdown_connections 2023-11-26T07:20:47.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.186+0000 7fc7b8e79640 1 -- 172.21.15.31:0/1308519983 wait complete. 2023-11-26T07:20:47.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:20:47.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.346+0000 7f3c6ad65640 1 Processor -- start 2023-11-26T07:20:47.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c6ad65640 1 -- start start 2023-11-26T07:20:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c6ad65640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c6406e410 0x7f3c640715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c6ad65640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3c64071af0 0x7f3c640b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c6ad65640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c640ad740 con 0x7f3c640b2ee0 2023-11-26T07:20:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c6ad65640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c640b3e20 con 0x7f3c6406e410 2023-11-26T07:20:47.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c6ad65640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6415dde0 con 0x7f3c64071af0 2023-11-26T07:20:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c68b01640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c6406e410 0x7f3c640715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c68b01640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3c6406e410 0x7f3c640715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51928/0 (socket says 172.21.15.31:51928) 2023-11-26T07:20:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c68b01640 1 -- 172.21.15.31:0/4203328456 learned_addr learned my addr 172.21.15.31:0/4203328456 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c63fff640 1 --2- 172.21.15.31:0/4203328456 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3c64071af0 0x7f3c640b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c68b01640 1 -- 172.21.15.31:0/4203328456 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3c64071af0 msgr2=0x7f3c640b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:47.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c68b01640 1 --2- 172.21.15.31:0/4203328456 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3c64071af0 0x7f3c640b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c68b01640 1 -- 172.21.15.31:0/4203328456 >> v1:172.21.15.97:6789/0 conn(0x7f3c640b2ee0 legacy=0x7f3c64161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.350+0000 7f3c68b01640 1 -- 172.21.15.31:0/4203328456 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c640b3580 con 0x7f3c6406e410 2023-11-26T07:20:47.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c68b01640 1 --2- 172.21.15.31:0/4203328456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c6406e410 0x7f3c640715b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009870 tx=0x7f3c5801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45955fae8d2541ea server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c637fe640 1 -- 172.21.15.31:0/4203328456 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c580ab600 con 0x7f3c6406e410 2023-11-26T07:20:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c637fe640 1 -- 172.21.15.31:0/4203328456 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c580a7050 con 0x7f3c6406e410 2023-11-26T07:20:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c637fe640 1 -- 172.21.15.31:0/4203328456 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c580ab980 con 0x7f3c6406e410 2023-11-26T07:20:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4203328456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c6406e410 msgr2=0x7f3c640715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c6ad65640 1 --2- 172.21.15.31:0/4203328456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c6406e410 0x7f3c640715b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009870 tx=0x7f3c5801c540 comp rx=0 tx=0).stop 2023-11-26T07:20:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4203328456 shutdown_connections 2023-11-26T07:20:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c6ad65640 1 --2- 172.21.15.31:0/4203328456 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3c64071af0 0x7f3c640b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c6ad65640 1 --2- 172.21.15.31:0/4203328456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c6406e410 0x7f3c640715b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4203328456 >> 172.21.15.31:0/4203328456 conn(0x7f3c640b1e40 msgr2=0x7f3c64070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:47.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4203328456 shutdown_connections 2023-11-26T07:20:47.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4203328456 wait complete. 2023-11-26T07:20:47.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c6ad65640 1 Processor -- start 2023-11-26T07:20:47.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.354+0000 7f3c6ad65640 1 -- start start 2023-11-26T07:20:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c6ad65640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c6406e410 0x7f3c6414d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c6ad65640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3c64071af0 0x7f3c6414f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c6ad65640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c640b2ee0 0x7f3c641518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c6ad65640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640cc320 con 0x7f3c640b2ee0 2023-11-26T07:20:47.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c6ad65640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6414d770 con 0x7f3c6406e410 2023-11-26T07:20:47.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c6ad65640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6414ec60 con 0x7f3c64071af0 2023-11-26T07:20:47.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c69302640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c640b2ee0 0x7f3c641518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c69302640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c640b2ee0 0x7f3c641518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35534/0 (socket says 172.21.15.31:35534) 2023-11-26T07:20:47.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c69302640 1 -- 172.21.15.31:0/4069083556 learned_addr learned my addr 172.21.15.31:0/4069083556 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:47.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c63fff640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3c64071af0 0x7f3c6414f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c68b01640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c6406e410 0x7f3c6414d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c69302640 1 -- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3c64071af0 msgr2=0x7f3c6414f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c69302640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3c64071af0 0x7f3c6414f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c69302640 1 -- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c6406e410 msgr2=0x7f3c6414d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c69302640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c6406e410 0x7f3c6414d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c69302640 1 -- 172.21.15.31:0/4069083556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c580094d0 con 0x7f3c640b2ee0 2023-11-26T07:20:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c68b01640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c6406e410 0x7f3c6414d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c69302640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c640b2ee0 0x7f3c641518c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c7a0 tx=0x7f3c5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:47.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c617fa640 1 -- 172.21.15.31:0/4069083556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c003b80 con 0x7f3c640b2ee0 2023-11-26T07:20:47.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c617fa640 1 -- 172.21.15.31:0/4069083556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c003ce0 con 0x7f3c640b2ee0 2023-11-26T07:20:47.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c617fa640 1 -- 172.21.15.31:0/4069083556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00aa90 con 0x7f3c640b2ee0 2023-11-26T07:20:47.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4069083556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c640b0290 con 0x7f3c640b2ee0 2023-11-26T07:20:47.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4069083556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c640b0850 con 0x7f3c640b2ee0 2023-11-26T07:20:47.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.358+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4069083556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c640cc460 con 0x7f3c640b2ee0 2023-11-26T07:20:47.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.362+0000 7f3c617fa640 1 -- 172.21.15.31:0/4069083556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f3c5c00abf0 con 0x7f3c640b2ee0 2023-11-26T07:20:47.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.362+0000 7f3c617fa640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c48066620 0x7f3c48068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.362+0000 7f3c617fa640 1 -- 172.21.15.31:0/4069083556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3c5c083f10 con 0x7f3c640b2ee0 2023-11-26T07:20:47.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.362+0000 7f3c68b01640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c48066620 0x7f3c48068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.362+0000 7f3c68b01640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c48066620 0x7f3c48068ae0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3c640af5e0 tx=0x7f3c580a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:47.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.366+0000 7f3c617fa640 1 -- 172.21.15.31:0/4069083556 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3c640cc460 con 0x7f3c640b2ee0 2023-11-26T07:20:47.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.522+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4069083556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c640cc460 con 0x7f3c640b2ee0 2023-11-26T07:20:47.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c617fa640 1 -- 172.21.15.31:0/4069083556 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f3c640cc460 con 0x7f3c640b2ee0 2023-11-26T07:20:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:47.528 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:19:15.232944+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:59.966450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:00.550730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.18","mappings":[{"from":0,"to":5}]},{"pgid":"7.2d","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:20:47.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c48066620 msgr2=0x7f3c48068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c48066620 0x7f3c48068ae0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3c640af5e0 tx=0x7f3c580a6000 comp rx=0 tx=0).stop 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4069083556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c640b2ee0 msgr2=0x7f3c641518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c640b2ee0 0x7f3c641518c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c7a0 tx=0x7f3c5c00cc60 comp rx=0 tx=0).stop 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4069083556 shutdown_connections 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3c48066620 0x7f3c48068ae0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c640b2ee0 0x7f3c641518c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3c64071af0 0x7f3c6414f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 --2- 172.21.15.31:0/4069083556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c6406e410 0x7f3c6414d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4069083556 >> 172.21.15.31:0/4069083556 conn(0x7f3c640b1e40 msgr2=0x7f3c6415dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4069083556 shutdown_connections 2023-11-26T07:20:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.526+0000 7f3c6ad65640 1 -- 172.21.15.31:0/4069083556 wait complete. 2023-11-26T07:20:47.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:18:59.966450+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:19:00.550730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:20:47.552 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:20:47.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.694+0000 7f059fbc1640 1 Processor -- start 2023-11-26T07:20:47.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.694+0000 7f059fbc1640 1 -- start start 2023-11-26T07:20:47.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.694+0000 7f059fbc1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05980cdfb0 0x7f05980abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.694+0000 7f059fbc1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05980cbd50 0x7f05980cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.694+0000 7f059fbc1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05980b2710 con 0x7f059815a710 2023-11-26T07:20:47.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.694+0000 7f059fbc1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980cca80 con 0x7f05980cbd50 2023-11-26T07:20:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.694+0000 7f059fbc1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980b29f0 con 0x7f05980cdfb0 2023-11-26T07:20:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059e15e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05980cbd50 0x7f05980cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059d15c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f059815a710 0x7f05980ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49526/0 (socket says 172.21.15.31:49526) 2023-11-26T07:20:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059d15c640 1 -- 172.21.15.31:0/551318930 learned_addr learned my addr 172.21.15.31:0/551318930 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059d95d640 1 --2- 172.21.15.31:0/551318930 >> v2:172.21.15.97:3300/0 conn(0x7f05980cdfb0 0x7f05980abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059e15e640 1 -- 172.21.15.31:0/551318930 >> v2:172.21.15.97:3300/0 conn(0x7f05980cdfb0 msgr2=0x7f05980abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059e15e640 1 --2- 172.21.15.31:0/551318930 >> v2:172.21.15.97:3300/0 conn(0x7f05980cdfb0 0x7f05980abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059e15e640 1 -- 172.21.15.31:0/551318930 >> v1:172.21.15.31:6789/0 conn(0x7f059815a710 legacy=0x7f05980ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059e15e640 1 -- 172.21.15.31:0/551318930 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980b2cd0 con 0x7f05980cbd50 2023-11-26T07:20:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059e15e640 1 --2- 172.21.15.31:0/551318930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05980cbd50 0x7f05980cc1e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f059806c270 tx=0x7f059001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3454bfd27ed22cc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:47.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059c95b640 1 -- 172.21.15.31:0/551318930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900a59c0 con 0x7f05980cbd50 2023-11-26T07:20:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059c95b640 1 -- 172.21.15.31:0/551318930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05900a5b20 con 0x7f05980cbd50 2023-11-26T07:20:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.698+0000 7f059c95b640 1 -- 172.21.15.31:0/551318930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900ad440 con 0x7f05980cbd50 2023-11-26T07:20:47.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 -- 172.21.15.31:0/551318930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05980cbd50 msgr2=0x7f05980cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 --2- 172.21.15.31:0/551318930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05980cbd50 0x7f05980cc1e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f059806c270 tx=0x7f059001c8e0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 -- 172.21.15.31:0/551318930 shutdown_connections 2023-11-26T07:20:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 --2- 172.21.15.31:0/551318930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05980cbd50 0x7f05980cc1e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 --2- 172.21.15.31:0/551318930 >> v2:172.21.15.97:3300/0 conn(0x7f05980cdfb0 0x7f05980abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 -- 172.21.15.31:0/551318930 >> 172.21.15.31:0/551318930 conn(0x7f05980b0cb0 msgr2=0x7f05980b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 -- 172.21.15.31:0/551318930 shutdown_connections 2023-11-26T07:20:47.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 -- 172.21.15.31:0/551318930 wait complete. 2023-11-26T07:20:47.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 Processor -- start 2023-11-26T07:20:47.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 -- start start 2023-11-26T07:20:47.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05980cbd50 0x7f059814e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f05980cdfb0 0x7f0598154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059815a710 0x7f0598152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059814eac0 con 0x7f059815a710 2023-11-26T07:20:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059814f8c0 con 0x7f05980cbd50 2023-11-26T07:20:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059fbc1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0598154b70 con 0x7f05980cdfb0 2023-11-26T07:20:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059e15e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059815a710 0x7f0598152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059e15e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059815a710 0x7f0598152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35552/0 (socket says 172.21.15.31:35552) 2023-11-26T07:20:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059e15e640 1 -- 172.21.15.31:0/714032502 learned_addr learned my addr 172.21.15.31:0/714032502 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059d95d640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05980cbd50 0x7f059814e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059d15c640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f05980cdfb0 0x7f0598154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059e15e640 1 -- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f05980cdfb0 msgr2=0x7f0598154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059e15e640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f05980cdfb0 0x7f0598154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059e15e640 1 -- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05980cbd50 msgr2=0x7f059814e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059e15e640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05980cbd50 0x7f059814e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.702+0000 7f059e15e640 1 -- 172.21.15.31:0/714032502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590009a70 con 0x7f059815a710 2023-11-26T07:20:47.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f059e15e640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059815a710 0x7f0598152a90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f05900194f0 tx=0x7f0590019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:47.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f058e7fc640 1 -- 172.21.15.31:0/714032502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900add70 con 0x7f059815a710 2023-11-26T07:20:47.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f058e7fc640 1 -- 172.21.15.31:0/714032502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0590009450 con 0x7f059815a710 2023-11-26T07:20:47.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f058e7fc640 1 -- 172.21.15.31:0/714032502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900b7650 con 0x7f059815a710 2023-11-26T07:20:47.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f059fbc1640 1 -- 172.21.15.31:0/714032502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059814ecb0 con 0x7f059815a710 2023-11-26T07:20:47.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f059fbc1640 1 -- 172.21.15.31:0/714032502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059814f270 con 0x7f059815a710 2023-11-26T07:20:47.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f059fbc1640 1 -- 172.21.15.31:0/714032502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05980671f0 con 0x7f059815a710 2023-11-26T07:20:47.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f058e7fc640 1 -- 172.21.15.31:0/714032502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f05900b4050 con 0x7f059815a710 2023-11-26T07:20:47.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f058e7fc640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f057c066620 0x7f057c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:47.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f058e7fc640 1 -- 172.21.15.31:0/714032502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f059011f490 con 0x7f059815a710 2023-11-26T07:20:47.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.706+0000 7f059d95d640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f057c066620 0x7f057c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:47.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.710+0000 7f059d95d640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f057c066620 0x7f057c068ae0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f059400b9d0 tx=0x7f0594008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:47.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.710+0000 7f058e7fc640 1 -- 172.21.15.31:0/714032502 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f05900ec5b0 con 0x7f059815a710 2023-11-26T07:20:47.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.866+0000 7f059fbc1640 1 -- 172.21.15.31:0/714032502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05981661d0 con 0x7f059815a710 2023-11-26T07:20:47.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.866+0000 7f058e7fc640 1 -- 172.21.15.31:0/714032502 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05981661d0 con 0x7f059815a710 2023-11-26T07:20:47.870 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:20:47.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 -- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f057c066620 msgr2=0x7f057c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f057c066620 0x7f057c068ae0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f059400b9d0 tx=0x7f0594008040 comp rx=0 tx=0).stop 2023-11-26T07:20:47.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 -- 172.21.15.31:0/714032502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059815a710 msgr2=0x7f0598152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:47.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059815a710 0x7f0598152a90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f05900194f0 tx=0x7f0590019680 comp rx=0 tx=0).stop 2023-11-26T07:20:47.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 -- 172.21.15.31:0/714032502 shutdown_connections 2023-11-26T07:20:47.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f057c066620 0x7f057c068ae0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f059815a710 0x7f0598152a90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f05980cdfb0 0x7f0598154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 --2- 172.21.15.31:0/714032502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05980cbd50 0x7f059814e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:47.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 -- 172.21.15.31:0/714032502 >> 172.21.15.31:0/714032502 conn(0x7f05980b0cb0 msgr2=0x7f05980acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:47.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 -- 172.21.15.31:0/714032502 shutdown_connections 2023-11-26T07:20:47.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:47.870+0000 7f059fbc1640 1 -- 172.21.15.31:0/714032502 wait complete. 2023-11-26T07:20:47.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:20:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.034+0000 7f2d5e65a640 1 Processor -- start 2023-11-26T07:20:48.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.034+0000 7f2d5e65a640 1 -- start start 2023-11-26T07:20:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5e65a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d580cdfb0 0x7f2d580abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5e65a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d580cbd50 0x7f2d580cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5e65a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d580b2710 con 0x7f2d5815a710 2023-11-26T07:20:48.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5e65a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d580cca80 con 0x7f2d580cbd50 2023-11-26T07:20:48.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5e65a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d580b29f0 con 0x7f2d580cdfb0 2023-11-26T07:20:48.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5cbf7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d580cbd50 0x7f2d580cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d577fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2d5815a710 0x7f2d580ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:58042/0 (socket says 172.21.15.31:58042) 2023-11-26T07:20:48.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d577fe640 1 -- 172.21.15.31:0/4191364363 learned_addr learned my addr 172.21.15.31:0/4191364363 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:48.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d57fff640 1 --2- 172.21.15.31:0/4191364363 >> v2:172.21.15.97:3300/0 conn(0x7f2d580cdfb0 0x7f2d580abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5cbf7640 1 -- 172.21.15.31:0/4191364363 >> v2:172.21.15.97:3300/0 conn(0x7f2d580cdfb0 msgr2=0x7f2d580abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5cbf7640 1 --2- 172.21.15.31:0/4191364363 >> v2:172.21.15.97:3300/0 conn(0x7f2d580cdfb0 0x7f2d580abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5cbf7640 1 -- 172.21.15.31:0/4191364363 >> v1:172.21.15.97:6789/0 conn(0x7f2d5815a710 legacy=0x7f2d580ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5cbf7640 1 -- 172.21.15.31:0/4191364363 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d580b2cd0 con 0x7f2d580cbd50 2023-11-26T07:20:48.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d56ffd640 1 -- 172.21.15.31:0/4191364363 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153232666 0 0) 0x7f2d580b2710 con 0x7f2d5815a710 2023-11-26T07:20:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.038+0000 7f2d5cbf7640 1 --2- 172.21.15.31:0/4191364363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cbd50 0x7f2d580cc1e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2d5806c270 tx=0x7f2d5001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c851eb59d396ee server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d56ffd640 1 -- 172.21.15.31:0/4191364363 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d500a59c0 con 0x7f2d580cbd50 2023-11-26T07:20:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d56ffd640 1 -- 172.21.15.31:0/4191364363 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d500a5b20 con 0x7f2d580cbd50 2023-11-26T07:20:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d56ffd640 1 -- 172.21.15.31:0/4191364363 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d500ad440 con 0x7f2d580cbd50 2023-11-26T07:20:48.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 -- 172.21.15.31:0/4191364363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cbd50 msgr2=0x7f2d580cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 --2- 172.21.15.31:0/4191364363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cbd50 0x7f2d580cc1e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2d5806c270 tx=0x7f2d5001c8e0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 -- 172.21.15.31:0/4191364363 shutdown_connections 2023-11-26T07:20:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 --2- 172.21.15.31:0/4191364363 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cbd50 0x7f2d580cc1e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 --2- 172.21.15.31:0/4191364363 >> v2:172.21.15.97:3300/0 conn(0x7f2d580cdfb0 0x7f2d580abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 -- 172.21.15.31:0/4191364363 >> 172.21.15.31:0/4191364363 conn(0x7f2d580b0cb0 msgr2=0x7f2d580b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 -- 172.21.15.31:0/4191364363 shutdown_connections 2023-11-26T07:20:48.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 -- 172.21.15.31:0/4191364363 wait complete. 2023-11-26T07:20:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 Processor -- start 2023-11-26T07:20:48.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 -- start start 2023-11-26T07:20:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d580cbd50 0x7f2d58166be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cdfb0 0x7f2d58151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5815a710 0x7f2d5815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d58151db0 con 0x7f2d580cdfb0 2023-11-26T07:20:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d58152090 con 0x7f2d5815a710 2023-11-26T07:20:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.042+0000 7f2d5e65a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d58152370 con 0x7f2d580cbd50 2023-11-26T07:20:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cdfb0 0x7f2d58151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d57fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d580cbd50 0x7f2d58166be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cdfb0 0x7f2d58151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35568/0 (socket says 172.21.15.31:35568) 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d5cbf7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5815a710 0x7f2d5815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d577fe640 1 -- 172.21.15.31:0/1877182760 learned_addr learned my addr 172.21.15.31:0/1877182760 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d577fe640 1 -- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d580cbd50 msgr2=0x7f2d58166be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d577fe640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d580cbd50 0x7f2d58166be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d577fe640 1 -- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5815a710 msgr2=0x7f2d5815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d577fe640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5815a710 0x7f2d5815ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d577fe640 1 -- 172.21.15.31:0/1877182760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d50009a70 con 0x7f2d580cdfb0 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d577fe640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cdfb0 0x7f2d58151350 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c450 tx=0x7f2d4c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d54ff9640 1 -- 172.21.15.31:0/1877182760 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0074e0 con 0x7f2d580cdfb0 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d54ff9640 1 -- 172.21.15.31:0/1877182760 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c007640 con 0x7f2d580cdfb0 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d54ff9640 1 -- 172.21.15.31:0/1877182760 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0049f0 con 0x7f2d580cdfb0 2023-11-26T07:20:48.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d5e65a640 1 -- 172.21.15.31:0/1877182760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d58152650 con 0x7f2d580cdfb0 2023-11-26T07:20:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d5e65a640 1 -- 172.21.15.31:0/1877182760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d58167fc0 con 0x7f2d580cdfb0 2023-11-26T07:20:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d5e65a640 1 -- 172.21.15.31:0/1877182760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d580671f0 con 0x7f2d580cdfb0 2023-11-26T07:20:48.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d54ff9640 1 -- 172.21.15.31:0/1877182760 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f2d4c004b50 con 0x7f2d580cdfb0 2023-11-26T07:20:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.046+0000 7f2d54ff9640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d44066620 0x7f2d44068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.050+0000 7f2d54ff9640 1 -- 172.21.15.31:0/1877182760 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2d4c083330 con 0x7f2d580cdfb0 2023-11-26T07:20:48.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.050+0000 7f2d57fff640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d44066620 0x7f2d44068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.050+0000 7f2d57fff640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d44066620 0x7f2d44068ae0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009b60 tx=0x7f2d48009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:48.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.050+0000 7f2d54ff9640 1 -- 172.21.15.31:0/1877182760 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d4c0505c0 con 0x7f2d580cdfb0 2023-11-26T07:20:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.206+0000 7f2d5e65a640 1 -- 172.21.15.31:0/1877182760 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d5806b460 con 0x7f2d580cdfb0 2023-11-26T07:20:48.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d54ff9640 1 -- 172.21.15.31:0/1877182760 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d4c0180a0 con 0x7f2d580cdfb0 2023-11-26T07:20:48.211 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:20:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 -- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d44066620 msgr2=0x7f2d44068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d44066620 0x7f2d44068ae0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009b60 tx=0x7f2d48009190 comp rx=0 tx=0).stop 2023-11-26T07:20:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 -- 172.21.15.31:0/1877182760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cdfb0 msgr2=0x7f2d58151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cdfb0 0x7f2d58151350 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c450 tx=0x7f2d4c00c910 comp rx=0 tx=0).stop 2023-11-26T07:20:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 -- 172.21.15.31:0/1877182760 shutdown_connections 2023-11-26T07:20:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d44066620 0x7f2d44068ae0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d5815a710 0x7f2d5815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d580cdfb0 0x7f2d58151350 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 --2- 172.21.15.31:0/1877182760 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d580cbd50 0x7f2d58166be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 -- 172.21.15.31:0/1877182760 >> 172.21.15.31:0/1877182760 conn(0x7f2d580b0cb0 msgr2=0x7f2d580ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 -- 172.21.15.31:0/1877182760 shutdown_connections 2023-11-26T07:20:48.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.210+0000 7f2d5e65a640 1 -- 172.21.15.31:0/1877182760 wait complete. 2023-11-26T07:20:48.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:20:48.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.374+0000 7faf9cb57640 1 Processor -- start 2023-11-26T07:20:48.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9cb57640 1 -- start start 2023-11-26T07:20:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9cb57640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf980cb9b0 0x7faf9806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9cb57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf980cbd90 0x7faf9815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9cb57640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf98159630 con 0x7faf980ce5d0 2023-11-26T07:20:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9cb57640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9815bba0 con 0x7faf980cb9b0 2023-11-26T07:20:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9cb57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf980cdd50 con 0x7faf980cbd90 2023-11-26T07:20:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9659c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf980cb9b0 0x7faf9806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9659c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7faf980cb9b0 0x7faf9806e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35574/0 (socket says 172.21.15.31:35574) 2023-11-26T07:20:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9659c640 1 -- 172.21.15.31:0/2871043221 learned_addr learned my addr 172.21.15.31:0/2871043221 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:48.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf8ffff640 1 --2- 172.21.15.31:0/2871043221 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf980cbd90 0x7faf9815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9659c640 1 -- 172.21.15.31:0/2871043221 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf980cbd90 msgr2=0x7faf9815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:48.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9659c640 1 --2- 172.21.15.31:0/2871043221 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf980cbd90 0x7faf9815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9659c640 1 -- 172.21.15.31:0/2871043221 >> v1:172.21.15.97:6789/0 conn(0x7faf980ce5d0 legacy=0x7faf9815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.378+0000 7faf9659c640 1 -- 172.21.15.31:0/2871043221 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9815f6d0 con 0x7faf980cb9b0 2023-11-26T07:20:48.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9659c640 1 --2- 172.21.15.31:0/2871043221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980cb9b0 0x7faf9806e3f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7faf88009870 tx=0x7faf8801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd5593a6056234f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:48.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf95d9b640 1 -- 172.21.15.31:0/2871043221 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf880aa670 con 0x7faf980cb9b0 2023-11-26T07:20:48.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf95d9b640 1 -- 172.21.15.31:0/2871043221 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf880aa7d0 con 0x7faf980cb9b0 2023-11-26T07:20:48.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf95d9b640 1 -- 172.21.15.31:0/2871043221 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf880aab60 con 0x7faf980cb9b0 2023-11-26T07:20:48.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9cb57640 1 -- 172.21.15.31:0/2871043221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980cb9b0 msgr2=0x7faf9806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9cb57640 1 --2- 172.21.15.31:0/2871043221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980cb9b0 0x7faf9806e3f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7faf88009870 tx=0x7faf8801c510 comp rx=0 tx=0).stop 2023-11-26T07:20:48.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9cb57640 1 -- 172.21.15.31:0/2871043221 shutdown_connections 2023-11-26T07:20:48.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9cb57640 1 --2- 172.21.15.31:0/2871043221 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf980cbd90 0x7faf9815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9cb57640 1 --2- 172.21.15.31:0/2871043221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980cb9b0 0x7faf9806e3f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9cb57640 1 -- 172.21.15.31:0/2871043221 >> 172.21.15.31:0/2871043221 conn(0x7faf980b0cb0 msgr2=0x7faf980b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:48.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9cb57640 1 -- 172.21.15.31:0/2871043221 shutdown_connections 2023-11-26T07:20:48.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9cb57640 1 -- 172.21.15.31:0/2871043221 wait complete. 2023-11-26T07:20:48.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9cb57640 1 Processor -- start 2023-11-26T07:20:48.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.382+0000 7faf9cb57640 1 -- start start 2023-11-26T07:20:48.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf9cb57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf980cb9b0 0x7faf980afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf9cb57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf980cbd90 0x7faf98155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf9cb57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980ce5d0 0x7faf981458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf9cb57640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf98156ea0 con 0x7faf980ce5d0 2023-11-26T07:20:48.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf9cb57640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf98157180 con 0x7faf980cb9b0 2023-11-26T07:20:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf9cb57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf98157460 con 0x7faf980cbd90 2023-11-26T07:20:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf96d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980ce5d0 0x7faf981458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf96d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980ce5d0 0x7faf981458c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35586/0 (socket says 172.21.15.31:35586) 2023-11-26T07:20:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf96d9d640 1 -- 172.21.15.31:0/990197144 learned_addr learned my addr 172.21.15.31:0/990197144 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf8ffff640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf980cbd90 0x7faf98155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf9659c640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf980cb9b0 0x7faf980afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf96d9d640 1 -- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf980cbd90 msgr2=0x7faf98155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf96d9d640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf980cbd90 0x7faf98155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf96d9d640 1 -- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf980cb9b0 msgr2=0x7faf980afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf96d9d640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf980cb9b0 0x7faf980afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf96d9d640 1 -- 172.21.15.31:0/990197144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf880094d0 con 0x7faf980ce5d0 2023-11-26T07:20:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf96d9d640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980ce5d0 0x7faf981458c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faf9000c730 tx=0x7faf9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf8effd640 1 -- 172.21.15.31:0/990197144 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90003b80 con 0x7faf980ce5d0 2023-11-26T07:20:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf8effd640 1 -- 172.21.15.31:0/990197144 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf90003ce0 con 0x7faf980ce5d0 2023-11-26T07:20:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf9cb57640 1 -- 172.21.15.31:0/990197144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf98157740 con 0x7faf980ce5d0 2023-11-26T07:20:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf8effd640 1 -- 172.21.15.31:0/990197144 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9000aa30 con 0x7faf980ce5d0 2023-11-26T07:20:48.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf9cb57640 1 -- 172.21.15.31:0/990197144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf98145e00 con 0x7faf980ce5d0 2023-11-26T07:20:48.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.386+0000 7faf9cb57640 1 -- 172.21.15.31:0/990197144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf980671f0 con 0x7faf980ce5d0 2023-11-26T07:20:48.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.390+0000 7faf8effd640 1 -- 172.21.15.31:0/990197144 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7faf9000ab90 con 0x7faf980ce5d0 2023-11-26T07:20:48.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.390+0000 7faf8effd640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf78066620 0x7faf78068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.390+0000 7faf8effd640 1 -- 172.21.15.31:0/990197144 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7faf90083e30 con 0x7faf980ce5d0 2023-11-26T07:20:48.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.390+0000 7faf9659c640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf78066620 0x7faf78068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.390+0000 7faf9659c640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf78066620 0x7faf78068ae0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf880ae000 tx=0x7faf880b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:48.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.394+0000 7faf8effd640 1 -- 172.21.15.31:0/990197144 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faf90051090 con 0x7faf980ce5d0 2023-11-26T07:20:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.550+0000 7faf9cb57640 1 -- 172.21.15.31:0/990197144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf9806b460 con 0x7faf980ce5d0 2023-11-26T07:20:48.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.550+0000 7faf8effd640 1 -- 172.21.15.31:0/990197144 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf900546b0 con 0x7faf980ce5d0 2023-11-26T07:20:48.554 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:20:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 -- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf78066620 msgr2=0x7faf78068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf78066620 0x7faf78068ae0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7faf880ae000 tx=0x7faf880b5040 comp rx=0 tx=0).stop 2023-11-26T07:20:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 -- 172.21.15.31:0/990197144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980ce5d0 msgr2=0x7faf981458c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980ce5d0 0x7faf981458c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7faf9000c730 tx=0x7faf9000cbf0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 -- 172.21.15.31:0/990197144 shutdown_connections 2023-11-26T07:20:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7faf78066620 0x7faf78068ae0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf980ce5d0 0x7faf981458c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faf980cbd90 0x7faf98155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 --2- 172.21.15.31:0/990197144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf980cb9b0 0x7faf980afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 -- 172.21.15.31:0/990197144 >> 172.21.15.31:0/990197144 conn(0x7faf980b0cb0 msgr2=0x7faf9806e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 -- 172.21.15.31:0/990197144 shutdown_connections 2023-11-26T07:20:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.554+0000 7faf9cb57640 1 -- 172.21.15.31:0/990197144 wait complete. 2023-11-26T07:20:48.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:48.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.714+0000 7fb9bb6f7640 1 Processor -- start 2023-11-26T07:20:48.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.714+0000 7fb9bb6f7640 1 -- start start 2023-11-26T07:20:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9bb6f7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9b4144c80 0x7fb9b40abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9bb6f7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9b4070df0 0x7fb9b406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9bb6f7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b4145060 con 0x7fb9b40ac460 2023-11-26T07:20:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9bb6f7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b4071fd0 con 0x7fb9b4144c80 2023-11-26T07:20:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9bb6f7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b40ac8c0 con 0x7fb9b4070df0 2023-11-26T07:20:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9b9493640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9b4144c80 0x7fb9b40abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9b9c94640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9b4070df0 0x7fb9b406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9b9493640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb9b4144c80 0x7fb9b40abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51988/0 (socket says 172.21.15.31:51988) 2023-11-26T07:20:48.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9b8c92640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fb9b40ac460 0x7fb9b40706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:58058/0 (socket says 172.21.15.31:58058) 2023-11-26T07:20:48.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9b9493640 1 -- 172.21.15.31:0/289956518 learned_addr learned my addr 172.21.15.31:0/289956518 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:48.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9abfff640 1 -- 172.21.15.31:0/289956518 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145021294 0 0) 0x7fb9b4145060 con 0x7fb9b40ac460 2023-11-26T07:20:48.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9b9493640 1 -- 172.21.15.31:0/289956518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9b4070df0 msgr2=0x7fb9b406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:48.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9b9493640 1 --2- 172.21.15.31:0/289956518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9b4070df0 0x7fb9b406c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9b9493640 1 -- 172.21.15.31:0/289956518 >> v1:172.21.15.97:6789/0 conn(0x7fb9b40ac460 legacy=0x7fb9b40706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9b9493640 1 -- 172.21.15.31:0/289956518 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b4071450 con 0x7fb9b4144c80 2023-11-26T07:20:48.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.718+0000 7fb9b9493640 1 --2- 172.21.15.31:0/289956518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9b4144c80 0x7fb9b40abf20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009870 tx=0x7fb9b001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb0668f0f7910968 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9abfff640 1 -- 172.21.15.31:0/289956518 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00a7070 con 0x7fb9b4144c80 2023-11-26T07:20:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9abfff640 1 -- 172.21.15.31:0/289956518 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b00ab430 con 0x7fb9b4144c80 2023-11-26T07:20:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9abfff640 1 -- 172.21.15.31:0/289956518 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00ab7c0 con 0x7fb9b4144c80 2023-11-26T07:20:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/289956518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9b4144c80 msgr2=0x7fb9b40abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 --2- 172.21.15.31:0/289956518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9b4144c80 0x7fb9b40abf20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009870 tx=0x7fb9b001c540 comp rx=0 tx=0).stop 2023-11-26T07:20:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/289956518 shutdown_connections 2023-11-26T07:20:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 --2- 172.21.15.31:0/289956518 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9b4070df0 0x7fb9b406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 --2- 172.21.15.31:0/289956518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9b4144c80 0x7fb9b40abf20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/289956518 >> 172.21.15.31:0/289956518 conn(0x7fb9b40b0c40 msgr2=0x7fb9b40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:48.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/289956518 shutdown_connections 2023-11-26T07:20:48.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/289956518 wait complete. 2023-11-26T07:20:48.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 Processor -- start 2023-11-26T07:20:48.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 -- start start 2023-11-26T07:20:48.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9b4070df0 0x7fb9b415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9b40ac460 0x7fb9b40b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9b4144c80 0x7fb9b40b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b40b2df0 con 0x7fb9b40ac460 2023-11-26T07:20:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b40b21e0 con 0x7fb9b4144c80 2023-11-26T07:20:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9bb6f7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b40b24c0 con 0x7fb9b4070df0 2023-11-26T07:20:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9b8c92640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9b40ac460 0x7fb9b40b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9b8c92640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9b40ac460 0x7fb9b40b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35602/0 (socket says 172.21.15.31:35602) 2023-11-26T07:20:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.722+0000 7fb9b8c92640 1 -- 172.21.15.31:0/1443770480 learned_addr learned my addr 172.21.15.31:0/1443770480 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9b9c94640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9b4144c80 0x7fb9b40b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9b9493640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9b4070df0 0x7fb9b415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9b8c92640 1 -- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9b4070df0 msgr2=0x7fb9b415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9b8c92640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9b4070df0 0x7fb9b415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9b8c92640 1 -- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9b4144c80 msgr2=0x7fb9b40b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9b8c92640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9b4144c80 0x7fb9b40b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9b8c92640 1 -- 172.21.15.31:0/1443770480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00094d0 con 0x7fb9b40ac460 2023-11-26T07:20:48.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9b8c92640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9b40ac460 0x7fb9b40b3280 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40011f0 tx=0x7fb9a400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:48.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9a9ffb640 1 -- 172.21.15.31:0/1443770480 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a40074e0 con 0x7fb9b40ac460 2023-11-26T07:20:48.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9a9ffb640 1 -- 172.21.15.31:0/1443770480 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a4007640 con 0x7fb9b40ac460 2023-11-26T07:20:48.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/1443770480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b40b27a0 con 0x7fb9b40ac460 2023-11-26T07:20:48.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9a9ffb640 1 -- 172.21.15.31:0/1443770480 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4004a10 con 0x7fb9b40ac460 2023-11-26T07:20:48.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/1443770480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b415b370 con 0x7fb9b40ac460 2023-11-26T07:20:48.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/1443770480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b40b1ba0 con 0x7fb9b40ac460 2023-11-26T07:20:48.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9a9ffb640 1 -- 172.21.15.31:0/1443770480 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fb9a4004b70 con 0x7fb9b40ac460 2023-11-26T07:20:48.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.726+0000 7fb9a9ffb640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb998066620 0x7fb998068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:48.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.730+0000 7fb9a9ffb640 1 -- 172.21.15.31:0/1443770480 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb9a4083640 con 0x7fb9b40ac460 2023-11-26T07:20:48.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.730+0000 7fb9b9493640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb998066620 0x7fb998068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:48.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.730+0000 7fb9b9493640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb998066620 0x7fb998068ae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00af000 tx=0x7fb9b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:48.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.730+0000 7fb9a9ffb640 1 -- 172.21.15.31:0/1443770480 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb9a40508c0 con 0x7fb9b40ac460 2023-11-26T07:20:48.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/1443770480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9b404e850 con 0x7fb9b40ac460 2023-11-26T07:20:48.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9a9ffb640 1 -- 172.21.15.31:0/1443770480 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fb9a4053ee0 con 0x7fb9b40ac460 2023-11-26T07:20:48.925 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:48.925 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:48.926 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb998066620 msgr2=0x7fb998068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb998066620 0x7fb998068ae0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00af000 tx=0x7fb9b0008040 comp rx=0 tx=0).stop 2023-11-26T07:20:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/1443770480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9b40ac460 msgr2=0x7fb9b40b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9b40ac460 0x7fb9b40b3280 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40011f0 tx=0x7fb9a400c9b0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/1443770480 shutdown_connections 2023-11-26T07:20:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb998066620 0x7fb998068ae0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9b4144c80 0x7fb9b40b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9b40ac460 0x7fb9b40b3280 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 --2- 172.21.15.31:0/1443770480 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb9b4070df0 0x7fb9b415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:48.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/1443770480 >> 172.21.15.31:0/1443770480 conn(0x7fb9b40b0c40 msgr2=0x7fb9b40b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/1443770480 shutdown_connections 2023-11-26T07:20:48.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:48.922+0000 7fb9bb6f7640 1 -- 172.21.15.31:0/1443770480 wait complete. 2023-11-26T07:20:48.945 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:20:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366edb8640 1 Processor -- start 2023-11-26T07:20:49.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366edb8640 1 -- start start 2023-11-26T07:20:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366edb8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3668144c80 0x7f36680abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366edb8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3668070df0 0x7f366806c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366edb8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3668145060 con 0x7f36680ac460 2023-11-26T07:20:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366edb8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3668071fd0 con 0x7f3668144c80 2023-11-26T07:20:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366edb8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36680ac8c0 con 0x7f3668070df0 2023-11-26T07:20:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366cb54640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3668144c80 0x7f36680abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f3667fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f36680ac460 0x7f36680706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49538/0 (socket says 172.21.15.31:49538) 2023-11-26T07:20:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f3667fff640 1 -- 172.21.15.31:0/2332136366 learned_addr learned my addr 172.21.15.31:0/2332136366 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366d355640 1 --2- 172.21.15.31:0/2332136366 >> v2:172.21.15.97:3300/0 conn(0x7f3668070df0 0x7f366806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366cb54640 1 -- 172.21.15.31:0/2332136366 >> v2:172.21.15.97:3300/0 conn(0x7f3668070df0 msgr2=0x7f366806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366cb54640 1 --2- 172.21.15.31:0/2332136366 >> v2:172.21.15.97:3300/0 conn(0x7f3668070df0 0x7f366806c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366cb54640 1 -- 172.21.15.31:0/2332136366 >> v1:172.21.15.31:6789/0 conn(0x7f36680ac460 legacy=0x7f36680706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f36677fe640 1 -- 172.21.15.31:0/2332136366 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615348673 0 0) 0x7f3668145060 con 0x7f36680ac460 2023-11-26T07:20:49.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.086+0000 7f366cb54640 1 -- 172.21.15.31:0/2332136366 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3668071450 con 0x7f3668144c80 2023-11-26T07:20:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f366cb54640 1 --2- 172.21.15.31:0/2332136366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3668144c80 0x7f36680abf20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f36680ce840 tx=0x7f365c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba1f24a1bcdf019 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:49.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f36677fe640 1 -- 172.21.15.31:0/2332136366 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c004230 con 0x7f3668144c80 2023-11-26T07:20:49.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f36677fe640 1 -- 172.21.15.31:0/2332136366 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c0a7050 con 0x7f3668144c80 2023-11-26T07:20:49.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f36677fe640 1 -- 172.21.15.31:0/2332136366 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c003620 con 0x7f3668144c80 2023-11-26T07:20:49.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f366edb8640 1 -- 172.21.15.31:0/2332136366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3668144c80 msgr2=0x7f36680abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f366edb8640 1 --2- 172.21.15.31:0/2332136366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3668144c80 0x7f36680abf20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f36680ce840 tx=0x7f365c01c460 comp rx=0 tx=0).stop 2023-11-26T07:20:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f366edb8640 1 -- 172.21.15.31:0/2332136366 shutdown_connections 2023-11-26T07:20:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f366edb8640 1 --2- 172.21.15.31:0/2332136366 >> v2:172.21.15.97:3300/0 conn(0x7f3668070df0 0x7f366806c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f366edb8640 1 --2- 172.21.15.31:0/2332136366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3668144c80 0x7f36680abf20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f366edb8640 1 -- 172.21.15.31:0/2332136366 >> 172.21.15.31:0/2332136366 conn(0x7f36680b0c40 msgr2=0x7f36680b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:49.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f366edb8640 1 -- 172.21.15.31:0/2332136366 shutdown_connections 2023-11-26T07:20:49.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.090+0000 7f366edb8640 1 -- 172.21.15.31:0/2332136366 wait complete. 2023-11-26T07:20:49.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366edb8640 1 Processor -- start 2023-11-26T07:20:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366edb8640 1 -- start start 2023-11-26T07:20:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366edb8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3668070df0 0x7f3668140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366edb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36680ac460 0x7f366813a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366edb8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3668144c80 0x7f366813a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366edb8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36681416a0 con 0x7f36680ac460 2023-11-26T07:20:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366edb8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36681f9160 con 0x7f3668070df0 2023-11-26T07:20:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366edb8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36681f9440 con 0x7f3668144c80 2023-11-26T07:20:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f3667fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36680ac460 0x7f366813a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f3667fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36680ac460 0x7f366813a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35626/0 (socket says 172.21.15.31:35626) 2023-11-26T07:20:49.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f3667fff640 1 -- 172.21.15.31:0/3651907623 learned_addr learned my addr 172.21.15.31:0/3651907623 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366d355640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3668144c80 0x7f366813a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366cb54640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3668070df0 0x7f3668140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f3667fff640 1 -- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3668144c80 msgr2=0x7f366813a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f3667fff640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3668144c80 0x7f366813a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f3667fff640 1 -- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3668070df0 msgr2=0x7f3668140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f3667fff640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3668070df0 0x7f3668140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f3667fff640 1 -- 172.21.15.31:0/3651907623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c009420 con 0x7f36680ac460 2023-11-26T07:20:49.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366d355640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3668144c80 0x7f366813a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:49.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f3667fff640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36680ac460 0x7f366813a170 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f36540011f0 tx=0x7f365400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f36657fa640 1 -- 172.21.15.31:0/3651907623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540074e0 con 0x7f36680ac460 2023-11-26T07:20:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f36657fa640 1 -- 172.21.15.31:0/3651907623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3654007640 con 0x7f36680ac460 2023-11-26T07:20:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366edb8640 1 -- 172.21.15.31:0/3651907623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36681f9720 con 0x7f36680ac460 2023-11-26T07:20:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f36657fa640 1 -- 172.21.15.31:0/3651907623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654004a10 con 0x7f36680ac460 2023-11-26T07:20:49.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.094+0000 7f366edb8640 1 -- 172.21.15.31:0/3651907623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3668141300 con 0x7f36680ac460 2023-11-26T07:20:49.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.098+0000 7f366edb8640 1 -- 172.21.15.31:0/3651907623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36680b1ba0 con 0x7f36680ac460 2023-11-26T07:20:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.098+0000 7f36657fa640 1 -- 172.21.15.31:0/3651907623 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f3654004b70 con 0x7f36680ac460 2023-11-26T07:20:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.098+0000 7f36657fa640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f364c066670 0x7f364c068b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.098+0000 7f36657fa640 1 -- 172.21.15.31:0/3651907623 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f36540838e0 con 0x7f36680ac460 2023-11-26T07:20:49.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.098+0000 7f366cb54640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f364c066670 0x7f364c068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.098+0000 7f366cb54640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f364c066670 0x7f364c068b30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f365c0ad000 tx=0x7f365c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:49.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.102+0000 7f36657fa640 1 -- 172.21.15.31:0/3651907623 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3654050a80 con 0x7f36680ac460 2023-11-26T07:20:49.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.262+0000 7f366edb8640 1 -- 172.21.15.31:0/3651907623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f366804e850 con 0x7f36680ac460 2023-11-26T07:20:49.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.262+0000 7f36657fa640 1 -- 172.21.15.31:0/3651907623 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f3654018090 con 0x7f36680ac460 2023-11-26T07:20:49.265 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:49.266 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:19:15.232944+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:59.966450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:00.550730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.18","mappings":[{"from":0,"to":5}]},{"pgid":"7.2d","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 -- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f364c066670 msgr2=0x7f364c068b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f364c066670 0x7f364c068b30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f365c0ad000 tx=0x7f365c0a6000 comp rx=0 tx=0).stop 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 -- 172.21.15.31:0/3651907623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36680ac460 msgr2=0x7f366813a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36680ac460 0x7f366813a170 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f36540011f0 tx=0x7f365400c9b0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 -- 172.21.15.31:0/3651907623 shutdown_connections 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f364c066670 0x7f364c068b30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3668144c80 0x7f366813a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36680ac460 0x7f366813a170 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 --2- 172.21.15.31:0/3651907623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3668070df0 0x7f3668140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 -- 172.21.15.31:0/3651907623 >> 172.21.15.31:0/3651907623 conn(0x7f36680b0c40 msgr2=0x7f36680b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:49.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 -- 172.21.15.31:0/3651907623 shutdown_connections 2023-11-26T07:20:49.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.266+0000 7f366edb8640 1 -- 172.21.15.31:0/3651907623 wait complete. 2023-11-26T07:20:49.288 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:20:49.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bf59e640 1 Processor -- start 2023-11-26T07:20:49.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bf59e640 1 -- start start 2023-11-26T07:20:49.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bf59e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9c015d880 0x7fd9c00b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bf59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00d3e70 0x7fd9c00cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bf59e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c00d15f0 con 0x7fd9c00b6c70 2023-11-26T07:20:49.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bf59e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c00ce000 con 0x7fd9c015d880 2023-11-26T07:20:49.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bf59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00d0630 con 0x7fd9c00d3e70 2023-11-26T07:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9be59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9c015d880 0x7fd9c00b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bed9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00d3e70 0x7fd9c00cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9b7fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fd9c00b6c70 0x7fd9c00d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:58060/0 (socket says 172.21.15.31:58060) 2023-11-26T07:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9be59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd9c015d880 0x7fd9c00b6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:52028/0 (socket says 172.21.15.31:52028) 2023-11-26T07:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bed9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00d3e70 0x7fd9c00cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:60730/0 (socket says 172.21.15.31:60730) 2023-11-26T07:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9b7fff640 1 -- 172.21.15.31:0/3839467320 learned_addr learned my addr 172.21.15.31:0/3839467320 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bed9d640 1 -- 172.21.15.31:0/3839467320 >> v2:172.21.15.97:3300/0 conn(0x7fd9c015d880 msgr2=0x7fd9c00b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bed9d640 1 --2- 172.21.15.31:0/3839467320 >> v2:172.21.15.97:3300/0 conn(0x7fd9c015d880 0x7fd9c00b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bed9d640 1 -- 172.21.15.31:0/3839467320 >> v1:172.21.15.97:6789/0 conn(0x7fd9c00b6c70 legacy=0x7fd9c00d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bdd9b640 1 -- 172.21.15.31:0/3839467320 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237718077 0 0) 0x7fd9c00d15f0 con 0x7fd9c00b6c70 2023-11-26T07:20:49.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.430+0000 7fd9bed9d640 1 -- 172.21.15.31:0/3839467320 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00d0910 con 0x7fd9c00d3e70 2023-11-26T07:20:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bed9d640 1 --2- 172.21.15.31:0/3839467320 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00d3e70 0x7fd9c00cfa20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009d90 tx=0x7fd9b801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd9cc28406dd9c6f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bdd9b640 1 -- 172.21.15.31:0/3839467320 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80a5a30 con 0x7fd9c00d3e70 2023-11-26T07:20:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bdd9b640 1 -- 172.21.15.31:0/3839467320 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b80a5b90 con 0x7fd9c00d3e70 2023-11-26T07:20:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bdd9b640 1 -- 172.21.15.31:0/3839467320 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80ad440 con 0x7fd9c00d3e70 2023-11-26T07:20:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bf59e640 1 -- 172.21.15.31:0/3839467320 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00d3e70 msgr2=0x7fd9c00cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bf59e640 1 --2- 172.21.15.31:0/3839467320 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00d3e70 0x7fd9c00cfa20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009d90 tx=0x7fd9b801c950 comp rx=0 tx=0).stop 2023-11-26T07:20:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bf59e640 1 -- 172.21.15.31:0/3839467320 shutdown_connections 2023-11-26T07:20:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bf59e640 1 --2- 172.21.15.31:0/3839467320 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00d3e70 0x7fd9c00cfa20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bf59e640 1 --2- 172.21.15.31:0/3839467320 >> v2:172.21.15.97:3300/0 conn(0x7fd9c015d880 0x7fd9c00b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bf59e640 1 -- 172.21.15.31:0/3839467320 >> 172.21.15.31:0/3839467320 conn(0x7fd9c006b9f0 msgr2=0x7fd9c006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bf59e640 1 -- 172.21.15.31:0/3839467320 shutdown_connections 2023-11-26T07:20:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.434+0000 7fd9bf59e640 1 -- 172.21.15.31:0/3839467320 wait complete. 2023-11-26T07:20:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bf59e640 1 Processor -- start 2023-11-26T07:20:49.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bf59e640 1 -- start start 2023-11-26T07:20:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bf59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00b6c70 0x7fd9c00b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bf59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00d3e70 0x7fd9c00cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bf59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c015d880 0x7fd9c00cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bf59e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0157f00 con 0x7fd9c00d3e70 2023-11-26T07:20:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bf59e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00ca4a0 con 0x7fd9c015d880 2023-11-26T07:20:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bf59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00ca780 con 0x7fd9c00b6c70 2023-11-26T07:20:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00d3e70 0x7fd9c00cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00d3e70 0x7fd9c00cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35630/0 (socket says 172.21.15.31:35630) 2023-11-26T07:20:49.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b7fff640 1 -- 172.21.15.31:0/2403216265 learned_addr learned my addr 172.21.15.31:0/2403216265 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:49.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bed9d640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c015d880 0x7fd9c00cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9be59c640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00b6c70 0x7fd9c00b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b7fff640 1 -- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00b6c70 msgr2=0x7fd9c00b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b7fff640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00b6c70 0x7fd9c00b83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b7fff640 1 -- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c015d880 msgr2=0x7fd9c00cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b7fff640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c015d880 0x7fd9c00cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b7fff640 1 -- 172.21.15.31:0/2403216265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b8009a70 con 0x7fd9c00d3e70 2023-11-26T07:20:49.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b7fff640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00d3e70 0x7fd9c00cc290 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80011f0 tx=0x7fd9a800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:49.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b6ffd640 1 -- 172.21.15.31:0/2403216265 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80074e0 con 0x7fd9c00d3e70 2023-11-26T07:20:49.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b6ffd640 1 -- 172.21.15.31:0/2403216265 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a8007640 con 0x7fd9c00d3e70 2023-11-26T07:20:49.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bf59e640 1 -- 172.21.15.31:0/2403216265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c00caa60 con 0x7fd9c00d3e70 2023-11-26T07:20:49.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9b6ffd640 1 -- 172.21.15.31:0/2403216265 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8004a10 con 0x7fd9c00d3e70 2023-11-26T07:20:49.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.438+0000 7fd9bf59e640 1 -- 172.21.15.31:0/2403216265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c00b8920 con 0x7fd9c00d3e70 2023-11-26T07:20:49.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.442+0000 7fd9bf59e640 1 -- 172.21.15.31:0/2403216265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c0074a00 con 0x7fd9c00d3e70 2023-11-26T07:20:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.442+0000 7fd9b6ffd640 1 -- 172.21.15.31:0/2403216265 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd9a8004b70 con 0x7fd9c00d3e70 2023-11-26T07:20:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.442+0000 7fd9b6ffd640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd9a0066650 0x7fd9a0068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.442+0000 7fd9b6ffd640 1 -- 172.21.15.31:0/2403216265 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd9a8083380 con 0x7fd9c00d3e70 2023-11-26T07:20:49.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.442+0000 7fd9be59c640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd9a0066650 0x7fd9a0068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.442+0000 7fd9be59c640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd9a0066650 0x7fd9a0068b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009a50 tx=0x7fd9b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:49.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.446+0000 7fd9b6ffd640 1 -- 172.21.15.31:0/2403216265 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd9a8050540 con 0x7fd9c00d3e70 2023-11-26T07:20:49.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.638+0000 7fd9bf59e640 1 -- 172.21.15.31:0/2403216265 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9c00cdbc0 con 0x7fd9c00d3e70 2023-11-26T07:20:49.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.638+0000 7fd9b6ffd640 1 -- 172.21.15.31:0/2403216265 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd9c00cdbc0 con 0x7fd9c00d3e70 2023-11-26T07:20:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:49.642 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:49.642 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:49.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.638+0000 7fd9bf59e640 1 -- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd9a0066650 msgr2=0x7fd9a0068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.638+0000 7fd9bf59e640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd9a0066650 0x7fd9a0068b10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009a50 tx=0x7fd9b0008040 comp rx=0 tx=0).stop 2023-11-26T07:20:49.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.638+0000 7fd9bf59e640 1 -- 172.21.15.31:0/2403216265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00d3e70 msgr2=0x7fd9c00cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.638+0000 7fd9bf59e640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00d3e70 0x7fd9c00cc290 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80011f0 tx=0x7fd9a800c9b0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.642+0000 7fd9bf59e640 1 -- 172.21.15.31:0/2403216265 shutdown_connections 2023-11-26T07:20:49.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.642+0000 7fd9bf59e640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd9a0066650 0x7fd9a0068b10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.642+0000 7fd9bf59e640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c015d880 0x7fd9c00cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.642+0000 7fd9bf59e640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9c00d3e70 0x7fd9c00cc290 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.642+0000 7fd9bf59e640 1 --2- 172.21.15.31:0/2403216265 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9c00b6c70 0x7fd9c00b83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.642+0000 7fd9bf59e640 1 -- 172.21.15.31:0/2403216265 >> 172.21.15.31:0/2403216265 conn(0x7fd9c006b9f0 msgr2=0x7fd9c015dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:49.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.642+0000 7fd9bf59e640 1 -- 172.21.15.31:0/2403216265 shutdown_connections 2023-11-26T07:20:49.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.642+0000 7fd9bf59e640 1 -- 172.21.15.31:0/2403216265 wait complete. 2023-11-26T07:20:49.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:49.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.802+0000 7f819ea68640 1 Processor -- start 2023-11-26T07:20:49.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.802+0000 7f819ea68640 1 -- start start 2023-11-26T07:20:49.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.802+0000 7f819ea68640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8198146c60 0x7f8198070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.802+0000 7f819ea68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8198147040 0x7f8198070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.802+0000 7f819ea68640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819815dba0 con 0x7f819806bb10 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.802+0000 7f819ea68640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819806d100 con 0x7f8198146c60 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.802+0000 7f819ea68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819806c660 con 0x7f8198147040 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.802+0000 7f819c804640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8198146c60 0x7f8198070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.802+0000 7f819c804640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8198146c60 0x7f8198070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35646/0 (socket says 172.21.15.31:35646) 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.802+0000 7f819c804640 1 -- 172.21.15.31:0/1731030862 learned_addr learned my addr 172.21.15.31:0/1731030862 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f8197fff640 1 --2- 172.21.15.31:0/1731030862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8198147040 0x7f8198070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f819c804640 1 -- 172.21.15.31:0/1731030862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8198147040 msgr2=0x7f8198070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f819c804640 1 --2- 172.21.15.31:0/1731030862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8198147040 0x7f8198070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f819c804640 1 -- 172.21.15.31:0/1731030862 >> v1:172.21.15.97:6789/0 conn(0x7f819806bb10 legacy=0x7f819806bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f819c804640 1 -- 172.21.15.31:0/1731030862 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819806c860 con 0x7f8198146c60 2023-11-26T07:20:49.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f8197fff640 1 --2- 172.21.15.31:0/1731030862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8198147040 0x7f8198070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:49.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f819c804640 1 --2- 172.21.15.31:0/1731030862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198146c60 0x7f8198070020 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f818c009870 tx=0x7f818c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f4bdfa0bf89f12 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:49.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f81977fe640 1 -- 172.21.15.31:0/1731030862 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c0aa670 con 0x7f8198146c60 2023-11-26T07:20:49.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f81977fe640 1 -- 172.21.15.31:0/1731030862 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818c0aa7d0 con 0x7f8198146c60 2023-11-26T07:20:49.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f81977fe640 1 -- 172.21.15.31:0/1731030862 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c0aab60 con 0x7f8198146c60 2023-11-26T07:20:49.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f819ea68640 1 -- 172.21.15.31:0/1731030862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198146c60 msgr2=0x7f8198070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.806+0000 7f819ea68640 1 --2- 172.21.15.31:0/1731030862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198146c60 0x7f8198070020 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f818c009870 tx=0x7f818c01c510 comp rx=0 tx=0).stop 2023-11-26T07:20:49.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 -- 172.21.15.31:0/1731030862 shutdown_connections 2023-11-26T07:20:49.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 --2- 172.21.15.31:0/1731030862 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8198147040 0x7f8198070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 --2- 172.21.15.31:0/1731030862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198146c60 0x7f8198070020 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 -- 172.21.15.31:0/1731030862 >> 172.21.15.31:0/1731030862 conn(0x7f81980b2ea0 msgr2=0x7f81980b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 -- 172.21.15.31:0/1731030862 shutdown_connections 2023-11-26T07:20:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 -- 172.21.15.31:0/1731030862 wait complete. 2023-11-26T07:20:49.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 Processor -- start 2023-11-26T07:20:49.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 -- start start 2023-11-26T07:20:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f819806bb10 0x7f819813afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8198146c60 0x7f819813c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198147040 0x7f8198157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819813e530 con 0x7f8198147040 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819813e810 con 0x7f8198146c60 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819ea68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819813eaf0 con 0x7f819806bb10 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819d005640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198147040 0x7f8198157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819d005640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198147040 0x7f8198157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35662/0 (socket says 172.21.15.31:35662) 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819d005640 1 -- 172.21.15.31:0/1883292197 learned_addr learned my addr 172.21.15.31:0/1883292197 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819c804640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f819806bb10 0x7f819813afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f8197fff640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8198146c60 0x7f819813c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819d005640 1 -- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f819806bb10 msgr2=0x7f819813afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819d005640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f819806bb10 0x7f819813afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819d005640 1 -- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8198146c60 msgr2=0x7f819813c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:49.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819d005640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8198146c60 0x7f819813c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:49.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f819d005640 1 -- 172.21.15.31:0/1883292197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0094d0 con 0x7f8198147040 2023-11-26T07:20:49.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.810+0000 7f8197fff640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8198146c60 0x7f819813c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:49.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f819d005640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198147040 0x7f8198157ec0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f819000c7a0 tx=0x7f819000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:49.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f81957fa640 1 -- 172.21.15.31:0/1883292197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190003ba0 con 0x7f8198147040 2023-11-26T07:20:49.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f81957fa640 1 -- 172.21.15.31:0/1883292197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8190003d00 con 0x7f8198147040 2023-11-26T07:20:49.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f81957fa640 1 -- 172.21.15.31:0/1883292197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819000aa30 con 0x7f8198147040 2023-11-26T07:20:49.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f819ea68640 1 -- 172.21.15.31:0/1883292197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819813edd0 con 0x7f8198147040 2023-11-26T07:20:49.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f819ea68640 1 -- 172.21.15.31:0/1883292197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819813b520 con 0x7f8198147040 2023-11-26T07:20:49.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f819ea68640 1 -- 172.21.15.31:0/1883292197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819804e850 con 0x7f8198147040 2023-11-26T07:20:49.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f81957fa640 1 -- 172.21.15.31:0/1883292197 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f819000ab90 con 0x7f8198147040 2023-11-26T07:20:49.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f81957fa640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f817c066620 0x7f817c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:49.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f81957fa640 1 -- 172.21.15.31:0/1883292197 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8190083f70 con 0x7f8198147040 2023-11-26T07:20:49.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.814+0000 7f819c804640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f817c066620 0x7f817c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:49.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.818+0000 7f819c804640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f817c066620 0x7f817c068ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f818c0ae000 tx=0x7f818c0b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:49.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:49.818+0000 7f81957fa640 1 -- 172.21.15.31:0/1883292197 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8190051120 con 0x7f8198147040 2023-11-26T07:20:50.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.006+0000 7f819ea68640 1 -- 172.21.15.31:0/1883292197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f819813f280 con 0x7f8198147040 2023-11-26T07:20:50.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.006+0000 7f81957fa640 1 -- 172.21.15.31:0/1883292197 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f8190054740 con 0x7f8198147040 2023-11-26T07:20:50.010 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:50.010 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:19:01.563598+0000","modified":"2023-11-26T07:19:04.566715+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5622},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5622":{"gid":5622,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1115502954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1115502954},{"type":"v1","addr":"172.21.15.31:6835","nonce":1115502954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-26T07:20:50.010 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2023-11-26T07:20:50.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 -- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f817c066620 msgr2=0x7f817c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f817c066620 0x7f817c068ae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f818c0ae000 tx=0x7f818c0b5040 comp rx=0 tx=0).stop 2023-11-26T07:20:50.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 -- 172.21.15.31:0/1883292197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198147040 msgr2=0x7f8198157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198147040 0x7f8198157ec0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f819000c7a0 tx=0x7f819000cc60 comp rx=0 tx=0).stop 2023-11-26T07:20:50.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 -- 172.21.15.31:0/1883292197 shutdown_connections 2023-11-26T07:20:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f817c066620 0x7f817c068ae0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8198147040 0x7f8198157ec0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8198146c60 0x7f819813c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 --2- 172.21.15.31:0/1883292197 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f819806bb10 0x7f819813afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 -- 172.21.15.31:0/1883292197 >> 172.21.15.31:0/1883292197 conn(0x7f81980b2ea0 msgr2=0x7f8198143b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 -- 172.21.15.31:0/1883292197 shutdown_connections 2023-11-26T07:20:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.010+0000 7f819ea68640 1 -- 172.21.15.31:0/1883292197 wait complete. 2023-11-26T07:20:50.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:20:50.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.170+0000 7f0b1efa1640 1 Processor -- start 2023-11-26T07:20:50.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.170+0000 7f0b1efa1640 1 -- start start 2023-11-26T07:20:50.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b1efa1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0b180d2080 0x7f0b18168160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b1efa1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b180cefd0 con 0x7f0b180cead0 2023-11-26T07:20:50.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b1efa1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b180cf2b0 con 0x7f0b1815d7b0 2023-11-26T07:20:50.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b1efa1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b180d17e0 con 0x7f0b180d2080 2023-11-26T07:20:50.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b17fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0b180cead0 0x7f0b180d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49546/0 (socket says 172.21.15.31:49546) 2023-11-26T07:20:50.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b17fff640 1 -- 172.21.15.31:0/1910330821 learned_addr learned my addr 172.21.15.31:0/1910330821 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:50.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b1d53e640 1 --2- 172.21.15.31:0/1910330821 >> v2:172.21.15.31:3300/0 conn(0x7f0b180d2080 0x7f0b18168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b177fe640 1 -- 172.21.15.31:0/1910330821 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165037569 0 0) 0x7f0b180cefd0 con 0x7f0b180cead0 2023-11-26T07:20:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b177fe640 1 -- 172.21.15.31:0/1910330821 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b08003530 con 0x7f0b180cead0 2023-11-26T07:20:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b177fe640 1 -- 172.21.15.31:0/1910330821 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799498687 0 0) 0x7f0b180cf2b0 con 0x7f0b1815d7b0 2023-11-26T07:20:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b177fe640 1 -- 172.21.15.31:0/1910330821 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b180cefd0 con 0x7f0b1815d7b0 2023-11-26T07:20:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b1d53e640 1 -- 172.21.15.31:0/1910330821 >> v1:172.21.15.97:6789/0 conn(0x7f0b1815d7b0 legacy=0x7f0b180d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b1d53e640 1 -- 172.21.15.31:0/1910330821 >> v1:172.21.15.31:6789/0 conn(0x7f0b180cead0 legacy=0x7f0b180d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.174+0000 7f0b1d53e640 1 -- 172.21.15.31:0/1910330821 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1806b3f0 con 0x7f0b180d2080 2023-11-26T07:20:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b1d53e640 1 --2- 172.21.15.31:0/1910330821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180d2080 0x7f0b18168160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009d90 tx=0x7f0b1001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26c0fa8bf437a98 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b177fe640 1 -- 172.21.15.31:0/1910330821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100a58c0 con 0x7f0b180d2080 2023-11-26T07:20:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b177fe640 1 -- 172.21.15.31:0/1910330821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100a5a20 con 0x7f0b180d2080 2023-11-26T07:20:50.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b177fe640 1 -- 172.21.15.31:0/1910330821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100a5db0 con 0x7f0b180d2080 2023-11-26T07:20:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b1efa1640 1 -- 172.21.15.31:0/1910330821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180d2080 msgr2=0x7f0b18168160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b1efa1640 1 --2- 172.21.15.31:0/1910330821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180d2080 0x7f0b18168160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009d90 tx=0x7f0b1001c800 comp rx=0 tx=0).stop 2023-11-26T07:20:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b1efa1640 1 -- 172.21.15.31:0/1910330821 shutdown_connections 2023-11-26T07:20:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b1efa1640 1 --2- 172.21.15.31:0/1910330821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180d2080 0x7f0b18168160 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b1efa1640 1 -- 172.21.15.31:0/1910330821 >> 172.21.15.31:0/1910330821 conn(0x7f0b180d33b0 msgr2=0x7f0b180d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:50.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b1efa1640 1 -- 172.21.15.31:0/1910330821 shutdown_connections 2023-11-26T07:20:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.178+0000 7f0b1efa1640 1 -- 172.21.15.31:0/1910330821 wait complete. 2023-11-26T07:20:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1efa1640 1 Processor -- start 2023-11-26T07:20:50.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1efa1640 1 -- start start 2023-11-26T07:20:50.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1efa1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180cead0 0x7f0b180cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1efa1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b180d2080 0x7f0b180c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1efa1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b1815d7b0 0x7f0b180c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1efa1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180b86c0 con 0x7f0b180cead0 2023-11-26T07:20:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1efa1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180b8d80 con 0x7f0b1815d7b0 2023-11-26T07:20:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1efa1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180b7a40 con 0x7f0b180d2080 2023-11-26T07:20:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1cd3d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180cead0 0x7f0b180cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1cd3d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180cead0 0x7f0b180cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35686/0 (socket says 172.21.15.31:35686) 2023-11-26T07:20:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1cd3d640 1 -- 172.21.15.31:0/3411814266 learned_addr learned my addr 172.21.15.31:0/3411814266 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b17fff640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b180d2080 0x7f0b180c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1d53e640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b1815d7b0 0x7f0b180c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1cd3d640 1 -- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b180d2080 msgr2=0x7f0b180c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1cd3d640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b180d2080 0x7f0b180c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1cd3d640 1 -- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b1815d7b0 msgr2=0x7f0b180c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1cd3d640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b1815d7b0 0x7f0b180c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1cd3d640 1 -- 172.21.15.31:0/3411814266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10009a70 con 0x7f0b180cead0 2023-11-26T07:20:50.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1cd3d640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180cead0 0x7f0b180cc330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00c490 tx=0x7f0b0c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b157fa640 1 -- 172.21.15.31:0/3411814266 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c0052d0 con 0x7f0b180cead0 2023-11-26T07:20:50.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b157fa640 1 -- 172.21.15.31:0/3411814266 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c005430 con 0x7f0b180cead0 2023-11-26T07:20:50.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b157fa640 1 -- 172.21.15.31:0/3411814266 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c012870 con 0x7f0b180cead0 2023-11-26T07:20:50.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1efa1640 1 -- 172.21.15.31:0/3411814266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b180b7d20 con 0x7f0b180cead0 2023-11-26T07:20:50.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.182+0000 7f0b1efa1640 1 -- 172.21.15.31:0/3411814266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b180b82e0 con 0x7f0b180cead0 2023-11-26T07:20:50.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.186+0000 7f0b1efa1640 1 -- 172.21.15.31:0/3411814266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1804e850 con 0x7f0b180cead0 2023-11-26T07:20:50.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.186+0000 7f0b157fa640 1 -- 172.21.15.31:0/3411814266 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f0b0c0129d0 con 0x7f0b180cead0 2023-11-26T07:20:50.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.186+0000 7f0b157fa640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0afc066620 0x7f0afc068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.186+0000 7f0b157fa640 1 -- 172.21.15.31:0/3411814266 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0b0c083c20 con 0x7f0b180cead0 2023-11-26T07:20:50.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.186+0000 7f0b17fff640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0afc066620 0x7f0afc068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.186+0000 7f0b17fff640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0afc066620 0x7f0afc068ae0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0b18167e00 tx=0x7f0b04007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:50.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.190+0000 7f0b157fa640 1 -- 172.21.15.31:0/3411814266 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0b0c050e90 con 0x7f0b180cead0 2023-11-26T07:20:50.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 -- 172.21.15.31:0/3411814266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b1806a5b0 con 0x7f0b180cead0 2023-11-26T07:20:50.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b157fa640 1 -- 172.21.15.31:0/3411814266 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16581 (secure 0 0 0) 0x7f0b0c0544b0 con 0x7f0b180cead0 2023-11-26T07:20:50.357 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:50.357 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:19:15.232944+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:18:59.966450+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-26T07:19:00.550730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.18","mappings":[{"from":0,"to":5}]},{"pgid":"7.2d","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:20:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 -- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0afc066620 msgr2=0x7f0afc068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0afc066620 0x7f0afc068ae0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0b18167e00 tx=0x7f0b04007d40 comp rx=0 tx=0).stop 2023-11-26T07:20:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 -- 172.21.15.31:0/3411814266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180cead0 msgr2=0x7f0b180cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180cead0 0x7f0b180cc330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00c490 tx=0x7f0b0c00c950 comp rx=0 tx=0).stop 2023-11-26T07:20:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 -- 172.21.15.31:0/3411814266 shutdown_connections 2023-11-26T07:20:50.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0afc066620 0x7f0afc068ae0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b1815d7b0 0x7f0b180c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0b180d2080 0x7f0b180c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 --2- 172.21.15.31:0/3411814266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b180cead0 0x7f0b180cc330 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.354+0000 7f0b1efa1640 1 -- 172.21.15.31:0/3411814266 >> 172.21.15.31:0/3411814266 conn(0x7f0b180d33b0 msgr2=0x7f0b180d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:50.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.358+0000 7f0b1efa1640 1 -- 172.21.15.31:0/3411814266 shutdown_connections 2023-11-26T07:20:50.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.358+0000 7f0b1efa1640 1 -- 172.21.15.31:0/3411814266 wait complete. 2023-11-26T07:20:50.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:20:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f349069c640 1 Processor -- start 2023-11-26T07:20:50.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f349069c640 1 -- start start 2023-11-26T07:20:50.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f349069c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34880cdfb0 0x7f34880abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f349069c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 0x7f34880ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f349069c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34880ce8d0 con 0x7f34880cbd50 2023-11-26T07:20:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f349069c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34880cca80 con 0x7f34880cdfb0 2023-11-26T07:20:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f349069c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34880b2710 con 0x7f348815a710 2023-11-26T07:20:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f348ec39640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f34880cbd50 0x7f34880cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49550/0 (socket says 172.21.15.31:49550) 2023-11-26T07:20:50.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f348ec39640 1 -- 172.21.15.31:0/2891125118 learned_addr learned my addr 172.21.15.31:0/2891125118 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f348dc37640 1 --2- 172.21.15.31:0/2891125118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 0x7f34880ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f348e438640 1 --2- 172.21.15.31:0/2891125118 >> v2:172.21.15.97:3300/0 conn(0x7f34880cdfb0 0x7f34880abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.522+0000 7f348d436640 1 -- 172.21.15.31:0/2891125118 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433026888 0 0) 0x7f34880ce8d0 con 0x7f34880cbd50 2023-11-26T07:20:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348d436640 1 -- 172.21.15.31:0/2891125118 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3474003530 con 0x7f34880cbd50 2023-11-26T07:20:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348dc37640 1 -- 172.21.15.31:0/2891125118 >> v2:172.21.15.97:3300/0 conn(0x7f34880cdfb0 msgr2=0x7f34880abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348dc37640 1 --2- 172.21.15.31:0/2891125118 >> v2:172.21.15.97:3300/0 conn(0x7f34880cdfb0 0x7f34880abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348dc37640 1 -- 172.21.15.31:0/2891125118 >> v1:172.21.15.31:6789/0 conn(0x7f34880cbd50 legacy=0x7f34880cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348d436640 1 -- 172.21.15.31:0/2891125118 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3633223893 0 0) 0x7f3474003530 con 0x7f34880cbd50 2023-11-26T07:20:50.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348dc37640 1 -- 172.21.15.31:0/2891125118 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34880b29f0 con 0x7f348815a710 2023-11-26T07:20:50.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348e438640 1 --2- 172.21.15.31:0/2891125118 >> v2:172.21.15.97:3300/0 conn(0x7f34880cdfb0 0x7f34880abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:20:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348dc37640 1 --2- 172.21.15.31:0/2891125118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 0x7f34880ce390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f34780097c0 tx=0x7f347801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adea6f8c3cdba20d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348d436640 1 -- 172.21.15.31:0/2891125118 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3480001a00 con 0x7f348815a710 2023-11-26T07:20:50.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348d436640 1 -- 172.21.15.31:0/2891125118 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34780a7050 con 0x7f348815a710 2023-11-26T07:20:50.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.526+0000 7f348d436640 1 -- 172.21.15.31:0/2891125118 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34780ab600 con 0x7f348815a710 2023-11-26T07:20:50.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 -- 172.21.15.31:0/2891125118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 msgr2=0x7f34880ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 --2- 172.21.15.31:0/2891125118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 0x7f34880ce390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f34780097c0 tx=0x7f347801c370 comp rx=0 tx=0).stop 2023-11-26T07:20:50.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 -- 172.21.15.31:0/2891125118 shutdown_connections 2023-11-26T07:20:50.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 --2- 172.21.15.31:0/2891125118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 0x7f34880ce390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 --2- 172.21.15.31:0/2891125118 >> v2:172.21.15.97:3300/0 conn(0x7f34880cdfb0 0x7f34880abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 -- 172.21.15.31:0/2891125118 >> 172.21.15.31:0/2891125118 conn(0x7f34880b0cb0 msgr2=0x7f34880b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:50.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 -- 172.21.15.31:0/2891125118 shutdown_connections 2023-11-26T07:20:50.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 -- 172.21.15.31:0/2891125118 wait complete. 2023-11-26T07:20:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 Processor -- start 2023-11-26T07:20:50.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 -- start start 2023-11-26T07:20:50.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34880cbd50 0x7f3488157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34880cdfb0 0x7f3488163800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 0x7f34881591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3488152e50 con 0x7f34880cdfb0 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3488158150 con 0x7f34880cbd50 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f349069c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3488158430 con 0x7f348815a710 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348dc37640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34880cdfb0 0x7f3488163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348dc37640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34880cdfb0 0x7f3488163800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35694/0 (socket says 172.21.15.31:35694) 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348dc37640 1 -- 172.21.15.31:0/3825383094 learned_addr learned my addr 172.21.15.31:0/3825383094 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348e438640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34880cbd50 0x7f3488157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348ec39640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 0x7f34881591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348dc37640 1 -- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 msgr2=0x7f34881591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348dc37640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 0x7f34881591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348dc37640 1 -- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34880cbd50 msgr2=0x7f3488157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348dc37640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34880cbd50 0x7f3488157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348dc37640 1 -- 172.21.15.31:0/3825383094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3484009420 con 0x7f34880cdfb0 2023-11-26T07:20:50.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348ec39640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 0x7f34881591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:50.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.530+0000 7f348dc37640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34880cdfb0 0x7f3488163800 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3478000c10 tx=0x7f3478019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:50.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.534+0000 7f347effd640 1 -- 172.21.15.31:0/3825383094 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34780b9070 con 0x7f34880cdfb0 2023-11-26T07:20:50.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.534+0000 7f347effd640 1 -- 172.21.15.31:0/3825383094 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3478003830 con 0x7f34880cdfb0 2023-11-26T07:20:50.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.534+0000 7f349069c640 1 -- 172.21.15.31:0/3825383094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3478009420 con 0x7f34880cdfb0 2023-11-26T07:20:50.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.534+0000 7f347effd640 1 -- 172.21.15.31:0/3825383094 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3478003ba0 con 0x7f34880cdfb0 2023-11-26T07:20:50.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.534+0000 7f349069c640 1 -- 172.21.15.31:0/3825383094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34881589f0 con 0x7f34880cdfb0 2023-11-26T07:20:50.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.534+0000 7f349069c640 1 -- 172.21.15.31:0/3825383094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34880671f0 con 0x7f34880cdfb0 2023-11-26T07:20:50.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.534+0000 7f347effd640 1 -- 172.21.15.31:0/3825383094 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f34780b2040 con 0x7f34880cdfb0 2023-11-26T07:20:50.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.534+0000 7f347effd640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f346c066620 0x7f346c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.534+0000 7f347effd640 1 -- 172.21.15.31:0/3825383094 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f347811da40 con 0x7f34880cdfb0 2023-11-26T07:20:50.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.534+0000 7f348e438640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f346c066620 0x7f346c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.538+0000 7f348e438640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f346c066620 0x7f346c068ae0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3484009c80 tx=0x7f3484008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:50.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.538+0000 7f347effd640 1 -- 172.21.15.31:0/3825383094 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f34780eaa50 con 0x7f34880cdfb0 2023-11-26T07:20:50.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.734+0000 7f349069c640 1 -- 172.21.15.31:0/3825383094 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f348806b460 con 0x7f34880cdfb0 2023-11-26T07:20:50.738 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-11-26T07:20:50.734+0000 7fe86499d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:20:50.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.738+0000 7f347effd640 1 -- 172.21.15.31:0/3825383094 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7f34780b0020 con 0x7f34880cdfb0 2023-11-26T07:20:50.741 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:20:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 -- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f346c066620 msgr2=0x7f346c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f346c066620 0x7f346c068ae0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3484009c80 tx=0x7f3484008040 comp rx=0 tx=0).stop 2023-11-26T07:20:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 -- 172.21.15.31:0/3825383094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34880cdfb0 msgr2=0x7f3488163800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34880cdfb0 0x7f3488163800 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f3478000c10 tx=0x7f3478019110 comp rx=0 tx=0).stop 2023-11-26T07:20:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 -- 172.21.15.31:0/3825383094 shutdown_connections 2023-11-26T07:20:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f346c066620 0x7f346c068ae0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f348815a710 0x7f34881591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34880cdfb0 0x7f3488163800 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 --2- 172.21.15.31:0/3825383094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34880cbd50 0x7f3488157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 -- 172.21.15.31:0/3825383094 >> 172.21.15.31:0/3825383094 conn(0x7f34880b0cb0 msgr2=0x7f348815dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 -- 172.21.15.31:0/3825383094 shutdown_connections 2023-11-26T07:20:50.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.742+0000 7f349069c640 1 -- 172.21.15.31:0/3825383094 wait complete. 2023-11-26T07:20:50.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:20:50.821 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-11-26T07:20:50.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.918+0000 7f054f8e3640 1 Processor -- start 2023-11-26T07:20:50.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.918+0000 7f054f8e3640 1 -- start start 2023-11-26T07:20:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054f8e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 0x7f054815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054f8e3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05480ce5d0 0x7f054815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054f8e3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0548159630 con 0x7f05480cb9b0 2023-11-26T07:20:50.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054f8e3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054815bba0 con 0x7f05480ce5d0 2023-11-26T07:20:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054f8e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480cdd50 con 0x7f054806e3f0 2023-11-26T07:20:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054d67f640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f05480cb9b0 0x7f05480cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49566/0 (socket says 172.21.15.31:49566) 2023-11-26T07:20:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054d67f640 1 -- 172.21.15.31:0/1075591634 learned_addr learned my addr 172.21.15.31:0/1075591634 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054de80640 1 --2- 172.21.15.31:0/1075591634 >> v2:172.21.15.97:3300/0 conn(0x7f05480ce5d0 0x7f054815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054ce7e640 1 --2- 172.21.15.31:0/1075591634 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 0x7f054815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f053ffff640 1 -- 172.21.15.31:0/1075591634 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857898073 0 0) 0x7f0548159630 con 0x7f05480cb9b0 2023-11-26T07:20:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f053ffff640 1 -- 172.21.15.31:0/1075591634 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0534003530 con 0x7f05480cb9b0 2023-11-26T07:20:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f053ffff640 1 -- 172.21.15.31:0/1075591634 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1232895081 0 0) 0x7f0534003530 con 0x7f05480cb9b0 2023-11-26T07:20:50.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054de80640 1 -- 172.21.15.31:0/1075591634 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 msgr2=0x7f054815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:50.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054de80640 1 --2- 172.21.15.31:0/1075591634 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 0x7f054815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054de80640 1 -- 172.21.15.31:0/1075591634 >> v1:172.21.15.31:6789/0 conn(0x7f05480cb9b0 legacy=0x7f05480cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.922+0000 7f054de80640 1 -- 172.21.15.31:0/1075591634 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054815f6d0 con 0x7f05480ce5d0 2023-11-26T07:20:50.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f054de80640 1 --2- 172.21.15.31:0/1075591634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce5d0 0x7f054815bfd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0540009d90 tx=0x7f054001c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=626e7749fafb069a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:50.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f053ffff640 1 -- 172.21.15.31:0/1075591634 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544001680 con 0x7f05480ce5d0 2023-11-26T07:20:50.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f053ffff640 1 -- 172.21.15.31:0/1075591634 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05400a5540 con 0x7f05480ce5d0 2023-11-26T07:20:50.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f053ffff640 1 -- 172.21.15.31:0/1075591634 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400a58d0 con 0x7f05480ce5d0 2023-11-26T07:20:50.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f054f8e3640 1 -- 172.21.15.31:0/1075591634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce5d0 msgr2=0x7f054815bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f054f8e3640 1 --2- 172.21.15.31:0/1075591634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce5d0 0x7f054815bfd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0540009d90 tx=0x7f054001c870 comp rx=0 tx=0).stop 2023-11-26T07:20:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f054f8e3640 1 -- 172.21.15.31:0/1075591634 shutdown_connections 2023-11-26T07:20:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f054f8e3640 1 --2- 172.21.15.31:0/1075591634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce5d0 0x7f054815bfd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f054f8e3640 1 --2- 172.21.15.31:0/1075591634 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 0x7f054815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f054f8e3640 1 -- 172.21.15.31:0/1075591634 >> 172.21.15.31:0/1075591634 conn(0x7f05480b0cb0 msgr2=0x7f05480b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f054f8e3640 1 -- 172.21.15.31:0/1075591634 shutdown_connections 2023-11-26T07:20:50.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f054f8e3640 1 -- 172.21.15.31:0/1075591634 wait complete. 2023-11-26T07:20:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.926+0000 7f054f8e3640 1 Processor -- start 2023-11-26T07:20:50.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054f8e3640 1 -- start start 2023-11-26T07:20:50.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054f8e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 0x7f05480aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054f8e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05480cb9b0 0x7f054813e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054f8e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce5d0 0x7f05480b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054f8e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054815aef0 con 0x7f05480cb9b0 2023-11-26T07:20:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054f8e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054815b1d0 con 0x7f05480ce5d0 2023-11-26T07:20:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054f8e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054815b4b0 con 0x7f054806e3f0 2023-11-26T07:20:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054ce7e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05480cb9b0 0x7f054813e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054ce7e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05480cb9b0 0x7f054813e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35710/0 (socket says 172.21.15.31:35710) 2023-11-26T07:20:50.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054ce7e640 1 -- 172.21.15.31:0/769667495 learned_addr learned my addr 172.21.15.31:0/769667495 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054d67f640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 0x7f05480aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054ce7e640 1 -- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 msgr2=0x7f05480aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054ce7e640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 0x7f05480aedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054ce7e640 1 -- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce5d0 msgr2=0x7f05480b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054de80640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce5d0 0x7f05480b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054ce7e640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce5d0 0x7f05480b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054ce7e640 1 -- 172.21.15.31:0/769667495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540009a70 con 0x7f05480cb9b0 2023-11-26T07:20:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054d67f640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 0x7f05480aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:50.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054de80640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce5d0 0x7f05480b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054ce7e640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05480cb9b0 0x7f054813e0c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f053800e930 tx=0x7f053800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f053dffb640 1 -- 172.21.15.31:0/769667495 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538009690 con 0x7f05480cb9b0 2023-11-26T07:20:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f053dffb640 1 -- 172.21.15.31:0/769667495 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053800ca00 con 0x7f05480cb9b0 2023-11-26T07:20:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054f8e3640 1 -- 172.21.15.31:0/769667495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0548157ed0 con 0x7f05480cb9b0 2023-11-26T07:20:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f053dffb640 1 -- 172.21.15.31:0/769667495 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053800cd70 con 0x7f05480cb9b0 2023-11-26T07:20:50.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054f8e3640 1 -- 172.21.15.31:0/769667495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054813e600 con 0x7f05480cb9b0 2023-11-26T07:20:50.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.930+0000 7f054f8e3640 1 -- 172.21.15.31:0/769667495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05480671f0 con 0x7f05480cb9b0 2023-11-26T07:20:50.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.934+0000 7f053dffb640 1 -- 172.21.15.31:0/769667495 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f0538013470 con 0x7f05480cb9b0 2023-11-26T07:20:50.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.934+0000 7f053dffb640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f052c066620 0x7f052c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:50.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.934+0000 7f053dffb640 1 -- 172.21.15.31:0/769667495 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0538084590 con 0x7f05480cb9b0 2023-11-26T07:20:50.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.934+0000 7f054d67f640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f052c066620 0x7f052c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:50.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.934+0000 7f054d67f640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f052c066620 0x7f052c068ae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f05440011f0 tx=0x7f0544009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:50.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:50.938+0000 7f053dffb640 1 -- 172.21.15.31:0/769667495 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0538051720 con 0x7f05480cb9b0 2023-11-26T07:20:51.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.130+0000 7f054f8e3640 1 -- 172.21.15.31:0/769667495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0548158060 con 0x7f05480cb9b0 2023-11-26T07:20:51.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.742+0000 7f053dffb640 1 -- 172.21.15.31:0/769667495 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0538054d40 con 0x7f05480cb9b0 2023-11-26T07:20:51.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.742+0000 7f054f8e3640 1 -- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f052c066620 msgr2=0x7f052c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:51.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.742+0000 7f054f8e3640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f052c066620 0x7f052c068ae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f05440011f0 tx=0x7f0544009040 comp rx=0 tx=0).stop 2023-11-26T07:20:51.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.746+0000 7f054f8e3640 1 -- 172.21.15.31:0/769667495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05480cb9b0 msgr2=0x7f054813e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:51.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.746+0000 7f054f8e3640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05480cb9b0 0x7f054813e0c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f053800e930 tx=0x7f053800edf0 comp rx=0 tx=0).stop 2023-11-26T07:20:51.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.746+0000 7f054f8e3640 1 -- 172.21.15.31:0/769667495 shutdown_connections 2023-11-26T07:20:51.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.746+0000 7f054f8e3640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f052c066620 0x7f052c068ae0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:51.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.746+0000 7f054f8e3640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05480ce5d0 0x7f05480b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:51.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.746+0000 7f054f8e3640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05480cb9b0 0x7f054813e0c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:51.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.746+0000 7f054f8e3640 1 --2- 172.21.15.31:0/769667495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f054806e3f0 0x7f05480aedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:51.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.746+0000 7f054f8e3640 1 -- 172.21.15.31:0/769667495 >> 172.21.15.31:0/769667495 conn(0x7f05480b0cb0 msgr2=0x7f05480cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:51.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.746+0000 7f054f8e3640 1 -- 172.21.15.31:0/769667495 shutdown_connections 2023-11-26T07:20:51.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.746+0000 7f054f8e3640 1 -- 172.21.15.31:0/769667495 wait complete. 2023-11-26T07:20:51.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:20:51.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.958+0000 7f4326c6b640 1 Processor -- start 2023-11-26T07:20:51.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.958+0000 7f4326c6b640 1 -- start start 2023-11-26T07:20:51.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.958+0000 7f4326c6b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b07a0 0x7f43201619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:51.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.958+0000 7f4326c6b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4320161290 con 0x7f4320161f20 2023-11-26T07:20:51.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.958+0000 7f4326c6b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4320161550 con 0x7f43200b15f0 2023-11-26T07:20:51.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.958+0000 7f4326c6b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200b0b80 con 0x7f43200b07a0 2023-11-26T07:20:51.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.958+0000 7f431ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4320161f20 0x7f43200b0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49576/0 (socket says 172.21.15.31:49576) 2023-11-26T07:20:51.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.958+0000 7f431ffff640 1 -- 172.21.15.31:0/1999871383 learned_addr learned my addr 172.21.15.31:0/1999871383 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:51.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.958+0000 7f4324a07640 1 --2- 172.21.15.31:0/1999871383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b07a0 0x7f43201619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f431f7fe640 1 -- 172.21.15.31:0/1999871383 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709498653 0 0) 0x7f4320161290 con 0x7f4320161f20 2023-11-26T07:20:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f431f7fe640 1 -- 172.21.15.31:0/1999871383 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4310003530 con 0x7f4320161f20 2023-11-26T07:20:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f431f7fe640 1 -- 172.21.15.31:0/1999871383 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060789083 0 0) 0x7f4320161550 con 0x7f43200b15f0 2023-11-26T07:20:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f431f7fe640 1 -- 172.21.15.31:0/1999871383 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4320161290 con 0x7f43200b15f0 2023-11-26T07:20:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f4324a07640 1 -- 172.21.15.31:0/1999871383 >> v1:172.21.15.97:6789/0 conn(0x7f43200b15f0 legacy=0x7f43200cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f4324a07640 1 -- 172.21.15.31:0/1999871383 >> v1:172.21.15.31:6789/0 conn(0x7f4320161f20 legacy=0x7f43200b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:51.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f4324a07640 1 -- 172.21.15.31:0/1999871383 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43200c65a0 con 0x7f43200b07a0 2023-11-26T07:20:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f4324a07640 1 --2- 172.21.15.31:0/1999871383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b07a0 0x7f43201619e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43140097f0 tx=0x7f431401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b87df071c6de49 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f431f7fe640 1 -- 172.21.15.31:0/1999871383 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140ac410 con 0x7f43200b07a0 2023-11-26T07:20:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f431f7fe640 1 -- 172.21.15.31:0/1999871383 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43140ac570 con 0x7f43200b07a0 2023-11-26T07:20:51.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f431f7fe640 1 -- 172.21.15.31:0/1999871383 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140ac900 con 0x7f43200b07a0 2023-11-26T07:20:51.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f4326c6b640 1 -- 172.21.15.31:0/1999871383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b07a0 msgr2=0x7f43201619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:51.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.962+0000 7f4326c6b640 1 --2- 172.21.15.31:0/1999871383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b07a0 0x7f43201619e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43140097f0 tx=0x7f431401c4c0 comp rx=0 tx=0).stop 2023-11-26T07:20:51.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 -- 172.21.15.31:0/1999871383 shutdown_connections 2023-11-26T07:20:51.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 --2- 172.21.15.31:0/1999871383 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b07a0 0x7f43201619e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 -- 172.21.15.31:0/1999871383 >> 172.21.15.31:0/1999871383 conn(0x7f43200b1d50 msgr2=0x7f43200b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 -- 172.21.15.31:0/1999871383 shutdown_connections 2023-11-26T07:20:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 -- 172.21.15.31:0/1999871383 wait complete. 2023-11-26T07:20:51.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 Processor -- start 2023-11-26T07:20:51.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 -- start start 2023-11-26T07:20:51.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43200b07a0 0x7f4320156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:51.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b15f0 0x7f432015f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:51.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4320161f20 0x7f432015fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:51.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43201602c0 con 0x7f43200b07a0 2023-11-26T07:20:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43201605a0 con 0x7f4320161f20 2023-11-26T07:20:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4326c6b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4320160880 con 0x7f43200b15f0 2023-11-26T07:20:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4324a07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43200b07a0 0x7f4320156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4324a07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43200b07a0 0x7f4320156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35722/0 (socket says 172.21.15.31:35722) 2023-11-26T07:20:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4324a07640 1 -- 172.21.15.31:0/309857151 learned_addr learned my addr 172.21.15.31:0/309857151 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f431ffff640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b15f0 0x7f432015f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4325208640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4320161f20 0x7f432015fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4324a07640 1 -- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b15f0 msgr2=0x7f432015f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:51.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4324a07640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b15f0 0x7f432015f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:51.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4324a07640 1 -- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4320161f20 msgr2=0x7f432015fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:51.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4324a07640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4320161f20 0x7f432015fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:51.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.966+0000 7f4324a07640 1 -- 172.21.15.31:0/309857151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4314009420 con 0x7f43200b07a0 2023-11-26T07:20:51.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f4324a07640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43200b07a0 0x7f4320156ec0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f43140097f0 tx=0x7f4314019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:51.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f431d7fa640 1 -- 172.21.15.31:0/309857151 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314006e20 con 0x7f43200b07a0 2023-11-26T07:20:51.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f431d7fa640 1 -- 172.21.15.31:0/309857151 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43140ace20 con 0x7f43200b07a0 2023-11-26T07:20:51.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f4326c6b640 1 -- 172.21.15.31:0/309857151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4320160b60 con 0x7f43200b07a0 2023-11-26T07:20:51.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f431d7fa640 1 -- 172.21.15.31:0/309857151 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43140b5860 con 0x7f43200b07a0 2023-11-26T07:20:51.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f4326c6b640 1 -- 172.21.15.31:0/309857151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432015ff80 con 0x7f43200b07a0 2023-11-26T07:20:51.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f4326c6b640 1 -- 172.21.15.31:0/309857151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43200ad5f0 con 0x7f43200b07a0 2023-11-26T07:20:51.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f431d7fa640 1 -- 172.21.15.31:0/309857151 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f43140b3040 con 0x7f43200b07a0 2023-11-26T07:20:51.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f431d7fa640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4304066650 0x7f4304068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f431d7fa640 1 -- 172.21.15.31:0/309857151 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f431411db40 con 0x7f43200b07a0 2023-11-26T07:20:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.970+0000 7f431ffff640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4304066650 0x7f4304068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:51.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.974+0000 7f431ffff640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4304066650 0x7f4304068b10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f43200afdd0 tx=0x7f430c007d80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:51.974+0000 7f431d7fa640 1 -- 172.21.15.31:0/309857151 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f43200ad5f0 con 0x7f43200b07a0 2023-11-26T07:20:52.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:52.138+0000 7f4326c6b640 1 -- 172.21.15.31:0/309857151 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f432004e850 con 0x7f43200b07a0 2023-11-26T07:20:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.038+0000 7f431d7fa640 1 -- 172.21.15.31:0/309857151 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) v1 ==== 181+0+0 (secure 0 0 0) 0x7f43140ee5c0 con 0x7f43200b07a0 2023-11-26T07:20:53.041 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:20:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 -- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4304066650 msgr2=0x7f4304068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4304066650 0x7f4304068b10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f43200afdd0 tx=0x7f430c007d80 comp rx=0 tx=0).stop 2023-11-26T07:20:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 -- 172.21.15.31:0/309857151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43200b07a0 msgr2=0x7f4320156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43200b07a0 0x7f4320156ec0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f43140097f0 tx=0x7f4314019260 comp rx=0 tx=0).stop 2023-11-26T07:20:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 -- 172.21.15.31:0/309857151 shutdown_connections 2023-11-26T07:20:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4304066650 0x7f4304068b10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4320161f20 0x7f432015fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43200b15f0 0x7f432015f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 --2- 172.21.15.31:0/309857151 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43200b07a0 0x7f4320156ec0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 -- 172.21.15.31:0/309857151 >> 172.21.15.31:0/309857151 conn(0x7f43200b1d50 msgr2=0x7f4320168850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 -- 172.21.15.31:0/309857151 shutdown_connections 2023-11-26T07:20:53.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.042+0000 7f4326c6b640 1 -- 172.21.15.31:0/309857151 wait complete. 2023-11-26T07:20:53.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:20:53.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.226+0000 7f6f5a26b640 1 Processor -- start 2023-11-26T07:20:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.226+0000 7f6f5a26b640 1 -- start start 2023-11-26T07:20:53.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.226+0000 7f6f5a26b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6f540cc460 0x7f6f540abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.226+0000 7f6f5a26b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f540b2710 con 0x7f6f5415a710 2023-11-26T07:20:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.226+0000 7f6f5a26b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f540b29f0 con 0x7f6f540cdfb0 2023-11-26T07:20:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.226+0000 7f6f5a26b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f540cca80 con 0x7f6f540cc460 2023-11-26T07:20:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.226+0000 7f6f537fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6f5415a710 0x7f6f540cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49588/0 (socket says 172.21.15.31:49588) 2023-11-26T07:20:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.226+0000 7f6f537fe640 1 -- 172.21.15.31:0/115726650 learned_addr learned my addr 172.21.15.31:0/115726650 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:53.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f58808640 1 --2- 172.21.15.31:0/115726650 >> v2:172.21.15.97:3300/0 conn(0x7f6f540cc460 0x7f6f540abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f52ffd640 1 -- 172.21.15.31:0/115726650 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969870999 0 0) 0x7f6f540b2710 con 0x7f6f5415a710 2023-11-26T07:20:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f52ffd640 1 -- 172.21.15.31:0/115726650 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f3c003530 con 0x7f6f5415a710 2023-11-26T07:20:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f52ffd640 1 -- 172.21.15.31:0/115726650 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910547026 0 0) 0x7f6f540b29f0 con 0x7f6f540cdfb0 2023-11-26T07:20:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f52ffd640 1 -- 172.21.15.31:0/115726650 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f540b2710 con 0x7f6f540cdfb0 2023-11-26T07:20:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f52ffd640 1 -- 172.21.15.31:0/115726650 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 418765586 0 0) 0x7f6f3c003530 con 0x7f6f5415a710 2023-11-26T07:20:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f58808640 1 -- 172.21.15.31:0/115726650 >> v1:172.21.15.97:6789/0 conn(0x7f6f540cdfb0 legacy=0x7f6f540ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f58808640 1 -- 172.21.15.31:0/115726650 >> v1:172.21.15.31:6789/0 conn(0x7f6f5415a710 legacy=0x7f6f540cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:53.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f58808640 1 -- 172.21.15.31:0/115726650 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540b2cd0 con 0x7f6f540cc460 2023-11-26T07:20:53.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f58808640 1 --2- 172.21.15.31:0/115726650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f540cc460 0x7f6f540abf90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009d90 tx=0x7f6f4c01c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bb493db69dc6e05 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:53.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f52ffd640 1 -- 172.21.15.31:0/115726650 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48001680 con 0x7f6f540cc460 2023-11-26T07:20:53.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f52ffd640 1 -- 172.21.15.31:0/115726650 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4c0a56a0 con 0x7f6f540cc460 2023-11-26T07:20:53.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.230+0000 7f6f52ffd640 1 -- 172.21.15.31:0/115726650 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c0a5a30 con 0x7f6f540cc460 2023-11-26T07:20:53.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 -- 172.21.15.31:0/115726650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f540cc460 msgr2=0x7f6f540abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:53.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 --2- 172.21.15.31:0/115726650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f540cc460 0x7f6f540abf90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009d90 tx=0x7f6f4c01c840 comp rx=0 tx=0).stop 2023-11-26T07:20:53.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 -- 172.21.15.31:0/115726650 shutdown_connections 2023-11-26T07:20:53.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 --2- 172.21.15.31:0/115726650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f540cc460 0x7f6f540abf90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:53.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 -- 172.21.15.31:0/115726650 >> 172.21.15.31:0/115726650 conn(0x7f6f540b0cb0 msgr2=0x7f6f540b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:53.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 -- 172.21.15.31:0/115726650 shutdown_connections 2023-11-26T07:20:53.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 -- 172.21.15.31:0/115726650 wait complete. 2023-11-26T07:20:53.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 Processor -- start 2023-11-26T07:20:53.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 -- start start 2023-11-26T07:20:53.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f540cc460 0x7f6f54152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f540cdfb0 0x7f6f5415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f5415a710 0x7f6f5415d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f54072900 con 0x7f6f540cdfb0 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5415a060 con 0x7f6f5415a710 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f5a26b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f54071d80 con 0x7f6f540cc460 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f537fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f540cdfb0 0x7f6f5415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f537fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f540cdfb0 0x7f6f5415d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35734/0 (socket says 172.21.15.31:35734) 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f537fe640 1 -- 172.21.15.31:0/3460771941 learned_addr learned my addr 172.21.15.31:0/3460771941 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f53fff640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f540cc460 0x7f6f54152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.234+0000 7f6f58808640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f5415a710 0x7f6f5415d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f537fe640 1 -- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f540cc460 msgr2=0x7f6f54152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:53.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f537fe640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f540cc460 0x7f6f54152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f537fe640 1 -- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f5415a710 msgr2=0x7f6f5415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f537fe640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f5415a710 0x7f6f5415d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f537fe640 1 -- 172.21.15.31:0/3460771941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c009a70 con 0x7f6f540cdfb0 2023-11-26T07:20:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f537fe640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f540cdfb0 0x7f6f5415d120 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800c750 tx=0x7f6f4800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f50ff9640 1 -- 172.21.15.31:0/3460771941 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48005ae0 con 0x7f6f540cdfb0 2023-11-26T07:20:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f50ff9640 1 -- 172.21.15.31:0/3460771941 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f48005c40 con 0x7f6f540cdfb0 2023-11-26T07:20:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f5a26b640 1 -- 172.21.15.31:0/3460771941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f54072060 con 0x7f6f540cdfb0 2023-11-26T07:20:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f50ff9640 1 -- 172.21.15.31:0/3460771941 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48007070 con 0x7f6f540cdfb0 2023-11-26T07:20:53.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f5a26b640 1 -- 172.21.15.31:0/3460771941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f54072620 con 0x7f6f540cdfb0 2023-11-26T07:20:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f5a26b640 1 -- 172.21.15.31:0/3460771941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f540671f0 con 0x7f6f540cdfb0 2023-11-26T07:20:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f50ff9640 1 -- 172.21.15.31:0/3460771941 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f6f480071f0 con 0x7f6f540cdfb0 2023-11-26T07:20:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f50ff9640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6f40066650 0x7f6f40068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f50ff9640 1 -- 172.21.15.31:0/3460771941 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f6f48083c20 con 0x7f6f540cdfb0 2023-11-26T07:20:53.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.238+0000 7f6f53fff640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6f40066650 0x7f6f40068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:53.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.242+0000 7f6f53fff640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6f40066650 0x7f6f40068b10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6f440011f0 tx=0x7f6f44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:53.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.242+0000 7f6f50ff9640 1 -- 172.21.15.31:0/3460771941 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6f48050fe0 con 0x7f6f540cdfb0 2023-11-26T07:20:53.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:53.406+0000 7f6f5a26b640 1 -- 172.21.15.31:0/3460771941 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6f5406b460 con 0x7f6f540cdfb0 2023-11-26T07:20:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.286+0000 7f6f50ff9640 1 -- 172.21.15.31:0/3460771941 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6f48054600 con 0x7f6f540cdfb0 2023-11-26T07:20:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-11-26T07:20:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 -- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6f40066650 msgr2=0x7f6f40068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6f40066650 0x7f6f40068b10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6f440011f0 tx=0x7f6f44009040 comp rx=0 tx=0).stop 2023-11-26T07:20:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 -- 172.21.15.31:0/3460771941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f540cdfb0 msgr2=0x7f6f5415d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f540cdfb0 0x7f6f5415d120 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800c750 tx=0x7f6f4800cc10 comp rx=0 tx=0).stop 2023-11-26T07:20:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 -- 172.21.15.31:0/3460771941 shutdown_connections 2023-11-26T07:20:54.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6f40066650 0x7f6f40068b10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f5415a710 0x7f6f5415d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f540cdfb0 0x7f6f5415d120 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 --2- 172.21.15.31:0/3460771941 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6f540cc460 0x7f6f54152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 -- 172.21.15.31:0/3460771941 >> 172.21.15.31:0/3460771941 conn(0x7f6f540b0cb0 msgr2=0x7f6f540ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:54.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 -- 172.21.15.31:0/3460771941 shutdown_connections 2023-11-26T07:20:54.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.290+0000 7f6f5a26b640 1 -- 172.21.15.31:0/3460771941 wait complete. 2023-11-26T07:20:54.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:20:54.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f3297b2a640 1 Processor -- start 2023-11-26T07:20:54.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f3297b2a640 1 -- start start 2023-11-26T07:20:54.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f3297b2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f329006e3f0 0x7f329015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f3297b2a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32900ce5d0 0x7f329015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f3297b2a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3290159630 con 0x7f32900cb9b0 2023-11-26T07:20:54.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f3297b2a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329015bba0 con 0x7f32900ce5d0 2023-11-26T07:20:54.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f3297b2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32900cdd50 con 0x7f329006e3f0 2023-11-26T07:20:54.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f32960c7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32900ce5d0 0x7f329015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f32960c7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f32900ce5d0 0x7f329015bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40604/0 (socket says 172.21.15.31:40604) 2023-11-26T07:20:54.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f32960c7640 1 -- 172.21.15.31:0/3216962606 learned_addr learned my addr 172.21.15.31:0/3216962606 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:54.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f32950c5640 1 --2- 172.21.15.31:0/3216962606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f329006e3f0 0x7f329015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f32960c7640 1 -- 172.21.15.31:0/3216962606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f329006e3f0 msgr2=0x7f329015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:54.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f32960c7640 1 --2- 172.21.15.31:0/3216962606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f329006e3f0 0x7f329015f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f32960c7640 1 -- 172.21.15.31:0/3216962606 >> v1:172.21.15.97:6789/0 conn(0x7f32900cb9b0 legacy=0x7f32900cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.470+0000 7f32960c7640 1 -- 172.21.15.31:0/3216962606 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329015f6d0 con 0x7f32900ce5d0 2023-11-26T07:20:54.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f32960c7640 1 --2- 172.21.15.31:0/3216962606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32900ce5d0 0x7f329015bfd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3288009e50 tx=0x7f328801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8f5b93534d510f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:54.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f32948c4640 1 -- 172.21.15.31:0/3216962606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32880a57b0 con 0x7f32900ce5d0 2023-11-26T07:20:54.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f32948c4640 1 -- 172.21.15.31:0/3216962606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32880a5910 con 0x7f32900ce5d0 2023-11-26T07:20:54.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f32948c4640 1 -- 172.21.15.31:0/3216962606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32880a5ca0 con 0x7f32900ce5d0 2023-11-26T07:20:54.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f3297b2a640 1 -- 172.21.15.31:0/3216962606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32900ce5d0 msgr2=0x7f329015bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f3297b2a640 1 --2- 172.21.15.31:0/3216962606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32900ce5d0 0x7f329015bfd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3288009e50 tx=0x7f328801ca10 comp rx=0 tx=0).stop 2023-11-26T07:20:54.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f3297b2a640 1 -- 172.21.15.31:0/3216962606 shutdown_connections 2023-11-26T07:20:54.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f3297b2a640 1 --2- 172.21.15.31:0/3216962606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32900ce5d0 0x7f329015bfd0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f3297b2a640 1 --2- 172.21.15.31:0/3216962606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f329006e3f0 0x7f329015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f3297b2a640 1 -- 172.21.15.31:0/3216962606 >> 172.21.15.31:0/3216962606 conn(0x7f32900b0cb0 msgr2=0x7f32900b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:54.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f3297b2a640 1 -- 172.21.15.31:0/3216962606 shutdown_connections 2023-11-26T07:20:54.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f3297b2a640 1 -- 172.21.15.31:0/3216962606 wait complete. 2023-11-26T07:20:54.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.474+0000 7f3297b2a640 1 Processor -- start 2023-11-26T07:20:54.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f3297b2a640 1 -- start start 2023-11-26T07:20:54.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f3297b2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329006e3f0 0x7f3290099ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f3297b2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32900cb9b0 0x7f329015afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f3297b2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32900ce5d0 0x7f3290071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f3297b2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329015cde0 con 0x7f329006e3f0 2023-11-26T07:20:54.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f3297b2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329015d0c0 con 0x7f32900ce5d0 2023-11-26T07:20:54.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f3297b2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329015d3a0 con 0x7f32900cb9b0 2023-11-26T07:20:54.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32958c6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329006e3f0 0x7f3290099ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32958c6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329006e3f0 0x7f3290099ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40610/0 (socket says 172.21.15.31:40610) 2023-11-26T07:20:54.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32958c6640 1 -- 172.21.15.31:0/2328613562 learned_addr learned my addr 172.21.15.31:0/2328613562 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:54.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32950c5640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32900cb9b0 0x7f329015afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32960c7640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32900ce5d0 0x7f3290071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32958c6640 1 -- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32900cb9b0 msgr2=0x7f329015afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32958c6640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32900cb9b0 0x7f329015afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32958c6640 1 -- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32900ce5d0 msgr2=0x7f3290071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32958c6640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32900ce5d0 0x7f3290071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32958c6640 1 -- 172.21.15.31:0/2328613562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3288009b30 con 0x7f329006e3f0 2023-11-26T07:20:54.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32958c6640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329006e3f0 0x7f3290099ff0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f328c00c500 tx=0x7f328c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:54.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32867fc640 1 -- 172.21.15.31:0/2328613562 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c005220 con 0x7f329006e3f0 2023-11-26T07:20:54.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32867fc640 1 -- 172.21.15.31:0/2328613562 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c005380 con 0x7f329006e3f0 2023-11-26T07:20:54.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f3297b2a640 1 -- 172.21.15.31:0/2328613562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329015d680 con 0x7f329006e3f0 2023-11-26T07:20:54.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f32867fc640 1 -- 172.21.15.31:0/2328613562 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c007270 con 0x7f329006e3f0 2023-11-26T07:20:54.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f3297b2a640 1 -- 172.21.15.31:0/2328613562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329015b500 con 0x7f329006e3f0 2023-11-26T07:20:54.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.478+0000 7f3297b2a640 1 -- 172.21.15.31:0/2328613562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32900671f0 con 0x7f329006e3f0 2023-11-26T07:20:54.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.482+0000 7f32867fc640 1 -- 172.21.15.31:0/2328613562 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f328c0073d0 con 0x7f329006e3f0 2023-11-26T07:20:54.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.482+0000 7f32867fc640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3274066620 0x7f3274068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.482+0000 7f32867fc640 1 -- 172.21.15.31:0/2328613562 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f328c083690 con 0x7f329006e3f0 2023-11-26T07:20:54.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.482+0000 7f32950c5640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3274066620 0x7f3274068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.482+0000 7f32950c5640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3274066620 0x7f3274068ae0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f32901f5da0 tx=0x7f3280009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:54.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.486+0000 7f32867fc640 1 -- 172.21.15.31:0/2328613562 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f328c050c50 con 0x7f329006e3f0 2023-11-26T07:20:54.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.642+0000 7f3297b2a640 1 -- 172.21.15.31:0/2328613562 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f329006b460 con 0x7f329006e3f0 2023-11-26T07:20:54.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.646+0000 7f32867fc640 1 -- 172.21.15.31:0/2328613562 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f328c054270 con 0x7f329006e3f0 2023-11-26T07:20:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 -- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3274066620 msgr2=0x7f3274068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3274066620 0x7f3274068ae0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f32901f5da0 tx=0x7f3280009190 comp rx=0 tx=0).stop 2023-11-26T07:20:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 -- 172.21.15.31:0/2328613562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329006e3f0 msgr2=0x7f3290099ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329006e3f0 0x7f3290099ff0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f328c00c500 tx=0x7f328c00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 -- 172.21.15.31:0/2328613562 shutdown_connections 2023-11-26T07:20:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3274066620 0x7f3274068ae0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32900ce5d0 0x7f3290071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f32900cb9b0 0x7f329015afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 --2- 172.21.15.31:0/2328613562 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f329006e3f0 0x7f3290099ff0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 -- 172.21.15.31:0/2328613562 >> 172.21.15.31:0/2328613562 conn(0x7f32900b0cb0 msgr2=0x7f32900b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 -- 172.21.15.31:0/2328613562 shutdown_connections 2023-11-26T07:20:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.650+0000 7f3297b2a640 1 -- 172.21.15.31:0/2328613562 wait complete. 2023-11-26T07:20:54.671 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2023-11-26T07:20:54.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.810+0000 7f34fcb2a640 1 Processor -- start 2023-11-26T07:20:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.810+0000 7f34fcb2a640 1 -- start start 2023-11-26T07:20:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34fcb2a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34f80cfdb0 0x7f34f80d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34fcb2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34f81654c0 0x7f34f81678b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34fcb2a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f80d12e0 con 0x7f34f815d530 2023-11-26T07:20:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34fcb2a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f80d0760 con 0x7f34f80cfdb0 2023-11-26T07:20:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34fcb2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f80d15c0 con 0x7f34f81654c0 2023-11-26T07:20:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34effff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34f80cfdb0 0x7f34f80d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34effff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34f80cfdb0 0x7f34f80d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40612/0 (socket says 172.21.15.31:40612) 2023-11-26T07:20:54.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34effff640 1 -- 172.21.15.31:0/511551423 learned_addr learned my addr 172.21.15.31:0/511551423 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:54.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34f6d9d640 1 --2- 172.21.15.31:0/511551423 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34f81654c0 0x7f34f81678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34effff640 1 -- 172.21.15.31:0/511551423 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34f81654c0 msgr2=0x7f34f81678b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:54.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34effff640 1 --2- 172.21.15.31:0/511551423 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34f81654c0 0x7f34f81678b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34effff640 1 -- 172.21.15.31:0/511551423 >> v1:172.21.15.97:6789/0 conn(0x7f34f815d530 legacy=0x7f34f80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.814+0000 7f34effff640 1 -- 172.21.15.31:0/511551423 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f80d18a0 con 0x7f34f80cfdb0 2023-11-26T07:20:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34effff640 1 --2- 172.21.15.31:0/511551423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 0x7f34f80d0220 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f34e00098a0 tx=0x7f34e001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a19232cc86c3d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34f5d9b640 1 -- 172.21.15.31:0/511551423 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00a8a10 con 0x7f34f80cfdb0 2023-11-26T07:20:54.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34f5d9b640 1 -- 172.21.15.31:0/511551423 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e00a8b70 con 0x7f34f80cfdb0 2023-11-26T07:20:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34f5d9b640 1 -- 172.21.15.31:0/511551423 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e00b0440 con 0x7f34f80cfdb0 2023-11-26T07:20:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34fcb2a640 1 -- 172.21.15.31:0/511551423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 msgr2=0x7f34f80d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34fcb2a640 1 --2- 172.21.15.31:0/511551423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 0x7f34f80d0220 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f34e00098a0 tx=0x7f34e001ed10 comp rx=0 tx=0).stop 2023-11-26T07:20:54.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34fcb2a640 1 -- 172.21.15.31:0/511551423 shutdown_connections 2023-11-26T07:20:54.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34fcb2a640 1 --2- 172.21.15.31:0/511551423 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34f81654c0 0x7f34f81678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34fcb2a640 1 --2- 172.21.15.31:0/511551423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 0x7f34f80d0220 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34fcb2a640 1 -- 172.21.15.31:0/511551423 >> 172.21.15.31:0/511551423 conn(0x7f34f809c6d0 msgr2=0x7f34f80d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:54.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34fcb2a640 1 -- 172.21.15.31:0/511551423 shutdown_connections 2023-11-26T07:20:54.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34fcb2a640 1 -- 172.21.15.31:0/511551423 wait complete. 2023-11-26T07:20:54.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34fcb2a640 1 Processor -- start 2023-11-26T07:20:54.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.818+0000 7f34fcb2a640 1 -- start start 2023-11-26T07:20:54.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34fcb2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 0x7f34f815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34fcb2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f815d530 0x7f34f8155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34fcb2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34f81654c0 0x7f34f81575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34fcb2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8155f40 con 0x7f34f80cfdb0 2023-11-26T07:20:54.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34fcb2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f815c5a0 con 0x7f34f815d530 2023-11-26T07:20:54.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34fcb2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f815a990 con 0x7f34f81654c0 2023-11-26T07:20:54.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34f659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 0x7f34f815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34f659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 0x7f34f815a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40624/0 (socket says 172.21.15.31:40624) 2023-11-26T07:20:54.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34f659c640 1 -- 172.21.15.31:0/1086638497 learned_addr learned my addr 172.21.15.31:0/1086638497 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:54.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34f6d9d640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34f81654c0 0x7f34f81575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34effff640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f815d530 0x7f34f8155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34f659c640 1 -- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34f81654c0 msgr2=0x7f34f81575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34f659c640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34f81654c0 0x7f34f81575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34f659c640 1 -- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f815d530 msgr2=0x7f34f8155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:54.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34f659c640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f815d530 0x7f34f8155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:54.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34f659c640 1 -- 172.21.15.31:0/1086638497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e00094d0 con 0x7f34f80cfdb0 2023-11-26T07:20:54.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34f659c640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 0x7f34f815a2b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c530 tx=0x7f34e800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34eeffd640 1 -- 172.21.15.31:0/1086638497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8005220 con 0x7f34f80cfdb0 2023-11-26T07:20:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34eeffd640 1 -- 172.21.15.31:0/1086638497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e8005380 con 0x7f34f80cfdb0 2023-11-26T07:20:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34eeffd640 1 -- 172.21.15.31:0/1086638497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8007270 con 0x7f34f80cfdb0 2023-11-26T07:20:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34fcb2a640 1 -- 172.21.15.31:0/1086638497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f815ac70 con 0x7f34f80cfdb0 2023-11-26T07:20:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34fcb2a640 1 -- 172.21.15.31:0/1086638497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f815b230 con 0x7f34f80cfdb0 2023-11-26T07:20:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34fcb2a640 1 -- 172.21.15.31:0/1086638497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f804e850 con 0x7f34f80cfdb0 2023-11-26T07:20:54.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.822+0000 7f34eeffd640 1 -- 172.21.15.31:0/1086638497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f34e80073d0 con 0x7f34f80cfdb0 2023-11-26T07:20:54.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.826+0000 7f34eeffd640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34d8066620 0x7f34d8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:54.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.826+0000 7f34eeffd640 1 -- 172.21.15.31:0/1086638497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f34e80837b0 con 0x7f34f80cfdb0 2023-11-26T07:20:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.826+0000 7f34effff640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34d8066620 0x7f34d8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.826+0000 7f34effff640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34d8066620 0x7f34d8068ae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f34e00b2000 tx=0x7f34e0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:54.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:54.830+0000 7f34eeffd640 1 -- 172.21.15.31:0/1086638497 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f34e8050ca0 con 0x7f34f80cfdb0 2023-11-26T07:20:55.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.042+0000 7f34fcb2a640 1 -- 172.21.15.31:0/1086638497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) v1 -- 0x7f34f80cf980 con 0x7f34f80cfdb0 2023-11-26T07:20:55.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.294+0000 7f34eeffd640 1 -- 172.21.15.31:0/1086638497 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v182) v1 ==== 140+0+0 (secure 0 0 0) 0x7f34e80542c0 con 0x7f34f80cfdb0 2023-11-26T07:20:55.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 -- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34d8066620 msgr2=0x7f34d8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34d8066620 0x7f34d8068ae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f34e00b2000 tx=0x7f34e0008040 comp rx=0 tx=0).stop 2023-11-26T07:20:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 -- 172.21.15.31:0/1086638497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 msgr2=0x7f34f815a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 0x7f34f815a2b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c530 tx=0x7f34e800c9f0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 -- 172.21.15.31:0/1086638497 shutdown_connections 2023-11-26T07:20:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34d8066620 0x7f34d8068ae0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34f81654c0 0x7f34f81575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34f815d530 0x7f34f8155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 --2- 172.21.15.31:0/1086638497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34f80cfdb0 0x7f34f815a2b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 -- 172.21.15.31:0/1086638497 >> 172.21.15.31:0/1086638497 conn(0x7f34f809c6d0 msgr2=0x7f34f80b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 -- 172.21.15.31:0/1086638497 shutdown_connections 2023-11-26T07:20:55.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.298+0000 7f34fcb2a640 1 -- 172.21.15.31:0/1086638497 wait complete. 2023-11-26T07:20:55.319 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-11-26T07:20:55.319 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2023-11-26T07:20:55.320 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:20:55.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:20:55.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.458+0000 7f677889c640 1 Processor -- start 2023-11-26T07:20:55.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677889c640 1 -- start start 2023-11-26T07:20:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677889c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f67740ce5c0 0x7f67740cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677889c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67740b0a70 con 0x7f677406a5b0 2023-11-26T07:20:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677889c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67740b4100 con 0x7f677415d8b0 2023-11-26T07:20:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677889c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67740cddb0 con 0x7f67740ce5c0 2023-11-26T07:20:55.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6772d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f677406a5b0 0x7f677406aa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38964/0 (socket says 172.21.15.31:38964) 2023-11-26T07:20:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6772d9d640 1 -- 172.21.15.31:0/2534514880 learned_addr learned my addr 172.21.15.31:0/2534514880 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f676bfff640 1 --2- 172.21.15.31:0/2534514880 >> v2:172.21.15.97:3300/0 conn(0x7f67740ce5c0 0x7f67740cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277625055 0 0) 0x7f67740b0a70 con 0x7f677406a5b0 2023-11-26T07:20:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6760003530 con 0x7f677406a5b0 2023-11-26T07:20:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 754665611 0 0) 0x7f6760003530 con 0x7f677406a5b0 2023-11-26T07:20:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 >> v2:172.21.15.97:3300/0 conn(0x7f67740ce5c0 msgr2=0x7f67740cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 --2- 172.21.15.31:0/2534514880 >> v2:172.21.15.97:3300/0 conn(0x7f67740ce5c0 0x7f67740cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 >> v1:172.21.15.97:6789/0 conn(0x7f677415d8b0 legacy=0x7f67740b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740b43e0 con 0x7f677406a5b0 2023-11-26T07:20:55.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f676c001a00 con 0x7f677406a5b0 2023-11-26T07:20:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 >> v1:172.21.15.31:6789/0 conn(0x7f677406a5b0 legacy=0x7f677406aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f6760003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67600047d0 0x7f6760004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6760005190 0x7f676000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67740cddb0 con 0x7f67600039d0 2023-11-26T07:20:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676000c070 con 0x7f6760005190 2023-11-26T07:20:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676000c230 con 0x7f67600047d0 2023-11-26T07:20:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677259c640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f6760003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6772d9d640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6760005190 0x7f676000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f676bfff640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67600047d0 0x7f6760004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677259c640 1 -- 172.21.15.31:0/2534514880 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67600047d0 msgr2=0x7f6760004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677259c640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67600047d0 0x7f6760004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677259c640 1 -- 172.21.15.31:0/2534514880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6760005190 msgr2=0x7f676000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677259c640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6760005190 0x7f676000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f677259c640 1 -- 172.21.15.31:0/2534514880 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6760003530 con 0x7f67600039d0 2023-11-26T07:20:55.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.462+0000 7f6772d9d640 1 -- 172.21.15.31:0/2534514880 reap_dead start 2023-11-26T07:20:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f677259c640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f6760003df0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6764001780 tx=0x7f676401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac74b0b79066732 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67640a4ea0 con 0x7f67600039d0 2023-11-26T07:20:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67640a7050 con 0x7f67600039d0 2023-11-26T07:20:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f6771d9b640 1 -- 172.21.15.31:0/2534514880 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764003600 con 0x7f67600039d0 2023-11-26T07:20:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f677889c640 1 -- 172.21.15.31:0/2534514880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 msgr2=0x7f6760003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f677889c640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f6760003df0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6764001780 tx=0x7f676401c3e0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f677889c640 1 -- 172.21.15.31:0/2534514880 shutdown_connections 2023-11-26T07:20:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f677889c640 1 --2- 172.21.15.31:0/2534514880 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f6760003df0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f677889c640 1 -- 172.21.15.31:0/2534514880 >> 172.21.15.31:0/2534514880 conn(0x7f67740b5570 msgr2=0x7f67740b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f677889c640 1 -- 172.21.15.31:0/2534514880 shutdown_connections 2023-11-26T07:20:55.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.466+0000 7f677889c640 1 -- 172.21.15.31:0/2534514880 wait complete. 2023-11-26T07:20:55.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677889c640 1 Processor -- start 2023-11-26T07:20:55.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677889c640 1 -- start start 2023-11-26T07:20:55.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677889c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f677415e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677889c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67741609b0 0x7f677414f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677889c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677415b560 0x7f677415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677889c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677415bf20 con 0x7f67600039d0 2023-11-26T07:20:55.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677889c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677415ec20 con 0x7f677415b560 2023-11-26T07:20:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677889c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677414f730 con 0x7f67741609b0 2023-11-26T07:20:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f677415e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f677415e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40650/0 (socket says 172.21.15.31:40650) 2023-11-26T07:20:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677259c640 1 -- 172.21.15.31:0/2001522341 learned_addr learned my addr 172.21.15.31:0/2001522341 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f6772d9d640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677415b560 0x7f677415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f676bfff640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67741609b0 0x7f677414f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677259c640 1 -- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67741609b0 msgr2=0x7f677414f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677259c640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67741609b0 0x7f677414f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677259c640 1 -- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677415b560 msgr2=0x7f677415b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677259c640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677415b560 0x7f677415b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677259c640 1 -- 172.21.15.31:0/2001522341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677415ee50 con 0x7f67600039d0 2023-11-26T07:20:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f6772d9d640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677415b560 0x7f677415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677259c640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f677415e6e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6764005040 tx=0x7f6760005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f676affd640 1 -- 172.21.15.31:0/2001522341 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764003e40 con 0x7f67600039d0 2023-11-26T07:20:55.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f676affd640 1 -- 172.21.15.31:0/2001522341 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67640a4ea0 con 0x7f67600039d0 2023-11-26T07:20:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677889c640 1 -- 172.21.15.31:0/2001522341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67740c9550 con 0x7f67600039d0 2023-11-26T07:20:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f676affd640 1 -- 172.21.15.31:0/2001522341 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764007250 con 0x7f67600039d0 2023-11-26T07:20:55.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.470+0000 7f677889c640 1 -- 172.21.15.31:0/2001522341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67740c9b10 con 0x7f67600039d0 2023-11-26T07:20:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.474+0000 7f677889c640 1 -- 172.21.15.31:0/2001522341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677404e850 con 0x7f67600039d0 2023-11-26T07:20:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.474+0000 7f676affd640 1 -- 172.21.15.31:0/2001522341 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f67640073b0 con 0x7f67600039d0 2023-11-26T07:20:55.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.474+0000 7f676affd640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67540666a0 0x7f6754068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.474+0000 7f676affd640 1 -- 172.21.15.31:0/2001522341 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f676411e2b0 con 0x7f67600039d0 2023-11-26T07:20:55.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.474+0000 7f676bfff640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67540666a0 0x7f6754068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.474+0000 7f676bfff640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67540666a0 0x7f6754068b60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f67740cd650 tx=0x7f675c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:55.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.478+0000 7f676affd640 1 -- 172.21.15.31:0/2001522341 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f67640eb810 con 0x7f67600039d0 2023-11-26T07:20:55.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 -- 172.21.15.31:0/2001522341 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f677406de30 con 0x7f67600039d0 2023-11-26T07:20:55.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f676affd640 1 -- 172.21.15.31:0/2001522341 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+12902 (secure 0 0 0) 0x7f67640eee30 con 0x7f67600039d0 2023-11-26T07:20:55.645 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:55.645 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":49,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:20:54.287678+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6835/1115502954":"2023-11-27T07:20:50.737990+0000","172.21.15.31:6834/1115502954":"2023-11-27T07:20:50.737990+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:20:55.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 -- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67540666a0 msgr2=0x7f6754068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67540666a0 0x7f6754068b60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f67740cd650 tx=0x7f675c008040 comp rx=0 tx=0).stop 2023-11-26T07:20:55.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 -- 172.21.15.31:0/2001522341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 msgr2=0x7f677415e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f677415e6e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6764005040 tx=0x7f6760005800 comp rx=0 tx=0).stop 2023-11-26T07:20:55.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 -- 172.21.15.31:0/2001522341 shutdown_connections 2023-11-26T07:20:55.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67540666a0 0x7f6754068b60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f677415b560 0x7f677415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f67741609b0 0x7f677414f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 --2- 172.21.15.31:0/2001522341 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67600039d0 0x7f677415e6e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.642+0000 7f677889c640 1 -- 172.21.15.31:0/2001522341 >> 172.21.15.31:0/2001522341 conn(0x7f67740b5570 msgr2=0x7f677406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:55.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.646+0000 7f677889c640 1 -- 172.21.15.31:0/2001522341 shutdown_connections 2023-11-26T07:20:55.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.646+0000 7f677889c640 1 -- 172.21.15.31:0/2001522341 wait complete. 2023-11-26T07:20:55.667 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:20:55.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:20:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0ce079640 1 Processor -- start 2023-11-26T07:20:55.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0ce079640 1 -- start start 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0ce079640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb0c806e3f0 0x7fb0c815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0ce079640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c8159630 con 0x7fb0c80cb9b0 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0ce079640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c815f6d0 con 0x7fb0c80ce5d0 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0ce079640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c806e860 con 0x7fb0c806e3f0 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0c77fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb0c80cb9b0 0x7fb0c80cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38972/0 (socket says 172.21.15.31:38972) 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0c77fe640 1 -- 172.21.15.31:0/3281806472 learned_addr learned my addr 172.21.15.31:0/3281806472 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0c6ffd640 1 --2- 172.21.15.31:0/3281806472 >> v2:172.21.15.97:3300/0 conn(0x7fb0c806e3f0 0x7fb0c815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772311280 0 0) 0x7fb0c8159630 con 0x7fb0c80cb9b0 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.810+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0b0003530 con 0x7fb0c80cb9b0 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2354101901 0 0) 0x7fb0b0003530 con 0x7fb0c80cb9b0 2023-11-26T07:20:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 >> v2:172.21.15.97:3300/0 conn(0x7fb0c806e3f0 msgr2=0x7fb0c815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 --2- 172.21.15.31:0/3281806472 >> v2:172.21.15.97:3300/0 conn(0x7fb0c806e3f0 0x7fb0c815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 >> v1:172.21.15.97:6789/0 conn(0x7fb0c80ce5d0 legacy=0x7fb0c815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80cd1d0 con 0x7fb0c80cb9b0 2023-11-26T07:20:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fb0b8001680 con 0x7fb0c80cb9b0 2023-11-26T07:20:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 >> v1:172.21.15.31:6789/0 conn(0x7fb0c80cb9b0 legacy=0x7fb0c80cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb0b00047d0 0x7fb0b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b0005190 0x7fb0b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c806e860 con 0x7fb0b0003980 2023-11-26T07:20:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b000c030 con 0x7fb0b0005190 2023-11-26T07:20:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b000c1f0 con 0x7fb0b00047d0 2023-11-26T07:20:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c77fe640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c7fff640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b0005190 0x7fb0b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c6ffd640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb0b00047d0 0x7fb0b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c77fe640 1 -- 172.21.15.31:0/3281806472 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb0b00047d0 msgr2=0x7fb0b0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c77fe640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb0b00047d0 0x7fb0b0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c77fe640 1 -- 172.21.15.31:0/3281806472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b0005190 msgr2=0x7fb0b000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:55.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c77fe640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b0005190 0x7fb0b000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c77fe640 1 -- 172.21.15.31:0/3281806472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b0003530 con 0x7fb0b0003980 2023-11-26T07:20:55.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c7fff640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0b0005190 0x7fb0b000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:55.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c7fff640 1 -- 172.21.15.31:0/3281806472 reap_dead start 2023-11-26T07:20:55.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.814+0000 7fb0c77fe640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0b0003da0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c240 tx=0x7fb0b801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9c61de894a7e13 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:55.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8005ae0 con 0x7fb0b0003980 2023-11-26T07:20:55.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b8005c40 con 0x7fb0b0003980 2023-11-26T07:20:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0c67fc640 1 -- 172.21.15.31:0/3281806472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80072b0 con 0x7fb0b0003980 2023-11-26T07:20:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 -- 172.21.15.31:0/3281806472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 msgr2=0x7fb0b0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0b0003da0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c240 tx=0x7fb0b801cc40 comp rx=0 tx=0).stop 2023-11-26T07:20:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 -- 172.21.15.31:0/3281806472 shutdown_connections 2023-11-26T07:20:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 --2- 172.21.15.31:0/3281806472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0b0003da0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 -- 172.21.15.31:0/3281806472 >> 172.21.15.31:0/3281806472 conn(0x7fb0c80b0cb0 msgr2=0x7fb0c80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 -- 172.21.15.31:0/3281806472 shutdown_connections 2023-11-26T07:20:55.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 -- 172.21.15.31:0/3281806472 wait complete. 2023-11-26T07:20:55.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 Processor -- start 2023-11-26T07:20:55.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 -- start start 2023-11-26T07:20:55.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0c81f7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c80b3d60 0x7fb0c80b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb0c809bdf0 0x7fb0c809c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c81f8140 con 0x7fb0b0003980 2023-11-26T07:20:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c809a340 con 0x7fb0c80b3d60 2023-11-26T07:20:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.818+0000 7fb0ce079640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c809a5e0 con 0x7fb0c809bdf0 2023-11-26T07:20:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0c81f7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0c81f7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40680/0 (socket says 172.21.15.31:40680) 2023-11-26T07:20:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c77fe640 1 -- 172.21.15.31:0/742270747 learned_addr learned my addr 172.21.15.31:0/742270747 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c7fff640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb0c809bdf0 0x7fb0c809c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c6ffd640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c80b3d60 0x7fb0c80b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c77fe640 1 -- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb0c809bdf0 msgr2=0x7fb0c809c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c77fe640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb0c809bdf0 0x7fb0c809c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c77fe640 1 -- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c80b3d60 msgr2=0x7fb0c80b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c77fe640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c80b3d60 0x7fb0c80b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c77fe640 1 -- 172.21.15.31:0/742270747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80cae10 con 0x7fb0b0003980 2023-11-26T07:20:55.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0c77fe640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0c81f7c00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c240 tx=0x7fb0b0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:55.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0affff640 1 -- 172.21.15.31:0/742270747 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80b4070 con 0x7fb0b0003980 2023-11-26T07:20:55.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0affff640 1 -- 172.21.15.31:0/742270747 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b8001500 con 0x7fb0b0003980 2023-11-26T07:20:55.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0affff640 1 -- 172.21.15.31:0/742270747 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8007070 con 0x7fb0b0003980 2023-11-26T07:20:55.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0ce079640 1 -- 172.21.15.31:0/742270747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c80cb0f0 con 0x7fb0b0003980 2023-11-26T07:20:55.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0ce079640 1 -- 172.21.15.31:0/742270747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c809c7e0 con 0x7fb0b0003980 2023-11-26T07:20:55.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0ce079640 1 -- 172.21.15.31:0/742270747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c80671f0 con 0x7fb0b0003980 2023-11-26T07:20:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.822+0000 7fb0affff640 1 -- 172.21.15.31:0/742270747 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fb0b80071d0 con 0x7fb0b0003980 2023-11-26T07:20:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.826+0000 7fb0affff640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0b4066620 0x7fb0b4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.826+0000 7fb0affff640 1 -- 172.21.15.31:0/742270747 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb0b811daa0 con 0x7fb0b0003980 2023-11-26T07:20:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.826+0000 7fb0c6ffd640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0b4066620 0x7fb0b4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:55.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.826+0000 7fb0c6ffd640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0b4066620 0x7fb0b4068ae0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8099fa0 tx=0x7fb0bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:55.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.830+0000 7fb0affff640 1 -- 172.21.15.31:0/742270747 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb0b80eb030 con 0x7fb0b0003980 2023-11-26T07:20:55.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 -- 172.21.15.31:0/742270747 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0c806b460 con 0x7fb0b0003980 2023-11-26T07:20:55.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0affff640 1 -- 172.21.15.31:0/742270747 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0b80ee650 con 0x7fb0b0003980 2023-11-26T07:20:55.988 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:20:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 -- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0b4066620 msgr2=0x7fb0b4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0b4066620 0x7fb0b4068ae0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8099fa0 tx=0x7fb0bc008040 comp rx=0 tx=0).stop 2023-11-26T07:20:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 -- 172.21.15.31:0/742270747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 msgr2=0x7fb0c81f7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0c81f7c00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c240 tx=0x7fb0b0005800 comp rx=0 tx=0).stop 2023-11-26T07:20:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 -- 172.21.15.31:0/742270747 shutdown_connections 2023-11-26T07:20:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0b4066620 0x7fb0b4068ae0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb0c809bdf0 0x7fb0c809c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0c80b3d60 0x7fb0c80b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 --2- 172.21.15.31:0/742270747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0b0003980 0x7fb0c81f7c00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:55.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 -- 172.21.15.31:0/742270747 >> 172.21.15.31:0/742270747 conn(0x7fb0c80b0cb0 msgr2=0x7fb0c80cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:55.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 -- 172.21.15.31:0/742270747 shutdown_connections 2023-11-26T07:20:55.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:55.986+0000 7fb0ce079640 1 -- 172.21.15.31:0/742270747 wait complete. 2023-11-26T07:20:56.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:20:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb10c7f8640 1 Processor -- start 2023-11-26T07:20:56.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb10c7f8640 1 -- start start 2023-11-26T07:20:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb10c7f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1040cead0 0x7fb1040d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb10c7f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 0x7fb10406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb10c7f8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1040d2300 con 0x7fb10415d7b0 2023-11-26T07:20:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb10c7f8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1040d17e0 con 0x7fb1040cead0 2023-11-26T07:20:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb10c7f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040cefd0 con 0x7fb1040d1eb0 2023-11-26T07:20:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb109d93640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1040cead0 0x7fb1040d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb10ad95640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 0x7fb10406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:56.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb109d93640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1040cead0 0x7fb1040d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:44264/0 (socket says 172.21.15.31:44264) 2023-11-26T07:20:56.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb109d93640 1 -- 172.21.15.31:0/2076975971 learned_addr learned my addr 172.21.15.31:0/2076975971 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:56.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb109d93640 1 -- 172.21.15.31:0/2076975971 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 msgr2=0x7fb10406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:56.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb109d93640 1 --2- 172.21.15.31:0/2076975971 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 0x7fb10406bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:56.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb109d93640 1 -- 172.21.15.31:0/2076975971 >> v1:172.21.15.97:6789/0 conn(0x7fb10415d7b0 legacy=0x7fb1040d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:56.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.166+0000 7fb109d93640 1 -- 172.21.15.31:0/2076975971 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040cf2b0 con 0x7fb1040cead0 2023-11-26T07:20:56.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb109d93640 1 --2- 172.21.15.31:0/2076975971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1040cead0 0x7fb1040d1970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009870 tx=0x7fb0f401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b44178fb9bee3d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:56.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb109592640 1 -- 172.21.15.31:0/2076975971 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40ab5f0 con 0x7fb1040cead0 2023-11-26T07:20:56.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb109592640 1 -- 172.21.15.31:0/2076975971 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f40ab750 con 0x7fb1040cead0 2023-11-26T07:20:56.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb109592640 1 -- 172.21.15.31:0/2076975971 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40abae0 con 0x7fb1040cead0 2023-11-26T07:20:56.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2076975971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1040cead0 msgr2=0x7fb1040d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:56.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb10c7f8640 1 --2- 172.21.15.31:0/2076975971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1040cead0 0x7fb1040d1970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009870 tx=0x7fb0f401c3f0 comp rx=0 tx=0).stop 2023-11-26T07:20:56.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2076975971 shutdown_connections 2023-11-26T07:20:56.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb10c7f8640 1 --2- 172.21.15.31:0/2076975971 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 0x7fb10406bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:56.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb10c7f8640 1 --2- 172.21.15.31:0/2076975971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1040cead0 0x7fb1040d1970 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:56.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2076975971 >> 172.21.15.31:0/2076975971 conn(0x7fb1040d33b0 msgr2=0x7fb1040d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:56.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2076975971 shutdown_connections 2023-11-26T07:20:56.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.170+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2076975971 wait complete. 2023-11-26T07:20:56.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10c7f8640 1 Processor -- start 2023-11-26T07:20:56.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10c7f8640 1 -- start start 2023-11-26T07:20:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10c7f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040cead0 0x7fb104159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10c7f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 0x7fb1040b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10c7f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10415d7b0 0x7fb1041548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10c7f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104157380 con 0x7fb1040cead0 2023-11-26T07:20:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10c7f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040c93e0 con 0x7fb10415d7b0 2023-11-26T07:20:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10c7f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040c96c0 con 0x7fb1040d1eb0 2023-11-26T07:20:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10a594640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040cead0 0x7fb104159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10a594640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040cead0 0x7fb104159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40692/0 (socket says 172.21.15.31:40692) 2023-11-26T07:20:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10a594640 1 -- 172.21.15.31:0/2683536668 learned_addr learned my addr 172.21.15.31:0/2683536668 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10ad95640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10415d7b0 0x7fb1041548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb109d93640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 0x7fb1040b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10a594640 1 -- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 msgr2=0x7fb1040b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10a594640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 0x7fb1040b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10a594640 1 -- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10415d7b0 msgr2=0x7fb1041548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10a594640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10415d7b0 0x7fb1041548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10a594640 1 -- 172.21.15.31:0/2683536668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40094d0 con 0x7fb1040cead0 2023-11-26T07:20:56.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb109d93640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 0x7fb1040b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:56.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10ad95640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10415d7b0 0x7fb1041548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:56.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10a594640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040cead0 0x7fb104159b90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb10000c3c0 tx=0x7fb10000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:56.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb0faffd640 1 -- 172.21.15.31:0/2683536668 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000052d0 con 0x7fb1040cead0 2023-11-26T07:20:56.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb0faffd640 1 -- 172.21.15.31:0/2683536668 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb100005430 con 0x7fb1040cead0 2023-11-26T07:20:56.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2683536668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1040c99a0 con 0x7fb1040cead0 2023-11-26T07:20:56.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb0faffd640 1 -- 172.21.15.31:0/2683536668 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb100003840 con 0x7fb1040cead0 2023-11-26T07:20:56.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.174+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2683536668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10415a0d0 con 0x7fb1040cead0 2023-11-26T07:20:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.178+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2683536668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10404e850 con 0x7fb1040cead0 2023-11-26T07:20:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.178+0000 7fb0faffd640 1 -- 172.21.15.31:0/2683536668 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fb1000039a0 con 0x7fb1040cead0 2023-11-26T07:20:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.178+0000 7fb0faffd640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0e8066620 0x7fb0e8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.178+0000 7fb0faffd640 1 -- 172.21.15.31:0/2683536668 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb100083270 con 0x7fb1040cead0 2023-11-26T07:20:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.178+0000 7fb109d93640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0e8066620 0x7fb0e8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.178+0000 7fb109d93640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0e8066620 0x7fb0e8068ae0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40af000 tx=0x7fb0f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.182+0000 7fb0faffd640 1 -- 172.21.15.31:0/2683536668 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb100050760 con 0x7fb1040cead0 2023-11-26T07:20:56.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:56.394+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2683536668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7fb1040d2300 con 0x7fb1040cead0 2023-11-26T07:20:57.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.298+0000 7fb0faffd640 1 -- 172.21.15.31:0/2683536668 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v184) v1 ==== 164+0+0 (secure 0 0 0) 0x7fb1040d2300 con 0x7fb1040cead0 2023-11-26T07:20:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0e8066620 msgr2=0x7fb0e8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0e8066620 0x7fb0e8068ae0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40af000 tx=0x7fb0f4008040 comp rx=0 tx=0).stop 2023-11-26T07:20:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2683536668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040cead0 msgr2=0x7fb104159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040cead0 0x7fb104159b90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb10000c3c0 tx=0x7fb10000c880 comp rx=0 tx=0).stop 2023-11-26T07:20:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2683536668 shutdown_connections 2023-11-26T07:20:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb0e8066620 0x7fb0e8068ae0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb10415d7b0 0x7fb1041548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb1040d1eb0 0x7fb1040b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 --2- 172.21.15.31:0/2683536668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040cead0 0x7fb104159b90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2683536668 >> 172.21.15.31:0/2683536668 conn(0x7fb1040d33b0 msgr2=0x7fb1040d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2683536668 shutdown_connections 2023-11-26T07:20:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.302+0000 7fb10c7f8640 1 -- 172.21.15.31:0/2683536668 wait complete. 2023-11-26T07:20:57.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:20:57.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f62c16640 1 Processor -- start 2023-11-26T07:20:57.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f62c16640 1 -- start start 2023-11-26T07:20:57.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f62c16640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cb9b0 0x7f5f5c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f62c16640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f5c0cbd90 0x7f5f5c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f62c16640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5c159630 con 0x7f5f5c0ce5d0 2023-11-26T07:20:57.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f62c16640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5c15bba0 con 0x7f5f5c0cbd90 2023-11-26T07:20:57.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f62c16640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0cdd50 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f5bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f5c0cbd90 0x7f5f5c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f611b3640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5f5c0ce5d0 0x7f5f5c15bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38986/0 (socket says 172.21.15.31:38986) 2023-11-26T07:20:57.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f611b3640 1 -- 172.21.15.31:0/3032638250 learned_addr learned my addr 172.21.15.31:0/3032638250 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:57.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f609b2640 1 --2- 172.21.15.31:0/3032638250 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cb9b0 0x7f5f5c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f5bfff640 1 -- 172.21.15.31:0/3032638250 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cb9b0 msgr2=0x7f5f5c06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:57.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f5bfff640 1 --2- 172.21.15.31:0/3032638250 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cb9b0 0x7f5f5c06e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f5bfff640 1 -- 172.21.15.31:0/3032638250 >> v1:172.21.15.31:6789/0 conn(0x7f5f5c0ce5d0 legacy=0x7f5f5c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f5bfff640 1 -- 172.21.15.31:0/3032638250 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c15f6d0 con 0x7f5f5c0cbd90 2023-11-26T07:20:57.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.478+0000 7f5f5b7fe640 1 -- 172.21.15.31:0/3032638250 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605840500 0 0) 0x7f5f5c159630 con 0x7f5f5c0ce5d0 2023-11-26T07:20:57.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f5bfff640 1 --2- 172.21.15.31:0/3032638250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cbd90 0x7f5f5c15f190 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5f480097c0 tx=0x7f5f4801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7205ae765deccef5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:57.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f5b7fe640 1 -- 172.21.15.31:0/3032638250 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480ab650 con 0x7f5f5c0cbd90 2023-11-26T07:20:57.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f5b7fe640 1 -- 172.21.15.31:0/3032638250 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f480ab7b0 con 0x7f5f5c0cbd90 2023-11-26T07:20:57.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f5b7fe640 1 -- 172.21.15.31:0/3032638250 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480abb40 con 0x7f5f5c0cbd90 2023-11-26T07:20:57.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f62c16640 1 -- 172.21.15.31:0/3032638250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cbd90 msgr2=0x7f5f5c15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f62c16640 1 --2- 172.21.15.31:0/3032638250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cbd90 0x7f5f5c15f190 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5f480097c0 tx=0x7f5f4801c450 comp rx=0 tx=0).stop 2023-11-26T07:20:57.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f62c16640 1 -- 172.21.15.31:0/3032638250 shutdown_connections 2023-11-26T07:20:57.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f62c16640 1 --2- 172.21.15.31:0/3032638250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cbd90 0x7f5f5c15f190 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f62c16640 1 --2- 172.21.15.31:0/3032638250 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cb9b0 0x7f5f5c06e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f62c16640 1 -- 172.21.15.31:0/3032638250 >> 172.21.15.31:0/3032638250 conn(0x7f5f5c0b0cb0 msgr2=0x7f5f5c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:57.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f62c16640 1 -- 172.21.15.31:0/3032638250 shutdown_connections 2023-11-26T07:20:57.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f62c16640 1 -- 172.21.15.31:0/3032638250 wait complete. 2023-11-26T07:20:57.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.482+0000 7f5f62c16640 1 Processor -- start 2023-11-26T07:20:57.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f62c16640 1 -- start start 2023-11-26T07:20:57.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f62c16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cb9b0 0x7f5f5c0ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f62c16640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cbd90 0x7f5f5c155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f62c16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f5c0ce5d0 0x7f5f5c0b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f62c16640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0ace80 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f62c16640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0abf90 con 0x7f5f5c0ce5d0 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f62c16640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0ac270 con 0x7f5f5c0cbd90 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f609b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cb9b0 0x7f5f5c0ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f609b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cb9b0 0x7f5f5c0ad310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40704/0 (socket says 172.21.15.31:40704) 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f609b2640 1 -- 172.21.15.31:0/1986353352 learned_addr learned my addr 172.21.15.31:0/1986353352 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f5bfff640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cbd90 0x7f5f5c155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f611b3640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f5c0ce5d0 0x7f5f5c0b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f609b2640 1 -- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cbd90 msgr2=0x7f5f5c155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f609b2640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cbd90 0x7f5f5c155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f609b2640 1 -- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f5c0ce5d0 msgr2=0x7f5f5c0b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f609b2640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f5c0ce5d0 0x7f5f5c0b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f609b2640 1 -- 172.21.15.31:0/1986353352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48009420 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f5bfff640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cbd90 0x7f5f5c155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:57.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f609b2640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cb9b0 0x7f5f5c0ad310 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000d800 tx=0x7f5f5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:57.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f597fa640 1 -- 172.21.15.31:0/1986353352 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5000b6c0 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f597fa640 1 -- 172.21.15.31:0/1986353352 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5000b820 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f597fa640 1 -- 172.21.15.31:0/1986353352 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5000bb90 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f62c16640 1 -- 172.21.15.31:0/1986353352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c0ac550 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f62c16640 1 -- 172.21.15.31:0/1986353352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c0acb10 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.486+0000 7f5f62c16640 1 -- 172.21.15.31:0/1986353352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5c0671f0 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.490+0000 7f5f597fa640 1 -- 172.21.15.31:0/1986353352 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f5f5000bcf0 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.490+0000 7f5f597fa640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5f40066620 0x7f5f40068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:57.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.490+0000 7f5f597fa640 1 -- 172.21.15.31:0/1986353352 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5f500844d0 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.490+0000 7f5f5bfff640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5f40066620 0x7f5f40068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:57.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.490+0000 7f5f5bfff640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5f40066620 0x7f5f40068ae0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5f480af000 tx=0x7f5f480a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:57.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.494+0000 7f5f597fa640 1 -- 172.21.15.31:0/1986353352 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5f50051930 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.650+0000 7f5f62c16640 1 -- 172.21.15.31:0/1986353352 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5f5c06b460 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f597fa640 1 -- 172.21.15.31:0/1986353352 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5f50011340 con 0x7f5f5c0cb9b0 2023-11-26T07:20:57.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 -- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5f40066620 msgr2=0x7f5f40068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5f40066620 0x7f5f40068ae0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5f480af000 tx=0x7f5f480a7000 comp rx=0 tx=0).stop 2023-11-26T07:20:57.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 -- 172.21.15.31:0/1986353352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cb9b0 msgr2=0x7f5f5c0ad310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:57.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cb9b0 0x7f5f5c0ad310 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5f5000d800 tx=0x7f5f5000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 -- 172.21.15.31:0/1986353352 shutdown_connections 2023-11-26T07:20:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5f40066620 0x7f5f40068ae0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f5c0ce5d0 0x7f5f5c0b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5f5c0cbd90 0x7f5f5c155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 --2- 172.21.15.31:0/1986353352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5c0cb9b0 0x7f5f5c0ad310 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 -- 172.21.15.31:0/1986353352 >> 172.21.15.31:0/1986353352 conn(0x7f5f5c0b0cb0 msgr2=0x7f5f5c1654d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 -- 172.21.15.31:0/1986353352 shutdown_connections 2023-11-26T07:20:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:57.658+0000 7f5f62c16640 1 -- 172.21.15.31:0/1986353352 wait complete. 2023-11-26T07:20:57.680 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:20:57.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:20:57.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:20:57.861 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:20:57.861 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:20:57.861 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:20:57.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:20:57.870 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:20:57.870 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:20:57.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:20:57.880 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:20:57.880 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:20:57.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:20:57.923 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:20:57.923 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:20:57.932 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:20:57.932 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:20:57.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:20:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.106+0000 7f3a22b94640 1 Processor -- start 2023-11-26T07:20:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.106+0000 7f3a22b94640 1 -- start start 2023-11-26T07:20:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a22b94640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3a1c16a210 0x7f3a1c070670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a22b94640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a1c070bb0 0x7f3a1c06d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a22b94640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3a1c06d7f0 0x7f3a1c06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a22b94640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b4300 con 0x7f3a1c070bb0 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a22b94640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c071040 con 0x7f3a1c16a210 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a22b94640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c16a5f0 con 0x7f3a1c06d7f0 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a1bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a1c070bb0 0x7f3a1c06d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a1bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a1c070bb0 0x7f3a1c06d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40718/0 (socket says 172.21.15.31:40718) 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a20930640 1 --2- 172.21.15.31:0/670866952 >> v2:172.21.15.97:3300/0 conn(0x7f3a1c16a210 0x7f3a1c070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a21131640 1 --2- 172.21.15.31:0/670866952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3a1c06d7f0 0x7f3a1c06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a1bfff640 1 -- 172.21.15.31:0/670866952 learned_addr learned my addr 172.21.15.31:0/670866952 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a1bfff640 1 -- 172.21.15.31:0/670866952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3a1c06d7f0 msgr2=0x7f3a1c06dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a1bfff640 1 --2- 172.21.15.31:0/670866952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3a1c06d7f0 0x7f3a1c06dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a1bfff640 1 -- 172.21.15.31:0/670866952 >> v2:172.21.15.97:3300/0 conn(0x7f3a1c16a210 msgr2=0x7f3a1c070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a1bfff640 1 --2- 172.21.15.31:0/670866952 >> v2:172.21.15.97:3300/0 conn(0x7f3a1c16a210 0x7f3a1c070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.110+0000 7f3a1bfff640 1 -- 172.21.15.31:0/670866952 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0b4020 con 0x7f3a1c070bb0 2023-11-26T07:20:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a1bfff640 1 --2- 172.21.15.31:0/670866952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c070bb0 0x7f3a1c06d2b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c068a60 tx=0x7f3a080253a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bf4d574fe283dd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:58.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a1b7fe640 1 -- 172.21.15.31:0/670866952 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08004230 con 0x7f3a1c070bb0 2023-11-26T07:20:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a1b7fe640 1 -- 172.21.15.31:0/670866952 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a08004390 con 0x7f3a1c070bb0 2023-11-26T07:20:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a1b7fe640 1 -- 172.21.15.31:0/670866952 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08003820 con 0x7f3a1c070bb0 2023-11-26T07:20:58.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 -- 172.21.15.31:0/670866952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c070bb0 msgr2=0x7f3a1c06d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 --2- 172.21.15.31:0/670866952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c070bb0 0x7f3a1c06d2b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c068a60 tx=0x7f3a080253a0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 -- 172.21.15.31:0/670866952 shutdown_connections 2023-11-26T07:20:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 --2- 172.21.15.31:0/670866952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3a1c06d7f0 0x7f3a1c06dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 --2- 172.21.15.31:0/670866952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c070bb0 0x7f3a1c06d2b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 --2- 172.21.15.31:0/670866952 >> v2:172.21.15.97:3300/0 conn(0x7f3a1c16a210 0x7f3a1c070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 -- 172.21.15.31:0/670866952 >> 172.21.15.31:0/670866952 conn(0x7f3a1c0b0c40 msgr2=0x7f3a1c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 -- 172.21.15.31:0/670866952 shutdown_connections 2023-11-26T07:20:58.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 -- 172.21.15.31:0/670866952 wait complete. 2023-11-26T07:20:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 Processor -- start 2023-11-26T07:20:58.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.114+0000 7f3a22b94640 1 -- start start 2023-11-26T07:20:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a22b94640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3a1c06d7f0 0x7f3a1c16a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a22b94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1c070bb0 0x7f3a1c15ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a22b94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c16a210 0x7f3a1c160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a22b94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c15e910 con 0x7f3a1c16a210 2023-11-26T07:20:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a22b94640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c162750 con 0x7f3a1c070bb0 2023-11-26T07:20:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a22b94640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c161f40 con 0x7f3a1c06d7f0 2023-11-26T07:20:58.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a21131640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c16a210 0x7f3a1c160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a21131640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c16a210 0x7f3a1c160480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40732/0 (socket says 172.21.15.31:40732) 2023-11-26T07:20:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a21131640 1 -- 172.21.15.31:0/1321801936 learned_addr learned my addr 172.21.15.31:0/1321801936 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a20930640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3a1c06d7f0 0x7f3a1c16a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a1bfff640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1c070bb0 0x7f3a1c15ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a21131640 1 -- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3a1c06d7f0 msgr2=0x7f3a1c16a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a21131640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3a1c06d7f0 0x7f3a1c16a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a21131640 1 -- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1c070bb0 msgr2=0x7f3a1c15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a21131640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1c070bb0 0x7f3a1c15ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a21131640 1 -- 172.21.15.31:0/1321801936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0800b590 con 0x7f3a1c16a210 2023-11-26T07:20:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a21131640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c16a210 0x7f3a1c160480 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400fb90 tx=0x7f3a1400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a197fa640 1 -- 172.21.15.31:0/1321801936 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1400dc30 con 0x7f3a1c16a210 2023-11-26T07:20:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a197fa640 1 -- 172.21.15.31:0/1321801936 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1400ddb0 con 0x7f3a1c16a210 2023-11-26T07:20:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a22b94640 1 -- 172.21.15.31:0/1321801936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c15e150 con 0x7f3a1c16a210 2023-11-26T07:20:58.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a197fa640 1 -- 172.21.15.31:0/1321801936 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a140146b0 con 0x7f3a1c16a210 2023-11-26T07:20:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a22b94640 1 -- 172.21.15.31:0/1321801936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c162190 con 0x7f3a1c16a210 2023-11-26T07:20:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.118+0000 7f3a22b94640 1 -- 172.21.15.31:0/1321801936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c04e850 con 0x7f3a1c16a210 2023-11-26T07:20:58.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.122+0000 7f3a197fa640 1 -- 172.21.15.31:0/1321801936 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f3a14014810 con 0x7f3a1c16a210 2023-11-26T07:20:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.122+0000 7f3a197fa640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3a00066620 0x7f3a00068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.122+0000 7f3a197fa640 1 -- 172.21.15.31:0/1321801936 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3a14084630 con 0x7f3a1c16a210 2023-11-26T07:20:58.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.122+0000 7f3a20930640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3a00066620 0x7f3a00068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.122+0000 7f3a20930640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3a00066620 0x7f3a00068ae0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000dd60 tx=0x7f3a1000f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:58.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.126+0000 7f3a197fa640 1 -- 172.21.15.31:0/1321801936 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3a14051ab0 con 0x7f3a1c16a210 2023-11-26T07:20:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 -- 172.21.15.31:0/1321801936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a1c16d9c0 con 0x7f3a1c16a210 2023-11-26T07:20:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a197fa640 1 -- 172.21.15.31:0/1321801936 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f3a14018020 con 0x7f3a1c16a210 2023-11-26T07:20:58.325 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:58.325 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5434,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/2358434581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2358434581},{"type":"v1","addr":"172.21.15.97:6836","nonce":2358434581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2592011924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2592011924},{"type":"v1","addr":"172.21.15.97:6837","nonce":2592011924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1136407751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1136407751},{"type":"v1","addr":"172.21.15.31:6833","nonce":1136407751}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/568649764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":568649764},{"type":"v1","addr":"172.21.15.31:6835","nonce":568649764}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[]} 2023-11-26T07:20:58.326 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 30 2023-11-26T07:20:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 -- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3a00066620 msgr2=0x7f3a00068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3a00066620 0x7f3a00068ae0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000dd60 tx=0x7f3a1000f040 comp rx=0 tx=0).stop 2023-11-26T07:20:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 -- 172.21.15.31:0/1321801936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c16a210 msgr2=0x7f3a1c160480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c16a210 0x7f3a1c160480 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400fb90 tx=0x7f3a1400d520 comp rx=0 tx=0).stop 2023-11-26T07:20:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 -- 172.21.15.31:0/1321801936 shutdown_connections 2023-11-26T07:20:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3a00066620 0x7f3a00068ae0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a1c16a210 0x7f3a1c160480 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a1c070bb0 0x7f3a1c15ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 --2- 172.21.15.31:0/1321801936 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3a1c06d7f0 0x7f3a1c16a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.322+0000 7f3a22b94640 1 -- 172.21.15.31:0/1321801936 >> 172.21.15.31:0/1321801936 conn(0x7f3a1c0b0c40 msgr2=0x7f3a1c0b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.326+0000 7f3a22b94640 1 -- 172.21.15.31:0/1321801936 shutdown_connections 2023-11-26T07:20:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.326+0000 7f3a22b94640 1 -- 172.21.15.31:0/1321801936 wait complete. 2023-11-26T07:20:58.347 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:20:58.347 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:20:58.347 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:20:58.347 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:20:58.348 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:20:58.348 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:20:58.348 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:20:58.348 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:20:58.348 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:20:58.348 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:20:58.349 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:20:58.349 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:20:58.349 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:20:58.349 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:20:58.349 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:20:58.349 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:20:58.361 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:20:58.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:20:58.362 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:20:58.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:20:58.366 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:20:58.366 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:20:58.366 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:20:58.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:20:58.409 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:20:58.410 DEBUG:teuthology.parallel:result is None 2023-11-26T07:20:58.410 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:20:58.411 DEBUG:teuthology.parallel:result is None 2023-11-26T07:20:58.411 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:20:58.412 DEBUG:teuthology.parallel:result is None 2023-11-26T07:20:58.412 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:20:58.413 DEBUG:teuthology.parallel:result is None 2023-11-26T07:20:58.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:20:58.499 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-11-26T07:20:58.500 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-11-26T07:20:58.501 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2023-11-26T07:20:58.502 INFO:tasks.ceph.mds.d.smithi097.stdout:starting mds.d at 2023-11-26T07:20:58.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a9a0c9640 1 Processor -- start 2023-11-26T07:20:58.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a9a0c9640 1 -- start start 2023-11-26T07:20:58.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a9a0c9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 0x7f8a9406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a9a0c9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a940cbd90 0x7f8a9415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a9a0c9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a94159630 con 0x7f8a940ce5d0 2023-11-26T07:20:58.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a9a0c9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9415bba0 con 0x7f8a940cbd90 2023-11-26T07:20:58.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a9a0c9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a940cdd50 con 0x7f8a940cb9b0 2023-11-26T07:20:58.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a92ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a940cbd90 0x7f8a9415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a93fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8a940ce5d0 0x7f8a9415bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38994/0 (socket says 172.21.15.31:38994) 2023-11-26T07:20:58.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a92ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8a940cbd90 0x7f8a9415f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40770/0 (socket says 172.21.15.31:40770) 2023-11-26T07:20:58.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a93fff640 1 -- 172.21.15.31:0/4292185898 learned_addr learned my addr 172.21.15.31:0/4292185898 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:58.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a937fe640 1 --2- 172.21.15.31:0/4292185898 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 0x7f8a9406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a92ffd640 1 -- 172.21.15.31:0/4292185898 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 msgr2=0x7f8a9406e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a92ffd640 1 --2- 172.21.15.31:0/4292185898 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 0x7f8a9406e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a92ffd640 1 -- 172.21.15.31:0/4292185898 >> v1:172.21.15.31:6789/0 conn(0x7f8a940ce5d0 legacy=0x7f8a9415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a92ffd640 1 -- 172.21.15.31:0/4292185898 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9415f6d0 con 0x7f8a940cbd90 2023-11-26T07:20:58.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.558+0000 7f8a927fc640 1 -- 172.21.15.31:0/4292185898 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822268535 0 0) 0x7f8a94159630 con 0x7f8a940ce5d0 2023-11-26T07:20:58.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a92ffd640 1 --2- 172.21.15.31:0/4292185898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 0x7f8a9415f190 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8a880097c0 tx=0x7f8a8801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8d51776f8c223f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:58.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a927fc640 1 -- 172.21.15.31:0/4292185898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a880ab650 con 0x7f8a940cbd90 2023-11-26T07:20:58.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a927fc640 1 -- 172.21.15.31:0/4292185898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a880ab7b0 con 0x7f8a940cbd90 2023-11-26T07:20:58.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a927fc640 1 -- 172.21.15.31:0/4292185898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a880abb40 con 0x7f8a940cbd90 2023-11-26T07:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/4292185898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 msgr2=0x7f8a9415f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a9a0c9640 1 --2- 172.21.15.31:0/4292185898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 0x7f8a9415f190 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8a880097c0 tx=0x7f8a8801c450 comp rx=0 tx=0).stop 2023-11-26T07:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/4292185898 shutdown_connections 2023-11-26T07:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a9a0c9640 1 --2- 172.21.15.31:0/4292185898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 0x7f8a9415f190 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a9a0c9640 1 --2- 172.21.15.31:0/4292185898 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 0x7f8a9406e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/4292185898 >> 172.21.15.31:0/4292185898 conn(0x7f8a940b0cb0 msgr2=0x7f8a940b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/4292185898 shutdown_connections 2023-11-26T07:20:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.562+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/4292185898 wait complete. 2023-11-26T07:20:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a9a0c9640 1 Processor -- start 2023-11-26T07:20:58.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a9a0c9640 1 -- start start 2023-11-26T07:20:58.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a9a0c9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 0x7f8a941fae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a9a0c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 0x7f8a940afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a9a0c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a940ce5d0 0x7f8a94153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a9a0c9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9415e380 con 0x7f8a940cbd90 2023-11-26T07:20:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a9a0c9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a941fa9f0 con 0x7f8a940ce5d0 2023-11-26T07:20:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a9a0c9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9415e840 con 0x7f8a940cb9b0 2023-11-26T07:20:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 0x7f8a940afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 0x7f8a940afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40786/0 (socket says 172.21.15.31:40786) 2023-11-26T07:20:58.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a92ffd640 1 -- 172.21.15.31:0/2340823259 learned_addr learned my addr 172.21.15.31:0/2340823259 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:58.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a93fff640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a940ce5d0 0x7f8a94153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a937fe640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 0x7f8a941fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a92ffd640 1 -- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 msgr2=0x7f8a941fae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:58.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a92ffd640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 0x7f8a941fae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a92ffd640 1 -- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a940ce5d0 msgr2=0x7f8a94153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:58.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a92ffd640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a940ce5d0 0x7f8a94153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:58.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a92ffd640 1 -- 172.21.15.31:0/2340823259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a88009420 con 0x7f8a940cbd90 2023-11-26T07:20:58.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a937fe640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 0x7f8a941fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:58.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a93fff640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a940ce5d0 0x7f8a94153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:58.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a92ffd640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 0x7f8a940afe70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8a880ad040 tx=0x7f8a8801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:58.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a7bfff640 1 -- 172.21.15.31:0/2340823259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a880bf610 con 0x7f8a940cbd90 2023-11-26T07:20:58.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a7bfff640 1 -- 172.21.15.31:0/2340823259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a880bf790 con 0x7f8a940cbd90 2023-11-26T07:20:58.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a7bfff640 1 -- 172.21.15.31:0/2340823259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a880bfb40 con 0x7f8a940cbd90 2023-11-26T07:20:58.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/2340823259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a941fa650 con 0x7f8a940cbd90 2023-11-26T07:20:58.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.566+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/2340823259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a941fb350 con 0x7f8a940cbd90 2023-11-26T07:20:58.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.570+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/2340823259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a940671f0 con 0x7f8a940cbd90 2023-11-26T07:20:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.570+0000 7f8a7bfff640 1 -- 172.21.15.31:0/2340823259 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f8a880b5400 con 0x7f8a940cbd90 2023-11-26T07:20:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.570+0000 7f8a7bfff640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a80066620 0x7f8a80068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.570+0000 7f8a7bfff640 1 -- 172.21.15.31:0/2340823259 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a8811ed00 con 0x7f8a940cbd90 2023-11-26T07:20:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.570+0000 7f8a937fe640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a80066620 0x7f8a80068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:58.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.570+0000 7f8a937fe640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a80066620 0x7f8a80068ae0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8a940ae230 tx=0x7f8a84008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:58.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.574+0000 7f8a7bfff640 1 -- 172.21.15.31:0/2340823259 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a880ec310 con 0x7f8a940cbd90 2023-11-26T07:20:58.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:58.742+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/2340823259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8a941fa4b0 con 0x7f8a940cbd90 2023-11-26T07:20:59.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a7bfff640 1 -- 172.21.15.31:0/2340823259 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8a941fa4b0 con 0x7f8a940cbd90 2023-11-26T07:20:59.319 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-11-26T07:20:59.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a80066620 msgr2=0x7f8a80068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a80066620 0x7f8a80068ae0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8a940ae230 tx=0x7f8a84008040 comp rx=0 tx=0).stop 2023-11-26T07:20:59.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/2340823259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 msgr2=0x7f8a940afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 0x7f8a940afe70 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8a880ad040 tx=0x7f8a8801ac90 comp rx=0 tx=0).stop 2023-11-26T07:20:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/2340823259 shutdown_connections 2023-11-26T07:20:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8a80066620 0x7f8a80068ae0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a940ce5d0 0x7f8a94153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a940cbd90 0x7f8a940afe70 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 --2- 172.21.15.31:0/2340823259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8a940cb9b0 0x7f8a941fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/2340823259 >> 172.21.15.31:0/2340823259 conn(0x7f8a940b0cb0 msgr2=0x7f8a941581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/2340823259 shutdown_connections 2023-11-26T07:20:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.318+0000 7f8a9a0c9640 1 -- 172.21.15.31:0/2340823259 wait complete. 2023-11-26T07:20:59.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:20:59.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9fa1fdf640 1 Processor -- start 2023-11-26T07:20:59.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9fa1fdf640 1 -- start start 2023-11-26T07:20:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9fa1fdf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f9c146c60 0x7f9f9c070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9fa1fdf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9f9c147040 0x7f9f9c070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9fa1fdf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f9c06bb10 0x7f9f9c06bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9fa1fdf640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c06d100 con 0x7f9f9c146c60 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9fa1fdf640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c15dba0 con 0x7f9f9c147040 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9fa1fdf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c06c860 con 0x7f9f9c06bb10 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9f9b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f9c146c60 0x7f9f9c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9f9b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f9c146c60 0x7f9f9c070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40790/0 (socket says 172.21.15.31:40790) 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9f9b7fe640 1 -- 172.21.15.31:0/3219177661 learned_addr learned my addr 172.21.15.31:0/3219177661 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9f9affd640 1 --2- 172.21.15.31:0/3219177661 >> v2:172.21.15.97:3300/0 conn(0x7f9f9c147040 0x7f9f9c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9f9bfff640 1 --2- 172.21.15.31:0/3219177661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f9c06bb10 0x7f9f9c06bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9f9b7fe640 1 -- 172.21.15.31:0/3219177661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f9c06bb10 msgr2=0x7f9f9c06bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9f9b7fe640 1 --2- 172.21.15.31:0/3219177661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f9c06bb10 0x7f9f9c06bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9f9b7fe640 1 -- 172.21.15.31:0/3219177661 >> v2:172.21.15.97:3300/0 conn(0x7f9f9c147040 msgr2=0x7f9f9c070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9f9b7fe640 1 --2- 172.21.15.31:0/3219177661 >> v2:172.21.15.97:3300/0 conn(0x7f9f9c147040 0x7f9f9c070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.486+0000 7f9f9b7fe640 1 -- 172.21.15.31:0/3219177661 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c06cb40 con 0x7f9f9c146c60 2023-11-26T07:20:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9f9b7fe640 1 --2- 172.21.15.31:0/3219177661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c146c60 0x7f9f9c070020 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c06e1f0 tx=0x7f9f8c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e79627d8c8e924 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9f9a7fc640 1 -- 172.21.15.31:0/3219177661 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0aa5f0 con 0x7f9f9c146c60 2023-11-26T07:20:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9f9a7fc640 1 -- 172.21.15.31:0/3219177661 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0aa750 con 0x7f9f9c146c60 2023-11-26T07:20:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9f9a7fc640 1 -- 172.21.15.31:0/3219177661 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0aaae0 con 0x7f9f9c146c60 2023-11-26T07:20:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3219177661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c146c60 msgr2=0x7f9f9c070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9fa1fdf640 1 --2- 172.21.15.31:0/3219177661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c146c60 0x7f9f9c070020 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c06e1f0 tx=0x7f9f8c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3219177661 shutdown_connections 2023-11-26T07:20:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9fa1fdf640 1 --2- 172.21.15.31:0/3219177661 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f9c06bb10 0x7f9f9c06bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9fa1fdf640 1 --2- 172.21.15.31:0/3219177661 >> v2:172.21.15.97:3300/0 conn(0x7f9f9c147040 0x7f9f9c070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9fa1fdf640 1 --2- 172.21.15.31:0/3219177661 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c146c60 0x7f9f9c070020 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3219177661 >> 172.21.15.31:0/3219177661 conn(0x7f9f9c0b2ea0 msgr2=0x7f9f9c0b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3219177661 shutdown_connections 2023-11-26T07:20:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.490+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3219177661 wait complete. 2023-11-26T07:20:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 Processor -- start 2023-11-26T07:20:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 -- start start 2023-11-26T07:20:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c06bb10 0x7f9f9c13c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f9c146c60 0x7f9f9c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f9c147040 0x7f9f9c13ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c13e980 con 0x7f9f9c06bb10 2023-11-26T07:20:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c13c8f0 con 0x7f9f9c147040 2023-11-26T07:20:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c13ce00 con 0x7f9f9c146c60 2023-11-26T07:20:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c06bb10 0x7f9f9c13c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c06bb10 0x7f9f9c13c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40794/0 (socket says 172.21.15.31:40794) 2023-11-26T07:20:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9b7fe640 1 -- 172.21.15.31:0/3073693144 learned_addr learned my addr 172.21.15.31:0/3073693144 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9bfff640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f9c147040 0x7f9f9c13ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9affd640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f9c146c60 0x7f9f9c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9b7fe640 1 -- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f9c146c60 msgr2=0x7f9f9c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9b7fe640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f9c146c60 0x7f9f9c15c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9b7fe640 1 -- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f9c147040 msgr2=0x7f9f9c13ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9b7fe640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f9c147040 0x7f9f9c13ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9b7fe640 1 -- 172.21.15.31:0/3073693144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0094d0 con 0x7f9f9c06bb10 2023-11-26T07:20:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9bfff640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f9c147040 0x7f9f9c13ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:20:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f9b7fe640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c06bb10 0x7f9f9c13c3b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0ac040 tx=0x7f9f8c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f83fff640 1 -- 172.21.15.31:0/3073693144 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0b3800 con 0x7f9f9c06bb10 2023-11-26T07:20:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f83fff640 1 -- 172.21.15.31:0/3073693144 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0b3960 con 0x7f9f9c06bb10 2023-11-26T07:20:59.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9f83fff640 1 -- 172.21.15.31:0/3073693144 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0b3cf0 con 0x7f9f9c06bb10 2023-11-26T07:20:59.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3073693144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c13dd70 con 0x7f9f9c06bb10 2023-11-26T07:20:59.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3073693144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c13e330 con 0x7f9f9c06bb10 2023-11-26T07:20:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.494+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3073693144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c04e850 con 0x7f9f9c06bb10 2023-11-26T07:20:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.498+0000 7f9f83fff640 1 -- 172.21.15.31:0/3073693144 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f9f8c0b2380 con 0x7f9f9c06bb10 2023-11-26T07:20:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.498+0000 7f9f83fff640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9f88066620 0x7f9f88068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.498+0000 7f9f83fff640 1 -- 172.21.15.31:0/3073693144 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9f8c0a7070 con 0x7f9f9c06bb10 2023-11-26T07:20:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.498+0000 7f9f9affd640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9f88066620 0x7f9f88068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.498+0000 7f9f9affd640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9f88066620 0x7f9f88068ae0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c142120 tx=0x7f9f90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.502+0000 7f9f83fff640 1 -- 172.21.15.31:0/3073693144 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9f8c0eba50 con 0x7f9f9c06bb10 2023-11-26T07:20:59.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.718+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3073693144 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9f9c0d1f60 con 0x7f9f9c06bb10 2023-11-26T07:20:59.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.718+0000 7f9f83fff640 1 -- 172.21.15.31:0/3073693144 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9f8c0c3090 con 0x7f9f9c06bb10 2023-11-26T07:20:59.721 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:20:59.722 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:20:59.722 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-11-26T07:20:59.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.722 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-11-26T07:20:59.722 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlRJmWGBAAuUAS68YFt9PaqDV8RFi+6w==", 2023-11-26T07:20:59.722 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.722 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:20:59.722 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:20:59.722 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:20:59.722 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlIdH1JBAAmPlAQXq78ht4pJf6I7Cd5w==", 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlfuXoHxAAf7vKZr+LVZaN6HR3qiks5w==", 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.724 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlWmgkLBAAojAzX06O7kbwmj154qUGog==", 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDU8GJlc/bqEBAAueOQvSeK6MW+WOT746yQzA==", 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.725 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDX8GJlEaW6FxAA8q4rEKSJtIl/Hr2uCNSMUg==", 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:20:59.726 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDa8GJldV90HxAAQRgcLf8ABSU4Sfb3A1Xa4g==", 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDd8GJl2erPJhAAoC71eDwCOmskb+VCLYQyUw==", 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDj8GJllq08LxAAGuz9+Ou/ybWxdkN3PSMaEQ==", 2023-11-26T07:20:59.728 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDm8GJlww4eNxAAjRaBq45nrNwJPlopyOhXNg==", 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:20:59.729 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDq8GJl2jxiAhAAhu2OULy84YRJYxTmShoVBg==", 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDt8GJld8HNCBAA3deA4HKTqRggbVc73Y8efg==", 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA==", 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.731 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlEBAQOhAAx5TsMN17L2fNm+j3yqUIOQ==", 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:20:59.732 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl/CN+NhAAeyry8gIOtxwLxCQb0t1eKw==", 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.734 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-11-26T07:20:59.734 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDR8GJlA9e7BBAATe3WYA79PgfLc5UHgKGkpw==", 2023-11-26T07:20:59.734 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:20:59.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:20:59.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:20:59.734 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:20:59.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-11-26T07:20:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDP8GJlfT+4NhAACA0t23bQtWvdFLsB9X+X8w==", 2023-11-26T07:20:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:20:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:20:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:20:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-11-26T07:20:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlnh/pExAA6qyPM7So2BLbxbGbquLZZA==", 2023-11-26T07:20:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.737 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:20:59.737 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:20:59.737 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:20:59.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:20:59.737 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-11-26T07:20:59.738 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlYhb/DxAAx1G3Ve23IdZ7cX9nPb2WhA==", 2023-11-26T07:20:59.738 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:20:59.738 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:20:59.738 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:20:59.738 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:20:59.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:20:59.739 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:20:59.739 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:20:59.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9f88066620 msgr2=0x7f9f88068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9f88066620 0x7f9f88068ae0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c142120 tx=0x7f9f90008040 comp rx=0 tx=0).stop 2023-11-26T07:20:59.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3073693144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c06bb10 msgr2=0x7f9f9c13c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c06bb10 0x7f9f9c13c3b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c0ac040 tx=0x7f9f8c01ad10 comp rx=0 tx=0).stop 2023-11-26T07:20:59.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3073693144 shutdown_connections 2023-11-26T07:20:59.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9f88066620 0x7f9f88068ae0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f9c147040 0x7f9f9c13ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9f9c146c60 0x7f9f9c15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 --2- 172.21.15.31:0/3073693144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f9c06bb10 0x7f9f9c13c3b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3073693144 >> 172.21.15.31:0/3073693144 conn(0x7f9f9c0b2ea0 msgr2=0x7f9f9c0d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3073693144 shutdown_connections 2023-11-26T07:20:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.722+0000 7f9fa1fdf640 1 -- 172.21.15.31:0/3073693144 wait complete. 2023-11-26T07:20:59.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:20:59.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9bae06640 1 Processor -- start 2023-11-26T07:20:59.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9bae06640 1 -- start start 2023-11-26T07:20:59.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9bae06640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9b415d880 0x7fe9b40b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9bae06640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe9b40b6c70 0x7fe9b40d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9bae06640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b40d15f0 con 0x7fe9b40d3ca0 2023-11-26T07:20:59.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9bae06640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b40d40d0 con 0x7fe9b415d880 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9bae06640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b40ce000 con 0x7fe9b40b6c70 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9ba605640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe9b40d3ca0 0x7fe9b40cfa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38998/0 (socket says 172.21.15.31:38998) 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9ba605640 1 -- 172.21.15.31:0/760265242 learned_addr learned my addr 172.21.15.31:0/760265242 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9b9e04640 1 --2- 172.21.15.31:0/760265242 >> v2:172.21.15.31:3300/0 conn(0x7fe9b415d880 0x7fe9b40b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9b9603640 1 --2- 172.21.15.31:0/760265242 >> v2:172.21.15.97:3300/0 conn(0x7fe9b40b6c70 0x7fe9b40d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9b8e02640 1 -- 172.21.15.31:0/760265242 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644038165 0 0) 0x7fe9b40d15f0 con 0x7fe9b40d3ca0 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9b9e04640 1 -- 172.21.15.31:0/760265242 >> v2:172.21.15.97:3300/0 conn(0x7fe9b40b6c70 msgr2=0x7fe9b40d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9b9e04640 1 --2- 172.21.15.31:0/760265242 >> v2:172.21.15.97:3300/0 conn(0x7fe9b40b6c70 0x7fe9b40d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9b9e04640 1 -- 172.21.15.31:0/760265242 >> v1:172.21.15.31:6789/0 conn(0x7fe9b40d3ca0 legacy=0x7fe9b40cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9b9e04640 1 -- 172.21.15.31:0/760265242 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b40d0630 con 0x7fe9b415d880 2023-11-26T07:20:59.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.886+0000 7fe9b9603640 1 --2- 172.21.15.31:0/760265242 >> v2:172.21.15.97:3300/0 conn(0x7fe9b40b6c70 0x7fe9b40d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:59.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9b9e04640 1 --2- 172.21.15.31:0/760265242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b415d880 0x7fe9b40b6730 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40cd850 tx=0x7fe9b001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17f39e269696cd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:59.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9b8e02640 1 -- 172.21.15.31:0/760265242 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00ab660 con 0x7fe9b415d880 2023-11-26T07:20:59.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9b8e02640 1 -- 172.21.15.31:0/760265242 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b00ab7c0 con 0x7fe9b415d880 2023-11-26T07:20:59.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9b8e02640 1 -- 172.21.15.31:0/760265242 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00abb50 con 0x7fe9b415d880 2023-11-26T07:20:59.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9bae06640 1 -- 172.21.15.31:0/760265242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b415d880 msgr2=0x7fe9b40b6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9bae06640 1 --2- 172.21.15.31:0/760265242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b415d880 0x7fe9b40b6730 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40cd850 tx=0x7fe9b001c460 comp rx=0 tx=0).stop 2023-11-26T07:20:59.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9bae06640 1 -- 172.21.15.31:0/760265242 shutdown_connections 2023-11-26T07:20:59.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9bae06640 1 --2- 172.21.15.31:0/760265242 >> v2:172.21.15.97:3300/0 conn(0x7fe9b40b6c70 0x7fe9b40d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9bae06640 1 --2- 172.21.15.31:0/760265242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b415d880 0x7fe9b40b6730 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9bae06640 1 -- 172.21.15.31:0/760265242 >> 172.21.15.31:0/760265242 conn(0x7fe9b406b9f0 msgr2=0x7fe9b406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:20:59.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9bae06640 1 -- 172.21.15.31:0/760265242 shutdown_connections 2023-11-26T07:20:59.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.890+0000 7fe9bae06640 1 -- 172.21.15.31:0/760265242 wait complete. 2023-11-26T07:20:59.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 Processor -- start 2023-11-26T07:20:59.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 -- start start 2023-11-26T07:20:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b40b6c70 0x7fe9b4157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40d3ca0 0x7fe9b40c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b415d880 0x7fe9b4162370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b41578b0 con 0x7fe9b40b6c70 2023-11-26T07:20:59.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40ccc60 con 0x7fe9b40d3ca0 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40b3ca0 con 0x7fe9b415d880 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9e04640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b40b6c70 0x7fe9b4157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9e04640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b40b6c70 0x7fe9b4157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40810/0 (socket says 172.21.15.31:40810) 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9e04640 1 -- 172.21.15.31:0/1902518364 learned_addr learned my addr 172.21.15.31:0/1902518364 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9603640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40d3ca0 0x7fe9b40c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9e04640 1 -- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b415d880 msgr2=0x7fe9b4162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9ba605640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b415d880 0x7fe9b4162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9e04640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b415d880 0x7fe9b4162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9e04640 1 -- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40d3ca0 msgr2=0x7fe9b40c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9e04640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40d3ca0 0x7fe9b40c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9e04640 1 -- 172.21.15.31:0/1902518364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b0009420 con 0x7fe9b40b6c70 2023-11-26T07:20:59.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9603640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40d3ca0 0x7fe9b40c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9ba605640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b415d880 0x7fe9b4162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:20:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9b9e04640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b40b6c70 0x7fe9b4157370 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00ad040 tx=0x7fe9b001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1902518364 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00b5810 con 0x7fe9b40b6c70 2023-11-26T07:20:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1902518364 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b00b5990 con 0x7fe9b40b6c70 2023-11-26T07:20:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 -- 172.21.15.31:0/1902518364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b40b3f40 con 0x7fe9b40b6c70 2023-11-26T07:20:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1902518364 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00b5d40 con 0x7fe9b40b6c70 2023-11-26T07:20:59.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 -- 172.21.15.31:0/1902518364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b40b4500 con 0x7fe9b40b6c70 2023-11-26T07:20:59.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.894+0000 7fe9bae06640 1 -- 172.21.15.31:0/1902518364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b4074a00 con 0x7fe9b40b6c70 2023-11-26T07:20:59.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.898+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1902518364 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fe9b00a9070 con 0x7fe9b40b6c70 2023-11-26T07:20:59.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.898+0000 7fe9aa7fc640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe998066620 0x7fe998068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:20:59.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.898+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1902518364 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe9b011e4f0 con 0x7fe9b40b6c70 2023-11-26T07:20:59.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.898+0000 7fe9b9603640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe998066620 0x7fe998068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:20:59.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.898+0000 7fe9b9603640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe998066620 0x7fe998068ae0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe9b41683b0 tx=0x7fe9a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:20:59.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:20:59.902+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1902518364 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe9b00eba40 con 0x7fe9b40b6c70 2023-11-26T07:21:00.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.058+0000 7fe9bae06640 1 -- 172.21.15.31:0/1902518364 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9b40cd600 con 0x7fe9b40b6c70 2023-11-26T07:21:00.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.058+0000 7fe9aa7fc640 1 -- 172.21.15.31:0/1902518364 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12773 (secure 0 0 0) 0x7fe9b00ef060 con 0x7fe9b40b6c70 2023-11-26T07:21:00.063 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:00.063 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":53,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:20:59.329904+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:00.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 -- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe998066620 msgr2=0x7fe998068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe998066620 0x7fe998068ae0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe9b41683b0 tx=0x7fe9a4008040 comp rx=0 tx=0).stop 2023-11-26T07:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 -- 172.21.15.31:0/1902518364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b40b6c70 msgr2=0x7fe9b4157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b40b6c70 0x7fe9b4157370 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00ad040 tx=0x7fe9b001aca0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 -- 172.21.15.31:0/1902518364 shutdown_connections 2023-11-26T07:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe998066620 0x7fe998068ae0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe9b415d880 0x7fe9b4162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe9b40d3ca0 0x7fe9b40c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 --2- 172.21.15.31:0/1902518364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b40b6c70 0x7fe9b4157370 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 -- 172.21.15.31:0/1902518364 >> 172.21.15.31:0/1902518364 conn(0x7fe9b406b9f0 msgr2=0x7fe9b415dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 -- 172.21.15.31:0/1902518364 shutdown_connections 2023-11-26T07:21:00.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.062+0000 7fe9bae06640 1 -- 172.21.15.31:0/1902518364 wait complete. 2023-11-26T07:21:00.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:21:00.084 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:21:00.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c34e3640 1 Processor -- start 2023-11-26T07:21:00.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c34e3640 1 -- start start 2023-11-26T07:21:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c34e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53bc0cdfb0 0x7f53bc0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c34e3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f53bc15a710 0x7f53bc0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c34e3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53bc0cbd50 0x7f53bc0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c34e3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53bc0cca80 con 0x7f53bc0cbd50 2023-11-26T07:21:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c34e3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53bc0ce8d0 con 0x7f53bc15a710 2023-11-26T07:21:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c34e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0b2710 con 0x7f53bc0cdfb0 2023-11-26T07:21:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c1a80640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53bc0cbd50 0x7f53bc0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c1a80640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53bc0cbd50 0x7f53bc0cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40822/0 (socket says 172.21.15.31:40822) 2023-11-26T07:21:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c1a80640 1 -- 172.21.15.31:0/110865322 learned_addr learned my addr 172.21.15.31:0/110865322 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c0a7e640 1 --2- 172.21.15.31:0/110865322 >> v2:172.21.15.97:3300/0 conn(0x7f53bc15a710 0x7f53bc0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c127f640 1 --2- 172.21.15.31:0/110865322 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53bc0cdfb0 0x7f53bc0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c1a80640 1 -- 172.21.15.31:0/110865322 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53bc0cdfb0 msgr2=0x7f53bc0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c1a80640 1 --2- 172.21.15.31:0/110865322 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53bc0cdfb0 0x7f53bc0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c1a80640 1 -- 172.21.15.31:0/110865322 >> v2:172.21.15.97:3300/0 conn(0x7f53bc15a710 msgr2=0x7f53bc0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:00.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c1a80640 1 --2- 172.21.15.31:0/110865322 >> v2:172.21.15.97:3300/0 conn(0x7f53bc15a710 0x7f53bc0ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.230+0000 7f53c1a80640 1 -- 172.21.15.31:0/110865322 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc0b29f0 con 0x7f53bc0cbd50 2023-11-26T07:21:00.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53c1a80640 1 --2- 172.21.15.31:0/110865322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc0cbd50 0x7f53bc0cc1e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f53bc074340 tx=0x7f53b401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f35e5acfa1e4973 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53b3fff640 1 -- 172.21.15.31:0/110865322 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b40a5820 con 0x7f53bc0cbd50 2023-11-26T07:21:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53b3fff640 1 -- 172.21.15.31:0/110865322 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b40a5980 con 0x7f53bc0cbd50 2023-11-26T07:21:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53b3fff640 1 -- 172.21.15.31:0/110865322 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b40a5d10 con 0x7f53bc0cbd50 2023-11-26T07:21:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53c34e3640 1 -- 172.21.15.31:0/110865322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc0cbd50 msgr2=0x7f53bc0cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53c34e3640 1 --2- 172.21.15.31:0/110865322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc0cbd50 0x7f53bc0cc1e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f53bc074340 tx=0x7f53b401c9a0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53c34e3640 1 -- 172.21.15.31:0/110865322 shutdown_connections 2023-11-26T07:21:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53c34e3640 1 --2- 172.21.15.31:0/110865322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc0cbd50 0x7f53bc0cc1e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53c34e3640 1 --2- 172.21.15.31:0/110865322 >> v2:172.21.15.97:3300/0 conn(0x7f53bc15a710 0x7f53bc0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53c34e3640 1 --2- 172.21.15.31:0/110865322 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53bc0cdfb0 0x7f53bc0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53c34e3640 1 -- 172.21.15.31:0/110865322 >> 172.21.15.31:0/110865322 conn(0x7f53bc0b0cb0 msgr2=0x7f53bc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53c34e3640 1 -- 172.21.15.31:0/110865322 shutdown_connections 2023-11-26T07:21:00.239 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.234+0000 7f53c34e3640 1 -- 172.21.15.31:0/110865322 wait complete. 2023-11-26T07:21:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c34e3640 1 Processor -- start 2023-11-26T07:21:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c34e3640 1 -- start start 2023-11-26T07:21:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c34e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc0cbd50 0x7f53bc14fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c34e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53bc0cdfb0 0x7f53bc1590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c34e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc15a710 0x7f53bc163880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c34e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc15ab40 con 0x7f53bc15a710 2023-11-26T07:21:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c34e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc153a10 con 0x7f53bc0cbd50 2023-11-26T07:21:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c34e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc15ad90 con 0x7f53bc0cdfb0 2023-11-26T07:21:00.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c1a80640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc15a710 0x7f53bc163880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c1a80640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc15a710 0x7f53bc163880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40832/0 (socket says 172.21.15.31:40832) 2023-11-26T07:21:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c1a80640 1 -- 172.21.15.31:0/3852613345 learned_addr learned my addr 172.21.15.31:0/3852613345 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c127f640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc0cbd50 0x7f53bc14fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c0a7e640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53bc0cdfb0 0x7f53bc1590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c1a80640 1 -- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53bc0cdfb0 msgr2=0x7f53bc1590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c1a80640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53bc0cdfb0 0x7f53bc1590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c1a80640 1 -- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc0cbd50 msgr2=0x7f53bc14fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c1a80640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc0cbd50 0x7f53bc14fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c1a80640 1 -- 172.21.15.31:0/3852613345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4009b30 con 0x7f53bc15a710 2023-11-26T07:21:00.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c127f640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc0cbd50 0x7f53bc14fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:00.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c1a80640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc15a710 0x7f53bc163880 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f53b40af040 tx=0x7f53b4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:00.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53b1ffb640 1 -- 172.21.15.31:0/3852613345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b40b8670 con 0x7f53bc15a710 2023-11-26T07:21:00.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53b1ffb640 1 -- 172.21.15.31:0/3852613345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b40b87d0 con 0x7f53bc15a710 2023-11-26T07:21:00.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53b1ffb640 1 -- 172.21.15.31:0/3852613345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b40b8b80 con 0x7f53bc15a710 2023-11-26T07:21:00.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c34e3640 1 -- 172.21.15.31:0/3852613345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc15b070 con 0x7f53bc15a710 2023-11-26T07:21:00.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.238+0000 7f53c34e3640 1 -- 172.21.15.31:0/3852613345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc15b630 con 0x7f53bc15a710 2023-11-26T07:21:00.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.242+0000 7f53c34e3640 1 -- 172.21.15.31:0/3852613345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53bc0671f0 con 0x7f53bc15a710 2023-11-26T07:21:00.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.242+0000 7f53b1ffb640 1 -- 172.21.15.31:0/3852613345 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f53b40b8ce0 con 0x7f53bc15a710 2023-11-26T07:21:00.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.242+0000 7f53b1ffb640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f53a0066650 0x7f53a0068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.242+0000 7f53b1ffb640 1 -- 172.21.15.31:0/3852613345 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f53b411f510 con 0x7f53bc15a710 2023-11-26T07:21:00.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.242+0000 7f53c127f640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f53a0066650 0x7f53a0068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.242+0000 7f53c127f640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f53a0066650 0x7f53a0068b10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009f50 tx=0x7f53b8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:00.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.246+0000 7f53b1ffb640 1 -- 172.21.15.31:0/3852613345 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f53b40ecc50 con 0x7f53bc15a710 2023-11-26T07:21:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.406+0000 7f53c34e3640 1 -- 172.21.15.31:0/3852613345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53bc06b460 con 0x7f53bc15a710 2023-11-26T07:21:00.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.406+0000 7f53b1ffb640 1 -- 172.21.15.31:0/3852613345 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53b40f0270 con 0x7f53bc15a710 2023-11-26T07:21:00.410 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:21:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 -- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f53a0066650 msgr2=0x7f53a0068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f53a0066650 0x7f53a0068b10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009f50 tx=0x7f53b8009190 comp rx=0 tx=0).stop 2023-11-26T07:21:00.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 -- 172.21.15.31:0/3852613345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc15a710 msgr2=0x7f53bc163880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc15a710 0x7f53bc163880 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f53b40af040 tx=0x7f53b4019740 comp rx=0 tx=0).stop 2023-11-26T07:21:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 -- 172.21.15.31:0/3852613345 shutdown_connections 2023-11-26T07:21:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f53a0066650 0x7f53a0068b10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53bc15a710 0x7f53bc163880 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f53bc0cdfb0 0x7f53bc1590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 --2- 172.21.15.31:0/3852613345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53bc0cbd50 0x7f53bc14fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 -- 172.21.15.31:0/3852613345 >> 172.21.15.31:0/3852613345 conn(0x7f53bc0b0cb0 msgr2=0x7f53bc15e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 -- 172.21.15.31:0/3852613345 shutdown_connections 2023-11-26T07:21:00.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.410+0000 7f53c34e3640 1 -- 172.21.15.31:0/3852613345 wait complete. 2023-11-26T07:21:00.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:21:00.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:21:00.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:21:00.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4be4d80640 1 Processor -- start 2023-11-26T07:21:00.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4be4d80640 1 -- start start 2023-11-26T07:21:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4be4d80640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be00cead0 0x7f4be00d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4be4d80640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be00cefd0 con 0x7f4be015d7b0 2023-11-26T07:21:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4be4d80640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be00cf2b0 con 0x7f4be00d1eb0 2023-11-26T07:21:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4be4d80640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00d17e0 con 0x7f4be00cead0 2023-11-26T07:21:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4bde59c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4be015d7b0 0x7f4be00d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39008/0 (socket says 172.21.15.31:39008) 2023-11-26T07:21:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4bde59c640 1 -- 172.21.15.31:0/3958359873 learned_addr learned my addr 172.21.15.31:0/3958359873 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4bd7fff640 1 --2- 172.21.15.31:0/3958359873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be00cead0 0x7f4be00d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4bddd9b640 1 -- 172.21.15.31:0/3958359873 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704969262 0 0) 0x7f4be00cefd0 con 0x7f4be015d7b0 2023-11-26T07:21:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4bddd9b640 1 -- 172.21.15.31:0/3958359873 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bcc003530 con 0x7f4be015d7b0 2023-11-26T07:21:00.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4bddd9b640 1 -- 172.21.15.31:0/3958359873 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430320036 0 0) 0x7f4be00cf2b0 con 0x7f4be00d1eb0 2023-11-26T07:21:00.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4bddd9b640 1 -- 172.21.15.31:0/3958359873 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be00cefd0 con 0x7f4be00d1eb0 2023-11-26T07:21:00.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4bd7fff640 1 -- 172.21.15.31:0/3958359873 >> v1:172.21.15.97:6789/0 conn(0x7f4be00d1eb0 legacy=0x7f4be006bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4bd7fff640 1 -- 172.21.15.31:0/3958359873 >> v1:172.21.15.31:6789/0 conn(0x7f4be015d7b0 legacy=0x7f4be00d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.574+0000 7f4bd7fff640 1 -- 172.21.15.31:0/3958359873 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be006b3f0 con 0x7f4be00cead0 2023-11-26T07:21:00.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4bd7fff640 1 --2- 172.21.15.31:0/3958359873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be00cead0 0x7f4be00d1970 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80097f0 tx=0x7f4bc801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5e228ed2e53d2da server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:00.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4bddd9b640 1 -- 172.21.15.31:0/3958359873 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80ab600 con 0x7f4be00cead0 2023-11-26T07:21:00.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4bddd9b640 1 -- 172.21.15.31:0/3958359873 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc80a7050 con 0x7f4be00cead0 2023-11-26T07:21:00.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4bddd9b640 1 -- 172.21.15.31:0/3958359873 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80ab980 con 0x7f4be00cead0 2023-11-26T07:21:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4be4d80640 1 -- 172.21.15.31:0/3958359873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be00cead0 msgr2=0x7f4be00d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4be4d80640 1 --2- 172.21.15.31:0/3958359873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be00cead0 0x7f4be00d1970 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80097f0 tx=0x7f4bc801c3a0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4be4d80640 1 -- 172.21.15.31:0/3958359873 shutdown_connections 2023-11-26T07:21:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4be4d80640 1 --2- 172.21.15.31:0/3958359873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be00cead0 0x7f4be00d1970 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4be4d80640 1 -- 172.21.15.31:0/3958359873 >> 172.21.15.31:0/3958359873 conn(0x7f4be00d33b0 msgr2=0x7f4be00d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4be4d80640 1 -- 172.21.15.31:0/3958359873 shutdown_connections 2023-11-26T07:21:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.578+0000 7f4be4d80640 1 -- 172.21.15.31:0/3958359873 wait complete. 2023-11-26T07:21:00.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4be4d80640 1 Processor -- start 2023-11-26T07:21:00.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4be4d80640 1 -- start start 2023-11-26T07:21:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4be4d80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4be00cead0 0x7f4be0153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4be4d80640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4be00d1eb0 0x7f4be00cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4be4d80640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be015d7b0 0x7f4be00cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4be4d80640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00cc2f0 con 0x7f4be00d1eb0 2023-11-26T07:21:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4be4d80640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00cc5d0 con 0x7f4be00cead0 2023-11-26T07:21:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4be4d80640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00cc8b0 con 0x7f4be015d7b0 2023-11-26T07:21:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4be00d1eb0 0x7f4be00cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4be00d1eb0 0x7f4be00cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40840/0 (socket says 172.21.15.31:40840) 2023-11-26T07:21:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd7fff640 1 -- 172.21.15.31:0/3204097888 learned_addr learned my addr 172.21.15.31:0/3204097888 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bde59c640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4be00cead0 0x7f4be0153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bded9d640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be015d7b0 0x7f4be00cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd7fff640 1 -- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be015d7b0 msgr2=0x7f4be00cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd7fff640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be015d7b0 0x7f4be00cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd7fff640 1 -- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4be00cead0 msgr2=0x7f4be0153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd7fff640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4be00cead0 0x7f4be0153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd7fff640 1 -- 172.21.15.31:0/3204097888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8009420 con 0x7f4be00d1eb0 2023-11-26T07:21:00.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd7fff640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4be00d1eb0 0x7f4be00cb7a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8018fb0 tx=0x7f4bc8019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:00.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd6ffd640 1 -- 172.21.15.31:0/3204097888 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80b4430 con 0x7f4be00d1eb0 2023-11-26T07:21:00.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd6ffd640 1 -- 172.21.15.31:0/3204097888 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc80b45b0 con 0x7f4be00d1eb0 2023-11-26T07:21:00.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4bd6ffd640 1 -- 172.21.15.31:0/3204097888 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80b4960 con 0x7f4be00d1eb0 2023-11-26T07:21:00.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4be4d80640 1 -- 172.21.15.31:0/3204097888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be00ccb90 con 0x7f4be00d1eb0 2023-11-26T07:21:00.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.582+0000 7f4be4d80640 1 -- 172.21.15.31:0/3204097888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be0154200 con 0x7f4be00d1eb0 2023-11-26T07:21:00.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.586+0000 7f4be4d80640 1 -- 172.21.15.31:0/3204097888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be004e850 con 0x7f4be00d1eb0 2023-11-26T07:21:00.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.586+0000 7f4bd6ffd640 1 -- 172.21.15.31:0/3204097888 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f4bc80b2040 con 0x7f4be00d1eb0 2023-11-26T07:21:00.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.586+0000 7f4bd6ffd640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bc0066620 0x7f4bc0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.586+0000 7f4bd6ffd640 1 -- 172.21.15.31:0/3204097888 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4bc811dcd0 con 0x7f4be00d1eb0 2023-11-26T07:21:00.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.586+0000 7f4bde59c640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bc0066620 0x7f4bc0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.586+0000 7f4bde59c640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bc0066620 0x7f4bc0068ae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0007fd0 tx=0x7f4bd0007d60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:00.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.590+0000 7f4bd6ffd640 1 -- 172.21.15.31:0/3204097888 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4bc80eb120 con 0x7f4be00d1eb0 2023-11-26T07:21:00.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.750+0000 7f4be4d80640 1 -- 172.21.15.31:0/3204097888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4be00cd8e0 con 0x7f4be00d1eb0 2023-11-26T07:21:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4bd6ffd640 1 -- 172.21.15.31:0/3204097888 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4bc80ee740 con 0x7f4be00d1eb0 2023-11-26T07:21:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:00.755 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:21:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-11-26T07:21:00.755 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-11-26T07:21:00.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:00.756 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:21:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 -- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bc0066620 msgr2=0x7f4bc0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bc0066620 0x7f4bc0068ae0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0007fd0 tx=0x7f4bd0007d60 comp rx=0 tx=0).stop 2023-11-26T07:21:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 -- 172.21.15.31:0/3204097888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4be00d1eb0 msgr2=0x7f4be00cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4be00d1eb0 0x7f4be00cb7a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8018fb0 tx=0x7f4bc8019140 comp rx=0 tx=0).stop 2023-11-26T07:21:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 -- 172.21.15.31:0/3204097888 shutdown_connections 2023-11-26T07:21:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4bc0066620 0x7f4bc0068ae0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4be015d7b0 0x7f4be00cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4be00d1eb0 0x7f4be00cb7a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 --2- 172.21.15.31:0/3204097888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4be00cead0 0x7f4be0153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 -- 172.21.15.31:0/3204097888 >> 172.21.15.31:0/3204097888 conn(0x7f4be00d33b0 msgr2=0x7f4be00d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 -- 172.21.15.31:0/3204097888 shutdown_connections 2023-11-26T07:21:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.754+0000 7f4be4d80640 1 -- 172.21.15.31:0/3204097888 wait complete. 2023-11-26T07:21:00.776 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:21:00.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:21:00.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd413f8640 1 Processor -- start 2023-11-26T07:21:00.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd413f8640 1 -- start start 2023-11-26T07:21:00.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd413f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdd3c146820 0x7fdd3c070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd413f8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd3c15eb50 0x7fdd3c0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd413f8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd3c147190 con 0x7fdd3c0b4210 2023-11-26T07:21:00.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd413f8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3c0b4be0 con 0x7fdd3c15eb50 2023-11-26T07:21:00.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd413f8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3c146c00 con 0x7fdd3c146820 2023-11-26T07:21:00.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd3b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd3c15eb50 0x7fdd3c0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd3b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd3c15eb50 0x7fdd3c0b46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40852/0 (socket says 172.21.15.31:40852) 2023-11-26T07:21:00.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd3b7fe640 1 -- 172.21.15.31:0/215928599 learned_addr learned my addr 172.21.15.31:0/215928599 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:00.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd3a7fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdd3c0b4210 0x7fdd3c161180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:39012/0 (socket says 172.21.15.31:39012) 2023-11-26T07:21:00.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd3affd640 1 --2- 172.21.15.31:0/215928599 >> v2:172.21.15.97:3300/0 conn(0x7fdd3c146820 0x7fdd3c070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd3b7fe640 1 -- 172.21.15.31:0/215928599 >> v2:172.21.15.97:3300/0 conn(0x7fdd3c146820 msgr2=0x7fdd3c070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:00.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd3b7fe640 1 --2- 172.21.15.31:0/215928599 >> v2:172.21.15.97:3300/0 conn(0x7fdd3c146820 0x7fdd3c070a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd3b7fe640 1 -- 172.21.15.31:0/215928599 >> v1:172.21.15.31:6789/0 conn(0x7fdd3c0b4210 legacy=0x7fdd3c161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.922+0000 7fdd3b7fe640 1 -- 172.21.15.31:0/215928599 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c06d030 con 0x7fdd3c15eb50 2023-11-26T07:21:00.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd3b7fe640 1 --2- 172.21.15.31:0/215928599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd3c15eb50 0x7fdd3c0b46a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c06e450 tx=0x7fdd3401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee92173838202509 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:00.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd39ffb640 1 -- 172.21.15.31:0/215928599 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd340a5780 con 0x7fdd3c15eb50 2023-11-26T07:21:00.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd39ffb640 1 -- 172.21.15.31:0/215928599 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd340a58e0 con 0x7fdd3c15eb50 2023-11-26T07:21:00.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd39ffb640 1 -- 172.21.15.31:0/215928599 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd340a5c70 con 0x7fdd3c15eb50 2023-11-26T07:21:00.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd413f8640 1 -- 172.21.15.31:0/215928599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd3c15eb50 msgr2=0x7fdd3c0b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd413f8640 1 --2- 172.21.15.31:0/215928599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd3c15eb50 0x7fdd3c0b46a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c06e450 tx=0x7fdd3401c9a0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd413f8640 1 -- 172.21.15.31:0/215928599 shutdown_connections 2023-11-26T07:21:00.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd413f8640 1 --2- 172.21.15.31:0/215928599 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd3c15eb50 0x7fdd3c0b46a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd413f8640 1 --2- 172.21.15.31:0/215928599 >> v2:172.21.15.97:3300/0 conn(0x7fdd3c146820 0x7fdd3c070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd413f8640 1 -- 172.21.15.31:0/215928599 >> 172.21.15.31:0/215928599 conn(0x7fdd3c0b2f10 msgr2=0x7fdd3c0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:00.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd413f8640 1 -- 172.21.15.31:0/215928599 shutdown_connections 2023-11-26T07:21:00.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.926+0000 7fdd413f8640 1 -- 172.21.15.31:0/215928599 wait complete. 2023-11-26T07:21:00.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 Processor -- start 2023-11-26T07:21:00.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 -- start start 2023-11-26T07:21:00.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd3c0b4210 0x7fdd3c13e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd3c146820 0x7fdd3c145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c15eb50 0x7fdd3c15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c164700 con 0x7fdd3c146820 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c13f220 con 0x7fdd3c15eb50 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c13f500 con 0x7fdd3c0b4210 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd3c146820 0x7fdd3c145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c15eb50 0x7fdd3c15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd3c146820 0x7fdd3c145230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40864/0 (socket says 172.21.15.31:40864) 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3affd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd3c0b4210 0x7fdd3c13e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3a7fc640 1 -- 172.21.15.31:0/2281623869 learned_addr learned my addr 172.21.15.31:0/2281623869 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3b7fe640 1 -- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd3c0b4210 msgr2=0x7fdd3c13e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3b7fe640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd3c0b4210 0x7fdd3c13e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3b7fe640 1 -- 172.21.15.31:0/2281623869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd3c146820 msgr2=0x7fdd3c145230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3b7fe640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd3c146820 0x7fdd3c145230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3b7fe640 1 -- 172.21.15.31:0/2281623869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd34009b30 con 0x7fdd3c15eb50 2023-11-26T07:21:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd3b7fe640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c15eb50 0x7fdd3c15c180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdd340ab000 tx=0x7fdd34019740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:00.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd237fe640 1 -- 172.21.15.31:0/2281623869 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34009510 con 0x7fdd3c15eb50 2023-11-26T07:21:00.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd237fe640 1 -- 172.21.15.31:0/2281623869 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd340b8470 con 0x7fdd3c15eb50 2023-11-26T07:21:00.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 -- 172.21.15.31:0/2281623869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3c13f7e0 con 0x7fdd3c15eb50 2023-11-26T07:21:00.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd237fe640 1 -- 172.21.15.31:0/2281623869 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd340b8800 con 0x7fdd3c15eb50 2023-11-26T07:21:00.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 -- 172.21.15.31:0/2281623869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3c13ee10 con 0x7fdd3c15eb50 2023-11-26T07:21:00.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.930+0000 7fdd413f8640 1 -- 172.21.15.31:0/2281623869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3c164870 con 0x7fdd3c15eb50 2023-11-26T07:21:00.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.934+0000 7fdd237fe640 1 -- 172.21.15.31:0/2281623869 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fdd340b8960 con 0x7fdd3c15eb50 2023-11-26T07:21:00.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.934+0000 7fdd237fe640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd28066620 0x7fdd28068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:00.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.934+0000 7fdd237fe640 1 -- 172.21.15.31:0/2281623869 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdd3411efc0 con 0x7fdd3c15eb50 2023-11-26T07:21:00.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.934+0000 7fdd3affd640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd28066620 0x7fdd28068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:00.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.934+0000 7fdd3affd640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd28066620 0x7fdd28068ae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009b60 tx=0x7fdd2c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:00.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:00.938+0000 7fdd237fe640 1 -- 172.21.15.31:0/2281623869 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdd340ec630 con 0x7fdd3c15eb50 2023-11-26T07:21:01.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.094+0000 7fdd413f8640 1 -- 172.21.15.31:0/2281623869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdd3c04e850 con 0x7fdd3c15eb50 2023-11-26T07:21:01.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.326+0000 7fdd237fe640 1 -- 172.21.15.31:0/2281623869 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v54) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdd340efc50 con 0x7fdd3c15eb50 2023-11-26T07:21:01.327 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-11-26T07:21:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 -- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd28066620 msgr2=0x7fdd28068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd28066620 0x7fdd28068ae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009b60 tx=0x7fdd2c009190 comp rx=0 tx=0).stop 2023-11-26T07:21:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 -- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c15eb50 msgr2=0x7fdd3c15c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c15eb50 0x7fdd3c15c180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdd340ab000 tx=0x7fdd34019740 comp rx=0 tx=0).stop 2023-11-26T07:21:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 -- 172.21.15.31:0/2281623869 shutdown_connections 2023-11-26T07:21:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd28066620 0x7fdd28068ae0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:01.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd3c15eb50 0x7fdd3c15c180 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:01.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd3c146820 0x7fdd3c145230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:01.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 --2- 172.21.15.31:0/2281623869 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd3c0b4210 0x7fdd3c13e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:01.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 -- 172.21.15.31:0/2281623869 >> 172.21.15.31:0/2281623869 conn(0x7fdd3c0b2f10 msgr2=0x7fdd3c0b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:01.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 -- 172.21.15.31:0/2281623869 shutdown_connections 2023-11-26T07:21:01.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.330+0000 7fdd413f8640 1 -- 172.21.15.31:0/2281623869 wait complete. 2023-11-26T07:21:01.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:21:01.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.498+0000 7f6e0d77d640 1 Processor -- start 2023-11-26T07:21:01.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e0d77d640 1 -- start start 2023-11-26T07:21:01.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e0d77d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e0815d2a0 0x7f6e080d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e0d77d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6e080d2c20 0x7f6e080b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e0d77d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e080b6250 con 0x7f6e080b5180 2023-11-26T07:21:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e0d77d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e080ce8c0 con 0x7f6e080d2c20 2023-11-26T07:21:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e0d77d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e08161cf0 con 0x7f6e0815d2a0 2023-11-26T07:21:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e0815d2a0 0x7f6e080d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e077fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6e080d2c20 0x7f6e080b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e0cf7c640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6e080b5180 0x7f6e0806b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:49014/0 (socket says 172.21.15.31:49014) 2023-11-26T07:21:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e0cf7c640 1 -- 172.21.15.31:0/1361874190 learned_addr learned my addr 172.21.15.31:0/1361874190 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:01.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e0815d2a0 0x7f6e080d26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:57380/0 (socket says 172.21.15.31:57380) 2023-11-26T07:21:01.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e06ffd640 1 -- 172.21.15.31:0/1361874190 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325102785 0 0) 0x7f6e080b6250 con 0x7f6e080b5180 2023-11-26T07:21:01.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e07fff640 1 -- 172.21.15.31:0/1361874190 >> v2:172.21.15.97:3300/0 conn(0x7f6e080d2c20 msgr2=0x7f6e080b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e07fff640 1 --2- 172.21.15.31:0/1361874190 >> v2:172.21.15.97:3300/0 conn(0x7f6e080d2c20 0x7f6e080b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e07fff640 1 -- 172.21.15.31:0/1361874190 >> v1:172.21.15.97:6789/0 conn(0x7f6e080b5180 legacy=0x7f6e0806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e07fff640 1 -- 172.21.15.31:0/1361874190 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e080b5590 con 0x7f6e0815d2a0 2023-11-26T07:21:01.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.502+0000 7f6e077fe640 1 --2- 172.21.15.31:0/1361874190 >> v2:172.21.15.97:3300/0 conn(0x7f6e080d2c20 0x7f6e080b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:01.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e07fff640 1 --2- 172.21.15.31:0/1361874190 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e0815d2a0 0x7f6e080d26e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6df80097c0 tx=0x7f6df801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f0a733b52306cf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e06ffd640 1 -- 172.21.15.31:0/1361874190 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df80ab600 con 0x7f6e0815d2a0 2023-11-26T07:21:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e06ffd640 1 -- 172.21.15.31:0/1361874190 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df80a7050 con 0x7f6e0815d2a0 2023-11-26T07:21:01.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e06ffd640 1 -- 172.21.15.31:0/1361874190 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df80ab980 con 0x7f6e0815d2a0 2023-11-26T07:21:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e0d77d640 1 -- 172.21.15.31:0/1361874190 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e0815d2a0 msgr2=0x7f6e080d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e0d77d640 1 --2- 172.21.15.31:0/1361874190 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e0815d2a0 0x7f6e080d26e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6df80097c0 tx=0x7f6df801c450 comp rx=0 tx=0).stop 2023-11-26T07:21:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e0d77d640 1 -- 172.21.15.31:0/1361874190 shutdown_connections 2023-11-26T07:21:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e0d77d640 1 --2- 172.21.15.31:0/1361874190 >> v2:172.21.15.97:3300/0 conn(0x7f6e080d2c20 0x7f6e080b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e0d77d640 1 --2- 172.21.15.31:0/1361874190 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e0815d2a0 0x7f6e080d26e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:01.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e0d77d640 1 -- 172.21.15.31:0/1361874190 >> 172.21.15.31:0/1361874190 conn(0x7f6e080668a0 msgr2=0x7f6e080cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e0d77d640 1 -- 172.21.15.31:0/1361874190 shutdown_connections 2023-11-26T07:21:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.506+0000 7f6e0d77d640 1 -- 172.21.15.31:0/1361874190 wait complete. 2023-11-26T07:21:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 Processor -- start 2023-11-26T07:21:01.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 -- start start 2023-11-26T07:21:01.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e080b5180 0x7f6e080c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e080d2c20 0x7f6e08162c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e0815d2a0 0x7f6e0815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e080cc3b0 con 0x7f6e080d2c20 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e080c93c0 con 0x7f6e0815d2a0 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e080cb730 con 0x7f6e080b5180 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e080d2c20 0x7f6e08162c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e077fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e080d2c20 0x7f6e08162c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40876/0 (socket says 172.21.15.31:40876) 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e077fe640 1 -- 172.21.15.31:0/3988379500 learned_addr learned my addr 172.21.15.31:0/3988379500 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e07fff640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e080b5180 0x7f6e080c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0cf7c640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e0815d2a0 0x7f6e0815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:01.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e077fe640 1 -- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e080b5180 msgr2=0x7f6e080c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e077fe640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e080b5180 0x7f6e080c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e077fe640 1 -- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e0815d2a0 msgr2=0x7f6e0815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e077fe640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e0815d2a0 0x7f6e0815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e077fe640 1 -- 172.21.15.31:0/3988379500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc009420 con 0x7f6e080d2c20 2023-11-26T07:21:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e077fe640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e080d2c20 0x7f6e08162c60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00ea00 tx=0x7f6dfc00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e04ff9640 1 -- 172.21.15.31:0/3988379500 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc00ca60 con 0x7f6e080d2c20 2023-11-26T07:21:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e04ff9640 1 -- 172.21.15.31:0/3988379500 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc00cbc0 con 0x7f6e080d2c20 2023-11-26T07:21:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e04ff9640 1 -- 172.21.15.31:0/3988379500 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc013470 con 0x7f6e080d2c20 2023-11-26T07:21:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 -- 172.21.15.31:0/3988379500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df8009420 con 0x7f6e080d2c20 2023-11-26T07:21:01.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 -- 172.21.15.31:0/3988379500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e080cbcf0 con 0x7f6e080d2c20 2023-11-26T07:21:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.510+0000 7f6e0d77d640 1 -- 172.21.15.31:0/3988379500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e08074990 con 0x7f6e080d2c20 2023-11-26T07:21:01.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.514+0000 7f6e04ff9640 1 -- 172.21.15.31:0/3988379500 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f6dfc004020 con 0x7f6e080d2c20 2023-11-26T07:21:01.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.514+0000 7f6e04ff9640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6df4066650 0x7f6df4068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:01.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.514+0000 7f6e04ff9640 1 -- 172.21.15.31:0/3988379500 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6dfc084230 con 0x7f6e080d2c20 2023-11-26T07:21:01.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.514+0000 7f6e07fff640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6df4066650 0x7f6df4068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:01.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.514+0000 7f6e07fff640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6df4066650 0x7f6df4068b10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6df80ad000 tx=0x7f6df8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:01.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.518+0000 7f6e04ff9640 1 -- 172.21.15.31:0/3988379500 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6dfc051680 con 0x7f6e080d2c20 2023-11-26T07:21:01.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:01.674+0000 7f6e0d77d640 1 -- 172.21.15.31:0/3988379500 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6e080c9750 con 0x7f6e080d2c20 2023-11-26T07:21:02.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.322+0000 7f6e04ff9640 1 -- 172.21.15.31:0/3988379500 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v55) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6dfc054ca0 con 0x7f6e080d2c20 2023-11-26T07:21:02.325 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-11-26T07:21:02.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 -- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6df4066650 msgr2=0x7f6df4068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:02.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6df4066650 0x7f6df4068b10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6df80ad000 tx=0x7f6df8008040 comp rx=0 tx=0).stop 2023-11-26T07:21:02.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 -- 172.21.15.31:0/3988379500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e080d2c20 msgr2=0x7f6e08162c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:02.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e080d2c20 0x7f6e08162c60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00ea00 tx=0x7f6dfc00eec0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 -- 172.21.15.31:0/3988379500 shutdown_connections 2023-11-26T07:21:02.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6df4066650 0x7f6df4068b10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e0815d2a0 0x7f6e0815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e080d2c20 0x7f6e08162c60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 --2- 172.21.15.31:0/3988379500 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6e080b5180 0x7f6e080c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 -- 172.21.15.31:0/3988379500 >> 172.21.15.31:0/3988379500 conn(0x7f6e080668a0 msgr2=0x7f6e0815f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:02.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 -- 172.21.15.31:0/3988379500 shutdown_connections 2023-11-26T07:21:02.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.326+0000 7f6e0d77d640 1 -- 172.21.15.31:0/3988379500 wait complete. 2023-11-26T07:21:02.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:21:02.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.510+0000 7fb27ae32640 1 Processor -- start 2023-11-26T07:21:02.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.510+0000 7fb27ae32640 1 -- start start 2023-11-26T07:21:02.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb27ae32640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb274159f00 0x7fb2740ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:02.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb27ae32640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb274070890 0x7fb274070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:02.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb27ae32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb274072950 con 0x7fb2740ae680 2023-11-26T07:21:02.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb27ae32640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27406e2b0 con 0x7fb274070890 2023-11-26T07:21:02.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb27ae32640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27406c6f0 con 0x7fb274159f00 2023-11-26T07:21:02.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb2793cf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb274070890 0x7fb274070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:02.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb2793cf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb274070890 0x7fb274070d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40882/0 (socket says 172.21.15.31:40882) 2023-11-26T07:21:02.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb2793cf640 1 -- 172.21.15.31:0/1936650106 learned_addr learned my addr 172.21.15.31:0/1936650106 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb278bce640 1 --2- 172.21.15.31:0/1936650106 >> v2:172.21.15.97:3300/0 conn(0x7fb274159f00 0x7fb2740ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb2793cf640 1 -- 172.21.15.31:0/1936650106 >> v2:172.21.15.97:3300/0 conn(0x7fb274159f00 msgr2=0x7fb2740ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb2793cf640 1 --2- 172.21.15.31:0/1936650106 >> v2:172.21.15.97:3300/0 conn(0x7fb274159f00 0x7fb2740ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb2793cf640 1 -- 172.21.15.31:0/1936650106 >> v1:172.21.15.97:6789/0 conn(0x7fb2740ae680 legacy=0x7fb2740cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb2793cf640 1 -- 172.21.15.31:0/1936650106 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27406e550 con 0x7fb274070890 2023-11-26T07:21:02.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb278bce640 1 --2- 172.21.15.31:0/1936650106 >> v2:172.21.15.97:3300/0 conn(0x7fb274159f00 0x7fb2740ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:02.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.514+0000 7fb2793cf640 1 --2- 172.21.15.31:0/1936650106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb274070890 0x7fb274070d20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb27406c110 tx=0x7fb26c01ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cfb50f79b302fe server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:02.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb2737fe640 1 -- 172.21.15.31:0/1936650106 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0a7a10 con 0x7fb274070890 2023-11-26T07:21:02.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb2737fe640 1 -- 172.21.15.31:0/1936650106 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c0a7b70 con 0x7fb274070890 2023-11-26T07:21:02.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb2737fe640 1 -- 172.21.15.31:0/1936650106 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0af3f0 con 0x7fb274070890 2023-11-26T07:21:02.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 -- 172.21.15.31:0/1936650106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb274070890 msgr2=0x7fb274070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:02.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 --2- 172.21.15.31:0/1936650106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb274070890 0x7fb274070d20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb27406c110 tx=0x7fb26c01ebc0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 -- 172.21.15.31:0/1936650106 shutdown_connections 2023-11-26T07:21:02.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 --2- 172.21.15.31:0/1936650106 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb274070890 0x7fb274070d20 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 --2- 172.21.15.31:0/1936650106 >> v2:172.21.15.97:3300/0 conn(0x7fb274159f00 0x7fb2740ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 -- 172.21.15.31:0/1936650106 >> 172.21.15.31:0/1936650106 conn(0x7fb2740b0cb0 msgr2=0x7fb2740b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:02.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 -- 172.21.15.31:0/1936650106 shutdown_connections 2023-11-26T07:21:02.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 -- 172.21.15.31:0/1936650106 wait complete. 2023-11-26T07:21:02.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 Processor -- start 2023-11-26T07:21:02.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 -- start start 2023-11-26T07:21:02.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb274070890 0x7fb274156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:02.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2740ae680 0x7fb274155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:02.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb27414caf0 0x7fb27415b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:02.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27415af30 con 0x7fb2740ae680 2023-11-26T07:21:02.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27415a390 con 0x7fb274070890 2023-11-26T07:21:02.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.518+0000 7fb27ae32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27415a600 con 0x7fb27414caf0 2023-11-26T07:21:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb273fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2740ae680 0x7fb274155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb273fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2740ae680 0x7fb274155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40892/0 (socket says 172.21.15.31:40892) 2023-11-26T07:21:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb273fff640 1 -- 172.21.15.31:0/3794865670 learned_addr learned my addr 172.21.15.31:0/3794865670 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb278bce640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb274070890 0x7fb274156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb2793cf640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb27414caf0 0x7fb27415b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb273fff640 1 -- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb27414caf0 msgr2=0x7fb27415b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb273fff640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb27414caf0 0x7fb27415b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb273fff640 1 -- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb274070890 msgr2=0x7fb274156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb273fff640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb274070890 0x7fb274156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:02.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb273fff640 1 -- 172.21.15.31:0/3794865670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c009b30 con 0x7fb2740ae680 2023-11-26T07:21:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb273fff640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2740ae680 0x7fb274155980 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb26000c530 tx=0x7fb26000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb2717fa640 1 -- 172.21.15.31:0/3794865670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb260005220 con 0x7fb2740ae680 2023-11-26T07:21:02.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb2717fa640 1 -- 172.21.15.31:0/3794865670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb260005380 con 0x7fb2740ae680 2023-11-26T07:21:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb2717fa640 1 -- 172.21.15.31:0/3794865670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb260007270 con 0x7fb2740ae680 2023-11-26T07:21:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb27ae32640 1 -- 172.21.15.31:0/3794865670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27415a8e0 con 0x7fb2740ae680 2023-11-26T07:21:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb27ae32640 1 -- 172.21.15.31:0/3794865670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb274155ec0 con 0x7fb2740ae680 2023-11-26T07:21:02.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.522+0000 7fb27ae32640 1 -- 172.21.15.31:0/3794865670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2740671f0 con 0x7fb2740ae680 2023-11-26T07:21:02.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.526+0000 7fb2717fa640 1 -- 172.21.15.31:0/3794865670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fb2600073d0 con 0x7fb2740ae680 2023-11-26T07:21:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.526+0000 7fb2717fa640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb258066620 0x7fb258068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.526+0000 7fb2717fa640 1 -- 172.21.15.31:0/3794865670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fb260084560 con 0x7fb2740ae680 2023-11-26T07:21:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.526+0000 7fb278bce640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb258066620 0x7fb258068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:02.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.526+0000 7fb278bce640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb258066620 0x7fb258068ae0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb2741530d0 tx=0x7fb268008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.530+0000 7fb2717fa640 1 -- 172.21.15.31:0/3794865670 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb2600519e0 con 0x7fb2740ae680 2023-11-26T07:21:02.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:02.718+0000 7fb27ae32640 1 -- 172.21.15.31:0/3794865670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb27415b0e0 con 0x7fb2740ae680 2023-11-26T07:21:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.334+0000 7fb2717fa640 1 -- 172.21.15.31:0/3794865670 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v33) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb27415b0e0 con 0x7fb2740ae680 2023-11-26T07:21:03.339 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 8 and data pool 9 2023-11-26T07:21:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.338+0000 7fb27ae32640 1 -- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb258066620 msgr2=0x7fb258068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.338+0000 7fb27ae32640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb258066620 0x7fb258068ae0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb2741530d0 tx=0x7fb268008040 comp rx=0 tx=0).stop 2023-11-26T07:21:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.338+0000 7fb27ae32640 1 -- 172.21.15.31:0/3794865670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2740ae680 msgr2=0x7fb274155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.338+0000 7fb27ae32640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2740ae680 0x7fb274155980 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb26000c530 tx=0x7fb26000c9f0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.342+0000 7fb27ae32640 1 -- 172.21.15.31:0/3794865670 shutdown_connections 2023-11-26T07:21:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.342+0000 7fb27ae32640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb258066620 0x7fb258068ae0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.342+0000 7fb27ae32640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb27414caf0 0x7fb27415b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.342+0000 7fb27ae32640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2740ae680 0x7fb274155980 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.342+0000 7fb27ae32640 1 --2- 172.21.15.31:0/3794865670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb274070890 0x7fb274156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.342+0000 7fb27ae32640 1 -- 172.21.15.31:0/3794865670 >> 172.21.15.31:0/3794865670 conn(0x7fb2740b0cb0 msgr2=0x7fb2740b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.342+0000 7fb27ae32640 1 -- 172.21.15.31:0/3794865670 shutdown_connections 2023-11-26T07:21:03.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.342+0000 7fb27ae32640 1 -- 172.21.15.31:0/3794865670 wait complete. 2023-11-26T07:21:03.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.534+0000 7f0dc8879640 1 Processor -- start 2023-11-26T07:21:03.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.534+0000 7f0dc8879640 1 -- start start 2023-11-26T07:21:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.534+0000 7f0dc8879640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0dc4143ae0 0x7f0dc40b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.534+0000 7f0dc8879640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 0x7f0dc406e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.534+0000 7f0dc8879640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0dc40adcc0 0x7f0dc40ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.534+0000 7f0dc8879640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc40b2480 con 0x7f0dc40adcc0 2023-11-26T07:21:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.534+0000 7f0dc8879640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc406e950 con 0x7f0dc4143ae0 2023-11-26T07:21:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.534+0000 7f0dc8879640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc4070670 con 0x7f0dc406dfd0 2023-11-26T07:21:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.534+0000 7f0dc2d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0dc40adcc0 0x7f0dc40ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc2d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0dc40adcc0 0x7f0dc40ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40896/0 (socket says 172.21.15.31:40896) 2023-11-26T07:21:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc2d9d640 1 -- 172.21.15.31:0/3253075887 learned_addr learned my addr 172.21.15.31:0/3253075887 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:03.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc1d9b640 1 --2- 172.21.15.31:0/3253075887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 0x7f0dc406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc259c640 1 --2- 172.21.15.31:0/3253075887 >> v2:172.21.15.97:3300/0 conn(0x7f0dc4143ae0 0x7f0dc40b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc2d9d640 1 -- 172.21.15.31:0/3253075887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 msgr2=0x7f0dc406e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc2d9d640 1 --2- 172.21.15.31:0/3253075887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 0x7f0dc406e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc2d9d640 1 -- 172.21.15.31:0/3253075887 >> v2:172.21.15.97:3300/0 conn(0x7f0dc4143ae0 msgr2=0x7f0dc40b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc2d9d640 1 --2- 172.21.15.31:0/3253075887 >> v2:172.21.15.97:3300/0 conn(0x7f0dc4143ae0 0x7f0dc40b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc2d9d640 1 -- 172.21.15.31:0/3253075887 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc4070950 con 0x7f0dc40adcc0 2023-11-26T07:21:03.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc1d9b640 1 --2- 172.21.15.31:0/3253075887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 0x7f0dc406e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:03.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc2d9d640 1 --2- 172.21.15.31:0/3253075887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 0x7f0dc40ae110 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4070080 tx=0x7f0dbc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf7346b9548c98f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:03.542 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc159a640 1 -- 172.21.15.31:0/3253075887 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0a5820 con 0x7f0dc40adcc0 2023-11-26T07:21:03.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc159a640 1 -- 172.21.15.31:0/3253075887 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dbc0a5980 con 0x7f0dc40adcc0 2023-11-26T07:21:03.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.538+0000 7f0dc159a640 1 -- 172.21.15.31:0/3253075887 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0a5d10 con 0x7f0dc40adcc0 2023-11-26T07:21:03.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 -- 172.21.15.31:0/3253075887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 msgr2=0x7f0dc40ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 --2- 172.21.15.31:0/3253075887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 0x7f0dc40ae110 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4070080 tx=0x7f0dbc01c9a0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 -- 172.21.15.31:0/3253075887 shutdown_connections 2023-11-26T07:21:03.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 --2- 172.21.15.31:0/3253075887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 0x7f0dc40ae110 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 --2- 172.21.15.31:0/3253075887 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 0x7f0dc406e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 --2- 172.21.15.31:0/3253075887 >> v2:172.21.15.97:3300/0 conn(0x7f0dc4143ae0 0x7f0dc40b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 -- 172.21.15.31:0/3253075887 >> 172.21.15.31:0/3253075887 conn(0x7f0dc40b0c40 msgr2=0x7f0dc40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:03.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 -- 172.21.15.31:0/3253075887 shutdown_connections 2023-11-26T07:21:03.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 -- 172.21.15.31:0/3253075887 wait complete. 2023-11-26T07:21:03.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 Processor -- start 2023-11-26T07:21:03.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 -- start start 2023-11-26T07:21:03.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 0x7f0dc40aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 0x7f0dc415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dc4143ae0 0x7f0dc4159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc41f5e30 con 0x7f0dc40adcc0 2023-11-26T07:21:03.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc41f6bb0 con 0x7f0dc4143ae0 2023-11-26T07:21:03.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc8879640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc41f60d0 con 0x7f0dc406dfd0 2023-11-26T07:21:03.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 0x7f0dc415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 0x7f0dc415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40904/0 (socket says 172.21.15.31:40904) 2023-11-26T07:21:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc1d9b640 1 -- 172.21.15.31:0/307206118 learned_addr learned my addr 172.21.15.31:0/307206118 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc259c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 0x7f0dc40aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.542+0000 7f0dc2d9d640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dc4143ae0 0x7f0dc4159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0dc1d9b640 1 -- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 msgr2=0x7f0dc40aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0dc1d9b640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 0x7f0dc40aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0dc1d9b640 1 -- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dc4143ae0 msgr2=0x7f0dc4159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0dc1d9b640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dc4143ae0 0x7f0dc4159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0dc1d9b640 1 -- 172.21.15.31:0/307206118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc009b30 con 0x7f0dc40adcc0 2023-11-26T07:21:03.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0dc1d9b640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 0x7f0dc415b1f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0db800d800 tx=0x7f0db800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:03.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0db2ffd640 1 -- 172.21.15.31:0/307206118 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800b6c0 con 0x7f0dc40adcc0 2023-11-26T07:21:03.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0db2ffd640 1 -- 172.21.15.31:0/307206118 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db800b820 con 0x7f0dc40adcc0 2023-11-26T07:21:03.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0db2ffd640 1 -- 172.21.15.31:0/307206118 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800bbd0 con 0x7f0dc40adcc0 2023-11-26T07:21:03.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0dc8879640 1 -- 172.21.15.31:0/307206118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc41f63b0 con 0x7f0dc40adcc0 2023-11-26T07:21:03.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0dc8879640 1 -- 172.21.15.31:0/307206118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc415bb80 con 0x7f0dc40adcc0 2023-11-26T07:21:03.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0dc8879640 1 -- 172.21.15.31:0/307206118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc404e850 con 0x7f0dc40adcc0 2023-11-26T07:21:03.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0db2ffd640 1 -- 172.21.15.31:0/307206118 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f0db800bd30 con 0x7f0dc40adcc0 2023-11-26T07:21:03.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0db2ffd640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0da4066620 0x7f0da4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.546+0000 7f0db2ffd640 1 -- 172.21.15.31:0/307206118 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0db8084610 con 0x7f0dc40adcc0 2023-11-26T07:21:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.550+0000 7f0dc259c640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0da4066620 0x7f0da4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.550+0000 7f0dc259c640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0da4066620 0x7f0da4068ae0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40b3ef0 tx=0x7f0db4009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:03.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.554+0000 7f0db2ffd640 1 -- 172.21.15.31:0/307206118 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0db8051820 con 0x7f0dc40adcc0 2023-11-26T07:21:03.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.710+0000 7f0dc8879640 1 -- 172.21.15.31:0/307206118 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0dc406cc40 con 0x7f0dc40adcc0 2023-11-26T07:21:03.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.710+0000 7f0db2ffd640 1 -- 172.21.15.31:0/307206118 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16494 (secure 0 0 0) 0x7f0db8054e40 con 0x7f0dc40adcc0 2023-11-26T07:21:03.713 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:03.713 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":56,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:02.722229+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:01.142504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:01.723895+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.710+0000 7f0dc8879640 1 -- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0da4066620 msgr2=0x7f0da4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.710+0000 7f0dc8879640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0da4066620 0x7f0da4068ae0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40b3ef0 tx=0x7f0db4009190 comp rx=0 tx=0).stop 2023-11-26T07:21:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.710+0000 7f0dc8879640 1 -- 172.21.15.31:0/307206118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 msgr2=0x7f0dc415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.710+0000 7f0dc8879640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 0x7f0dc415b1f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0db800d800 tx=0x7f0db800dcc0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.714+0000 7f0dc8879640 1 -- 172.21.15.31:0/307206118 shutdown_connections 2023-11-26T07:21:03.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.714+0000 7f0dc8879640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0da4066620 0x7f0da4068ae0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.714+0000 7f0dc8879640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0dc4143ae0 0x7f0dc4159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.714+0000 7f0dc8879640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0dc40adcc0 0x7f0dc415b1f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.714+0000 7f0dc8879640 1 --2- 172.21.15.31:0/307206118 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0dc406dfd0 0x7f0dc40aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.714+0000 7f0dc8879640 1 -- 172.21.15.31:0/307206118 >> 172.21.15.31:0/307206118 conn(0x7f0dc40b0c40 msgr2=0x7f0dc40b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.714+0000 7f0dc8879640 1 -- 172.21.15.31:0/307206118 shutdown_connections 2023-11-26T07:21:03.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.714+0000 7f0dc8879640 1 -- 172.21.15.31:0/307206118 wait complete. 2023-11-26T07:21:03.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:03.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.874+0000 7f93f2ebb640 1 Processor -- start 2023-11-26T07:21:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.874+0000 7f93f2ebb640 1 -- start start 2023-11-26T07:21:03.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.874+0000 7f93f2ebb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93ec06e410 0x7f93ec0715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.874+0000 7f93f2ebb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f93ec071af0 0x7f93ec0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.874+0000 7f93f2ebb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ec0b2ee0 0x7f93ec161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.874+0000 7f93f2ebb640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ec0b3e20 con 0x7f93ec06e410 2023-11-26T07:21:03.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.874+0000 7f93f2ebb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ec071fa0 con 0x7f93ec071af0 2023-11-26T07:21:03.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.874+0000 7f93f2ebb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec15dde0 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93f0c57640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93ec06e410 0x7f93ec0715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93f0c57640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93ec06e410 0x7f93ec0715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34426/0 (socket says 172.21.15.31:34426) 2023-11-26T07:21:03.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93f0c57640 1 -- 172.21.15.31:0/3930369746 learned_addr learned my addr 172.21.15.31:0/3930369746 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:03.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93f1458640 1 --2- 172.21.15.31:0/3930369746 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ec0b2ee0 0x7f93ec161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93ebfff640 1 --2- 172.21.15.31:0/3930369746 >> v2:172.21.15.97:3300/0 conn(0x7f93ec071af0 0x7f93ec0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93f0c57640 1 -- 172.21.15.31:0/3930369746 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ec0b2ee0 msgr2=0x7f93ec161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:03.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93f0c57640 1 --2- 172.21.15.31:0/3930369746 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ec0b2ee0 0x7f93ec161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93f0c57640 1 -- 172.21.15.31:0/3930369746 >> v2:172.21.15.97:3300/0 conn(0x7f93ec071af0 msgr2=0x7f93ec0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:03.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93f0c57640 1 --2- 172.21.15.31:0/3930369746 >> v2:172.21.15.97:3300/0 conn(0x7f93ec071af0 0x7f93ec0b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93f0c57640 1 -- 172.21.15.31:0/3930369746 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec0ad740 con 0x7f93ec06e410 2023-11-26T07:21:03.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.878+0000 7f93f0c57640 1 --2- 172.21.15.31:0/3930369746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec06e410 0x7f93ec0715b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009870 tx=0x7f93e001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ee34c5c5df5c11 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:03.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93eb7fe640 1 -- 172.21.15.31:0/3930369746 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00ab600 con 0x7f93ec06e410 2023-11-26T07:21:03.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93eb7fe640 1 -- 172.21.15.31:0/3930369746 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e00a7050 con 0x7f93ec06e410 2023-11-26T07:21:03.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93eb7fe640 1 -- 172.21.15.31:0/3930369746 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00ab980 con 0x7f93ec06e410 2023-11-26T07:21:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 -- 172.21.15.31:0/3930369746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec06e410 msgr2=0x7f93ec0715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 --2- 172.21.15.31:0/3930369746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec06e410 0x7f93ec0715b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009870 tx=0x7f93e001c240 comp rx=0 tx=0).stop 2023-11-26T07:21:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 -- 172.21.15.31:0/3930369746 shutdown_connections 2023-11-26T07:21:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 --2- 172.21.15.31:0/3930369746 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ec0b2ee0 0x7f93ec161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 --2- 172.21.15.31:0/3930369746 >> v2:172.21.15.97:3300/0 conn(0x7f93ec071af0 0x7f93ec0b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 --2- 172.21.15.31:0/3930369746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec06e410 0x7f93ec0715b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 -- 172.21.15.31:0/3930369746 >> 172.21.15.31:0/3930369746 conn(0x7f93ec0b1e40 msgr2=0x7f93ec070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 -- 172.21.15.31:0/3930369746 shutdown_connections 2023-11-26T07:21:03.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 -- 172.21.15.31:0/3930369746 wait complete. 2023-11-26T07:21:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 Processor -- start 2023-11-26T07:21:03.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 -- start start 2023-11-26T07:21:03.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ec06e410 0x7f93ec14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ec071af0 0x7f93ec0af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec0b2ee0 0x7f93ec0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec0b0c00 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec0afff0 con 0x7f93ec06e410 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f2ebb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec0b02d0 con 0x7f93ec071af0 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f1458640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec0b2ee0 0x7f93ec0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f1458640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec0b2ee0 0x7f93ec0ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34434/0 (socket says 172.21.15.31:34434) 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f1458640 1 -- 172.21.15.31:0/2587024252 learned_addr learned my addr 172.21.15.31:0/2587024252 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93f0c57640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ec06e410 0x7f93ec14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.882+0000 7f93ebfff640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ec071af0 0x7f93ec0af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93f1458640 1 -- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ec071af0 msgr2=0x7f93ec0af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93f1458640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ec071af0 0x7f93ec0af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93f1458640 1 -- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ec06e410 msgr2=0x7f93ec14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93f1458640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ec06e410 0x7f93ec14cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93f1458640 1 -- 172.21.15.31:0/2587024252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e00094d0 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93f1458640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec0b2ee0 0x7f93ec0ce520 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f93e400efc0 tx=0x7f93e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93e97fa640 1 -- 172.21.15.31:0/2587024252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4014480 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93e97fa640 1 -- 172.21.15.31:0/2587024252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e400f040 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93f2ebb640 1 -- 172.21.15.31:0/2587024252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec0b05b0 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93e97fa640 1 -- 172.21.15.31:0/2587024252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4014820 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93f2ebb640 1 -- 172.21.15.31:0/2587024252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec0b1090 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93f2ebb640 1 -- 172.21.15.31:0/2587024252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ec0b0db0 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93e97fa640 1 -- 172.21.15.31:0/2587024252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f93e4014980 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93e97fa640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93d0066620 0x7f93d0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:03.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.886+0000 7f93e97fa640 1 -- 172.21.15.31:0/2587024252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93e4084f40 con 0x7f93ec0b2ee0 2023-11-26T07:21:03.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.890+0000 7f93f0c57640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93d0066620 0x7f93d0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:03.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.890+0000 7f93f0c57640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93d0066620 0x7f93d0068ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f93e00ad000 tx=0x7f93e00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:03.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:03.894+0000 7f93e97fa640 1 -- 172.21.15.31:0/2587024252 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f93ec0b0db0 con 0x7f93ec0b2ee0 2023-11-26T07:21:04.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 -- 172.21.15.31:0/2587024252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93ec04e850 con 0x7f93ec0b2ee0 2023-11-26T07:21:04.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93e97fa640 1 -- 172.21.15.31:0/2587024252 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16494 (secure 0 0 0) 0x7f93e40520e0 con 0x7f93ec0b2ee0 2023-11-26T07:21:04.056 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:04.056 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":56,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:02.722229+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:01.142504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:01.723895+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 -- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93d0066620 msgr2=0x7f93d0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93d0066620 0x7f93d0068ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f93e00ad000 tx=0x7f93e00a6000 comp rx=0 tx=0).stop 2023-11-26T07:21:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 -- 172.21.15.31:0/2587024252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec0b2ee0 msgr2=0x7f93ec0ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:04.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec0b2ee0 0x7f93ec0ce520 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f93e400efc0 tx=0x7f93e400c490 comp rx=0 tx=0).stop 2023-11-26T07:21:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 -- 172.21.15.31:0/2587024252 shutdown_connections 2023-11-26T07:21:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93d0066620 0x7f93d0068ae0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93ec0b2ee0 0x7f93ec0ce520 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f93ec071af0 0x7f93ec0af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 --2- 172.21.15.31:0/2587024252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93ec06e410 0x7f93ec14cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 -- 172.21.15.31:0/2587024252 >> 172.21.15.31:0/2587024252 conn(0x7f93ec0b1e40 msgr2=0x7f93ec0cc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 -- 172.21.15.31:0/2587024252 shutdown_connections 2023-11-26T07:21:04.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.054+0000 7f93f2ebb640 1 -- 172.21.15.31:0/2587024252 wait complete. 2023-11-26T07:21:04.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:21:04.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6b0e16640 1 Processor -- start 2023-11-26T07:21:04.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6b0e16640 1 -- start start 2023-11-26T07:21:04.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6b0e16640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6ac0ced30 0x7ff6ac0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:04.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6b0e16640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6ac0cf1e0 0x7ff6ac0d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:04.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6b0e16640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ac0cdfb0 con 0x7ff6ac15d8f0 2023-11-26T07:21:04.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6b0e16640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ac0d4420 con 0x7ff6ac0ced30 2023-11-26T07:21:04.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6b0e16640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0d4140 con 0x7ff6ac0cf1e0 2023-11-26T07:21:04.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6a3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6ac0ced30 0x7ff6ac0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:04.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6a3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6ac0ced30 0x7ff6ac0cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34450/0 (socket says 172.21.15.31:34450) 2023-11-26T07:21:04.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6a3fff640 1 -- 172.21.15.31:0/988780723 learned_addr learned my addr 172.21.15.31:0/988780723 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:04.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6aad9d640 1 --2- 172.21.15.31:0/988780723 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6ac0cf1e0 0x7ff6ac0d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:04.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6a3fff640 1 -- 172.21.15.31:0/988780723 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6ac0cf1e0 msgr2=0x7ff6ac0d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:04.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6a3fff640 1 --2- 172.21.15.31:0/988780723 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6ac0cf1e0 0x7ff6ac0d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:04.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6a3fff640 1 -- 172.21.15.31:0/988780723 >> v1:172.21.15.97:6789/0 conn(0x7ff6ac15d8f0 legacy=0x7ff6ac15e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:04.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.222+0000 7ff6a3fff640 1 -- 172.21.15.31:0/988780723 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac0ce250 con 0x7ff6ac0ced30 2023-11-26T07:21:04.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6a3fff640 1 --2- 172.21.15.31:0/988780723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 0x7ff6ac0cfe00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff694009870 tx=0x7ff69401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c011a6350d2a7fd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:04.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6a9d9b640 1 -- 172.21.15.31:0/988780723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6940aa670 con 0x7ff6ac0ced30 2023-11-26T07:21:04.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6a9d9b640 1 -- 172.21.15.31:0/988780723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6940aa7d0 con 0x7ff6ac0ced30 2023-11-26T07:21:04.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6a9d9b640 1 -- 172.21.15.31:0/988780723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6940aab60 con 0x7ff6ac0ced30 2023-11-26T07:21:04.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6b0e16640 1 -- 172.21.15.31:0/988780723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 msgr2=0x7ff6ac0cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:04.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6b0e16640 1 --2- 172.21.15.31:0/988780723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 0x7ff6ac0cfe00 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff694009870 tx=0x7ff69401c510 comp rx=0 tx=0).stop 2023-11-26T07:21:04.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6b0e16640 1 -- 172.21.15.31:0/988780723 shutdown_connections 2023-11-26T07:21:04.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6b0e16640 1 --2- 172.21.15.31:0/988780723 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6ac0cf1e0 0x7ff6ac0d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:04.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6b0e16640 1 --2- 172.21.15.31:0/988780723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 0x7ff6ac0cfe00 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:04.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6b0e16640 1 -- 172.21.15.31:0/988780723 >> 172.21.15.31:0/988780723 conn(0x7ff6ac09c750 msgr2=0x7ff6ac0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:04.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6b0e16640 1 -- 172.21.15.31:0/988780723 shutdown_connections 2023-11-26T07:21:04.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.226+0000 7ff6b0e16640 1 -- 172.21.15.31:0/988780723 wait complete. 2023-11-26T07:21:04.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6b0e16640 1 Processor -- start 2023-11-26T07:21:04.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6b0e16640 1 -- start start 2023-11-26T07:21:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6b0e16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 0x7ff6ac1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6b0e16640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6ac0cf1e0 0x7ff6ac0cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6b0e16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6ac15d8f0 0x7ff6ac0cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6b0e16640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0caf10 con 0x7ff6ac0ced30 2023-11-26T07:21:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6b0e16640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0cb740 con 0x7ff6ac15d8f0 2023-11-26T07:21:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6b0e16640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac0a0640 con 0x7ff6ac0cf1e0 2023-11-26T07:21:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6aa59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 0x7ff6ac1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:04.234 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6aa59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 0x7ff6ac1558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34460/0 (socket says 172.21.15.31:34460) 2023-11-26T07:21:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6aa59c640 1 -- 172.21.15.31:0/3110786979 learned_addr learned my addr 172.21.15.31:0/3110786979 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6aad9d640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6ac15d8f0 0x7ff6ac0cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6a3fff640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6ac0cf1e0 0x7ff6ac0cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6aa59c640 1 -- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6ac0cf1e0 msgr2=0x7ff6ac0cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6aa59c640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6ac0cf1e0 0x7ff6ac0cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6aa59c640 1 -- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6ac15d8f0 msgr2=0x7ff6ac0cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6aa59c640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6ac15d8f0 0x7ff6ac0cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6aa59c640 1 -- 172.21.15.31:0/3110786979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6940094d0 con 0x7ff6ac0ced30 2023-11-26T07:21:04.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6aa59c640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 0x7ff6ac1558e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00c500 tx=0x7ff69c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:04.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6a2ffd640 1 -- 172.21.15.31:0/3110786979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c005220 con 0x7ff6ac0ced30 2023-11-26T07:21:04.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6a2ffd640 1 -- 172.21.15.31:0/3110786979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c005380 con 0x7ff6ac0ced30 2023-11-26T07:21:04.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6a2ffd640 1 -- 172.21.15.31:0/3110786979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c007270 con 0x7ff6ac0ced30 2023-11-26T07:21:04.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6b0e16640 1 -- 172.21.15.31:0/3110786979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac0cb0f0 con 0x7ff6ac0ced30 2023-11-26T07:21:04.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.230+0000 7ff6b0e16640 1 -- 172.21.15.31:0/3110786979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac155e20 con 0x7ff6ac0ced30 2023-11-26T07:21:04.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.234+0000 7ff6b0e16640 1 -- 172.21.15.31:0/3110786979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ac04e850 con 0x7ff6ac0ced30 2023-11-26T07:21:04.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.234+0000 7ff6a2ffd640 1 -- 172.21.15.31:0/3110786979 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff69c0073d0 con 0x7ff6ac0ced30 2023-11-26T07:21:04.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.234+0000 7ff6a2ffd640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff68c066650 0x7ff68c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:04.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.234+0000 7ff6a2ffd640 1 -- 172.21.15.31:0/3110786979 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff69c0835f0 con 0x7ff6ac0ced30 2023-11-26T07:21:04.237 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.234+0000 7ff6a3fff640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff68c066650 0x7ff68c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:04.238 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.234+0000 7ff6a3fff640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff68c066650 0x7ff68c068b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff6940ae000 tx=0x7ff6940a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:04.241 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.238+0000 7ff6a2ffd640 1 -- 172.21.15.31:0/3110786979 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff69c050760 con 0x7ff6ac0ced30 2023-11-26T07:21:04.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:04.430+0000 7ff6b0e16640 1 -- 172.21.15.31:0/3110786979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff6ac0cf9d0 con 0x7ff6ac0ced30 2023-11-26T07:21:05.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.370+0000 7ff6a2ffd640 1 -- 172.21.15.31:0/3110786979 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v36) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff69c053d80 con 0x7ff6ac0ced30 2023-11-26T07:21:05.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 -- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff68c066650 msgr2=0x7ff68c068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:05.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff68c066650 0x7ff68c068b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff6940ae000 tx=0x7ff6940a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:05.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 -- 172.21.15.31:0/3110786979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 msgr2=0x7ff6ac1558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:05.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 0x7ff6ac1558e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00c500 tx=0x7ff69c00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:21:05.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 -- 172.21.15.31:0/3110786979 shutdown_connections 2023-11-26T07:21:05.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff68c066650 0x7ff68c068b10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:05.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6ac15d8f0 0x7ff6ac0cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:05.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6ac0cf1e0 0x7ff6ac0cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:05.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 --2- 172.21.15.31:0/3110786979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6ac0ced30 0x7ff6ac1558e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:05.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 -- 172.21.15.31:0/3110786979 >> 172.21.15.31:0/3110786979 conn(0x7ff6ac09c750 msgr2=0x7ff6ac0b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:05.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 -- 172.21.15.31:0/3110786979 shutdown_connections 2023-11-26T07:21:05.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.374+0000 7ff6b0e16640 1 -- 172.21.15.31:0/3110786979 wait complete. 2023-11-26T07:21:05.400 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:21:05.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:21:05.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.546+0000 7f2362254640 1 Processor -- start 2023-11-26T07:21:05.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.546+0000 7f2362254640 1 -- start start 2023-11-26T07:21:05.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f2362254640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cdfb0 0x7f235c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:05.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f2362254640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c0b2710 con 0x7f235c0cbd50 2023-11-26T07:21:05.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f2362254640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c0b29f0 con 0x7f235c15a710 2023-11-26T07:21:05.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f2362254640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0cca80 con 0x7f235c0cdfb0 2023-11-26T07:21:05.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f235bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f235c0cbd50 0x7f235c0cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35492/0 (socket says 172.21.15.31:35492) 2023-11-26T07:21:05.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f235bfff640 1 -- 172.21.15.31:0/3934247286 learned_addr learned my addr 172.21.15.31:0/3934247286 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:05.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f235b7fe640 1 --2- 172.21.15.31:0/3934247286 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cdfb0 0x7f235c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:05.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f235a7fc640 1 -- 172.21.15.31:0/3934247286 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663444458 0 0) 0x7f235c0b2710 con 0x7f235c0cbd50 2023-11-26T07:21:05.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f235a7fc640 1 -- 172.21.15.31:0/3934247286 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2344003530 con 0x7f235c0cbd50 2023-11-26T07:21:05.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f235a7fc640 1 -- 172.21.15.31:0/3934247286 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850763611 0 0) 0x7f235c0b29f0 con 0x7f235c15a710 2023-11-26T07:21:05.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f235a7fc640 1 -- 172.21.15.31:0/3934247286 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f235c0b2710 con 0x7f235c15a710 2023-11-26T07:21:05.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f235b7fe640 1 -- 172.21.15.31:0/3934247286 >> v1:172.21.15.97:6789/0 conn(0x7f235c15a710 legacy=0x7f235c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:05.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f235b7fe640 1 -- 172.21.15.31:0/3934247286 >> v1:172.21.15.31:6789/0 conn(0x7f235c0cbd50 legacy=0x7f235c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:05.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.550+0000 7f235b7fe640 1 -- 172.21.15.31:0/3934247286 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c0b2cd0 con 0x7f235c0cdfb0 2023-11-26T07:21:05.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f235b7fe640 1 --2- 172.21.15.31:0/3934247286 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cdfb0 0x7f235c0abf90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f234c0097f0 tx=0x7f234c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa122abe752c928 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:05.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f235a7fc640 1 -- 172.21.15.31:0/3934247286 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c0aa610 con 0x7f235c0cdfb0 2023-11-26T07:21:05.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f235a7fc640 1 -- 172.21.15.31:0/3934247286 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c0a6040 con 0x7f235c0cdfb0 2023-11-26T07:21:05.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f235a7fc640 1 -- 172.21.15.31:0/3934247286 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c0aa990 con 0x7f235c0cdfb0 2023-11-26T07:21:05.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f2362254640 1 -- 172.21.15.31:0/3934247286 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cdfb0 msgr2=0x7f235c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:05.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f2362254640 1 --2- 172.21.15.31:0/3934247286 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cdfb0 0x7f235c0abf90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f234c0097f0 tx=0x7f234c01c490 comp rx=0 tx=0).stop 2023-11-26T07:21:05.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f2362254640 1 -- 172.21.15.31:0/3934247286 shutdown_connections 2023-11-26T07:21:05.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f2362254640 1 --2- 172.21.15.31:0/3934247286 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cdfb0 0x7f235c0abf90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:05.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f2362254640 1 -- 172.21.15.31:0/3934247286 >> 172.21.15.31:0/3934247286 conn(0x7f235c0b0cb0 msgr2=0x7f235c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:05.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f2362254640 1 -- 172.21.15.31:0/3934247286 shutdown_connections 2023-11-26T07:21:05.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.554+0000 7f2362254640 1 -- 172.21.15.31:0/3934247286 wait complete. 2023-11-26T07:21:05.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 Processor -- start 2023-11-26T07:21:05.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 -- start start 2023-11-26T07:21:05.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cbd50 0x7f235c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:05.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c0cdfb0 0x7f235c0afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f235c15a710 0x7f235c0b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c09a8d0 con 0x7f235c0cdfb0 2023-11-26T07:21:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c09ab70 con 0x7f235c15a710 2023-11-26T07:21:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c09ae50 con 0x7f235c0cbd50 2023-11-26T07:21:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c0cdfb0 0x7f235c0afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c0cdfb0 0x7f235c0afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34462/0 (socket says 172.21.15.31:34462) 2023-11-26T07:21:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235affd640 1 -- 172.21.15.31:0/3338242995 learned_addr learned my addr 172.21.15.31:0/3338242995 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235b7fe640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cbd50 0x7f235c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235bfff640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f235c15a710 0x7f235c0b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235affd640 1 -- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cbd50 msgr2=0x7f235c152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235affd640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cbd50 0x7f235c152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235affd640 1 -- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f235c15a710 msgr2=0x7f235c0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235affd640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f235c15a710 0x7f235c0b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235affd640 1 -- 172.21.15.31:0/3338242995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c009420 con 0x7f235c0cdfb0 2023-11-26T07:21:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235bfff640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f235c15a710 0x7f235c0b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f235affd640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c0cdfb0 0x7f235c0afb30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f235000c430 tx=0x7f235000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2343fff640 1 -- 172.21.15.31:0/3338242995 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23500052d0 con 0x7f235c0cdfb0 2023-11-26T07:21:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2343fff640 1 -- 172.21.15.31:0/3338242995 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2350005450 con 0x7f235c0cdfb0 2023-11-26T07:21:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2343fff640 1 -- 172.21.15.31:0/3338242995 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350003840 con 0x7f235c0cdfb0 2023-11-26T07:21:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 -- 172.21.15.31:0/3338242995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c09b130 con 0x7f235c0cdfb0 2023-11-26T07:21:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 -- 172.21.15.31:0/3338242995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c09b6f0 con 0x7f235c0cdfb0 2023-11-26T07:21:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.558+0000 7f2362254640 1 -- 172.21.15.31:0/3338242995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c0671f0 con 0x7f235c0cdfb0 2023-11-26T07:21:05.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.562+0000 7f2343fff640 1 -- 172.21.15.31:0/3338242995 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f23500039a0 con 0x7f235c0cdfb0 2023-11-26T07:21:05.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.562+0000 7f2343fff640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2348066620 0x7f2348068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:05.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.562+0000 7f2343fff640 1 -- 172.21.15.31:0/3338242995 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23500837c0 con 0x7f235c0cdfb0 2023-11-26T07:21:05.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.562+0000 7f235b7fe640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2348066620 0x7f2348068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:05.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.562+0000 7f235b7fe640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2348066620 0x7f2348068ae0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f234c0ae000 tx=0x7f234c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:05.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.566+0000 7f2343fff640 1 -- 172.21.15.31:0/3338242995 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f23500509b0 con 0x7f235c0cdfb0 2023-11-26T07:21:05.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:05.754+0000 7f2362254640 1 -- 172.21.15.31:0/3338242995 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f235c06b460 con 0x7f235c0cdfb0 2023-11-26T07:21:06.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.378+0000 7f2343fff640 1 -- 172.21.15.31:0/3338242995 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v37) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2350053fd0 con 0x7f235c0cdfb0 2023-11-26T07:21:06.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 -- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2348066620 msgr2=0x7f2348068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2348066620 0x7f2348068ae0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f234c0ae000 tx=0x7f234c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 -- 172.21.15.31:0/3338242995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c0cdfb0 msgr2=0x7f235c0afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c0cdfb0 0x7f235c0afb30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f235000c430 tx=0x7f235000c8f0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 -- 172.21.15.31:0/3338242995 shutdown_connections 2023-11-26T07:21:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2348066620 0x7f2348068ae0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f235c15a710 0x7f235c0b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f235c0cdfb0 0x7f235c0afb30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 --2- 172.21.15.31:0/3338242995 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f235c0cbd50 0x7f235c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 -- 172.21.15.31:0/3338242995 >> 172.21.15.31:0/3338242995 conn(0x7f235c0b0cb0 msgr2=0x7f235c0cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 -- 172.21.15.31:0/3338242995 shutdown_connections 2023-11-26T07:21:06.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.382+0000 7f2362254640 1 -- 172.21.15.31:0/3338242995 wait complete. 2023-11-26T07:21:06.407 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:21:06.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:06.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c58f32640 1 Processor -- start 2023-11-26T07:21:06.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c58f32640 1 -- start start 2023-11-26T07:21:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c58f32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5415d880 0x7f0c540b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c58f32640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c540d15f0 con 0x7f0c540b6c70 2023-11-26T07:21:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c58f32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c540ce000 con 0x7f0c540d3e70 2023-11-26T07:21:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c58f32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540d0630 con 0x7f0c5415d880 2023-11-26T07:21:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c52ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0c540b6c70 0x7f0c540d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35502/0 (socket says 172.21.15.31:35502) 2023-11-26T07:21:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c52ffd640 1 -- 172.21.15.31:0/3878409662 learned_addr learned my addr 172.21.15.31:0/3878409662 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c537fe640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5415d880 0x7f0c540b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449234946 0 0) 0x7f0c540d15f0 con 0x7f0c540b6c70 2023-11-26T07:21:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c3c003530 con 0x7f0c540b6c70 2023-11-26T07:21:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 507448790 0 0) 0x7f0c3c003530 con 0x7f0c540b6c70 2023-11-26T07:21:06.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5415d880 msgr2=0x7f0c540b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5415d880 0x7f0c540b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 >> v1:172.21.15.97:6789/0 conn(0x7f0c540d3e70 legacy=0x7f0c540cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c540d0910 con 0x7f0c540b6c70 2023-11-26T07:21:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f0c48001680 con 0x7f0c540b6c70 2023-11-26T07:21:06.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 >> v1:172.21.15.31:6789/0 conn(0x7f0c540b6c70 legacy=0x7f0c540d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 0x7f0c3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c3c0047d0 0x7f0c3c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5415d880 0x7f0c3c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540d0630 con 0x7f0c3c0039b0 2023-11-26T07:21:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c00ba70 con 0x7f0c3c0047d0 2023-11-26T07:21:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c00bc30 con 0x7f0c5415d880 2023-11-26T07:21:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c537fe640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 0x7f0c3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c52ffd640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c3c0047d0 0x7f0c3c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c53fff640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5415d880 0x7f0c3c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c537fe640 1 -- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5415d880 msgr2=0x7f0c3c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c537fe640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5415d880 0x7f0c3c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c537fe640 1 -- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c3c0047d0 msgr2=0x7f0c3c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:06.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c537fe640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c3c0047d0 0x7f0c3c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.554+0000 7f0c537fe640 1 -- 172.21.15.31:0/3878409662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c003530 con 0x7f0c3c0039b0 2023-11-26T07:21:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c537fe640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 0x7f0c3c003dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009ab0 tx=0x7f0c4401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff277c5b76109ccf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c440a55e0 con 0x7f0c3c0039b0 2023-11-26T07:21:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c440a5740 con 0x7f0c3c0039b0 2023-11-26T07:21:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c527fc640 1 -- 172.21.15.31:0/3878409662 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c440a5af0 con 0x7f0c3c0039b0 2023-11-26T07:21:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c58f32640 1 -- 172.21.15.31:0/3878409662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 msgr2=0x7f0c3c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c58f32640 1 --2- 172.21.15.31:0/3878409662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 0x7f0c3c003dd0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009ab0 tx=0x7f0c4401ca10 comp rx=0 tx=0).stop 2023-11-26T07:21:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c53fff640 1 -- 172.21.15.31:0/3878409662 reap_dead start 2023-11-26T07:21:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c58f32640 1 -- 172.21.15.31:0/3878409662 shutdown_connections 2023-11-26T07:21:06.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c58f32640 1 -- 172.21.15.31:0/3878409662 >> 172.21.15.31:0/3878409662 conn(0x7f0c5406b9f0 msgr2=0x7f0c5406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:06.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c58f32640 1 -- 172.21.15.31:0/3878409662 shutdown_connections 2023-11-26T07:21:06.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.558+0000 7f0c58f32640 1 -- 172.21.15.31:0/3878409662 wait complete. 2023-11-26T07:21:06.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c58f32640 1 Processor -- start 2023-11-26T07:21:06.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c58f32640 1 -- start start 2023-11-26T07:21:06.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c58f32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 0x7f0c54168bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c58f32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5406d940 0x7f0c540714f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c58f32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c5415c300 0x7f0c540b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c58f32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54169430 con 0x7f0c3c0039b0 2023-11-26T07:21:06.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c58f32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54071060 con 0x7f0c5415c300 2023-11-26T07:21:06.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c58f32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54070490 con 0x7f0c5406d940 2023-11-26T07:21:06.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c537fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 0x7f0c54168bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c537fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 0x7f0c54168bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34490/0 (socket says 172.21.15.31:34490) 2023-11-26T07:21:06.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c537fe640 1 -- 172.21.15.31:0/1241467323 learned_addr learned my addr 172.21.15.31:0/1241467323 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:06.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c52ffd640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5406d940 0x7f0c540714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c53fff640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c5415c300 0x7f0c540b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c537fe640 1 -- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5406d940 msgr2=0x7f0c540714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c537fe640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5406d940 0x7f0c540714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c537fe640 1 -- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c5415c300 msgr2=0x7f0c540b4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c537fe640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c5415c300 0x7f0c540b4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c537fe640 1 -- 172.21.15.31:0/1241467323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c540d2ae0 con 0x7f0c3c0039b0 2023-11-26T07:21:06.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c537fe640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 0x7f0c54168bc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0c440b0040 tx=0x7f0c3c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:06.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c3bfff640 1 -- 172.21.15.31:0/1241467323 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c440a55e0 con 0x7f0c3c0039b0 2023-11-26T07:21:06.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c3bfff640 1 -- 172.21.15.31:0/1241467323 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c44004020 con 0x7f0c3c0039b0 2023-11-26T07:21:06.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c3bfff640 1 -- 172.21.15.31:0/1241467323 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c440043b0 con 0x7f0c3c0039b0 2023-11-26T07:21:06.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c58f32640 1 -- 172.21.15.31:0/1241467323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c54070730 con 0x7f0c3c0039b0 2023-11-26T07:21:06.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.562+0000 7f0c58f32640 1 -- 172.21.15.31:0/1241467323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c54070cf0 con 0x7f0c3c0039b0 2023-11-26T07:21:06.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.566+0000 7f0c58f32640 1 -- 172.21.15.31:0/1241467323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c540cd8f0 con 0x7f0c3c0039b0 2023-11-26T07:21:06.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.566+0000 7f0c3bfff640 1 -- 172.21.15.31:0/1241467323 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f0c440b4040 con 0x7f0c3c0039b0 2023-11-26T07:21:06.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.566+0000 7f0c3bfff640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066650 0x7f0c40068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.566+0000 7f0c3bfff640 1 -- 172.21.15.31:0/1241467323 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c4411d750 con 0x7f0c3c0039b0 2023-11-26T07:21:06.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.566+0000 7f0c52ffd640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066650 0x7f0c40068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.566+0000 7f0c52ffd640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066650 0x7f0c40068b10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0c541fe780 tx=0x7f0c48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:06.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.570+0000 7f0c3bfff640 1 -- 172.21.15.31:0/1241467323 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0c440ea8b0 con 0x7f0c3c0039b0 2023-11-26T07:21:06.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 -- 172.21.15.31:0/1241467323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c540d11a0 con 0x7f0c3c0039b0 2023-11-26T07:21:06.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c3bfff640 1 -- 172.21.15.31:0/1241467323 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f0c440eded0 con 0x7f0c3c0039b0 2023-11-26T07:21:06.768 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:06.769 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3429205589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3429205589},{"type":"v1","addr":"172.21.15.97:6836","nonce":3429205589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5959,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3569564413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3569564413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3569564413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5965,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690524463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690524463},{"type":"v1","addr":"172.21.15.97:6837","nonce":690524463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:02.722567+0000","modified":"2023-11-26T07:21:05.757700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1053315203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1053315203},{"type":"v1","addr":"172.21.15.31:6834","nonce":1053315203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:21:06.769 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 37 2023-11-26T07:21:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 -- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066650 msgr2=0x7f0c40068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066650 0x7f0c40068b10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0c541fe780 tx=0x7f0c48009040 comp rx=0 tx=0).stop 2023-11-26T07:21:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 -- 172.21.15.31:0/1241467323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 msgr2=0x7f0c54168bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 0x7f0c54168bc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f0c440b0040 tx=0x7f0c3c005800 comp rx=0 tx=0).stop 2023-11-26T07:21:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 -- 172.21.15.31:0/1241467323 shutdown_connections 2023-11-26T07:21:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0c40066650 0x7f0c40068b10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c5415c300 0x7f0c540b4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0c5406d940 0x7f0c540714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 --2- 172.21.15.31:0/1241467323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3c0039b0 0x7f0c54168bc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 -- 172.21.15.31:0/1241467323 >> 172.21.15.31:0/1241467323 conn(0x7f0c5406b9f0 msgr2=0x7f0c541603f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 -- 172.21.15.31:0/1241467323 shutdown_connections 2023-11-26T07:21:06.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.766+0000 7f0c58f32640 1 -- 172.21.15.31:0/1241467323 wait complete. 2023-11-26T07:21:06.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.930+0000 7fb7ee101640 1 Processor -- start 2023-11-26T07:21:06.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.930+0000 7fb7ee101640 1 -- start start 2023-11-26T07:21:06.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.930+0000 7fb7ee101640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7e8071cc0 0x7fb7e80b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.930+0000 7fb7ee101640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb7e80b2ee0 0x7fb7e80adee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.930+0000 7fb7ee101640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e80ad740 con 0x7fb7e806e410 2023-11-26T07:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.930+0000 7fb7ee101640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e80b3e20 con 0x7fb7e80b2ee0 2023-11-26T07:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.930+0000 7fb7ee101640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80b3390 con 0x7fb7e8071cc0 2023-11-26T07:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.930+0000 7fb7e77fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb7e806e410 0x7fb7e80715b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35510/0 (socket says 172.21.15.31:35510) 2023-11-26T07:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.930+0000 7fb7e77fe640 1 -- 172.21.15.31:0/2588637804 learned_addr learned my addr 172.21.15.31:0/2588637804 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.930+0000 7fb7e6ffd640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7e8071cc0 0x7fb7e80b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e7fff640 1 --2- 172.21.15.31:0/2588637804 >> v2:172.21.15.97:3300/0 conn(0x7fb7e80b2ee0 0x7fb7e80adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058477511 0 0) 0x7fb7e80ad740 con 0x7fb7e806e410 2023-11-26T07:21:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d0003530 con 0x7fb7e806e410 2023-11-26T07:21:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 707431934 0 0) 0x7fb7d0003530 con 0x7fb7e806e410 2023-11-26T07:21:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7e8071cc0 msgr2=0x7fb7e80b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7e8071cc0 0x7fb7e80b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 >> v2:172.21.15.97:3300/0 conn(0x7fb7e80b2ee0 msgr2=0x7fb7e80adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:06.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 --2- 172.21.15.31:0/2588637804 >> v2:172.21.15.97:3300/0 conn(0x7fb7e80b2ee0 0x7fb7e80adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e80b3580 con 0x7fb7e806e410 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fb7d8001680 con 0x7fb7e806e410 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 >> v1:172.21.15.31:6789/0 conn(0x7fb7e806e410 legacy=0x7fb7e80715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d0003980 0x7fb7d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7e8071cc0 0x7fb7d00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d0004d10 0x7fb7d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80b3e20 con 0x7fb7d0004d10 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d000b990 con 0x7fb7d0003980 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d000bad0 con 0x7fb7e8071cc0 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e7fff640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d0004d10 0x7fb7d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e6ffd640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7e8071cc0 0x7fb7d00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e77fe640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d0003980 0x7fb7d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e7fff640 1 -- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7e8071cc0 msgr2=0x7fb7d00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:06.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e7fff640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7e8071cc0 0x7fb7d00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e7fff640 1 -- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d0003980 msgr2=0x7fb7d0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:06.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e7fff640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7d0003980 0x7fb7d0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.934+0000 7fb7e7fff640 1 -- 172.21.15.31:0/2588637804 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d0003530 con 0x7fb7d0004d10 2023-11-26T07:21:06.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7e7fff640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d0004d10 0x7fb7d0005190 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e170 tx=0x7fb7e001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95b89dacc036391 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:06.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00a5c90 con 0x7fb7d0004d10 2023-11-26T07:21:06.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e000f040 con 0x7fb7d0004d10 2023-11-26T07:21:06.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7e67fc640 1 -- 172.21.15.31:0/2588637804 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00ad440 con 0x7fb7d0004d10 2023-11-26T07:21:06.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7ee101640 1 -- 172.21.15.31:0/2588637804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d0004d10 msgr2=0x7fb7d0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7ee101640 1 --2- 172.21.15.31:0/2588637804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7d0004d10 0x7fb7d0005190 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb7e000e170 tx=0x7fb7e001cc80 comp rx=0 tx=0).stop 2023-11-26T07:21:06.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7e7fff640 1 -- 172.21.15.31:0/2588637804 reap_dead start 2023-11-26T07:21:06.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7ee101640 1 -- 172.21.15.31:0/2588637804 shutdown_connections 2023-11-26T07:21:06.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7ee101640 1 -- 172.21.15.31:0/2588637804 >> 172.21.15.31:0/2588637804 conn(0x7fb7e80b1e40 msgr2=0x7fb7e8070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:06.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7ee101640 1 -- 172.21.15.31:0/2588637804 shutdown_connections 2023-11-26T07:21:06.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7ee101640 1 -- 172.21.15.31:0/2588637804 wait complete. 2023-11-26T07:21:06.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7ee101640 1 Processor -- start 2023-11-26T07:21:06.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.938+0000 7fb7ee101640 1 -- start start 2023-11-26T07:21:06.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7ee101640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7d0004d10 0x7fb7e814c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7ee101640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e814f6d0 0x7fb7e814fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7ee101640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e814d0a0 0x7fb7e814d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7ee101640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8145da0 con 0x7fb7e814f6d0 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7ee101640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e814f240 con 0x7fb7e814d0a0 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7ee101640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e814cec0 con 0x7fb7d0004d10 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e814f6d0 0x7fb7e814fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e814f6d0 0x7fb7e814fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34510/0 (socket says 172.21.15.31:34510) 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e6ffd640 1 -- 172.21.15.31:0/2966985007 learned_addr learned my addr 172.21.15.31:0/2966985007 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e77fe640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7d0004d10 0x7fb7e814c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e7fff640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e814d0a0 0x7fb7e814d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e6ffd640 1 -- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7d0004d10 msgr2=0x7fb7e814c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e6ffd640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7d0004d10 0x7fb7e814c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e6ffd640 1 -- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e814d0a0 msgr2=0x7fb7e814d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:06.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e6ffd640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e814d0a0 0x7fb7e814d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e6ffd640 1 -- 172.21.15.31:0/2966985007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e814cbe0 con 0x7fb7e814f6d0 2023-11-26T07:21:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e7fff640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e814d0a0 0x7fb7e814d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7e6ffd640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e814f6d0 0x7fb7e814fb50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0119d0 tx=0x7fb7dc011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7cffff640 1 -- 172.21.15.31:0/2966985007 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00faf0 con 0x7fb7e814f6d0 2023-11-26T07:21:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7cffff640 1 -- 172.21.15.31:0/2966985007 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7dc00fc50 con 0x7fb7e814f6d0 2023-11-26T07:21:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7cffff640 1 -- 172.21.15.31:0/2966985007 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc014440 con 0x7fb7e814f6d0 2023-11-26T07:21:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7ee101640 1 -- 172.21.15.31:0/2966985007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e814ebf0 con 0x7fb7e814f6d0 2023-11-26T07:21:06.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7ee101640 1 -- 172.21.15.31:0/2966985007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e814da60 con 0x7fb7e814f6d0 2023-11-26T07:21:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7ee101640 1 -- 172.21.15.31:0/2966985007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e804e850 con 0x7fb7e814f6d0 2023-11-26T07:21:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.942+0000 7fb7cffff640 1 -- 172.21.15.31:0/2966985007 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fb7dc0145a0 con 0x7fb7e814f6d0 2023-11-26T07:21:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.946+0000 7fb7cffff640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7d4066650 0x7fb7d4068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.946+0000 7fb7cffff640 1 -- 172.21.15.31:0/2966985007 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7dc0849b0 con 0x7fb7e814f6d0 2023-11-26T07:21:06.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.946+0000 7fb7e77fe640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7d4066650 0x7fb7d4068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:06.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.946+0000 7fb7e77fe640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7d4066650 0x7fb7d4068b10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8000c10 tx=0x7fb7d8007900 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:06.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:06.950+0000 7fb7cffff640 1 -- 172.21.15.31:0/2966985007 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb7dc051c60 con 0x7fb7e814f6d0 2023-11-26T07:21:07.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.106+0000 7fb7ee101640 1 -- 172.21.15.31:0/2966985007 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7e814f380 con 0x7fb7e814f6d0 2023-11-26T07:21:07.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.106+0000 7fb7cffff640 1 -- 172.21.15.31:0/2966985007 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fb7dc022020 con 0x7fb7e814f6d0 2023-11-26T07:21:07.110 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:07.111 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:04.352334+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:01.142504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:01.723895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:07.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 -- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7d4066650 msgr2=0x7fb7d4068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7d4066650 0x7fb7d4068b10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8000c10 tx=0x7fb7d8007900 comp rx=0 tx=0).stop 2023-11-26T07:21:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 -- 172.21.15.31:0/2966985007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e814f6d0 msgr2=0x7fb7e814fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e814f6d0 0x7fb7e814fb50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0119d0 tx=0x7fb7dc011dd0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 -- 172.21.15.31:0/2966985007 shutdown_connections 2023-11-26T07:21:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb7d4066650 0x7fb7d4068b10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7e814d0a0 0x7fb7e814d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e814f6d0 0x7fb7e814fb50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 --2- 172.21.15.31:0/2966985007 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb7d0004d10 0x7fb7e814c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 -- 172.21.15.31:0/2966985007 >> 172.21.15.31:0/2966985007 conn(0x7fb7e80b1e40 msgr2=0x7fb7e8152a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 -- 172.21.15.31:0/2966985007 shutdown_connections 2023-11-26T07:21:07.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.110+0000 7fb7ee101640 1 -- 172.21.15.31:0/2966985007 wait complete. 2023-11-26T07:21:07.132 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:21:07.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:21:07.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff938b1c640 1 Processor -- start 2023-11-26T07:21:07.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff938b1c640 1 -- start start 2023-11-26T07:21:07.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff938b1c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff9340d2080 0x7ff934168160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff938b1c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9340cefd0 con 0x7ff9340cead0 2023-11-26T07:21:07.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff938b1c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9340cf2b0 con 0x7ff93415d7b0 2023-11-26T07:21:07.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff938b1c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9340d17e0 con 0x7ff9340d2080 2023-11-26T07:21:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff931d9b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff9340cead0 0x7ff9340d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35522/0 (socket says 172.21.15.31:35522) 2023-11-26T07:21:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff931d9b640 1 -- 172.21.15.31:0/2045761273 learned_addr learned my addr 172.21.15.31:0/2045761273 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff932d9d640 1 --2- 172.21.15.31:0/2045761273 >> v2:172.21.15.97:3300/0 conn(0x7ff9340d2080 0x7ff934168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff93159a640 1 -- 172.21.15.31:0/2045761273 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628383959 0 0) 0x7ff9340cefd0 con 0x7ff9340cead0 2023-11-26T07:21:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff93159a640 1 -- 172.21.15.31:0/2045761273 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff91c003530 con 0x7ff9340cead0 2023-11-26T07:21:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff932d9d640 1 -- 172.21.15.31:0/2045761273 >> v1:172.21.15.97:6789/0 conn(0x7ff93415d7b0 legacy=0x7ff9340d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff93159a640 1 -- 172.21.15.31:0/2045761273 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498597851 0 0) 0x7ff9340cf2b0 con 0x7ff93415d7b0 2023-11-26T07:21:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff932d9d640 1 -- 172.21.15.31:0/2045761273 >> v1:172.21.15.31:6789/0 conn(0x7ff9340cead0 legacy=0x7ff9340d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.274+0000 7ff932d9d640 1 -- 172.21.15.31:0/2045761273 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93406b3f0 con 0x7ff9340d2080 2023-11-26T07:21:07.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff932d9d640 1 --2- 172.21.15.31:0/2045761273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9340d2080 0x7ff934168160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009d90 tx=0x7ff92c01c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5989e22d70d0ac server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff93159a640 1 -- 172.21.15.31:0/2045761273 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0a58c0 con 0x7ff9340d2080 2023-11-26T07:21:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff93159a640 1 -- 172.21.15.31:0/2045761273 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c0a5a20 con 0x7ff9340d2080 2023-11-26T07:21:07.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff93159a640 1 -- 172.21.15.31:0/2045761273 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0a5d90 con 0x7ff9340d2080 2023-11-26T07:21:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff938b1c640 1 -- 172.21.15.31:0/2045761273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9340d2080 msgr2=0x7ff934168160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff938b1c640 1 --2- 172.21.15.31:0/2045761273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9340d2080 0x7ff934168160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009d90 tx=0x7ff92c01c800 comp rx=0 tx=0).stop 2023-11-26T07:21:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff938b1c640 1 -- 172.21.15.31:0/2045761273 shutdown_connections 2023-11-26T07:21:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff938b1c640 1 --2- 172.21.15.31:0/2045761273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9340d2080 0x7ff934168160 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff938b1c640 1 -- 172.21.15.31:0/2045761273 >> 172.21.15.31:0/2045761273 conn(0x7ff9340d33b0 msgr2=0x7ff9340d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff938b1c640 1 -- 172.21.15.31:0/2045761273 shutdown_connections 2023-11-26T07:21:07.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.278+0000 7ff938b1c640 1 -- 172.21.15.31:0/2045761273 wait complete. 2023-11-26T07:21:07.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff938b1c640 1 Processor -- start 2023-11-26T07:21:07.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff938b1c640 1 -- start start 2023-11-26T07:21:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff938b1c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9340cead0 0x7ff9340c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff938b1c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9340d2080 0x7ff9341556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff938b1c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff93415d7b0 0x7ff9340b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff938b1c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340c95d0 con 0x7ff9340d2080 2023-11-26T07:21:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff938b1c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340c83c0 con 0x7ff9340cead0 2023-11-26T07:21:07.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff938b1c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9340c7a90 con 0x7ff93415d7b0 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff931d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9340d2080 0x7ff9341556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff931d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9340d2080 0x7ff9341556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34518/0 (socket says 172.21.15.31:34518) 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff931d9b640 1 -- 172.21.15.31:0/3189456670 learned_addr learned my addr 172.21.15.31:0/3189456670 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff932d9d640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff93415d7b0 0x7ff9340b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff93259c640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9340cead0 0x7ff9340c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff931d9b640 1 -- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff93415d7b0 msgr2=0x7ff9340b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff931d9b640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff93415d7b0 0x7ff9340b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff931d9b640 1 -- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9340cead0 msgr2=0x7ff9340c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff931d9b640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9340cead0 0x7ff9340c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff931d9b640 1 -- 172.21.15.31:0/3189456670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c009a70 con 0x7ff9340d2080 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff931d9b640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9340d2080 0x7ff9341556f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff92800c390 tx=0x7ff92800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:07.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff922ffd640 1 -- 172.21.15.31:0/3189456670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928005cf0 con 0x7ff9340d2080 2023-11-26T07:21:07.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff922ffd640 1 -- 172.21.15.31:0/3189456670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff928005e70 con 0x7ff9340d2080 2023-11-26T07:21:07.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff922ffd640 1 -- 172.21.15.31:0/3189456670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9280072f0 con 0x7ff9340d2080 2023-11-26T07:21:07.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff938b1c640 1 -- 172.21.15.31:0/3189456670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9340c7d70 con 0x7ff9340d2080 2023-11-26T07:21:07.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.282+0000 7ff938b1c640 1 -- 172.21.15.31:0/3189456670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9340b6cc0 con 0x7ff9340d2080 2023-11-26T07:21:07.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.286+0000 7ff938b1c640 1 -- 172.21.15.31:0/3189456670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9340c8570 con 0x7ff9340d2080 2023-11-26T07:21:07.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.286+0000 7ff922ffd640 1 -- 172.21.15.31:0/3189456670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7ff928007450 con 0x7ff9340d2080 2023-11-26T07:21:07.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.286+0000 7ff922ffd640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff914066620 0x7ff914068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.286+0000 7ff922ffd640 1 -- 172.21.15.31:0/3189456670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff928083640 con 0x7ff9340d2080 2023-11-26T07:21:07.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.286+0000 7ff93259c640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff914066620 0x7ff914068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.286+0000 7ff93259c640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff914066620 0x7ff914068ae0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff93415d520 tx=0x7ff924009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:07.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.290+0000 7ff922ffd640 1 -- 172.21.15.31:0/3189456670 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff9340c8570 con 0x7ff9340d2080 2023-11-26T07:21:07.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.446+0000 7ff938b1c640 1 -- 172.21.15.31:0/3189456670 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9340c7c00 con 0x7ff914066620 2023-11-26T07:21:07.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.454+0000 7ff922ffd640 1 -- 172.21.15.31:0/3189456670 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288593 (secure 0 0 0) 0x7ff9340c7c00 con 0x7ff914066620 2023-11-26T07:21:07.461 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:07.463 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:21:07.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 -- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff914066620 msgr2=0x7ff914068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff914066620 0x7ff914068ae0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff93415d520 tx=0x7ff924009040 comp rx=0 tx=0).stop 2023-11-26T07:21:07.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 -- 172.21.15.31:0/3189456670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9340d2080 msgr2=0x7ff9341556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9340d2080 0x7ff9341556f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff92800c390 tx=0x7ff92800c850 comp rx=0 tx=0).stop 2023-11-26T07:21:07.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 -- 172.21.15.31:0/3189456670 shutdown_connections 2023-11-26T07:21:07.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff914066620 0x7ff914068ae0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff93415d7b0 0x7ff9340b61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9340d2080 0x7ff9341556f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 --2- 172.21.15.31:0/3189456670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9340cead0 0x7ff9340c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 -- 172.21.15.31:0/3189456670 >> 172.21.15.31:0/3189456670 conn(0x7ff9340d33b0 msgr2=0x7ff9340d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 -- 172.21.15.31:0/3189456670 shutdown_connections 2023-11-26T07:21:07.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.458+0000 7ff938b1c640 1 -- 172.21.15.31:0/3189456670 wait complete. 2023-11-26T07:21:07.482 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2023-11-26T07:21:06.360148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239368,"kb_used_data":6540,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735352,"statfs":{"total":773094113280,"available":772849000448,"internally_reserved":0,"allocated":6696960,"data_stored":3451969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.037496"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364844+0000","last_change":"2023-11-26T07:21:03.386155+0000","last_active":"2023-11-26T07:21:04.364844+0000","last_peered":"2023-11-26T07:21:04.364844+0000","last_clean":"2023-11-26T07:21:04.364844+0000","last_became_active":"2023-11-26T07:21:03.384875+0000","last_became_peered":"2023-11-26T07:21:03.384875+0000","last_unstale":"2023-11-26T07:21:04.364844+0000","last_undegraded":"2023-11-26T07:21:04.364844+0000","last_fullsized":"2023-11-26T07:21:04.364844+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:44:46.591162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396308+0000","last_change":"2023-11-26T07:21:03.386199+0000","last_active":"2023-11-26T07:21:04.396308+0000","last_peered":"2023-11-26T07:21:04.396308+0000","last_clean":"2023-11-26T07:21:04.396308+0000","last_became_active":"2023-11-26T07:21:03.385928+0000","last_became_peered":"2023-11-26T07:21:03.385928+0000","last_unstale":"2023-11-26T07:21:04.396308+0000","last_undegraded":"2023-11-26T07:21:04.396308+0000","last_fullsized":"2023-11-26T07:21:04.396308+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:57.208084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252745+0000","last_change":"2023-11-26T07:21:03.369998+0000","last_active":"2023-11-26T07:21:05.252745+0000","last_peered":"2023-11-26T07:21:05.252745+0000","last_clean":"2023-11-26T07:21:05.252745+0000","last_became_active":"2023-11-26T07:21:03.369708+0000","last_became_peered":"2023-11-26T07:21:03.369708+0000","last_unstale":"2023-11-26T07:21:05.252745+0000","last_undegraded":"2023-11-26T07:21:05.252745+0000","last_fullsized":"2023-11-26T07:21:05.252745+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:30:57.184043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252226+0000","last_change":"2023-11-26T07:21:03.362433+0000","last_active":"2023-11-26T07:21:05.252226+0000","last_peered":"2023-11-26T07:21:05.252226+0000","last_clean":"2023-11-26T07:21:05.252226+0000","last_became_active":"2023-11-26T07:21:03.361118+0000","last_became_peered":"2023-11-26T07:21:03.361118+0000","last_unstale":"2023-11-26T07:21:05.252226+0000","last_undegraded":"2023-11-26T07:21:05.252226+0000","last_fullsized":"2023-11-26T07:21:05.252226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:52:38.839238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364456+0000","last_change":"2023-11-26T07:21:03.364370+0000","last_active":"2023-11-26T07:21:04.364456+0000","last_peered":"2023-11-26T07:21:04.364456+0000","last_clean":"2023-11-26T07:21:04.364456+0000","last_became_active":"2023-11-26T07:21:03.362262+0000","last_became_peered":"2023-11-26T07:21:03.362262+0000","last_unstale":"2023-11-26T07:21:04.364456+0000","last_undegraded":"2023-11-26T07:21:04.364456+0000","last_fullsized":"2023-11-26T07:21:04.364456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:36:14.945015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977105+0000","last_change":"2023-11-26T07:21:03.385073+0000","last_active":"2023-11-26T07:21:04.977105+0000","last_peered":"2023-11-26T07:21:04.977105+0000","last_clean":"2023-11-26T07:21:04.977105+0000","last_became_active":"2023-11-26T07:21:03.383028+0000","last_became_peered":"2023-11-26T07:21:03.383028+0000","last_unstale":"2023-11-26T07:21:04.977105+0000","last_undegraded":"2023-11-26T07:21:04.977105+0000","last_fullsized":"2023-11-26T07:21:04.977105+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:18:12.211335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368122+0000","last_change":"2023-11-26T07:21:03.363504+0000","last_active":"2023-11-26T07:21:04.368122+0000","last_peered":"2023-11-26T07:21:04.368122+0000","last_clean":"2023-11-26T07:21:04.368122+0000","last_became_active":"2023-11-26T07:21:03.362775+0000","last_became_peered":"2023-11-26T07:21:03.362775+0000","last_unstale":"2023-11-26T07:21:04.368122+0000","last_undegraded":"2023-11-26T07:21:04.368122+0000","last_fullsized":"2023-11-26T07:21:04.368122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:55:55.475667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.395995+0000","last_change":"2023-11-26T07:21:03.366970+0000","last_active":"2023-11-26T07:21:04.395995+0000","last_peered":"2023-11-26T07:21:04.395995+0000","last_clean":"2023-11-26T07:21:04.395995+0000","last_became_active":"2023-11-26T07:21:03.366644+0000","last_became_peered":"2023-11-26T07:21:03.366644+0000","last_unstale":"2023-11-26T07:21:04.395995+0000","last_undegraded":"2023-11-26T07:21:04.395995+0000","last_fullsized":"2023-11-26T07:21:04.395995+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:38.343545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368682+0000","last_change":"2023-11-26T07:21:03.387977+0000","last_active":"2023-11-26T07:21:04.368682+0000","last_peered":"2023-11-26T07:21:04.368682+0000","last_clean":"2023-11-26T07:21:04.368682+0000","last_became_active":"2023-11-26T07:21:03.387712+0000","last_became_peered":"2023-11-26T07:21:03.387712+0000","last_unstale":"2023-11-26T07:21:04.368682+0000","last_undegraded":"2023-11-26T07:21:04.368682+0000","last_fullsized":"2023-11-26T07:21:04.368682+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:32:10.045656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252369+0000","last_change":"2023-11-26T07:21:03.375803+0000","last_active":"2023-11-26T07:21:05.252369+0000","last_peered":"2023-11-26T07:21:05.252369+0000","last_clean":"2023-11-26T07:21:05.252369+0000","last_became_active":"2023-11-26T07:21:03.375542+0000","last_became_peered":"2023-11-26T07:21:03.375542+0000","last_unstale":"2023-11-26T07:21:05.252369+0000","last_undegraded":"2023-11-26T07:21:05.252369+0000","last_fullsized":"2023-11-26T07:21:05.252369+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:06:06.243732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252373+0000","last_change":"2023-11-26T07:21:03.383851+0000","last_active":"2023-11-26T07:21:05.252373+0000","last_peered":"2023-11-26T07:21:05.252373+0000","last_clean":"2023-11-26T07:21:05.252373+0000","last_became_active":"2023-11-26T07:21:03.383486+0000","last_became_peered":"2023-11-26T07:21:03.383486+0000","last_unstale":"2023-11-26T07:21:05.252373+0000","last_undegraded":"2023-11-26T07:21:05.252373+0000","last_fullsized":"2023-11-26T07:21:05.252373+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:28:02.849233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252374+0000","last_change":"2023-11-26T07:21:03.364033+0000","last_active":"2023-11-26T07:21:05.252374+0000","last_peered":"2023-11-26T07:21:05.252374+0000","last_clean":"2023-11-26T07:21:05.252374+0000","last_became_active":"2023-11-26T07:21:03.363755+0000","last_became_peered":"2023-11-26T07:21:03.363755+0000","last_unstale":"2023-11-26T07:21:05.252374+0000","last_undegraded":"2023-11-26T07:21:05.252374+0000","last_fullsized":"2023-11-26T07:21:05.252374+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:30:57.184043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366675+0000","last_change":"2023-11-26T07:21:03.385356+0000","last_active":"2023-11-26T07:21:04.366675+0000","last_peered":"2023-11-26T07:21:04.366675+0000","last_clean":"2023-11-26T07:21:04.366675+0000","last_became_active":"2023-11-26T07:21:03.385064+0000","last_became_peered":"2023-11-26T07:21:03.385064+0000","last_unstale":"2023-11-26T07:21:04.366675+0000","last_undegraded":"2023-11-26T07:21:04.366675+0000","last_fullsized":"2023-11-26T07:21:04.366675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:57:04.036866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368355+0000","last_change":"2023-11-26T07:21:03.357913+0000","last_active":"2023-11-26T07:21:04.368355+0000","last_peered":"2023-11-26T07:21:04.368355+0000","last_clean":"2023-11-26T07:21:04.368355+0000","last_became_active":"2023-11-26T07:21:03.357622+0000","last_became_peered":"2023-11-26T07:21:03.357622+0000","last_unstale":"2023-11-26T07:21:04.368355+0000","last_undegraded":"2023-11-26T07:21:04.368355+0000","last_fullsized":"2023-11-26T07:21:04.368355+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:51:37.114357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368316+0000","last_change":"2023-11-26T07:21:03.363297+0000","last_active":"2023-11-26T07:21:04.368316+0000","last_peered":"2023-11-26T07:21:04.368316+0000","last_clean":"2023-11-26T07:21:04.368316+0000","last_became_active":"2023-11-26T07:21:03.362988+0000","last_became_peered":"2023-11-26T07:21:03.362988+0000","last_unstale":"2023-11-26T07:21:04.368316+0000","last_undegraded":"2023-11-26T07:21:04.368316+0000","last_fullsized":"2023-11-26T07:21:04.368316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:51:38.023765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977041+0000","last_change":"2023-11-26T07:21:03.373542+0000","last_active":"2023-11-26T07:21:04.977041+0000","last_peered":"2023-11-26T07:21:04.977041+0000","last_clean":"2023-11-26T07:21:04.977041+0000","last_became_active":"2023-11-26T07:21:03.373142+0000","last_became_peered":"2023-11-26T07:21:03.373142+0000","last_unstale":"2023-11-26T07:21:04.977041+0000","last_undegraded":"2023-11-26T07:21:04.977041+0000","last_fullsized":"2023-11-26T07:21:04.977041+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:42.015742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396605+0000","last_change":"2023-11-26T07:21:03.381010+0000","last_active":"2023-11-26T07:21:04.396605+0000","last_peered":"2023-11-26T07:21:04.396605+0000","last_clean":"2023-11-26T07:21:04.396605+0000","last_became_active":"2023-11-26T07:21:03.380702+0000","last_became_peered":"2023-11-26T07:21:03.380702+0000","last_unstale":"2023-11-26T07:21:04.396605+0000","last_undegraded":"2023-11-26T07:21:04.396605+0000","last_fullsized":"2023-11-26T07:21:04.396605+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:34:48.265696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252799+0000","last_change":"2023-11-26T07:21:03.390617+0000","last_active":"2023-11-26T07:21:05.252799+0000","last_peered":"2023-11-26T07:21:05.252799+0000","last_clean":"2023-11-26T07:21:05.252799+0000","last_became_active":"2023-11-26T07:21:03.390328+0000","last_became_peered":"2023-11-26T07:21:03.390328+0000","last_unstale":"2023-11-26T07:21:05.252799+0000","last_undegraded":"2023-11-26T07:21:05.252799+0000","last_fullsized":"2023-11-26T07:21:05.252799+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:05.628845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364659+0000","last_change":"2023-11-26T07:21:03.368334+0000","last_active":"2023-11-26T07:21:04.364659+0000","last_peered":"2023-11-26T07:21:04.364659+0000","last_clean":"2023-11-26T07:21:04.364659+0000","last_became_active":"2023-11-26T07:21:03.368053+0000","last_became_peered":"2023-11-26T07:21:03.368053+0000","last_unstale":"2023-11-26T07:21:04.364659+0000","last_undegraded":"2023-11-26T07:21:04.364659+0000","last_fullsized":"2023-11-26T07:21:04.364659+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:57:16.891965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396417+0000","last_change":"2023-11-26T07:21:03.387499+0000","last_active":"2023-11-26T07:21:04.396417+0000","last_peered":"2023-11-26T07:21:04.396417+0000","last_clean":"2023-11-26T07:21:04.396417+0000","last_became_active":"2023-11-26T07:21:03.387160+0000","last_became_peered":"2023-11-26T07:21:03.387160+0000","last_unstale":"2023-11-26T07:21:04.396417+0000","last_undegraded":"2023-11-26T07:21:04.396417+0000","last_fullsized":"2023-11-26T07:21:04.396417+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:28:55.054249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252793+0000","last_change":"2023-11-26T07:21:03.364127+0000","last_active":"2023-11-26T07:21:05.252793+0000","last_peered":"2023-11-26T07:21:05.252793+0000","last_clean":"2023-11-26T07:21:05.252793+0000","last_became_active":"2023-11-26T07:21:03.362777+0000","last_became_peered":"2023-11-26T07:21:03.362777+0000","last_unstale":"2023-11-26T07:21:05.252793+0000","last_undegraded":"2023-11-26T07:21:05.252793+0000","last_fullsized":"2023-11-26T07:21:05.252793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:49.720970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396216+0000","last_change":"2023-11-26T07:21:03.372588+0000","last_active":"2023-11-26T07:21:04.396216+0000","last_peered":"2023-11-26T07:21:04.396216+0000","last_clean":"2023-11-26T07:21:04.396216+0000","last_became_active":"2023-11-26T07:21:03.372319+0000","last_became_peered":"2023-11-26T07:21:03.372319+0000","last_unstale":"2023-11-26T07:21:04.396216+0000","last_undegraded":"2023-11-26T07:21:04.396216+0000","last_fullsized":"2023-11-26T07:21:04.396216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:29:37.921422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252226+0000","last_change":"2023-11-26T07:21:03.373638+0000","last_active":"2023-11-26T07:21:05.252226+0000","last_peered":"2023-11-26T07:21:05.252226+0000","last_clean":"2023-11-26T07:21:05.252226+0000","last_became_active":"2023-11-26T07:21:03.373363+0000","last_became_peered":"2023-11-26T07:21:03.373363+0000","last_unstale":"2023-11-26T07:21:05.252226+0000","last_undegraded":"2023-11-26T07:21:05.252226+0000","last_fullsized":"2023-11-26T07:21:05.252226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:18:12.211335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366341+0000","last_change":"2023-11-26T07:21:03.385458+0000","last_active":"2023-11-26T07:21:04.366341+0000","last_peered":"2023-11-26T07:21:04.366341+0000","last_clean":"2023-11-26T07:21:04.366341+0000","last_became_active":"2023-11-26T07:21:03.384962+0000","last_became_peered":"2023-11-26T07:21:03.384962+0000","last_unstale":"2023-11-26T07:21:04.366341+0000","last_undegraded":"2023-11-26T07:21:04.366341+0000","last_fullsized":"2023-11-26T07:21:04.366341+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:16.189058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364886+0000","last_change":"2023-11-26T07:21:03.369176+0000","last_active":"2023-11-26T07:21:04.364886+0000","last_peered":"2023-11-26T07:21:04.364886+0000","last_clean":"2023-11-26T07:21:04.364886+0000","last_became_active":"2023-11-26T07:21:03.368878+0000","last_became_peered":"2023-11-26T07:21:03.368878+0000","last_unstale":"2023-11-26T07:21:04.364886+0000","last_undegraded":"2023-11-26T07:21:04.364886+0000","last_fullsized":"2023-11-26T07:21:04.364886+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:35:31.473926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252049+0000","last_change":"2023-11-26T07:21:03.367364+0000","last_active":"2023-11-26T07:21:05.252049+0000","last_peered":"2023-11-26T07:21:05.252049+0000","last_clean":"2023-11-26T07:21:05.252049+0000","last_became_active":"2023-11-26T07:21:03.367066+0000","last_became_peered":"2023-11-26T07:21:03.367066+0000","last_unstale":"2023-11-26T07:21:05.252049+0000","last_undegraded":"2023-11-26T07:21:05.252049+0000","last_fullsized":"2023-11-26T07:21:05.252049+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:01.130100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977389+0000","last_change":"2023-11-26T07:21:03.384928+0000","last_active":"2023-11-26T07:21:04.977389+0000","last_peered":"2023-11-26T07:21:04.977389+0000","last_clean":"2023-11-26T07:21:04.977389+0000","last_became_active":"2023-11-26T07:21:03.384481+0000","last_became_peered":"2023-11-26T07:21:03.384481+0000","last_unstale":"2023-11-26T07:21:04.977389+0000","last_undegraded":"2023-11-26T07:21:04.977389+0000","last_fullsized":"2023-11-26T07:21:04.977389+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:52:38.839238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364669+0000","last_change":"2023-11-26T07:21:03.363854+0000","last_active":"2023-11-26T07:21:04.364669+0000","last_peered":"2023-11-26T07:21:04.364669+0000","last_clean":"2023-11-26T07:21:04.364669+0000","last_became_active":"2023-11-26T07:21:03.363553+0000","last_became_peered":"2023-11-26T07:21:03.363553+0000","last_unstale":"2023-11-26T07:21:04.364669+0000","last_undegraded":"2023-11-26T07:21:04.364669+0000","last_fullsized":"2023-11-26T07:21:04.364669+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:00.811057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396401+0000","last_change":"2023-11-26T07:21:03.379625+0000","last_active":"2023-11-26T07:21:04.396401+0000","last_peered":"2023-11-26T07:21:04.396401+0000","last_clean":"2023-11-26T07:21:04.396401+0000","last_became_active":"2023-11-26T07:21:03.376175+0000","last_became_peered":"2023-11-26T07:21:03.376175+0000","last_unstale":"2023-11-26T07:21:04.396401+0000","last_undegraded":"2023-11-26T07:21:04.396401+0000","last_fullsized":"2023-11-26T07:21:04.396401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:53:14.886903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252491+0000","last_change":"2023-11-26T07:21:03.365468+0000","last_active":"2023-11-26T07:21:05.252491+0000","last_peered":"2023-11-26T07:21:05.252491+0000","last_clean":"2023-11-26T07:21:05.252491+0000","last_became_active":"2023-11-26T07:21:03.365147+0000","last_became_peered":"2023-11-26T07:21:03.365147+0000","last_unstale":"2023-11-26T07:21:05.252491+0000","last_undegraded":"2023-11-26T07:21:05.252491+0000","last_fullsized":"2023-11-26T07:21:05.252491+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:27:35.853867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364379+0000","last_change":"2023-11-26T07:21:03.369981+0000","last_active":"2023-11-26T07:21:04.364379+0000","last_peered":"2023-11-26T07:21:04.364379+0000","last_clean":"2023-11-26T07:21:04.364379+0000","last_became_active":"2023-11-26T07:21:03.369706+0000","last_became_peered":"2023-11-26T07:21:03.369706+0000","last_unstale":"2023-11-26T07:21:04.364379+0000","last_undegraded":"2023-11-26T07:21:04.364379+0000","last_fullsized":"2023-11-26T07:21:04.364379+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:21.217395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977150+0000","last_change":"2023-11-26T07:21:03.392306+0000","last_active":"2023-11-26T07:21:04.977150+0000","last_peered":"2023-11-26T07:21:04.977150+0000","last_clean":"2023-11-26T07:21:04.977150+0000","last_became_active":"2023-11-26T07:21:03.391824+0000","last_became_peered":"2023-11-26T07:21:03.391824+0000","last_unstale":"2023-11-26T07:21:04.977150+0000","last_undegraded":"2023-11-26T07:21:04.977150+0000","last_fullsized":"2023-11-26T07:21:04.977150+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:01.130100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977522+0000","last_change":"2023-11-26T07:21:03.385658+0000","last_active":"2023-11-26T07:21:04.977522+0000","last_peered":"2023-11-26T07:21:04.977522+0000","last_clean":"2023-11-26T07:21:04.977522+0000","last_became_active":"2023-11-26T07:21:03.385381+0000","last_became_peered":"2023-11-26T07:21:03.385381+0000","last_unstale":"2023-11-26T07:21:04.977522+0000","last_undegraded":"2023-11-26T07:21:04.977522+0000","last_fullsized":"2023-11-26T07:21:04.977522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:30:57.184043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977569+0000","last_change":"2023-11-26T07:21:02.355959+0000","last_active":"2023-11-26T07:21:04.977569+0000","last_peered":"2023-11-26T07:21:04.977569+0000","last_clean":"2023-11-26T07:21:04.977569+0000","last_became_active":"2023-11-26T07:21:02.355075+0000","last_became_peered":"2023-11-26T07:21:02.355075+0000","last_unstale":"2023-11-26T07:21:04.977569+0000","last_undegraded":"2023-11-26T07:21:04.977569+0000","last_fullsized":"2023-11-26T07:21:04.977569+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:19:48.708001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252347+0000","last_change":"2023-11-26T07:21:03.375105+0000","last_active":"2023-11-26T07:21:05.252347+0000","last_peered":"2023-11-26T07:21:05.252347+0000","last_clean":"2023-11-26T07:21:05.252347+0000","last_became_active":"2023-11-26T07:21:03.374701+0000","last_became_peered":"2023-11-26T07:21:03.374701+0000","last_unstale":"2023-11-26T07:21:05.252347+0000","last_undegraded":"2023-11-26T07:21:05.252347+0000","last_fullsized":"2023-11-26T07:21:05.252347+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:26:41.546697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396436+0000","last_change":"2023-11-26T07:21:02.348430+0000","last_active":"2023-11-26T07:21:04.396436+0000","last_peered":"2023-11-26T07:21:04.396436+0000","last_clean":"2023-11-26T07:21:04.396436+0000","last_became_active":"2023-11-26T07:21:02.348140+0000","last_became_peered":"2023-11-26T07:21:02.348140+0000","last_unstale":"2023-11-26T07:21:04.396436+0000","last_undegraded":"2023-11-26T07:21:04.396436+0000","last_fullsized":"2023-11-26T07:21:04.396436+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:13:18.729520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977467+0000","last_change":"2023-11-26T07:21:03.383668+0000","last_active":"2023-11-26T07:21:04.977467+0000","last_peered":"2023-11-26T07:21:04.977467+0000","last_clean":"2023-11-26T07:21:04.977467+0000","last_became_active":"2023-11-26T07:21:03.383392+0000","last_became_peered":"2023-11-26T07:21:03.383392+0000","last_unstale":"2023-11-26T07:21:04.977467+0000","last_undegraded":"2023-11-26T07:21:04.977467+0000","last_fullsized":"2023-11-26T07:21:04.977467+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:05.628845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977565+0000","last_change":"2023-11-26T07:21:02.350619+0000","last_active":"2023-11-26T07:21:04.977565+0000","last_peered":"2023-11-26T07:21:04.977565+0000","last_clean":"2023-11-26T07:21:04.977565+0000","last_became_active":"2023-11-26T07:21:02.350345+0000","last_became_peered":"2023-11-26T07:21:02.350345+0000","last_unstale":"2023-11-26T07:21:04.977565+0000","last_undegraded":"2023-11-26T07:21:04.977565+0000","last_fullsized":"2023-11-26T07:21:04.977565+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:48:21.733306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977400+0000","last_change":"2023-11-26T07:21:03.384746+0000","last_active":"2023-11-26T07:21:04.977400+0000","last_peered":"2023-11-26T07:21:04.977400+0000","last_clean":"2023-11-26T07:21:04.977400+0000","last_became_active":"2023-11-26T07:21:03.384155+0000","last_became_peered":"2023-11-26T07:21:03.384155+0000","last_unstale":"2023-11-26T07:21:04.977400+0000","last_undegraded":"2023-11-26T07:21:04.977400+0000","last_fullsized":"2023-11-26T07:21:04.977400+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:49.720970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252418+0000","last_change":"2023-11-26T07:21:02.347495+0000","last_active":"2023-11-26T07:21:05.252418+0000","last_peered":"2023-11-26T07:21:05.252418+0000","last_clean":"2023-11-26T07:21:05.252418+0000","last_became_active":"2023-11-26T07:21:02.347214+0000","last_became_peered":"2023-11-26T07:21:02.347214+0000","last_unstale":"2023-11-26T07:21:05.252418+0000","last_undegraded":"2023-11-26T07:21:05.252418+0000","last_fullsized":"2023-11-26T07:21:05.252418+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:50:08.586552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396056+0000","last_change":"2023-11-26T07:21:03.371745+0000","last_active":"2023-11-26T07:21:04.396056+0000","last_peered":"2023-11-26T07:21:04.396056+0000","last_clean":"2023-11-26T07:21:04.396056+0000","last_became_active":"2023-11-26T07:21:03.371443+0000","last_became_peered":"2023-11-26T07:21:03.371443+0000","last_unstale":"2023-11-26T07:21:04.396056+0000","last_undegraded":"2023-11-26T07:21:04.396056+0000","last_fullsized":"2023-11-26T07:21:04.396056+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:29.519759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368332+0000","last_change":"2023-11-26T07:21:02.352669+0000","last_active":"2023-11-26T07:21:04.368332+0000","last_peered":"2023-11-26T07:21:04.368332+0000","last_clean":"2023-11-26T07:21:04.368332+0000","last_became_active":"2023-11-26T07:21:02.352409+0000","last_became_peered":"2023-11-26T07:21:02.352409+0000","last_unstale":"2023-11-26T07:21:04.368332+0000","last_undegraded":"2023-11-26T07:21:04.368332+0000","last_fullsized":"2023-11-26T07:21:04.368332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:06:05.237512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396244+0000","last_change":"2023-11-26T07:21:03.388999+0000","last_active":"2023-11-26T07:21:04.396244+0000","last_peered":"2023-11-26T07:21:04.396244+0000","last_clean":"2023-11-26T07:21:04.396244+0000","last_became_active":"2023-11-26T07:21:03.388752+0000","last_became_peered":"2023-11-26T07:21:03.388752+0000","last_unstale":"2023-11-26T07:21:04.396244+0000","last_undegraded":"2023-11-26T07:21:04.396244+0000","last_fullsized":"2023-11-26T07:21:04.396244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:33:58.718469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977198+0000","last_change":"2023-11-26T07:21:02.345184+0000","last_active":"2023-11-26T07:21:04.977198+0000","last_peered":"2023-11-26T07:21:04.977198+0000","last_clean":"2023-11-26T07:21:04.977198+0000","last_became_active":"2023-11-26T07:21:02.344821+0000","last_became_peered":"2023-11-26T07:21:02.344821+0000","last_unstale":"2023-11-26T07:21:04.977198+0000","last_undegraded":"2023-11-26T07:21:04.977198+0000","last_fullsized":"2023-11-26T07:21:04.977198+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:46:38.191366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366454+0000","last_change":"2023-11-26T07:21:03.359640+0000","last_active":"2023-11-26T07:21:04.366454+0000","last_peered":"2023-11-26T07:21:04.366454+0000","last_clean":"2023-11-26T07:21:04.366454+0000","last_became_active":"2023-11-26T07:21:03.359361+0000","last_became_peered":"2023-11-26T07:21:03.359361+0000","last_unstale":"2023-11-26T07:21:04.366454+0000","last_undegraded":"2023-11-26T07:21:04.366454+0000","last_fullsized":"2023-11-26T07:21:04.366454+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:46:29.111583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251853+0000","last_change":"2023-11-26T07:21:02.345247+0000","last_active":"2023-11-26T07:21:05.251853+0000","last_peered":"2023-11-26T07:21:05.251853+0000","last_clean":"2023-11-26T07:21:05.251853+0000","last_became_active":"2023-11-26T07:21:02.344934+0000","last_became_peered":"2023-11-26T07:21:02.344934+0000","last_unstale":"2023-11-26T07:21:05.251853+0000","last_undegraded":"2023-11-26T07:21:05.251853+0000","last_fullsized":"2023-11-26T07:21:05.251853+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:48:44.815091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396063+0000","last_change":"2023-11-26T07:21:03.378854+0000","last_active":"2023-11-26T07:21:04.396063+0000","last_peered":"2023-11-26T07:21:04.396063+0000","last_clean":"2023-11-26T07:21:04.396063+0000","last_became_active":"2023-11-26T07:21:03.378578+0000","last_became_peered":"2023-11-26T07:21:03.378578+0000","last_unstale":"2023-11-26T07:21:04.396063+0000","last_undegraded":"2023-11-26T07:21:04.396063+0000","last_fullsized":"2023-11-26T07:21:04.396063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:38:13.589648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368121+0000","last_change":"2023-11-26T07:21:02.351579+0000","last_active":"2023-11-26T07:21:04.368121+0000","last_peered":"2023-11-26T07:21:04.368121+0000","last_clean":"2023-11-26T07:21:04.368121+0000","last_became_active":"2023-11-26T07:21:02.351203+0000","last_became_peered":"2023-11-26T07:21:02.351203+0000","last_unstale":"2023-11-26T07:21:04.368121+0000","last_undegraded":"2023-11-26T07:21:04.368121+0000","last_fullsized":"2023-11-26T07:21:04.368121+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:46:28.105363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368724+0000","last_change":"2023-11-26T07:21:03.362630+0000","last_active":"2023-11-26T07:21:04.368724+0000","last_peered":"2023-11-26T07:21:04.368724+0000","last_clean":"2023-11-26T07:21:04.368724+0000","last_became_active":"2023-11-26T07:21:03.362301+0000","last_became_peered":"2023-11-26T07:21:03.362301+0000","last_unstale":"2023-11-26T07:21:04.368724+0000","last_undegraded":"2023-11-26T07:21:04.368724+0000","last_fullsized":"2023-11-26T07:21:04.368724+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:36.512866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.253072+0000","last_change":"2023-11-26T07:21:02.350168+0000","last_active":"2023-11-26T07:21:05.253072+0000","last_peered":"2023-11-26T07:21:05.253072+0000","last_clean":"2023-11-26T07:21:05.253072+0000","last_became_active":"2023-11-26T07:21:02.349846+0000","last_became_peered":"2023-11-26T07:21:02.349846+0000","last_unstale":"2023-11-26T07:21:05.253072+0000","last_undegraded":"2023-11-26T07:21:05.253072+0000","last_fullsized":"2023-11-26T07:21:05.253072+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:12:25.647531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.253053+0000","last_change":"2023-11-26T07:21:03.366130+0000","last_active":"2023-11-26T07:21:05.253053+0000","last_peered":"2023-11-26T07:21:05.253053+0000","last_clean":"2023-11-26T07:21:05.253053+0000","last_became_active":"2023-11-26T07:21:03.365855+0000","last_became_peered":"2023-11-26T07:21:03.365855+0000","last_unstale":"2023-11-26T07:21:05.253053+0000","last_undegraded":"2023-11-26T07:21:05.253053+0000","last_fullsized":"2023-11-26T07:21:05.253053+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:57.566419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396513+0000","last_change":"2023-11-26T07:21:02.358713+0000","last_active":"2023-11-26T07:21:04.396513+0000","last_peered":"2023-11-26T07:21:04.396513+0000","last_clean":"2023-11-26T07:21:04.396513+0000","last_became_active":"2023-11-26T07:21:02.358441+0000","last_became_peered":"2023-11-26T07:21:02.358441+0000","last_unstale":"2023-11-26T07:21:04.396513+0000","last_undegraded":"2023-11-26T07:21:04.396513+0000","last_fullsized":"2023-11-26T07:21:04.396513+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:29.613665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252203+0000","last_change":"2023-11-26T07:21:03.372076+0000","last_active":"2023-11-26T07:21:05.252203+0000","last_peered":"2023-11-26T07:21:05.252203+0000","last_clean":"2023-11-26T07:21:05.252203+0000","last_became_active":"2023-11-26T07:21:03.371656+0000","last_became_peered":"2023-11-26T07:21:03.371656+0000","last_unstale":"2023-11-26T07:21:05.252203+0000","last_undegraded":"2023-11-26T07:21:05.252203+0000","last_fullsized":"2023-11-26T07:21:05.252203+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:57:04.036866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252993+0000","last_change":"2023-11-26T07:21:02.350387+0000","last_active":"2023-11-26T07:21:05.252993+0000","last_peered":"2023-11-26T07:21:05.252993+0000","last_clean":"2023-11-26T07:21:05.252993+0000","last_became_active":"2023-11-26T07:21:02.350099+0000","last_became_peered":"2023-11-26T07:21:02.350099+0000","last_unstale":"2023-11-26T07:21:05.252993+0000","last_undegraded":"2023-11-26T07:21:05.252993+0000","last_fullsized":"2023-11-26T07:21:05.252993+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:40:17.484190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.253010+0000","last_change":"2023-11-26T07:21:03.368500+0000","last_active":"2023-11-26T07:21:05.253010+0000","last_peered":"2023-11-26T07:21:05.253010+0000","last_clean":"2023-11-26T07:21:05.253010+0000","last_became_active":"2023-11-26T07:21:03.368222+0000","last_became_peered":"2023-11-26T07:21:03.368222+0000","last_unstale":"2023-11-26T07:21:05.253010+0000","last_undegraded":"2023-11-26T07:21:05.253010+0000","last_fullsized":"2023-11-26T07:21:05.253010+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:02.820690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368585+0000","last_change":"2023-11-26T07:21:02.351440+0000","last_active":"2023-11-26T07:21:04.368585+0000","last_peered":"2023-11-26T07:21:04.368585+0000","last_clean":"2023-11-26T07:21:04.368585+0000","last_became_active":"2023-11-26T07:21:02.350504+0000","last_became_peered":"2023-11-26T07:21:02.350504+0000","last_unstale":"2023-11-26T07:21:04.368585+0000","last_undegraded":"2023-11-26T07:21:04.368585+0000","last_fullsized":"2023-11-26T07:21:04.368585+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:26:40.540477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251811+0000","last_change":"2023-11-26T07:21:03.367583+0000","last_active":"2023-11-26T07:21:05.251811+0000","last_peered":"2023-11-26T07:21:05.251811+0000","last_clean":"2023-11-26T07:21:05.251811+0000","last_became_active":"2023-11-26T07:21:03.367210+0000","last_became_peered":"2023-11-26T07:21:03.367210+0000","last_unstale":"2023-11-26T07:21:05.251811+0000","last_undegraded":"2023-11-26T07:21:05.251811+0000","last_fullsized":"2023-11-26T07:21:05.251811+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:42.015742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396246+0000","last_change":"2023-11-26T07:21:02.359252+0000","last_active":"2023-11-26T07:21:04.396246+0000","last_peered":"2023-11-26T07:21:04.396246+0000","last_clean":"2023-11-26T07:21:04.396246+0000","last_became_active":"2023-11-26T07:21:02.358217+0000","last_became_peered":"2023-11-26T07:21:02.358217+0000","last_unstale":"2023-11-26T07:21:04.396246+0000","last_undegraded":"2023-11-26T07:21:04.396246+0000","last_fullsized":"2023-11-26T07:21:04.396246+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:01.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252688+0000","last_change":"2023-11-26T07:21:03.364704+0000","last_active":"2023-11-26T07:21:05.252688+0000","last_peered":"2023-11-26T07:21:05.252688+0000","last_clean":"2023-11-26T07:21:05.252688+0000","last_became_active":"2023-11-26T07:21:03.364416+0000","last_became_peered":"2023-11-26T07:21:03.364416+0000","last_unstale":"2023-11-26T07:21:05.252688+0000","last_undegraded":"2023-11-26T07:21:05.252688+0000","last_fullsized":"2023-11-26T07:21:05.252688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:52:38.839238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366408+0000","last_change":"2023-11-26T07:21:02.349014+0000","last_active":"2023-11-26T07:21:04.366408+0000","last_peered":"2023-11-26T07:21:04.366408+0000","last_clean":"2023-11-26T07:21:04.366408+0000","last_became_active":"2023-11-26T07:21:02.348699+0000","last_became_peered":"2023-11-26T07:21:02.348699+0000","last_unstale":"2023-11-26T07:21:04.366408+0000","last_undegraded":"2023-11-26T07:21:04.366408+0000","last_fullsized":"2023-11-26T07:21:04.366408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:11:37.171832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368122+0000","last_change":"2023-11-26T07:21:03.355702+0000","last_active":"2023-11-26T07:21:04.368122+0000","last_peered":"2023-11-26T07:21:04.368122+0000","last_clean":"2023-11-26T07:21:04.368122+0000","last_became_active":"2023-11-26T07:21:03.355386+0000","last_became_peered":"2023-11-26T07:21:03.355386+0000","last_unstale":"2023-11-26T07:21:04.368122+0000","last_undegraded":"2023-11-26T07:21:04.368122+0000","last_fullsized":"2023-11-26T07:21:04.368122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:36:24.491317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251760+0000","last_change":"2023-11-26T07:21:02.351084+0000","last_active":"2023-11-26T07:21:05.251760+0000","last_peered":"2023-11-26T07:21:05.251760+0000","last_clean":"2023-11-26T07:21:05.251760+0000","last_became_active":"2023-11-26T07:21:02.350796+0000","last_became_peered":"2023-11-26T07:21:02.350796+0000","last_unstale":"2023-11-26T07:21:05.251760+0000","last_undegraded":"2023-11-26T07:21:05.251760+0000","last_fullsized":"2023-11-26T07:21:05.251760+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:19:48.708001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252385+0000","last_change":"2023-11-26T07:21:03.373608+0000","last_active":"2023-11-26T07:21:05.252385+0000","last_peered":"2023-11-26T07:21:05.252385+0000","last_clean":"2023-11-26T07:21:05.252385+0000","last_became_active":"2023-11-26T07:21:03.373277+0000","last_became_peered":"2023-11-26T07:21:03.373277+0000","last_unstale":"2023-11-26T07:21:05.252385+0000","last_undegraded":"2023-11-26T07:21:05.252385+0000","last_fullsized":"2023-11-26T07:21:05.252385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:01.130100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251802+0000","last_change":"2023-11-26T07:21:02.352787+0000","last_active":"2023-11-26T07:21:05.251802+0000","last_peered":"2023-11-26T07:21:05.251802+0000","last_clean":"2023-11-26T07:21:05.251802+0000","last_became_active":"2023-11-26T07:21:02.352517+0000","last_became_peered":"2023-11-26T07:21:02.352517+0000","last_unstale":"2023-11-26T07:21:05.251802+0000","last_undegraded":"2023-11-26T07:21:05.251802+0000","last_fullsized":"2023-11-26T07:21:05.251802+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:51:37.017545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252852+0000","last_change":"2023-11-26T07:21:03.365379+0000","last_active":"2023-11-26T07:21:05.252852+0000","last_peered":"2023-11-26T07:21:05.252852+0000","last_clean":"2023-11-26T07:21:05.252852+0000","last_became_active":"2023-11-26T07:21:03.365089+0000","last_became_peered":"2023-11-26T07:21:03.365089+0000","last_unstale":"2023-11-26T07:21:05.252852+0000","last_undegraded":"2023-11-26T07:21:05.252852+0000","last_fullsized":"2023-11-26T07:21:05.252852+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:57.039182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396587+0000","last_change":"2023-11-26T07:21:02.359000+0000","last_active":"2023-11-26T07:21:04.396587+0000","last_peered":"2023-11-26T07:21:04.396587+0000","last_clean":"2023-11-26T07:21:04.396587+0000","last_became_active":"2023-11-26T07:21:02.358717+0000","last_became_peered":"2023-11-26T07:21:02.358717+0000","last_unstale":"2023-11-26T07:21:04.396587+0000","last_undegraded":"2023-11-26T07:21:04.396587+0000","last_fullsized":"2023-11-26T07:21:04.396587+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:56.032962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252717+0000","last_change":"2023-11-26T07:21:03.367661+0000","last_active":"2023-11-26T07:21:05.252717+0000","last_peered":"2023-11-26T07:21:05.252717+0000","last_clean":"2023-11-26T07:21:05.252717+0000","last_became_active":"2023-11-26T07:21:03.367381+0000","last_became_peered":"2023-11-26T07:21:03.367381+0000","last_unstale":"2023-11-26T07:21:05.252717+0000","last_undegraded":"2023-11-26T07:21:05.252717+0000","last_fullsized":"2023-11-26T07:21:05.252717+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:49.720970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977597+0000","last_change":"2023-11-26T07:21:02.358649+0000","last_active":"2023-11-26T07:21:04.977597+0000","last_peered":"2023-11-26T07:21:04.977597+0000","last_clean":"2023-11-26T07:21:04.977597+0000","last_became_active":"2023-11-26T07:21:02.358346+0000","last_became_peered":"2023-11-26T07:21:02.358346+0000","last_unstale":"2023-11-26T07:21:04.977597+0000","last_undegraded":"2023-11-26T07:21:04.977597+0000","last_fullsized":"2023-11-26T07:21:04.977597+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:48:44.815091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396398+0000","last_change":"2023-11-26T07:21:03.367409+0000","last_active":"2023-11-26T07:21:04.396398+0000","last_peered":"2023-11-26T07:21:04.396398+0000","last_clean":"2023-11-26T07:21:04.396398+0000","last_became_active":"2023-11-26T07:21:03.367094+0000","last_became_peered":"2023-11-26T07:21:03.367094+0000","last_unstale":"2023-11-26T07:21:04.396398+0000","last_undegraded":"2023-11-26T07:21:04.396398+0000","last_fullsized":"2023-11-26T07:21:04.396398+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:25.313667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252114+0000","last_change":"2023-11-26T07:21:02.345391+0000","last_active":"2023-11-26T07:21:05.252114+0000","last_peered":"2023-11-26T07:21:05.252114+0000","last_clean":"2023-11-26T07:21:05.252114+0000","last_became_active":"2023-11-26T07:21:02.345091+0000","last_became_peered":"2023-11-26T07:21:02.345091+0000","last_unstale":"2023-11-26T07:21:05.252114+0000","last_undegraded":"2023-11-26T07:21:05.252114+0000","last_fullsized":"2023-11-26T07:21:05.252114+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:48:21.733306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252901+0000","last_change":"2023-11-26T07:21:03.364513+0000","last_active":"2023-11-26T07:21:05.252901+0000","last_peered":"2023-11-26T07:21:05.252901+0000","last_clean":"2023-11-26T07:21:05.252901+0000","last_became_active":"2023-11-26T07:21:03.361707+0000","last_became_peered":"2023-11-26T07:21:03.361707+0000","last_unstale":"2023-11-26T07:21:05.252901+0000","last_undegraded":"2023-11-26T07:21:05.252901+0000","last_fullsized":"2023-11-26T07:21:05.252901+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:44.853586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977736+0000","last_change":"2023-11-26T07:21:02.351812+0000","last_active":"2023-11-26T07:21:04.977736+0000","last_peered":"2023-11-26T07:21:04.977736+0000","last_clean":"2023-11-26T07:21:04.977736+0000","last_became_active":"2023-11-26T07:21:02.351518+0000","last_became_peered":"2023-11-26T07:21:02.351518+0000","last_unstale":"2023-11-26T07:21:04.977736+0000","last_undegraded":"2023-11-26T07:21:04.977736+0000","last_fullsized":"2023-11-26T07:21:04.977736+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:12:25.647531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251969+0000","last_change":"2023-11-26T07:21:03.367980+0000","last_active":"2023-11-26T07:21:05.251969+0000","last_peered":"2023-11-26T07:21:05.251969+0000","last_clean":"2023-11-26T07:21:05.251969+0000","last_became_active":"2023-11-26T07:21:03.367645+0000","last_became_peered":"2023-11-26T07:21:03.367645+0000","last_unstale":"2023-11-26T07:21:05.251969+0000","last_undegraded":"2023-11-26T07:21:05.251969+0000","last_fullsized":"2023-11-26T07:21:05.251969+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:18:12.211335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396374+0000","last_change":"2023-11-26T07:21:02.349587+0000","last_active":"2023-11-26T07:21:04.396374+0000","last_peered":"2023-11-26T07:21:04.396374+0000","last_clean":"2023-11-26T07:21:04.396374+0000","last_became_active":"2023-11-26T07:21:02.349322+0000","last_became_peered":"2023-11-26T07:21:02.349322+0000","last_unstale":"2023-11-26T07:21:04.396374+0000","last_undegraded":"2023-11-26T07:21:04.396374+0000","last_fullsized":"2023-11-26T07:21:04.396374+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:56.560199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977267+0000","last_change":"2023-11-26T07:21:03.383400+0000","last_active":"2023-11-26T07:21:04.977267+0000","last_peered":"2023-11-26T07:21:04.977267+0000","last_clean":"2023-11-26T07:21:04.977267+0000","last_became_active":"2023-11-26T07:21:03.382982+0000","last_became_peered":"2023-11-26T07:21:03.382982+0000","last_unstale":"2023-11-26T07:21:04.977267+0000","last_undegraded":"2023-11-26T07:21:04.977267+0000","last_fullsized":"2023-11-26T07:21:04.977267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:27:35.853867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252031+0000","last_change":"2023-11-26T07:21:02.359603+0000","last_active":"2023-11-26T07:21:05.252031+0000","last_peered":"2023-11-26T07:21:05.252031+0000","last_clean":"2023-11-26T07:21:05.252031+0000","last_became_active":"2023-11-26T07:21:02.359312+0000","last_became_peered":"2023-11-26T07:21:02.359312+0000","last_unstale":"2023-11-26T07:21:05.252031+0000","last_undegraded":"2023-11-26T07:21:05.252031+0000","last_fullsized":"2023-11-26T07:21:05.252031+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:56:55.153902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977032+0000","last_change":"2023-11-26T07:21:03.373309+0000","last_active":"2023-11-26T07:21:04.977032+0000","last_peered":"2023-11-26T07:21:04.977032+0000","last_clean":"2023-11-26T07:21:04.977032+0000","last_became_active":"2023-11-26T07:21:03.372872+0000","last_became_peered":"2023-11-26T07:21:03.372872+0000","last_unstale":"2023-11-26T07:21:04.977032+0000","last_undegraded":"2023-11-26T07:21:04.977032+0000","last_fullsized":"2023-11-26T07:21:04.977032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:25.274024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251635+0000","last_change":"2023-11-26T07:21:02.347554+0000","last_active":"2023-11-26T07:21:05.251635+0000","last_peered":"2023-11-26T07:21:05.251635+0000","last_clean":"2023-11-26T07:21:05.251635+0000","last_became_active":"2023-11-26T07:21:02.347273+0000","last_became_peered":"2023-11-26T07:21:02.347273+0000","last_unstale":"2023-11-26T07:21:05.251635+0000","last_undegraded":"2023-11-26T07:21:05.251635+0000","last_fullsized":"2023-11-26T07:21:05.251635+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:15:20.377323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252172+0000","last_change":"2023-11-26T07:21:03.362042+0000","last_active":"2023-11-26T07:21:05.252172+0000","last_peered":"2023-11-26T07:21:05.252172+0000","last_clean":"2023-11-26T07:21:05.252172+0000","last_became_active":"2023-11-26T07:21:03.361727+0000","last_became_peered":"2023-11-26T07:21:03.361727+0000","last_unstale":"2023-11-26T07:21:05.252172+0000","last_undegraded":"2023-11-26T07:21:05.252172+0000","last_fullsized":"2023-11-26T07:21:05.252172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:27:35.853867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252408+0000","last_change":"2023-11-26T07:21:02.355296+0000","last_active":"2023-11-26T07:21:05.252408+0000","last_peered":"2023-11-26T07:21:05.252408+0000","last_clean":"2023-11-26T07:21:05.252408+0000","last_became_active":"2023-11-26T07:21:02.354680+0000","last_became_peered":"2023-11-26T07:21:02.354680+0000","last_unstale":"2023-11-26T07:21:05.252408+0000","last_undegraded":"2023-11-26T07:21:05.252408+0000","last_fullsized":"2023-11-26T07:21:05.252408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:15.182838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252849+0000","last_change":"2023-11-26T07:21:03.363559+0000","last_active":"2023-11-26T07:21:05.252849+0000","last_peered":"2023-11-26T07:21:05.252849+0000","last_clean":"2023-11-26T07:21:05.252849+0000","last_became_active":"2023-11-26T07:21:03.363125+0000","last_became_peered":"2023-11-26T07:21:03.363125+0000","last_unstale":"2023-11-26T07:21:05.252849+0000","last_undegraded":"2023-11-26T07:21:05.252849+0000","last_fullsized":"2023-11-26T07:21:05.252849+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:05.628845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977636+0000","last_change":"2023-11-26T07:21:02.345125+0000","last_active":"2023-11-26T07:21:04.977636+0000","last_peered":"2023-11-26T07:21:04.977636+0000","last_clean":"2023-11-26T07:21:04.977636+0000","last_became_active":"2023-11-26T07:21:02.344732+0000","last_became_peered":"2023-11-26T07:21:02.344732+0000","last_unstale":"2023-11-26T07:21:04.977636+0000","last_undegraded":"2023-11-26T07:21:04.977636+0000","last_fullsized":"2023-11-26T07:21:04.977636+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:50:08.586552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252315+0000","last_change":"2023-11-26T07:21:03.371605+0000","last_active":"2023-11-26T07:21:05.252315+0000","last_peered":"2023-11-26T07:21:05.252315+0000","last_clean":"2023-11-26T07:21:05.252315+0000","last_became_active":"2023-11-26T07:21:03.371267+0000","last_became_peered":"2023-11-26T07:21:03.371267+0000","last_unstale":"2023-11-26T07:21:05.252315+0000","last_undegraded":"2023-11-26T07:21:05.252315+0000","last_fullsized":"2023-11-26T07:21:05.252315+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:08:09.640208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366580+0000","last_change":"2023-11-26T07:21:02.346301+0000","last_active":"2023-11-26T07:21:04.366580+0000","last_peered":"2023-11-26T07:21:04.366580+0000","last_clean":"2023-11-26T07:21:04.366580+0000","last_became_active":"2023-11-26T07:21:02.345981+0000","last_became_peered":"2023-11-26T07:21:02.345981+0000","last_unstale":"2023-11-26T07:21:04.366580+0000","last_undegraded":"2023-11-26T07:21:04.366580+0000","last_fullsized":"2023-11-26T07:21:04.366580+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:56:55.153902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977658+0000","last_change":"2023-11-26T07:21:03.386337+0000","last_active":"2023-11-26T07:21:04.977658+0000","last_peered":"2023-11-26T07:21:04.977658+0000","last_clean":"2023-11-26T07:21:04.977658+0000","last_became_active":"2023-11-26T07:21:03.386053+0000","last_became_peered":"2023-11-26T07:21:03.386053+0000","last_unstale":"2023-11-26T07:21:04.977658+0000","last_undegraded":"2023-11-26T07:21:04.977658+0000","last_fullsized":"2023-11-26T07:21:04.977658+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:58.898933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252467+0000","last_change":"2023-11-26T07:21:02.346915+0000","last_active":"2023-11-26T07:21:05.252467+0000","last_peered":"2023-11-26T07:21:05.252467+0000","last_clean":"2023-11-26T07:21:05.252467+0000","last_became_active":"2023-11-26T07:21:02.346640+0000","last_became_peered":"2023-11-26T07:21:02.346640+0000","last_unstale":"2023-11-26T07:21:05.252467+0000","last_undegraded":"2023-11-26T07:21:05.252467+0000","last_fullsized":"2023-11-26T07:21:05.252467+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:40:17.484190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396201+0000","last_change":"2023-11-26T07:21:03.375636+0000","last_active":"2023-11-26T07:21:04.396201+0000","last_peered":"2023-11-26T07:21:04.396201+0000","last_clean":"2023-11-26T07:21:04.396201+0000","last_became_active":"2023-11-26T07:21:03.375470+0000","last_became_peered":"2023-11-26T07:21:03.375470+0000","last_unstale":"2023-11-26T07:21:04.396201+0000","last_undegraded":"2023-11-26T07:21:04.396201+0000","last_fullsized":"2023-11-26T07:21:04.396201+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:45:49.937288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251990+0000","last_change":"2023-11-26T07:21:02.356049+0000","last_active":"2023-11-26T07:21:05.251990+0000","last_peered":"2023-11-26T07:21:05.251990+0000","last_clean":"2023-11-26T07:21:05.251990+0000","last_became_active":"2023-11-26T07:21:02.355685+0000","last_became_peered":"2023-11-26T07:21:02.355685+0000","last_unstale":"2023-11-26T07:21:05.251990+0000","last_undegraded":"2023-11-26T07:21:05.251990+0000","last_fullsized":"2023-11-26T07:21:05.251990+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:11:37.171832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252839+0000","last_change":"2023-11-26T07:21:03.379311+0000","last_active":"2023-11-26T07:21:05.252839+0000","last_peered":"2023-11-26T07:21:05.252839+0000","last_clean":"2023-11-26T07:21:05.252839+0000","last_became_active":"2023-11-26T07:21:03.379010+0000","last_became_peered":"2023-11-26T07:21:03.379010+0000","last_unstale":"2023-11-26T07:21:05.252839+0000","last_undegraded":"2023-11-26T07:21:05.252839+0000","last_fullsized":"2023-11-26T07:21:05.252839+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:58.898933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368262+0000","last_change":"2023-11-26T07:21:02.353249+0000","last_active":"2023-11-26T07:21:04.368262+0000","last_peered":"2023-11-26T07:21:04.368262+0000","last_clean":"2023-11-26T07:21:04.368262+0000","last_became_active":"2023-11-26T07:21:02.352948+0000","last_became_peered":"2023-11-26T07:21:02.352948+0000","last_unstale":"2023-11-26T07:21:04.368262+0000","last_undegraded":"2023-11-26T07:21:04.368262+0000","last_fullsized":"2023-11-26T07:21:04.368262+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:51:53.928403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251853+0000","last_change":"2023-11-26T07:21:03.374236+0000","last_active":"2023-11-26T07:21:05.251853+0000","last_peered":"2023-11-26T07:21:05.251853+0000","last_clean":"2023-11-26T07:21:05.251853+0000","last_became_active":"2023-11-26T07:21:03.373912+0000","last_became_peered":"2023-11-26T07:21:03.373912+0000","last_unstale":"2023-11-26T07:21:05.251853+0000","last_undegraded":"2023-11-26T07:21:05.251853+0000","last_fullsized":"2023-11-26T07:21:05.251853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:46:29.111583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251883+0000","last_change":"2023-11-26T07:21:02.351786+0000","last_active":"2023-11-26T07:21:05.251883+0000","last_peered":"2023-11-26T07:21:05.251883+0000","last_clean":"2023-11-26T07:21:05.251883+0000","last_became_active":"2023-11-26T07:21:02.351507+0000","last_became_peered":"2023-11-26T07:21:02.351507+0000","last_unstale":"2023-11-26T07:21:05.251883+0000","last_undegraded":"2023-11-26T07:21:05.251883+0000","last_fullsized":"2023-11-26T07:21:05.251883+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:04:13.358930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396092+0000","last_change":"2023-11-26T07:21:03.379423+0000","last_active":"2023-11-26T07:21:04.396092+0000","last_peered":"2023-11-26T07:21:04.396092+0000","last_clean":"2023-11-26T07:21:04.396092+0000","last_became_active":"2023-11-26T07:21:03.378829+0000","last_became_peered":"2023-11-26T07:21:03.378829+0000","last_unstale":"2023-11-26T07:21:04.396092+0000","last_undegraded":"2023-11-26T07:21:04.396092+0000","last_fullsized":"2023-11-26T07:21:04.396092+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:06:15.219155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396086+0000","last_change":"2023-11-26T07:21:02.352891+0000","last_active":"2023-11-26T07:21:04.396086+0000","last_peered":"2023-11-26T07:21:04.396086+0000","last_clean":"2023-11-26T07:21:04.396086+0000","last_became_active":"2023-11-26T07:21:02.352704+0000","last_became_peered":"2023-11-26T07:21:02.352704+0000","last_unstale":"2023-11-26T07:21:04.396086+0000","last_undegraded":"2023-11-26T07:21:04.396086+0000","last_fullsized":"2023-11-26T07:21:04.396086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:36.459993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.395974+0000","last_change":"2023-11-26T07:21:03.387578+0000","last_active":"2023-11-26T07:21:04.395974+0000","last_peered":"2023-11-26T07:21:04.395974+0000","last_clean":"2023-11-26T07:21:04.395974+0000","last_became_active":"2023-11-26T07:21:03.387273+0000","last_became_peered":"2023-11-26T07:21:03.387273+0000","last_unstale":"2023-11-26T07:21:04.395974+0000","last_undegraded":"2023-11-26T07:21:04.395974+0000","last_fullsized":"2023-11-26T07:21:04.395974+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:43:54.528101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366332+0000","last_change":"2023-11-26T07:21:02.352401+0000","last_active":"2023-11-26T07:21:04.366332+0000","last_peered":"2023-11-26T07:21:04.366332+0000","last_clean":"2023-11-26T07:21:04.366332+0000","last_became_active":"2023-11-26T07:21:02.352137+0000","last_became_peered":"2023-11-26T07:21:02.352137+0000","last_unstale":"2023-11-26T07:21:04.366332+0000","last_undegraded":"2023-11-26T07:21:04.366332+0000","last_fullsized":"2023-11-26T07:21:04.366332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:32:24.841087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:21:03.350205+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:21:03.350205+0000","last_peered":"2023-11-26T07:21:03.350205+0000","last_clean":"2023-11-26T07:21:03.350205+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:21:03.350205+0000","last_undegraded":"2023-11-26T07:21:03.350205+0000","last_fullsized":"2023-11-26T07:21:03.350205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":875532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.51000000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92200000000000004,"5min":0.995,"15min":0.995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67300000000000004,"15min":0.67200000000000004},"min":{"1min":0.48599999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94199999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68200000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.50700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.98799999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70199999999999996,"15min":0.71599999999999997},"min":{"1min":0.54800000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86799999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.66400000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.55800000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90400000000000003,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.879},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92400000000000004,"5min":1.016,"15min":1.016},"last":0.72199999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95199999999999996,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.80400000000000005,"15min":0.82199999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.97999999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.79200000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.67400000000000004,"15min":0.71799999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.66200000000000003,"15min":0.69399999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81399999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.58499999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.753,"15min":0.78100000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83199999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.81000000000000005,"15min":0.85599999999999998},"min":{"1min":0.51800000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94799999999999995,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.73499999999999999,"15min":0.76700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88200000000000001,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.80300000000000005,"15min":0.83499999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88500000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.69899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":684032,"data_stored":285004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.45700000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97799999999999998,"5min":2.492,"15min":2.492},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71499999999999997,"15min":0.73799999999999999},"min":{"1min":0.49099999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.73599999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.749},"min":{"1min":0.504,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92000000000000004,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.78200000000000003,"15min":0.81599999999999995},"min":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91600000000000004,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.97499999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.60399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95499999999999996,"5min":1.677,"15min":1.677},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.67800000000000005,"15min":0.67300000000000004},"min":{"1min":0.54400000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":1.367,"15min":1.367},"last":0.78100000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95799999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79200000000000004,"15min":0.82299999999999995},"min":{"1min":0.52800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.874,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.91800000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63800000000000001,"15min":0.64700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89500000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.751},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.69399999999999995,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85499999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.68999999999999995,"15min":0.72399999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.83399999999999996,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.751,"15min":0.78600000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97699999999999998,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.80300000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.75600000000000001,"15min":0.78800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.93899999999999995,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.754,"15min":0.78400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.86299999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.71499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":285656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.60799999999999998,"15min":0.58099999999999996},"min":{"1min":0.56200000000000006,"5min":0.316,"15min":0.316},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.49299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.44400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.55800000000000005,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.51000000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.018,"5min":1.036,"15min":1.036},"last":0.54500000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.60299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91700000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.93799999999999994,"5min":1.071,"15min":1.071},"last":0.57599999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66900000000000004,"15min":0.69199999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83999999999999997,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.40500000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.0569999999999999,"5min":1.206,"15min":1.206},"last":0.68000000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.64300000000000002,"15min":0.67800000000000005},"min":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80800000000000005,"5min":1.143,"15min":1.143},"last":0.628},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.70899999999999996,"15min":0.73899999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.032,"5min":1.228,"15min":1.228},"last":0.60999999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73799999999999999,"15min":0.754},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95799999999999996,"5min":1.264,"15min":1.264},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78800000000000003,"15min":0.81799999999999995},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97999999999999998,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.76300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":286616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.68100000000000005,"15min":0.68600000000000005},"min":{"1min":0.502,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98899999999999999,"5min":1.016,"15min":1.016},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84599999999999997,"5min":1.26,"15min":1.26},"last":0.50700000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95899999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.53200000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.88800000000000001,"5min":1.077,"15min":1.077},"last":0.67700000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96899999999999997,"5min":1.302,"15min":1.302},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.47799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96099999999999997,"5min":1.181,"15min":1.181},"last":0.61899999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73799999999999999,"15min":0.752},"min":{"1min":0.57599999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93200000000000005,"5min":1.702,"15min":1.702},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.78900000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.002,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67700000000000005,"15min":0.69599999999999995},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80600000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85199999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.45800000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.48499999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86699999999999999,"5min":1.129,"15min":1.129},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76600000000000001,"15min":0.79200000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91800000000000004,"5min":1.012,"15min":1.012},"last":0.79700000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.499,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.879,"5min":1.016,"15min":1.016},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.93300000000000005,"5min":1.242,"15min":1.242},"last":0.69999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":285060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47499999999999998},"min":{"1min":0.315,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.629,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71799999999999997,"5min":1.294,"15min":1.294},"last":0.48199999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.44600000000000001},"min":{"1min":0.32500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76900000000000002,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52200000000000002,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76600000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45000000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.436,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82699999999999996,"5min":1.456,"15min":1.456},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.58299999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.60599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1579999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0680000000000001,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.72199999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.61099999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0229999999999999,"5min":1.395,"15min":1.395},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.66500000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1200000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.79300000000000004,"15min":0.79400000000000004},"min":{"1min":0.628,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0960000000000001,"5min":1.333,"15min":1.333},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80700000000000005,"15min":0.81100000000000005},"min":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1180000000000001,"5min":1.347,"15min":1.347},"last":0.69699999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80400000000000005},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0700000000000001,"5min":1.397,"15min":1.397},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.61599999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.087,"5min":1.363,"15min":1.363},"last":0.56399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":285060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.60399999999999998,"15min":0.66300000000000003},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.746,"5min":3.931,"15min":3.931},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.67200000000000004,"15min":0.72399999999999998},"min":{"1min":0.432,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.747,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.64100000000000001,"15min":0.70499999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.75600000000000001,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.64200000000000002,"15min":0.69999999999999996},"min":{"1min":0.39500000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78800000000000003,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.57299999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.501,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.76900000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.82699999999999996,"15min":0.92800000000000005},"min":{"1min":0.46200000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.76500000000000001,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.70799999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.88900000000000001,"15min":0.94099999999999995},"min":{"1min":0.58499999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.046,"5min":4.431,"15min":4.431},"last":1.099},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.876,"15min":0.91700000000000004},"min":{"1min":0.56799999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3049999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":1.024}]},{"osd":5,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.101,"5min":1.575,"15min":1.575},"last":1.21},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.91500000000000004,"15min":0.97499999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.038,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.94899999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.93700000000000006,"15min":0.998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.256,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1359999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.1240000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.94299999999999995,"15min":0.996},"min":{"1min":0.61599999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.84799999999999998},"min":{"1min":0.59099999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.054,"5min":1.609,"15min":1.609},"last":1.052}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":278794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.27500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.63300000000000001,"5min":1.034,"15min":1.034},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76100000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.39500000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58299999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71499999999999997,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.58899999999999997},"min":{"1min":0.47499999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82199999999999995,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.5,"15min":0.504},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68799999999999994,"5min":0.753,"15min":0.753},"last":0.44400000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.50800000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53100000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.216,"15min":1.216},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.56499999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.175,"5min":1.262,"15min":1.262},"last":0.754},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.75,"15min":0.75},"min":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0780000000000001,"5min":1.54,"15min":1.54},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.74299999999999999,"15min":0.73099999999999998},"min":{"1min":0.55400000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.095,"5min":1.145,"15min":1.145},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1180000000000001,"5min":1.333,"15min":1.333},"last":0.63100000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76100000000000001,"15min":0.76200000000000001},"min":{"1min":0.61399999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.04,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.61599999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1930000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.53200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":870247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.32000000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88500000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.36499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73399999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.47599999999999998},"min":{"1min":0.35499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63700000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.66600000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.80400000000000005,"15min":0.80900000000000005},"min":{"1min":0.59399999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0349999999999999,"5min":1.091,"15min":1.091},"last":0.999}]},{"osd":5,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.58299999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98799999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":1.042},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.83199999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.61099999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96499999999999997,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.024},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.79100000000000004,"15min":0.80500000000000005},"min":{"1min":0.50800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.0129999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.0549999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0169999999999999,"5min":1.123,"15min":1.123},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.81499999999999995,"15min":0.81399999999999995},"min":{"1min":0.71899999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.071,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0820000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:21:07.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:21:07.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.670+0000 7fae12613640 1 Processor -- start 2023-11-26T07:21:07.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.670+0000 7fae12613640 1 -- start start 2023-11-26T07:21:07.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.670+0000 7fae12613640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae0c0cead0 0x7fae0c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.670+0000 7fae12613640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae0c0d1eb0 0x7fae0c06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.670+0000 7fae12613640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c0d2300 con 0x7fae0c15d7b0 2023-11-26T07:21:07.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.670+0000 7fae12613640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c0d17e0 con 0x7fae0c0cead0 2023-11-26T07:21:07.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.670+0000 7fae12613640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0cefd0 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.670+0000 7fae0b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae0c0cead0 0x7fae0c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae0b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae0c0cead0 0x7fae0c0d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34532/0 (socket says 172.21.15.31:34532) 2023-11-26T07:21:07.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae0b7fe640 1 -- 172.21.15.31:0/1977042384 learned_addr learned my addr 172.21.15.31:0/1977042384 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:07.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae10bb0640 1 --2- 172.21.15.31:0/1977042384 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae0c0d1eb0 0x7fae0c06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae0b7fe640 1 -- 172.21.15.31:0/1977042384 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae0c0d1eb0 msgr2=0x7fae0c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:07.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae0b7fe640 1 --2- 172.21.15.31:0/1977042384 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae0c0d1eb0 0x7fae0c06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae0b7fe640 1 -- 172.21.15.31:0/1977042384 >> v1:172.21.15.97:6789/0 conn(0x7fae0c15d7b0 legacy=0x7fae0c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae0b7fe640 1 -- 172.21.15.31:0/1977042384 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0cf2b0 con 0x7fae0c0cead0 2023-11-26T07:21:07.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae0b7fe640 1 --2- 172.21.15.31:0/1977042384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0cead0 0x7fae0c0d1970 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fae00009870 tx=0x7fae0001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2b5db3737ec1ab server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:07.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae0affd640 1 -- 172.21.15.31:0/1977042384 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae000ab5f0 con 0x7fae0c0cead0 2023-11-26T07:21:07.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae0affd640 1 -- 172.21.15.31:0/1977042384 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae000ab750 con 0x7fae0c0cead0 2023-11-26T07:21:07.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.674+0000 7fae0affd640 1 -- 172.21.15.31:0/1977042384 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae000abae0 con 0x7fae0c0cead0 2023-11-26T07:21:07.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 -- 172.21.15.31:0/1977042384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0cead0 msgr2=0x7fae0c0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 --2- 172.21.15.31:0/1977042384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0cead0 0x7fae0c0d1970 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fae00009870 tx=0x7fae0001c3f0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 -- 172.21.15.31:0/1977042384 shutdown_connections 2023-11-26T07:21:07.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 --2- 172.21.15.31:0/1977042384 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae0c0d1eb0 0x7fae0c06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 --2- 172.21.15.31:0/1977042384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0cead0 0x7fae0c0d1970 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 -- 172.21.15.31:0/1977042384 >> 172.21.15.31:0/1977042384 conn(0x7fae0c0d33b0 msgr2=0x7fae0c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:07.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 -- 172.21.15.31:0/1977042384 shutdown_connections 2023-11-26T07:21:07.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 -- 172.21.15.31:0/1977042384 wait complete. 2023-11-26T07:21:07.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 Processor -- start 2023-11-26T07:21:07.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 -- start start 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae0c0cead0 0x7fae0c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0d1eb0 0x7fae0c0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae0c15d7b0 0x7fae0c1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c157380 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0c93e0 con 0x7fae0c0cead0 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae12613640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0c96c0 con 0x7fae0c15d7b0 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae0b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0d1eb0 0x7fae0c0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae0b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0d1eb0 0x7fae0c0b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34534/0 (socket says 172.21.15.31:34534) 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae0bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae0c0cead0 0x7fae0c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae10bb0640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae0c15d7b0 0x7fae0c1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.678+0000 7fae0b7fe640 1 -- 172.21.15.31:0/95232233 learned_addr learned my addr 172.21.15.31:0/95232233 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae0b7fe640 1 -- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae0c15d7b0 msgr2=0x7fae0c1548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae0b7fe640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae0c15d7b0 0x7fae0c1548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae0b7fe640 1 -- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae0c0cead0 msgr2=0x7fae0c159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae0b7fe640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae0c0cead0 0x7fae0c159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae0b7fe640 1 -- 172.21.15.31:0/95232233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae000094d0 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae0b7fe640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0d1eb0 0x7fae0c0b1410 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae0001a020 tx=0x7fae0001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:07.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae08ff9640 1 -- 172.21.15.31:0/95232233 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae000be5f0 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae08ff9640 1 -- 172.21.15.31:0/95232233 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae000be750 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae08ff9640 1 -- 172.21.15.31:0/95232233 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae000beb00 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae12613640 1 -- 172.21.15.31:0/95232233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c0c99a0 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae12613640 1 -- 172.21.15.31:0/95232233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c15a0d0 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae12613640 1 -- 172.21.15.31:0/95232233 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c1574f0 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae08ff9640 1 -- 172.21.15.31:0/95232233 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fae000b43f0 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae08ff9640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadf8066620 0x7fadf8068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:07.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae08ff9640 1 -- 172.21.15.31:0/95232233 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae000a8050 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.682+0000 7fae0bfff640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadf8066620 0x7fadf8068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:07.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.686+0000 7fae0bfff640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadf8066620 0x7fadf8068ae0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0ce080 tx=0x7fadfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:07.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.686+0000 7fae08ff9640 1 -- 172.21.15.31:0/95232233 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae0c1574f0 con 0x7fae0c0d1eb0 2023-11-26T07:21:07.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.846+0000 7fae12613640 1 -- 172.21.15.31:0/95232233 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fae0c0c9e50 con 0x7fadf8066620 2023-11-26T07:21:07.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.854+0000 7fae08ff9640 1 -- 172.21.15.31:0/95232233 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288593 (secure 0 0 0) 0x7fae0c0c9e50 con 0x7fadf8066620 2023-11-26T07:21:07.860 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:07.861 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:21:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 -- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadf8066620 msgr2=0x7fadf8068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadf8066620 0x7fadf8068ae0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fae0c0ce080 tx=0x7fadfc009040 comp rx=0 tx=0).stop 2023-11-26T07:21:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 -- 172.21.15.31:0/95232233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0d1eb0 msgr2=0x7fae0c0b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0d1eb0 0x7fae0c0b1410 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae0001a020 tx=0x7fae0001ac30 comp rx=0 tx=0).stop 2023-11-26T07:21:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 -- 172.21.15.31:0/95232233 shutdown_connections 2023-11-26T07:21:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadf8066620 0x7fadf8068ae0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae0c15d7b0 0x7fae0c1548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae0c0d1eb0 0x7fae0c0b1410 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 --2- 172.21.15.31:0/95232233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae0c0cead0 0x7fae0c159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 -- 172.21.15.31:0/95232233 >> 172.21.15.31:0/95232233 conn(0x7fae0c0d33b0 msgr2=0x7fae0c0d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 -- 172.21.15.31:0/95232233 shutdown_connections 2023-11-26T07:21:07.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:07.858+0000 7fae12613640 1 -- 172.21.15.31:0/95232233 wait complete. 2023-11-26T07:21:07.880 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2023-11-26T07:21:06.360148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239368,"kb_used_data":6540,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735352,"statfs":{"total":773094113280,"available":772849000448,"internally_reserved":0,"allocated":6696960,"data_stored":3451969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.037496"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364844+0000","last_change":"2023-11-26T07:21:03.386155+0000","last_active":"2023-11-26T07:21:04.364844+0000","last_peered":"2023-11-26T07:21:04.364844+0000","last_clean":"2023-11-26T07:21:04.364844+0000","last_became_active":"2023-11-26T07:21:03.384875+0000","last_became_peered":"2023-11-26T07:21:03.384875+0000","last_unstale":"2023-11-26T07:21:04.364844+0000","last_undegraded":"2023-11-26T07:21:04.364844+0000","last_fullsized":"2023-11-26T07:21:04.364844+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:44:46.591162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396308+0000","last_change":"2023-11-26T07:21:03.386199+0000","last_active":"2023-11-26T07:21:04.396308+0000","last_peered":"2023-11-26T07:21:04.396308+0000","last_clean":"2023-11-26T07:21:04.396308+0000","last_became_active":"2023-11-26T07:21:03.385928+0000","last_became_peered":"2023-11-26T07:21:03.385928+0000","last_unstale":"2023-11-26T07:21:04.396308+0000","last_undegraded":"2023-11-26T07:21:04.396308+0000","last_fullsized":"2023-11-26T07:21:04.396308+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:14:57.208084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252745+0000","last_change":"2023-11-26T07:21:03.369998+0000","last_active":"2023-11-26T07:21:05.252745+0000","last_peered":"2023-11-26T07:21:05.252745+0000","last_clean":"2023-11-26T07:21:05.252745+0000","last_became_active":"2023-11-26T07:21:03.369708+0000","last_became_peered":"2023-11-26T07:21:03.369708+0000","last_unstale":"2023-11-26T07:21:05.252745+0000","last_undegraded":"2023-11-26T07:21:05.252745+0000","last_fullsized":"2023-11-26T07:21:05.252745+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:30:57.184043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252226+0000","last_change":"2023-11-26T07:21:03.362433+0000","last_active":"2023-11-26T07:21:05.252226+0000","last_peered":"2023-11-26T07:21:05.252226+0000","last_clean":"2023-11-26T07:21:05.252226+0000","last_became_active":"2023-11-26T07:21:03.361118+0000","last_became_peered":"2023-11-26T07:21:03.361118+0000","last_unstale":"2023-11-26T07:21:05.252226+0000","last_undegraded":"2023-11-26T07:21:05.252226+0000","last_fullsized":"2023-11-26T07:21:05.252226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:52:38.839238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364456+0000","last_change":"2023-11-26T07:21:03.364370+0000","last_active":"2023-11-26T07:21:04.364456+0000","last_peered":"2023-11-26T07:21:04.364456+0000","last_clean":"2023-11-26T07:21:04.364456+0000","last_became_active":"2023-11-26T07:21:03.362262+0000","last_became_peered":"2023-11-26T07:21:03.362262+0000","last_unstale":"2023-11-26T07:21:04.364456+0000","last_undegraded":"2023-11-26T07:21:04.364456+0000","last_fullsized":"2023-11-26T07:21:04.364456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:36:14.945015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977105+0000","last_change":"2023-11-26T07:21:03.385073+0000","last_active":"2023-11-26T07:21:04.977105+0000","last_peered":"2023-11-26T07:21:04.977105+0000","last_clean":"2023-11-26T07:21:04.977105+0000","last_became_active":"2023-11-26T07:21:03.383028+0000","last_became_peered":"2023-11-26T07:21:03.383028+0000","last_unstale":"2023-11-26T07:21:04.977105+0000","last_undegraded":"2023-11-26T07:21:04.977105+0000","last_fullsized":"2023-11-26T07:21:04.977105+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:18:12.211335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368122+0000","last_change":"2023-11-26T07:21:03.363504+0000","last_active":"2023-11-26T07:21:04.368122+0000","last_peered":"2023-11-26T07:21:04.368122+0000","last_clean":"2023-11-26T07:21:04.368122+0000","last_became_active":"2023-11-26T07:21:03.362775+0000","last_became_peered":"2023-11-26T07:21:03.362775+0000","last_unstale":"2023-11-26T07:21:04.368122+0000","last_undegraded":"2023-11-26T07:21:04.368122+0000","last_fullsized":"2023-11-26T07:21:04.368122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:55:55.475667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.395995+0000","last_change":"2023-11-26T07:21:03.366970+0000","last_active":"2023-11-26T07:21:04.395995+0000","last_peered":"2023-11-26T07:21:04.395995+0000","last_clean":"2023-11-26T07:21:04.395995+0000","last_became_active":"2023-11-26T07:21:03.366644+0000","last_became_peered":"2023-11-26T07:21:03.366644+0000","last_unstale":"2023-11-26T07:21:04.395995+0000","last_undegraded":"2023-11-26T07:21:04.395995+0000","last_fullsized":"2023-11-26T07:21:04.395995+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:40:38.343545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368682+0000","last_change":"2023-11-26T07:21:03.387977+0000","last_active":"2023-11-26T07:21:04.368682+0000","last_peered":"2023-11-26T07:21:04.368682+0000","last_clean":"2023-11-26T07:21:04.368682+0000","last_became_active":"2023-11-26T07:21:03.387712+0000","last_became_peered":"2023-11-26T07:21:03.387712+0000","last_unstale":"2023-11-26T07:21:04.368682+0000","last_undegraded":"2023-11-26T07:21:04.368682+0000","last_fullsized":"2023-11-26T07:21:04.368682+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:32:10.045656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252369+0000","last_change":"2023-11-26T07:21:03.375803+0000","last_active":"2023-11-26T07:21:05.252369+0000","last_peered":"2023-11-26T07:21:05.252369+0000","last_clean":"2023-11-26T07:21:05.252369+0000","last_became_active":"2023-11-26T07:21:03.375542+0000","last_became_peered":"2023-11-26T07:21:03.375542+0000","last_unstale":"2023-11-26T07:21:05.252369+0000","last_undegraded":"2023-11-26T07:21:05.252369+0000","last_fullsized":"2023-11-26T07:21:05.252369+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:06:06.243732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252373+0000","last_change":"2023-11-26T07:21:03.383851+0000","last_active":"2023-11-26T07:21:05.252373+0000","last_peered":"2023-11-26T07:21:05.252373+0000","last_clean":"2023-11-26T07:21:05.252373+0000","last_became_active":"2023-11-26T07:21:03.383486+0000","last_became_peered":"2023-11-26T07:21:03.383486+0000","last_unstale":"2023-11-26T07:21:05.252373+0000","last_undegraded":"2023-11-26T07:21:05.252373+0000","last_fullsized":"2023-11-26T07:21:05.252373+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:28:02.849233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252374+0000","last_change":"2023-11-26T07:21:03.364033+0000","last_active":"2023-11-26T07:21:05.252374+0000","last_peered":"2023-11-26T07:21:05.252374+0000","last_clean":"2023-11-26T07:21:05.252374+0000","last_became_active":"2023-11-26T07:21:03.363755+0000","last_became_peered":"2023-11-26T07:21:03.363755+0000","last_unstale":"2023-11-26T07:21:05.252374+0000","last_undegraded":"2023-11-26T07:21:05.252374+0000","last_fullsized":"2023-11-26T07:21:05.252374+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:30:57.184043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366675+0000","last_change":"2023-11-26T07:21:03.385356+0000","last_active":"2023-11-26T07:21:04.366675+0000","last_peered":"2023-11-26T07:21:04.366675+0000","last_clean":"2023-11-26T07:21:04.366675+0000","last_became_active":"2023-11-26T07:21:03.385064+0000","last_became_peered":"2023-11-26T07:21:03.385064+0000","last_unstale":"2023-11-26T07:21:04.366675+0000","last_undegraded":"2023-11-26T07:21:04.366675+0000","last_fullsized":"2023-11-26T07:21:04.366675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:57:04.036866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368355+0000","last_change":"2023-11-26T07:21:03.357913+0000","last_active":"2023-11-26T07:21:04.368355+0000","last_peered":"2023-11-26T07:21:04.368355+0000","last_clean":"2023-11-26T07:21:04.368355+0000","last_became_active":"2023-11-26T07:21:03.357622+0000","last_became_peered":"2023-11-26T07:21:03.357622+0000","last_unstale":"2023-11-26T07:21:04.368355+0000","last_undegraded":"2023-11-26T07:21:04.368355+0000","last_fullsized":"2023-11-26T07:21:04.368355+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:51:37.114357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368316+0000","last_change":"2023-11-26T07:21:03.363297+0000","last_active":"2023-11-26T07:21:04.368316+0000","last_peered":"2023-11-26T07:21:04.368316+0000","last_clean":"2023-11-26T07:21:04.368316+0000","last_became_active":"2023-11-26T07:21:03.362988+0000","last_became_peered":"2023-11-26T07:21:03.362988+0000","last_unstale":"2023-11-26T07:21:04.368316+0000","last_undegraded":"2023-11-26T07:21:04.368316+0000","last_fullsized":"2023-11-26T07:21:04.368316+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:51:38.023765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977041+0000","last_change":"2023-11-26T07:21:03.373542+0000","last_active":"2023-11-26T07:21:04.977041+0000","last_peered":"2023-11-26T07:21:04.977041+0000","last_clean":"2023-11-26T07:21:04.977041+0000","last_became_active":"2023-11-26T07:21:03.373142+0000","last_became_peered":"2023-11-26T07:21:03.373142+0000","last_unstale":"2023-11-26T07:21:04.977041+0000","last_undegraded":"2023-11-26T07:21:04.977041+0000","last_fullsized":"2023-11-26T07:21:04.977041+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:42.015742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396605+0000","last_change":"2023-11-26T07:21:03.381010+0000","last_active":"2023-11-26T07:21:04.396605+0000","last_peered":"2023-11-26T07:21:04.396605+0000","last_clean":"2023-11-26T07:21:04.396605+0000","last_became_active":"2023-11-26T07:21:03.380702+0000","last_became_peered":"2023-11-26T07:21:03.380702+0000","last_unstale":"2023-11-26T07:21:04.396605+0000","last_undegraded":"2023-11-26T07:21:04.396605+0000","last_fullsized":"2023-11-26T07:21:04.396605+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:34:48.265696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252799+0000","last_change":"2023-11-26T07:21:03.390617+0000","last_active":"2023-11-26T07:21:05.252799+0000","last_peered":"2023-11-26T07:21:05.252799+0000","last_clean":"2023-11-26T07:21:05.252799+0000","last_became_active":"2023-11-26T07:21:03.390328+0000","last_became_peered":"2023-11-26T07:21:03.390328+0000","last_unstale":"2023-11-26T07:21:05.252799+0000","last_undegraded":"2023-11-26T07:21:05.252799+0000","last_fullsized":"2023-11-26T07:21:05.252799+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:05.628845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364659+0000","last_change":"2023-11-26T07:21:03.368334+0000","last_active":"2023-11-26T07:21:04.364659+0000","last_peered":"2023-11-26T07:21:04.364659+0000","last_clean":"2023-11-26T07:21:04.364659+0000","last_became_active":"2023-11-26T07:21:03.368053+0000","last_became_peered":"2023-11-26T07:21:03.368053+0000","last_unstale":"2023-11-26T07:21:04.364659+0000","last_undegraded":"2023-11-26T07:21:04.364659+0000","last_fullsized":"2023-11-26T07:21:04.364659+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:57:16.891965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396417+0000","last_change":"2023-11-26T07:21:03.387499+0000","last_active":"2023-11-26T07:21:04.396417+0000","last_peered":"2023-11-26T07:21:04.396417+0000","last_clean":"2023-11-26T07:21:04.396417+0000","last_became_active":"2023-11-26T07:21:03.387160+0000","last_became_peered":"2023-11-26T07:21:03.387160+0000","last_unstale":"2023-11-26T07:21:04.396417+0000","last_undegraded":"2023-11-26T07:21:04.396417+0000","last_fullsized":"2023-11-26T07:21:04.396417+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:28:55.054249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252793+0000","last_change":"2023-11-26T07:21:03.364127+0000","last_active":"2023-11-26T07:21:05.252793+0000","last_peered":"2023-11-26T07:21:05.252793+0000","last_clean":"2023-11-26T07:21:05.252793+0000","last_became_active":"2023-11-26T07:21:03.362777+0000","last_became_peered":"2023-11-26T07:21:03.362777+0000","last_unstale":"2023-11-26T07:21:05.252793+0000","last_undegraded":"2023-11-26T07:21:05.252793+0000","last_fullsized":"2023-11-26T07:21:05.252793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:49.720970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396216+0000","last_change":"2023-11-26T07:21:03.372588+0000","last_active":"2023-11-26T07:21:04.396216+0000","last_peered":"2023-11-26T07:21:04.396216+0000","last_clean":"2023-11-26T07:21:04.396216+0000","last_became_active":"2023-11-26T07:21:03.372319+0000","last_became_peered":"2023-11-26T07:21:03.372319+0000","last_unstale":"2023-11-26T07:21:04.396216+0000","last_undegraded":"2023-11-26T07:21:04.396216+0000","last_fullsized":"2023-11-26T07:21:04.396216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:29:37.921422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252226+0000","last_change":"2023-11-26T07:21:03.373638+0000","last_active":"2023-11-26T07:21:05.252226+0000","last_peered":"2023-11-26T07:21:05.252226+0000","last_clean":"2023-11-26T07:21:05.252226+0000","last_became_active":"2023-11-26T07:21:03.373363+0000","last_became_peered":"2023-11-26T07:21:03.373363+0000","last_unstale":"2023-11-26T07:21:05.252226+0000","last_undegraded":"2023-11-26T07:21:05.252226+0000","last_fullsized":"2023-11-26T07:21:05.252226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:18:12.211335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366341+0000","last_change":"2023-11-26T07:21:03.385458+0000","last_active":"2023-11-26T07:21:04.366341+0000","last_peered":"2023-11-26T07:21:04.366341+0000","last_clean":"2023-11-26T07:21:04.366341+0000","last_became_active":"2023-11-26T07:21:03.384962+0000","last_became_peered":"2023-11-26T07:21:03.384962+0000","last_unstale":"2023-11-26T07:21:04.366341+0000","last_undegraded":"2023-11-26T07:21:04.366341+0000","last_fullsized":"2023-11-26T07:21:04.366341+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:16.189058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364886+0000","last_change":"2023-11-26T07:21:03.369176+0000","last_active":"2023-11-26T07:21:04.364886+0000","last_peered":"2023-11-26T07:21:04.364886+0000","last_clean":"2023-11-26T07:21:04.364886+0000","last_became_active":"2023-11-26T07:21:03.368878+0000","last_became_peered":"2023-11-26T07:21:03.368878+0000","last_unstale":"2023-11-26T07:21:04.364886+0000","last_undegraded":"2023-11-26T07:21:04.364886+0000","last_fullsized":"2023-11-26T07:21:04.364886+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:35:31.473926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252049+0000","last_change":"2023-11-26T07:21:03.367364+0000","last_active":"2023-11-26T07:21:05.252049+0000","last_peered":"2023-11-26T07:21:05.252049+0000","last_clean":"2023-11-26T07:21:05.252049+0000","last_became_active":"2023-11-26T07:21:03.367066+0000","last_became_peered":"2023-11-26T07:21:03.367066+0000","last_unstale":"2023-11-26T07:21:05.252049+0000","last_undegraded":"2023-11-26T07:21:05.252049+0000","last_fullsized":"2023-11-26T07:21:05.252049+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:01.130100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977389+0000","last_change":"2023-11-26T07:21:03.384928+0000","last_active":"2023-11-26T07:21:04.977389+0000","last_peered":"2023-11-26T07:21:04.977389+0000","last_clean":"2023-11-26T07:21:04.977389+0000","last_became_active":"2023-11-26T07:21:03.384481+0000","last_became_peered":"2023-11-26T07:21:03.384481+0000","last_unstale":"2023-11-26T07:21:04.977389+0000","last_undegraded":"2023-11-26T07:21:04.977389+0000","last_fullsized":"2023-11-26T07:21:04.977389+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:52:38.839238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364669+0000","last_change":"2023-11-26T07:21:03.363854+0000","last_active":"2023-11-26T07:21:04.364669+0000","last_peered":"2023-11-26T07:21:04.364669+0000","last_clean":"2023-11-26T07:21:04.364669+0000","last_became_active":"2023-11-26T07:21:03.363553+0000","last_became_peered":"2023-11-26T07:21:03.363553+0000","last_unstale":"2023-11-26T07:21:04.364669+0000","last_undegraded":"2023-11-26T07:21:04.364669+0000","last_fullsized":"2023-11-26T07:21:04.364669+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:39:00.811057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396401+0000","last_change":"2023-11-26T07:21:03.379625+0000","last_active":"2023-11-26T07:21:04.396401+0000","last_peered":"2023-11-26T07:21:04.396401+0000","last_clean":"2023-11-26T07:21:04.396401+0000","last_became_active":"2023-11-26T07:21:03.376175+0000","last_became_peered":"2023-11-26T07:21:03.376175+0000","last_unstale":"2023-11-26T07:21:04.396401+0000","last_undegraded":"2023-11-26T07:21:04.396401+0000","last_fullsized":"2023-11-26T07:21:04.396401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:53:14.886903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252491+0000","last_change":"2023-11-26T07:21:03.365468+0000","last_active":"2023-11-26T07:21:05.252491+0000","last_peered":"2023-11-26T07:21:05.252491+0000","last_clean":"2023-11-26T07:21:05.252491+0000","last_became_active":"2023-11-26T07:21:03.365147+0000","last_became_peered":"2023-11-26T07:21:03.365147+0000","last_unstale":"2023-11-26T07:21:05.252491+0000","last_undegraded":"2023-11-26T07:21:05.252491+0000","last_fullsized":"2023-11-26T07:21:05.252491+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:27:35.853867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.364379+0000","last_change":"2023-11-26T07:21:03.369981+0000","last_active":"2023-11-26T07:21:04.364379+0000","last_peered":"2023-11-26T07:21:04.364379+0000","last_clean":"2023-11-26T07:21:04.364379+0000","last_became_active":"2023-11-26T07:21:03.369706+0000","last_became_peered":"2023-11-26T07:21:03.369706+0000","last_unstale":"2023-11-26T07:21:04.364379+0000","last_undegraded":"2023-11-26T07:21:04.364379+0000","last_fullsized":"2023-11-26T07:21:04.364379+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:21.217395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977150+0000","last_change":"2023-11-26T07:21:03.392306+0000","last_active":"2023-11-26T07:21:04.977150+0000","last_peered":"2023-11-26T07:21:04.977150+0000","last_clean":"2023-11-26T07:21:04.977150+0000","last_became_active":"2023-11-26T07:21:03.391824+0000","last_became_peered":"2023-11-26T07:21:03.391824+0000","last_unstale":"2023-11-26T07:21:04.977150+0000","last_undegraded":"2023-11-26T07:21:04.977150+0000","last_fullsized":"2023-11-26T07:21:04.977150+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:01.130100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977522+0000","last_change":"2023-11-26T07:21:03.385658+0000","last_active":"2023-11-26T07:21:04.977522+0000","last_peered":"2023-11-26T07:21:04.977522+0000","last_clean":"2023-11-26T07:21:04.977522+0000","last_became_active":"2023-11-26T07:21:03.385381+0000","last_became_peered":"2023-11-26T07:21:03.385381+0000","last_unstale":"2023-11-26T07:21:04.977522+0000","last_undegraded":"2023-11-26T07:21:04.977522+0000","last_fullsized":"2023-11-26T07:21:04.977522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:30:57.184043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977569+0000","last_change":"2023-11-26T07:21:02.355959+0000","last_active":"2023-11-26T07:21:04.977569+0000","last_peered":"2023-11-26T07:21:04.977569+0000","last_clean":"2023-11-26T07:21:04.977569+0000","last_became_active":"2023-11-26T07:21:02.355075+0000","last_became_peered":"2023-11-26T07:21:02.355075+0000","last_unstale":"2023-11-26T07:21:04.977569+0000","last_undegraded":"2023-11-26T07:21:04.977569+0000","last_fullsized":"2023-11-26T07:21:04.977569+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:19:48.708001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252347+0000","last_change":"2023-11-26T07:21:03.375105+0000","last_active":"2023-11-26T07:21:05.252347+0000","last_peered":"2023-11-26T07:21:05.252347+0000","last_clean":"2023-11-26T07:21:05.252347+0000","last_became_active":"2023-11-26T07:21:03.374701+0000","last_became_peered":"2023-11-26T07:21:03.374701+0000","last_unstale":"2023-11-26T07:21:05.252347+0000","last_undegraded":"2023-11-26T07:21:05.252347+0000","last_fullsized":"2023-11-26T07:21:05.252347+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:26:41.546697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396436+0000","last_change":"2023-11-26T07:21:02.348430+0000","last_active":"2023-11-26T07:21:04.396436+0000","last_peered":"2023-11-26T07:21:04.396436+0000","last_clean":"2023-11-26T07:21:04.396436+0000","last_became_active":"2023-11-26T07:21:02.348140+0000","last_became_peered":"2023-11-26T07:21:02.348140+0000","last_unstale":"2023-11-26T07:21:04.396436+0000","last_undegraded":"2023-11-26T07:21:04.396436+0000","last_fullsized":"2023-11-26T07:21:04.396436+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:13:18.729520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977467+0000","last_change":"2023-11-26T07:21:03.383668+0000","last_active":"2023-11-26T07:21:04.977467+0000","last_peered":"2023-11-26T07:21:04.977467+0000","last_clean":"2023-11-26T07:21:04.977467+0000","last_became_active":"2023-11-26T07:21:03.383392+0000","last_became_peered":"2023-11-26T07:21:03.383392+0000","last_unstale":"2023-11-26T07:21:04.977467+0000","last_undegraded":"2023-11-26T07:21:04.977467+0000","last_fullsized":"2023-11-26T07:21:04.977467+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:05.628845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977565+0000","last_change":"2023-11-26T07:21:02.350619+0000","last_active":"2023-11-26T07:21:04.977565+0000","last_peered":"2023-11-26T07:21:04.977565+0000","last_clean":"2023-11-26T07:21:04.977565+0000","last_became_active":"2023-11-26T07:21:02.350345+0000","last_became_peered":"2023-11-26T07:21:02.350345+0000","last_unstale":"2023-11-26T07:21:04.977565+0000","last_undegraded":"2023-11-26T07:21:04.977565+0000","last_fullsized":"2023-11-26T07:21:04.977565+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:48:21.733306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977400+0000","last_change":"2023-11-26T07:21:03.384746+0000","last_active":"2023-11-26T07:21:04.977400+0000","last_peered":"2023-11-26T07:21:04.977400+0000","last_clean":"2023-11-26T07:21:04.977400+0000","last_became_active":"2023-11-26T07:21:03.384155+0000","last_became_peered":"2023-11-26T07:21:03.384155+0000","last_unstale":"2023-11-26T07:21:04.977400+0000","last_undegraded":"2023-11-26T07:21:04.977400+0000","last_fullsized":"2023-11-26T07:21:04.977400+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:49.720970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252418+0000","last_change":"2023-11-26T07:21:02.347495+0000","last_active":"2023-11-26T07:21:05.252418+0000","last_peered":"2023-11-26T07:21:05.252418+0000","last_clean":"2023-11-26T07:21:05.252418+0000","last_became_active":"2023-11-26T07:21:02.347214+0000","last_became_peered":"2023-11-26T07:21:02.347214+0000","last_unstale":"2023-11-26T07:21:05.252418+0000","last_undegraded":"2023-11-26T07:21:05.252418+0000","last_fullsized":"2023-11-26T07:21:05.252418+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:50:08.586552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396056+0000","last_change":"2023-11-26T07:21:03.371745+0000","last_active":"2023-11-26T07:21:04.396056+0000","last_peered":"2023-11-26T07:21:04.396056+0000","last_clean":"2023-11-26T07:21:04.396056+0000","last_became_active":"2023-11-26T07:21:03.371443+0000","last_became_peered":"2023-11-26T07:21:03.371443+0000","last_unstale":"2023-11-26T07:21:04.396056+0000","last_undegraded":"2023-11-26T07:21:04.396056+0000","last_fullsized":"2023-11-26T07:21:04.396056+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:18:29.519759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368332+0000","last_change":"2023-11-26T07:21:02.352669+0000","last_active":"2023-11-26T07:21:04.368332+0000","last_peered":"2023-11-26T07:21:04.368332+0000","last_clean":"2023-11-26T07:21:04.368332+0000","last_became_active":"2023-11-26T07:21:02.352409+0000","last_became_peered":"2023-11-26T07:21:02.352409+0000","last_unstale":"2023-11-26T07:21:04.368332+0000","last_undegraded":"2023-11-26T07:21:04.368332+0000","last_fullsized":"2023-11-26T07:21:04.368332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:06:05.237512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396244+0000","last_change":"2023-11-26T07:21:03.388999+0000","last_active":"2023-11-26T07:21:04.396244+0000","last_peered":"2023-11-26T07:21:04.396244+0000","last_clean":"2023-11-26T07:21:04.396244+0000","last_became_active":"2023-11-26T07:21:03.388752+0000","last_became_peered":"2023-11-26T07:21:03.388752+0000","last_unstale":"2023-11-26T07:21:04.396244+0000","last_undegraded":"2023-11-26T07:21:04.396244+0000","last_fullsized":"2023-11-26T07:21:04.396244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:33:58.718469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977198+0000","last_change":"2023-11-26T07:21:02.345184+0000","last_active":"2023-11-26T07:21:04.977198+0000","last_peered":"2023-11-26T07:21:04.977198+0000","last_clean":"2023-11-26T07:21:04.977198+0000","last_became_active":"2023-11-26T07:21:02.344821+0000","last_became_peered":"2023-11-26T07:21:02.344821+0000","last_unstale":"2023-11-26T07:21:04.977198+0000","last_undegraded":"2023-11-26T07:21:04.977198+0000","last_fullsized":"2023-11-26T07:21:04.977198+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:46:38.191366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366454+0000","last_change":"2023-11-26T07:21:03.359640+0000","last_active":"2023-11-26T07:21:04.366454+0000","last_peered":"2023-11-26T07:21:04.366454+0000","last_clean":"2023-11-26T07:21:04.366454+0000","last_became_active":"2023-11-26T07:21:03.359361+0000","last_became_peered":"2023-11-26T07:21:03.359361+0000","last_unstale":"2023-11-26T07:21:04.366454+0000","last_undegraded":"2023-11-26T07:21:04.366454+0000","last_fullsized":"2023-11-26T07:21:04.366454+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:46:29.111583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251853+0000","last_change":"2023-11-26T07:21:02.345247+0000","last_active":"2023-11-26T07:21:05.251853+0000","last_peered":"2023-11-26T07:21:05.251853+0000","last_clean":"2023-11-26T07:21:05.251853+0000","last_became_active":"2023-11-26T07:21:02.344934+0000","last_became_peered":"2023-11-26T07:21:02.344934+0000","last_unstale":"2023-11-26T07:21:05.251853+0000","last_undegraded":"2023-11-26T07:21:05.251853+0000","last_fullsized":"2023-11-26T07:21:05.251853+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:48:44.815091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396063+0000","last_change":"2023-11-26T07:21:03.378854+0000","last_active":"2023-11-26T07:21:04.396063+0000","last_peered":"2023-11-26T07:21:04.396063+0000","last_clean":"2023-11-26T07:21:04.396063+0000","last_became_active":"2023-11-26T07:21:03.378578+0000","last_became_peered":"2023-11-26T07:21:03.378578+0000","last_unstale":"2023-11-26T07:21:04.396063+0000","last_undegraded":"2023-11-26T07:21:04.396063+0000","last_fullsized":"2023-11-26T07:21:04.396063+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:38:13.589648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368121+0000","last_change":"2023-11-26T07:21:02.351579+0000","last_active":"2023-11-26T07:21:04.368121+0000","last_peered":"2023-11-26T07:21:04.368121+0000","last_clean":"2023-11-26T07:21:04.368121+0000","last_became_active":"2023-11-26T07:21:02.351203+0000","last_became_peered":"2023-11-26T07:21:02.351203+0000","last_unstale":"2023-11-26T07:21:04.368121+0000","last_undegraded":"2023-11-26T07:21:04.368121+0000","last_fullsized":"2023-11-26T07:21:04.368121+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:46:28.105363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368724+0000","last_change":"2023-11-26T07:21:03.362630+0000","last_active":"2023-11-26T07:21:04.368724+0000","last_peered":"2023-11-26T07:21:04.368724+0000","last_clean":"2023-11-26T07:21:04.368724+0000","last_became_active":"2023-11-26T07:21:03.362301+0000","last_became_peered":"2023-11-26T07:21:03.362301+0000","last_unstale":"2023-11-26T07:21:04.368724+0000","last_undegraded":"2023-11-26T07:21:04.368724+0000","last_fullsized":"2023-11-26T07:21:04.368724+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:09:36.512866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.253072+0000","last_change":"2023-11-26T07:21:02.350168+0000","last_active":"2023-11-26T07:21:05.253072+0000","last_peered":"2023-11-26T07:21:05.253072+0000","last_clean":"2023-11-26T07:21:05.253072+0000","last_became_active":"2023-11-26T07:21:02.349846+0000","last_became_peered":"2023-11-26T07:21:02.349846+0000","last_unstale":"2023-11-26T07:21:05.253072+0000","last_undegraded":"2023-11-26T07:21:05.253072+0000","last_fullsized":"2023-11-26T07:21:05.253072+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:12:25.647531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.253053+0000","last_change":"2023-11-26T07:21:03.366130+0000","last_active":"2023-11-26T07:21:05.253053+0000","last_peered":"2023-11-26T07:21:05.253053+0000","last_clean":"2023-11-26T07:21:05.253053+0000","last_became_active":"2023-11-26T07:21:03.365855+0000","last_became_peered":"2023-11-26T07:21:03.365855+0000","last_unstale":"2023-11-26T07:21:05.253053+0000","last_undegraded":"2023-11-26T07:21:05.253053+0000","last_fullsized":"2023-11-26T07:21:05.253053+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:57.566419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396513+0000","last_change":"2023-11-26T07:21:02.358713+0000","last_active":"2023-11-26T07:21:04.396513+0000","last_peered":"2023-11-26T07:21:04.396513+0000","last_clean":"2023-11-26T07:21:04.396513+0000","last_became_active":"2023-11-26T07:21:02.358441+0000","last_became_peered":"2023-11-26T07:21:02.358441+0000","last_unstale":"2023-11-26T07:21:04.396513+0000","last_undegraded":"2023-11-26T07:21:04.396513+0000","last_fullsized":"2023-11-26T07:21:04.396513+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:29:29.613665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252203+0000","last_change":"2023-11-26T07:21:03.372076+0000","last_active":"2023-11-26T07:21:05.252203+0000","last_peered":"2023-11-26T07:21:05.252203+0000","last_clean":"2023-11-26T07:21:05.252203+0000","last_became_active":"2023-11-26T07:21:03.371656+0000","last_became_peered":"2023-11-26T07:21:03.371656+0000","last_unstale":"2023-11-26T07:21:05.252203+0000","last_undegraded":"2023-11-26T07:21:05.252203+0000","last_fullsized":"2023-11-26T07:21:05.252203+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:57:04.036866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252993+0000","last_change":"2023-11-26T07:21:02.350387+0000","last_active":"2023-11-26T07:21:05.252993+0000","last_peered":"2023-11-26T07:21:05.252993+0000","last_clean":"2023-11-26T07:21:05.252993+0000","last_became_active":"2023-11-26T07:21:02.350099+0000","last_became_peered":"2023-11-26T07:21:02.350099+0000","last_unstale":"2023-11-26T07:21:05.252993+0000","last_undegraded":"2023-11-26T07:21:05.252993+0000","last_fullsized":"2023-11-26T07:21:05.252993+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:40:17.484190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.253010+0000","last_change":"2023-11-26T07:21:03.368500+0000","last_active":"2023-11-26T07:21:05.253010+0000","last_peered":"2023-11-26T07:21:05.253010+0000","last_clean":"2023-11-26T07:21:05.253010+0000","last_became_active":"2023-11-26T07:21:03.368222+0000","last_became_peered":"2023-11-26T07:21:03.368222+0000","last_unstale":"2023-11-26T07:21:05.253010+0000","last_undegraded":"2023-11-26T07:21:05.253010+0000","last_fullsized":"2023-11-26T07:21:05.253010+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:02.820690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368585+0000","last_change":"2023-11-26T07:21:02.351440+0000","last_active":"2023-11-26T07:21:04.368585+0000","last_peered":"2023-11-26T07:21:04.368585+0000","last_clean":"2023-11-26T07:21:04.368585+0000","last_became_active":"2023-11-26T07:21:02.350504+0000","last_became_peered":"2023-11-26T07:21:02.350504+0000","last_unstale":"2023-11-26T07:21:04.368585+0000","last_undegraded":"2023-11-26T07:21:04.368585+0000","last_fullsized":"2023-11-26T07:21:04.368585+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:26:40.540477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251811+0000","last_change":"2023-11-26T07:21:03.367583+0000","last_active":"2023-11-26T07:21:05.251811+0000","last_peered":"2023-11-26T07:21:05.251811+0000","last_clean":"2023-11-26T07:21:05.251811+0000","last_became_active":"2023-11-26T07:21:03.367210+0000","last_became_peered":"2023-11-26T07:21:03.367210+0000","last_unstale":"2023-11-26T07:21:05.251811+0000","last_undegraded":"2023-11-26T07:21:05.251811+0000","last_fullsized":"2023-11-26T07:21:05.251811+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:51:42.015742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396246+0000","last_change":"2023-11-26T07:21:02.359252+0000","last_active":"2023-11-26T07:21:04.396246+0000","last_peered":"2023-11-26T07:21:04.396246+0000","last_clean":"2023-11-26T07:21:04.396246+0000","last_became_active":"2023-11-26T07:21:02.358217+0000","last_became_peered":"2023-11-26T07:21:02.358217+0000","last_unstale":"2023-11-26T07:21:04.396246+0000","last_undegraded":"2023-11-26T07:21:04.396246+0000","last_fullsized":"2023-11-26T07:21:04.396246+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:01.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252688+0000","last_change":"2023-11-26T07:21:03.364704+0000","last_active":"2023-11-26T07:21:05.252688+0000","last_peered":"2023-11-26T07:21:05.252688+0000","last_clean":"2023-11-26T07:21:05.252688+0000","last_became_active":"2023-11-26T07:21:03.364416+0000","last_became_peered":"2023-11-26T07:21:03.364416+0000","last_unstale":"2023-11-26T07:21:05.252688+0000","last_undegraded":"2023-11-26T07:21:05.252688+0000","last_fullsized":"2023-11-26T07:21:05.252688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:52:38.839238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366408+0000","last_change":"2023-11-26T07:21:02.349014+0000","last_active":"2023-11-26T07:21:04.366408+0000","last_peered":"2023-11-26T07:21:04.366408+0000","last_clean":"2023-11-26T07:21:04.366408+0000","last_became_active":"2023-11-26T07:21:02.348699+0000","last_became_peered":"2023-11-26T07:21:02.348699+0000","last_unstale":"2023-11-26T07:21:04.366408+0000","last_undegraded":"2023-11-26T07:21:04.366408+0000","last_fullsized":"2023-11-26T07:21:04.366408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:11:37.171832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368122+0000","last_change":"2023-11-26T07:21:03.355702+0000","last_active":"2023-11-26T07:21:04.368122+0000","last_peered":"2023-11-26T07:21:04.368122+0000","last_clean":"2023-11-26T07:21:04.368122+0000","last_became_active":"2023-11-26T07:21:03.355386+0000","last_became_peered":"2023-11-26T07:21:03.355386+0000","last_unstale":"2023-11-26T07:21:04.368122+0000","last_undegraded":"2023-11-26T07:21:04.368122+0000","last_fullsized":"2023-11-26T07:21:04.368122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:36:24.491317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251760+0000","last_change":"2023-11-26T07:21:02.351084+0000","last_active":"2023-11-26T07:21:05.251760+0000","last_peered":"2023-11-26T07:21:05.251760+0000","last_clean":"2023-11-26T07:21:05.251760+0000","last_became_active":"2023-11-26T07:21:02.350796+0000","last_became_peered":"2023-11-26T07:21:02.350796+0000","last_unstale":"2023-11-26T07:21:05.251760+0000","last_undegraded":"2023-11-26T07:21:05.251760+0000","last_fullsized":"2023-11-26T07:21:05.251760+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:19:48.708001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252385+0000","last_change":"2023-11-26T07:21:03.373608+0000","last_active":"2023-11-26T07:21:05.252385+0000","last_peered":"2023-11-26T07:21:05.252385+0000","last_clean":"2023-11-26T07:21:05.252385+0000","last_became_active":"2023-11-26T07:21:03.373277+0000","last_became_peered":"2023-11-26T07:21:03.373277+0000","last_unstale":"2023-11-26T07:21:05.252385+0000","last_undegraded":"2023-11-26T07:21:05.252385+0000","last_fullsized":"2023-11-26T07:21:05.252385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:01.130100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251802+0000","last_change":"2023-11-26T07:21:02.352787+0000","last_active":"2023-11-26T07:21:05.251802+0000","last_peered":"2023-11-26T07:21:05.251802+0000","last_clean":"2023-11-26T07:21:05.251802+0000","last_became_active":"2023-11-26T07:21:02.352517+0000","last_became_peered":"2023-11-26T07:21:02.352517+0000","last_unstale":"2023-11-26T07:21:05.251802+0000","last_undegraded":"2023-11-26T07:21:05.251802+0000","last_fullsized":"2023-11-26T07:21:05.251802+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:51:37.017545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252852+0000","last_change":"2023-11-26T07:21:03.365379+0000","last_active":"2023-11-26T07:21:05.252852+0000","last_peered":"2023-11-26T07:21:05.252852+0000","last_clean":"2023-11-26T07:21:05.252852+0000","last_became_active":"2023-11-26T07:21:03.365089+0000","last_became_peered":"2023-11-26T07:21:03.365089+0000","last_unstale":"2023-11-26T07:21:05.252852+0000","last_undegraded":"2023-11-26T07:21:05.252852+0000","last_fullsized":"2023-11-26T07:21:05.252852+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:57.039182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396587+0000","last_change":"2023-11-26T07:21:02.359000+0000","last_active":"2023-11-26T07:21:04.396587+0000","last_peered":"2023-11-26T07:21:04.396587+0000","last_clean":"2023-11-26T07:21:04.396587+0000","last_became_active":"2023-11-26T07:21:02.358717+0000","last_became_peered":"2023-11-26T07:21:02.358717+0000","last_unstale":"2023-11-26T07:21:04.396587+0000","last_undegraded":"2023-11-26T07:21:04.396587+0000","last_fullsized":"2023-11-26T07:21:04.396587+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:56.032962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252717+0000","last_change":"2023-11-26T07:21:03.367661+0000","last_active":"2023-11-26T07:21:05.252717+0000","last_peered":"2023-11-26T07:21:05.252717+0000","last_clean":"2023-11-26T07:21:05.252717+0000","last_became_active":"2023-11-26T07:21:03.367381+0000","last_became_peered":"2023-11-26T07:21:03.367381+0000","last_unstale":"2023-11-26T07:21:05.252717+0000","last_undegraded":"2023-11-26T07:21:05.252717+0000","last_fullsized":"2023-11-26T07:21:05.252717+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:03:49.720970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977597+0000","last_change":"2023-11-26T07:21:02.358649+0000","last_active":"2023-11-26T07:21:04.977597+0000","last_peered":"2023-11-26T07:21:04.977597+0000","last_clean":"2023-11-26T07:21:04.977597+0000","last_became_active":"2023-11-26T07:21:02.358346+0000","last_became_peered":"2023-11-26T07:21:02.358346+0000","last_unstale":"2023-11-26T07:21:04.977597+0000","last_undegraded":"2023-11-26T07:21:04.977597+0000","last_fullsized":"2023-11-26T07:21:04.977597+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:48:44.815091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396398+0000","last_change":"2023-11-26T07:21:03.367409+0000","last_active":"2023-11-26T07:21:04.396398+0000","last_peered":"2023-11-26T07:21:04.396398+0000","last_clean":"2023-11-26T07:21:04.396398+0000","last_became_active":"2023-11-26T07:21:03.367094+0000","last_became_peered":"2023-11-26T07:21:03.367094+0000","last_unstale":"2023-11-26T07:21:04.396398+0000","last_undegraded":"2023-11-26T07:21:04.396398+0000","last_fullsized":"2023-11-26T07:21:04.396398+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:25.313667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252114+0000","last_change":"2023-11-26T07:21:02.345391+0000","last_active":"2023-11-26T07:21:05.252114+0000","last_peered":"2023-11-26T07:21:05.252114+0000","last_clean":"2023-11-26T07:21:05.252114+0000","last_became_active":"2023-11-26T07:21:02.345091+0000","last_became_peered":"2023-11-26T07:21:02.345091+0000","last_unstale":"2023-11-26T07:21:05.252114+0000","last_undegraded":"2023-11-26T07:21:05.252114+0000","last_fullsized":"2023-11-26T07:21:05.252114+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:48:21.733306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252901+0000","last_change":"2023-11-26T07:21:03.364513+0000","last_active":"2023-11-26T07:21:05.252901+0000","last_peered":"2023-11-26T07:21:05.252901+0000","last_clean":"2023-11-26T07:21:05.252901+0000","last_became_active":"2023-11-26T07:21:03.361707+0000","last_became_peered":"2023-11-26T07:21:03.361707+0000","last_unstale":"2023-11-26T07:21:05.252901+0000","last_undegraded":"2023-11-26T07:21:05.252901+0000","last_fullsized":"2023-11-26T07:21:05.252901+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:23:44.853586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977736+0000","last_change":"2023-11-26T07:21:02.351812+0000","last_active":"2023-11-26T07:21:04.977736+0000","last_peered":"2023-11-26T07:21:04.977736+0000","last_clean":"2023-11-26T07:21:04.977736+0000","last_became_active":"2023-11-26T07:21:02.351518+0000","last_became_peered":"2023-11-26T07:21:02.351518+0000","last_unstale":"2023-11-26T07:21:04.977736+0000","last_undegraded":"2023-11-26T07:21:04.977736+0000","last_fullsized":"2023-11-26T07:21:04.977736+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:12:25.647531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251969+0000","last_change":"2023-11-26T07:21:03.367980+0000","last_active":"2023-11-26T07:21:05.251969+0000","last_peered":"2023-11-26T07:21:05.251969+0000","last_clean":"2023-11-26T07:21:05.251969+0000","last_became_active":"2023-11-26T07:21:03.367645+0000","last_became_peered":"2023-11-26T07:21:03.367645+0000","last_unstale":"2023-11-26T07:21:05.251969+0000","last_undegraded":"2023-11-26T07:21:05.251969+0000","last_fullsized":"2023-11-26T07:21:05.251969+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:18:12.211335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396374+0000","last_change":"2023-11-26T07:21:02.349587+0000","last_active":"2023-11-26T07:21:04.396374+0000","last_peered":"2023-11-26T07:21:04.396374+0000","last_clean":"2023-11-26T07:21:04.396374+0000","last_became_active":"2023-11-26T07:21:02.349322+0000","last_became_peered":"2023-11-26T07:21:02.349322+0000","last_unstale":"2023-11-26T07:21:04.396374+0000","last_undegraded":"2023-11-26T07:21:04.396374+0000","last_fullsized":"2023-11-26T07:21:04.396374+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:57:56.560199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977267+0000","last_change":"2023-11-26T07:21:03.383400+0000","last_active":"2023-11-26T07:21:04.977267+0000","last_peered":"2023-11-26T07:21:04.977267+0000","last_clean":"2023-11-26T07:21:04.977267+0000","last_became_active":"2023-11-26T07:21:03.382982+0000","last_became_peered":"2023-11-26T07:21:03.382982+0000","last_unstale":"2023-11-26T07:21:04.977267+0000","last_undegraded":"2023-11-26T07:21:04.977267+0000","last_fullsized":"2023-11-26T07:21:04.977267+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:27:35.853867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252031+0000","last_change":"2023-11-26T07:21:02.359603+0000","last_active":"2023-11-26T07:21:05.252031+0000","last_peered":"2023-11-26T07:21:05.252031+0000","last_clean":"2023-11-26T07:21:05.252031+0000","last_became_active":"2023-11-26T07:21:02.359312+0000","last_became_peered":"2023-11-26T07:21:02.359312+0000","last_unstale":"2023-11-26T07:21:05.252031+0000","last_undegraded":"2023-11-26T07:21:05.252031+0000","last_fullsized":"2023-11-26T07:21:05.252031+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:56:55.153902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977032+0000","last_change":"2023-11-26T07:21:03.373309+0000","last_active":"2023-11-26T07:21:04.977032+0000","last_peered":"2023-11-26T07:21:04.977032+0000","last_clean":"2023-11-26T07:21:04.977032+0000","last_became_active":"2023-11-26T07:21:03.372872+0000","last_became_peered":"2023-11-26T07:21:03.372872+0000","last_unstale":"2023-11-26T07:21:04.977032+0000","last_undegraded":"2023-11-26T07:21:04.977032+0000","last_fullsized":"2023-11-26T07:21:04.977032+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:27:25.274024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251635+0000","last_change":"2023-11-26T07:21:02.347554+0000","last_active":"2023-11-26T07:21:05.251635+0000","last_peered":"2023-11-26T07:21:05.251635+0000","last_clean":"2023-11-26T07:21:05.251635+0000","last_became_active":"2023-11-26T07:21:02.347273+0000","last_became_peered":"2023-11-26T07:21:02.347273+0000","last_unstale":"2023-11-26T07:21:05.251635+0000","last_undegraded":"2023-11-26T07:21:05.251635+0000","last_fullsized":"2023-11-26T07:21:05.251635+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:15:20.377323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252172+0000","last_change":"2023-11-26T07:21:03.362042+0000","last_active":"2023-11-26T07:21:05.252172+0000","last_peered":"2023-11-26T07:21:05.252172+0000","last_clean":"2023-11-26T07:21:05.252172+0000","last_became_active":"2023-11-26T07:21:03.361727+0000","last_became_peered":"2023-11-26T07:21:03.361727+0000","last_unstale":"2023-11-26T07:21:05.252172+0000","last_undegraded":"2023-11-26T07:21:05.252172+0000","last_fullsized":"2023-11-26T07:21:05.252172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:27:35.853867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252408+0000","last_change":"2023-11-26T07:21:02.355296+0000","last_active":"2023-11-26T07:21:05.252408+0000","last_peered":"2023-11-26T07:21:05.252408+0000","last_clean":"2023-11-26T07:21:05.252408+0000","last_became_active":"2023-11-26T07:21:02.354680+0000","last_became_peered":"2023-11-26T07:21:02.354680+0000","last_unstale":"2023-11-26T07:21:05.252408+0000","last_undegraded":"2023-11-26T07:21:05.252408+0000","last_fullsized":"2023-11-26T07:21:05.252408+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:57:15.182838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252849+0000","last_change":"2023-11-26T07:21:03.363559+0000","last_active":"2023-11-26T07:21:05.252849+0000","last_peered":"2023-11-26T07:21:05.252849+0000","last_clean":"2023-11-26T07:21:05.252849+0000","last_became_active":"2023-11-26T07:21:03.363125+0000","last_became_peered":"2023-11-26T07:21:03.363125+0000","last_unstale":"2023-11-26T07:21:05.252849+0000","last_undegraded":"2023-11-26T07:21:05.252849+0000","last_fullsized":"2023-11-26T07:21:05.252849+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:05.628845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977636+0000","last_change":"2023-11-26T07:21:02.345125+0000","last_active":"2023-11-26T07:21:04.977636+0000","last_peered":"2023-11-26T07:21:04.977636+0000","last_clean":"2023-11-26T07:21:04.977636+0000","last_became_active":"2023-11-26T07:21:02.344732+0000","last_became_peered":"2023-11-26T07:21:02.344732+0000","last_unstale":"2023-11-26T07:21:04.977636+0000","last_undegraded":"2023-11-26T07:21:04.977636+0000","last_fullsized":"2023-11-26T07:21:04.977636+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:50:08.586552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252315+0000","last_change":"2023-11-26T07:21:03.371605+0000","last_active":"2023-11-26T07:21:05.252315+0000","last_peered":"2023-11-26T07:21:05.252315+0000","last_clean":"2023-11-26T07:21:05.252315+0000","last_became_active":"2023-11-26T07:21:03.371267+0000","last_became_peered":"2023-11-26T07:21:03.371267+0000","last_unstale":"2023-11-26T07:21:05.252315+0000","last_undegraded":"2023-11-26T07:21:05.252315+0000","last_fullsized":"2023-11-26T07:21:05.252315+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:08:09.640208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366580+0000","last_change":"2023-11-26T07:21:02.346301+0000","last_active":"2023-11-26T07:21:04.366580+0000","last_peered":"2023-11-26T07:21:04.366580+0000","last_clean":"2023-11-26T07:21:04.366580+0000","last_became_active":"2023-11-26T07:21:02.345981+0000","last_became_peered":"2023-11-26T07:21:02.345981+0000","last_unstale":"2023-11-26T07:21:04.366580+0000","last_undegraded":"2023-11-26T07:21:04.366580+0000","last_fullsized":"2023-11-26T07:21:04.366580+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:56:55.153902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.977658+0000","last_change":"2023-11-26T07:21:03.386337+0000","last_active":"2023-11-26T07:21:04.977658+0000","last_peered":"2023-11-26T07:21:04.977658+0000","last_clean":"2023-11-26T07:21:04.977658+0000","last_became_active":"2023-11-26T07:21:03.386053+0000","last_became_peered":"2023-11-26T07:21:03.386053+0000","last_unstale":"2023-11-26T07:21:04.977658+0000","last_undegraded":"2023-11-26T07:21:04.977658+0000","last_fullsized":"2023-11-26T07:21:04.977658+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:58.898933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252467+0000","last_change":"2023-11-26T07:21:02.346915+0000","last_active":"2023-11-26T07:21:05.252467+0000","last_peered":"2023-11-26T07:21:05.252467+0000","last_clean":"2023-11-26T07:21:05.252467+0000","last_became_active":"2023-11-26T07:21:02.346640+0000","last_became_peered":"2023-11-26T07:21:02.346640+0000","last_unstale":"2023-11-26T07:21:05.252467+0000","last_undegraded":"2023-11-26T07:21:05.252467+0000","last_fullsized":"2023-11-26T07:21:05.252467+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:40:17.484190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396201+0000","last_change":"2023-11-26T07:21:03.375636+0000","last_active":"2023-11-26T07:21:04.396201+0000","last_peered":"2023-11-26T07:21:04.396201+0000","last_clean":"2023-11-26T07:21:04.396201+0000","last_became_active":"2023-11-26T07:21:03.375470+0000","last_became_peered":"2023-11-26T07:21:03.375470+0000","last_unstale":"2023-11-26T07:21:04.396201+0000","last_undegraded":"2023-11-26T07:21:04.396201+0000","last_fullsized":"2023-11-26T07:21:04.396201+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:45:49.937288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251990+0000","last_change":"2023-11-26T07:21:02.356049+0000","last_active":"2023-11-26T07:21:05.251990+0000","last_peered":"2023-11-26T07:21:05.251990+0000","last_clean":"2023-11-26T07:21:05.251990+0000","last_became_active":"2023-11-26T07:21:02.355685+0000","last_became_peered":"2023-11-26T07:21:02.355685+0000","last_unstale":"2023-11-26T07:21:05.251990+0000","last_undegraded":"2023-11-26T07:21:05.251990+0000","last_fullsized":"2023-11-26T07:21:05.251990+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:11:37.171832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.252839+0000","last_change":"2023-11-26T07:21:03.379311+0000","last_active":"2023-11-26T07:21:05.252839+0000","last_peered":"2023-11-26T07:21:05.252839+0000","last_clean":"2023-11-26T07:21:05.252839+0000","last_became_active":"2023-11-26T07:21:03.379010+0000","last_became_peered":"2023-11-26T07:21:03.379010+0000","last_unstale":"2023-11-26T07:21:05.252839+0000","last_undegraded":"2023-11-26T07:21:05.252839+0000","last_fullsized":"2023-11-26T07:21:05.252839+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:06:58.898933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.368262+0000","last_change":"2023-11-26T07:21:02.353249+0000","last_active":"2023-11-26T07:21:04.368262+0000","last_peered":"2023-11-26T07:21:04.368262+0000","last_clean":"2023-11-26T07:21:04.368262+0000","last_became_active":"2023-11-26T07:21:02.352948+0000","last_became_peered":"2023-11-26T07:21:02.352948+0000","last_unstale":"2023-11-26T07:21:04.368262+0000","last_undegraded":"2023-11-26T07:21:04.368262+0000","last_fullsized":"2023-11-26T07:21:04.368262+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:51:53.928403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251853+0000","last_change":"2023-11-26T07:21:03.374236+0000","last_active":"2023-11-26T07:21:05.251853+0000","last_peered":"2023-11-26T07:21:05.251853+0000","last_clean":"2023-11-26T07:21:05.251853+0000","last_became_active":"2023-11-26T07:21:03.373912+0000","last_became_peered":"2023-11-26T07:21:03.373912+0000","last_unstale":"2023-11-26T07:21:05.251853+0000","last_undegraded":"2023-11-26T07:21:05.251853+0000","last_fullsized":"2023-11-26T07:21:05.251853+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:46:29.111583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:05.251883+0000","last_change":"2023-11-26T07:21:02.351786+0000","last_active":"2023-11-26T07:21:05.251883+0000","last_peered":"2023-11-26T07:21:05.251883+0000","last_clean":"2023-11-26T07:21:05.251883+0000","last_became_active":"2023-11-26T07:21:02.351507+0000","last_became_peered":"2023-11-26T07:21:02.351507+0000","last_unstale":"2023-11-26T07:21:05.251883+0000","last_undegraded":"2023-11-26T07:21:05.251883+0000","last_fullsized":"2023-11-26T07:21:05.251883+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:04:13.358930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396092+0000","last_change":"2023-11-26T07:21:03.379423+0000","last_active":"2023-11-26T07:21:04.396092+0000","last_peered":"2023-11-26T07:21:04.396092+0000","last_clean":"2023-11-26T07:21:04.396092+0000","last_became_active":"2023-11-26T07:21:03.378829+0000","last_became_peered":"2023-11-26T07:21:03.378829+0000","last_unstale":"2023-11-26T07:21:04.396092+0000","last_undegraded":"2023-11-26T07:21:04.396092+0000","last_fullsized":"2023-11-26T07:21:04.396092+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:06:15.219155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.396086+0000","last_change":"2023-11-26T07:21:02.352891+0000","last_active":"2023-11-26T07:21:04.396086+0000","last_peered":"2023-11-26T07:21:04.396086+0000","last_clean":"2023-11-26T07:21:04.396086+0000","last_became_active":"2023-11-26T07:21:02.352704+0000","last_became_peered":"2023-11-26T07:21:02.352704+0000","last_unstale":"2023-11-26T07:21:04.396086+0000","last_undegraded":"2023-11-26T07:21:04.396086+0000","last_fullsized":"2023-11-26T07:21:04.396086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:54:36.459993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.395974+0000","last_change":"2023-11-26T07:21:03.387578+0000","last_active":"2023-11-26T07:21:04.395974+0000","last_peered":"2023-11-26T07:21:04.395974+0000","last_clean":"2023-11-26T07:21:04.395974+0000","last_became_active":"2023-11-26T07:21:03.387273+0000","last_became_peered":"2023-11-26T07:21:03.387273+0000","last_unstale":"2023-11-26T07:21:04.395974+0000","last_undegraded":"2023-11-26T07:21:04.395974+0000","last_fullsized":"2023-11-26T07:21:04.395974+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:02.321976+0000","last_clean_scrub_stamp":"2023-11-26T07:21:02.321976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:43:54.528101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-26T07:21:04.366332+0000","last_change":"2023-11-26T07:21:02.352401+0000","last_active":"2023-11-26T07:21:04.366332+0000","last_peered":"2023-11-26T07:21:04.366332+0000","last_clean":"2023-11-26T07:21:04.366332+0000","last_became_active":"2023-11-26T07:21:02.352137+0000","last_became_peered":"2023-11-26T07:21:02.352137+0000","last_unstale":"2023-11-26T07:21:04.366332+0000","last_undegraded":"2023-11-26T07:21:04.366332+0000","last_fullsized":"2023-11-26T07:21:04.366332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:01.315756+0000","last_clean_scrub_stamp":"2023-11-26T07:21:01.315756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:32:24.841087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-26T07:21:03.350205+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:21:03.350205+0000","last_peered":"2023-11-26T07:21:03.350205+0000","last_clean":"2023-11-26T07:21:03.350205+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:21:03.350205+0000","last_undegraded":"2023-11-26T07:21:03.350205+0000","last_fullsized":"2023-11-26T07:21:03.350205+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":875532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.51000000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.92200000000000004,"5min":0.995,"15min":0.995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67300000000000004,"15min":0.67200000000000004},"min":{"1min":0.48599999999999999,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94199999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68200000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.50700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.98799999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70199999999999996,"15min":0.71599999999999997},"min":{"1min":0.54800000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86799999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.66400000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.55800000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.90400000000000003,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.879},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92400000000000004,"5min":1.016,"15min":1.016},"last":0.72199999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.95199999999999996,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.80400000000000005,"15min":0.82199999999999995},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.97999999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.79200000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.67400000000000004,"15min":0.71799999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.66200000000000003,"15min":0.69399999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.81399999999999995,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.58499999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.753,"15min":0.78100000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.83199999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.81000000000000005,"15min":0.85599999999999998},"min":{"1min":0.51800000000000002,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94799999999999995,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.73499999999999999,"15min":0.76700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88200000000000001,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.80300000000000005,"15min":0.83499999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88500000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.69899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":684032,"data_stored":285004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.45700000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97799999999999998,"5min":2.492,"15min":2.492},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71499999999999997,"15min":0.73799999999999999},"min":{"1min":0.49099999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.89900000000000002,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":0.73599999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.749},"min":{"1min":0.504,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92000000000000004,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.78200000000000003,"15min":0.81599999999999995},"min":{"1min":0.47299999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91600000000000004,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.97499999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.73299999999999998,"15min":0.745},"min":{"1min":0.60399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95499999999999996,"5min":1.677,"15min":1.677},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.67800000000000005,"15min":0.67300000000000004},"min":{"1min":0.54400000000000004,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":1.367,"15min":1.367},"last":0.78100000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.62,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95799999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.79200000000000004,"15min":0.82299999999999995},"min":{"1min":0.52800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.874,"5min":2.9340000000000002,"15min":2.9340000000000002},"last":0.91800000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.63800000000000001,"15min":0.64700000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89500000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.751},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.69399999999999995,"15min":0.72199999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.85499999999999998,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.68999999999999995,"15min":0.72399999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.83399999999999996,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.751,"15min":0.78600000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97699999999999998,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.80300000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.75600000000000001,"15min":0.78800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.93899999999999995,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.754,"15min":0.78400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.86299999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.71499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":285656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.60799999999999998,"15min":0.58099999999999996},"min":{"1min":0.56200000000000006,"5min":0.316,"15min":0.316},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.49299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.44400000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.55800000000000005,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.51000000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.018,"5min":1.036,"15min":1.036},"last":0.54500000000000004}]},{"osd":2,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.60299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91700000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.54400000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.54100000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.93799999999999994,"5min":1.071,"15min":1.071},"last":0.57599999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.66900000000000004,"15min":0.69199999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83999999999999997,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.40500000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.0569999999999999,"5min":1.206,"15min":1.206},"last":0.68000000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.64300000000000002,"15min":0.67800000000000005},"min":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80800000000000005,"5min":1.143,"15min":1.143},"last":0.628},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.70899999999999996,"15min":0.73899999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.032,"5min":1.228,"15min":1.228},"last":0.60999999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73799999999999999,"15min":0.754},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95799999999999996,"5min":1.264,"15min":1.264},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78800000000000003,"15min":0.81799999999999995},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97999999999999998,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.76300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":696320,"data_stored":286616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.68100000000000005,"15min":0.68600000000000005},"min":{"1min":0.502,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.98899999999999999,"5min":1.016,"15min":1.016},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65900000000000003,"15min":0.67300000000000004},"min":{"1min":0.40600000000000003,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84599999999999997,"5min":1.26,"15min":1.26},"last":0.50700000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.68100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95899999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.71499999999999997},"min":{"1min":0.53200000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.88800000000000001,"5min":1.077,"15min":1.077},"last":0.67700000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70599999999999996,"15min":0.70899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.96899999999999997,"5min":1.302,"15min":1.302},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73599999999999999,"15min":0.75600000000000001},"min":{"1min":0.47799999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96099999999999997,"5min":1.181,"15min":1.181},"last":0.61899999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.73799999999999999,"15min":0.752},"min":{"1min":0.57599999999999996,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93200000000000005,"5min":1.702,"15min":1.702},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.78900000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.002,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.53000000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67700000000000005,"15min":0.69599999999999995},"min":{"1min":0.44900000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.80600000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85199999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.45800000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.48499999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86699999999999999,"5min":1.129,"15min":1.129},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76600000000000001,"15min":0.79200000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.91800000000000004,"5min":1.012,"15min":1.012},"last":0.79700000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.499,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.879,"5min":1.016,"15min":1.016},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.52700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.93300000000000005,"5min":1.242,"15min":1.242},"last":0.69999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":692224,"data_stored":285060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47699999999999998,"15min":0.47499999999999998},"min":{"1min":0.315,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.629,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.71799999999999997,"5min":1.294,"15min":1.294},"last":0.48199999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.44600000000000001},"min":{"1min":0.32500000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76900000000000002,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.52200000000000002,"15min":0.51900000000000002},"min":{"1min":0.40699999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76600000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45000000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.436,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82699999999999996,"5min":1.456,"15min":1.456},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58899999999999997,"15min":0.58499999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.81299999999999994,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.58299999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.60599999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1579999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0680000000000001,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.72199999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77200000000000002,"15min":0.78100000000000003},"min":{"1min":0.61099999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0229999999999999,"5min":1.395,"15min":1.395},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.66500000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1200000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.79300000000000004,"15min":0.79400000000000004},"min":{"1min":0.628,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0960000000000001,"5min":1.333,"15min":1.333},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80700000000000005,"15min":0.81100000000000005},"min":{"1min":0.66200000000000003,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.1180000000000001,"5min":1.347,"15min":1.347},"last":0.69699999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80600000000000005,"15min":0.80400000000000005},"min":{"1min":0.66100000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0700000000000001,"5min":1.397,"15min":1.397},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81200000000000006,"15min":0.81599999999999995},"min":{"1min":0.61599999999999999,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.087,"5min":1.363,"15min":1.363},"last":0.56399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":285060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.60399999999999998,"15min":0.66300000000000003},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.746,"5min":3.931,"15min":3.931},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.67200000000000004,"15min":0.72399999999999998},"min":{"1min":0.432,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.747,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.64100000000000001,"15min":0.70499999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.75600000000000001,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.64200000000000002,"15min":0.69999999999999996},"min":{"1min":0.39500000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.78800000000000003,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.57299999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.501,"15min":0.50900000000000001},"min":{"1min":0.36799999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.76900000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.82699999999999996,"15min":0.92800000000000005},"min":{"1min":0.46200000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.76500000000000001,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.70799999999999996}]},{"osd":4,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.88900000000000001,"15min":0.94099999999999995},"min":{"1min":0.58499999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.046,"5min":4.431,"15min":4.431},"last":1.099},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.876,"15min":0.91700000000000004},"min":{"1min":0.56799999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.3049999999999999,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":1.024}]},{"osd":5,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.101,"5min":1.575,"15min":1.575},"last":1.21},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.91500000000000004,"15min":0.97499999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.038,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.94899999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.93700000000000006,"15min":0.998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.256,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1359999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.1240000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.94299999999999995,"15min":0.996},"min":{"1min":0.61599999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.204,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.82199999999999995,"15min":0.84799999999999998},"min":{"1min":0.59099999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.054,"5min":1.609,"15min":1.609},"last":1.052}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":675840,"data_stored":278794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.27500000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.63300000000000001,"5min":1.034,"15min":1.034},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.41499999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.76100000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.39500000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44800000000000001,"15min":0.45700000000000002},"min":{"1min":0.31,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.58299999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51500000000000001,"15min":0.52700000000000002},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71499999999999997,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.58899999999999997},"min":{"1min":0.47499999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82199999999999995,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.5,"15min":0.504},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68799999999999994,"5min":0.753,"15min":0.753},"last":0.44400000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.50800000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.53100000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.216,"15min":1.216},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.56499999999999995,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.175,"5min":1.262,"15min":1.262},"last":0.754},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.75,"15min":0.75},"min":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0780000000000001,"5min":1.54,"15min":1.54},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.74299999999999999,"15min":0.73099999999999998},"min":{"1min":0.55400000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.095,"5min":1.145,"15min":1.145},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.1180000000000001,"5min":1.333,"15min":1.333},"last":0.63100000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76100000000000001,"15min":0.76200000000000001},"min":{"1min":0.61399999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.04,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.61599999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1930000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.53200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":870247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.32000000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47599999999999998},"min":{"1min":0.33600000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88500000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.36499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51500000000000001},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73399999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.47599999999999998},"min":{"1min":0.35499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63700000000000001,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.66600000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.80400000000000005,"15min":0.80900000000000005},"min":{"1min":0.59399999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0349999999999999,"5min":1.091,"15min":1.091},"last":0.999}]},{"osd":5,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.58299999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.98799999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":1.042},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.83199999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.79300000000000004,"15min":0.79900000000000004},"min":{"1min":0.61099999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.96499999999999997,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":1.024},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.79100000000000004,"15min":0.80500000000000005},"min":{"1min":0.50800000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.0129999999999999,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.0549999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0169999999999999,"5min":1.123,"15min":1.123},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.81499999999999995,"15min":0.81399999999999995},"min":{"1min":0.71899999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.071,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":1.0820000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:21:07.885 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:21:07.885 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:21:08.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.070+0000 7f15737f8640 1 Processor -- start 2023-11-26T07:21:08.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.070+0000 7f15737f8640 1 -- start start 2023-11-26T07:21:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.070+0000 7f15737f8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f156c0b4040 0x7f156c0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.070+0000 7f15737f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f156c162190 0x7f156c1625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.070+0000 7f15737f8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0b2590 con 0x7f156c0b3c60 2023-11-26T07:21:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.070+0000 7f15737f8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0b2870 con 0x7f156c0b4040 2023-11-26T07:21:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.070+0000 7f15737f8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0b2b50 con 0x7f156c162190 2023-11-26T07:21:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.070+0000 7f1570d93640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f156c0b4040 0x7f156c0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.070+0000 7f1570d93640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f156c0b4040 0x7f156c0b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34542/0 (socket says 172.21.15.31:34542) 2023-11-26T07:21:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.070+0000 7f1570d93640 1 -- 172.21.15.31:0/2765520820 learned_addr learned my addr 172.21.15.31:0/2765520820 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.074+0000 7f1571d95640 1 --2- 172.21.15.31:0/2765520820 >> v2:172.21.15.97:3300/0 conn(0x7f156c162190 0x7f156c1625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.074+0000 7f1570d93640 1 -- 172.21.15.31:0/2765520820 >> v2:172.21.15.97:3300/0 conn(0x7f156c162190 msgr2=0x7f156c1625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.074+0000 7f1570d93640 1 --2- 172.21.15.31:0/2765520820 >> v2:172.21.15.97:3300/0 conn(0x7f156c162190 0x7f156c1625a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.074+0000 7f1570d93640 1 -- 172.21.15.31:0/2765520820 >> v1:172.21.15.97:6789/0 conn(0x7f156c0b3c60 legacy=0x7f156c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.074+0000 7f1570d93640 1 -- 172.21.15.31:0/2765520820 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0b2e30 con 0x7f156c0b4040 2023-11-26T07:21:08.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.074+0000 7f1570d93640 1 --2- 172.21.15.31:0/2765520820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b4040 0x7f156c0b1fb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f156c0b1cd0 tx=0x7f155c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29016de395a5db9f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:08.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.074+0000 7f1563fff640 1 -- 172.21.15.31:0/2765520820 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c0aa650 con 0x7f156c0b4040 2023-11-26T07:21:08.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.074+0000 7f1563fff640 1 -- 172.21.15.31:0/2765520820 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c0aa7b0 con 0x7f156c0b4040 2023-11-26T07:21:08.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.074+0000 7f1563fff640 1 -- 172.21.15.31:0/2765520820 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c0aab40 con 0x7f156c0b4040 2023-11-26T07:21:08.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 -- 172.21.15.31:0/2765520820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b4040 msgr2=0x7f156c0b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 --2- 172.21.15.31:0/2765520820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b4040 0x7f156c0b1fb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f156c0b1cd0 tx=0x7f155c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 -- 172.21.15.31:0/2765520820 shutdown_connections 2023-11-26T07:21:08.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 --2- 172.21.15.31:0/2765520820 >> v2:172.21.15.97:3300/0 conn(0x7f156c162190 0x7f156c1625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 --2- 172.21.15.31:0/2765520820 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c0b4040 0x7f156c0b1fb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 -- 172.21.15.31:0/2765520820 >> 172.21.15.31:0/2765520820 conn(0x7f156c0b0cb0 msgr2=0x7f156c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:08.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 -- 172.21.15.31:0/2765520820 shutdown_connections 2023-11-26T07:21:08.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 -- 172.21.15.31:0/2765520820 wait complete. 2023-11-26T07:21:08.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 Processor -- start 2023-11-26T07:21:08.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 -- start start 2023-11-26T07:21:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c0b3c60 0x7f156c09c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c162190 0x7f156c1481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c0710a0 0x7f156c09a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c148700 con 0x7f156c162190 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1628d0 con 0x7f156c0710a0 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f15737f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1625a0 con 0x7f156c0b3c60 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1570d93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c162190 0x7f156c1481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1570d93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c162190 0x7f156c1481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34544/0 (socket says 172.21.15.31:34544) 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1570d93640 1 -- 172.21.15.31:0/3553436854 learned_addr learned my addr 172.21.15.31:0/3553436854 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1571d95640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c0710a0 0x7f156c09a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1571594640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c0b3c60 0x7f156c09c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1570d93640 1 -- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c0b3c60 msgr2=0x7f156c09c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1570d93640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c0b3c60 0x7f156c09c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1570d93640 1 -- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c0710a0 msgr2=0x7f156c09a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1570d93640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c0710a0 0x7f156c09a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1570d93640 1 -- 172.21.15.31:0/3553436854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c0094d0 con 0x7f156c162190 2023-11-26T07:21:08.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1571594640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c0b3c60 0x7f156c09c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:08.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1571d95640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c0710a0 0x7f156c09a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:08.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.078+0000 7f1570d93640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c162190 0x7f156c1481c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f155c0190e0 tx=0x7f155c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:08.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.082+0000 7f1561ffb640 1 -- 172.21.15.31:0/3553436854 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c0b3800 con 0x7f156c162190 2023-11-26T07:21:08.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.082+0000 7f1561ffb640 1 -- 172.21.15.31:0/3553436854 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c0b3960 con 0x7f156c162190 2023-11-26T07:21:08.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.082+0000 7f1561ffb640 1 -- 172.21.15.31:0/3553436854 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c0b3d10 con 0x7f156c162190 2023-11-26T07:21:08.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.082+0000 7f15737f8640 1 -- 172.21.15.31:0/3553436854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c0718f0 con 0x7f156c162190 2023-11-26T07:21:08.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.082+0000 7f15737f8640 1 -- 172.21.15.31:0/3553436854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c071eb0 con 0x7f156c162190 2023-11-26T07:21:08.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.082+0000 7f15737f8640 1 -- 172.21.15.31:0/3553436854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c0671f0 con 0x7f156c162190 2023-11-26T07:21:08.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.082+0000 7f1561ffb640 1 -- 172.21.15.31:0/3553436854 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f155c0a9430 con 0x7f156c162190 2023-11-26T07:21:08.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.082+0000 7f1561ffb640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f15500666a0 0x7f1550068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.082+0000 7f1561ffb640 1 -- 172.21.15.31:0/3553436854 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f155c11efa0 con 0x7f156c162190 2023-11-26T07:21:08.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.082+0000 7f1571594640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f15500666a0 0x7f1550068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.086+0000 7f1571594640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f15500666a0 0x7f1550068b60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1568000f70 tx=0x7f1568009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:08.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.086+0000 7f1561ffb640 1 -- 172.21.15.31:0/3553436854 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f155c0ec070 con 0x7f156c162190 2023-11-26T07:21:08.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.298+0000 7f15737f8640 1 -- 172.21.15.31:0/3553436854 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f156c072070 con 0x7f156c162190 2023-11-26T07:21:08.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f1561ffb640 1 -- 172.21.15.31:0/3553436854 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f155c0ef690 con 0x7f156c162190 2023-11-26T07:21:08.303 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-11-26T07:21:08.303 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA== 2023-11-26T07:21:08.303 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2023-11-26T07:21:08.303 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2023-11-26T07:21:08.303 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:21:08.304 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-11-26T07:21:08.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 -- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f15500666a0 msgr2=0x7f1550068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f15500666a0 0x7f1550068b60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1568000f70 tx=0x7f1568009040 comp rx=0 tx=0).stop 2023-11-26T07:21:08.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 -- 172.21.15.31:0/3553436854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c162190 msgr2=0x7f156c1481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c162190 0x7f156c1481c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f155c0190e0 tx=0x7f155c01ad10 comp rx=0 tx=0).stop 2023-11-26T07:21:08.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 -- 172.21.15.31:0/3553436854 shutdown_connections 2023-11-26T07:21:08.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f15500666a0 0x7f1550068b60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f156c0710a0 0x7f156c09a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f156c162190 0x7f156c1481c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 --2- 172.21.15.31:0/3553436854 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f156c0b3c60 0x7f156c09c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 -- 172.21.15.31:0/3553436854 >> 172.21.15.31:0/3553436854 conn(0x7f156c0b0cb0 msgr2=0x7f156c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:08.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 -- 172.21.15.31:0/3553436854 shutdown_connections 2023-11-26T07:21:08.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.302+0000 7f15737f8640 1 -- 172.21.15.31:0/3553436854 wait complete. 2023-11-26T07:21:08.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddefdf640 1 Processor -- start 2023-11-26T07:21:08.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddefdf640 1 -- start start 2023-11-26T07:21:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddefdf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fadd80b3410 0x7fadd806e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddefdf640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd806e120 con 0x7fadd80ac020 2023-11-26T07:21:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddefdf640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd80ac4b0 con 0x7fadd80ad260 2023-11-26T07:21:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddefdf640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd806d7f0 con 0x7fadd80b3410 2023-11-26T07:21:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddcd7b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fadd80b3410 0x7fadd806e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddcd7b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fadd80b3410 0x7fadd806e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34554/0 (socket says 172.21.15.31:34554) 2023-11-26T07:21:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7fadd7fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fadd80ac020 0x7fadd80acb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35532/0 (socket says 172.21.15.31:35532) 2023-11-26T07:21:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddcd7b640 1 -- 172.21.15.31:0/4181108300 learned_addr learned my addr 172.21.15.31:0/4181108300 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:08.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddcd7b640 1 -- 172.21.15.31:0/4181108300 >> v1:172.21.15.97:6789/0 conn(0x7fadd80ad260 legacy=0x7fadd8151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:08.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddcd7b640 1 -- 172.21.15.31:0/4181108300 >> v1:172.21.15.31:6789/0 conn(0x7fadd80ac020 legacy=0x7fadd80acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7faddcd7b640 1 -- 172.21.15.31:0/4181108300 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd806dad0 con 0x7fadd80b3410 2023-11-26T07:21:08.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.466+0000 7fadd77fe640 1 -- 172.21.15.31:0/4181108300 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107062394 0 0) 0x7fadd806e120 con 0x7fadd80ac020 2023-11-26T07:21:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7faddcd7b640 1 --2- 172.21.15.31:0/4181108300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd80b3410 0x7fadd806e5b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009870 tx=0x7fadcc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26cd708c57fad791 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7fadd77fe640 1 -- 172.21.15.31:0/4181108300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc0ac410 con 0x7fadd80b3410 2023-11-26T07:21:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7fadd77fe640 1 -- 172.21.15.31:0/4181108300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc0ac570 con 0x7fadd80b3410 2023-11-26T07:21:08.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7fadd77fe640 1 -- 172.21.15.31:0/4181108300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc0ac900 con 0x7fadd80b3410 2023-11-26T07:21:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7faddefdf640 1 -- 172.21.15.31:0/4181108300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd80b3410 msgr2=0x7fadd806e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7faddefdf640 1 --2- 172.21.15.31:0/4181108300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd80b3410 0x7fadd806e5b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009870 tx=0x7fadcc01c480 comp rx=0 tx=0).stop 2023-11-26T07:21:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7faddefdf640 1 -- 172.21.15.31:0/4181108300 shutdown_connections 2023-11-26T07:21:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7faddefdf640 1 --2- 172.21.15.31:0/4181108300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd80b3410 0x7fadd806e5b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7faddefdf640 1 -- 172.21.15.31:0/4181108300 >> 172.21.15.31:0/4181108300 conn(0x7fadd806a5b0 msgr2=0x7fadd806a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:08.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7faddefdf640 1 -- 172.21.15.31:0/4181108300 shutdown_connections 2023-11-26T07:21:08.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.470+0000 7faddefdf640 1 -- 172.21.15.31:0/4181108300 wait complete. 2023-11-26T07:21:08.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddefdf640 1 Processor -- start 2023-11-26T07:21:08.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddefdf640 1 -- start start 2023-11-26T07:21:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddefdf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fadd80ad260 0x7fadd81451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddefdf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd80b3410 0x7fadd814e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddefdf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd815f990 0x7fadd814eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddefdf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd814f000 con 0x7fadd815f990 2023-11-26T07:21:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddefdf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd814f7a0 con 0x7fadd80b3410 2023-11-26T07:21:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddefdf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd814f1e0 con 0x7fadd80ad260 2023-11-26T07:21:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddd57c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd815f990 0x7fadd814eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddd57c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd815f990 0x7fadd814eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34558/0 (socket says 172.21.15.31:34558) 2023-11-26T07:21:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddd57c640 1 -- 172.21.15.31:0/4004177385 learned_addr learned my addr 172.21.15.31:0/4004177385 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7fadd7fff640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd80b3410 0x7fadd814e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddcd7b640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fadd80ad260 0x7fadd81451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddd57c640 1 -- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fadd80ad260 msgr2=0x7fadd81451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddd57c640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fadd80ad260 0x7fadd81451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddd57c640 1 -- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd80b3410 msgr2=0x7fadd814e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddd57c640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd80b3410 0x7fadd814e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddd57c640 1 -- 172.21.15.31:0/4004177385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc0094d0 con 0x7fadd815f990 2023-11-26T07:21:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddcd7b640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fadd80ad260 0x7fadd81451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7fadd7fff640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd80b3410 0x7fadd814e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddd57c640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd815f990 0x7fadd814eac0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c700 tx=0x7fadd000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7fadd57fa640 1 -- 172.21.15.31:0/4004177385 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0003b80 con 0x7fadd815f990 2023-11-26T07:21:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7fadd57fa640 1 -- 172.21.15.31:0/4004177385 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0003ce0 con 0x7fadd815f990 2023-11-26T07:21:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7fadd57fa640 1 -- 172.21.15.31:0/4004177385 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000aa90 con 0x7fadd815f990 2023-11-26T07:21:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddefdf640 1 -- 172.21.15.31:0/4004177385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd814f4c0 con 0x7fadd815f990 2023-11-26T07:21:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.474+0000 7faddefdf640 1 -- 172.21.15.31:0/4004177385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd814b1e0 con 0x7fadd815f990 2023-11-26T07:21:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.478+0000 7faddefdf640 1 -- 172.21.15.31:0/4004177385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd814ad30 con 0x7fadd815f990 2023-11-26T07:21:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.478+0000 7fadd57fa640 1 -- 172.21.15.31:0/4004177385 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fadd000abf0 con 0x7fadd815f990 2023-11-26T07:21:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.478+0000 7fadd57fa640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadbc066650 0x7fadbc068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.478+0000 7fadd57fa640 1 -- 172.21.15.31:0/4004177385 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadd0084010 con 0x7fadd815f990 2023-11-26T07:21:08.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.478+0000 7faddcd7b640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadbc066650 0x7fadbc068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.478+0000 7faddcd7b640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadbc066650 0x7fadbc068b10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0ae000 tx=0x7fadcc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:08.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.482+0000 7fadd57fa640 1 -- 172.21.15.31:0/4004177385 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fadd0051180 con 0x7fadd815f990 2023-11-26T07:21:08.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.670+0000 7faddefdf640 1 -- 172.21.15.31:0/4004177385 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadd804e850 con 0x7fadd815f990 2023-11-26T07:21:08.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7fadd57fa640 1 -- 172.21.15.31:0/4004177385 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+4910 (secure 0 0 0) 0x7fadd0058340 con 0x7fadd815f990 2023-11-26T07:21:08.675 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:08.676 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3429205589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3429205589},{"type":"v1","addr":"172.21.15.97:6836","nonce":3429205589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5959,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3569564413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3569564413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3569564413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5965,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690524463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690524463},{"type":"v1","addr":"172.21.15.97:6837","nonce":690524463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:02.722567+0000","modified":"2023-11-26T07:21:05.757700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1053315203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1053315203},{"type":"v1","addr":"172.21.15.31:6834","nonce":1053315203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:21:08.676 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 37 2023-11-26T07:21:08.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 -- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadbc066650 msgr2=0x7fadbc068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadbc066650 0x7fadbc068b10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0ae000 tx=0x7fadcc008040 comp rx=0 tx=0).stop 2023-11-26T07:21:08.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 -- 172.21.15.31:0/4004177385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd815f990 msgr2=0x7fadd814eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:08.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd815f990 0x7fadd814eac0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c700 tx=0x7fadd000cbc0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 -- 172.21.15.31:0/4004177385 shutdown_connections 2023-11-26T07:21:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fadbc066650 0x7fadbc068b10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadd815f990 0x7fadd814eac0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fadd80b3410 0x7fadd814e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 --2- 172.21.15.31:0/4004177385 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fadd80ad260 0x7fadd81451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 -- 172.21.15.31:0/4004177385 >> 172.21.15.31:0/4004177385 conn(0x7fadd806a5b0 msgr2=0x7fadd81504e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 -- 172.21.15.31:0/4004177385 shutdown_connections 2023-11-26T07:21:08.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:08.674+0000 7faddefdf640 1 -- 172.21.15.31:0/4004177385 wait complete. 2023-11-26T07:21:09.098 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:21:02.722567+0000', 'modified': '2023-11-26T07:21:05.757700+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6141}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6141': {'gid': 6141, 'name': 'a', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6834/1053315203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6832', 'nonce': 1053315203}, {'type': 'v1', 'addr': '172.21.15.31:6834', 'nonce': 1053315203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:21:09.099 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:21:09.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-26T07:21:09.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.242+0000 7fa0cd068640 1 Processor -- start 2023-11-26T07:21:09.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.242+0000 7fa0cd068640 1 -- start start 2023-11-26T07:21:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.242+0000 7fa0cd068640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 0x7fa0c8070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.242+0000 7fa0cd068640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0c815eb50 0x7fa0c80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.242+0000 7fa0cd068640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c8147190 con 0x7fa0c80b4210 2023-11-26T07:21:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.242+0000 7fa0cd068640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c8146c00 con 0x7fa0c815eb50 2023-11-26T07:21:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.242+0000 7fa0cd068640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c806bf90 con 0x7fa0c8146820 2023-11-26T07:21:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.242+0000 7fa0c759e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0c815eb50 0x7fa0c80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.242+0000 7fa0c759e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa0c815eb50 0x7fa0c80b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34562/0 (socket says 172.21.15.31:34562) 2023-11-26T07:21:09.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.242+0000 7fa0c759e640 1 -- 172.21.15.31:0/606449631 learned_addr learned my addr 172.21.15.31:0/606449631 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.246+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/606449631 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 0x7fa0c8070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.246+0000 7fa0c759e640 1 -- 172.21.15.31:0/606449631 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 msgr2=0x7fa0c8070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.246+0000 7fa0c759e640 1 --2- 172.21.15.31:0/606449631 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 0x7fa0c8070a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.246+0000 7fa0c759e640 1 -- 172.21.15.31:0/606449631 >> v1:172.21.15.97:6789/0 conn(0x7fa0c80b4210 legacy=0x7fa0c8161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.246+0000 7fa0c759e640 1 -- 172.21.15.31:0/606449631 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c80b4bb0 con 0x7fa0c815eb50 2023-11-26T07:21:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.246+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/606449631 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 0x7fa0c8070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.246+0000 7fa0c759e640 1 --2- 172.21.15.31:0/606449631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c815eb50 0x7fa0c80b4670 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009e50 tx=0x7fa0c001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e220deea2fabb3e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.246+0000 7fa0c5d9b640 1 -- 172.21.15.31:0/606449631 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00a57c0 con 0x7fa0c815eb50 2023-11-26T07:21:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.246+0000 7fa0c5d9b640 1 -- 172.21.15.31:0/606449631 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c00a5920 con 0x7fa0c815eb50 2023-11-26T07:21:09.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.246+0000 7fa0c5d9b640 1 -- 172.21.15.31:0/606449631 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00a5cb0 con 0x7fa0c815eb50 2023-11-26T07:21:09.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 -- 172.21.15.31:0/606449631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c815eb50 msgr2=0x7fa0c80b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 --2- 172.21.15.31:0/606449631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c815eb50 0x7fa0c80b4670 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009e50 tx=0x7fa0c001c9e0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 -- 172.21.15.31:0/606449631 shutdown_connections 2023-11-26T07:21:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 --2- 172.21.15.31:0/606449631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c815eb50 0x7fa0c80b4670 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 --2- 172.21.15.31:0/606449631 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 0x7fa0c8070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 -- 172.21.15.31:0/606449631 >> 172.21.15.31:0/606449631 conn(0x7fa0c80b2f10 msgr2=0x7fa0c80b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 -- 172.21.15.31:0/606449631 shutdown_connections 2023-11-26T07:21:09.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 -- 172.21.15.31:0/606449631 wait complete. 2023-11-26T07:21:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 Processor -- start 2023-11-26T07:21:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 -- start start 2023-11-26T07:21:09.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c80b4210 0x7fa0c8167670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 0x7fa0c8167bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0c815eb50 0x7fa0c815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c814b810 con 0x7fa0c80b4210 2023-11-26T07:21:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c813e3c0 con 0x7fa0c815eb50 2023-11-26T07:21:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0cd068640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c814c070 con 0x7fa0c8146820 2023-11-26T07:21:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0c6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c80b4210 0x7fa0c8167670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0c6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c80b4210 0x7fa0c8167670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34568/0 (socket says 172.21.15.31:34568) 2023-11-26T07:21:09.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/629603731 learned_addr learned my addr 172.21.15.31:0/629603731 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0c659c640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 0x7fa0c8167bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.250+0000 7fa0c759e640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0c815eb50 0x7fa0c815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 msgr2=0x7fa0c8167bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 0x7fa0c8167bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0c815eb50 msgr2=0x7fa0c815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0c815eb50 0x7fa0c815c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/629603731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c0009b30 con 0x7fa0c80b4210 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c80b4210 0x7fa0c8167670 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800d830 tx=0x7fa0b800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0af7fe640 1 -- 172.21.15.31:0/629603731 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8009660 con 0x7fa0c80b4210 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0af7fe640 1 -- 172.21.15.31:0/629603731 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b80097c0 con 0x7fa0c80b4210 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0af7fe640 1 -- 172.21.15.31:0/629603731 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8009b70 con 0x7fa0c80b4210 2023-11-26T07:21:09.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0cd068640 1 -- 172.21.15.31:0/629603731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c806d030 con 0x7fa0c80b4210 2023-11-26T07:21:09.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0cd068640 1 -- 172.21.15.31:0/629603731 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c81680f0 con 0x7fa0c80b4210 2023-11-26T07:21:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0af7fe640 1 -- 172.21.15.31:0/629603731 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fa0b8014410 con 0x7fa0c80b4210 2023-11-26T07:21:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.254+0000 7fa0af7fe640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa0b4066620 0x7fa0b4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0af7fe640 1 -- 172.21.15.31:0/629603731 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0b80849e0 con 0x7fa0c80b4210 2023-11-26T07:21:09.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c659c640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa0b4066620 0x7fa0b4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 Processor -- start 2023-11-26T07:21:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 -- start start 2023-11-26T07:21:09.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c659c640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa0b4066620 0x7fa0b4068ae0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa0c815d7e0 tx=0x7fa0bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7e4e0 0x55962ec7e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7ee00 0x55962ec2bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec2c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962ea96130 con 0x55962ec2c320 2023-11-26T07:21:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962ea839a0 con 0x55962ec7e4e0 2023-11-26T07:21:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55962ea80ed0 con 0x55962ec7ee00 2023-11-26T07:21:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec2c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c659c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7ee00 0x55962ec2bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec2c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34572/0 (socket says 172.21.15.31:34572) 2023-11-26T07:21:09.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c759e640 1 -- 172.21.15.31:0/459604774 learned_addr learned my addr 172.21.15.31:0/459604774 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c6d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7e4e0 0x55962ec7e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c759e640 1 -- 172.21.15.31:0/459604774 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7ee00 msgr2=0x55962ec2bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c759e640 1 --2- 172.21.15.31:0/459604774 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7ee00 0x55962ec2bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c759e640 1 -- 172.21.15.31:0/459604774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7e4e0 msgr2=0x55962ec7e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c759e640 1 --2- 172.21.15.31:0/459604774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7e4e0 0x55962ec7e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c759e640 1 -- 172.21.15.31:0/459604774 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55962e774890 con 0x55962ec2c320 2023-11-26T07:21:09.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0c759e640 1 --2- 172.21.15.31:0/459604774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec2c700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa0c001b090 tx=0x7fa0c0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152d9dd8f6f6031a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0acff9640 1 -- 172.21.15.31:0/459604774 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00a5c90 con 0x55962ec2c320 2023-11-26T07:21:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0acff9640 1 -- 172.21.15.31:0/459604774 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c00a5df0 con 0x55962ec2c320 2023-11-26T07:21:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0acff9640 1 -- 172.21.15.31:0/459604774 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00b6610 con 0x55962ec2c320 2023-11-26T07:21:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 -- 172.21.15.31:0/459604774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 msgr2=0x55962ec2c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/459604774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec2c700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa0c001b090 tx=0x7fa0c0004ac0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 -- 172.21.15.31:0/459604774 shutdown_connections 2023-11-26T07:21:09.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/459604774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec2c700 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/459604774 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7ee00 0x55962ec2bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/459604774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7e4e0 0x55962ec7e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.258+0000 7fa0d03c7000 1 -- 172.21.15.31:0/459604774 >> 172.21.15.31:0/459604774 conn(0x55962ec282d0 msgr2=0x55962ec2a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 -- 172.21.15.31:0/459604774 shutdown_connections 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 -- 172.21.15.31:0/459604774 wait complete. 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 -- start start 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 Processor -- start 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec3ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec3f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7ee00 0x55962ec3f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962ea7c7c0 con 0x55962ec2c320 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962ea34bf0 con 0x55962ec7ee00 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55962ea3aa90 con 0x55962ec7e4e0 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 10 client.0 _ll_register_callbacks cb 0x55962ec173c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec3ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec3ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34586/0 (socket says 172.21.15.31:34586) 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/2257881868 learned_addr learned my addr 172.21.15.31:0/2257881868 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c659c640 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec3f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c759e640 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7ee00 0x55962ec3f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/2257881868 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 msgr2=0x55962ec3f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec3f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/2257881868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7ee00 msgr2=0x55962ec3f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:09.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7ee00 0x55962ec3f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/2257881868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c80b4bb0 con 0x55962ec2c320 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c759e640 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7ee00 0x55962ec3f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec3ee30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8051db0 tx=0x7fa0b80559e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9402b0762be61e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/2257881868 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8082fd0 con 0x55962ec2c320 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/2257881868 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0b80828b0 con 0x55962ec2c320 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 -- 172.21.15.31:0/2257881868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55962ea3e470 con 0x55962ec2c320 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/2257881868 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0b805d670 con 0x55962ec2c320 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 10 client.6279 fetch_fsmap learned FSMap version 37 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 -- 172.21.15.31:0/2257881868 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55962ea3d190 con 0x55962ec2c320 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/2257881868 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55962ea3e470 con 0x55962ec2c320 2023-11-26T07:21:09.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/2257881868 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa0b8056b30 con 0x55962ec2c320 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 10 client.6279 fetch_fsmap finished waiting for FSMap version 37 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 10 client.6279 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 20 client.6279 populate_metadata read hostname 'smithi031' 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.31:6832/1053315203,v1:172.21.15.31:6834/1053315203] conn(0x55962ec402c0 0x55962ec907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 4 client.6279 mds_command: new command op to 6141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.262+0000 7fa0d03c7000 1 -- 172.21.15.31:0/2257881868 --> [v2:172.21.15.31:6832/1053315203,v1:172.21.15.31:6834/1053315203] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55962ea41b50 con 0x55962ec402c0 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0c659c640 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.31:6832/1053315203,v1:172.21.15.31:6834/1053315203] conn(0x55962ec402c0 0x55962ec907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0c659c640 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.31:6832/1053315203,v1:172.21.15.31:6834/1053315203] conn(0x55962ec402c0 0x55962ec907a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45100f282e672700 server_cookie=baeaf927456e08d7 in_seq=0 out_seq=0 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0a5ffb640 10 client.6279 ms_handle_connect on v2:172.21.15.31:6832/1053315203 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/2257881868 <== mds.0 v2:172.21.15.31:6832/1053315203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55962ea41b50 con 0x55962ec402c0 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0a5ffb640 10 client.6279 handle_command_reply: tid=0 2023-11-26T07:21:09.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 client.6279 shutdown 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 -- 172.21.15.31:0/2257881868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 msgr2=0x55962ec3ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec3ee30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8051db0 tx=0x7fa0b80559e0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 -- 172.21.15.31:0/2257881868 shutdown_connections 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec7ee00 0x55962ec3f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.31:6832/1053315203,v1:172.21.15.31:6834/1053315203] conn(0x55962ec402c0 0x55962ec907a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec3f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/2257881868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec3ee30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0a5ffb640 0 client.6279 ms_handle_reset on v2:172.21.15.31:6832/1053315203 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 -- 172.21.15.31:0/2257881868 >> 172.21.15.31:0/2257881868 conn(0x55962ec282d0 msgr2=0x55962ec2af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 -- 172.21.15.31:0/2257881868 shutdown_connections 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 1 -- 172.21.15.31:0/2257881868 wait complete. 2023-11-26T07:21:09.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.266+0000 7fa0d03c7000 20 client.6279 trim_cache size 0 max 16384 2023-11-26T07:21:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0d03c7000 1 Processor -- start 2023-11-26T07:21:09.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0d03c7000 1 -- start start 2023-11-26T07:21:09.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec282d0 0x55962ec8b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec2c320 0x55962ec8be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec907a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962e735940 con 0x55962ec282d0 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962ea79bc0 con 0x55962ec2c320 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55962ea45910 con 0x55962ec7e4e0 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec282d0 0x55962ec8b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec282d0 0x55962ec8b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34600/0 (socket says 172.21.15.31:34600) 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/3897321091 learned_addr learned my addr 172.21.15.31:0/3897321091 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c759e640 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec907a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/3897321091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 msgr2=0x55962ec907a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c659c640 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec2c320 0x55962ec8be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec907a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/3897321091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec2c320 msgr2=0x55962ec8be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:09.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec2c320 0x55962ec8be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c6d9d640 1 -- 172.21.15.31:0/3897321091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55962ea23490 con 0x55962ec282d0 2023-11-26T07:21:09.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c759e640 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec907a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:09.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c659c640 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec2c320 0x55962ec8be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:09.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec282d0 0x55962ec8b950 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa0b805d5d0 tx=0x7fa0b8057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3d3e22aa83007c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:09.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0ad7fa640 1 -- 172.21.15.31:0/3897321091 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8082b70 con 0x55962ec282d0 2023-11-26T07:21:09.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0ad7fa640 1 -- 172.21.15.31:0/3897321091 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8055210 con 0x55962ec282d0 2023-11-26T07:21:09.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.282+0000 7fa0ad7fa640 1 -- 172.21.15.31:0/3897321091 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8082ff0 con 0x55962ec282d0 2023-11-26T07:21:09.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- 172.21.15.31:0/3897321091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec282d0 msgr2=0x55962ec8b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec282d0 0x55962ec8b950 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa0b805d5d0 tx=0x7fa0b8057000 comp rx=0 tx=0).stop 2023-11-26T07:21:09.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- 172.21.15.31:0/3897321091 shutdown_connections 2023-11-26T07:21:09.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec907a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec2c320 0x55962ec8be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/3897321091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec282d0 0x55962ec8b950 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- 172.21.15.31:0/3897321091 >> 172.21.15.31:0/3897321091 conn(0x55962ec31490 msgr2=0x55962ec8b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- 172.21.15.31:0/3897321091 shutdown_connections 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- 172.21.15.31:0/3897321091 wait complete. 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- start start 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 Processor -- start 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec282d0 0x55962ec927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec9b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec9b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962ec24d60 con 0x55962ec2c320 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55962ea462e0 con 0x55962ec282d0 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55962ea72aa0 con 0x55962ec7e4e0 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 10 client.0 _ll_register_callbacks cb 0x55962eacac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec9b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec9b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34616/0 (socket says 172.21.15.31:34616) 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c659c640 1 -- 172.21.15.31:0/705391645 learned_addr learned my addr 172.21.15.31:0/705391645 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec282d0 0x55962ec927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c759e640 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec9b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c659c640 1 -- 172.21.15.31:0/705391645 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 msgr2=0x55962ec9b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c659c640 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec9b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c659c640 1 -- 172.21.15.31:0/705391645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec282d0 msgr2=0x55962ec927d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:09.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c659c640 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec282d0 0x55962ec927d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c659c640 1 -- 172.21.15.31:0/705391645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55962ea3d190 con 0x55962ec2c320 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0c659c640 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec9b270 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc001ec0 tx=0x7fa0bc001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5418dbd107af6ad server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/705391645 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc029070 con 0x55962ec2c320 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/705391645 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0bc01c040 con 0x55962ec2c320 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- 172.21.15.31:0/705391645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55962ea7b110 con 0x55962ec2c320 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/705391645 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0bc024ac0 con 0x55962ec2c320 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 10 client.6285 fetch_fsmap learned FSMap version 37 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 -- 172.21.15.31:0/705391645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) v3 -- 0x55962e774cc0 con 0x55962ec2c320 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/705391645 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55962ea7b110 con 0x55962ec2c320 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/705391645 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 37) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa0bc012070 con 0x55962ec2c320 2023-11-26T07:21:09.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 10 client.6285 fetch_fsmap finished waiting for FSMap version 37 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 10 client.6285 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 20 client.6285 populate_metadata read hostname 'smithi031' 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.286+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.31:6832/1053315203,v1:172.21.15.31:6834/1053315203] conn(0x55962ec7ee00 0x55962ecdeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 4 client.6285 mds_command: new command op to 6141 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 -- 172.21.15.31:0/705391645 --> [v2:172.21.15.31:6832/1053315203,v1:172.21.15.31:6834/1053315203] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55962eaaa9a0 con 0x55962ec7ee00 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.31:6832/1053315203,v1:172.21.15.31:6834/1053315203] conn(0x55962ec7ee00 0x55962ecdeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0c6d9d640 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.31:6832/1053315203,v1:172.21.15.31:6834/1053315203] conn(0x55962ec7ee00 0x55962ecdeee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a3951d5898f0dd server_cookie=4beb631765c0ffc4 in_seq=0 out_seq=0 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0a5ffb640 10 client.6285 ms_handle_connect on v2:172.21.15.31:6832/1053315203 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0a5ffb640 1 -- 172.21.15.31:0/705391645 <== mds.0 v2:172.21.15.31:6832/1053315203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55962eaaa9a0 con 0x55962ec7ee00 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0a5ffb640 10 client.6285 handle_command_reply: tid=0 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 client.6285 shutdown 2023-11-26T07:21:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 -- 172.21.15.31:0/705391645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 msgr2=0x55962ec9b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec9b270 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc001ec0 tx=0x7fa0bc001090 comp rx=0 tx=0).stop 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 -- 172.21.15.31:0/705391645 shutdown_connections 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.31:6832/1053315203,v1:172.21.15.31:6834/1053315203] conn(0x55962ec7ee00 0x55962ecdeee0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55962ec7e4e0 0x55962ec9b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55962ec2c320 0x55962ec9b270 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 --2- 172.21.15.31:0/705391645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55962ec282d0 0x55962ec927d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0a5ffb640 0 client.6285 ms_handle_reset on v2:172.21.15.31:6832/1053315203 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 -- 172.21.15.31:0/705391645 >> 172.21.15.31:0/705391645 conn(0x55962ec31490 msgr2=0x55962ec8bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 -- 172.21.15.31:0/705391645 shutdown_connections 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 1 -- 172.21.15.31:0/705391645 wait complete. 2023-11-26T07:21:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0d03c7000 20 client.6285 trim_cache size 0 max 16384 2023-11-26T07:21:09.298 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:21:09.298 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "4c2b4df1-71b1-4841-8807-af144d2610e7", 2023-11-26T07:21:09.298 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-11-26T07:21:09.298 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 6141, 2023-11-26T07:21:09.298 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-11-26T07:21:09.298 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-11-26T07:21:09.298 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-11-26T07:21:09.298 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 5.9522250019999996, 2023-11-26T07:21:09.298 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 37, 2023-11-26T07:21:09.299 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 57, 2023-11-26T07:21:09.299 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 57, 2023-11-26T07:21:09.299 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 10.788807667 2023-11-26T07:21:09.299 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:21:09.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 -- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa0b4066620 msgr2=0x7fa0b4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa0b4066620 0x7fa0b4068ae0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa0c815d7e0 tx=0x7fa0bc009040 comp rx=0 tx=0).stop 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 -- 172.21.15.31:0/629603731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c80b4210 msgr2=0x7fa0c8167670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c80b4210 0x7fa0c8167670 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800d830 tx=0x7fa0b800dcf0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 -- 172.21.15.31:0/629603731 shutdown_connections 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa0b4066620 0x7fa0b4068ae0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0c815eb50 0x7fa0c815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa0c8146820 0x7fa0c8167bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 --2- 172.21.15.31:0/629603731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c80b4210 0x7fa0c8167670 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 -- 172.21.15.31:0/629603731 >> 172.21.15.31:0/629603731 conn(0x7fa0c80b2f10 msgr2=0x7fa0c806e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 -- 172.21.15.31:0/629603731 shutdown_connections 2023-11-26T07:21:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.290+0000 7fa0cd068640 1 -- 172.21.15.31:0/629603731 wait complete. 2023-11-26T07:21:09.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:21:09.322 INFO:tasks.ceph.mds.a.smithi031.stderr:2023-11-26T07:21:09.318+0000 7f005ee42640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:21:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0b0957640 1 Processor -- start 2023-11-26T07:21:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0b0957640 1 -- start start 2023-11-26T07:21:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0b0957640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0ac144c80 0x7fd0ac0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0b0957640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ac145060 con 0x7fd0ac0ac460 2023-11-26T07:21:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0b0957640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ac071fd0 con 0x7fd0ac070df0 2023-11-26T07:21:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0b0957640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac0ac8c0 con 0x7fd0ac144c80 2023-11-26T07:21:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0aa59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0ac144c80 0x7fd0ac0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0a3fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd0ac0ac460 0x7fd0ac0706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35536/0 (socket says 172.21.15.31:35536) 2023-11-26T07:21:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0a3fff640 1 -- 172.21.15.31:0/286808723 learned_addr learned my addr 172.21.15.31:0/286808723 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0aa59c640 1 -- 172.21.15.31:0/286808723 >> v1:172.21.15.97:6789/0 conn(0x7fd0ac070df0 legacy=0x7fd0ac06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0aa59c640 1 -- 172.21.15.31:0/286808723 >> v1:172.21.15.31:6789/0 conn(0x7fd0ac0ac460 legacy=0x7fd0ac0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0aa59c640 1 -- 172.21.15.31:0/286808723 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac071450 con 0x7fd0ac144c80 2023-11-26T07:21:09.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.454+0000 7fd0a9d9b640 1 -- 172.21.15.31:0/286808723 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422816229 0 0) 0x7fd0ac145060 con 0x7fd0ac0ac460 2023-11-26T07:21:09.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0aa59c640 1 --2- 172.21.15.31:0/286808723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac144c80 0x7fd0ac0abf20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0097c0 tx=0x7fd09c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fabfb76496507f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0a9d9b640 1 -- 172.21.15.31:0/286808723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0ab600 con 0x7fd0ac144c80 2023-11-26T07:21:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0a9d9b640 1 -- 172.21.15.31:0/286808723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c0a7050 con 0x7fd0ac144c80 2023-11-26T07:21:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0a9d9b640 1 -- 172.21.15.31:0/286808723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0ab980 con 0x7fd0ac144c80 2023-11-26T07:21:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0b0957640 1 -- 172.21.15.31:0/286808723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac144c80 msgr2=0x7fd0ac0abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0b0957640 1 --2- 172.21.15.31:0/286808723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac144c80 0x7fd0ac0abf20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0097c0 tx=0x7fd09c01c480 comp rx=0 tx=0).stop 2023-11-26T07:21:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0b0957640 1 -- 172.21.15.31:0/286808723 shutdown_connections 2023-11-26T07:21:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0b0957640 1 --2- 172.21.15.31:0/286808723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac144c80 0x7fd0ac0abf20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0b0957640 1 -- 172.21.15.31:0/286808723 >> 172.21.15.31:0/286808723 conn(0x7fd0ac0b0c40 msgr2=0x7fd0ac0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0b0957640 1 -- 172.21.15.31:0/286808723 shutdown_connections 2023-11-26T07:21:09.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.458+0000 7fd0b0957640 1 -- 172.21.15.31:0/286808723 wait complete. 2023-11-26T07:21:09.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0b0957640 1 Processor -- start 2023-11-26T07:21:09.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0b0957640 1 -- start start 2023-11-26T07:21:09.465 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0b0957640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac070df0 0x7fd0ac15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0b0957640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0ac0ac460 0x7fd0ac156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0b0957640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0ac144c80 0x7fd0ac142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0b0957640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac15b3c0 con 0x7fd0ac070df0 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0b0957640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac1643d0 con 0x7fd0ac0ac460 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0b0957640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac1646b0 con 0x7fd0ac144c80 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aa59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac070df0 0x7fd0ac15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aa59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac070df0 0x7fd0ac15ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34630/0 (socket says 172.21.15.31:34630) 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aa59c640 1 -- 172.21.15.31:0/1611285198 learned_addr learned my addr 172.21.15.31:0/1611285198 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0a3fff640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0ac0ac460 0x7fd0ac156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aad9d640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0ac144c80 0x7fd0ac142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aa59c640 1 -- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0ac144c80 msgr2=0x7fd0ac142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aa59c640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0ac144c80 0x7fd0ac142bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aa59c640 1 -- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0ac0ac460 msgr2=0x7fd0ac156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aa59c640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0ac0ac460 0x7fd0ac156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aa59c640 1 -- 172.21.15.31:0/1611285198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c009420 con 0x7fd0ac070df0 2023-11-26T07:21:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0a3fff640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0ac0ac460 0x7fd0ac156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aad9d640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0ac144c80 0x7fd0ac142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0aa59c640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac070df0 0x7fd0ac15ae80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0ae040 tx=0x7fd09c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0a2ffd640 1 -- 172.21.15.31:0/1611285198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0b7650 con 0x7fd0ac070df0 2023-11-26T07:21:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0a2ffd640 1 -- 172.21.15.31:0/1611285198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c0b77b0 con 0x7fd0ac070df0 2023-11-26T07:21:09.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0b0957640 1 -- 172.21.15.31:0/1611285198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ac164990 con 0x7fd0ac070df0 2023-11-26T07:21:09.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0a2ffd640 1 -- 172.21.15.31:0/1611285198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0b7b60 con 0x7fd0ac070df0 2023-11-26T07:21:09.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.462+0000 7fd0b0957640 1 -- 172.21.15.31:0/1611285198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ac164f50 con 0x7fd0ac070df0 2023-11-26T07:21:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.466+0000 7fd0b0957640 1 -- 172.21.15.31:0/1611285198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ac143b80 con 0x7fd0ac070df0 2023-11-26T07:21:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.466+0000 7fd0a2ffd640 1 -- 172.21.15.31:0/1611285198 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fd09c0b7cc0 con 0x7fd0ac070df0 2023-11-26T07:21:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.466+0000 7fd0a2ffd640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd08c066620 0x7fd08c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.466+0000 7fd0a2ffd640 1 -- 172.21.15.31:0/1611285198 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd09c11ed50 con 0x7fd0ac070df0 2023-11-26T07:21:09.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.466+0000 7fd0a3fff640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd08c066620 0x7fd08c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:09.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.466+0000 7fd0a3fff640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd08c066620 0x7fd08c068ae0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac15fc80 tx=0x7fd094007c80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:09.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.470+0000 7fd0a2ffd640 1 -- 172.21.15.31:0/1611285198 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd09c0ebd10 con 0x7fd0ac070df0 2023-11-26T07:21:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 -- 172.21.15.31:0/1611285198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd0ac0b1ba0 con 0x7fd0ac070df0 2023-11-26T07:21:09.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0a2ffd640 1 -- 172.21.15.31:0/1611285198 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd09c0b0070 con 0x7fd0ac070df0 2023-11-26T07:21:09.668 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:21:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 -- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd08c066620 msgr2=0x7fd08c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd08c066620 0x7fd08c068ae0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac15fc80 tx=0x7fd094007c80 comp rx=0 tx=0).stop 2023-11-26T07:21:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 -- 172.21.15.31:0/1611285198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac070df0 msgr2=0x7fd0ac15ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac070df0 0x7fd0ac15ae80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0ae040 tx=0x7fd09c01acc0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 -- 172.21.15.31:0/1611285198 shutdown_connections 2023-11-26T07:21:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd08c066620 0x7fd08c068ae0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd0ac144c80 0x7fd0ac142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0ac0ac460 0x7fd0ac156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 --2- 172.21.15.31:0/1611285198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0ac070df0 0x7fd0ac15ae80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 -- 172.21.15.31:0/1611285198 >> 172.21.15.31:0/1611285198 conn(0x7fd0ac0b0c40 msgr2=0x7fd0ac0aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:09.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 -- 172.21.15.31:0/1611285198 shutdown_connections 2023-11-26T07:21:09.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:09.666+0000 7fd0b0957640 1 -- 172.21.15.31:0/1611285198 wait complete. 2023-11-26T07:21:09.689 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:21:09.689 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:21:09.689 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-11-26T07:21:09.689 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:21:09.689 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:21:09.689 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:21:09.689 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:21:09.689 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:21:09.690 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:21:09.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:09.690 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-11-26T07:21:09.740 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:21:09.740 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:21:09.740 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:21:09.740 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:09.740 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-11-26T07:21:09.740 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:09.740 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:09.740 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:09.740 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-11-26T07:21:09.740 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42170sec preferred_lft 42170sec 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:09.741 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 02:42:1b:e4:a9:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:21:09.742 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:21:09.743 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:21:09.743 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:09.743 INFO:teuthology.orchestra.run.smithi031.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:21:09.743 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:21:09.743 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:21:09.743 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:09.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:09.744 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:21:09.806 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-11-26T07:21:09.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:09.808 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-11-26T07:21:09.864 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-11-26T07:21:09.864 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:21:09.864 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-26T07:21:09.865 INFO:teuthology.orchestra.run.smithi031.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:21:09.865 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-26T07:21:09.865 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:21:09.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:09.865 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:21:09.865 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:21:09.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-26T07:21:09.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-26T07:21:09.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-26T07:21:09.866 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:21:09.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:09.959 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:21:09.969 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:21:09.969 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:21:09.969 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:21:09.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:09.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-11-26T07:21:10.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:21:10.032 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:21:10.083 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:21:10.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:21:10.083 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:21:10.134 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:21:10.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:10.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:10.233 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:10.233 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:21:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:21:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:21:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:21:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:21:10.250 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:21:10.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:10.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:21:10.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:10.271 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:21:10.331 INFO:teuthology.orchestra.run.smithi031.stdout:client6297 2023-11-26T07:21:10.332 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:21:10.332 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:10.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:21:10.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.574+0000 7f5c3ed01640 1 Processor -- start 2023-11-26T07:21:10.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.574+0000 7f5c3ed01640 1 -- start start 2023-11-26T07:21:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.574+0000 7f5c3ed01640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c38144c80 0x7f5c380abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.574+0000 7f5c3ed01640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c38145060 con 0x7f5c38070df0 2023-11-26T07:21:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.574+0000 7f5c3ed01640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c38071fd0 con 0x7f5c380ac460 2023-11-26T07:21:10.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.574+0000 7f5c3ed01640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c380ac8c0 con 0x7f5c38144c80 2023-11-26T07:21:10.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.574+0000 7f5c3ca9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5c38144c80 0x7f5c380abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:10.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.574+0000 7f5c3d29e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5c38070df0 0x7f5c3806c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35546/0 (socket says 172.21.15.31:35546) 2023-11-26T07:21:10.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.574+0000 7f5c3d29e640 1 -- 172.21.15.31:0/2996792694 learned_addr learned my addr 172.21.15.31:0/2996792694 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:10.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ca9d640 1 -- 172.21.15.31:0/2996792694 >> v1:172.21.15.97:6789/0 conn(0x7f5c380ac460 legacy=0x7f5c380706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:10.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ca9d640 1 -- 172.21.15.31:0/2996792694 >> v1:172.21.15.31:6789/0 conn(0x7f5c38070df0 legacy=0x7f5c3806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:10.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c377fe640 1 -- 172.21.15.31:0/2996792694 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1312659104 0 0) 0x7f5c38145060 con 0x7f5c38070df0 2023-11-26T07:21:10.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ca9d640 1 -- 172.21.15.31:0/2996792694 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c38071450 con 0x7f5c38144c80 2023-11-26T07:21:10.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ca9d640 1 --2- 172.21.15.31:0/2996792694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 0x7f5c380abf20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0097c0 tx=0x7f5c2c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ebb762a8fb1638 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:10.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c377fe640 1 -- 172.21.15.31:0/2996792694 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c0ab600 con 0x7f5c38144c80 2023-11-26T07:21:10.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c377fe640 1 -- 172.21.15.31:0/2996792694 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c0a7050 con 0x7f5c38144c80 2023-11-26T07:21:10.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c377fe640 1 -- 172.21.15.31:0/2996792694 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c0ab980 con 0x7f5c38144c80 2023-11-26T07:21:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ed01640 1 -- 172.21.15.31:0/2996792694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 msgr2=0x7f5c380abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ed01640 1 --2- 172.21.15.31:0/2996792694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 0x7f5c380abf20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0097c0 tx=0x7f5c2c01c440 comp rx=0 tx=0).stop 2023-11-26T07:21:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ed01640 1 -- 172.21.15.31:0/2996792694 shutdown_connections 2023-11-26T07:21:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ed01640 1 --2- 172.21.15.31:0/2996792694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 0x7f5c380abf20 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ed01640 1 -- 172.21.15.31:0/2996792694 >> 172.21.15.31:0/2996792694 conn(0x7f5c380b0c40 msgr2=0x7f5c380b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ed01640 1 -- 172.21.15.31:0/2996792694 shutdown_connections 2023-11-26T07:21:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.578+0000 7f5c3ed01640 1 -- 172.21.15.31:0/2996792694 wait complete. 2023-11-26T07:21:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ed01640 1 Processor -- start 2023-11-26T07:21:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ed01640 1 -- start start 2023-11-26T07:21:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ed01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c38070df0 0x7f5c3815f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ed01640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5c380ac460 0x7f5c38140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ed01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 0x7f5c3815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ed01640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38099e00 con 0x7f5c38144c80 2023-11-26T07:21:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ed01640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3815cae0 con 0x7f5c38070df0 2023-11-26T07:21:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ed01640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38099ad0 con 0x7f5c380ac460 2023-11-26T07:21:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3d29e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 0x7f5c3815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ca9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c38070df0 0x7f5c3815f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3d29e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 0x7f5c3815d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34642/0 (socket says 172.21.15.31:34642) 2023-11-26T07:21:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c37fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5c380ac460 0x7f5c38140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ca9d640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c38070df0 0x7f5c3815f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:58536/0 (socket says 172.21.15.31:58536) 2023-11-26T07:21:10.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3d29e640 1 -- 172.21.15.31:0/170017001 learned_addr learned my addr 172.21.15.31:0/170017001 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:10.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3d29e640 1 -- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5c380ac460 msgr2=0x7f5c38140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:10.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3d29e640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5c380ac460 0x7f5c38140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:10.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3d29e640 1 -- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c38070df0 msgr2=0x7f5c3815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3d29e640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c38070df0 0x7f5c3815f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3d29e640 1 -- 172.21.15.31:0/170017001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c009420 con 0x7f5c38144c80 2023-11-26T07:21:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3ca9d640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c38070df0 0x7f5c3815f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.582+0000 7f5c3d29e640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 0x7f5c3815d3e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5c30001570 tx=0x7f5c3000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.586+0000 7f5c357fa640 1 -- 172.21.15.31:0/170017001 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3000a640 con 0x7f5c38144c80 2023-11-26T07:21:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.586+0000 7f5c357fa640 1 -- 172.21.15.31:0/170017001 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3000a7c0 con 0x7f5c38144c80 2023-11-26T07:21:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.586+0000 7f5c357fa640 1 -- 172.21.15.31:0/170017001 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3000ab70 con 0x7f5c38144c80 2023-11-26T07:21:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.586+0000 7f5c3ed01640 1 -- 172.21.15.31:0/170017001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3815cd80 con 0x7f5c38144c80 2023-11-26T07:21:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.586+0000 7f5c3ed01640 1 -- 172.21.15.31:0/170017001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3815f950 con 0x7f5c38144c80 2023-11-26T07:21:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.586+0000 7f5c3ed01640 1 -- 172.21.15.31:0/170017001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c380b1ba0 con 0x7f5c38144c80 2023-11-26T07:21:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.586+0000 7f5c357fa640 1 -- 172.21.15.31:0/170017001 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f5c3000acd0 con 0x7f5c38144c80 2023-11-26T07:21:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.586+0000 7f5c357fa640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c1c066650 0x7f5c1c068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:10.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.586+0000 7f5c357fa640 1 -- 172.21.15.31:0/170017001 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c300834b0 con 0x7f5c38144c80 2023-11-26T07:21:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.590+0000 7f5c3ca9d640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c1c066650 0x7f5c1c068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:10.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.590+0000 7f5c3ca9d640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c1c066650 0x7f5c1c068b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0ad000 tx=0x7f5c2c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:10.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.590+0000 7f5c357fa640 1 -- 172.21.15.31:0/170017001 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c30050800 con 0x7f5c38144c80 2023-11-26T07:21:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.782+0000 7f5c3ed01640 1 -- 172.21.15.31:0/170017001 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5c3804e850 con 0x7f5c38144c80 2023-11-26T07:21:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c357fa640 1 -- 172.21.15.31:0/170017001 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v37) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5c30018060 con 0x7f5c38144c80 2023-11-26T07:21:10.787 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:21:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 -- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c1c066650 msgr2=0x7f5c1c068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c1c066650 0x7f5c1c068b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0ad000 tx=0x7f5c2c0a6000 comp rx=0 tx=0).stop 2023-11-26T07:21:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 -- 172.21.15.31:0/170017001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 msgr2=0x7f5c3815d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 0x7f5c3815d3e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5c30001570 tx=0x7f5c3000cca0 comp rx=0 tx=0).stop 2023-11-26T07:21:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 -- 172.21.15.31:0/170017001 shutdown_connections 2023-11-26T07:21:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5c1c066650 0x7f5c1c068b10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c38144c80 0x7f5c3815d3e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5c380ac460 0x7f5c38140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 --2- 172.21.15.31:0/170017001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c38070df0 0x7f5c3815f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 -- 172.21.15.31:0/170017001 >> 172.21.15.31:0/170017001 conn(0x7f5c380b0c40 msgr2=0x7f5c380add30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 -- 172.21.15.31:0/170017001 shutdown_connections 2023-11-26T07:21:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:10.786+0000 7f5c3ed01640 1 -- 172.21.15.31:0/170017001 wait complete. 2023-11-26T07:21:10.809 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:21:10.809 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:21:10.809 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-26T07:21:10.809 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:21:10.809 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:21:10.809 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:21:10.809 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:21:10.809 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:21:10.809 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:21:10.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:10.810 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-26T07:21:10.820 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:21:10.821 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:21:10.821 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:21:10.821 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:10.821 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-26T07:21:10.821 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:10.821 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:10.821 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-11-26T07:21:10.821 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:10.821 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42192sec preferred_lft 42192sec 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:21:10.822 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:61:bc:5b:3d brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:21:10.823 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:10.824 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:21:10.824 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:21:10.824 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:21:10.824 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:10.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:10.825 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:21:10.890 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-26T07:21:10.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:10.893 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-26T07:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-26T07:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:21:10.950 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:21:10.951 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:21:10.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:10.951 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:21:10.951 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:21:10.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:21:10.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:21:10.952 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:21:10.952 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:21:11.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:11.049 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:21:11.058 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:21:11.058 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:21:11.059 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:21:11.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:11.059 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-26T07:21:11.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:21:11.123 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:21:11.174 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:21:11.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:21:11.175 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:21:11.226 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:21:11.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:11.226 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:11.321 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:11.321 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:21:11.334 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:11.334 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:21:11.335 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:21:11.335 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 1@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:21:11.335 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:21:11.338 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:21:11.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:11.339 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:21:11.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:11.358 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:21:11.413 INFO:teuthology.orchestra.run.smithi097.stdout:client5600 2023-11-26T07:21:11.413 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:21:11.413 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:11.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:21:11.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.610+0000 7f7523fc4640 1 Processor -- start 2023-11-26T07:21:11.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.610+0000 7f7523fc4640 1 -- start start 2023-11-26T07:21:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7523fc4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f751c0b3c60 0x7f751c0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7523fc4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0b4040 0x7f751c0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7523fc4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f751c162190 0x7f751c162600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7523fc4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751c0b2590 con 0x7f751c0b3c60 2023-11-26T07:21:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7523fc4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751c0b2870 con 0x7f751c162190 2023-11-26T07:21:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7523fc4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751c0b2b50 con 0x7f751c0b4040 2023-11-26T07:21:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7521d60640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f751c0b3c60 0x7f751c0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:11.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7521d60640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f751c0b3c60 0x7f751c0b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34654/0 (socket says 172.21.15.31:34654) 2023-11-26T07:21:11.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7521d60640 1 -- 172.21.15.31:0/146343441 learned_addr learned my addr 172.21.15.31:0/146343441 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:11.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7522561640 1 --2- 172.21.15.31:0/146343441 >> v2:172.21.15.97:3300/0 conn(0x7f751c162190 0x7f751c162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:11.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f752155f640 1 --2- 172.21.15.31:0/146343441 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0b4040 0x7f751c0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:11.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7521d60640 1 -- 172.21.15.31:0/146343441 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0b4040 msgr2=0x7f751c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:11.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7521d60640 1 --2- 172.21.15.31:0/146343441 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0b4040 0x7f751c0ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:11.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7521d60640 1 -- 172.21.15.31:0/146343441 >> v2:172.21.15.97:3300/0 conn(0x7f751c162190 msgr2=0x7f751c162600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:11.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7521d60640 1 --2- 172.21.15.31:0/146343441 >> v2:172.21.15.97:3300/0 conn(0x7f751c162190 0x7f751c162600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:11.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.614+0000 7f7521d60640 1 -- 172.21.15.31:0/146343441 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c0b2e30 con 0x7f751c0b3c60 2023-11-26T07:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7521d60640 1 --2- 172.21.15.31:0/146343441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0b3c60 0x7f751c0b1fb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f751c06c0c0 tx=0x7f751801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287497bf5b7c1b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7520d5e640 1 -- 172.21.15.31:0/146343441 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180ab620 con 0x7f751c0b3c60 2023-11-26T07:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7520d5e640 1 -- 172.21.15.31:0/146343441 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75180ab780 con 0x7f751c0b3c60 2023-11-26T07:21:11.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7520d5e640 1 -- 172.21.15.31:0/146343441 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180abb10 con 0x7f751c0b3c60 2023-11-26T07:21:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 -- 172.21.15.31:0/146343441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0b3c60 msgr2=0x7f751c0b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 --2- 172.21.15.31:0/146343441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0b3c60 0x7f751c0b1fb0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f751c06c0c0 tx=0x7f751801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:21:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 -- 172.21.15.31:0/146343441 shutdown_connections 2023-11-26T07:21:11.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 --2- 172.21.15.31:0/146343441 >> v2:172.21.15.97:3300/0 conn(0x7f751c162190 0x7f751c162600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 --2- 172.21.15.31:0/146343441 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c0b4040 0x7f751c0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 --2- 172.21.15.31:0/146343441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c0b3c60 0x7f751c0b1fb0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 -- 172.21.15.31:0/146343441 >> 172.21.15.31:0/146343441 conn(0x7f751c0b0cb0 msgr2=0x7f751c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 -- 172.21.15.31:0/146343441 shutdown_connections 2023-11-26T07:21:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 -- 172.21.15.31:0/146343441 wait complete. 2023-11-26T07:21:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 Processor -- start 2023-11-26T07:21:11.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.618+0000 7f7523fc4640 1 -- start start 2023-11-26T07:21:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7523fc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c0b3c60 0x7f751c1668b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7523fc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c162190 0x7f751c09b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7523fc4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c09c360 0x7f751c15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:11.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7523fc4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c165220 con 0x7f751c162190 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7523fc4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c166fb0 con 0x7f751c0b3c60 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7523fc4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751c09a570 con 0x7f751c09c360 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f752155f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c162190 0x7f751c09b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7521d60640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c0b3c60 0x7f751c1668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f752155f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c162190 0x7f751c09b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34656/0 (socket says 172.21.15.31:34656) 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f752155f640 1 -- 172.21.15.31:0/236026128 learned_addr learned my addr 172.21.15.31:0/236026128 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7522561640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c09c360 0x7f751c15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7521d60640 1 -- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c09c360 msgr2=0x7f751c15d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7521d60640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c09c360 0x7f751c15d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7521d60640 1 -- 172.21.15.31:0/236026128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c162190 msgr2=0x7f751c09b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7521d60640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c162190 0x7f751c09b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:11.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7521d60640 1 -- 172.21.15.31:0/236026128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75180094d0 con 0x7f751c0b3c60 2023-11-26T07:21:11.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f752155f640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c162190 0x7f751c09b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:11.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7521d60640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c0b3c60 0x7f751c1668b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f75180ac000 tx=0x7f751801ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:11.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f75127fc640 1 -- 172.21.15.31:0/236026128 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180be430 con 0x7f751c0b3c60 2023-11-26T07:21:11.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f75127fc640 1 -- 172.21.15.31:0/236026128 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75180be5d0 con 0x7f751c0b3c60 2023-11-26T07:21:11.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f75127fc640 1 -- 172.21.15.31:0/236026128 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180be980 con 0x7f751c0b3c60 2023-11-26T07:21:11.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7523fc4640 1 -- 172.21.15.31:0/236026128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751c09a850 con 0x7f751c0b3c60 2023-11-26T07:21:11.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7523fc4640 1 -- 172.21.15.31:0/236026128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751c09ae10 con 0x7f751c0b3c60 2023-11-26T07:21:11.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f7523fc4640 1 -- 172.21.15.31:0/236026128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751c0671f0 con 0x7f751c0b3c60 2023-11-26T07:21:11.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.622+0000 7f75127fc640 1 -- 172.21.15.31:0/236026128 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f75180b43f0 con 0x7f751c0b3c60 2023-11-26T07:21:11.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.626+0000 7f75127fc640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066650 0x7f7500068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:11.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.626+0000 7f75127fc640 1 -- 172.21.15.31:0/236026128 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f751811e4f0 con 0x7f751c0b3c60 2023-11-26T07:21:11.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.626+0000 7f752155f640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066650 0x7f7500068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:11.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.626+0000 7f752155f640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066650 0x7f7500068b10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f751c16ca80 tx=0x7f750c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:11.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.630+0000 7f75127fc640 1 -- 172.21.15.31:0/236026128 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f75180eb730 con 0x7f751c0b3c60 2023-11-26T07:21:11.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:11.822+0000 7f7523fc4640 1 -- 172.21.15.31:0/236026128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f751c09a6b0 con 0x7f751c0b3c60 2023-11-26T07:21:12.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.402+0000 7f75127fc640 1 -- 172.21.15.31:0/236026128 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 136+0+0 (secure 0 0 0) 0x7f751c09a6b0 con 0x7f751c0b3c60 2023-11-26T07:21:12.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.402+0000 7f7523fc4640 1 -- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066650 msgr2=0x7f7500068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:12.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.402+0000 7f7523fc4640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066650 0x7f7500068b10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f751c16ca80 tx=0x7f750c008040 comp rx=0 tx=0).stop 2023-11-26T07:21:12.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.402+0000 7f7523fc4640 1 -- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c0b3c60 msgr2=0x7f751c1668b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:12.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.406+0000 7f7523fc4640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c0b3c60 0x7f751c1668b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f75180ac000 tx=0x7f751801ad10 comp rx=0 tx=0).stop 2023-11-26T07:21:12.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.406+0000 7f7523fc4640 1 -- 172.21.15.31:0/236026128 shutdown_connections 2023-11-26T07:21:12.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.406+0000 7f7523fc4640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7500066650 0x7f7500068b10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.406+0000 7f7523fc4640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f751c09c360 0x7f751c15d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.406+0000 7f7523fc4640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f751c162190 0x7f751c09b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.406+0000 7f7523fc4640 1 --2- 172.21.15.31:0/236026128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f751c0b3c60 0x7f751c1668b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.406+0000 7f7523fc4640 1 -- 172.21.15.31:0/236026128 >> 172.21.15.31:0/236026128 conn(0x7f751c0b0cb0 msgr2=0x7f751c06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:12.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.406+0000 7f7523fc4640 1 -- 172.21.15.31:0/236026128 shutdown_connections 2023-11-26T07:21:12.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.406+0000 7f7523fc4640 1 -- 172.21.15.31:0/236026128 wait complete. 2023-11-26T07:21:12.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:21:12.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.570+0000 7f59481e7640 1 Processor -- start 2023-11-26T07:21:12.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.570+0000 7f59481e7640 1 -- start start 2023-11-26T07:21:12.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f59481e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59400cb9b0 0x7f594006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f59481e7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59400cbd90 0x7f594015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f59481e7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f59400ce5d0 0x7f594015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f59481e7640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f594015bba0 con 0x7f59400cbd90 2023-11-26T07:21:12.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f59481e7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59400cdd50 con 0x7f59400ce5d0 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f59481e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5940159630 con 0x7f59400cb9b0 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f5945782640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59400cbd90 0x7f594015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f5945782640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59400cbd90 0x7f594015f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34658/0 (socket says 172.21.15.31:34658) 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f5945782640 1 -- 172.21.15.31:0/1451991821 learned_addr learned my addr 172.21.15.31:0/1451991821 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f5946784640 1 --2- 172.21.15.31:0/1451991821 >> v2:172.21.15.97:3300/0 conn(0x7f59400ce5d0 0x7f594015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f5945f83640 1 --2- 172.21.15.31:0/1451991821 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59400cb9b0 0x7f594006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f5945782640 1 -- 172.21.15.31:0/1451991821 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59400cb9b0 msgr2=0x7f594006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f5945782640 1 --2- 172.21.15.31:0/1451991821 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59400cb9b0 0x7f594006e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f5945782640 1 -- 172.21.15.31:0/1451991821 >> v2:172.21.15.97:3300/0 conn(0x7f59400ce5d0 msgr2=0x7f594015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f5945782640 1 --2- 172.21.15.31:0/1451991821 >> v2:172.21.15.97:3300/0 conn(0x7f59400ce5d0 0x7f594015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.574+0000 7f5945782640 1 -- 172.21.15.31:0/1451991821 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594015f6d0 con 0x7f59400cbd90 2023-11-26T07:21:12.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f5945782640 1 --2- 172.21.15.31:0/1451991821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400cbd90 0x7f594015f190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f59400ae970 tx=0x7f593001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b42048d096bd225 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:12.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f5944f81640 1 -- 172.21.15.31:0/1451991821 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300ab670 con 0x7f59400cbd90 2023-11-26T07:21:12.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f5944f81640 1 -- 172.21.15.31:0/1451991821 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59300ab7d0 con 0x7f59400cbd90 2023-11-26T07:21:12.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f5944f81640 1 -- 172.21.15.31:0/1451991821 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300abb60 con 0x7f59400cbd90 2023-11-26T07:21:12.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 -- 172.21.15.31:0/1451991821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400cbd90 msgr2=0x7f594015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:12.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 --2- 172.21.15.31:0/1451991821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400cbd90 0x7f594015f190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f59400ae970 tx=0x7f593001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 -- 172.21.15.31:0/1451991821 shutdown_connections 2023-11-26T07:21:12.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 --2- 172.21.15.31:0/1451991821 >> v2:172.21.15.97:3300/0 conn(0x7f59400ce5d0 0x7f594015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 --2- 172.21.15.31:0/1451991821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400cbd90 0x7f594015f190 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 --2- 172.21.15.31:0/1451991821 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59400cb9b0 0x7f594006e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 -- 172.21.15.31:0/1451991821 >> 172.21.15.31:0/1451991821 conn(0x7f59400b0cb0 msgr2=0x7f59400b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:12.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 -- 172.21.15.31:0/1451991821 shutdown_connections 2023-11-26T07:21:12.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 -- 172.21.15.31:0/1451991821 wait complete. 2023-11-26T07:21:12.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 Processor -- start 2023-11-26T07:21:12.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.578+0000 7f59481e7640 1 -- start start 2023-11-26T07:21:12.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59481e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59400cb9b0 0x7f59401577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59481e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59400cbd90 0x7f5940157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59481e7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400ce5d0 0x7f59401f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59481e7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401f7eb0 con 0x7f59400ce5d0 2023-11-26T07:21:12.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59481e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401f7880 con 0x7f59400cbd90 2023-11-26T07:21:12.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59481e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5940158360 con 0x7f59400cb9b0 2023-11-26T07:21:12.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5946784640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400ce5d0 0x7f59401f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5946784640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400ce5d0 0x7f59401f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34660/0 (socket says 172.21.15.31:34660) 2023-11-26T07:21:12.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5945782640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59400cbd90 0x7f5940157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5946784640 1 -- 172.21.15.31:0/852933648 learned_addr learned my addr 172.21.15.31:0/852933648 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:12.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5945f83640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59400cb9b0 0x7f59401577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5946784640 1 -- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59400cb9b0 msgr2=0x7f59401577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:12.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5946784640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59400cb9b0 0x7f59401577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5946784640 1 -- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59400cbd90 msgr2=0x7f5940157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:12.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5946784640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59400cbd90 0x7f5940157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5946784640 1 -- 172.21.15.31:0/852933648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59300094d0 con 0x7f59400ce5d0 2023-11-26T07:21:12.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f5946784640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400ce5d0 0x7f59401f66c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5938007e90 tx=0x7f593800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:12.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59367fc640 1 -- 172.21.15.31:0/852933648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938014420 con 0x7f59400ce5d0 2023-11-26T07:21:12.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59367fc640 1 -- 172.21.15.31:0/852933648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5938014580 con 0x7f59400ce5d0 2023-11-26T07:21:12.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59367fc640 1 -- 172.21.15.31:0/852933648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5938014930 con 0x7f59400ce5d0 2023-11-26T07:21:12.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59481e7640 1 -- 172.21.15.31:0/852933648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5940158640 con 0x7f59400ce5d0 2023-11-26T07:21:12.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59481e7640 1 -- 172.21.15.31:0/852933648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59401558b0 con 0x7f59400ce5d0 2023-11-26T07:21:12.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59481e7640 1 -- 172.21.15.31:0/852933648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59400671f0 con 0x7f59400ce5d0 2023-11-26T07:21:12.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.582+0000 7f59367fc640 1 -- 172.21.15.31:0/852933648 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f5938014a90 con 0x7f59400ce5d0 2023-11-26T07:21:12.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.586+0000 7f59367fc640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5924066620 0x7f5924068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.586+0000 7f59367fc640 1 -- 172.21.15.31:0/852933648 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59380848c0 con 0x7f59400ce5d0 2023-11-26T07:21:12.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.586+0000 7f5945f83640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5924066620 0x7f5924068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.586+0000 7f5945f83640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5924066620 0x7f5924068ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f593c007a60 tx=0x7f593c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:12.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.590+0000 7f59367fc640 1 -- 172.21.15.31:0/852933648 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5938051b60 con 0x7f59400ce5d0 2023-11-26T07:21:12.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.766+0000 7f59481e7640 1 -- 172.21.15.31:0/852933648 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f59400ccea0 con 0x7f5924066620 2023-11-26T07:21:12.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.766+0000 7f59367fc640 1 -- 172.21.15.31:0/852933648 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f593c010990 con 0x7f5924066620 2023-11-26T07:21:12.769 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:21:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-11-26T07:21:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:12.770 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:21:12.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.766+0000 7f59481e7640 1 -- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5924066620 msgr2=0x7f5924068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:12.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.766+0000 7f59481e7640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5924066620 0x7f5924068ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f593c007a60 tx=0x7f593c008040 comp rx=0 tx=0).stop 2023-11-26T07:21:12.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.766+0000 7f59481e7640 1 -- 172.21.15.31:0/852933648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400ce5d0 msgr2=0x7f59401f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:12.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.766+0000 7f59481e7640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400ce5d0 0x7f59401f66c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5938007e90 tx=0x7f593800e4b0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.770+0000 7f59481e7640 1 -- 172.21.15.31:0/852933648 shutdown_connections 2023-11-26T07:21:12.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.770+0000 7f59481e7640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5924066620 0x7f5924068ae0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.770+0000 7f59481e7640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59400ce5d0 0x7f59401f66c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.770+0000 7f59481e7640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59400cbd90 0x7f5940157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.770+0000 7f59481e7640 1 --2- 172.21.15.31:0/852933648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59400cb9b0 0x7f59401577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.770+0000 7f59481e7640 1 -- 172.21.15.31:0/852933648 >> 172.21.15.31:0/852933648 conn(0x7f59400b0cb0 msgr2=0x7f594015e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:12.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.770+0000 7f59481e7640 1 -- 172.21.15.31:0/852933648 shutdown_connections 2023-11-26T07:21:12.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.770+0000 7f59481e7640 1 -- 172.21.15.31:0/852933648 wait complete. 2023-11-26T07:21:12.790 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:21:12.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20fd079640 1 Processor -- start 2023-11-26T07:21:12.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20fd079640 1 -- start start 2023-11-26T07:21:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20fd079640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20f80cdfb0 0x7f20f80abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20fd079640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f815a710 0x7f20f80ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20fd079640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f20f80cbd50 0x7f20f80cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20fd079640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f80cca80 con 0x7f20f80cdfb0 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20fd079640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f80ce8d0 con 0x7f20f80cbd50 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20fd079640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80b2710 con 0x7f20f815a710 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f6d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20f80cdfb0 0x7f20f80abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f6d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f20f80cdfb0 0x7f20f80abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34666/0 (socket says 172.21.15.31:34666) 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f6d9d640 1 -- 172.21.15.31:0/1363711764 learned_addr learned my addr 172.21.15.31:0/1363711764 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f759e640 1 --2- 172.21.15.31:0/1363711764 >> v2:172.21.15.97:3300/0 conn(0x7f20f80cbd50 0x7f20f80cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f659c640 1 --2- 172.21.15.31:0/1363711764 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f815a710 0x7f20f80ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f6d9d640 1 -- 172.21.15.31:0/1363711764 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f815a710 msgr2=0x7f20f80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f6d9d640 1 --2- 172.21.15.31:0/1363711764 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f815a710 0x7f20f80ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f6d9d640 1 -- 172.21.15.31:0/1363711764 >> v2:172.21.15.97:3300/0 conn(0x7f20f80cbd50 msgr2=0x7f20f80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f6d9d640 1 --2- 172.21.15.31:0/1363711764 >> v2:172.21.15.97:3300/0 conn(0x7f20f80cbd50 0x7f20f80cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f6d9d640 1 -- 172.21.15.31:0/1363711764 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f80b29f0 con 0x7f20f80cdfb0 2023-11-26T07:21:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.934+0000 7f20f659c640 1 --2- 172.21.15.31:0/1363711764 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f815a710 0x7f20f80ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20f6d9d640 1 --2- 172.21.15.31:0/1363711764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f80cdfb0 0x7f20f80abf90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f20f809d1d0 tx=0x7f20e801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7a40c4567f08f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20f5d9b640 1 -- 172.21.15.31:0/1363711764 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80aa5f0 con 0x7f20f80cdfb0 2023-11-26T07:21:12.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20f5d9b640 1 -- 172.21.15.31:0/1363711764 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e80aa750 con 0x7f20f80cdfb0 2023-11-26T07:21:12.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20f5d9b640 1 -- 172.21.15.31:0/1363711764 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80aaae0 con 0x7f20f80cdfb0 2023-11-26T07:21:12.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20fd079640 1 -- 172.21.15.31:0/1363711764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f80cdfb0 msgr2=0x7f20f80abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:12.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20fd079640 1 --2- 172.21.15.31:0/1363711764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f80cdfb0 0x7f20f80abf90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f20f809d1d0 tx=0x7f20e801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20fd079640 1 -- 172.21.15.31:0/1363711764 shutdown_connections 2023-11-26T07:21:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20fd079640 1 --2- 172.21.15.31:0/1363711764 >> v2:172.21.15.97:3300/0 conn(0x7f20f80cbd50 0x7f20f80cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20fd079640 1 --2- 172.21.15.31:0/1363711764 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f815a710 0x7f20f80ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20fd079640 1 --2- 172.21.15.31:0/1363711764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f80cdfb0 0x7f20f80abf90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20fd079640 1 -- 172.21.15.31:0/1363711764 >> 172.21.15.31:0/1363711764 conn(0x7f20f80b0cb0 msgr2=0x7f20f80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20fd079640 1 -- 172.21.15.31:0/1363711764 shutdown_connections 2023-11-26T07:21:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.938+0000 7f20fd079640 1 -- 172.21.15.31:0/1363711764 wait complete. 2023-11-26T07:21:12.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20fd079640 1 Processor -- start 2023-11-26T07:21:12.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20fd079640 1 -- start start 2023-11-26T07:21:12.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20fd079640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f80cbd50 0x7f20f8152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20fd079640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cdfb0 0x7f20f81635a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20fd079640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f815a710 0x7f20f8163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20fd079640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f81530a0 con 0x7f20f815a710 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20fd079640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8152e80 con 0x7f20f80cdfb0 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20fd079640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8153be0 con 0x7f20f80cbd50 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f815a710 0x7f20f8163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f815a710 0x7f20f8163ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34672/0 (socket says 172.21.15.31:34672) 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f759e640 1 -- 172.21.15.31:0/2662670568 learned_addr learned my addr 172.21.15.31:0/2662670568 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f659c640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cdfb0 0x7f20f81635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f6d9d640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f80cbd50 0x7f20f8152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f759e640 1 -- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f80cbd50 msgr2=0x7f20f8152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f759e640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f80cbd50 0x7f20f8152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f759e640 1 -- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cdfb0 msgr2=0x7f20f81635a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:12.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f759e640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cdfb0 0x7f20f81635a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:12.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f759e640 1 -- 172.21.15.31:0/2662670568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e80094d0 con 0x7f20f815a710 2023-11-26T07:21:12.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f659c640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cdfb0 0x7f20f81635a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:12.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f6d9d640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f80cbd50 0x7f20f8152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:12.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.942+0000 7f20f759e640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f815a710 0x7f20f8163ae0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f20f000ef00 tx=0x7f20f000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:12.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.946+0000 7f20df7fe640 1 -- 172.21.15.31:0/2662670568 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0010040 con 0x7f20f815a710 2023-11-26T07:21:12.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.946+0000 7f20df7fe640 1 -- 172.21.15.31:0/2662670568 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f0014400 con 0x7f20f815a710 2023-11-26T07:21:12.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.946+0000 7f20df7fe640 1 -- 172.21.15.31:0/2662670568 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0009ea0 con 0x7f20f815a710 2023-11-26T07:21:12.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.946+0000 7f20fd079640 1 -- 172.21.15.31:0/2662670568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f81532b0 con 0x7f20f815a710 2023-11-26T07:21:12.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.946+0000 7f20fd079640 1 -- 172.21.15.31:0/2662670568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f8153870 con 0x7f20f815a710 2023-11-26T07:21:12.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.946+0000 7f20fd079640 1 -- 172.21.15.31:0/2662670568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f80671f0 con 0x7f20f815a710 2023-11-26T07:21:12.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.946+0000 7f20df7fe640 1 -- 172.21.15.31:0/2662670568 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f20f00147b0 con 0x7f20f815a710 2023-11-26T07:21:12.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.946+0000 7f20df7fe640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20e4066650 0x7f20e4068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:12.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.946+0000 7f20df7fe640 1 -- 172.21.15.31:0/2662670568 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20f0084a00 con 0x7f20f815a710 2023-11-26T07:21:12.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.946+0000 7f20f6d9d640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20e4066650 0x7f20e4068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:12.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.950+0000 7f20f6d9d640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20e4066650 0x7f20e4068b10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f20e80ae000 tx=0x7f20e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:12.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:12.950+0000 7f20df7fe640 1 -- 172.21.15.31:0/2662670568 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f20f0051a50 con 0x7f20f815a710 2023-11-26T07:21:13.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.110+0000 7f20fd079640 1 -- 172.21.15.31:0/2662670568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f20f806b460 con 0x7f20f815a710 2023-11-26T07:21:13.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.110+0000 7f20df7fe640 1 -- 172.21.15.31:0/2662670568 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f20f00180c0 con 0x7f20f815a710 2023-11-26T07:21:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 -- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20e4066650 msgr2=0x7f20e4068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20e4066650 0x7f20e4068b10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f20e80ae000 tx=0x7f20e8008040 comp rx=0 tx=0).stop 2023-11-26T07:21:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 -- 172.21.15.31:0/2662670568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f815a710 msgr2=0x7f20f8163ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f815a710 0x7f20f8163ae0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f20f000ef00 tx=0x7f20f000b480 comp rx=0 tx=0).stop 2023-11-26T07:21:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 -- 172.21.15.31:0/2662670568 shutdown_connections 2023-11-26T07:21:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f20e4066650 0x7f20e4068b10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f815a710 0x7f20f8163ae0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f80cdfb0 0x7f20f81635a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 --2- 172.21.15.31:0/2662670568 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f20f80cbd50 0x7f20f8152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 -- 172.21.15.31:0/2662670568 >> 172.21.15.31:0/2662670568 conn(0x7f20f80b0cb0 msgr2=0x7f20f80ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:13.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 -- 172.21.15.31:0/2662670568 shutdown_connections 2023-11-26T07:21:13.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.114+0000 7f20fd079640 1 -- 172.21.15.31:0/2662670568 wait complete. 2023-11-26T07:21:13.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:13.136 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol13010) 2023-11-26T07:21:13.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:13.164 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-11-26T07:21:13.216 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-11-26T07:21:13.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:13.217 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol13010) 2023-11-26T07:21:13.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol52791 2023-11-26T07:21:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f9431fc1640 1 Processor -- start 2023-11-26T07:21:13.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f9431fc1640 1 -- start start 2023-11-26T07:21:13.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f9431fc1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f942c15d8f0 0x7f942c15e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:13.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f9431fc1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f942c0ced30 0x7f942c0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:13.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f9431fc1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c15ecc0 con 0x7f942c0cf1e0 2023-11-26T07:21:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f9431fc1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c0d4420 con 0x7f942c15d8f0 2023-11-26T07:21:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f9431fc1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c0cdfb0 con 0x7f942c0ced30 2023-11-26T07:21:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f942bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f942c0cf1e0 0x7f942c0d1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:35552/0 (socket says 172.21.15.31:35552) 2023-11-26T07:21:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f942bfff640 1 -- 172.21.15.31:0/2302619127 learned_addr learned my addr 172.21.15.31:0/2302619127 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f942b7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f942c15d8f0 0x7f942c15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f942affd640 1 --2- 172.21.15.31:0/2302619127 >> v2:172.21.15.97:3300/0 conn(0x7f942c0ced30 0x7f942c0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f942b7fe640 1 -- 172.21.15.31:0/2302619127 >> v2:172.21.15.97:3300/0 conn(0x7f942c0ced30 msgr2=0x7f942c0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:13.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f942b7fe640 1 --2- 172.21.15.31:0/2302619127 >> v2:172.21.15.97:3300/0 conn(0x7f942c0ced30 0x7f942c0cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f942b7fe640 1 -- 172.21.15.31:0/2302619127 >> v1:172.21.15.31:6789/0 conn(0x7f942c0cf1e0 legacy=0x7f942c0d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:13.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f942b7fe640 1 -- 172.21.15.31:0/2302619127 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0d4140 con 0x7f942c15d8f0 2023-11-26T07:21:13.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.470+0000 7f942a7fc640 1 -- 172.21.15.31:0/2302619127 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357356837 0 0) 0x7f942c15ecc0 con 0x7f942c0cf1e0 2023-11-26T07:21:13.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f942b7fe640 1 --2- 172.21.15.31:0/2302619127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 0x7f942c15e780 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f941c0097c0 tx=0x7f941c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53a7d56110ea811 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:13.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f942a7fc640 1 -- 172.21.15.31:0/2302619127 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0ab610 con 0x7f942c15d8f0 2023-11-26T07:21:13.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f942a7fc640 1 -- 172.21.15.31:0/2302619127 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c0ab770 con 0x7f942c15d8f0 2023-11-26T07:21:13.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f942a7fc640 1 -- 172.21.15.31:0/2302619127 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0abb00 con 0x7f942c15d8f0 2023-11-26T07:21:13.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f9431fc1640 1 -- 172.21.15.31:0/2302619127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 msgr2=0x7f942c15e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:13.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f9431fc1640 1 --2- 172.21.15.31:0/2302619127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 0x7f942c15e780 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f941c0097c0 tx=0x7f941c01c410 comp rx=0 tx=0).stop 2023-11-26T07:21:13.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f9431fc1640 1 -- 172.21.15.31:0/2302619127 shutdown_connections 2023-11-26T07:21:13.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f9431fc1640 1 --2- 172.21.15.31:0/2302619127 >> v2:172.21.15.97:3300/0 conn(0x7f942c0ced30 0x7f942c0cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f9431fc1640 1 --2- 172.21.15.31:0/2302619127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 0x7f942c15e780 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f9431fc1640 1 -- 172.21.15.31:0/2302619127 >> 172.21.15.31:0/2302619127 conn(0x7f942c09c750 msgr2=0x7f942c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:13.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f9431fc1640 1 -- 172.21.15.31:0/2302619127 shutdown_connections 2023-11-26T07:21:13.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.474+0000 7f9431fc1640 1 -- 172.21.15.31:0/2302619127 wait complete. 2023-11-26T07:21:13.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9431fc1640 1 Processor -- start 2023-11-26T07:21:13.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9431fc1640 1 -- start start 2023-11-26T07:21:13.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9431fc1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0ced30 0x7f942c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:13.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9431fc1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f942c0cf1e0 0x7f942c0c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:13.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9431fc1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 0x7f942c157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:13.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9431fc1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c15ad60 con 0x7f942c15d8f0 2023-11-26T07:21:13.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9431fc1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0b4820 con 0x7f942c0ced30 2023-11-26T07:21:13.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9431fc1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c15aa80 con 0x7f942c0cf1e0 2023-11-26T07:21:13.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 0x7f942c157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:13.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 0x7f942c157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34686/0 (socket says 172.21.15.31:34686) 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942bfff640 1 -- 172.21.15.31:0/177092606 learned_addr learned my addr 172.21.15.31:0/177092606 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942b7fe640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0ced30 0x7f942c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942bfff640 1 -- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f942c0cf1e0 msgr2=0x7f942c0c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942bfff640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f942c0cf1e0 0x7f942c0c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942bfff640 1 -- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0ced30 msgr2=0x7f942c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942bfff640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0ced30 0x7f942c0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942bfff640 1 -- 172.21.15.31:0/177092606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c009420 con 0x7f942c15d8f0 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942b7fe640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0ced30 0x7f942c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f942bfff640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 0x7f942c157930 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f942400c780 tx=0x7f942400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9413fff640 1 -- 172.21.15.31:0/177092606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942400ce90 con 0x7f942c15d8f0 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9413fff640 1 -- 172.21.15.31:0/177092606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942400a640 con 0x7f942c15d8f0 2023-11-26T07:21:13.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9413fff640 1 -- 172.21.15.31:0/177092606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942400a9d0 con 0x7f942c15d8f0 2023-11-26T07:21:13.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9431fc1640 1 -- 172.21.15.31:0/177092606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c0b41d0 con 0x7f942c15d8f0 2023-11-26T07:21:13.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.478+0000 7f9431fc1640 1 -- 172.21.15.31:0/177092606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c0c8fe0 con 0x7f942c15d8f0 2023-11-26T07:21:13.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.482+0000 7f9431fc1640 1 -- 172.21.15.31:0/177092606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c0b4030 con 0x7f942c15d8f0 2023-11-26T07:21:13.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.482+0000 7f9413fff640 1 -- 172.21.15.31:0/177092606 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f942400ab30 con 0x7f942c15d8f0 2023-11-26T07:21:13.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.482+0000 7f9413fff640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9418066650 0x7f9418068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:13.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.482+0000 7f9413fff640 1 -- 172.21.15.31:0/177092606 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9424083620 con 0x7f942c15d8f0 2023-11-26T07:21:13.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.482+0000 7f942b7fe640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9418066650 0x7f9418068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:13.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.486+0000 7f942b7fe640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9418066650 0x7f9418068b10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f941c0af000 tx=0x7f941c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:13.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.486+0000 7f9413fff640 1 -- 172.21.15.31:0/177092606 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f942c0b4030 con 0x7f942c15d8f0 2023-11-26T07:21:13.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.666+0000 7f9431fc1640 1 -- 172.21.15.31:0/177092606 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol52791", "target": ["mon-mgr", ""]}) v1 -- 0x7f942c067730 con 0x7f9418066650 2023-11-26T07:21:13.676 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:13.673+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:13.676 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:13.673+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:13.677 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:13.673+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:13.677 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:13.673+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:13.677 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:13.673+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:13.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9413fff640 1 -- 172.21.15.31:0/177092606 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f942c067730 con 0x7f9418066650 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9431fc1640 1 -- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9418066650 msgr2=0x7f9418068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9431fc1640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9418066650 0x7f9418068b10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f941c0af000 tx=0x7f941c008040 comp rx=0 tx=0).stop 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9431fc1640 1 -- 172.21.15.31:0/177092606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 msgr2=0x7f942c157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9431fc1640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 0x7f942c157930 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f942400c780 tx=0x7f942400cc40 comp rx=0 tx=0).stop 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9431fc1640 1 -- 172.21.15.31:0/177092606 shutdown_connections 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9431fc1640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9418066650 0x7f9418068b10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9431fc1640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c15d8f0 0x7f942c157930 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9431fc1640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f942c0cf1e0 0x7f942c0c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9431fc1640 1 --2- 172.21.15.31:0/177092606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f942c0ced30 0x7f942c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.718+0000 7f9431fc1640 1 -- 172.21.15.31:0/177092606 >> 172.21.15.31:0/177092606 conn(0x7f942c09c750 msgr2=0x7f942c0b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:13.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.722+0000 7f9431fc1640 1 -- 172.21.15.31:0/177092606 shutdown_connections 2023-11-26T07:21:13.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:13.722+0000 7f9431fc1640 1 -- 172.21.15.31:0/177092606 wait complete. 2023-11-26T07:21:13.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:13.742 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol52791/.meta ./volumes/_nogroup/subvol13010) 2023-11-26T07:21:18.521 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol13010 2023-11-26T07:21:18.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f9361e32640 1 Processor -- start 2023-11-26T07:21:18.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f9361e32640 1 -- start start 2023-11-26T07:21:18.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f9361e32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15a710 0x7f935c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:18.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f9361e32640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c0b2710 con 0x7f935c0cdfb0 2023-11-26T07:21:18.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f9361e32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c0b29f0 con 0x7f935c0cbd50 2023-11-26T07:21:18.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f9361e32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0cca80 con 0x7f935c15a710 2023-11-26T07:21:18.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f935c0cdfb0 0x7f935c0ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56084/0 (socket says 172.21.15.31:56084) 2023-11-26T07:21:18.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935b7fe640 1 -- 172.21.15.31:0/633760935 learned_addr learned my addr 172.21.15.31:0/633760935 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:18.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935affd640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15a710 0x7f935c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:18.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630928627 0 0) 0x7f935c0b2710 con 0x7f935c0cdfb0 2023-11-26T07:21:18.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9344003530 con 0x7f935c0cdfb0 2023-11-26T07:21:18.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1595555016 0 0) 0x7f9344003530 con 0x7f935c0cdfb0 2023-11-26T07:21:18.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15a710 msgr2=0x7f935c0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:18.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15a710 0x7f935c0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 >> v1:172.21.15.97:6789/0 conn(0x7f935c0cbd50 legacy=0x7f935c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:18.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c0b2cd0 con 0x7f935c0cdfb0 2023-11-26T07:21:18.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f934c001680 con 0x7f935c0cdfb0 2023-11-26T07:21:18.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 >> v1:172.21.15.31:6789/0 conn(0x7f935c0cdfb0 legacy=0x7f935c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:18.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440039b0 0x7f9344003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93440047d0 0x7f9344004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15a710 0x7f9344005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0cca80 con 0x7f93440047d0 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934400ba70 con 0x7f93440039b0 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934400bc30 con 0x7f935c15a710 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935affd640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93440047d0 0x7f9344004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935b7fe640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440039b0 0x7f9344003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935bfff640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15a710 0x7f9344005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935affd640 1 -- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15a710 msgr2=0x7f9344005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935affd640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15a710 0x7f9344005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935affd640 1 -- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440039b0 msgr2=0x7f9344003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:18.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935affd640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440039b0 0x7f9344003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.674+0000 7f935affd640 1 -- 172.21.15.31:0/633760935 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9344003530 con 0x7f93440047d0 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f935affd640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93440047d0 0x7f9344004c50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f935c15ab10 tx=0x7f935001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fc4065a3df3a27 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500a64c0 con 0x7f93440047d0 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93500a6620 con 0x7f93440047d0 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f935a7fc640 1 -- 172.21.15.31:0/633760935 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500a69d0 con 0x7f93440047d0 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f9361e32640 1 -- 172.21.15.31:0/633760935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93440047d0 msgr2=0x7f9344004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f9361e32640 1 --2- 172.21.15.31:0/633760935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93440047d0 0x7f9344004c50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f935c15ab10 tx=0x7f935001c9d0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f935bfff640 1 -- 172.21.15.31:0/633760935 reap_dead start 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f9361e32640 1 -- 172.21.15.31:0/633760935 shutdown_connections 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f9361e32640 1 -- 172.21.15.31:0/633760935 >> 172.21.15.31:0/633760935 conn(0x7f935c0b0cb0 msgr2=0x7f935c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f9361e32640 1 -- 172.21.15.31:0/633760935 shutdown_connections 2023-11-26T07:21:18.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.678+0000 7f9361e32640 1 -- 172.21.15.31:0/633760935 wait complete. 2023-11-26T07:21:18.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9361e32640 1 Processor -- start 2023-11-26T07:21:18.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9361e32640 1 -- start start 2023-11-26T07:21:18.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9361e32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440047d0 0x7f935c155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:18.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9361e32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15b770 0x7f935c0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:18.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9361e32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f935c0cf470 0x7f935c09b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:18.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9361e32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c1f3f60 con 0x7f935c0cf470 2023-11-26T07:21:18.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9361e32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0cf900 con 0x7f93440047d0 2023-11-26T07:21:18.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9361e32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c1f32e0 con 0x7f935c15b770 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f935c0cf470 0x7f935c09b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f935c0cf470 0x7f935c09b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57934/0 (socket says 172.21.15.31:57934) 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935bfff640 1 -- 172.21.15.31:0/324743942 learned_addr learned my addr 172.21.15.31:0/324743942 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935affd640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15b770 0x7f935c0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935b7fe640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440047d0 0x7f935c155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935bfff640 1 -- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15b770 msgr2=0x7f935c0cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935bfff640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15b770 0x7f935c0cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935bfff640 1 -- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440047d0 msgr2=0x7f935c155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935bfff640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440047d0 0x7f935c155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935bfff640 1 -- 172.21.15.31:0/324743942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c1f35c0 con 0x7f935c0cf470 2023-11-26T07:21:18.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f935bfff640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f935c0cf470 0x7f935c09b730 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f935c15f120 tx=0x7f935400fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:18.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9343fff640 1 -- 172.21.15.31:0/324743942 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354006d70 con 0x7f935c0cf470 2023-11-26T07:21:18.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9343fff640 1 -- 172.21.15.31:0/324743942 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9354003980 con 0x7f935c0cf470 2023-11-26T07:21:18.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9343fff640 1 -- 172.21.15.31:0/324743942 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354003d30 con 0x7f935c0cf470 2023-11-26T07:21:18.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.682+0000 7f9361e32640 1 -- 172.21.15.31:0/324743942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c1f38a0 con 0x7f935c0cf470 2023-11-26T07:21:18.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.686+0000 7f9361e32640 1 -- 172.21.15.31:0/324743942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c1559d0 con 0x7f935c0cf470 2023-11-26T07:21:18.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.686+0000 7f9361e32640 1 -- 172.21.15.31:0/324743942 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f935c0671f0 con 0x7f935c0cf470 2023-11-26T07:21:18.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.686+0000 7f9343fff640 1 -- 172.21.15.31:0/324743942 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f9354005110 con 0x7f935c0cf470 2023-11-26T07:21:18.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.686+0000 7f9343fff640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93480666b0 0x7f9348068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:18.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.686+0000 7f9343fff640 1 -- 172.21.15.31:0/324743942 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9354083710 con 0x7f935c0cf470 2023-11-26T07:21:18.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.686+0000 7f935b7fe640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93480666b0 0x7f9348068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:18.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.690+0000 7f935b7fe640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93480666b0 0x7f9348068b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f935c072780 tx=0x7f934c014040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:18.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.690+0000 7f9343fff640 1 -- 172.21.15.31:0/324743942 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f93540508d0 con 0x7f935c0cf470 2023-11-26T07:21:18.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.874+0000 7f9361e32640 1 -- 172.21.15.31:0/324743942 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol13010", "target": ["mon-mgr", ""]}) v1 -- 0x7f935c06b460 con 0x7f93480666b0 2023-11-26T07:21:18.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9343fff640 1 -- 172.21.15.31:0/324743942 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f935c06b460 con 0x7f93480666b0 2023-11-26T07:21:18.912 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvol13010 2023-11-26T07:21:18.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 -- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93480666b0 msgr2=0x7f9348068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93480666b0 0x7f9348068b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f935c072780 tx=0x7f934c014040 comp rx=0 tx=0).stop 2023-11-26T07:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 -- 172.21.15.31:0/324743942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f935c0cf470 msgr2=0x7f935c09b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f935c0cf470 0x7f935c09b730 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f935c15f120 tx=0x7f935400fcd0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 -- 172.21.15.31:0/324743942 shutdown_connections 2023-11-26T07:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93480666b0 0x7f9348068b70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f935c0cf470 0x7f935c09b730 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f935c15b770 0x7f935c0cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 --2- 172.21.15.31:0/324743942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93440047d0 0x7f935c155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 -- 172.21.15.31:0/324743942 >> 172.21.15.31:0/324743942 conn(0x7f935c0b0cb0 msgr2=0x7f935c159c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:18.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 -- 172.21.15.31:0/324743942 shutdown_connections 2023-11-26T07:21:18.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:18.910+0000 7f9361e32640 1 -- 172.21.15.31:0/324743942 wait complete. 2023-11-26T07:21:18.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol13010 2023-11-26T07:21:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae2466b640 1 Processor -- start 2023-11-26T07:21:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae2466b640 1 -- start start 2023-11-26T07:21:19.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae2466b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 0x7fae1c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae2466b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae1c15a710 0x7fae1c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae2466b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c0ce8d0 con 0x7fae1c0cbd50 2023-11-26T07:21:19.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae2466b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c0cca80 con 0x7fae1c15a710 2023-11-26T07:21:19.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae2466b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0b2710 con 0x7fae1c0cdfb0 2023-11-26T07:21:19.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae22c08640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fae1c0cbd50 0x7fae1c0cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56086/0 (socket says 172.21.15.31:56086) 2023-11-26T07:21:19.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae22c08640 1 -- 172.21.15.31:0/3427280284 learned_addr learned my addr 172.21.15.31:0/3427280284 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:19.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae22407640 1 --2- 172.21.15.31:0/3427280284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 0x7fae1c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae21c06640 1 --2- 172.21.15.31:0/3427280284 >> v2:172.21.15.97:3300/0 conn(0x7fae1c15a710 0x7fae1c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae21405640 1 -- 172.21.15.31:0/3427280284 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160991605 0 0) 0x7fae1c0ce8d0 con 0x7fae1c0cbd50 2023-11-26T07:21:19.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae22407640 1 -- 172.21.15.31:0/3427280284 >> v2:172.21.15.97:3300/0 conn(0x7fae1c15a710 msgr2=0x7fae1c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:19.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae22407640 1 --2- 172.21.15.31:0/3427280284 >> v2:172.21.15.97:3300/0 conn(0x7fae1c15a710 0x7fae1c0ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae22407640 1 -- 172.21.15.31:0/3427280284 >> v1:172.21.15.31:6789/0 conn(0x7fae1c0cbd50 legacy=0x7fae1c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:19.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.078+0000 7fae22407640 1 -- 172.21.15.31:0/3427280284 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c0b29f0 con 0x7fae1c0cdfb0 2023-11-26T07:21:19.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae22407640 1 --2- 172.21.15.31:0/3427280284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 0x7fae1c0abf90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fae180097c0 tx=0x7fae1801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be55fddb85c58460 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:19.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae21405640 1 -- 172.21.15.31:0/3427280284 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180ab650 con 0x7fae1c0cdfb0 2023-11-26T07:21:19.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae21405640 1 -- 172.21.15.31:0/3427280284 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae180ab7b0 con 0x7fae1c0cdfb0 2023-11-26T07:21:19.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae21405640 1 -- 172.21.15.31:0/3427280284 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae180abb40 con 0x7fae1c0cdfb0 2023-11-26T07:21:19.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae2466b640 1 -- 172.21.15.31:0/3427280284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 msgr2=0x7fae1c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:19.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae2466b640 1 --2- 172.21.15.31:0/3427280284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 0x7fae1c0abf90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fae180097c0 tx=0x7fae1801c450 comp rx=0 tx=0).stop 2023-11-26T07:21:19.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae2466b640 1 -- 172.21.15.31:0/3427280284 shutdown_connections 2023-11-26T07:21:19.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae2466b640 1 --2- 172.21.15.31:0/3427280284 >> v2:172.21.15.97:3300/0 conn(0x7fae1c15a710 0x7fae1c0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae2466b640 1 --2- 172.21.15.31:0/3427280284 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 0x7fae1c0abf90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae2466b640 1 -- 172.21.15.31:0/3427280284 >> 172.21.15.31:0/3427280284 conn(0x7fae1c0b0cb0 msgr2=0x7fae1c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:19.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae2466b640 1 -- 172.21.15.31:0/3427280284 shutdown_connections 2023-11-26T07:21:19.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.082+0000 7fae2466b640 1 -- 172.21.15.31:0/3427280284 wait complete. 2023-11-26T07:21:19.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae2466b640 1 Processor -- start 2023-11-26T07:21:19.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae2466b640 1 -- start start 2023-11-26T07:21:19.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae2466b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae1c0cbd50 0x7fae1c158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae2466b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 0x7fae1c158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae2466b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c15a710 0x7fae1c1635a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae2466b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1f6f80 con 0x7fae1c15a710 2023-11-26T07:21:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae2466b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1557c0 con 0x7fae1c0cbd50 2023-11-26T07:21:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae2466b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c159200 con 0x7fae1c0cdfb0 2023-11-26T07:21:19.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae22c08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c15a710 0x7fae1c1635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae22c08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c15a710 0x7fae1c1635a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57942/0 (socket says 172.21.15.31:57942) 2023-11-26T07:21:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae22c08640 1 -- 172.21.15.31:0/2051421828 learned_addr learned my addr 172.21.15.31:0/2051421828 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae22407640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae1c0cbd50 0x7fae1c158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae21c06640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 0x7fae1c158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae22c08640 1 -- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 msgr2=0x7fae1c158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:19.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae22c08640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 0x7fae1c158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae22c08640 1 -- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae1c0cbd50 msgr2=0x7fae1c158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae22c08640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae1c0cbd50 0x7fae1c158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae22c08640 1 -- 172.21.15.31:0/2051421828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae18009420 con 0x7fae1c15a710 2023-11-26T07:21:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae22c08640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c15a710 0x7fae1c1635a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fae1400c7a0 tx=0x7fae1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae12ffd640 1 -- 172.21.15.31:0/2051421828 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1400a640 con 0x7fae1c15a710 2023-11-26T07:21:19.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae12ffd640 1 -- 172.21.15.31:0/2051421828 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1400a7a0 con 0x7fae1c15a710 2023-11-26T07:21:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae12ffd640 1 -- 172.21.15.31:0/2051421828 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1400abd0 con 0x7fae1c15a710 2023-11-26T07:21:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae2466b640 1 -- 172.21.15.31:0/2051421828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c1594e0 con 0x7fae1c15a710 2023-11-26T07:21:19.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.086+0000 7fae2466b640 1 -- 172.21.15.31:0/2051421828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c159aa0 con 0x7fae1c15a710 2023-11-26T07:21:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.090+0000 7fae2466b640 1 -- 172.21.15.31:0/2051421828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c0671f0 con 0x7fae1c15a710 2023-11-26T07:21:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.090+0000 7fae12ffd640 1 -- 172.21.15.31:0/2051421828 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 86056+0+0 (secure 0 0 0) 0x7fae1400ad30 con 0x7fae1c15a710 2023-11-26T07:21:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.090+0000 7fae12ffd640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae000666b0 0x7fae00068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.090+0000 7fae12ffd640 1 -- 172.21.15.31:0/2051421828 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae14083820 con 0x7fae1c15a710 2023-11-26T07:21:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.090+0000 7fae22407640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae000666b0 0x7fae00068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:19.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.090+0000 7fae22407640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae000666b0 0x7fae00068b70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fae180af000 tx=0x7fae180a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:19.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.094+0000 7fae12ffd640 1 -- 172.21.15.31:0/2051421828 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae14050950 con 0x7fae1c15a710 2023-11-26T07:21:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.278+0000 7fae2466b640 1 -- 172.21.15.31:0/2051421828 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol13010", "target": ["mon-mgr", ""]}) v1 -- 0x7fae1c159cc0 con 0x7fae000666b0 2023-11-26T07:21:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae12ffd640 1 -- 172.21.15.31:0/2051421828 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7fae1c159cc0 con 0x7fae000666b0 2023-11-26T07:21:19.284 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvol13010 2023-11-26T07:21:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 -- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae000666b0 msgr2=0x7fae00068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae000666b0 0x7fae00068b70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fae180af000 tx=0x7fae180a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 -- 172.21.15.31:0/2051421828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c15a710 msgr2=0x7fae1c1635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c15a710 0x7fae1c1635a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fae1400c7a0 tx=0x7fae1400cc60 comp rx=0 tx=0).stop 2023-11-26T07:21:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 -- 172.21.15.31:0/2051421828 shutdown_connections 2023-11-26T07:21:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fae000666b0 0x7fae00068b70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae1c15a710 0x7fae1c1635a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae1c0cdfb0 0x7fae1c158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 --2- 172.21.15.31:0/2051421828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae1c0cbd50 0x7fae1c158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 -- 172.21.15.31:0/2051421828 >> 172.21.15.31:0/2051421828 conn(0x7fae1c0b0cb0 msgr2=0x7fae1c0ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 -- 172.21.15.31:0/2051421828 shutdown_connections 2023-11-26T07:21:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:19.282+0000 7fae2466b640 1 -- 172.21.15.31:0/2051421828 wait complete. 2023-11-26T07:21:19.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:19.305 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/9e10d006eab936db59adcf48af840471.meta) 2023-11-26T07:21:23.512 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-11-26T07:21:23.512 INFO:teuthology.orchestra.run.smithi031.stdout:version = 1 2023-11-26T07:21:23.512 INFO:teuthology.orchestra.run.smithi031.stdout:type = subvolume 2023-11-26T07:21:23.513 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvol13010 2023-11-26T07:21:23.513 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-11-26T07:21:23.513 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:23.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol13010 alice 2023-11-26T07:21:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f4271fdf640 1 Processor -- start 2023-11-26T07:21:23.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f4271fdf640 1 -- start start 2023-11-26T07:21:23.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f4271fdf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c0cdfb0 0x7f426c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:23.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f4271fdf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f426c15a710 0x7f426c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:23.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f4271fdf640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426c0ce8d0 con 0x7f426c0cbd50 2023-11-26T07:21:23.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f4271fdf640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426c0cca80 con 0x7f426c15a710 2023-11-26T07:21:23.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f4271fdf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0b2710 con 0x7f426c0cdfb0 2023-11-26T07:21:23.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f426bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f426c0cbd50 0x7f426c0cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41100/0 (socket says 172.21.15.31:41100) 2023-11-26T07:21:23.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f426bfff640 1 -- 172.21.15.31:0/287566757 learned_addr learned my addr 172.21.15.31:0/287566757 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:23.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f426b7fe640 1 --2- 172.21.15.31:0/287566757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c0cdfb0 0x7f426c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:23.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f426affd640 1 --2- 172.21.15.31:0/287566757 >> v2:172.21.15.97:3300/0 conn(0x7f426c15a710 0x7f426c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:23.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f426a7fc640 1 -- 172.21.15.31:0/287566757 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985154828 0 0) 0x7f426c0ce8d0 con 0x7f426c0cbd50 2023-11-26T07:21:23.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f426a7fc640 1 -- 172.21.15.31:0/287566757 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4254003530 con 0x7f426c0cbd50 2023-11-26T07:21:23.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f426b7fe640 1 -- 172.21.15.31:0/287566757 >> v2:172.21.15.97:3300/0 conn(0x7f426c15a710 msgr2=0x7f426c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:23.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f426b7fe640 1 --2- 172.21.15.31:0/287566757 >> v2:172.21.15.97:3300/0 conn(0x7f426c15a710 0x7f426c0ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:23.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f426b7fe640 1 -- 172.21.15.31:0/287566757 >> v1:172.21.15.31:6789/0 conn(0x7f426c0cbd50 legacy=0x7f426c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:23.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.670+0000 7f426b7fe640 1 -- 172.21.15.31:0/287566757 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0b29f0 con 0x7f426c0cdfb0 2023-11-26T07:21:23.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f426b7fe640 1 --2- 172.21.15.31:0/287566757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c0cdfb0 0x7f426c0abf90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f425c0097c0 tx=0x7f425c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73147430bb606774 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:23.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f426a7fc640 1 -- 172.21.15.31:0/287566757 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c0ab650 con 0x7f426c0cdfb0 2023-11-26T07:21:23.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f426a7fc640 1 -- 172.21.15.31:0/287566757 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c0ab7b0 con 0x7f426c0cdfb0 2023-11-26T07:21:23.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f426a7fc640 1 -- 172.21.15.31:0/287566757 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c0abb20 con 0x7f426c0cdfb0 2023-11-26T07:21:23.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f4271fdf640 1 -- 172.21.15.31:0/287566757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c0cdfb0 msgr2=0x7f426c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:23.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f4271fdf640 1 --2- 172.21.15.31:0/287566757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c0cdfb0 0x7f426c0abf90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f425c0097c0 tx=0x7f425c01c450 comp rx=0 tx=0).stop 2023-11-26T07:21:23.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f4271fdf640 1 -- 172.21.15.31:0/287566757 shutdown_connections 2023-11-26T07:21:23.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f4271fdf640 1 --2- 172.21.15.31:0/287566757 >> v2:172.21.15.97:3300/0 conn(0x7f426c15a710 0x7f426c0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:23.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f4271fdf640 1 --2- 172.21.15.31:0/287566757 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c0cdfb0 0x7f426c0abf90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:23.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f4271fdf640 1 -- 172.21.15.31:0/287566757 >> 172.21.15.31:0/287566757 conn(0x7f426c0b0cb0 msgr2=0x7f426c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:23.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f4271fdf640 1 -- 172.21.15.31:0/287566757 shutdown_connections 2023-11-26T07:21:23.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.674+0000 7f4271fdf640 1 -- 172.21.15.31:0/287566757 wait complete. 2023-11-26T07:21:23.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4271fdf640 1 Processor -- start 2023-11-26T07:21:23.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4271fdf640 1 -- start start 2023-11-26T07:21:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4271fdf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f426c0cbd50 0x7f426c071070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4271fdf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f426c0cdfb0 0x7f426c15b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4271fdf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c15a710 0x7f426c1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:23.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4271fdf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c071730 con 0x7f426c0cbd50 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4271fdf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426c0aca60 con 0x7f426c0cdfb0 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4271fdf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426c1531d0 con 0x7f426c15a710 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f426c0cbd50 0x7f426c071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f426c0cbd50 0x7f426c071070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57896/0 (socket says 172.21.15.31:57896) 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426b7fe640 1 -- 172.21.15.31:0/3311277824 learned_addr learned my addr 172.21.15.31:0/3311277824 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426affd640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f426c0cdfb0 0x7f426c15b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426bfff640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c15a710 0x7f426c1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426b7fe640 1 -- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c15a710 msgr2=0x7f426c1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426b7fe640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c15a710 0x7f426c1445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426b7fe640 1 -- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f426c0cdfb0 msgr2=0x7f426c15b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:23.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426b7fe640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f426c0cdfb0 0x7f426c15b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:23.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426b7fe640 1 -- 172.21.15.31:0/3311277824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c009420 con 0x7f426c0cbd50 2023-11-26T07:21:23.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f426b7fe640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f426c0cbd50 0x7f426c071070 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f425c01ccf0 tx=0x7f425c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:23.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4253fff640 1 -- 172.21.15.31:0/3311277824 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c0bf600 con 0x7f426c0cbd50 2023-11-26T07:21:23.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4253fff640 1 -- 172.21.15.31:0/3311277824 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c0bf760 con 0x7f426c0cbd50 2023-11-26T07:21:23.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4253fff640 1 -- 172.21.15.31:0/3311277824 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c0bfad0 con 0x7f426c0cbd50 2023-11-26T07:21:23.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4271fdf640 1 -- 172.21.15.31:0/3311277824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426c0ac130 con 0x7f426c0cbd50 2023-11-26T07:21:23.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.678+0000 7f4271fdf640 1 -- 172.21.15.31:0/3311277824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426c0ac6f0 con 0x7f426c0cbd50 2023-11-26T07:21:23.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.682+0000 7f4271fdf640 1 -- 172.21.15.31:0/3311277824 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f426c0671f0 con 0x7f426c0cbd50 2023-11-26T07:21:23.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.682+0000 7f4253fff640 1 -- 172.21.15.31:0/3311277824 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f425c0b5400 con 0x7f426c0cbd50 2023-11-26T07:21:23.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.682+0000 7f4253fff640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42580666b0 0x7f4258068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:23.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.682+0000 7f4253fff640 1 -- 172.21.15.31:0/3311277824 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f425c11f520 con 0x7f426c0cbd50 2023-11-26T07:21:23.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.682+0000 7f426affd640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42580666b0 0x7f4258068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:23.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.682+0000 7f426affd640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42580666b0 0x7f4258068b70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f426c0ae4b0 tx=0x7f4260009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:23.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.686+0000 7f4253fff640 1 -- 172.21.15.31:0/3311277824 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f425c0ec6e0 con 0x7f426c0cbd50 2023-11-26T07:21:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.866+0000 7f4271fdf640 1 -- 172.21.15.31:0/3311277824 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol13010", "auth_id": "alice", "target": ["mon-mgr", ""]}) v1 -- 0x7f426c06b460 con 0x7f42580666b0 2023-11-26T07:21:23.882 INFO:tasks.ceph.mgr.x.smithi097.stderr:Creating meta for ID alice with tenant None 2023-11-26T07:21:23.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.914+0000 7f4253fff640 1 -- 172.21.15.31:0/3311277824 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f426c06b460 con 0x7f42580666b0 2023-11-26T07:21:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 -- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42580666b0 msgr2=0x7f4258068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42580666b0 0x7f4258068b70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f426c0ae4b0 tx=0x7f4260009190 comp rx=0 tx=0).stop 2023-11-26T07:21:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 -- 172.21.15.31:0/3311277824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f426c0cbd50 msgr2=0x7f426c071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f426c0cbd50 0x7f426c071070 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f425c01ccf0 tx=0x7f425c01ac90 comp rx=0 tx=0).stop 2023-11-26T07:21:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 -- 172.21.15.31:0/3311277824 shutdown_connections 2023-11-26T07:21:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42580666b0 0x7f4258068b70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f426c15a710 0x7f426c1445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f426c0cdfb0 0x7f426c15b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 --2- 172.21.15.31:0/3311277824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f426c0cbd50 0x7f426c071070 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 -- 172.21.15.31:0/3311277824 >> 172.21.15.31:0/3311277824 conn(0x7f426c0b0cb0 msgr2=0x7f426c0afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 -- 172.21.15.31:0/3311277824 shutdown_connections 2023-11-26T07:21:23.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:23.918+0000 7f4271fdf640 1 -- 172.21.15.31:0/3311277824 wait complete. 2023-11-26T07:21:23.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2023-11-26T07:21:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.078+0000 7f7caca06640 1 Processor -- start 2023-11-26T07:21:24.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.078+0000 7f7caca06640 1 -- start start 2023-11-26T07:21:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7caca06640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b07a0 0x7f7ca81619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7caca06640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ca8161f20 0x7f7ca80b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7caca06640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca8161290 con 0x7f7ca80b2870 2023-11-26T07:21:24.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7caca06640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80b0b80 con 0x7f7ca8161f20 2023-11-26T07:21:24.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7caca06640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca815ebc0 con 0x7f7ca80b07a0 2023-11-26T07:21:24.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7c9ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ca8161f20 0x7f7ca80b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:24.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7ca659c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b07a0 0x7f7ca81619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7ca6d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f7ca80b2870 0x7f7ca80cc9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:55458/0 (socket says 172.21.15.31:55458) 2023-11-26T07:21:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7ca659c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b07a0 0x7f7ca81619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:45950/0 (socket says 172.21.15.31:45950) 2023-11-26T07:21:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7ca6d9d640 1 -- 172.21.15.31:0/1194248365 learned_addr learned my addr 172.21.15.31:0/1194248365 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7c9ffff640 1 --2- 172.21.15.31:0/1194248365 >> v2:172.21.15.97:3300/0 conn(0x7f7ca8161f20 0x7f7ca80b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:60110/0 (socket says 172.21.15.31:60110) 2023-11-26T07:21:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7ca659c640 1 -- 172.21.15.31:0/1194248365 >> v2:172.21.15.97:3300/0 conn(0x7f7ca8161f20 msgr2=0x7f7ca80b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7ca659c640 1 --2- 172.21.15.31:0/1194248365 >> v2:172.21.15.97:3300/0 conn(0x7f7ca8161f20 0x7f7ca80b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7ca659c640 1 -- 172.21.15.31:0/1194248365 >> v1:172.21.15.97:6789/0 conn(0x7f7ca80b2870 legacy=0x7f7ca80cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7ca659c640 1 -- 172.21.15.31:0/1194248365 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca8161550 con 0x7f7ca80b07a0 2023-11-26T07:21:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7c9ffff640 1 --2- 172.21.15.31:0/1194248365 >> v2:172.21.15.97:3300/0 conn(0x7f7ca8161f20 0x7f7ca80b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:24.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.082+0000 7f7ca5d9b640 1 -- 172.21.15.31:0/1194248365 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663387728 0 0) 0x7f7ca8161290 con 0x7f7ca80b2870 2023-11-26T07:21:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7ca659c640 1 --2- 172.21.15.31:0/1194248365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b07a0 0x7f7ca81619e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c980097c0 tx=0x7f7c9801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e192f09de59ddf8b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7ca5d9b640 1 -- 172.21.15.31:0/1194248365 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c980ab600 con 0x7f7ca80b07a0 2023-11-26T07:21:24.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7ca5d9b640 1 -- 172.21.15.31:0/1194248365 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c980a7050 con 0x7f7ca80b07a0 2023-11-26T07:21:24.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7ca5d9b640 1 -- 172.21.15.31:0/1194248365 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c980ab980 con 0x7f7ca80b07a0 2023-11-26T07:21:24.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7caca06640 1 -- 172.21.15.31:0/1194248365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b07a0 msgr2=0x7f7ca81619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:24.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7caca06640 1 --2- 172.21.15.31:0/1194248365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b07a0 0x7f7ca81619e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7c980097c0 tx=0x7f7c9801c480 comp rx=0 tx=0).stop 2023-11-26T07:21:24.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7caca06640 1 -- 172.21.15.31:0/1194248365 shutdown_connections 2023-11-26T07:21:24.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7caca06640 1 --2- 172.21.15.31:0/1194248365 >> v2:172.21.15.97:3300/0 conn(0x7f7ca8161f20 0x7f7ca80b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:24.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7caca06640 1 --2- 172.21.15.31:0/1194248365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b07a0 0x7f7ca81619e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:24.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7caca06640 1 -- 172.21.15.31:0/1194248365 >> 172.21.15.31:0/1194248365 conn(0x7f7ca80b1d50 msgr2=0x7f7ca80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:24.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7caca06640 1 -- 172.21.15.31:0/1194248365 shutdown_connections 2023-11-26T07:21:24.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7caca06640 1 -- 172.21.15.31:0/1194248365 wait complete. 2023-11-26T07:21:24.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7caca06640 1 Processor -- start 2023-11-26T07:21:24.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.086+0000 7f7caca06640 1 -- start start 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7caca06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80b07a0 0x7f7ca80ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7caca06640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b2870 0x7f7ca8158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7caca06640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca8161f20 0x7f7ca80b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7caca06640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80ae2a0 con 0x7f7ca80b07a0 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7caca06640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80aff70 con 0x7f7ca8161f20 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7caca06640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80adbe0 con 0x7f7ca80b2870 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7ca659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80b07a0 0x7f7ca80ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7ca659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80b07a0 0x7f7ca80ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57898/0 (socket says 172.21.15.31:57898) 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7ca659c640 1 -- 172.21.15.31:0/2754441589 learned_addr learned my addr 172.21.15.31:0/2754441589 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7ca6d9d640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca8161f20 0x7f7ca80b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:24.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7c9ffff640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b2870 0x7f7ca8158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7ca659c640 1 -- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b2870 msgr2=0x7f7ca8158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7ca659c640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b2870 0x7f7ca8158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7ca659c640 1 -- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca8161f20 msgr2=0x7f7ca80b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7ca659c640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca8161f20 0x7f7ca80b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7ca659c640 1 -- 172.21.15.31:0/2754441589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c90009420 con 0x7f7ca80b07a0 2023-11-26T07:21:24.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7ca659c640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80b07a0 0x7f7ca80ae6c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7c9801a0b0 tx=0x7f7c98019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:24.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7c9effd640 1 -- 172.21.15.31:0/2754441589 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98004030 con 0x7f7ca80b07a0 2023-11-26T07:21:24.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7c9effd640 1 -- 172.21.15.31:0/2754441589 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c98004190 con 0x7f7ca80b07a0 2023-11-26T07:21:24.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7c9effd640 1 -- 172.21.15.31:0/2754441589 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98004500 con 0x7f7ca80b07a0 2023-11-26T07:21:24.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7caca06640 1 -- 172.21.15.31:0/2754441589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c98009420 con 0x7f7ca80b07a0 2023-11-26T07:21:24.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7caca06640 1 -- 172.21.15.31:0/2754441589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca8159d80 con 0x7f7ca80b07a0 2023-11-26T07:21:24.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7caca06640 1 -- 172.21.15.31:0/2754441589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca80ada40 con 0x7f7ca80b07a0 2023-11-26T07:21:24.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.090+0000 7f7c9effd640 1 -- 172.21.15.31:0/2754441589 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f7c980b2040 con 0x7f7ca80b07a0 2023-11-26T07:21:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.094+0000 7f7c9effd640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7c880666b0 0x7f7c88068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:24.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.094+0000 7f7c9effd640 1 -- 172.21.15.31:0/2754441589 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c9811d550 con 0x7f7ca80b07a0 2023-11-26T07:21:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.094+0000 7f7c9ffff640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7c880666b0 0x7f7c88068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:24.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.094+0000 7f7c9ffff640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7c880666b0 0x7f7c88068b70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7ca81f9d60 tx=0x7f7c90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:24.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.098+0000 7f7c9effd640 1 -- 172.21.15.31:0/2754441589 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7ca80ada40 con 0x7f7ca80b07a0 2023-11-26T07:21:24.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.310+0000 7f7caca06640 1 -- 172.21.15.31:0/2754441589 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) v1 -- 0x7f7ca80ada40 con 0x7f7ca80b07a0 2023-11-26T07:21:27.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.310+0000 7f7c9effd640 1 -- 172.21.15.31:0/2754441589 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v18) v1 ==== 107+0+280 (secure 0 0 0) 0x7f7ca80ada40 con 0x7f7ca80b07a0 2023-11-26T07:21:27.493 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:21:27.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:27.493 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.alice", 2023-11-26T07:21:27.494 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDz8WJlsXRVNRAANIcQYI6xYe6U5js+J39pug==", 2023-11-26T07:21:27.494 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:27.494 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol13010", 2023-11-26T07:21:27.494 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-11-26T07:21:27.494 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-26T07:21:27.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:27.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:27.494 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:21:27.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 -- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7c880666b0 msgr2=0x7f7c88068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7c880666b0 0x7f7c88068b70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7ca81f9d60 tx=0x7f7c90008040 comp rx=0 tx=0).stop 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 -- 172.21.15.31:0/2754441589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80b07a0 msgr2=0x7f7ca80ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80b07a0 0x7f7ca80ae6c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7c9801a0b0 tx=0x7f7c98019220 comp rx=0 tx=0).stop 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 -- 172.21.15.31:0/2754441589 shutdown_connections 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7c880666b0 0x7f7c88068b70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca8161f20 0x7f7ca80b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ca80b2870 0x7f7ca8158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 --2- 172.21.15.31:0/2754441589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80b07a0 0x7f7ca80ae6c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 -- 172.21.15.31:0/2754441589 >> 172.21.15.31:0/2754441589 conn(0x7f7ca80b1d50 msgr2=0x7f7ca806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 -- 172.21.15.31:0/2754441589 shutdown_connections 2023-11-26T07:21:27.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:24.314+0000 7f7caca06640 1 -- 172.21.15.31:0/2754441589 wait complete. 2023-11-26T07:21:27.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol13010 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.650+0000 7f2257e68640 1 Processor -- start 2023-11-26T07:21:27.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.650+0000 7f2257e68640 1 -- start start 2023-11-26T07:21:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.650+0000 7f2257e68640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22500cdfb0 0x7f22500abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.650+0000 7f2257e68640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500b2710 con 0x7f225015a710 2023-11-26T07:21:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.650+0000 7f2257e68640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22500b29f0 con 0x7f22500cbd50 2023-11-26T07:21:27.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.650+0000 7f2257e68640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22500cca80 con 0x7f22500cdfb0 2023-11-26T07:21:27.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2255403640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f225015a710 0x7f22500ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41112/0 (socket says 172.21.15.31:41112) 2023-11-26T07:21:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2255403640 1 -- 172.21.15.31:0/3405791208 learned_addr learned my addr 172.21.15.31:0/3405791208 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2255c04640 1 --2- 172.21.15.31:0/3405791208 >> v2:172.21.15.31:3300/0 conn(0x7f22500cdfb0 0x7f22500abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2254c02640 1 -- 172.21.15.31:0/3405791208 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051303810 0 0) 0x7f22500b2710 con 0x7f225015a710 2023-11-26T07:21:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2254c02640 1 -- 172.21.15.31:0/3405791208 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f223c003530 con 0x7f225015a710 2023-11-26T07:21:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2255c04640 1 -- 172.21.15.31:0/3405791208 >> v1:172.21.15.97:6789/0 conn(0x7f22500cbd50 legacy=0x7f22500cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2255c04640 1 -- 172.21.15.31:0/3405791208 >> v1:172.21.15.31:6789/0 conn(0x7f225015a710 legacy=0x7f22500ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2255c04640 1 -- 172.21.15.31:0/3405791208 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22500b2cd0 con 0x7f22500cdfb0 2023-11-26T07:21:27.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2255c04640 1 --2- 172.21.15.31:0/3405791208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cdfb0 0x7f22500abf90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f224c0097c0 tx=0x7f224c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a78b5c50be908cf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:27.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2254c02640 1 -- 172.21.15.31:0/3405791208 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c0ab670 con 0x7f22500cdfb0 2023-11-26T07:21:27.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2254c02640 1 -- 172.21.15.31:0/3405791208 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c0ab7d0 con 0x7f22500cdfb0 2023-11-26T07:21:27.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.654+0000 7f2254c02640 1 -- 172.21.15.31:0/3405791208 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c0abb60 con 0x7f22500cdfb0 2023-11-26T07:21:27.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 -- 172.21.15.31:0/3405791208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cdfb0 msgr2=0x7f22500abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 --2- 172.21.15.31:0/3405791208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cdfb0 0x7f22500abf90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f224c0097c0 tx=0x7f224c01c450 comp rx=0 tx=0).stop 2023-11-26T07:21:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 -- 172.21.15.31:0/3405791208 shutdown_connections 2023-11-26T07:21:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 --2- 172.21.15.31:0/3405791208 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cdfb0 0x7f22500abf90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 -- 172.21.15.31:0/3405791208 >> 172.21.15.31:0/3405791208 conn(0x7f22500b0cb0 msgr2=0x7f22500b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 -- 172.21.15.31:0/3405791208 shutdown_connections 2023-11-26T07:21:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 -- 172.21.15.31:0/3405791208 wait complete. 2023-11-26T07:21:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 Processor -- start 2023-11-26T07:21:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 -- start start 2023-11-26T07:21:27.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cbd50 0x7f22500afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22500cdfb0 0x7f2250146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f225015a710 0x7f225015d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250159e90 con 0x7f22500cbd50 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250147100 con 0x7f22500cdfb0 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2257e68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22500b05b0 con 0x7f225015a710 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2255c04640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cbd50 0x7f22500afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2255c04640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cbd50 0x7f22500afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57918/0 (socket says 172.21.15.31:57918) 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.658+0000 7f2255c04640 1 -- 172.21.15.31:0/1156722277 learned_addr learned my addr 172.21.15.31:0/1156722277 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2255403640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22500cdfb0 0x7f2250146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2255c04640 1 -- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f225015a710 msgr2=0x7f225015d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2255c04640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f225015a710 0x7f225015d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2255c04640 1 -- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22500cdfb0 msgr2=0x7f2250146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2255c04640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22500cdfb0 0x7f2250146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2255c04640 1 -- 172.21.15.31:0/1156722277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c009420 con 0x7f22500cbd50 2023-11-26T07:21:27.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2255403640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22500cdfb0 0x7f2250146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:27.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2255c04640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cbd50 0x7f22500afed0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f224c0ad040 tx=0x7f224c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:27.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f22467fc640 1 -- 172.21.15.31:0/1156722277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c0b58d0 con 0x7f22500cbd50 2023-11-26T07:21:27.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f22467fc640 1 -- 172.21.15.31:0/1156722277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c0b5a50 con 0x7f22500cbd50 2023-11-26T07:21:27.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f22467fc640 1 -- 172.21.15.31:0/1156722277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c0b5e00 con 0x7f22500cbd50 2023-11-26T07:21:27.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2257e68640 1 -- 172.21.15.31:0/1156722277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22500b0890 con 0x7f22500cbd50 2023-11-26T07:21:27.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2257e68640 1 -- 172.21.15.31:0/1156722277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22501466a0 con 0x7f22500cbd50 2023-11-26T07:21:27.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f2257e68640 1 -- 172.21.15.31:0/1156722277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22500671f0 con 0x7f22500cbd50 2023-11-26T07:21:27.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f22467fc640 1 -- 172.21.15.31:0/1156722277 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f224c0bf410 con 0x7f22500cbd50 2023-11-26T07:21:27.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f22467fc640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f22340666b0 0x7f2234068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:27.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.662+0000 7f22467fc640 1 -- 172.21.15.31:0/1156722277 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f224c11f2d0 con 0x7f22500cbd50 2023-11-26T07:21:27.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.666+0000 7f2255403640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f22340666b0 0x7f2234068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:27.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.666+0000 7f2255403640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f22340666b0 0x7f2234068b70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f22500ac040 tx=0x7f2240009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:27.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.670+0000 7f22467fc640 1 -- 172.21.15.31:0/1156722277 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f224c0ec3f0 con 0x7f22500cbd50 2023-11-26T07:21:27.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.850+0000 7f2257e68640 1 -- 172.21.15.31:0/1156722277 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol13010", "target": ["mon-mgr", ""]}) v1 -- 0x7f22500b0a80 con 0x7f22340666b0 2023-11-26T07:21:27.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f22467fc640 1 -- 172.21.15.31:0/1156722277 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f22500b0a80 con 0x7f22340666b0 2023-11-26T07:21:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 -- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f22340666b0 msgr2=0x7f2234068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f22340666b0 0x7f2234068b70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f22500ac040 tx=0x7f2240009040 comp rx=0 tx=0).stop 2023-11-26T07:21:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 -- 172.21.15.31:0/1156722277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cbd50 msgr2=0x7f22500afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cbd50 0x7f22500afed0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f224c0ad040 tx=0x7f224c01ac90 comp rx=0 tx=0).stop 2023-11-26T07:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 -- 172.21.15.31:0/1156722277 shutdown_connections 2023-11-26T07:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f22340666b0 0x7f2234068b70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f225015a710 0x7f225015d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22500cdfb0 0x7f2250146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 --2- 172.21.15.31:0/1156722277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22500cbd50 0x7f22500afed0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 -- 172.21.15.31:0/1156722277 >> 172.21.15.31:0/1156722277 conn(0x7f22500b0cb0 msgr2=0x7f225015e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 -- 172.21.15.31:0/1156722277 shutdown_connections 2023-11-26T07:21:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:27.870+0000 7f2257e68640 1 -- 172.21.15.31:0/1156722277 wait complete. 2023-11-26T07:21:27.876 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:27.873+0000 7f2402199640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:27.876 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:27.873+0000 7f2402199640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:27.876 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:27.873+0000 7f2402199640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:27.877 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:27.873+0000 7f2402199640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:27.877 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:27.873+0000 7f2402199640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:27.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol52791 2023-11-26T07:21:27.897 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:27.893+0000 7f240319b640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:27.898 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:27.893+0000 7f240319b640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:27.898 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:27.893+0000 7f240319b640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:27.898 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:27.893+0000 7f240319b640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:27.898 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:21:27.893+0000 7f240319b640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:21:28.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.042+0000 7f86e9f57640 1 Processor -- start 2023-11-26T07:21:28.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.042+0000 7f86e9f57640 1 -- start start 2023-11-26T07:21:28.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.042+0000 7f86e9f57640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86e415a710 0x7f86e40abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.042+0000 7f86e9f57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40cbd50 0x7f86e40cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.042+0000 7f86e9f57640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e40b2710 con 0x7f86e40cdfb0 2023-11-26T07:21:28.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.042+0000 7f86e9f57640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e40cca80 con 0x7f86e415a710 2023-11-26T07:21:28.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.042+0000 7f86e9f57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e40b29f0 con 0x7f86e40cbd50 2023-11-26T07:21:28.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e3fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40cbd50 0x7f86e40cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86e415a710 0x7f86e40abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e37fe640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f86e40cdfb0 0x7f86e40ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:55468/0 (socket says 172.21.15.31:55468) 2023-11-26T07:21:28.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e37fe640 1 -- 172.21.15.31:0/3728310597 learned_addr learned my addr 172.21.15.31:0/3728310597 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:28.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e2ffd640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f86e415a710 0x7f86e40abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:60132/0 (socket says 172.21.15.31:60132) 2023-11-26T07:21:28.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e27fc640 1 -- 172.21.15.31:0/3728310597 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146183823 0 0) 0x7f86e40b2710 con 0x7f86e40cdfb0 2023-11-26T07:21:28.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e27fc640 1 -- 172.21.15.31:0/3728310597 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86cc003530 con 0x7f86e40cdfb0 2023-11-26T07:21:28.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e2ffd640 1 -- 172.21.15.31:0/3728310597 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40cbd50 msgr2=0x7f86e40cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:28.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e2ffd640 1 --2- 172.21.15.31:0/3728310597 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40cbd50 0x7f86e40cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e2ffd640 1 -- 172.21.15.31:0/3728310597 >> v1:172.21.15.97:6789/0 conn(0x7f86e40cdfb0 legacy=0x7f86e40ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e2ffd640 1 -- 172.21.15.31:0/3728310597 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e40b2cd0 con 0x7f86e415a710 2023-11-26T07:21:28.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e3fff640 1 --2- 172.21.15.31:0/3728310597 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40cbd50 0x7f86e40cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:28.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.046+0000 7f86e2ffd640 1 --2- 172.21.15.31:0/3728310597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e415a710 0x7f86e40abf90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f86d80097c0 tx=0x7f86d801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f6af80e31dd3cb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:28.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e27fc640 1 -- 172.21.15.31:0/3728310597 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d80ab670 con 0x7f86e415a710 2023-11-26T07:21:28.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e27fc640 1 -- 172.21.15.31:0/3728310597 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d80ab7d0 con 0x7f86e415a710 2023-11-26T07:21:28.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e27fc640 1 -- 172.21.15.31:0/3728310597 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d80abb60 con 0x7f86e415a710 2023-11-26T07:21:28.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 -- 172.21.15.31:0/3728310597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e415a710 msgr2=0x7f86e40abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 --2- 172.21.15.31:0/3728310597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e415a710 0x7f86e40abf90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f86d80097c0 tx=0x7f86d801c450 comp rx=0 tx=0).stop 2023-11-26T07:21:28.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 -- 172.21.15.31:0/3728310597 shutdown_connections 2023-11-26T07:21:28.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 --2- 172.21.15.31:0/3728310597 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e40cbd50 0x7f86e40cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 --2- 172.21.15.31:0/3728310597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e415a710 0x7f86e40abf90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 -- 172.21.15.31:0/3728310597 >> 172.21.15.31:0/3728310597 conn(0x7f86e40b0cb0 msgr2=0x7f86e40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:28.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 -- 172.21.15.31:0/3728310597 shutdown_connections 2023-11-26T07:21:28.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 -- 172.21.15.31:0/3728310597 wait complete. 2023-11-26T07:21:28.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 Processor -- start 2023-11-26T07:21:28.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 -- start start 2023-11-26T07:21:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40cbd50 0x7f86e415ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86e40cdfb0 0x7f86e4152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e415a710 0x7f86e40ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e409ca90 con 0x7f86e40cdfb0 2023-11-26T07:21:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e415baa0 con 0x7f86e40cbd50 2023-11-26T07:21:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e9f57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e415b050 con 0x7f86e415a710 2023-11-26T07:21:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86e40cdfb0 0x7f86e4152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40cbd50 0x7f86e415ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86e40cdfb0 0x7f86e4152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57936/0 (socket says 172.21.15.31:57936) 2023-11-26T07:21:28.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e2ffd640 1 -- 172.21.15.31:0/3685226816 learned_addr learned my addr 172.21.15.31:0/3685226816 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:28.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.050+0000 7f86e3fff640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e415a710 0x7f86e40ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86e2ffd640 1 -- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e415a710 msgr2=0x7f86e40ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86e2ffd640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e415a710 0x7f86e40ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86e2ffd640 1 -- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40cbd50 msgr2=0x7f86e415ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86e2ffd640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40cbd50 0x7f86e415ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86e2ffd640 1 -- 172.21.15.31:0/3685226816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc009a70 con 0x7f86e40cdfb0 2023-11-26T07:21:28.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86e2ffd640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86e40cdfb0 0x7f86e4152e70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f86d801ab00 tx=0x7f86d801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86cbfff640 1 -- 172.21.15.31:0/3685226816 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d80bf610 con 0x7f86e40cdfb0 2023-11-26T07:21:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86cbfff640 1 -- 172.21.15.31:0/3685226816 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d80bf770 con 0x7f86e40cdfb0 2023-11-26T07:21:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86e9f57640 1 -- 172.21.15.31:0/3685226816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d8009420 con 0x7f86e40cdfb0 2023-11-26T07:21:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86cbfff640 1 -- 172.21.15.31:0/3685226816 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d80bfb20 con 0x7f86e40cdfb0 2023-11-26T07:21:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86e9f57640 1 -- 172.21.15.31:0/3685226816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e409cce0 con 0x7f86e40cdfb0 2023-11-26T07:21:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86e9f57640 1 -- 172.21.15.31:0/3685226816 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e40671f0 con 0x7f86e40cdfb0 2023-11-26T07:21:28.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86cbfff640 1 -- 172.21.15.31:0/3685226816 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f86d80bfc80 con 0x7f86e40cdfb0 2023-11-26T07:21:28.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.054+0000 7f86cbfff640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86d00666b0 0x7f86d0068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.058+0000 7f86cbfff640 1 -- 172.21.15.31:0/3685226816 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86d811efa0 con 0x7f86e40cdfb0 2023-11-26T07:21:28.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.058+0000 7f86e37fe640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86d00666b0 0x7f86d0068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.058+0000 7f86e37fe640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86d00666b0 0x7f86d0068b70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86d40011f0 tx=0x7f86d4007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:28.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.058+0000 7f86cbfff640 1 -- 172.21.15.31:0/3685226816 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86d80ec170 con 0x7f86e40cdfb0 2023-11-26T07:21:28.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.242+0000 7f86e9f57640 1 -- 172.21.15.31:0/3685226816 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol52791", "target": ["mon-mgr", ""]}) v1 -- 0x7f86e406b460 con 0x7f86d00666b0 2023-11-26T07:21:28.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.262+0000 7f86cbfff640 1 -- 172.21.15.31:0/3685226816 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f86e406b460 con 0x7f86d00666b0 2023-11-26T07:21:28.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 -- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86d00666b0 msgr2=0x7f86d0068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86d00666b0 0x7f86d0068b70 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86d40011f0 tx=0x7f86d4007d40 comp rx=0 tx=0).stop 2023-11-26T07:21:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 -- 172.21.15.31:0/3685226816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86e40cdfb0 msgr2=0x7f86e4152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86e40cdfb0 0x7f86e4152e70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f86d801ab00 tx=0x7f86d801ac90 comp rx=0 tx=0).stop 2023-11-26T07:21:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 -- 172.21.15.31:0/3685226816 shutdown_connections 2023-11-26T07:21:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86d00666b0 0x7f86d0068b70 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86e415a710 0x7f86e40ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86e40cdfb0 0x7f86e4152e70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 --2- 172.21.15.31:0/3685226816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f86e40cbd50 0x7f86e415ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 -- 172.21.15.31:0/3685226816 >> 172.21.15.31:0/3685226816 conn(0x7f86e40b0cb0 msgr2=0x7f86e40b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 -- 172.21.15.31:0/3685226816 shutdown_connections 2023-11-26T07:21:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.266+0000 7f86e9f57640 1 -- 172.21.15.31:0/3685226816 wait complete. 2023-11-26T07:21:28.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:28.290 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-26T07:21:28.304 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-11-26T07:21:28.305 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:21:28.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:21:28.425 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:21:28.496 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:21:28.497 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:21:28.497 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:28.506 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:21:28.506 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:28.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cb27f5640 1 Processor -- start 2023-11-26T07:21:28.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cb27f5640 1 -- start start 2023-11-26T07:21:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cb27f5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1cac0cb230 0x7f1cac159880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cb27f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1cac0b3320 0x7f1cac06c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cb27f5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1cac0b3740 0x7f1cac070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cb27f5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cac159dc0 con 0x7f1cac0cb230 2023-11-26T07:21:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cb27f5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cac0b41a0 con 0x7f1cac0b3740 2023-11-26T07:21:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cb27f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0b3be0 con 0x7f1cac0b3320 2023-11-26T07:21:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cabfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1cac0cb230 0x7f1cac159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cabfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1cac0cb230 0x7f1cac159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57938/0 (socket says 172.21.15.31:57938) 2023-11-26T07:21:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cabfff640 1 -- 172.21.15.31:0/1885417739 learned_addr learned my addr 172.21.15.31:0/1885417739 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:28.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.742+0000 7f1cab7fe640 1 --2- 172.21.15.31:0/1885417739 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1cac0b3320 0x7f1cac06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cb0d92640 1 --2- 172.21.15.31:0/1885417739 >> v2:172.21.15.97:3300/0 conn(0x7f1cac0b3740 0x7f1cac070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cabfff640 1 -- 172.21.15.31:0/1885417739 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1cac0b3320 msgr2=0x7f1cac06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cabfff640 1 --2- 172.21.15.31:0/1885417739 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1cac0b3320 0x7f1cac06c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cabfff640 1 -- 172.21.15.31:0/1885417739 >> v2:172.21.15.97:3300/0 conn(0x7f1cac0b3740 msgr2=0x7f1cac070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cabfff640 1 --2- 172.21.15.31:0/1885417739 >> v2:172.21.15.97:3300/0 conn(0x7f1cac0b3740 0x7f1cac070ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cabfff640 1 -- 172.21.15.31:0/1885417739 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0b3ec0 con 0x7f1cac0cb230 2023-11-26T07:21:28.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cabfff640 1 --2- 172.21.15.31:0/1885417739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 0x7f1cac159880 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009870 tx=0x7f1c9c01e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db70819880d9f129 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1caaffd640 1 -- 172.21.15.31:0/1885417739 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0a8040 con 0x7f1cac0cb230 2023-11-26T07:21:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1caaffd640 1 -- 172.21.15.31:0/1885417739 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c0ac600 con 0x7f1cac0cb230 2023-11-26T07:21:28.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1caaffd640 1 -- 172.21.15.31:0/1885417739 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0ac990 con 0x7f1cac0cb230 2023-11-26T07:21:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cb27f5640 1 -- 172.21.15.31:0/1885417739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 msgr2=0x7f1cac159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cb27f5640 1 --2- 172.21.15.31:0/1885417739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 0x7f1cac159880 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009870 tx=0x7f1c9c01e5c0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cb27f5640 1 -- 172.21.15.31:0/1885417739 shutdown_connections 2023-11-26T07:21:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.746+0000 7f1cb27f5640 1 --2- 172.21.15.31:0/1885417739 >> v2:172.21.15.97:3300/0 conn(0x7f1cac0b3740 0x7f1cac070ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 --2- 172.21.15.31:0/1885417739 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1cac0b3320 0x7f1cac06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 --2- 172.21.15.31:0/1885417739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 0x7f1cac159880 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 -- 172.21.15.31:0/1885417739 >> 172.21.15.31:0/1885417739 conn(0x7f1cac0b0c40 msgr2=0x7f1cac0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 -- 172.21.15.31:0/1885417739 shutdown_connections 2023-11-26T07:21:28.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 -- 172.21.15.31:0/1885417739 wait complete. 2023-11-26T07:21:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 Processor -- start 2023-11-26T07:21:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 -- start start 2023-11-26T07:21:28.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1cac0b3320 0x7f1cac15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cac0b3740 0x7f1cac14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 0x7f1cac15b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac14bac0 con 0x7f1cac0cb230 2023-11-26T07:21:28.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac155e60 con 0x7f1cac0b3740 2023-11-26T07:21:28.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb27f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac158eb0 con 0x7f1cac0b3320 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb0d92640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 0x7f1cac15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb0d92640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 0x7f1cac15b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57944/0 (socket says 172.21.15.31:57944) 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb0d92640 1 -- 172.21.15.31:0/2848544465 learned_addr learned my addr 172.21.15.31:0/2848544465 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cabfff640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1cac0b3320 0x7f1cac15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cab7fe640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cac0b3740 0x7f1cac14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb0d92640 1 -- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1cac0b3320 msgr2=0x7f1cac15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb0d92640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1cac0b3320 0x7f1cac15cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb0d92640 1 -- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cac0b3740 msgr2=0x7f1cac14e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb0d92640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cac0b3740 0x7f1cac14e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb0d92640 1 -- 172.21.15.31:0/2848544465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c0094d0 con 0x7f1cac0cb230 2023-11-26T07:21:28.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.750+0000 7f1cb0d92640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 0x7f1cac15b260 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4007de0 tx=0x7f1ca400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:28.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.754+0000 7f1ca8ff9640 1 -- 172.21.15.31:0/2848544465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4014620 con 0x7f1cac0cb230 2023-11-26T07:21:28.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.754+0000 7f1ca8ff9640 1 -- 172.21.15.31:0/2848544465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca400ee90 con 0x7f1cac0cb230 2023-11-26T07:21:28.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.754+0000 7f1ca8ff9640 1 -- 172.21.15.31:0/2848544465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40149e0 con 0x7f1cac0cb230 2023-11-26T07:21:28.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.754+0000 7f1cb27f5640 1 -- 172.21.15.31:0/2848544465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac156120 con 0x7f1cac0cb230 2023-11-26T07:21:28.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.754+0000 7f1cb27f5640 1 -- 172.21.15.31:0/2848544465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac14bef0 con 0x7f1cac0cb230 2023-11-26T07:21:28.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.754+0000 7f1cb27f5640 1 -- 172.21.15.31:0/2848544465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac04e850 con 0x7f1cac0cb230 2023-11-26T07:21:28.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.754+0000 7f1ca8ff9640 1 -- 172.21.15.31:0/2848544465 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 86056+0+0 (secure 0 0 0) 0x7f1ca4014b40 con 0x7f1cac0cb230 2023-11-26T07:21:28.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.754+0000 7f1ca8ff9640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1c980666b0 0x7f1c98068b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:28.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.754+0000 7f1ca8ff9640 1 -- 172.21.15.31:0/2848544465 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ca4084ad0 con 0x7f1cac0cb230 2023-11-26T07:21:28.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.754+0000 7f1cabfff640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1c980666b0 0x7f1c98068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:28.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.758+0000 7f1cabfff640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1c980666b0 0x7f1c98068b70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0b0000 tx=0x7f1c9c0a9000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:28.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.758+0000 7f1ca8ff9640 1 -- 172.21.15.31:0/2848544465 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1ca4051d70 con 0x7f1cac0cb230 2023-11-26T07:21:28.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 -- 172.21.15.31:0/2848544465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cac0b24c0 con 0x7f1cac0cb230 2023-11-26T07:21:28.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1ca8ff9640 1 -- 172.21.15.31:0/2848544465 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f1ca4055390 con 0x7f1cac0cb230 2023-11-26T07:21:28.932 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:28.932 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:04.352334+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:01.142504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:01.723895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:28.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 -- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1c980666b0 msgr2=0x7f1c98068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1c980666b0 0x7f1c98068b70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0b0000 tx=0x7f1c9c0a9000 comp rx=0 tx=0).stop 2023-11-26T07:21:28.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 -- 172.21.15.31:0/2848544465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 msgr2=0x7f1cac15b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 0x7f1cac15b260 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4007de0 tx=0x7f1ca400e4b0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 -- 172.21.15.31:0/2848544465 shutdown_connections 2023-11-26T07:21:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1c980666b0 0x7f1c98068b70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cac0cb230 0x7f1cac15b260 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cac0b3740 0x7f1cac14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 --2- 172.21.15.31:0/2848544465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1cac0b3320 0x7f1cac15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 -- 172.21.15.31:0/2848544465 >> 172.21.15.31:0/2848544465 conn(0x7f1cac0b0c40 msgr2=0x7f1cac06e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 -- 172.21.15.31:0/2848544465 shutdown_connections 2023-11-26T07:21:28.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:28.930+0000 7f1cb27f5640 1 -- 172.21.15.31:0/2848544465 wait complete. 2023-11-26T07:21:28.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:21:01.142504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:21:01.723895+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:21:28.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:21:29.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.098+0000 7fe06f816640 1 Processor -- start 2023-11-26T07:21:29.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06f816640 1 -- start start 2023-11-26T07:21:29.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06f816640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe0680cdfb0 0x7fe0680abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06f816640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe06815a710 0x7fe0680ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06f816640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0680ce8d0 con 0x7fe0680cbd50 2023-11-26T07:21:29.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06f816640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0680cca80 con 0x7fe0680cdfb0 2023-11-26T07:21:29.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06f816640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0680b2710 con 0x7fe06815a710 2023-11-26T07:21:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06d5b2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe0680cdfb0 0x7fe0680abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06ddb3640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe0680cbd50 0x7fe0680cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41114/0 (socket says 172.21.15.31:41114) 2023-11-26T07:21:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06ddb3640 1 -- 172.21.15.31:0/3973266827 learned_addr learned my addr 172.21.15.31:0/3973266827 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06cdb1640 1 --2- 172.21.15.31:0/3973266827 >> v2:172.21.15.97:3300/0 conn(0x7fe06815a710 0x7fe0680ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06d5b2640 1 -- 172.21.15.31:0/3973266827 >> v2:172.21.15.97:3300/0 conn(0x7fe06815a710 msgr2=0x7fe0680ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:29.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06d5b2640 1 --2- 172.21.15.31:0/3973266827 >> v2:172.21.15.97:3300/0 conn(0x7fe06815a710 0x7fe0680ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06d5b2640 1 -- 172.21.15.31:0/3973266827 >> v1:172.21.15.31:6789/0 conn(0x7fe0680cbd50 legacy=0x7fe0680cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe06d5b2640 1 -- 172.21.15.31:0/3973266827 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0680b29f0 con 0x7fe0680cdfb0 2023-11-26T07:21:29.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.102+0000 7fe05ffff640 1 -- 172.21.15.31:0/3973266827 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263681986 0 0) 0x7fe0680ce8d0 con 0x7fe0680cbd50 2023-11-26T07:21:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe06d5b2640 1 --2- 172.21.15.31:0/3973266827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 0x7fe0680abf90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe0680d0040 tx=0x7fe06401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6329dbd317028c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe05ffff640 1 -- 172.21.15.31:0/3973266827 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0640a7070 con 0x7fe0680cdfb0 2023-11-26T07:21:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe05ffff640 1 -- 172.21.15.31:0/3973266827 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0640ab430 con 0x7fe0680cdfb0 2023-11-26T07:21:29.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe05ffff640 1 -- 172.21.15.31:0/3973266827 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0640ab7c0 con 0x7fe0680cdfb0 2023-11-26T07:21:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe06f816640 1 -- 172.21.15.31:0/3973266827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 msgr2=0x7fe0680abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe06f816640 1 --2- 172.21.15.31:0/3973266827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 0x7fe0680abf90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe0680d0040 tx=0x7fe06401c430 comp rx=0 tx=0).stop 2023-11-26T07:21:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe06f816640 1 -- 172.21.15.31:0/3973266827 shutdown_connections 2023-11-26T07:21:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe06f816640 1 --2- 172.21.15.31:0/3973266827 >> v2:172.21.15.97:3300/0 conn(0x7fe06815a710 0x7fe0680ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe06f816640 1 --2- 172.21.15.31:0/3973266827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 0x7fe0680abf90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe06f816640 1 -- 172.21.15.31:0/3973266827 >> 172.21.15.31:0/3973266827 conn(0x7fe0680b0cb0 msgr2=0x7fe0680b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe06f816640 1 -- 172.21.15.31:0/3973266827 shutdown_connections 2023-11-26T07:21:29.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe06f816640 1 -- 172.21.15.31:0/3973266827 wait complete. 2023-11-26T07:21:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.106+0000 7fe06f816640 1 Processor -- start 2023-11-26T07:21:29.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06f816640 1 -- start start 2023-11-26T07:21:29.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06f816640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe0680cbd50 0x7fe068072020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06f816640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 0x7fe0681f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06f816640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe06815a710 0x7fe0680ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06f816640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06815b610 con 0x7fe0680cdfb0 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06f816640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe068158d20 con 0x7fe06815a710 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06f816640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe068159000 con 0x7fe0680cbd50 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06cdb1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 0x7fe0681f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06cdb1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 0x7fe0681f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57966/0 (socket says 172.21.15.31:57966) 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06cdb1640 1 -- 172.21.15.31:0/3587401085 learned_addr learned my addr 172.21.15.31:0/3587401085 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06d5b2640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe0680cbd50 0x7fe068072020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06ddb3640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe06815a710 0x7fe0680ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06cdb1640 1 -- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe0680cbd50 msgr2=0x7fe068072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06cdb1640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe0680cbd50 0x7fe068072020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06cdb1640 1 -- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe06815a710 msgr2=0x7fe0680ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06cdb1640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe06815a710 0x7fe0680ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06cdb1640 1 -- 172.21.15.31:0/3587401085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe064009420 con 0x7fe0680cdfb0 2023-11-26T07:21:29.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06cdb1640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 0x7fe0681f5620 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe05800e840 tx=0x7fe05800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:29.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe05dffb640 1 -- 172.21.15.31:0/3587401085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058009510 con 0x7fe0680cdfb0 2023-11-26T07:21:29.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe05dffb640 1 -- 172.21.15.31:0/3587401085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05800ee90 con 0x7fe0680cdfb0 2023-11-26T07:21:29.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe05dffb640 1 -- 172.21.15.31:0/3587401085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05800cb80 con 0x7fe0680cdfb0 2023-11-26T07:21:29.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06f816640 1 -- 172.21.15.31:0/3587401085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0681592e0 con 0x7fe0680cdfb0 2023-11-26T07:21:29.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06f816640 1 -- 172.21.15.31:0/3587401085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0681f5b60 con 0x7fe0680cdfb0 2023-11-26T07:21:29.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.110+0000 7fe06f816640 1 -- 172.21.15.31:0/3587401085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0680671f0 con 0x7fe0680cdfb0 2023-11-26T07:21:29.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.114+0000 7fe05dffb640 1 -- 172.21.15.31:0/3587401085 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fe05800cce0 con 0x7fe0680cdfb0 2023-11-26T07:21:29.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.114+0000 7fe05dffb640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe04c066800 0x7fe04c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.114+0000 7fe05dffb640 1 -- 172.21.15.31:0/3587401085 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe058084390 con 0x7fe0680cdfb0 2023-11-26T07:21:29.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.114+0000 7fe06d5b2640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe04c066800 0x7fe04c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.114+0000 7fe06d5b2640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe04c066800 0x7fe04c068cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe0640af000 tx=0x7fe0640a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:29.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.118+0000 7fe05dffb640 1 -- 172.21.15.31:0/3587401085 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe058051470 con 0x7fe0680cdfb0 2023-11-26T07:21:29.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.274+0000 7fe06f816640 1 -- 172.21.15.31:0/3587401085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe06815b450 con 0x7fe0680cdfb0 2023-11-26T07:21:29.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.274+0000 7fe05dffb640 1 -- 172.21.15.31:0/3587401085 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe06815b450 con 0x7fe0680cdfb0 2023-11-26T07:21:29.278 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:21:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 -- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe04c066800 msgr2=0x7fe04c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe04c066800 0x7fe04c068cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe0640af000 tx=0x7fe0640a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 -- 172.21.15.31:0/3587401085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 msgr2=0x7fe0681f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 0x7fe0681f5620 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe05800e840 tx=0x7fe05800ec40 comp rx=0 tx=0).stop 2023-11-26T07:21:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 -- 172.21.15.31:0/3587401085 shutdown_connections 2023-11-26T07:21:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe04c066800 0x7fe04c068cc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe06815a710 0x7fe0680ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0680cdfb0 0x7fe0681f5620 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 --2- 172.21.15.31:0/3587401085 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe0680cbd50 0x7fe068072020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 -- 172.21.15.31:0/3587401085 >> 172.21.15.31:0/3587401085 conn(0x7fe0680b0cb0 msgr2=0x7fe06815e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 -- 172.21.15.31:0/3587401085 shutdown_connections 2023-11-26T07:21:29.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.278+0000 7fe06f816640 1 -- 172.21.15.31:0/3587401085 wait complete. 2023-11-26T07:21:29.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:21:29.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdc05aa640 1 Processor -- start 2023-11-26T07:21:29.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdc05aa640 1 -- start start 2023-11-26T07:21:29.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdc05aa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbdb815d610 0x7fbdb80ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdc05aa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb80caef0 con 0x7fbdb80caa60 2023-11-26T07:21:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdc05aa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb80cbfe0 con 0x7fbdb806c520 2023-11-26T07:21:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdc05aa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb806b970 con 0x7fbdb815d610 2023-11-26T07:21:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdbdb45640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbdb80caa60 0x7fbdb80cb8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41124/0 (socket says 172.21.15.31:41124) 2023-11-26T07:21:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdbdb45640 1 -- 172.21.15.31:0/2891039590 learned_addr learned my addr 172.21.15.31:0/2891039590 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:29.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdbe346640 1 --2- 172.21.15.31:0/2891039590 >> v2:172.21.15.97:3300/0 conn(0x7fbdb815d610 0x7fbdb80ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdbd344640 1 -- 172.21.15.31:0/2891039590 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123355885 0 0) 0x7fbdb80caef0 con 0x7fbdb80caa60 2023-11-26T07:21:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdbd344640 1 -- 172.21.15.31:0/2891039590 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbda4003530 con 0x7fbdb80caa60 2023-11-26T07:21:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdbd344640 1 -- 172.21.15.31:0/2891039590 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254002222 0 0) 0x7fbdb80cbfe0 con 0x7fbdb806c520 2023-11-26T07:21:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdbe346640 1 -- 172.21.15.31:0/2891039590 >> v1:172.21.15.97:6789/0 conn(0x7fbdb806c520 legacy=0x7fbdb806c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdbe346640 1 -- 172.21.15.31:0/2891039590 >> v1:172.21.15.31:6789/0 conn(0x7fbdb80caa60 legacy=0x7fbdb80cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.442+0000 7fbdbe346640 1 -- 172.21.15.31:0/2891039590 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb806bc50 con 0x7fbdb815d610 2023-11-26T07:21:29.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdbe346640 1 --2- 172.21.15.31:0/2891039590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdb815d610 0x7fbdb80ce4b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40097c0 tx=0x7fbdb401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=812bc1bebaec9a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdbd344640 1 -- 172.21.15.31:0/2891039590 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40ab650 con 0x7fbdb815d610 2023-11-26T07:21:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdbd344640 1 -- 172.21.15.31:0/2891039590 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb40ab7b0 con 0x7fbdb815d610 2023-11-26T07:21:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdbd344640 1 -- 172.21.15.31:0/2891039590 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40abb20 con 0x7fbdb815d610 2023-11-26T07:21:29.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2891039590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdb815d610 msgr2=0x7fbdb80ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdc05aa640 1 --2- 172.21.15.31:0/2891039590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdb815d610 0x7fbdb80ce4b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40097c0 tx=0x7fbdb401c450 comp rx=0 tx=0).stop 2023-11-26T07:21:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2891039590 shutdown_connections 2023-11-26T07:21:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdc05aa640 1 --2- 172.21.15.31:0/2891039590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdb815d610 0x7fbdb80ce4b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2891039590 >> 172.21.15.31:0/2891039590 conn(0x7fbdb80b2f10 msgr2=0x7fbdb80b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2891039590 shutdown_connections 2023-11-26T07:21:29.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.446+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2891039590 wait complete. 2023-11-26T07:21:29.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdc05aa640 1 Processor -- start 2023-11-26T07:21:29.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdc05aa640 1 -- start start 2023-11-26T07:21:29.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdc05aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdb806c520 0x7fbdb80b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdc05aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdb80caa60 0x7fbdb81469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdc05aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbdb815d610 0x7fbdb814c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdc05aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b4dc0 con 0x7fbdb806c520 2023-11-26T07:21:29.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdc05aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b5530 con 0x7fbdb80caa60 2023-11-26T07:21:29.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdc05aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb80b5810 con 0x7fbdb815d610 2023-11-26T07:21:29.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbe346640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdb806c520 0x7fbdb80b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbe346640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdb806c520 0x7fbdb80b4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57982/0 (socket says 172.21.15.31:57982) 2023-11-26T07:21:29.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbe346640 1 -- 172.21.15.31:0/2180143985 learned_addr learned my addr 172.21.15.31:0/2180143985 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:29.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbeb47640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbdb815d610 0x7fbdb814c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbdb45640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdb80caa60 0x7fbdb81469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbe346640 1 -- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbdb815d610 msgr2=0x7fbdb814c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbe346640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbdb815d610 0x7fbdb814c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbe346640 1 -- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdb80caa60 msgr2=0x7fbdb81469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbe346640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdb80caa60 0x7fbdb81469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbe346640 1 -- 172.21.15.31:0/2180143985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4009420 con 0x7fbdb806c520 2023-11-26T07:21:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbdb45640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdb80caa60 0x7fbdb81469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdbe346640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdb806c520 0x7fbdb80b4780 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40ad040 tx=0x7fbdb401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdaeffd640 1 -- 172.21.15.31:0/2180143985 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40bf610 con 0x7fbdb806c520 2023-11-26T07:21:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdaeffd640 1 -- 172.21.15.31:0/2180143985 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb40bf770 con 0x7fbdb806c520 2023-11-26T07:21:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdaeffd640 1 -- 172.21.15.31:0/2180143985 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb40bfae0 con 0x7fbdb806c520 2023-11-26T07:21:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.450+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2180143985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb80b5af0 con 0x7fbdb806c520 2023-11-26T07:21:29.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.454+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2180143985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb814ce70 con 0x7fbdb806c520 2023-11-26T07:21:29.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.454+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2180143985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb804e850 con 0x7fbdb806c520 2023-11-26T07:21:29.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.454+0000 7fbdaeffd640 1 -- 172.21.15.31:0/2180143985 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fbdb40bfc40 con 0x7fbdb806c520 2023-11-26T07:21:29.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.454+0000 7fbdaeffd640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbd9c0667d0 0x7fbd9c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.454+0000 7fbdaeffd640 1 -- 172.21.15.31:0/2180143985 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdb411eee0 con 0x7fbdb806c520 2023-11-26T07:21:29.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.454+0000 7fbdbdb45640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbd9c0667d0 0x7fbd9c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.454+0000 7fbdbdb45640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbd9c0667d0 0x7fbd9c068c90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8160a40 tx=0x7fbda8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:29.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.458+0000 7fbdaeffd640 1 -- 172.21.15.31:0/2180143985 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbdb40ebf10 con 0x7fbdb806c520 2023-11-26T07:21:29.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.614+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2180143985 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbdb80cdf10 con 0x7fbdb806c520 2023-11-26T07:21:29.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.614+0000 7fbdaeffd640 1 -- 172.21.15.31:0/2180143985 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbdb40ef530 con 0x7fbdb806c520 2023-11-26T07:21:29.618 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:21:29.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbd9c0667d0 msgr2=0x7fbd9c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbd9c0667d0 0x7fbd9c068c90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8160a40 tx=0x7fbda8009040 comp rx=0 tx=0).stop 2023-11-26T07:21:29.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2180143985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdb806c520 msgr2=0x7fbdb80b4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdb806c520 0x7fbdb80b4780 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbdb40ad040 tx=0x7fbdb401ac90 comp rx=0 tx=0).stop 2023-11-26T07:21:29.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2180143985 shutdown_connections 2023-11-26T07:21:29.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbd9c0667d0 0x7fbd9c068c90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbdb815d610 0x7fbdb814c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdb80caa60 0x7fbdb81469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 --2- 172.21.15.31:0/2180143985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdb806c520 0x7fbdb80b4780 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2180143985 >> 172.21.15.31:0/2180143985 conn(0x7fbdb80b2f10 msgr2=0x7fbdb80cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:29.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2180143985 shutdown_connections 2023-11-26T07:21:29.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.618+0000 7fbdc05aa640 1 -- 172.21.15.31:0/2180143985 wait complete. 2023-11-26T07:21:29.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:21:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff45038d640 1 Processor -- start 2023-11-26T07:21:29.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff45038d640 1 -- start start 2023-11-26T07:21:29.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff45038d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff44806e3f0 0x7ff44815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff45038d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480ce5d0 0x7ff44815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff45038d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff448159630 con 0x7ff4480cb9b0 2023-11-26T07:21:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff45038d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44806e860 con 0x7ff44806e3f0 2023-11-26T07:21:29.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff45038d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44815bba0 con 0x7ff4480ce5d0 2023-11-26T07:21:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff44e92a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480ce5d0 0x7ff44815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff44d928640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff44806e3f0 0x7ff44815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff44e92a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480ce5d0 0x7ff44815bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:46010/0 (socket says 172.21.15.31:46010) 2023-11-26T07:21:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.782+0000 7ff44e92a640 1 -- 172.21.15.31:0/1740006000 learned_addr learned my addr 172.21.15.31:0/1740006000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.786+0000 7ff44d928640 1 -- 172.21.15.31:0/1740006000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480ce5d0 msgr2=0x7ff44815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.786+0000 7ff44d928640 1 --2- 172.21.15.31:0/1740006000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480ce5d0 0x7ff44815bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.786+0000 7ff44d928640 1 -- 172.21.15.31:0/1740006000 >> v1:172.21.15.97:6789/0 conn(0x7ff4480cb9b0 legacy=0x7ff4480cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.786+0000 7ff44d928640 1 -- 172.21.15.31:0/1740006000 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44815f6d0 con 0x7ff44806e3f0 2023-11-26T07:21:29.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.786+0000 7ff44d928640 1 --2- 172.21.15.31:0/1740006000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff44806e3f0 0x7ff44815f190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff4380097c0 tx=0x7ff43801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a6ca43d9b959b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.786+0000 7ff44d127640 1 -- 172.21.15.31:0/1740006000 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438004230 con 0x7ff44806e3f0 2023-11-26T07:21:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.786+0000 7ff44d127640 1 -- 172.21.15.31:0/1740006000 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4380a7050 con 0x7ff44806e3f0 2023-11-26T07:21:29.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.786+0000 7ff44d127640 1 -- 172.21.15.31:0/1740006000 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438003620 con 0x7ff44806e3f0 2023-11-26T07:21:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.786+0000 7ff45038d640 1 -- 172.21.15.31:0/1740006000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff44806e3f0 msgr2=0x7ff44815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.786+0000 7ff45038d640 1 --2- 172.21.15.31:0/1740006000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff44806e3f0 0x7ff44815f190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff4380097c0 tx=0x7ff43801c480 comp rx=0 tx=0).stop 2023-11-26T07:21:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 -- 172.21.15.31:0/1740006000 shutdown_connections 2023-11-26T07:21:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 --2- 172.21.15.31:0/1740006000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480ce5d0 0x7ff44815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 --2- 172.21.15.31:0/1740006000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff44806e3f0 0x7ff44815f190 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 -- 172.21.15.31:0/1740006000 >> 172.21.15.31:0/1740006000 conn(0x7ff4480b0cb0 msgr2=0x7ff4480b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 -- 172.21.15.31:0/1740006000 shutdown_connections 2023-11-26T07:21:29.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 -- 172.21.15.31:0/1740006000 wait complete. 2023-11-26T07:21:29.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 Processor -- start 2023-11-26T07:21:29.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 -- start start 2023-11-26T07:21:29.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff44806e3f0 0x7ff4480b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480cb9b0 0x7ff4480b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4480ce5d0 0x7ff448157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480b2420 con 0x7ff4480ce5d0 2023-11-26T07:21:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff448158e90 con 0x7ff44806e3f0 2023-11-26T07:21:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff45038d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480b2d80 con 0x7ff4480cb9b0 2023-11-26T07:21:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44e92a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4480ce5d0 0x7ff448157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44e92a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4480ce5d0 0x7ff448157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57988/0 (socket says 172.21.15.31:57988) 2023-11-26T07:21:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44d928640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480cb9b0 0x7ff4480b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44e92a640 1 -- 172.21.15.31:0/2065994667 learned_addr learned my addr 172.21.15.31:0/2065994667 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:29.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44e129640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff44806e3f0 0x7ff4480b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44e92a640 1 -- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480cb9b0 msgr2=0x7ff4480b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44e92a640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480cb9b0 0x7ff4480b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44e92a640 1 -- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff44806e3f0 msgr2=0x7ff4480b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44e92a640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff44806e3f0 0x7ff4480b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44e92a640 1 -- 172.21.15.31:0/2065994667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff438009420 con 0x7ff4480ce5d0 2023-11-26T07:21:29.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44d928640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480cb9b0 0x7ff4480b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.790+0000 7ff44e92a640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4480ce5d0 0x7ff448157ea0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff44000efc0 tx=0x7ff44000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.794+0000 7ff43effd640 1 -- 172.21.15.31:0/2065994667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440009b60 con 0x7ff4480ce5d0 2023-11-26T07:21:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.794+0000 7ff43effd640 1 -- 172.21.15.31:0/2065994667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44000f040 con 0x7ff4480ce5d0 2023-11-26T07:21:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.794+0000 7ff43effd640 1 -- 172.21.15.31:0/2065994667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4400039a0 con 0x7ff4480ce5d0 2023-11-26T07:21:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.794+0000 7ff45038d640 1 -- 172.21.15.31:0/2065994667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4481583e0 con 0x7ff4480ce5d0 2023-11-26T07:21:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.794+0000 7ff45038d640 1 -- 172.21.15.31:0/2065994667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff448099d70 con 0x7ff4480ce5d0 2023-11-26T07:21:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.794+0000 7ff45038d640 1 -- 172.21.15.31:0/2065994667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4480671f0 con 0x7ff4480ce5d0 2023-11-26T07:21:29.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.794+0000 7ff43effd640 1 -- 172.21.15.31:0/2065994667 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7ff440003b00 con 0x7ff4480ce5d0 2023-11-26T07:21:29.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.794+0000 7ff43effd640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff42c0667d0 0x7ff42c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:29.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.794+0000 7ff43effd640 1 -- 172.21.15.31:0/2065994667 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4400849e0 con 0x7ff4480ce5d0 2023-11-26T07:21:29.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.798+0000 7ff44e129640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff42c0667d0 0x7ff42c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:29.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.798+0000 7ff44e129640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff42c0667d0 0x7ff42c068c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff444007fd0 tx=0x7ff444009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:29.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.798+0000 7ff43effd640 1 -- 172.21.15.31:0/2065994667 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff440051940 con 0x7ff4480ce5d0 2023-11-26T07:21:29.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.954+0000 7ff45038d640 1 -- 172.21.15.31:0/2065994667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4480b2560 con 0x7ff4480ce5d0 2023-11-26T07:21:29.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.954+0000 7ff43effd640 1 -- 172.21.15.31:0/2065994667 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4480b2560 con 0x7ff4480ce5d0 2023-11-26T07:21:29.958 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:21:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 -- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff42c0667d0 msgr2=0x7ff42c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff42c0667d0 0x7ff42c068c90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff444007fd0 tx=0x7ff444009040 comp rx=0 tx=0).stop 2023-11-26T07:21:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 -- 172.21.15.31:0/2065994667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4480ce5d0 msgr2=0x7ff448157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4480ce5d0 0x7ff448157ea0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff44000efc0 tx=0x7ff44000c490 comp rx=0 tx=0).stop 2023-11-26T07:21:29.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 -- 172.21.15.31:0/2065994667 shutdown_connections 2023-11-26T07:21:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff42c0667d0 0x7ff42c068c90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4480ce5d0 0x7ff448157ea0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4480cb9b0 0x7ff4480b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 --2- 172.21.15.31:0/2065994667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff44806e3f0 0x7ff4480b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 -- 172.21.15.31:0/2065994667 >> 172.21.15.31:0/2065994667 conn(0x7ff4480b0cb0 msgr2=0x7ff448165940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 -- 172.21.15.31:0/2065994667 shutdown_connections 2023-11-26T07:21:29.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:29.958+0000 7ff45038d640 1 -- 172.21.15.31:0/2065994667 wait complete. 2023-11-26T07:21:29.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:21:30.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b9e3b8640 1 Processor -- start 2023-11-26T07:21:30.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b9e3b8640 1 -- start start 2023-11-26T07:21:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b9e3b8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b98146820 0x7f6b98070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b9e3b8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b980b4210 0x7f6b980b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b9e3b8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b98161180 0x7f6b981615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b9e3b8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b980b4be0 con 0x7f6b98146820 2023-11-26T07:21:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b9e3b8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b98147190 con 0x7f6b980b4210 2023-11-26T07:21:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b9e3b8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9806d030 con 0x7f6b98161180 2023-11-26T07:21:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b97fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b98146820 0x7f6b98070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b97fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6b98146820 0x7f6b98070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57996/0 (socket says 172.21.15.31:57996) 2023-11-26T07:21:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b97fff640 1 -- 172.21.15.31:0/2508514583 learned_addr learned my addr 172.21.15.31:0/2508514583 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:30.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b9c955640 1 --2- 172.21.15.31:0/2508514583 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b98161180 0x7f6b981615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b977fe640 1 --2- 172.21.15.31:0/2508514583 >> v2:172.21.15.97:3300/0 conn(0x7f6b980b4210 0x7f6b980b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b97fff640 1 -- 172.21.15.31:0/2508514583 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b98161180 msgr2=0x7f6b981615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b97fff640 1 --2- 172.21.15.31:0/2508514583 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b98161180 0x7f6b981615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b97fff640 1 -- 172.21.15.31:0/2508514583 >> v2:172.21.15.97:3300/0 conn(0x7f6b980b4210 msgr2=0x7f6b980b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b97fff640 1 --2- 172.21.15.31:0/2508514583 >> v2:172.21.15.97:3300/0 conn(0x7f6b980b4210 0x7f6b980b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b97fff640 1 -- 172.21.15.31:0/2508514583 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b98146c00 con 0x7f6b98146820 2023-11-26T07:21:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b977fe640 1 --2- 172.21.15.31:0/2508514583 >> v2:172.21.15.97:3300/0 conn(0x7f6b980b4210 0x7f6b980b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:30.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.122+0000 7f6b9c955640 1 --2- 172.21.15.31:0/2508514583 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b98161180 0x7f6b981615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:30.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b97fff640 1 --2- 172.21.15.31:0/2508514583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 0x7f6b98070a50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b5ad0 tx=0x7f6b8801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae44d7c79a87c18 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:30.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b96ffd640 1 -- 172.21.15.31:0/2508514583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b880aa5f0 con 0x7f6b98146820 2023-11-26T07:21:30.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b96ffd640 1 -- 172.21.15.31:0/2508514583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b880aa750 con 0x7f6b98146820 2023-11-26T07:21:30.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b96ffd640 1 -- 172.21.15.31:0/2508514583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b880aaae0 con 0x7f6b98146820 2023-11-26T07:21:30.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/2508514583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 msgr2=0x7f6b98070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:30.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b9e3b8640 1 --2- 172.21.15.31:0/2508514583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 0x7f6b98070a50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6b980b5ad0 tx=0x7f6b8801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/2508514583 shutdown_connections 2023-11-26T07:21:30.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b9e3b8640 1 --2- 172.21.15.31:0/2508514583 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b98161180 0x7f6b981615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b9e3b8640 1 --2- 172.21.15.31:0/2508514583 >> v2:172.21.15.97:3300/0 conn(0x7f6b980b4210 0x7f6b980b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b9e3b8640 1 --2- 172.21.15.31:0/2508514583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 0x7f6b98070a50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/2508514583 >> 172.21.15.31:0/2508514583 conn(0x7f6b980b2f10 msgr2=0x7f6b980b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:30.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/2508514583 shutdown_connections 2023-11-26T07:21:30.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.126+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/2508514583 wait complete. 2023-11-26T07:21:30.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9e3b8640 1 Processor -- start 2023-11-26T07:21:30.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9e3b8640 1 -- start start 2023-11-26T07:21:30.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9e3b8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b980b4210 0x7f6b9813fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9e3b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 0x7f6b98144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9e3b8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b98161180 0x7f6b98156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9e3b8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b98142ab0 con 0x7f6b98146820 2023-11-26T07:21:30.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9e3b8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b981424a0 con 0x7f6b98161180 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9e3b8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b981452d0 con 0x7f6b980b4210 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b977fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 0x7f6b98144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b977fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 0x7f6b98144d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58006/0 (socket says 172.21.15.31:58006) 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b977fe640 1 -- 172.21.15.31:0/397332779 learned_addr learned my addr 172.21.15.31:0/397332779 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b97fff640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b980b4210 0x7f6b9813fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9c955640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b98161180 0x7f6b98156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b977fe640 1 -- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b980b4210 msgr2=0x7f6b9813fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b977fe640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b980b4210 0x7f6b9813fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b977fe640 1 -- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b98161180 msgr2=0x7f6b98156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b977fe640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b98161180 0x7f6b98156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b977fe640 1 -- 172.21.15.31:0/397332779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b880094d0 con 0x7f6b98146820 2023-11-26T07:21:30.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b97fff640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b980b4210 0x7f6b9813fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:30.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9c955640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b98161180 0x7f6b98156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:30.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b977fe640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 0x7f6b98144d90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00e810 tx=0x7f6b8c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b94ff9640 1 -- 172.21.15.31:0/397332779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c004030 con 0x7f6b98146820 2023-11-26T07:21:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b94ff9640 1 -- 172.21.15.31:0/397332779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c004190 con 0x7f6b98146820 2023-11-26T07:21:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/397332779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b98145590 con 0x7f6b98146820 2023-11-26T07:21:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b94ff9640 1 -- 172.21.15.31:0/397332779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c004520 con 0x7f6b98146820 2023-11-26T07:21:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.130+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/397332779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b98145870 con 0x7f6b98146820 2023-11-26T07:21:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.134+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/397332779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b98142c60 con 0x7f6b98146820 2023-11-26T07:21:30.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.134+0000 7f6b94ff9640 1 -- 172.21.15.31:0/397332779 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f6b8c00c6d0 con 0x7f6b98146820 2023-11-26T07:21:30.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.134+0000 7f6b94ff9640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b840667d0 0x7f6b84068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.134+0000 7f6b94ff9640 1 -- 172.21.15.31:0/397332779 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b8c084090 con 0x7f6b98146820 2023-11-26T07:21:30.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.134+0000 7f6b97fff640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b840667d0 0x7f6b84068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.134+0000 7f6b97fff640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b840667d0 0x7f6b84068c90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6b880ae000 tx=0x7f6b880a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:30.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.138+0000 7f6b94ff9640 1 -- 172.21.15.31:0/397332779 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b98142c60 con 0x7f6b98146820 2023-11-26T07:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/397332779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b9804e850 con 0x7f6b98146820 2023-11-26T07:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b94ff9640 1 -- 172.21.15.31:0/397332779 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6b8c050fb0 con 0x7f6b98146820 2023-11-26T07:21:30.311 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:21:30.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b840667d0 msgr2=0x7f6b84068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:30.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b840667d0 0x7f6b84068c90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6b880ae000 tx=0x7f6b880a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:30.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/397332779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 msgr2=0x7f6b98144d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:30.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 0x7f6b98144d90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c00e810 tx=0x7f6b8c00ecd0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/397332779 shutdown_connections 2023-11-26T07:21:30.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b840667d0 0x7f6b84068c90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b98161180 0x7f6b98156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b98146820 0x7f6b98144d90 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 --2- 172.21.15.31:0/397332779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b980b4210 0x7f6b9813fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/397332779 >> 172.21.15.31:0/397332779 conn(0x7f6b980b2f10 msgr2=0x7f6b9815f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:30.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/397332779 shutdown_connections 2023-11-26T07:21:30.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.310+0000 7f6b9e3b8640 1 -- 172.21.15.31:0/397332779 wait complete. 2023-11-26T07:21:30.332 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:21:30.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:30.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:21:30.384 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:21:30.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:30.384 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:21:30.409 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:21:30.409 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:30.420 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:21:30.420 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:30.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:30.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:21:30.547 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:21:30.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.610+0000 7fd1c759e640 1 Processor -- start 2023-11-26T07:21:30.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.610+0000 7fd1c759e640 1 -- start start 2023-11-26T07:21:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c759e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1c815b130 0x7fd1c8070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c759e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1c80cc680 0x7fd1c80ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c759e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd1c80b4410 0x7fd1c80b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c759e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c80b4dd0 con 0x7fd1c815b130 2023-11-26T07:21:30.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c759e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c80709d0 con 0x7fd1c80b4410 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c759e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c80706a0 con 0x7fd1c80cc680 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c659c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1c815b130 0x7fd1c8070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c659c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd1c815b130 0x7fd1c8070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58008/0 (socket says 172.21.15.31:58008) 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c659c640 1 -- 172.21.15.31:0/45288387 learned_addr learned my addr 172.21.15.31:0/45288387 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1bffff640 1 --2- 172.21.15.31:0/45288387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1c80cc680 0x7fd1c80ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c6d9d640 1 --2- 172.21.15.31:0/45288387 >> v2:172.21.15.97:3300/0 conn(0x7fd1c80b4410 0x7fd1c80b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c659c640 1 -- 172.21.15.31:0/45288387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1c80cc680 msgr2=0x7fd1c80ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c659c640 1 --2- 172.21.15.31:0/45288387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1c80cc680 0x7fd1c80ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c659c640 1 -- 172.21.15.31:0/45288387 >> v2:172.21.15.97:3300/0 conn(0x7fd1c80b4410 msgr2=0x7fd1c80b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c659c640 1 --2- 172.21.15.31:0/45288387 >> v2:172.21.15.97:3300/0 conn(0x7fd1c80b4410 0x7fd1c80b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c659c640 1 -- 172.21.15.31:0/45288387 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c80cd690 con 0x7fd1c815b130 2023-11-26T07:21:30.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.614+0000 7fd1c6d9d640 1 --2- 172.21.15.31:0/45288387 >> v2:172.21.15.97:3300/0 conn(0x7fd1c80b4410 0x7fd1c80b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:30.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c659c640 1 --2- 172.21.15.31:0/45288387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 0x7fd1c8070e60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80cc2e0 tx=0x7fd1b801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b4e432a1133ad2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:30.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c5d9b640 1 -- 172.21.15.31:0/45288387 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b80aa5f0 con 0x7fd1c815b130 2023-11-26T07:21:30.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c5d9b640 1 -- 172.21.15.31:0/45288387 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b80aa750 con 0x7fd1c815b130 2023-11-26T07:21:30.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c5d9b640 1 -- 172.21.15.31:0/45288387 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b80aaae0 con 0x7fd1c815b130 2023-11-26T07:21:30.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 -- 172.21.15.31:0/45288387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 msgr2=0x7fd1c8070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:30.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 --2- 172.21.15.31:0/45288387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 0x7fd1c8070e60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80cc2e0 tx=0x7fd1b801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 -- 172.21.15.31:0/45288387 shutdown_connections 2023-11-26T07:21:30.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 --2- 172.21.15.31:0/45288387 >> v2:172.21.15.97:3300/0 conn(0x7fd1c80b4410 0x7fd1c80b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 --2- 172.21.15.31:0/45288387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1c80cc680 0x7fd1c80ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 --2- 172.21.15.31:0/45288387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 0x7fd1c8070e60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 -- 172.21.15.31:0/45288387 >> 172.21.15.31:0/45288387 conn(0x7fd1c80b0e00 msgr2=0x7fd1c80b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:30.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 -- 172.21.15.31:0/45288387 shutdown_connections 2023-11-26T07:21:30.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 -- 172.21.15.31:0/45288387 wait complete. 2023-11-26T07:21:30.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 Processor -- start 2023-11-26T07:21:30.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.618+0000 7fd1c759e640 1 -- start start 2023-11-26T07:21:30.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c759e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1c80b4410 0x7fd1c8155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c759e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c80cc680 0x7fd1c81489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 0x7fd1c80cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c759e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c80cad50 con 0x7fd1c815b130 2023-11-26T07:21:30.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c759e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c8148ee0 con 0x7fd1c80cc680 2023-11-26T07:21:30.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c759e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c80ca0d0 con 0x7fd1c80b4410 2023-11-26T07:21:30.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 0x7fd1c80cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c6d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 0x7fd1c80cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58018/0 (socket says 172.21.15.31:58018) 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c6d9d640 1 -- 172.21.15.31:0/919035312 learned_addr learned my addr 172.21.15.31:0/919035312 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c659c640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1c80b4410 0x7fd1c8155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1bffff640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c80cc680 0x7fd1c81489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c6d9d640 1 -- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1c80b4410 msgr2=0x7fd1c8155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c6d9d640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1c80b4410 0x7fd1c8155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c6d9d640 1 -- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c80cc680 msgr2=0x7fd1c81489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c6d9d640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c80cc680 0x7fd1c81489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c6d9d640 1 -- 172.21.15.31:0/919035312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b80094d0 con 0x7fd1c815b130 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1bffff640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c80cc680 0x7fd1c81489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c6d9d640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 0x7fd1c80cb170 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009fa0 tx=0x7fd1c000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1beffd640 1 -- 172.21.15.31:0/919035312 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0003ba0 con 0x7fd1c815b130 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1beffd640 1 -- 172.21.15.31:0/919035312 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c000f040 con 0x7fd1c815b130 2023-11-26T07:21:30.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1beffd640 1 -- 172.21.15.31:0/919035312 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0004030 con 0x7fd1c815b130 2023-11-26T07:21:30.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c759e640 1 -- 172.21.15.31:0/919035312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c80ca3b0 con 0x7fd1c815b130 2023-11-26T07:21:30.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c759e640 1 -- 172.21.15.31:0/919035312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c80ca970 con 0x7fd1c815b130 2023-11-26T07:21:30.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1c759e640 1 -- 172.21.15.31:0/919035312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c80cae90 con 0x7fd1c815b130 2023-11-26T07:21:30.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.622+0000 7fd1beffd640 1 -- 172.21.15.31:0/919035312 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fd1c0004190 con 0x7fd1c815b130 2023-11-26T07:21:30.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.626+0000 7fd1beffd640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1a80667d0 0x7fd1a8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:30.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.626+0000 7fd1beffd640 1 -- 172.21.15.31:0/919035312 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1c00846f0 con 0x7fd1c815b130 2023-11-26T07:21:30.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.626+0000 7fd1c659c640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1a80667d0 0x7fd1a8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:30.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.626+0000 7fd1c659c640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1a80667d0 0x7fd1a8068c90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80ae000 tx=0x7fd1b80b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:30.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.630+0000 7fd1beffd640 1 -- 172.21.15.31:0/919035312 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd1c80cae90 con 0x7fd1c815b130 2023-11-26T07:21:30.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.786+0000 7fd1c759e640 1 -- 172.21.15.31:0/919035312 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1c80cae90 con 0x7fd1c815b130 2023-11-26T07:21:30.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1beffd640 1 -- 172.21.15.31:0/919035312 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fd1c80cae90 con 0x7fd1c815b130 2023-11-26T07:21:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:30.791 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:04.352334+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:01.142504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:01.723895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 -- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1a80667d0 msgr2=0x7fd1a8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1a80667d0 0x7fd1a8068c90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80ae000 tx=0x7fd1b80b5040 comp rx=0 tx=0).stop 2023-11-26T07:21:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 -- 172.21.15.31:0/919035312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 msgr2=0x7fd1c80cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 0x7fd1c80cb170 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009fa0 tx=0x7fd1c000d560 comp rx=0 tx=0).stop 2023-11-26T07:21:30.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 -- 172.21.15.31:0/919035312 shutdown_connections 2023-11-26T07:21:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fd1a80667d0 0x7fd1a8068c90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1c815b130 0x7fd1c80cb170 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1c80cc680 0x7fd1c81489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 --2- 172.21.15.31:0/919035312 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd1c80b4410 0x7fd1c8155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 -- 172.21.15.31:0/919035312 >> 172.21.15.31:0/919035312 conn(0x7fd1c80b0e00 msgr2=0x7fd1c815ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 -- 172.21.15.31:0/919035312 shutdown_connections 2023-11-26T07:21:30.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:30.790+0000 7fd1c759e640 1 -- 172.21.15.31:0/919035312 wait complete. 2023-11-26T07:21:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:21:01.142504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:21:01.723895+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:21:30.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:21:31.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c594ea640 1 Processor -- start 2023-11-26T07:21:31.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c594ea640 1 -- start start 2023-11-26T07:21:31.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c594ea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6c5415d8f0 0x7f6c5415e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c594ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c540d0510 0x7f6c540d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c594ea640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5415ecc0 con 0x7f6c540ced30 2023-11-26T07:21:31.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c594ea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c540d4420 con 0x7f6c5415d8f0 2023-11-26T07:21:31.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c594ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540cdfb0 con 0x7f6c540d0510 2023-11-26T07:21:31.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c4bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6c540ced30 0x7f6c540cfe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41136/0 (socket says 172.21.15.31:41136) 2023-11-26T07:21:31.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c4bfff640 1 -- 172.21.15.31:0/3719366156 learned_addr learned my addr 172.21.15.31:0/3719366156 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:31.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c52ffd640 1 --2- 172.21.15.31:0/3719366156 >> v2:172.21.15.97:3300/0 conn(0x7f6c5415d8f0 0x7f6c5415e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c537fe640 1 --2- 172.21.15.31:0/3719366156 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c540d0510 0x7f6c540d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c527fc640 1 -- 172.21.15.31:0/3719366156 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455422304 0 0) 0x7f6c5415ecc0 con 0x7f6c540ced30 2023-11-26T07:21:31.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c527fc640 1 -- 172.21.15.31:0/3719366156 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c40003530 con 0x7f6c540ced30 2023-11-26T07:21:31.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c52ffd640 1 -- 172.21.15.31:0/3719366156 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c540d0510 msgr2=0x7f6c540d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:31.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c52ffd640 1 --2- 172.21.15.31:0/3719366156 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c540d0510 0x7f6c540d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c527fc640 1 -- 172.21.15.31:0/3719366156 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3280877405 0 0) 0x7f6c40003530 con 0x7f6c540ced30 2023-11-26T07:21:31.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c52ffd640 1 -- 172.21.15.31:0/3719366156 >> v1:172.21.15.31:6789/0 conn(0x7f6c540ced30 legacy=0x7f6c540cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c52ffd640 1 -- 172.21.15.31:0/3719366156 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540d4140 con 0x7f6c5415d8f0 2023-11-26T07:21:31.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.002+0000 7f6c537fe640 1 --2- 172.21.15.31:0/3719366156 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c540d0510 0x7f6c540d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:31.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c52ffd640 1 --2- 172.21.15.31:0/3719366156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c5415d8f0 0x7f6c5415e780 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6c440097f0 tx=0x7f6c4401c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42a20d26af93cb93 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:31.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c527fc640 1 -- 172.21.15.31:0/3719366156 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c001680 con 0x7f6c5415d8f0 2023-11-26T07:21:31.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c527fc640 1 -- 172.21.15.31:0/3719366156 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c440a7050 con 0x7f6c5415d8f0 2023-11-26T07:21:31.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c527fc640 1 -- 172.21.15.31:0/3719366156 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440ab600 con 0x7f6c5415d8f0 2023-11-26T07:21:31.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c594ea640 1 -- 172.21.15.31:0/3719366156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c5415d8f0 msgr2=0x7f6c5415e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c594ea640 1 --2- 172.21.15.31:0/3719366156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c5415d8f0 0x7f6c5415e780 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6c440097f0 tx=0x7f6c4401c370 comp rx=0 tx=0).stop 2023-11-26T07:21:31.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c594ea640 1 -- 172.21.15.31:0/3719366156 shutdown_connections 2023-11-26T07:21:31.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c594ea640 1 --2- 172.21.15.31:0/3719366156 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c540d0510 0x7f6c540d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c594ea640 1 --2- 172.21.15.31:0/3719366156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c5415d8f0 0x7f6c5415e780 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c594ea640 1 -- 172.21.15.31:0/3719366156 >> 172.21.15.31:0/3719366156 conn(0x7f6c5409c750 msgr2=0x7f6c540d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:31.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c594ea640 1 -- 172.21.15.31:0/3719366156 shutdown_connections 2023-11-26T07:21:31.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.006+0000 7f6c594ea640 1 -- 172.21.15.31:0/3719366156 wait complete. 2023-11-26T07:21:31.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c594ea640 1 Processor -- start 2023-11-26T07:21:31.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c594ea640 1 -- start start 2023-11-26T07:21:31.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c594ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c540ced30 0x7f6c54146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c594ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540d0510 0x7f6c541585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c594ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c5415d8f0 0x7f6c54158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c594ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540cccd0 con 0x7f6c540d0510 2023-11-26T07:21:31.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c594ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540ca500 con 0x7f6c540ced30 2023-11-26T07:21:31.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c594ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540ca710 con 0x7f6c5415d8f0 2023-11-26T07:21:31.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540d0510 0x7f6c541585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540d0510 0x7f6c541585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58030/0 (socket says 172.21.15.31:58030) 2023-11-26T07:21:31.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4bfff640 1 -- 172.21.15.31:0/3384427923 learned_addr learned my addr 172.21.15.31:0/3384427923 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:31.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c52ffd640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c540ced30 0x7f6c54146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c537fe640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c5415d8f0 0x7f6c54158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4bfff640 1 -- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c5415d8f0 msgr2=0x7f6c54158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4bfff640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c5415d8f0 0x7f6c54158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4bfff640 1 -- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c540ced30 msgr2=0x7f6c54146660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4bfff640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c540ced30 0x7f6c54146660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4bfff640 1 -- 172.21.15.31:0/3384427923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c009a70 con 0x7f6c540d0510 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c52ffd640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c540ced30 0x7f6c54146660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c537fe640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c5415d8f0 0x7f6c54158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4bfff640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540d0510 0x7f6c541585c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c580 tx=0x7f6c3c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4b7fe640 1 -- 172.21.15.31:0/3384427923 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c005cf0 con 0x7f6c540d0510 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4b7fe640 1 -- 172.21.15.31:0/3384427923 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c005e50 con 0x7f6c540d0510 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c4b7fe640 1 -- 172.21.15.31:0/3384427923 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c007330 con 0x7f6c540d0510 2023-11-26T07:21:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c594ea640 1 -- 172.21.15.31:0/3384427923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c44009420 con 0x7f6c540d0510 2023-11-26T07:21:31.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.010+0000 7f6c594ea640 1 -- 172.21.15.31:0/3384427923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c540cacd0 con 0x7f6c540d0510 2023-11-26T07:21:31.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.014+0000 7f6c594ea640 1 -- 172.21.15.31:0/3384427923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5404e850 con 0x7f6c540d0510 2023-11-26T07:21:31.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.014+0000 7f6c4b7fe640 1 -- 172.21.15.31:0/3384427923 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f6c3c007490 con 0x7f6c540d0510 2023-11-26T07:21:31.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.014+0000 7f6c4b7fe640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c340667d0 0x7f6c34068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.014+0000 7f6c4b7fe640 1 -- 172.21.15.31:0/3384427923 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c3c083aa0 con 0x7f6c540d0510 2023-11-26T07:21:31.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.014+0000 7f6c52ffd640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c340667d0 0x7f6c34068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.014+0000 7f6c52ffd640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c340667d0 0x7f6c34068c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6c440ad000 tx=0x7f6c44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.018+0000 7f6c4b7fe640 1 -- 172.21.15.31:0/3384427923 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c3c050b60 con 0x7f6c540d0510 2023-11-26T07:21:31.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.174+0000 7f6c594ea640 1 -- 172.21.15.31:0/3384427923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c540d0cf0 con 0x7f6c540d0510 2023-11-26T07:21:31.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c4b7fe640 1 -- 172.21.15.31:0/3384427923 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c3c017020 con 0x7f6c540d0510 2023-11-26T07:21:31.179 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:21:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 -- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c340667d0 msgr2=0x7f6c34068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c340667d0 0x7f6c34068c90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6c440ad000 tx=0x7f6c44008040 comp rx=0 tx=0).stop 2023-11-26T07:21:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 -- 172.21.15.31:0/3384427923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540d0510 msgr2=0x7f6c541585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540d0510 0x7f6c541585c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c580 tx=0x7f6c3c00ca40 comp rx=0 tx=0).stop 2023-11-26T07:21:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 -- 172.21.15.31:0/3384427923 shutdown_connections 2023-11-26T07:21:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6c340667d0 0x7f6c34068c90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c5415d8f0 0x7f6c54158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540d0510 0x7f6c541585c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 --2- 172.21.15.31:0/3384427923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c540ced30 0x7f6c54146660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 -- 172.21.15.31:0/3384427923 >> 172.21.15.31:0/3384427923 conn(0x7f6c5409c750 msgr2=0x7f6c540b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:31.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 -- 172.21.15.31:0/3384427923 shutdown_connections 2023-11-26T07:21:31.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.178+0000 7f6c594ea640 1 -- 172.21.15.31:0/3384427923 wait complete. 2023-11-26T07:21:31.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:21:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac96a32640 1 Processor -- start 2023-11-26T07:21:31.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac96a32640 1 -- start start 2023-11-26T07:21:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac96a32640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fac900b4210 0x7fac900b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac96a32640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac90147190 con 0x7fac9015db40 2023-11-26T07:21:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac96a32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9006d030 con 0x7fac90146820 2023-11-26T07:21:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac96a32640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac900b4be0 con 0x7fac900b4210 2023-11-26T07:21:31.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac94fcf640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fac9015db40 0x7fac9015df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41140/0 (socket says 172.21.15.31:41140) 2023-11-26T07:21:31.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac94fcf640 1 -- 172.21.15.31:0/1415309064 learned_addr learned my addr 172.21.15.31:0/1415309064 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8f7fe640 1 --2- 172.21.15.31:0/1415309064 >> v2:172.21.15.97:3300/0 conn(0x7fac900b4210 0x7fac900b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27629800 0 0) 0x7fac90147190 con 0x7fac9015db40 2023-11-26T07:21:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac78003530 con 0x7fac9015db40 2023-11-26T07:21:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332298792 0 0) 0x7fac9006d030 con 0x7fac90146820 2023-11-26T07:21:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac90147190 con 0x7fac90146820 2023-11-26T07:21:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4277667387 0 0) 0x7fac78003530 con 0x7fac9015db40 2023-11-26T07:21:31.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 >> v2:172.21.15.97:3300/0 conn(0x7fac900b4210 msgr2=0x7fac900b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 --2- 172.21.15.31:0/1415309064 >> v2:172.21.15.97:3300/0 conn(0x7fac900b4210 0x7fac900b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 >> v1:172.21.15.97:6789/0 conn(0x7fac90146820 legacy=0x7fac90070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac90146c00 con 0x7fac9015db40 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fac88001a00 con 0x7fac9015db40 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 >> v1:172.21.15.31:6789/0 conn(0x7fac9015db40 legacy=0x7fac9015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac78003b60 0x7fac78003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac780047d0 0x7fac78004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac78005190 0x7fac7800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac900b4be0 con 0x7fac78005190 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7800c060 con 0x7fac78003b60 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7800c220 con 0x7fac780047d0 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8f7fe640 1 --2- 172.21.15.31:0/1415309064 >> v2:172.21.15.97:3300/0 conn(0x7fac900b4210 0x7fac900b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:31.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac94fcf640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac78005190 0x7fac7800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8ffff640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac78003b60 0x7fac78003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac8f7fe640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac780047d0 0x7fac78004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac94fcf640 1 -- 172.21.15.31:0/1415309064 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac780047d0 msgr2=0x7fac78004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac94fcf640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac780047d0 0x7fac78004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac94fcf640 1 -- 172.21.15.31:0/1415309064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac78003b60 msgr2=0x7fac78003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac94fcf640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac78003b60 0x7fac78003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.346+0000 7fac94fcf640 1 -- 172.21.15.31:0/1415309064 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac78003530 con 0x7fac78005190 2023-11-26T07:21:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac94fcf640 1 -- 172.21.15.31:0/1415309064 reap_dead start 2023-11-26T07:21:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac94fcf640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac78005190 0x7fac7800b990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fac8800a710 tx=0x7fac8801f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ef00f650952353 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:31.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac880b0070 con 0x7fac78005190 2023-11-26T07:21:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac880ac410 con 0x7fac78005190 2023-11-26T07:21:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac8effd640 1 -- 172.21.15.31:0/1415309064 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac880ac7e0 con 0x7fac78005190 2023-11-26T07:21:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac96a32640 1 -- 172.21.15.31:0/1415309064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac78005190 msgr2=0x7fac7800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac96a32640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac78005190 0x7fac7800b990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fac8800a710 tx=0x7fac8801f960 comp rx=0 tx=0).stop 2023-11-26T07:21:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac96a32640 1 -- 172.21.15.31:0/1415309064 shutdown_connections 2023-11-26T07:21:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac96a32640 1 --2- 172.21.15.31:0/1415309064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac78005190 0x7fac7800b990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac96a32640 1 -- 172.21.15.31:0/1415309064 >> 172.21.15.31:0/1415309064 conn(0x7fac900b2f10 msgr2=0x7fac900b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac96a32640 1 -- 172.21.15.31:0/1415309064 shutdown_connections 2023-11-26T07:21:31.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.350+0000 7fac96a32640 1 -- 172.21.15.31:0/1415309064 wait complete. 2023-11-26T07:21:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac96a32640 1 Processor -- start 2023-11-26T07:21:31.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac96a32640 1 -- start start 2023-11-26T07:21:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac96a32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac78005190 0x7fac901e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac96a32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac9015cda0 0x7fac9013e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac96a32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac9013e590 0x7fac901f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac96a32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9013ea40 con 0x7fac9015cda0 2023-11-26T07:21:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac96a32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac901e2c30 con 0x7fac9013e590 2023-11-26T07:21:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac96a32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9013d300 con 0x7fac78005190 2023-11-26T07:21:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac9015cda0 0x7fac9013e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac9015cda0 0x7fac9013e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58044/0 (socket says 172.21.15.31:58044) 2023-11-26T07:21:31.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8f7fe640 1 -- 172.21.15.31:0/1569373303 learned_addr learned my addr 172.21.15.31:0/1569373303 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:31.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac94fcf640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac9013e590 0x7fac901f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8ffff640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac78005190 0x7fac901e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8f7fe640 1 -- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac78005190 msgr2=0x7fac901e4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8f7fe640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac78005190 0x7fac901e4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8f7fe640 1 -- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac9013e590 msgr2=0x7fac901f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8f7fe640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac9013e590 0x7fac901f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8f7fe640 1 -- 172.21.15.31:0/1569373303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9013d5e0 con 0x7fac9015cda0 2023-11-26T07:21:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8f7fe640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac9015cda0 0x7fac9013e050 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fac84011660 tx=0x7fac84011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8cff9640 1 -- 172.21.15.31:0/1569373303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400f960 con 0x7fac9015cda0 2023-11-26T07:21:31.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8cff9640 1 -- 172.21.15.31:0/1569373303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8400fac0 con 0x7fac9015cda0 2023-11-26T07:21:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac8cff9640 1 -- 172.21.15.31:0/1569373303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400fe30 con 0x7fac9015cda0 2023-11-26T07:21:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac96a32640 1 -- 172.21.15.31:0/1569373303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9013d8c0 con 0x7fac9015cda0 2023-11-26T07:21:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.354+0000 7fac96a32640 1 -- 172.21.15.31:0/1569373303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9015d220 con 0x7fac9015cda0 2023-11-26T07:21:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.358+0000 7fac96a32640 1 -- 172.21.15.31:0/1569373303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9004e850 con 0x7fac9015cda0 2023-11-26T07:21:31.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.358+0000 7fac8cff9640 1 -- 172.21.15.31:0/1569373303 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fac84015460 con 0x7fac9015cda0 2023-11-26T07:21:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.358+0000 7fac8cff9640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac7c0667d0 0x7fac7c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.358+0000 7fac8cff9640 1 -- 172.21.15.31:0/1569373303 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac84084870 con 0x7fac9015cda0 2023-11-26T07:21:31.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.358+0000 7fac8ffff640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac7c0667d0 0x7fac7c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.358+0000 7fac8ffff640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac7c0667d0 0x7fac7c068c90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac80000c10 tx=0x7fac80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:31.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.362+0000 7fac8cff9640 1 -- 172.21.15.31:0/1569373303 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fac84051950 con 0x7fac9015cda0 2023-11-26T07:21:31.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.518+0000 7fac96a32640 1 -- 172.21.15.31:0/1569373303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fac901e56d0 con 0x7fac9015cda0 2023-11-26T07:21:31.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac8cff9640 1 -- 172.21.15.31:0/1569373303 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fac84054f70 con 0x7fac9015cda0 2023-11-26T07:21:31.523 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:21:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 -- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac7c0667d0 msgr2=0x7fac7c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac7c0667d0 0x7fac7c068c90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac80000c10 tx=0x7fac80009040 comp rx=0 tx=0).stop 2023-11-26T07:21:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 -- 172.21.15.31:0/1569373303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac9015cda0 msgr2=0x7fac9013e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac9015cda0 0x7fac9013e050 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fac84011660 tx=0x7fac84011b20 comp rx=0 tx=0).stop 2023-11-26T07:21:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 -- 172.21.15.31:0/1569373303 shutdown_connections 2023-11-26T07:21:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fac7c0667d0 0x7fac7c068c90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac9013e590 0x7fac901f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac9015cda0 0x7fac9013e050 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 --2- 172.21.15.31:0/1569373303 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fac78005190 0x7fac901e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 -- 172.21.15.31:0/1569373303 >> 172.21.15.31:0/1569373303 conn(0x7fac900b2f10 msgr2=0x7fac901451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:31.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 -- 172.21.15.31:0/1569373303 shutdown_connections 2023-11-26T07:21:31.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.522+0000 7fac96a32640 1 -- 172.21.15.31:0/1569373303 wait complete. 2023-11-26T07:21:31.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:21:31.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b383df640 1 Processor -- start 2023-11-26T07:21:31.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b383df640 1 -- start start 2023-11-26T07:21:31.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b383df640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1b300ce5d0 0x7f1b3015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b383df640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b30159630 con 0x7f1b300cb9b0 2023-11-26T07:21:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b383df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b300cd1d0 con 0x7f1b3006e3f0 2023-11-26T07:21:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b383df640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b3006e860 con 0x7f1b300ce5d0 2023-11-26T07:21:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b3697c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1b300ce5d0 0x7f1b3015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b3697c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1b300ce5d0 0x7f1b3015bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58060/0 (socket says 172.21.15.31:58060) 2023-11-26T07:21:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b3697c640 1 -- 172.21.15.31:0/1403216174 learned_addr learned my addr 172.21.15.31:0/1403216174 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b3697c640 1 -- 172.21.15.31:0/1403216174 >> v1:172.21.15.97:6789/0 conn(0x7f1b3006e3f0 legacy=0x7f1b3015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b3697c640 1 -- 172.21.15.31:0/1403216174 >> v1:172.21.15.31:6789/0 conn(0x7f1b300cb9b0 legacy=0x7f1b300cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b3697c640 1 -- 172.21.15.31:0/1403216174 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b300cd4b0 con 0x7f1b300ce5d0 2023-11-26T07:21:31.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.690+0000 7f1b35179640 1 -- 172.21.15.31:0/1403216174 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390328750 0 0) 0x7f1b30159630 con 0x7f1b300cb9b0 2023-11-26T07:21:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b3697c640 1 --2- 172.21.15.31:0/1403216174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300ce5d0 0x7f1b3015bfd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009e50 tx=0x7f1b2801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a792fad6a7a9f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b35179640 1 -- 172.21.15.31:0/1403216174 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b280a56c0 con 0x7f1b300ce5d0 2023-11-26T07:21:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b35179640 1 -- 172.21.15.31:0/1403216174 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b280a5820 con 0x7f1b300ce5d0 2023-11-26T07:21:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b35179640 1 -- 172.21.15.31:0/1403216174 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b280a5bb0 con 0x7f1b300ce5d0 2023-11-26T07:21:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b383df640 1 -- 172.21.15.31:0/1403216174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300ce5d0 msgr2=0x7f1b3015bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b383df640 1 --2- 172.21.15.31:0/1403216174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300ce5d0 0x7f1b3015bfd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009e50 tx=0x7f1b2801ca10 comp rx=0 tx=0).stop 2023-11-26T07:21:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b383df640 1 -- 172.21.15.31:0/1403216174 shutdown_connections 2023-11-26T07:21:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b383df640 1 --2- 172.21.15.31:0/1403216174 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300ce5d0 0x7f1b3015bfd0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b383df640 1 -- 172.21.15.31:0/1403216174 >> 172.21.15.31:0/1403216174 conn(0x7f1b300b0cb0 msgr2=0x7f1b300b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b383df640 1 -- 172.21.15.31:0/1403216174 shutdown_connections 2023-11-26T07:21:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.694+0000 7f1b383df640 1 -- 172.21.15.31:0/1403216174 wait complete. 2023-11-26T07:21:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 Processor -- start 2023-11-26T07:21:31.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 -- start start 2023-11-26T07:21:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b3006e3f0 0x7f1b301505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300cb9b0 0x7f1b3014f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b300ce5d0 0x7f1b300b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300afbf0 con 0x7f1b300cb9b0 2023-11-26T07:21:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3015ae80 con 0x7f1b300ce5d0 2023-11-26T07:21:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3015b160 con 0x7f1b3006e3f0 2023-11-26T07:21:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3597a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300cb9b0 0x7f1b3014f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3597a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300cb9b0 0x7f1b3014f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58062/0 (socket says 172.21.15.31:58062) 2023-11-26T07:21:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3597a640 1 -- 172.21.15.31:0/2046322705 learned_addr learned my addr 172.21.15.31:0/2046322705 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:31.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3697c640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b300ce5d0 0x7f1b300b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3617b640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b3006e3f0 0x7f1b301505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3597a640 1 -- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b3006e3f0 msgr2=0x7f1b301505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3597a640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b3006e3f0 0x7f1b301505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3597a640 1 -- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b300ce5d0 msgr2=0x7f1b300b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3597a640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b300ce5d0 0x7f1b300b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3597a640 1 -- 172.21.15.31:0/2046322705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b28009b30 con 0x7f1b300cb9b0 2023-11-26T07:21:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3697c640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b300ce5d0 0x7f1b300b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3617b640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b3006e3f0 0x7f1b301505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b3597a640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300cb9b0 0x7f1b3014f880 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000c3c0 tx=0x7f1b2000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:31.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b26ffd640 1 -- 172.21.15.31:0/2046322705 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b200052d0 con 0x7f1b300cb9b0 2023-11-26T07:21:31.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b26ffd640 1 -- 172.21.15.31:0/2046322705 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b20005430 con 0x7f1b300cb9b0 2023-11-26T07:21:31.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b26ffd640 1 -- 172.21.15.31:0/2046322705 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20003840 con 0x7f1b300cb9b0 2023-11-26T07:21:31.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 -- 172.21.15.31:0/2046322705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3015b440 con 0x7f1b300cb9b0 2023-11-26T07:21:31.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 -- 172.21.15.31:0/2046322705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3014fdc0 con 0x7f1b300cb9b0 2023-11-26T07:21:31.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.698+0000 7f1b383df640 1 -- 172.21.15.31:0/2046322705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b300671f0 con 0x7f1b300cb9b0 2023-11-26T07:21:31.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.702+0000 7f1b26ffd640 1 -- 172.21.15.31:0/2046322705 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f1b200039a0 con 0x7f1b300cb9b0 2023-11-26T07:21:31.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.702+0000 7f1b26ffd640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b140667d0 0x7f1b14068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:31.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.702+0000 7f1b26ffd640 1 -- 172.21.15.31:0/2046322705 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b200838b0 con 0x7f1b300cb9b0 2023-11-26T07:21:31.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.702+0000 7f1b3617b640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b140667d0 0x7f1b14068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:31.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.702+0000 7f1b3617b640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b140667d0 0x7f1b14068c90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c000f70 tx=0x7f1b2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:31.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.706+0000 7f1b26ffd640 1 -- 172.21.15.31:0/2046322705 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1b20050970 con 0x7f1b300cb9b0 2023-11-26T07:21:31.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 -- 172.21.15.31:0/2046322705 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b3006b460 con 0x7f1b300cb9b0 2023-11-26T07:21:31.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b26ffd640 1 -- 172.21.15.31:0/2046322705 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b200170a0 con 0x7f1b300cb9b0 2023-11-26T07:21:31.868 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:21:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 -- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b140667d0 msgr2=0x7f1b14068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b140667d0 0x7f1b14068c90 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c000f70 tx=0x7f1b2c009040 comp rx=0 tx=0).stop 2023-11-26T07:21:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 -- 172.21.15.31:0/2046322705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300cb9b0 msgr2=0x7f1b3014f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300cb9b0 0x7f1b3014f880 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000c3c0 tx=0x7f1b2000c880 comp rx=0 tx=0).stop 2023-11-26T07:21:31.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 -- 172.21.15.31:0/2046322705 shutdown_connections 2023-11-26T07:21:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f1b140667d0 0x7f1b14068c90 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b300ce5d0 0x7f1b300b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b300cb9b0 0x7f1b3014f880 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 --2- 172.21.15.31:0/2046322705 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b3006e3f0 0x7f1b301505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 -- 172.21.15.31:0/2046322705 >> 172.21.15.31:0/2046322705 conn(0x7f1b300b0cb0 msgr2=0x7f1b300b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 -- 172.21.15.31:0/2046322705 shutdown_connections 2023-11-26T07:21:31.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:31.866+0000 7f1b383df640 1 -- 172.21.15.31:0/2046322705 wait complete. 2023-11-26T07:21:31.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:21:32.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.030+0000 7f6472a9c640 1 Processor -- start 2023-11-26T07:21:32.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.030+0000 7f6472a9c640 1 -- start start 2023-11-26T07:21:32.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.030+0000 7f6472a9c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f646c0b5830 0x7f646c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.030+0000 7f6472a9c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f646c0ad8e0 0x7f646c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.030+0000 7f6472a9c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646c0af630 con 0x7f646c0acd70 2023-11-26T07:21:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.030+0000 7f6472a9c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646c0acb10 con 0x7f646c0ad8e0 2023-11-26T07:21:32.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.030+0000 7f6472a9c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0ae5b0 con 0x7f646c0b5830 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f646c0acd70 0x7f646c0ad1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41162/0 (socket says 172.21.15.31:41162) 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646bfff640 1 -- 172.21.15.31:0/3814420090 learned_addr learned my addr 172.21.15.31:0/3814420090 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f6470838640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f646c0b5830 0x7f646c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f6471039640 1 --2- 172.21.15.31:0/3814420090 >> v2:172.21.15.97:3300/0 conn(0x7f646c0ad8e0 0x7f646c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701840781 0 0) 0x7f6458001120 con 0x7f646c0acd70 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f645c003530 con 0x7f646c0acd70 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 422610662 0 0) 0x7f645c003530 con 0x7f646c0acd70 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f646c0b5830 msgr2=0x7f646c15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f646c0b5830 0x7f646c15e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 >> v2:172.21.15.97:3300/0 conn(0x7f646c0ad8e0 msgr2=0x7f646c06d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:32.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 --2- 172.21.15.31:0/3814420090 >> v2:172.21.15.97:3300/0 conn(0x7f646c0ad8e0 0x7f646c06d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c0ae870 con 0x7f646c0acd70 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f6458001810 con 0x7f646c0acd70 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 >> v1:172.21.15.31:6789/0 conn(0x7f646c0acd70 legacy=0x7f646c0ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f646c0b5830 0x7f645c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c003f50 0x7f645c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c004d10 0x7f645c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0acb10 con 0x7f645c003f50 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0ae5b0 con 0x7f645c004d10 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c00b990 con 0x7f646c0b5830 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646bfff640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c003f50 0x7f645c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f6470838640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f646c0b5830 0x7f645c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f6471039640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c004d10 0x7f645c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646bfff640 1 -- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f646c0b5830 msgr2=0x7f645c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646bfff640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f646c0b5830 0x7f645c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646bfff640 1 -- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c004d10 msgr2=0x7f645c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646bfff640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f645c004d10 0x7f645c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.034+0000 7f646bfff640 1 -- 172.21.15.31:0/3814420090 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c003530 con 0x7f645c003f50 2023-11-26T07:21:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f646bfff640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c003f50 0x7f645c0047d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f645800c140 tx=0x7f645801f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22474358e010678a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458005ae0 con 0x7f645c003f50 2023-11-26T07:21:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64580ab070 con 0x7f645c003f50 2023-11-26T07:21:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f646b7fe640 1 -- 172.21.15.31:0/3814420090 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6458007070 con 0x7f645c003f50 2023-11-26T07:21:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f6472a9c640 1 -- 172.21.15.31:0/3814420090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c003f50 msgr2=0x7f645c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f6472a9c640 1 --2- 172.21.15.31:0/3814420090 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645c003f50 0x7f645c0047d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f645800c140 tx=0x7f645801f2f0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f6471039640 1 -- 172.21.15.31:0/3814420090 reap_dead start 2023-11-26T07:21:32.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f6472a9c640 1 -- 172.21.15.31:0/3814420090 shutdown_connections 2023-11-26T07:21:32.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f6472a9c640 1 -- 172.21.15.31:0/3814420090 >> 172.21.15.31:0/3814420090 conn(0x7f646c071d80 msgr2=0x7f646c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:32.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f6472a9c640 1 -- 172.21.15.31:0/3814420090 shutdown_connections 2023-11-26T07:21:32.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f6472a9c640 1 -- 172.21.15.31:0/3814420090 wait complete. 2023-11-26T07:21:32.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f6472a9c640 1 Processor -- start 2023-11-26T07:21:32.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.038+0000 7f6472a9c640 1 -- start start 2023-11-26T07:21:32.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6472a9c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f645c003f50 0x7f646c0ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6472a9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f646c159020 0x7f646c1f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6472a9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646c14c530 0x7f646c159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6472a9c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0cc4d0 con 0x7f646c14c530 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6472a9c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0cc7b0 con 0x7f646c159020 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6472a9c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0cca90 con 0x7f645c003f50 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6471039640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646c14c530 0x7f646c159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6471039640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646c14c530 0x7f646c159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58082/0 (socket says 172.21.15.31:58082) 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6471039640 1 -- 172.21.15.31:0/3918817656 learned_addr learned my addr 172.21.15.31:0/3918817656 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f646bfff640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f646c159020 0x7f646c1f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6470838640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f645c003f50 0x7f646c0ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6471039640 1 -- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f645c003f50 msgr2=0x7f646c0ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6471039640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f645c003f50 0x7f646c0ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6471039640 1 -- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f646c159020 msgr2=0x7f646c1f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6471039640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f646c159020 0x7f646c1f3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6471039640 1 -- 172.21.15.31:0/3918817656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c0ccd70 con 0x7f646c14c530 2023-11-26T07:21:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6471039640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646c14c530 0x7f646c159af0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f646c0cc320 tx=0x7f6464009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f64697fa640 1 -- 172.21.15.31:0/3918817656 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646400e040 con 0x7f646c14c530 2023-11-26T07:21:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f64697fa640 1 -- 172.21.15.31:0/3918817656 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646400f040 con 0x7f646c14c530 2023-11-26T07:21:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6472a9c640 1 -- 172.21.15.31:0/3918817656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646c0cf0b0 con 0x7f646c14c530 2023-11-26T07:21:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f64697fa640 1 -- 172.21.15.31:0/3918817656 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646400feb0 con 0x7f646c14c530 2023-11-26T07:21:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6472a9c640 1 -- 172.21.15.31:0/3918817656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c14c9c0 con 0x7f646c14c530 2023-11-26T07:21:32.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.042+0000 7f6472a9c640 1 -- 172.21.15.31:0/3918817656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646c0ccf60 con 0x7f646c14c530 2023-11-26T07:21:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.046+0000 7f64697fa640 1 -- 172.21.15.31:0/3918817656 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f6464022460 con 0x7f646c14c530 2023-11-26T07:21:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.046+0000 7f64697fa640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f64500667d0 0x7f6450068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.046+0000 7f64697fa640 1 -- 172.21.15.31:0/3918817656 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6464084ac0 con 0x7f646c14c530 2023-11-26T07:21:32.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.046+0000 7f6470838640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f64500667d0 0x7f6450068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.046+0000 7f6470838640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f64500667d0 0x7f6450068c90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6460009b00 tx=0x7f6460009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:32.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.050+0000 7f64697fa640 1 -- 172.21.15.31:0/3918817656 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6464086e60 con 0x7f646c14c530 2023-11-26T07:21:32.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.218+0000 7f6472a9c640 1 -- 172.21.15.31:0/3918817656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f646c0cc610 con 0x7f646c14c530 2023-11-26T07:21:32.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f64697fa640 1 -- 172.21.15.31:0/3918817656 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) v1 ==== 80+0+0 (secure 0 0 0) 0x7f646c0cc610 con 0x7f646c14c530 2023-11-26T07:21:32.223 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:21:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 -- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f64500667d0 msgr2=0x7f6450068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f64500667d0 0x7f6450068c90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6460009b00 tx=0x7f6460009190 comp rx=0 tx=0).stop 2023-11-26T07:21:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 -- 172.21.15.31:0/3918817656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646c14c530 msgr2=0x7f646c159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646c14c530 0x7f646c159af0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f646c0cc320 tx=0x7f6464009f70 comp rx=0 tx=0).stop 2023-11-26T07:21:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 -- 172.21.15.31:0/3918817656 shutdown_connections 2023-11-26T07:21:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f64500667d0 0x7f6450068c90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f646c14c530 0x7f646c159af0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f646c159020 0x7f646c1f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 --2- 172.21.15.31:0/3918817656 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f645c003f50 0x7f646c0ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 -- 172.21.15.31:0/3918817656 >> 172.21.15.31:0/3918817656 conn(0x7f646c071d80 msgr2=0x7f646c06dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:32.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 -- 172.21.15.31:0/3918817656 shutdown_connections 2023-11-26T07:21:32.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.222+0000 7f6472a9c640 1 -- 172.21.15.31:0/3918817656 wait complete. 2023-11-26T07:21:32.244 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:21:32.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:32.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:21:32.277 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:21:32.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:32.320 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:21:32.331 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:21:32.331 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:32.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:21:32.341 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:21:32.341 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:32.382 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:21:32.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:32.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.526+0000 7fa333157640 1 Processor -- start 2023-11-26T07:21:32.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.526+0000 7fa333157640 1 -- start start 2023-11-26T07:21:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa333157640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa32c06dfd0 0x7fa32c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa333157640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c0adcc0 0x7fa32c0ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa333157640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c06e980 con 0x7fa32c143ae0 2023-11-26T07:21:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa333157640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c070670 con 0x7fa32c06dfd0 2023-11-26T07:21:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa333157640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c070950 con 0x7fa32c0adcc0 2023-11-26T07:21:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa32bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa32c06dfd0 0x7fa32c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa32bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa32c06dfd0 0x7fa32c06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58084/0 (socket says 172.21.15.31:58084) 2023-11-26T07:21:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa32bfff640 1 -- 172.21.15.31:0/4150075795 learned_addr learned my addr 172.21.15.31:0/4150075795 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa3316f4640 1 --2- 172.21.15.31:0/4150075795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c0adcc0 0x7fa32c0ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa32bfff640 1 -- 172.21.15.31:0/4150075795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c0adcc0 msgr2=0x7fa32c0ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa32bfff640 1 --2- 172.21.15.31:0/4150075795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c0adcc0 0x7fa32c0ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa32bfff640 1 -- 172.21.15.31:0/4150075795 >> v1:172.21.15.97:6789/0 conn(0x7fa32c143ae0 legacy=0x7fa32c0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.530+0000 7fa32bfff640 1 -- 172.21.15.31:0/4150075795 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c070c30 con 0x7fa32c06dfd0 2023-11-26T07:21:32.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa32bfff640 1 --2- 172.21.15.31:0/4150075795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c06dfd0 0x7fa32c06e440 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa318009870 tx=0x7fa31801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195e3ebf8f5a7d87 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:32.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa32b7fe640 1 -- 172.21.15.31:0/4150075795 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180aa670 con 0x7fa32c06dfd0 2023-11-26T07:21:32.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa32b7fe640 1 -- 172.21.15.31:0/4150075795 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3180aa7d0 con 0x7fa32c06dfd0 2023-11-26T07:21:32.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa32b7fe640 1 -- 172.21.15.31:0/4150075795 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180aab60 con 0x7fa32c06dfd0 2023-11-26T07:21:32.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 -- 172.21.15.31:0/4150075795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c06dfd0 msgr2=0x7fa32c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 --2- 172.21.15.31:0/4150075795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c06dfd0 0x7fa32c06e440 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa318009870 tx=0x7fa31801c510 comp rx=0 tx=0).stop 2023-11-26T07:21:32.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 -- 172.21.15.31:0/4150075795 shutdown_connections 2023-11-26T07:21:32.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 --2- 172.21.15.31:0/4150075795 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c0adcc0 0x7fa32c0ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 --2- 172.21.15.31:0/4150075795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c06dfd0 0x7fa32c06e440 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 -- 172.21.15.31:0/4150075795 >> 172.21.15.31:0/4150075795 conn(0x7fa32c0b0c40 msgr2=0x7fa32c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:32.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 -- 172.21.15.31:0/4150075795 shutdown_connections 2023-11-26T07:21:32.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 -- 172.21.15.31:0/4150075795 wait complete. 2023-11-26T07:21:32.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 Processor -- start 2023-11-26T07:21:32.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 -- start start 2023-11-26T07:21:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.534+0000 7fa333157640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c06dfd0 0x7fa32c12e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa333157640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0adcc0 0x7fa32c0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa333157640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa32c143ae0 0x7fa32c12fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa333157640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c12ee40 con 0x7fa32c0adcc0 2023-11-26T07:21:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa333157640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c12f120 con 0x7fa32c143ae0 2023-11-26T07:21:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa333157640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c12f400 con 0x7fa32c06dfd0 2023-11-26T07:21:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa32bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0adcc0 0x7fa32c0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa32bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0adcc0 0x7fa32c0cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58086/0 (socket says 172.21.15.31:58086) 2023-11-26T07:21:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa32bfff640 1 -- 172.21.15.31:0/4135079439 learned_addr learned my addr 172.21.15.31:0/4135079439 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa330ef3640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c06dfd0 0x7fa32c12e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa3316f4640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa32c143ae0 0x7fa32c12fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa32bfff640 1 -- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c06dfd0 msgr2=0x7fa32c12e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa32bfff640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c06dfd0 0x7fa32c12e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa32bfff640 1 -- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa32c143ae0 msgr2=0x7fa32c12fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa32bfff640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa32c143ae0 0x7fa32c12fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa32bfff640 1 -- 172.21.15.31:0/4135079439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180094d0 con 0x7fa32c0adcc0 2023-11-26T07:21:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa330ef3640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c06dfd0 0x7fa32c12e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:32.541 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa3316f4640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa32c143ae0 0x7fa32c12fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa32bfff640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0adcc0 0x7fa32c0cd9d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa3180ac040 tx=0x7fa31801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa3297fa640 1 -- 172.21.15.31:0/4135079439 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180b3800 con 0x7fa32c0adcc0 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa3297fa640 1 -- 172.21.15.31:0/4135079439 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3180b3980 con 0x7fa32c0adcc0 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa3297fa640 1 -- 172.21.15.31:0/4135079439 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180b3d30 con 0x7fa32c0adcc0 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa333157640 1 -- 172.21.15.31:0/4135079439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c12f6e0 con 0x7fa32c0adcc0 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa333157640 1 -- 172.21.15.31:0/4135079439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c0cdf10 con 0x7fa32c0adcc0 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa333157640 1 -- 172.21.15.31:0/4135079439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c04e850 con 0x7fa32c0adcc0 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.538+0000 7fa3297fa640 1 -- 172.21.15.31:0/4135079439 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fa3180b2380 con 0x7fa32c0adcc0 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.542+0000 7fa3297fa640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa310066800 0x7fa310068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.542+0000 7fa3297fa640 1 -- 172.21.15.31:0/4135079439 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa31811ec80 con 0x7fa32c0adcc0 2023-11-26T07:21:32.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.542+0000 7fa330ef3640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa310066800 0x7fa310068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.544 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.542+0000 7fa330ef3640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa310066800 0x7fa310068cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa320000f70 tx=0x7fa320009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:32.546 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.546+0000 7fa3297fa640 1 -- 172.21.15.31:0/4135079439 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa318125080 con 0x7fa32c0adcc0 2023-11-26T07:21:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.734+0000 7fa333157640 1 -- 172.21.15.31:0/4135079439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa32c12fb90 con 0x7fa32c0adcc0 2023-11-26T07:21:32.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.734+0000 7fa3297fa640 1 -- 172.21.15.31:0/4135079439 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4910 (secure 0 0 0) 0x7fa3180c3090 con 0x7fa32c0adcc0 2023-11-26T07:21:32.737 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:32.737 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3429205589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3429205589},{"type":"v1","addr":"172.21.15.97:6836","nonce":3429205589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5959,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3569564413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3569564413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3569564413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5965,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690524463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690524463},{"type":"v1","addr":"172.21.15.97:6837","nonce":690524463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:02.722567+0000","modified":"2023-11-26T07:21:05.757700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1053315203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1053315203},{"type":"v1","addr":"172.21.15.31:6834","nonce":1053315203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:21:32.737 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2023-11-26T07:21:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.734+0000 7fa333157640 1 -- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa310066800 msgr2=0x7fa310068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.734+0000 7fa333157640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa310066800 0x7fa310068cc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa320000f70 tx=0x7fa320009040 comp rx=0 tx=0).stop 2023-11-26T07:21:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.734+0000 7fa333157640 1 -- 172.21.15.31:0/4135079439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0adcc0 msgr2=0x7fa32c0cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.734+0000 7fa333157640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0adcc0 0x7fa32c0cd9d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa3180ac040 tx=0x7fa31801ad50 comp rx=0 tx=0).stop 2023-11-26T07:21:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.738+0000 7fa333157640 1 -- 172.21.15.31:0/4135079439 shutdown_connections 2023-11-26T07:21:32.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.738+0000 7fa333157640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa310066800 0x7fa310068cc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.738+0000 7fa333157640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa32c143ae0 0x7fa32c12fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.738+0000 7fa333157640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa32c0adcc0 0x7fa32c0cd9d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.738+0000 7fa333157640 1 --2- 172.21.15.31:0/4135079439 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa32c06dfd0 0x7fa32c12e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.738+0000 7fa333157640 1 -- 172.21.15.31:0/4135079439 >> 172.21.15.31:0/4135079439 conn(0x7fa32c0b0c40 msgr2=0x7fa32c0afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.738+0000 7fa333157640 1 -- 172.21.15.31:0/4135079439 shutdown_connections 2023-11-26T07:21:32.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.738+0000 7fa333157640 1 -- 172.21.15.31:0/4135079439 wait complete. 2023-11-26T07:21:32.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:32.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac3454640 1 Processor -- start 2023-11-26T07:21:32.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac3454640 1 -- start start 2023-11-26T07:21:32.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac3454640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2abc144c80 0x7f2abc0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac3454640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2abc0ac460 0x7f2abc0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac3454640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2abc070c20 0x7f2abc15eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac3454640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2abc145060 con 0x7f2abc070c20 2023-11-26T07:21:32.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac3454640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2abc071fd0 con 0x7f2abc0ac460 2023-11-26T07:21:32.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac3454640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0ac8c0 con 0x7f2abc144c80 2023-11-26T07:21:32.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac19f1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2abc070c20 0x7f2abc15eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac19f1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2abc070c20 0x7f2abc15eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58100/0 (socket says 172.21.15.31:58100) 2023-11-26T07:21:32.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac19f1640 1 -- 172.21.15.31:0/2612295902 learned_addr learned my addr 172.21.15.31:0/2612295902 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:32.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac09ef640 1 --2- 172.21.15.31:0/2612295902 >> v2:172.21.15.97:3300/0 conn(0x7f2abc0ac460 0x7f2abc0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac11f0640 1 --2- 172.21.15.31:0/2612295902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2abc144c80 0x7f2abc0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac19f1640 1 -- 172.21.15.31:0/2612295902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2abc144c80 msgr2=0x7f2abc0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:32.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac19f1640 1 --2- 172.21.15.31:0/2612295902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2abc144c80 0x7f2abc0abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac19f1640 1 -- 172.21.15.31:0/2612295902 >> v2:172.21.15.97:3300/0 conn(0x7f2abc0ac460 msgr2=0x7f2abc0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:32.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac19f1640 1 --2- 172.21.15.31:0/2612295902 >> v2:172.21.15.97:3300/0 conn(0x7f2abc0ac460 0x7f2abc0706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.898+0000 7f2ac19f1640 1 -- 172.21.15.31:0/2612295902 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc071450 con 0x7f2abc070c20 2023-11-26T07:21:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ac19f1640 1 --2- 172.21.15.31:0/2612295902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc070c20 0x7f2abc15eed0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2abc06bf70 tx=0x7f2ab401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18ecc8005eff408 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:32.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ab3fff640 1 -- 172.21.15.31:0/2612295902 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab40a5560 con 0x7f2abc070c20 2023-11-26T07:21:32.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ab3fff640 1 -- 172.21.15.31:0/2612295902 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab40a56c0 con 0x7f2abc070c20 2023-11-26T07:21:32.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ab3fff640 1 -- 172.21.15.31:0/2612295902 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab40a5a50 con 0x7f2abc070c20 2023-11-26T07:21:32.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ac3454640 1 -- 172.21.15.31:0/2612295902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc070c20 msgr2=0x7f2abc15eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ac3454640 1 --2- 172.21.15.31:0/2612295902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc070c20 0x7f2abc15eed0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2abc06bf70 tx=0x7f2ab401c6e0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ac3454640 1 -- 172.21.15.31:0/2612295902 shutdown_connections 2023-11-26T07:21:32.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ac3454640 1 --2- 172.21.15.31:0/2612295902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc070c20 0x7f2abc15eed0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ac3454640 1 --2- 172.21.15.31:0/2612295902 >> v2:172.21.15.97:3300/0 conn(0x7f2abc0ac460 0x7f2abc0706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ac3454640 1 --2- 172.21.15.31:0/2612295902 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2abc144c80 0x7f2abc0abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ac3454640 1 -- 172.21.15.31:0/2612295902 >> 172.21.15.31:0/2612295902 conn(0x7f2abc0b0c40 msgr2=0x7f2abc0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:32.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ac3454640 1 -- 172.21.15.31:0/2612295902 shutdown_connections 2023-11-26T07:21:32.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.902+0000 7f2ac3454640 1 -- 172.21.15.31:0/2612295902 wait complete. 2023-11-26T07:21:32.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac3454640 1 Processor -- start 2023-11-26T07:21:32.908 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac3454640 1 -- start start 2023-11-26T07:21:32.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac3454640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc070c20 0x7f2abc09c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac3454640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc0ac460 0x7f2abc13c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac3454640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2abc144c80 0x7f2abc1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac3454640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc157710 con 0x7f2abc0ac460 2023-11-26T07:21:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac3454640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc1591a0 con 0x7f2abc070c20 2023-11-26T07:21:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac3454640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc15b9b0 con 0x7f2abc144c80 2023-11-26T07:21:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac09ef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc0ac460 0x7f2abc13c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac09ef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc0ac460 0x7f2abc13c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58116/0 (socket says 172.21.15.31:58116) 2023-11-26T07:21:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac09ef640 1 -- 172.21.15.31:0/2551778166 learned_addr learned my addr 172.21.15.31:0/2551778166 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac19f1640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2abc144c80 0x7f2abc1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac11f0640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc070c20 0x7f2abc09c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac09ef640 1 -- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2abc144c80 msgr2=0x7f2abc1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac09ef640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2abc144c80 0x7f2abc1578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac09ef640 1 -- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc070c20 msgr2=0x7f2abc09c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:32.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac09ef640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc070c20 0x7f2abc09c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:32.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac09ef640 1 -- 172.21.15.31:0/2551778166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4009b30 con 0x7f2abc0ac460 2023-11-26T07:21:32.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac09ef640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc0ac460 0x7f2abc13c840 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00e8b0 tx=0x7f2aac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:32.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ab1ffb640 1 -- 172.21.15.31:0/2551778166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00c940 con 0x7f2abc0ac460 2023-11-26T07:21:32.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ab1ffb640 1 -- 172.21.15.31:0/2551778166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac00caa0 con 0x7f2abc0ac460 2023-11-26T07:21:32.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac3454640 1 -- 172.21.15.31:0/2551778166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc099ec0 con 0x7f2abc0ac460 2023-11-26T07:21:32.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ab1ffb640 1 -- 172.21.15.31:0/2551778166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00ce30 con 0x7f2abc0ac460 2023-11-26T07:21:32.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.906+0000 7f2ac3454640 1 -- 172.21.15.31:0/2551778166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc09a480 con 0x7f2abc0ac460 2023-11-26T07:21:32.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.910+0000 7f2ac3454640 1 -- 172.21.15.31:0/2551778166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc0b1ba0 con 0x7f2abc0ac460 2023-11-26T07:21:32.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.910+0000 7f2ab1ffb640 1 -- 172.21.15.31:0/2551778166 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f2aac013470 con 0x7f2abc0ac460 2023-11-26T07:21:32.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.910+0000 7f2ab1ffb640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2aa00667d0 0x7f2aa0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:32.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.910+0000 7f2ab1ffb640 1 -- 172.21.15.31:0/2551778166 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2aac0845c0 con 0x7f2abc0ac460 2023-11-26T07:21:32.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.910+0000 7f2ac11f0640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2aa00667d0 0x7f2aa0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:32.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.910+0000 7f2ac11f0640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2aa00667d0 0x7f2aa0068c90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009ac0 tx=0x7f2ab8009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:32.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:32.914+0000 7f2ab1ffb640 1 -- 172.21.15.31:0/2551778166 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2aac051610 con 0x7f2abc0ac460 2023-11-26T07:21:33.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.070+0000 7f2ac3454640 1 -- 172.21.15.31:0/2551778166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2abc04e850 con 0x7f2abc0ac460 2023-11-26T07:21:33.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ab1ffb640 1 -- 172.21.15.31:0/2551778166 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f2aac054c30 con 0x7f2abc0ac460 2023-11-26T07:21:33.076 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:33.076 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:04.352334+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:01.142504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:01.723895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 -- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2aa00667d0 msgr2=0x7f2aa0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2aa00667d0 0x7f2aa0068c90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009ac0 tx=0x7f2ab8009190 comp rx=0 tx=0).stop 2023-11-26T07:21:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 -- 172.21.15.31:0/2551778166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc0ac460 msgr2=0x7f2abc13c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc0ac460 0x7f2abc13c840 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00e8b0 tx=0x7f2aac00ed70 comp rx=0 tx=0).stop 2023-11-26T07:21:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 -- 172.21.15.31:0/2551778166 shutdown_connections 2023-11-26T07:21:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2aa00667d0 0x7f2aa0068c90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2abc144c80 0x7f2abc1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2abc0ac460 0x7f2abc13c840 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 --2- 172.21.15.31:0/2551778166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2abc070c20 0x7f2abc09c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 -- 172.21.15.31:0/2551778166 >> 172.21.15.31:0/2551778166 conn(0x7f2abc0b0c40 msgr2=0x7f2abc0aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:33.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 -- 172.21.15.31:0/2551778166 shutdown_connections 2023-11-26T07:21:33.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.074+0000 7f2ac3454640 1 -- 172.21.15.31:0/2551778166 wait complete. 2023-11-26T07:21:33.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:21:01.142504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:21:01.723895+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:21:33.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:21:33.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73f7df640 1 Processor -- start 2023-11-26T07:21:33.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73f7df640 1 -- start start 2023-11-26T07:21:33.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73f7df640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff7380cdfb0 0x7ff7380abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73f7df640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7380b2710 con 0x7ff7380cbd50 2023-11-26T07:21:33.244 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73f7df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7380b29f0 con 0x7ff73815a710 2023-11-26T07:21:33.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73f7df640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7380cca80 con 0x7ff7380cdfb0 2023-11-26T07:21:33.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73dd7c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff7380cbd50 0x7ff7380cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:41174/0 (socket says 172.21.15.31:41174) 2023-11-26T07:21:33.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73dd7c640 1 -- 172.21.15.31:0/2727523490 learned_addr learned my addr 172.21.15.31:0/2727523490 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:33.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73d57b640 1 --2- 172.21.15.31:0/2727523490 >> v2:172.21.15.97:3300/0 conn(0x7ff7380cdfb0 0x7ff7380abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319409372 0 0) 0x7ff7380b2710 con 0x7ff7380cbd50 2023-11-26T07:21:33.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff724003530 con 0x7ff7380cbd50 2023-11-26T07:21:33.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 798080638 0 0) 0x7ff724003530 con 0x7ff7380cbd50 2023-11-26T07:21:33.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 >> v2:172.21.15.97:3300/0 conn(0x7ff7380cdfb0 msgr2=0x7ff7380abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:33.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 --2- 172.21.15.31:0/2727523490 >> v2:172.21.15.97:3300/0 conn(0x7ff7380cdfb0 0x7ff7380abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 >> v1:172.21.15.97:6789/0 conn(0x7ff73815a710 legacy=0x7ff7380ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380b2cd0 con 0x7ff7380cbd50 2023-11-26T07:21:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7ff730001a00 con 0x7ff7380cbd50 2023-11-26T07:21:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 >> v1:172.21.15.31:6789/0 conn(0x7ff7380cbd50 legacy=0x7ff7380cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff724003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7240047d0 0x7ff724004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff724005190 0x7ff72400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380cca80 con 0x7ff724003980 2023-11-26T07:21:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72400c030 con 0x7ff7240047d0 2023-11-26T07:21:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72400c1f0 con 0x7ff724005190 2023-11-26T07:21:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73d57b640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff724003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73cd7a640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7240047d0 0x7ff724004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73dd7c640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff724005190 0x7ff72400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73d57b640 1 -- 172.21.15.31:0/2727523490 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff724005190 msgr2=0x7ff72400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73d57b640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff724005190 0x7ff72400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73d57b640 1 -- 172.21.15.31:0/2727523490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7240047d0 msgr2=0x7ff724004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73d57b640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7240047d0 0x7ff724004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73d57b640 1 -- 172.21.15.31:0/2727523490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff724003530 con 0x7ff724003980 2023-11-26T07:21:33.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.242+0000 7ff73dd7c640 1 -- 172.21.15.31:0/2727523490 reap_dead start 2023-11-26T07:21:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff73d57b640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff724003da0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff73400bd80 tx=0x7ff73401f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2712bea9e2fc84df server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340af620 con 0x7ff724003980 2023-11-26T07:21:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7340af780 con 0x7ff724003980 2023-11-26T07:21:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff72ffff640 1 -- 172.21.15.31:0/2727523490 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340afb30 con 0x7ff724003980 2023-11-26T07:21:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff73f7df640 1 -- 172.21.15.31:0/2727523490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 msgr2=0x7ff724003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff73f7df640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff724003da0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff73400bd80 tx=0x7ff73401f370 comp rx=0 tx=0).stop 2023-11-26T07:21:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff73f7df640 1 -- 172.21.15.31:0/2727523490 shutdown_connections 2023-11-26T07:21:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff73f7df640 1 --2- 172.21.15.31:0/2727523490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff724003da0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff73f7df640 1 -- 172.21.15.31:0/2727523490 >> 172.21.15.31:0/2727523490 conn(0x7ff7380b0cb0 msgr2=0x7ff7380b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:33.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff73f7df640 1 -- 172.21.15.31:0/2727523490 shutdown_connections 2023-11-26T07:21:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.246+0000 7ff73f7df640 1 -- 172.21.15.31:0/2727523490 wait complete. 2023-11-26T07:21:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73f7df640 1 Processor -- start 2023-11-26T07:21:33.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73f7df640 1 -- start start 2023-11-26T07:21:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73f7df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff73815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73f7df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7380710b0 0x7ff7380c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73f7df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff738071530 0x7ff7381527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73f7df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73809a750 con 0x7ff724003980 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73f7df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff738099ad0 con 0x7ff738071530 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73f7df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff738099db0 con 0x7ff7380710b0 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73d57b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff73815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73d57b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff73815ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58144/0 (socket says 172.21.15.31:58144) 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73d57b640 1 -- 172.21.15.31:0/261760350 learned_addr learned my addr 172.21.15.31:0/261760350 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73dd7c640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff738071530 0x7ff7381527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73cd7a640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7380710b0 0x7ff7380c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73d57b640 1 -- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7380710b0 msgr2=0x7ff7380c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73d57b640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7380710b0 0x7ff7380c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73d57b640 1 -- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff738071530 msgr2=0x7ff7381527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73d57b640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff738071530 0x7ff7381527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73d57b640 1 -- 172.21.15.31:0/261760350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73809a090 con 0x7ff724003980 2023-11-26T07:21:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73d57b640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff73815ae90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff73401f1a0 tx=0x7ff73401c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff72dffb640 1 -- 172.21.15.31:0/261760350 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340ba440 con 0x7ff724003980 2023-11-26T07:21:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff72dffb640 1 -- 172.21.15.31:0/261760350 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7340ba5c0 con 0x7ff724003980 2023-11-26T07:21:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff72dffb640 1 -- 172.21.15.31:0/261760350 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340ba970 con 0x7ff724003980 2023-11-26T07:21:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73f7df640 1 -- 172.21.15.31:0/261760350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73809a370 con 0x7ff724003980 2023-11-26T07:21:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.250+0000 7ff73f7df640 1 -- 172.21.15.31:0/261760350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7380c9280 con 0x7ff724003980 2023-11-26T07:21:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.254+0000 7ff73f7df640 1 -- 172.21.15.31:0/261760350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73809a9c0 con 0x7ff724003980 2023-11-26T07:21:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.254+0000 7ff72dffb640 1 -- 172.21.15.31:0/261760350 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7ff7340b6040 con 0x7ff724003980 2023-11-26T07:21:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.254+0000 7ff72dffb640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff71c0667d0 0x7ff71c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.254+0000 7ff72dffb640 1 -- 172.21.15.31:0/261760350 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff73411f150 con 0x7ff724003980 2023-11-26T07:21:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.254+0000 7ff73cd7a640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff71c0667d0 0x7ff71c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.258+0000 7ff73cd7a640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff71c0667d0 0x7ff71c068c90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff728007fd0 tx=0x7ff728009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.258+0000 7ff72dffb640 1 -- 172.21.15.31:0/261760350 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff7340ec1b0 con 0x7ff724003980 2023-11-26T07:21:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 -- 172.21.15.31:0/261760350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7380671f0 con 0x7ff724003980 2023-11-26T07:21:33.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff72dffb640 1 -- 172.21.15.31:0/261760350 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7340ef7d0 con 0x7ff724003980 2023-11-26T07:21:33.420 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:21:33.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 -- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff71c0667d0 msgr2=0x7ff71c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff71c0667d0 0x7ff71c068c90 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff728007fd0 tx=0x7ff728009040 comp rx=0 tx=0).stop 2023-11-26T07:21:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 -- 172.21.15.31:0/261760350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 msgr2=0x7ff73815ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff73815ae90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff73401f1a0 tx=0x7ff73401c110 comp rx=0 tx=0).stop 2023-11-26T07:21:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 -- 172.21.15.31:0/261760350 shutdown_connections 2023-11-26T07:21:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff71c0667d0 0x7ff71c068c90 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff738071530 0x7ff7381527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff7380710b0 0x7ff7380c8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 --2- 172.21.15.31:0/261760350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff724003980 0x7ff73815ae90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 -- 172.21.15.31:0/261760350 >> 172.21.15.31:0/261760350 conn(0x7ff7380b0cb0 msgr2=0x7ff73806cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 -- 172.21.15.31:0/261760350 shutdown_connections 2023-11-26T07:21:33.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.418+0000 7ff73f7df640 1 -- 172.21.15.31:0/261760350 wait complete. 2023-11-26T07:21:33.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:21:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcda6fe640 1 Processor -- start 2023-11-26T07:21:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcda6fe640 1 -- start start 2023-11-26T07:21:33.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcda6fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdcd40b2870 0x7fdcd40b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcda6fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcd406afc0 0x7fdcd40b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcda6fe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd4161290 con 0x7fdcd40b07a0 2023-11-26T07:21:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcda6fe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd40b0b80 con 0x7fdcd40b2870 2023-11-26T07:21:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcda6fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd415ebc0 con 0x7fdcd406afc0 2023-11-26T07:21:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcd37fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdcd40b2870 0x7fdcd40b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcd37fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdcd40b2870 0x7fdcd40b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58160/0 (socket says 172.21.15.31:58160) 2023-11-26T07:21:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcd37fe640 1 -- 172.21.15.31:0/109619623 learned_addr learned my addr 172.21.15.31:0/109619623 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcd8c9b640 1 --2- 172.21.15.31:0/109619623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcd406afc0 0x7fdcd40b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcd37fe640 1 -- 172.21.15.31:0/109619623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcd406afc0 msgr2=0x7fdcd40b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcd37fe640 1 --2- 172.21.15.31:0/109619623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcd406afc0 0x7fdcd40b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcd37fe640 1 -- 172.21.15.31:0/109619623 >> v1:172.21.15.97:6789/0 conn(0x7fdcd40b07a0 legacy=0x7fdcd41619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcd37fe640 1 -- 172.21.15.31:0/109619623 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd4161550 con 0x7fdcd40b2870 2023-11-26T07:21:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.586+0000 7fdcd2ffd640 1 -- 172.21.15.31:0/109619623 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432005867 0 0) 0x7fdcd4161290 con 0x7fdcd40b07a0 2023-11-26T07:21:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcd37fe640 1 --2- 172.21.15.31:0/109619623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd40b2870 0x7fdcd40b0ee0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009870 tx=0x7fdcc801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f69a34b831a2e61 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcd2ffd640 1 -- 172.21.15.31:0/109619623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80a7070 con 0x7fdcd40b2870 2023-11-26T07:21:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcd2ffd640 1 -- 172.21.15.31:0/109619623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc80ab430 con 0x7fdcd40b2870 2023-11-26T07:21:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcd2ffd640 1 -- 172.21.15.31:0/109619623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc80ab7c0 con 0x7fdcd40b2870 2023-11-26T07:21:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcda6fe640 1 -- 172.21.15.31:0/109619623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd40b2870 msgr2=0x7fdcd40b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcda6fe640 1 --2- 172.21.15.31:0/109619623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd40b2870 0x7fdcd40b0ee0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009870 tx=0x7fdcc801c540 comp rx=0 tx=0).stop 2023-11-26T07:21:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcda6fe640 1 -- 172.21.15.31:0/109619623 shutdown_connections 2023-11-26T07:21:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcda6fe640 1 --2- 172.21.15.31:0/109619623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcd406afc0 0x7fdcd40b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcda6fe640 1 --2- 172.21.15.31:0/109619623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd40b2870 0x7fdcd40b0ee0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcda6fe640 1 -- 172.21.15.31:0/109619623 >> 172.21.15.31:0/109619623 conn(0x7fdcd40b1d50 msgr2=0x7fdcd40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcda6fe640 1 -- 172.21.15.31:0/109619623 shutdown_connections 2023-11-26T07:21:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.590+0000 7fdcda6fe640 1 -- 172.21.15.31:0/109619623 wait complete. 2023-11-26T07:21:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcda6fe640 1 Processor -- start 2023-11-26T07:21:33.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcda6fe640 1 -- start start 2023-11-26T07:21:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcda6fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd406afc0 0x7fdcd4155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcda6fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd40b07a0 0x7fdcd40af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcda6fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcd40b2870 0x7fdcd41550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcda6fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd41561c0 con 0x7fdcd406afc0 2023-11-26T07:21:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcda6fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd415a620 con 0x7fdcd40b07a0 2023-11-26T07:21:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcda6fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd4156f70 con 0x7fdcd40b2870 2023-11-26T07:21:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd406afc0 0x7fdcd4155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd406afc0 0x7fdcd4155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58164/0 (socket says 172.21.15.31:58164) 2023-11-26T07:21:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd3fff640 1 -- 172.21.15.31:0/1670283714 learned_addr learned my addr 172.21.15.31:0/1670283714 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd3fff640 1 -- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcd40b2870 msgr2=0x7fdcd41550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd3fff640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcd40b2870 0x7fdcd41550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd37fe640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd40b07a0 0x7fdcd40af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd3fff640 1 -- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd40b07a0 msgr2=0x7fdcd40af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd3fff640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd40b07a0 0x7fdcd40af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd3fff640 1 -- 172.21.15.31:0/1670283714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc80094d0 con 0x7fdcd406afc0 2023-11-26T07:21:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd37fe640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd40b07a0 0x7fdcd40af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:33.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd3fff640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd406afc0 0x7fdcd4155c80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400c480 tx=0x7fdcc400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd0ff9640 1 -- 172.21.15.31:0/1670283714 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4005480 con 0x7fdcd406afc0 2023-11-26T07:21:33.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd0ff9640 1 -- 172.21.15.31:0/1670283714 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc4003600 con 0x7fdcd406afc0 2023-11-26T07:21:33.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcd0ff9640 1 -- 172.21.15.31:0/1670283714 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40039b0 con 0x7fdcd406afc0 2023-11-26T07:21:33.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcda6fe640 1 -- 172.21.15.31:0/1670283714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd41563f0 con 0x7fdcd406afc0 2023-11-26T07:21:33.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.594+0000 7fdcda6fe640 1 -- 172.21.15.31:0/1670283714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd41569b0 con 0x7fdcd406afc0 2023-11-26T07:21:33.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.598+0000 7fdcda6fe640 1 -- 172.21.15.31:0/1670283714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd40ad030 con 0x7fdcd406afc0 2023-11-26T07:21:33.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.598+0000 7fdcd0ff9640 1 -- 172.21.15.31:0/1670283714 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fdcc4003b10 con 0x7fdcd406afc0 2023-11-26T07:21:33.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.598+0000 7fdcd0ff9640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcc00667d0 0x7fdcc0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.598+0000 7fdcd0ff9640 1 -- 172.21.15.31:0/1670283714 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcc40839a0 con 0x7fdcd406afc0 2023-11-26T07:21:33.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.598+0000 7fdcd37fe640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcc00667d0 0x7fdcc0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.598+0000 7fdcd37fe640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcc00667d0 0x7fdcc0068c90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80af000 tx=0x7fdcc80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.602+0000 7fdcd0ff9640 1 -- 172.21.15.31:0/1670283714 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdcd40ad030 con 0x7fdcd406afc0 2023-11-26T07:21:33.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 -- 172.21.15.31:0/1670283714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcd40ad030 con 0x7fdcd406afc0 2023-11-26T07:21:33.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcd0ff9640 1 -- 172.21.15.31:0/1670283714 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdcd40ad030 con 0x7fdcd406afc0 2023-11-26T07:21:33.764 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:21:33.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 -- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcc00667d0 msgr2=0x7fdcc0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcc00667d0 0x7fdcc0068c90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80af000 tx=0x7fdcc80a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:33.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 -- 172.21.15.31:0/1670283714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd406afc0 msgr2=0x7fdcd4155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd406afc0 0x7fdcd4155c80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400c480 tx=0x7fdcc400c940 comp rx=0 tx=0).stop 2023-11-26T07:21:33.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 -- 172.21.15.31:0/1670283714 shutdown_connections 2023-11-26T07:21:33.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdcc00667d0 0x7fdcc0068c90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdcd40b2870 0x7fdcd41550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdcd40b07a0 0x7fdcd40af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 --2- 172.21.15.31:0/1670283714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdcd406afc0 0x7fdcd4155c80 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 -- 172.21.15.31:0/1670283714 >> 172.21.15.31:0/1670283714 conn(0x7fdcd40b1d50 msgr2=0x7fdcd40c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:33.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 -- 172.21.15.31:0/1670283714 shutdown_connections 2023-11-26T07:21:33.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.762+0000 7fdcda6fe640 1 -- 172.21.15.31:0/1670283714 wait complete. 2023-11-26T07:21:33.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:21:33.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d5966b640 1 Processor -- start 2023-11-26T07:21:33.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d5966b640 1 -- start start 2023-11-26T07:21:33.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d5966b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7d540cdfb0 0x7f7d540abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d5966b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5415a710 0x7f7d540ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d5966b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d540ce8d0 con 0x7f7d540cbd50 2023-11-26T07:21:33.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d5966b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d540cca80 con 0x7f7d540cdfb0 2023-11-26T07:21:33.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d5966b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d540b2710 con 0x7f7d5415a710 2023-11-26T07:21:33.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d52ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7d540cdfb0 0x7f7d540abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d52ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7d540cdfb0 0x7f7d540abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37684/0 (socket says 172.21.15.31:37684) 2023-11-26T07:21:33.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d52ffd640 1 -- 172.21.15.31:0/2688856659 learned_addr learned my addr 172.21.15.31:0/2688856659 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:33.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d527fc640 1 --2- 172.21.15.31:0/2688856659 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5415a710 0x7f7d540ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d52ffd640 1 -- 172.21.15.31:0/2688856659 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5415a710 msgr2=0x7f7d540ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:33.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d52ffd640 1 --2- 172.21.15.31:0/2688856659 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5415a710 0x7f7d540ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d52ffd640 1 -- 172.21.15.31:0/2688856659 >> v1:172.21.15.97:6789/0 conn(0x7f7d540cbd50 legacy=0x7f7d540cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.930+0000 7f7d52ffd640 1 -- 172.21.15.31:0/2688856659 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d540b29f0 con 0x7f7d540cdfb0 2023-11-26T07:21:33.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d52ffd640 1 --2- 172.21.15.31:0/2688856659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d540cdfb0 0x7f7d540abf90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400bb10 tx=0x7f7d4401ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4decfae84ae6d73d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d51ffb640 1 -- 172.21.15.31:0/2688856659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440a7cb0 con 0x7f7d540cdfb0 2023-11-26T07:21:33.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d51ffb640 1 -- 172.21.15.31:0/2688856659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d440a7e10 con 0x7f7d540cdfb0 2023-11-26T07:21:33.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d51ffb640 1 -- 172.21.15.31:0/2688856659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440af660 con 0x7f7d540cdfb0 2023-11-26T07:21:33.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d5966b640 1 -- 172.21.15.31:0/2688856659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d540cdfb0 msgr2=0x7f7d540abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d5966b640 1 --2- 172.21.15.31:0/2688856659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d540cdfb0 0x7f7d540abf90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400bb10 tx=0x7f7d4401ebe0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d5966b640 1 -- 172.21.15.31:0/2688856659 shutdown_connections 2023-11-26T07:21:33.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d5966b640 1 --2- 172.21.15.31:0/2688856659 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d5415a710 0x7f7d540ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d5966b640 1 --2- 172.21.15.31:0/2688856659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d540cdfb0 0x7f7d540abf90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d5966b640 1 -- 172.21.15.31:0/2688856659 >> 172.21.15.31:0/2688856659 conn(0x7f7d540b0cb0 msgr2=0x7f7d540b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:33.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d5966b640 1 -- 172.21.15.31:0/2688856659 shutdown_connections 2023-11-26T07:21:33.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.934+0000 7f7d5966b640 1 -- 172.21.15.31:0/2688856659 wait complete. 2023-11-26T07:21:33.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d5966b640 1 Processor -- start 2023-11-26T07:21:33.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d5966b640 1 -- start start 2023-11-26T07:21:33.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d5966b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d540cbd50 0x7f7d54153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d5966b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d540cdfb0 0x7f7d541f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d5966b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5415a710 0x7f7d5415c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d5966b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d54158550 con 0x7f7d5415a710 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d5966b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d541578d0 con 0x7f7d540cbd50 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d5966b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d54157bb0 con 0x7f7d540cdfb0 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d537fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5415a710 0x7f7d5415c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d537fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5415a710 0x7f7d5415c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37686/0 (socket says 172.21.15.31:37686) 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d537fe640 1 -- 172.21.15.31:0/1296870473 learned_addr learned my addr 172.21.15.31:0/1296870473 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d52ffd640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d540cbd50 0x7f7d54153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d527fc640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d540cdfb0 0x7f7d541f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d537fe640 1 -- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d540cdfb0 msgr2=0x7f7d541f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d537fe640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d540cdfb0 0x7f7d541f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d537fe640 1 -- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d540cbd50 msgr2=0x7f7d54153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:33.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d537fe640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d540cbd50 0x7f7d54153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:33.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d537fe640 1 -- 172.21.15.31:0/1296870473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4400b770 con 0x7f7d5415a710 2023-11-26T07:21:33.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d52ffd640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d540cbd50 0x7f7d54153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:33.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d537fe640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5415a710 0x7f7d5415c750 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00c770 tx=0x7f7d4c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d3b7fe640 1 -- 172.21.15.31:0/1296870473 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c003b80 con 0x7f7d5415a710 2023-11-26T07:21:33.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d3b7fe640 1 -- 172.21.15.31:0/1296870473 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c003ce0 con 0x7f7d5415a710 2023-11-26T07:21:33.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d3b7fe640 1 -- 172.21.15.31:0/1296870473 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00aa90 con 0x7f7d5415a710 2023-11-26T07:21:33.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d5966b640 1 -- 172.21.15.31:0/1296870473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d54157e90 con 0x7f7d5415a710 2023-11-26T07:21:33.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.938+0000 7f7d5966b640 1 -- 172.21.15.31:0/1296870473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d541f5ba0 con 0x7f7d5415a710 2023-11-26T07:21:33.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.942+0000 7f7d5966b640 1 -- 172.21.15.31:0/1296870473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d540671f0 con 0x7f7d5415a710 2023-11-26T07:21:33.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.942+0000 7f7d3b7fe640 1 -- 172.21.15.31:0/1296870473 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f7d4c00abf0 con 0x7f7d5415a710 2023-11-26T07:21:33.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.942+0000 7f7d3b7fe640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7d400667d0 0x7f7d40068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:33.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.942+0000 7f7d3b7fe640 1 -- 172.21.15.31:0/1296870473 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d4c084290 con 0x7f7d5415a710 2023-11-26T07:21:33.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.942+0000 7f7d52ffd640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7d400667d0 0x7f7d40068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:33.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.942+0000 7f7d52ffd640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7d400667d0 0x7f7d40068c90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d440b1000 tx=0x7f7d44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:33.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:33.946+0000 7f7d3b7fe640 1 -- 172.21.15.31:0/1296870473 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7d4c051290 con 0x7f7d5415a710 2023-11-26T07:21:34.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.102+0000 7f7d5966b640 1 -- 172.21.15.31:0/1296870473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d5406b460 con 0x7f7d5415a710 2023-11-26T07:21:34.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.106+0000 7f7d3b7fe640 1 -- 172.21.15.31:0/1296870473 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d4c0548b0 con 0x7f7d5415a710 2023-11-26T07:21:34.109 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:21:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.106+0000 7f7d5966b640 1 -- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7d400667d0 msgr2=0x7f7d40068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.106+0000 7f7d5966b640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7d400667d0 0x7f7d40068c90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d440b1000 tx=0x7f7d44008040 comp rx=0 tx=0).stop 2023-11-26T07:21:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.106+0000 7f7d5966b640 1 -- 172.21.15.31:0/1296870473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5415a710 msgr2=0x7f7d5415c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.106+0000 7f7d5966b640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5415a710 0x7f7d5415c750 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00c770 tx=0x7f7d4c00cc30 comp rx=0 tx=0).stop 2023-11-26T07:21:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.110+0000 7f7d5966b640 1 -- 172.21.15.31:0/1296870473 shutdown_connections 2023-11-26T07:21:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.110+0000 7f7d5966b640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f7d400667d0 0x7f7d40068c90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.110+0000 7f7d5966b640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d5415a710 0x7f7d5415c750 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.110+0000 7f7d5966b640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7d540cdfb0 0x7f7d541f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.110+0000 7f7d5966b640 1 --2- 172.21.15.31:0/1296870473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d540cbd50 0x7f7d54153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.110+0000 7f7d5966b640 1 -- 172.21.15.31:0/1296870473 >> 172.21.15.31:0/1296870473 conn(0x7f7d540b0cb0 msgr2=0x7f7d540ad3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.110+0000 7f7d5966b640 1 -- 172.21.15.31:0/1296870473 shutdown_connections 2023-11-26T07:21:34.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.110+0000 7f7d5966b640 1 -- 172.21.15.31:0/1296870473 wait complete. 2023-11-26T07:21:34.131 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:34.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.270+0000 7f560b3b1640 1 Processor -- start 2023-11-26T07:21:34.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.270+0000 7f560b3b1640 1 -- start start 2023-11-26T07:21:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f560b3b1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f56040d04b0 0x7f56040b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f560b3b1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56040d0890 0x7f56040b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f560b3b1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56040cf1c0 con 0x7f560406d3f0 2023-11-26T07:21:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f560b3b1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560406c700 con 0x7f56040d04b0 2023-11-26T07:21:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f560b3b1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560415e930 con 0x7f56040d0890 2023-11-26T07:21:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f560abb0640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f560406d3f0 0x7f560415da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55152/0 (socket says 172.21.15.31:55152) 2023-11-26T07:21:34.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f560abb0640 1 -- 172.21.15.31:0/1355889232 learned_addr learned my addr 172.21.15.31:0/1355889232 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:34.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f5609bae640 1 --2- 172.21.15.31:0/1355889232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56040d0890 0x7f56040b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f560a3af640 1 --2- 172.21.15.31:0/1355889232 >> v2:172.21.15.97:3300/0 conn(0x7f56040d04b0 0x7f56040b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f56093ad640 1 -- 172.21.15.31:0/1355889232 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886537648 0 0) 0x7f56040cf1c0 con 0x7f560406d3f0 2023-11-26T07:21:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f56093ad640 1 -- 172.21.15.31:0/1355889232 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55f0003530 con 0x7f560406d3f0 2023-11-26T07:21:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f5609bae640 1 -- 172.21.15.31:0/1355889232 >> v2:172.21.15.97:3300/0 conn(0x7f56040d04b0 msgr2=0x7f56040b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f5609bae640 1 --2- 172.21.15.31:0/1355889232 >> v2:172.21.15.97:3300/0 conn(0x7f56040d04b0 0x7f56040b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f5609bae640 1 -- 172.21.15.31:0/1355889232 >> v1:172.21.15.31:6789/0 conn(0x7f560406d3f0 legacy=0x7f560415da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.274+0000 7f5609bae640 1 -- 172.21.15.31:0/1355889232 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560406cf60 con 0x7f56040d0890 2023-11-26T07:21:34.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f5609bae640 1 --2- 172.21.15.31:0/1355889232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56040d0890 0x7f56040b36e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f55f40097c0 tx=0x7f55f401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb6160c65eaf5075 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f56093ad640 1 -- 172.21.15.31:0/1355889232 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f40ac620 con 0x7f56040d0890 2023-11-26T07:21:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f56093ad640 1 -- 172.21.15.31:0/1355889232 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f40ac780 con 0x7f56040d0890 2023-11-26T07:21:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f56093ad640 1 -- 172.21.15.31:0/1355889232 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f40acb10 con 0x7f56040d0890 2023-11-26T07:21:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 -- 172.21.15.31:0/1355889232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56040d0890 msgr2=0x7f56040b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 --2- 172.21.15.31:0/1355889232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56040d0890 0x7f56040b36e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f55f40097c0 tx=0x7f55f401c3c0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 -- 172.21.15.31:0/1355889232 shutdown_connections 2023-11-26T07:21:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 --2- 172.21.15.31:0/1355889232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56040d0890 0x7f56040b36e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 --2- 172.21.15.31:0/1355889232 >> v2:172.21.15.97:3300/0 conn(0x7f56040d04b0 0x7f56040b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 -- 172.21.15.31:0/1355889232 >> 172.21.15.31:0/1355889232 conn(0x7f56040666a0 msgr2=0x7f5604066ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 -- 172.21.15.31:0/1355889232 shutdown_connections 2023-11-26T07:21:34.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 -- 172.21.15.31:0/1355889232 wait complete. 2023-11-26T07:21:34.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 Processor -- start 2023-11-26T07:21:34.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 -- start start 2023-11-26T07:21:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f560406d3f0 0x7f56041569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56040d04b0 0x7f56040cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56040d0890 0x7f5604157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56040cb370 con 0x7f56040d04b0 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56040cd840 con 0x7f56040d0890 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.278+0000 7f560b3b1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56040ccc30 con 0x7f560406d3f0 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f5609bae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56040d04b0 0x7f56040cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f5609bae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56040d04b0 0x7f56040cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37698/0 (socket says 172.21.15.31:37698) 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f5609bae640 1 -- 172.21.15.31:0/3210328102 learned_addr learned my addr 172.21.15.31:0/3210328102 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f560a3af640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f560406d3f0 0x7f56041569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f560abb0640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56040d0890 0x7f5604157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f5609bae640 1 -- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f560406d3f0 msgr2=0x7f56041569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f5609bae640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f560406d3f0 0x7f56041569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f5609bae640 1 -- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56040d0890 msgr2=0x7f5604157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f5609bae640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56040d0890 0x7f5604157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f5609bae640 1 -- 172.21.15.31:0/3210328102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f4009420 con 0x7f56040d04b0 2023-11-26T07:21:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f560abb0640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56040d0890 0x7f5604157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f5609bae640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56040d04b0 0x7f56040cdcd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f55f401aa70 tx=0x7f55f4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f55faffd640 1 -- 172.21.15.31:0/3210328102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f40ac620 con 0x7f56040d04b0 2023-11-26T07:21:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f55faffd640 1 -- 172.21.15.31:0/3210328102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f40b5400 con 0x7f56040d04b0 2023-11-26T07:21:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f560b3b1640 1 -- 172.21.15.31:0/3210328102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56040ccf10 con 0x7f56040d04b0 2023-11-26T07:21:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f55faffd640 1 -- 172.21.15.31:0/3210328102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f40b57d0 con 0x7f56040d04b0 2023-11-26T07:21:34.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f560b3b1640 1 -- 172.21.15.31:0/3210328102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56040cd4d0 con 0x7f56040d04b0 2023-11-26T07:21:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f560b3b1640 1 -- 172.21.15.31:0/3210328102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56040cd9d0 con 0x7f56040d04b0 2023-11-26T07:21:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f55faffd640 1 -- 172.21.15.31:0/3210328102 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f55f40b3040 con 0x7f56040d04b0 2023-11-26T07:21:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.282+0000 7f55faffd640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f55e80667d0 0x7f55e8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.286+0000 7f55faffd640 1 -- 172.21.15.31:0/3210328102 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55f411e4a0 con 0x7f56040d04b0 2023-11-26T07:21:34.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.286+0000 7f560a3af640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f55e80667d0 0x7f55e8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.286+0000 7f560a3af640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f55e80667d0 0x7f55e8068c90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5600009af0 tx=0x7f5600008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:34.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.290+0000 7f55faffd640 1 -- 172.21.15.31:0/3210328102 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f55f40eb4e0 con 0x7f56040d04b0 2023-11-26T07:21:34.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.478+0000 7f560b3b1640 1 -- 172.21.15.31:0/3210328102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5604074a00 con 0x7f56040d04b0 2023-11-26T07:21:34.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f55faffd640 1 -- 172.21.15.31:0/3210328102 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f55f40eeb00 con 0x7f56040d04b0 2023-11-26T07:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:34.483 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3429205589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3429205589},{"type":"v1","addr":"172.21.15.97:6836","nonce":3429205589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5959,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3569564413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3569564413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3569564413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5965,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690524463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690524463},{"type":"v1","addr":"172.21.15.97:6837","nonce":690524463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:02.722567+0000","modified":"2023-11-26T07:21:05.757700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1053315203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1053315203},{"type":"v1","addr":"172.21.15.31:6834","nonce":1053315203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:21:34.483 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2023-11-26T07:21:34.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 -- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f55e80667d0 msgr2=0x7f55e8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f55e80667d0 0x7f55e8068c90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5600009af0 tx=0x7f5600008040 comp rx=0 tx=0).stop 2023-11-26T07:21:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 -- 172.21.15.31:0/3210328102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56040d04b0 msgr2=0x7f56040cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56040d04b0 0x7f56040cdcd0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f55f401aa70 tx=0x7f55f4019160 comp rx=0 tx=0).stop 2023-11-26T07:21:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 -- 172.21.15.31:0/3210328102 shutdown_connections 2023-11-26T07:21:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f55e80667d0 0x7f55e8068c90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56040d0890 0x7f5604157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56040d04b0 0x7f56040cdcd0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 --2- 172.21.15.31:0/3210328102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f560406d3f0 0x7f56041569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 -- 172.21.15.31:0/3210328102 >> 172.21.15.31:0/3210328102 conn(0x7f56040666a0 msgr2=0x7f56040b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:34.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 -- 172.21.15.31:0/3210328102 shutdown_connections 2023-11-26T07:21:34.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.482+0000 7f560b3b1640 1 -- 172.21.15.31:0/3210328102 wait complete. 2023-11-26T07:21:34.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:34.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.642+0000 7f6612816640 1 Processor -- start 2023-11-26T07:21:34.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6612816640 1 -- start start 2023-11-26T07:21:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6612816640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f660c143ae0 0x7f660c0b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6612816640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f660c0adcc0 0x7f660c0ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6612816640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660c0b2480 con 0x7f660c06dfd0 2023-11-26T07:21:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6612816640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c070670 con 0x7f660c0adcc0 2023-11-26T07:21:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6612816640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c070950 con 0x7f660c143ae0 2023-11-26T07:21:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6610db3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f660c0adcc0 0x7f660c0ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6610db3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f660c0adcc0 0x7f660c0ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37710/0 (socket says 172.21.15.31:37710) 2023-11-26T07:21:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6610db3640 1 -- 172.21.15.31:0/3047254582 learned_addr learned my addr 172.21.15.31:0/3047254582 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:34.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f660b7fe640 1 --1- 172.21.15.31:0/3047254582 >> v1:172.21.15.31:6789/0 conn(0x7f660c06dfd0 0x7f660c06e410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55168/0 (socket says 172.21.15.31:55168) 2023-11-26T07:21:34.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f660bfff640 1 --2- 172.21.15.31:0/3047254582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f660c143ae0 0x7f660c0b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6610db3640 1 -- 172.21.15.31:0/3047254582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f660c143ae0 msgr2=0x7f660c0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:34.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6610db3640 1 --2- 172.21.15.31:0/3047254582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f660c143ae0 0x7f660c0b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6610db3640 1 -- 172.21.15.31:0/3047254582 >> v1:172.21.15.31:6789/0 conn(0x7f660c06dfd0 legacy=0x7f660c06e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f6610db3640 1 -- 172.21.15.31:0/3047254582 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c070c30 con 0x7f660c0adcc0 2023-11-26T07:21:34.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.646+0000 7f660affd640 1 -- 172.21.15.31:0/3047254582 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079557602 0 0) 0x7f660c0b2480 con 0x7f660c06dfd0 2023-11-26T07:21:34.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f6610db3640 1 --2- 172.21.15.31:0/3047254582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c0adcc0 0x7f660c0ae110 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6604009e50 tx=0x7f660401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0755f89b2119a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:34.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f660affd640 1 -- 172.21.15.31:0/3047254582 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040a58a0 con 0x7f660c0adcc0 2023-11-26T07:21:34.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f660affd640 1 -- 172.21.15.31:0/3047254582 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66040a5a00 con 0x7f660c0adcc0 2023-11-26T07:21:34.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f660affd640 1 -- 172.21.15.31:0/3047254582 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040a5d90 con 0x7f660c0adcc0 2023-11-26T07:21:34.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f6612816640 1 -- 172.21.15.31:0/3047254582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c0adcc0 msgr2=0x7f660c0ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f6612816640 1 --2- 172.21.15.31:0/3047254582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c0adcc0 0x7f660c0ae110 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6604009e50 tx=0x7f660401c9e0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f6612816640 1 -- 172.21.15.31:0/3047254582 shutdown_connections 2023-11-26T07:21:34.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f6612816640 1 --2- 172.21.15.31:0/3047254582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c0adcc0 0x7f660c0ae110 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f6612816640 1 --2- 172.21.15.31:0/3047254582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f660c143ae0 0x7f660c0b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f6612816640 1 -- 172.21.15.31:0/3047254582 >> 172.21.15.31:0/3047254582 conn(0x7f660c0b0c40 msgr2=0x7f660c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f6612816640 1 -- 172.21.15.31:0/3047254582 shutdown_connections 2023-11-26T07:21:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f6612816640 1 -- 172.21.15.31:0/3047254582 wait complete. 2023-11-26T07:21:34.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.650+0000 7f6612816640 1 Processor -- start 2023-11-26T07:21:34.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6612816640 1 -- start start 2023-11-26T07:21:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6612816640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f660c06dfd0 0x7f660c09c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6612816640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f660c0adcc0 0x7f660c149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6612816640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c143ae0 0x7f660c149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6612816640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c13dc60 con 0x7f660c143ae0 2023-11-26T07:21:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6612816640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c13d6a0 con 0x7f660c0adcc0 2023-11-26T07:21:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6612816640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c13d980 con 0x7f660c06dfd0 2023-11-26T07:21:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6610db3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c143ae0 0x7f660c149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6610db3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c143ae0 0x7f660c149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37724/0 (socket says 172.21.15.31:37724) 2023-11-26T07:21:34.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6610db3640 1 -- 172.21.15.31:0/1037824878 learned_addr learned my addr 172.21.15.31:0/1037824878 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f660bfff640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f660c06dfd0 0x7f660c09c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f660b7fe640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f660c0adcc0 0x7f660c149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6610db3640 1 -- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f660c06dfd0 msgr2=0x7f660c09c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6610db3640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f660c06dfd0 0x7f660c09c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6610db3640 1 -- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f660c0adcc0 msgr2=0x7f660c149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6610db3640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f660c0adcc0 0x7f660c149680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6610db3640 1 -- 172.21.15.31:0/1037824878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6604009b30 con 0x7f660c143ae0 2023-11-26T07:21:34.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f660b7fe640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f660c0adcc0 0x7f660c149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:34.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6610db3640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c143ae0 0x7f660c149bc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f66040af040 tx=0x7f6604019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:34.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6608ff9640 1 -- 172.21.15.31:0/1037824878 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040b8670 con 0x7f660c143ae0 2023-11-26T07:21:34.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6608ff9640 1 -- 172.21.15.31:0/1037824878 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66040b87d0 con 0x7f660c143ae0 2023-11-26T07:21:34.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6612816640 1 -- 172.21.15.31:0/1037824878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c147780 con 0x7f660c143ae0 2023-11-26T07:21:34.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6608ff9640 1 -- 172.21.15.31:0/1037824878 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66040b8b80 con 0x7f660c143ae0 2023-11-26T07:21:34.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6612816640 1 -- 172.21.15.31:0/1037824878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c09cbb0 con 0x7f660c143ae0 2023-11-26T07:21:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.654+0000 7f6612816640 1 -- 172.21.15.31:0/1037824878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c04e850 con 0x7f660c143ae0 2023-11-26T07:21:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.658+0000 7f6608ff9640 1 -- 172.21.15.31:0/1037824878 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f66040b8ce0 con 0x7f660c143ae0 2023-11-26T07:21:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.658+0000 7f6608ff9640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65f80667d0 0x7f65f8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.658+0000 7f6608ff9640 1 -- 172.21.15.31:0/1037824878 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f660411fa20 con 0x7f660c143ae0 2023-11-26T07:21:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.658+0000 7f660bfff640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65f80667d0 0x7f65f8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.658+0000 7f660bfff640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65f80667d0 0x7f65f8068c90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f660c09c3e0 tx=0x7f65fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:34.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.662+0000 7f6608ff9640 1 -- 172.21.15.31:0/1037824878 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f66040ecc00 con 0x7f660c143ae0 2023-11-26T07:21:34.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.818+0000 7f6612816640 1 -- 172.21.15.31:0/1037824878 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f660c147c30 con 0x7f660c143ae0 2023-11-26T07:21:34.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6608ff9640 1 -- 172.21.15.31:0/1037824878 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f66040f0220 con 0x7f660c143ae0 2023-11-26T07:21:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:34.824 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:04.352334+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:01.142504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:01.723895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:34.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 -- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65f80667d0 msgr2=0x7f65f8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65f80667d0 0x7f65f8068c90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f660c09c3e0 tx=0x7f65fc008040 comp rx=0 tx=0).stop 2023-11-26T07:21:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 -- 172.21.15.31:0/1037824878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c143ae0 msgr2=0x7f660c149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c143ae0 0x7f660c149bc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f66040af040 tx=0x7f6604019780 comp rx=0 tx=0).stop 2023-11-26T07:21:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 -- 172.21.15.31:0/1037824878 shutdown_connections 2023-11-26T07:21:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f65f80667d0 0x7f65f8068c90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f660c143ae0 0x7f660c149bc0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f660c0adcc0 0x7f660c149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 --2- 172.21.15.31:0/1037824878 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f660c06dfd0 0x7f660c09c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 -- 172.21.15.31:0/1037824878 >> 172.21.15.31:0/1037824878 conn(0x7f660c0b0c40 msgr2=0x7f660c0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 -- 172.21.15.31:0/1037824878 shutdown_connections 2023-11-26T07:21:34.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.822+0000 7f6612816640 1 -- 172.21.15.31:0/1037824878 wait complete. 2023-11-26T07:21:34.845 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:21:34.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:34.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.986+0000 7f9373a54640 1 Processor -- start 2023-11-26T07:21:34.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.986+0000 7f9373a54640 1 -- start start 2023-11-26T07:21:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.986+0000 7f9373a54640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f936c15f540 0x7f936c0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.986+0000 7f9373a54640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f936c0b1f40 0x7f936c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.986+0000 7f9373a54640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936c0b2920 con 0x7f936c0b23f0 2023-11-26T07:21:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.986+0000 7f9373a54640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c0b2c00 con 0x7f936c15f540 2023-11-26T07:21:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.986+0000 7f9373a54640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c0b2ee0 con 0x7f936c0b1f40 2023-11-26T07:21:34.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.986+0000 7f93717f0640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f936c15f540 0x7f936c0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.986+0000 7f9371ff1640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f936c0b23f0 0x7f936c070880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55172/0 (socket says 172.21.15.31:55172) 2023-11-26T07:21:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.986+0000 7f9371ff1640 1 -- 172.21.15.31:0/3796315014 learned_addr learned my addr 172.21.15.31:0/3796315014 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f9370fef640 1 --2- 172.21.15.31:0/3796315014 >> v2:172.21.15.97:3300/0 conn(0x7f936c0b1f40 0x7f936c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f93717f0640 1 -- 172.21.15.31:0/3796315014 >> v2:172.21.15.97:3300/0 conn(0x7f936c0b1f40 msgr2=0x7f936c06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f93717f0640 1 --2- 172.21.15.31:0/3796315014 >> v2:172.21.15.97:3300/0 conn(0x7f936c0b1f40 0x7f936c06c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f93717f0640 1 -- 172.21.15.31:0/3796315014 >> v1:172.21.15.31:6789/0 conn(0x7f936c0b23f0 legacy=0x7f936c070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f9363fff640 1 -- 172.21.15.31:0/3796315014 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248443174 0 0) 0x7f936c0b2920 con 0x7f936c0b23f0 2023-11-26T07:21:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f93717f0640 1 -- 172.21.15.31:0/3796315014 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c0b31c0 con 0x7f936c15f540 2023-11-26T07:21:34.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f9370fef640 1 --2- 172.21.15.31:0/3796315014 >> v2:172.21.15.97:3300/0 conn(0x7f936c0b1f40 0x7f936c06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f93717f0640 1 --2- 172.21.15.31:0/3796315014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c15f540 0x7f936c0b39c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f936c06eb50 tx=0x7f936c0717a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1de6bbd9b742c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f9363fff640 1 -- 172.21.15.31:0/3796315014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93680ab600 con 0x7f936c15f540 2023-11-26T07:21:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f9363fff640 1 -- 172.21.15.31:0/3796315014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93680a7050 con 0x7f936c15f540 2023-11-26T07:21:34.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f9363fff640 1 -- 172.21.15.31:0/3796315014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93680ab980 con 0x7f936c15f540 2023-11-26T07:21:34.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f9373a54640 1 -- 172.21.15.31:0/3796315014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c15f540 msgr2=0x7f936c0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:34.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.990+0000 7f9373a54640 1 --2- 172.21.15.31:0/3796315014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c15f540 0x7f936c0b39c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f936c06eb50 tx=0x7f936c0717a0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 -- 172.21.15.31:0/3796315014 shutdown_connections 2023-11-26T07:21:34.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 --2- 172.21.15.31:0/3796315014 >> v2:172.21.15.97:3300/0 conn(0x7f936c0b1f40 0x7f936c06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 --2- 172.21.15.31:0/3796315014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c15f540 0x7f936c0b39c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:34.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 -- 172.21.15.31:0/3796315014 >> 172.21.15.31:0/3796315014 conn(0x7f936c0b0c40 msgr2=0x7f936c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:34.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 -- 172.21.15.31:0/3796315014 shutdown_connections 2023-11-26T07:21:34.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 -- 172.21.15.31:0/3796315014 wait complete. 2023-11-26T07:21:35.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 Processor -- start 2023-11-26T07:21:35.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 -- start start 2023-11-26T07:21:35.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b1f40 0x7f936c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f936c0b23f0 0x7f936c154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936c15f540 0x7f936c1554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c15ec80 con 0x7f936c0b1f40 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0ad8e0 con 0x7f936c15f540 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9373a54640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0adbc0 con 0x7f936c0b23f0 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f93717f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b1f40 0x7f936c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f93717f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b1f40 0x7f936c15df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37746/0 (socket says 172.21.15.31:37746) 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f93717f0640 1 -- 172.21.15.31:0/86964736 learned_addr learned my addr 172.21.15.31:0/86964736 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9370fef640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f936c0b23f0 0x7f936c154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9371ff1640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936c15f540 0x7f936c1554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f93717f0640 1 -- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f936c0b23f0 msgr2=0x7f936c154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f93717f0640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f936c0b23f0 0x7f936c154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f93717f0640 1 -- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936c15f540 msgr2=0x7f936c1554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f93717f0640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936c15f540 0x7f936c1554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f93717f0640 1 -- 172.21.15.31:0/86964736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368009420 con 0x7f936c0b1f40 2023-11-26T07:21:35.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9371ff1640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936c15f540 0x7f936c1554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:35.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f93717f0640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b1f40 0x7f936c15df30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f936801a060 tx=0x7f93680191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:35.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.994+0000 7f9361ffb640 1 -- 172.21.15.31:0/86964736 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93680b4600 con 0x7f936c0b1f40 2023-11-26T07:21:35.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.998+0000 7f9361ffb640 1 -- 172.21.15.31:0/86964736 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93680b4760 con 0x7f936c0b1f40 2023-11-26T07:21:35.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.998+0000 7f9361ffb640 1 -- 172.21.15.31:0/86964736 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93680b4b10 con 0x7f936c0b1f40 2023-11-26T07:21:35.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.998+0000 7f9373a54640 1 -- 172.21.15.31:0/86964736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936c0adea0 con 0x7f936c0b1f40 2023-11-26T07:21:35.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.998+0000 7f9373a54640 1 -- 172.21.15.31:0/86964736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c15e510 con 0x7f936c0b1f40 2023-11-26T07:21:35.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.998+0000 7f9373a54640 1 -- 172.21.15.31:0/86964736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c04e850 con 0x7f936c0b1f40 2023-11-26T07:21:35.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.998+0000 7f9361ffb640 1 -- 172.21.15.31:0/86964736 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f93680b2040 con 0x7f936c0b1f40 2023-11-26T07:21:35.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.998+0000 7f9361ffb640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93500667d0 0x7f9350068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.998+0000 7f9361ffb640 1 -- 172.21.15.31:0/86964736 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f936811e520 con 0x7f936c0b1f40 2023-11-26T07:21:35.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:34.998+0000 7f9370fef640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93500667d0 0x7f9350068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.002+0000 7f9370fef640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93500667d0 0x7f9350068c90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f935c006fd0 tx=0x7f935c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:35.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.002+0000 7f9361ffb640 1 -- 172.21.15.31:0/86964736 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f93680eb440 con 0x7f936c0b1f40 2023-11-26T07:21:35.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 -- 172.21.15.31:0/86964736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f936c164f10 con 0x7f936c0b1f40 2023-11-26T07:21:35.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9361ffb640 1 -- 172.21.15.31:0/86964736 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f93680eea60 con 0x7f936c0b1f40 2023-11-26T07:21:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:35.202 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3429205589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3429205589},{"type":"v1","addr":"172.21.15.97:6836","nonce":3429205589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5959,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3569564413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3569564413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3569564413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5965,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690524463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690524463},{"type":"v1","addr":"172.21.15.97:6837","nonce":690524463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:02.722567+0000","modified":"2023-11-26T07:21:05.757700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1053315203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1053315203},{"type":"v1","addr":"172.21.15.31:6834","nonce":1053315203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:21:35.203 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2023-11-26T07:21:35.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 -- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93500667d0 msgr2=0x7f9350068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93500667d0 0x7f9350068c90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f935c006fd0 tx=0x7f935c008040 comp rx=0 tx=0).stop 2023-11-26T07:21:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 -- 172.21.15.31:0/86964736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b1f40 msgr2=0x7f936c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b1f40 0x7f936c15df30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f936801a060 tx=0x7f93680191d0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 -- 172.21.15.31:0/86964736 shutdown_connections 2023-11-26T07:21:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f93500667d0 0x7f9350068c90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f936c15f540 0x7f936c1554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f936c0b23f0 0x7f936c154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 --2- 172.21.15.31:0/86964736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f936c0b1f40 0x7f936c15df30 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 -- 172.21.15.31:0/86964736 >> 172.21.15.31:0/86964736 conn(0x7f936c0b0c40 msgr2=0x7f936c162330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:35.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 -- 172.21.15.31:0/86964736 shutdown_connections 2023-11-26T07:21:35.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.198+0000 7f9373a54640 1 -- 172.21.15.31:0/86964736 wait complete. 2023-11-26T07:21:35.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:35.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b979aa640 1 Processor -- start 2023-11-26T07:21:35.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b979aa640 1 -- start start 2023-11-26T07:21:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b979aa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b9015d7b0 0x7f6b900d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b979aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900cead0 0x7f6b900d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b979aa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b900cefd0 con 0x7f6b900d1eb0 2023-11-26T07:21:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b979aa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b900d2340 con 0x7f6b9015d7b0 2023-11-26T07:21:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b979aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900cf2b0 con 0x7f6b900cead0 2023-11-26T07:21:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b95746640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b9015d7b0 0x7f6b900d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b94f45640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900cead0 0x7f6b900d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b95f47640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f6b900d1eb0 0x7f6b900cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:45772/0 (socket says 172.21.15.31:45772) 2023-11-26T07:21:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b94f45640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900cead0 0x7f6b900d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:48410/0 (socket says 172.21.15.31:48410) 2023-11-26T07:21:35.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b94f45640 1 -- 172.21.15.31:0/2468052957 learned_addr learned my addr 172.21.15.31:0/2468052957 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:35.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.362+0000 7f6b95746640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6b9015d7b0 0x7f6b900d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:40128/0 (socket says 172.21.15.31:40128) 2023-11-26T07:21:35.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.366+0000 7f6b94f45640 1 -- 172.21.15.31:0/2468052957 >> v2:172.21.15.97:3300/0 conn(0x7f6b9015d7b0 msgr2=0x7f6b900d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:35.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.366+0000 7f6b94f45640 1 --2- 172.21.15.31:0/2468052957 >> v2:172.21.15.97:3300/0 conn(0x7f6b9015d7b0 0x7f6b900d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.366+0000 7f6b94f45640 1 -- 172.21.15.31:0/2468052957 >> v1:172.21.15.97:6789/0 conn(0x7f6b900d1eb0 legacy=0x7f6b900cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.366+0000 7f6b94f45640 1 -- 172.21.15.31:0/2468052957 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9006b3f0 con 0x7f6b900cead0 2023-11-26T07:21:35.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.366+0000 7f6b94f45640 1 --2- 172.21.15.31:0/2468052957 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900cead0 0x7f6b900d1970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009870 tx=0x7f6b8001c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f06cbeff28ea35 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:35.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.366+0000 7f6b87fff640 1 -- 172.21.15.31:0/2468052957 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b800ab5f0 con 0x7f6b900cead0 2023-11-26T07:21:35.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.366+0000 7f6b87fff640 1 -- 172.21.15.31:0/2468052957 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b800ab750 con 0x7f6b900cead0 2023-11-26T07:21:35.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.366+0000 7f6b87fff640 1 -- 172.21.15.31:0/2468052957 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b800abae0 con 0x7f6b900cead0 2023-11-26T07:21:35.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 -- 172.21.15.31:0/2468052957 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900cead0 msgr2=0x7f6b900d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 --2- 172.21.15.31:0/2468052957 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900cead0 0x7f6b900d1970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009870 tx=0x7f6b8001c3f0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 -- 172.21.15.31:0/2468052957 shutdown_connections 2023-11-26T07:21:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 --2- 172.21.15.31:0/2468052957 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900cead0 0x7f6b900d1970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 --2- 172.21.15.31:0/2468052957 >> v2:172.21.15.97:3300/0 conn(0x7f6b9015d7b0 0x7f6b900d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 -- 172.21.15.31:0/2468052957 >> 172.21.15.31:0/2468052957 conn(0x7f6b900d33b0 msgr2=0x7f6b900d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 -- 172.21.15.31:0/2468052957 shutdown_connections 2023-11-26T07:21:35.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 -- 172.21.15.31:0/2468052957 wait complete. 2023-11-26T07:21:35.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 Processor -- start 2023-11-26T07:21:35.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 -- start start 2023-11-26T07:21:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b900cead0 0x7f6b90153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900d1eb0 0x7f6b900c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9015d7b0 0x7f6b900c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900cb910 con 0x7f6b9015d7b0 2023-11-26T07:21:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900cad90 con 0x7f6b900cead0 2023-11-26T07:21:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.370+0000 7f6b979aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900cb070 con 0x7f6b900d1eb0 2023-11-26T07:21:35.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b95746640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b900cead0 0x7f6b90153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b94f45640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900d1eb0 0x7f6b900c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b95746640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b900cead0 0x7f6b90153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:40130/0 (socket says 172.21.15.31:40130) 2023-11-26T07:21:35.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b95f47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9015d7b0 0x7f6b900c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b95746640 1 -- 172.21.15.31:0/773422463 learned_addr learned my addr 172.21.15.31:0/773422463 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:35.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b94f45640 1 -- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b900cead0 msgr2=0x7f6b90153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b94f45640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b900cead0 0x7f6b90153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b94f45640 1 -- 172.21.15.31:0/773422463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9015d7b0 msgr2=0x7f6b900c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b94f45640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9015d7b0 0x7f6b900c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b94f45640 1 -- 172.21.15.31:0/773422463 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b800094d0 con 0x7f6b900d1eb0 2023-11-26T07:21:35.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b94f45640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900d1eb0 0x7f6b900c90b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b800ac000 tx=0x7f6b8001ac30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:35.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b85ffb640 1 -- 172.21.15.31:0/773422463 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b800be5f0 con 0x7f6b900d1eb0 2023-11-26T07:21:35.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b85ffb640 1 -- 172.21.15.31:0/773422463 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b800be790 con 0x7f6b900d1eb0 2023-11-26T07:21:35.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b979aa640 1 -- 172.21.15.31:0/773422463 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b900cb350 con 0x7f6b900d1eb0 2023-11-26T07:21:35.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b85ffb640 1 -- 172.21.15.31:0/773422463 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b800beb40 con 0x7f6b900d1eb0 2023-11-26T07:21:35.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b979aa640 1 -- 172.21.15.31:0/773422463 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b90153c50 con 0x7f6b900d1eb0 2023-11-26T07:21:35.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.374+0000 7f6b979aa640 1 -- 172.21.15.31:0/773422463 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9004e850 con 0x7f6b900d1eb0 2023-11-26T07:21:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.378+0000 7f6b85ffb640 1 -- 172.21.15.31:0/773422463 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f6b800059a0 con 0x7f6b900d1eb0 2023-11-26T07:21:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.378+0000 7f6b85ffb640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b740667d0 0x7f6b74068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.378+0000 7f6b85ffb640 1 -- 172.21.15.31:0/773422463 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b800a8050 con 0x7f6b900d1eb0 2023-11-26T07:21:35.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.378+0000 7f6b95746640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b740667d0 0x7f6b74068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.378+0000 7f6b95746640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b740667d0 0x7f6b74068c90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b900cdad0 tx=0x7f6b8c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:35.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.382+0000 7f6b85ffb640 1 -- 172.21.15.31:0/773422463 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b800ec370 con 0x7f6b900d1eb0 2023-11-26T07:21:35.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.570+0000 7f6b979aa640 1 -- 172.21.15.31:0/773422463 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b900cb1b0 con 0x7f6b900d1eb0 2023-11-26T07:21:35.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.570+0000 7f6b85ffb640 1 -- 172.21.15.31:0/773422463 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f6b900cb1b0 con 0x7f6b900d1eb0 2023-11-26T07:21:35.574 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:35.575 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3429205589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3429205589},{"type":"v1","addr":"172.21.15.97:6836","nonce":3429205589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5959,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3569564413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3569564413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3569564413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5965,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690524463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690524463},{"type":"v1","addr":"172.21.15.97:6837","nonce":690524463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:02.722567+0000","modified":"2023-11-26T07:21:05.757700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/1053315203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1053315203},{"type":"v1","addr":"172.21.15.31:6834","nonce":1053315203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-26T07:21:35.575 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2023-11-26T07:21:35.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 -- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b740667d0 msgr2=0x7f6b74068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b740667d0 0x7f6b74068c90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b900cdad0 tx=0x7f6b8c009190 comp rx=0 tx=0).stop 2023-11-26T07:21:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 -- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900d1eb0 msgr2=0x7f6b900c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900d1eb0 0x7f6b900c90b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b800ac000 tx=0x7f6b8001ac30 comp rx=0 tx=0).stop 2023-11-26T07:21:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 -- 172.21.15.31:0/773422463 shutdown_connections 2023-11-26T07:21:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6b740667d0 0x7f6b74068c90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b9015d7b0 0x7f6b900c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6b900d1eb0 0x7f6b900c90b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 --2- 172.21.15.31:0/773422463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b900cead0 0x7f6b90153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 -- 172.21.15.31:0/773422463 >> 172.21.15.31:0/773422463 conn(0x7f6b900d33b0 msgr2=0x7f6b900b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:35.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 -- 172.21.15.31:0/773422463 shutdown_connections 2023-11-26T07:21:35.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.574+0000 7f6b979aa640 1 -- 172.21.15.31:0/773422463 wait complete. 2023-11-26T07:21:35.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:35.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.734+0000 7f8dbc246640 1 Processor -- start 2023-11-26T07:21:35.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8dbc246640 1 -- start start 2023-11-26T07:21:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8dbc246640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8db415d7b0 0x7f8db40d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8dbc246640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8db40cead0 0x7f8db40d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8dbc246640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db40d22c0 con 0x7f8db40d1eb0 2023-11-26T07:21:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8dbc246640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db40cefd0 con 0x7f8db40cead0 2023-11-26T07:21:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8dbc246640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db40cf2b0 con 0x7f8db415d7b0 2023-11-26T07:21:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8db97e1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8db40cead0 0x7f8db40d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8db97e1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8db40cead0 0x7f8db40d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37764/0 (socket says 172.21.15.31:37764) 2023-11-26T07:21:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8db97e1640 1 -- 172.21.15.31:0/1777419670 learned_addr learned my addr 172.21.15.31:0/1777419670 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8dba7e3640 1 --1- 172.21.15.31:0/1777419670 >> v1:172.21.15.31:6789/0 conn(0x7f8db40d1eb0 0x7f8db40cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55178/0 (socket says 172.21.15.31:55178) 2023-11-26T07:21:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8db9fe2640 1 --2- 172.21.15.31:0/1777419670 >> v2:172.21.15.97:3300/0 conn(0x7f8db415d7b0 0x7f8db40d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8db97e1640 1 -- 172.21.15.31:0/1777419670 >> v2:172.21.15.97:3300/0 conn(0x7f8db415d7b0 msgr2=0x7f8db40d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8db97e1640 1 --2- 172.21.15.31:0/1777419670 >> v2:172.21.15.97:3300/0 conn(0x7f8db415d7b0 0x7f8db40d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8db97e1640 1 -- 172.21.15.31:0/1777419670 >> v1:172.21.15.31:6789/0 conn(0x7f8db40d1eb0 legacy=0x7f8db40cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8db97e1640 1 -- 172.21.15.31:0/1777419670 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db406b3f0 con 0x7f8db40cead0 2023-11-26T07:21:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.738+0000 7f8db8fe0640 1 -- 172.21.15.31:0/1777419670 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264152376 0 0) 0x7f8db40d22c0 con 0x7f8db40d1eb0 2023-11-26T07:21:35.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8db97e1640 1 --2- 172.21.15.31:0/1777419670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 0x7f8db40d1970 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8db40d3310 tx=0x7f8da401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f129df4a2641f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:35.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8db8fe0640 1 -- 172.21.15.31:0/1777419670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da40ac410 con 0x7f8db40cead0 2023-11-26T07:21:35.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8db8fe0640 1 -- 172.21.15.31:0/1777419670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da40ac570 con 0x7f8db40cead0 2023-11-26T07:21:35.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8db8fe0640 1 -- 172.21.15.31:0/1777419670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da40ac900 con 0x7f8db40cead0 2023-11-26T07:21:35.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8dbc246640 1 -- 172.21.15.31:0/1777419670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 msgr2=0x7f8db40d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8dbc246640 1 --2- 172.21.15.31:0/1777419670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 0x7f8db40d1970 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8db40d3310 tx=0x7f8da401c3b0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8dbc246640 1 -- 172.21.15.31:0/1777419670 shutdown_connections 2023-11-26T07:21:35.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8dbc246640 1 --2- 172.21.15.31:0/1777419670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 0x7f8db40d1970 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8dbc246640 1 --2- 172.21.15.31:0/1777419670 >> v2:172.21.15.97:3300/0 conn(0x7f8db415d7b0 0x7f8db40d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8dbc246640 1 -- 172.21.15.31:0/1777419670 >> 172.21.15.31:0/1777419670 conn(0x7f8db40d33b0 msgr2=0x7f8db40d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:35.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8dbc246640 1 -- 172.21.15.31:0/1777419670 shutdown_connections 2023-11-26T07:21:35.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8dbc246640 1 -- 172.21.15.31:0/1777419670 wait complete. 2023-11-26T07:21:35.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8dbc246640 1 Processor -- start 2023-11-26T07:21:35.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.742+0000 7f8dbc246640 1 -- start start 2023-11-26T07:21:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dbc246640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 0x7f8db4157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dbc246640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db40d1eb0 0x7f8db41401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dbc246640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8db415d7b0 0x7f8db406c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dbc246640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4157770 con 0x7f8db40cead0 2023-11-26T07:21:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dbc246640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4143050 con 0x7f8db40d1eb0 2023-11-26T07:21:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dbc246640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db4158f50 con 0x7f8db415d7b0 2023-11-26T07:21:35.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db9fe2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 0x7f8db4157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db9fe2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 0x7f8db4157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37770/0 (socket says 172.21.15.31:37770) 2023-11-26T07:21:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db9fe2640 1 -- 172.21.15.31:0/293995626 learned_addr learned my addr 172.21.15.31:0/293995626 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db97e1640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db40d1eb0 0x7f8db41401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dba7e3640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8db415d7b0 0x7f8db406c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db9fe2640 1 -- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8db415d7b0 msgr2=0x7f8db406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db9fe2640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8db415d7b0 0x7f8db406c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db9fe2640 1 -- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db40d1eb0 msgr2=0x7f8db41401f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db9fe2640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db40d1eb0 0x7f8db41401f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db9fe2640 1 -- 172.21.15.31:0/293995626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da40094d0 con 0x7f8db40cead0 2023-11-26T07:21:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db97e1640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db40d1eb0 0x7f8db41401f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dba7e3640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8db415d7b0 0x7f8db406c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8db9fe2640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 0x7f8db4157230 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8db000d5f0 tx=0x7f8db000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8daa7fc640 1 -- 172.21.15.31:0/293995626 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db00125f0 con 0x7f8db40cead0 2023-11-26T07:21:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8daa7fc640 1 -- 172.21.15.31:0/293995626 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db0012750 con 0x7f8db40cead0 2023-11-26T07:21:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8daa7fc640 1 -- 172.21.15.31:0/293995626 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0012ae0 con 0x7f8db40cead0 2023-11-26T07:21:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dbc246640 1 -- 172.21.15.31:0/293995626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db4159230 con 0x7f8db40cead0 2023-11-26T07:21:35.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dbc246640 1 -- 172.21.15.31:0/293995626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db4155af0 con 0x7f8db40cead0 2023-11-26T07:21:35.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.746+0000 7f8dbc246640 1 -- 172.21.15.31:0/293995626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db404e850 con 0x7f8db40cead0 2023-11-26T07:21:35.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.750+0000 7f8daa7fc640 1 -- 172.21.15.31:0/293995626 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f8db0012c40 con 0x7f8db40cead0 2023-11-26T07:21:35.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.750+0000 7f8daa7fc640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d980667d0 0x7f8d98068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:35.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.750+0000 7f8daa7fc640 1 -- 172.21.15.31:0/293995626 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8db00848c0 con 0x7f8db40cead0 2023-11-26T07:21:35.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.750+0000 7f8db97e1640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d980667d0 0x7f8d98068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:35.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.750+0000 7f8db97e1640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d980667d0 0x7f8d98068c90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8da40ae000 tx=0x7f8da40a6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:35.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.754+0000 7f8daa7fc640 1 -- 172.21.15.31:0/293995626 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8db0051930 con 0x7f8db40cead0 2023-11-26T07:21:35.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.910+0000 7f8dbc246640 1 -- 172.21.15.31:0/293995626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8db40b09e0 con 0x7f8db40cead0 2023-11-26T07:21:35.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8daa7fc640 1 -- 172.21.15.31:0/293995626 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f8db0007380 con 0x7f8db40cead0 2023-11-26T07:21:35.915 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:35.915 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:04.352334+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:01.142504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:01.723895+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:35.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 -- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d980667d0 msgr2=0x7f8d98068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d980667d0 0x7f8d98068c90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8da40ae000 tx=0x7f8da40a6040 comp rx=0 tx=0).stop 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 -- 172.21.15.31:0/293995626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 msgr2=0x7f8db4157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 0x7f8db4157230 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8db000d5f0 tx=0x7f8db000dab0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 -- 172.21.15.31:0/293995626 shutdown_connections 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f8d980667d0 0x7f8d98068c90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8db415d7b0 0x7f8db406c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8db40d1eb0 0x7f8db41401f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 --2- 172.21.15.31:0/293995626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8db40cead0 0x7f8db4157230 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 -- 172.21.15.31:0/293995626 >> 172.21.15.31:0/293995626 conn(0x7f8db40d33b0 msgr2=0x7f8db40d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 -- 172.21.15.31:0/293995626 shutdown_connections 2023-11-26T07:21:35.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:35.914+0000 7f8dbc246640 1 -- 172.21.15.31:0/293995626 wait complete. 2023-11-26T07:21:35.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:21:36.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.078+0000 7f0fd2217640 1 Processor -- start 2023-11-26T07:21:36.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.078+0000 7f0fd2217640 1 -- start start 2023-11-26T07:21:36.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.078+0000 7f0fd2217640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0fcc070a50 0x7f0fcc070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.078+0000 7f0fd2217640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc0713e0 con 0x7f0fcc147060 2023-11-26T07:21:36.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.078+0000 7f0fd2217640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fcc06d4f0 con 0x7f0fcc0ad000 2023-11-26T07:21:36.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.078+0000 7f0fd2217640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fcc0b2cb0 con 0x7f0fcc070a50 2023-11-26T07:21:36.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fcb7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0fcc147060 0x7f0fcc0b2460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55184/0 (socket says 172.21.15.31:55184) 2023-11-26T07:21:36.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fcb7fe640 1 -- 172.21.15.31:0/1310707538 learned_addr learned my addr 172.21.15.31:0/1310707538 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:36.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fcbfff640 1 --2- 172.21.15.31:0/1310707538 >> v2:172.21.15.31:3300/0 conn(0x7f0fcc070a50 0x7f0fcc070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fca7fc640 1 -- 172.21.15.31:0/1310707538 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804309341 0 0) 0x7f0fbc001120 con 0x7f0fcc147060 2023-11-26T07:21:36.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fca7fc640 1 -- 172.21.15.31:0/1310707538 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fb4003530 con 0x7f0fcc147060 2023-11-26T07:21:36.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fcbfff640 1 -- 172.21.15.31:0/1310707538 >> v1:172.21.15.97:6789/0 conn(0x7f0fcc0ad000 legacy=0x7f0fcc06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fcbfff640 1 -- 172.21.15.31:0/1310707538 >> v1:172.21.15.31:6789/0 conn(0x7f0fcc147060 legacy=0x7f0fcc0b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fcbfff640 1 -- 172.21.15.31:0/1310707538 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc0b2f90 con 0x7f0fcc070a50 2023-11-26T07:21:36.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fcbfff640 1 --2- 172.21.15.31:0/1310707538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc070a50 0x7f0fcc070ea0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009d90 tx=0x7f0fc401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a895dc6feb21c27e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:36.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fca7fc640 1 -- 172.21.15.31:0/1310707538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40a5900 con 0x7f0fcc070a50 2023-11-26T07:21:36.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fca7fc640 1 -- 172.21.15.31:0/1310707538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc40a5a60 con 0x7f0fcc070a50 2023-11-26T07:21:36.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.082+0000 7f0fca7fc640 1 -- 172.21.15.31:0/1310707538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40a5df0 con 0x7f0fcc070a50 2023-11-26T07:21:36.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 -- 172.21.15.31:0/1310707538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc070a50 msgr2=0x7f0fcc070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 --2- 172.21.15.31:0/1310707538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc070a50 0x7f0fcc070ea0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009d90 tx=0x7f0fc401c950 comp rx=0 tx=0).stop 2023-11-26T07:21:36.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 -- 172.21.15.31:0/1310707538 shutdown_connections 2023-11-26T07:21:36.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 --2- 172.21.15.31:0/1310707538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc070a50 0x7f0fcc070ea0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 -- 172.21.15.31:0/1310707538 >> 172.21.15.31:0/1310707538 conn(0x7f0fcc0cf3b0 msgr2=0x7f0fcc0cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:36.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 -- 172.21.15.31:0/1310707538 shutdown_connections 2023-11-26T07:21:36.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 -- 172.21.15.31:0/1310707538 wait complete. 2023-11-26T07:21:36.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 Processor -- start 2023-11-26T07:21:36.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 -- start start 2023-11-26T07:21:36.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fcc070a50 0x7f0fcc15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc0ad000 0x7f0fcc162dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fcc147060 0x7f0fcc12f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc163e20 con 0x7f0fcc0ad000 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc163860 con 0x7f0fcc147060 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fd2217640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc163b40 con 0x7f0fcc070a50 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc0ad000 0x7f0fcc162dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc0ad000 0x7f0fcc162dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37796/0 (socket says 172.21.15.31:37796) 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcaffd640 1 -- 172.21.15.31:0/2715947282 learned_addr learned my addr 172.21.15.31:0/2715947282 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcbfff640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fcc147060 0x7f0fcc12f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcb7fe640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fcc070a50 0x7f0fcc15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcaffd640 1 -- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fcc070a50 msgr2=0x7f0fcc15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcaffd640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fcc070a50 0x7f0fcc15e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcaffd640 1 -- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fcc147060 msgr2=0x7f0fcc12f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcaffd640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fcc147060 0x7f0fcc12f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcaffd640 1 -- 172.21.15.31:0/2715947282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4009a70 con 0x7f0fcc0ad000 2023-11-26T07:21:36.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.086+0000 7f0fcb7fe640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fcc070a50 0x7f0fcc15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:36.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fcaffd640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc0ad000 0x7f0fcc162dc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c3c0 tx=0x7f0fc000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:36.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fb3fff640 1 -- 172.21.15.31:0/2715947282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00052d0 con 0x7f0fcc0ad000 2023-11-26T07:21:36.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fb3fff640 1 -- 172.21.15.31:0/2715947282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0005430 con 0x7f0fcc0ad000 2023-11-26T07:21:36.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fd2217640 1 -- 172.21.15.31:0/2715947282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc141610 con 0x7f0fcc0ad000 2023-11-26T07:21:36.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fb3fff640 1 -- 172.21.15.31:0/2715947282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0003840 con 0x7f0fcc0ad000 2023-11-26T07:21:36.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fd2217640 1 -- 172.21.15.31:0/2715947282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc143c80 con 0x7f0fcc0ad000 2023-11-26T07:21:36.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fd2217640 1 -- 172.21.15.31:0/2715947282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc04e850 con 0x7f0fcc0ad000 2023-11-26T07:21:36.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fb3fff640 1 -- 172.21.15.31:0/2715947282 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f0fc00039a0 con 0x7f0fcc0ad000 2023-11-26T07:21:36.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fb3fff640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0fb80667d0 0x7f0fb8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fb3fff640 1 -- 172.21.15.31:0/2715947282 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fc00838c0 con 0x7f0fcc0ad000 2023-11-26T07:21:36.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.090+0000 7f0fcb7fe640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0fb80667d0 0x7f0fb8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.094+0000 7f0fcb7fe640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0fb80667d0 0x7f0fb8068c90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc007fd0 tx=0x7f0fbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:36.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.094+0000 7f0fb3fff640 1 -- 172.21.15.31:0/2715947282 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0fc0050980 con 0x7f0fcc0ad000 2023-11-26T07:21:36.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.290+0000 7f0fd2217640 1 -- 172.21.15.31:0/2715947282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0fcc141ac0 con 0x7f0fcc0ad000 2023-11-26T07:21:36.292 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-11-26T07:21:36.290+0000 7fe86499d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:21:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fb3fff640 1 -- 172.21.15.31:0/2715947282 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0fc0017090 con 0x7f0fcc0ad000 2023-11-26T07:21:36.295 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:21:36.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 -- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0fb80667d0 msgr2=0x7f0fb8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0fb80667d0 0x7f0fb8068c90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc007fd0 tx=0x7f0fbc009040 comp rx=0 tx=0).stop 2023-11-26T07:21:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 -- 172.21.15.31:0/2715947282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc0ad000 msgr2=0x7f0fcc162dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc0ad000 0x7f0fcc162dc0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c3c0 tx=0x7f0fc000c880 comp rx=0 tx=0).stop 2023-11-26T07:21:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 -- 172.21.15.31:0/2715947282 shutdown_connections 2023-11-26T07:21:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0fb80667d0 0x7f0fb8068c90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fcc147060 0x7f0fcc12f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fcc0ad000 0x7f0fcc162dc0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 --2- 172.21.15.31:0/2715947282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fcc070a50 0x7f0fcc15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 -- 172.21.15.31:0/2715947282 >> 172.21.15.31:0/2715947282 conn(0x7f0fcc0cf3b0 msgr2=0x7f0fcc06e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:36.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 -- 172.21.15.31:0/2715947282 shutdown_connections 2023-11-26T07:21:36.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.294+0000 7f0fd2217640 1 -- 172.21.15.31:0/2715947282 wait complete. 2023-11-26T07:21:36.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:21:36.365 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-11-26T07:21:36.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.462+0000 7f5fc31d1640 1 Processor -- start 2023-11-26T07:21:36.467 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fc31d1640 1 -- start start 2023-11-26T07:21:36.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fc31d1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fbc06e3f0 0x7f5fbc15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fc31d1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0ce5d0 0x7f5fbc15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fc31d1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fbc159630 con 0x7f5fbc0cb9b0 2023-11-26T07:21:36.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fc31d1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fbc06e860 con 0x7f5fbc06e3f0 2023-11-26T07:21:36.468 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fc31d1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc15bba0 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fc176e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0ce5d0 0x7f5fbc15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fbbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fbc06e3f0 0x7f5fbc15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fc0f6d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f5fbc0cb9b0 0x7f5fbc0cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:45816/0 (socket says 172.21.15.31:45816) 2023-11-26T07:21:36.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fbbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5fbc06e3f0 0x7f5fbc15f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:40178/0 (socket says 172.21.15.31:40178) 2023-11-26T07:21:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fc0f6d640 1 -- 172.21.15.31:0/3310469017 learned_addr learned my addr 172.21.15.31:0/3310469017 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fbbfff640 1 -- 172.21.15.31:0/3310469017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0ce5d0 msgr2=0x7f5fbc15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fbbfff640 1 --2- 172.21.15.31:0/3310469017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0ce5d0 0x7f5fbc15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fbbfff640 1 -- 172.21.15.31:0/3310469017 >> v1:172.21.15.97:6789/0 conn(0x7f5fbc0cb9b0 legacy=0x7f5fbc0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fbbfff640 1 -- 172.21.15.31:0/3310469017 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fbc15f6d0 con 0x7f5fbc06e3f0 2023-11-26T07:21:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fbb7fe640 1 -- 172.21.15.31:0/3310469017 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576516483 0 0) 0x7f5fbc159630 con 0x7f5fbc0cb9b0 2023-11-26T07:21:36.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.466+0000 7f5fc176e640 1 --2- 172.21.15.31:0/3310469017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0ce5d0 0x7f5fbc15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fbbfff640 1 --2- 172.21.15.31:0/3310469017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc06e3f0 0x7f5fbc15f190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80097c0 tx=0x7f5fa801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd5619b06a0c21a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fbb7fe640 1 -- 172.21.15.31:0/3310469017 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8004230 con 0x7f5fbc06e3f0 2023-11-26T07:21:36.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fbb7fe640 1 -- 172.21.15.31:0/3310469017 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa80a7050 con 0x7f5fbc06e3f0 2023-11-26T07:21:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fbb7fe640 1 -- 172.21.15.31:0/3310469017 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8003620 con 0x7f5fbc06e3f0 2023-11-26T07:21:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fc31d1640 1 -- 172.21.15.31:0/3310469017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc06e3f0 msgr2=0x7f5fbc15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fc31d1640 1 --2- 172.21.15.31:0/3310469017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc06e3f0 0x7f5fbc15f190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80097c0 tx=0x7f5fa801c480 comp rx=0 tx=0).stop 2023-11-26T07:21:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fc31d1640 1 -- 172.21.15.31:0/3310469017 shutdown_connections 2023-11-26T07:21:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fc31d1640 1 --2- 172.21.15.31:0/3310469017 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0ce5d0 0x7f5fbc15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fc31d1640 1 --2- 172.21.15.31:0/3310469017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc06e3f0 0x7f5fbc15f190 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fc31d1640 1 -- 172.21.15.31:0/3310469017 >> 172.21.15.31:0/3310469017 conn(0x7f5fbc0b0cb0 msgr2=0x7f5fbc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fc31d1640 1 -- 172.21.15.31:0/3310469017 shutdown_connections 2023-11-26T07:21:36.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.470+0000 7f5fc31d1640 1 -- 172.21.15.31:0/3310469017 wait complete. 2023-11-26T07:21:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc31d1640 1 Processor -- start 2023-11-26T07:21:36.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc31d1640 1 -- start start 2023-11-26T07:21:36.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc31d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc06e3f0 0x7f5fbc14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc31d1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0cb9b0 0x7f5fbc1f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc31d1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc0ce5d0 0x7f5fbc1f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc31d1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc14d950 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc31d1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc0b08a0 con 0x7f5fbc06e3f0 2023-11-26T07:21:36.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc31d1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fbc0b05c0 con 0x7f5fbc0cb9b0 2023-11-26T07:21:36.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc176e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc0ce5d0 0x7f5fbc1f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc176e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc0ce5d0 0x7f5fbc1f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37834/0 (socket says 172.21.15.31:37834) 2023-11-26T07:21:36.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc176e640 1 -- 172.21.15.31:0/4021979389 learned_addr learned my addr 172.21.15.31:0/4021979389 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:36.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fbbfff640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0cb9b0 0x7f5fbc1f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc0f6d640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc06e3f0 0x7f5fbc14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc176e640 1 -- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0cb9b0 msgr2=0x7f5fbc1f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc176e640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0cb9b0 0x7f5fbc1f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc176e640 1 -- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc06e3f0 msgr2=0x7f5fbc14d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc176e640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc06e3f0 0x7f5fbc14d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc176e640 1 -- 172.21.15.31:0/4021979389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb4009a70 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc176e640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc0ce5d0 0x7f5fbc1f5cf0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400efc0 tx=0x7f5fb400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fb97fa640 1 -- 172.21.15.31:0/4021979389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4009cd0 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fb97fa640 1 -- 172.21.15.31:0/4021979389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb400f040 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fb97fa640 1 -- 172.21.15.31:0/4021979389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40039a0 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc31d1640 1 -- 172.21.15.31:0/4021979389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa8009420 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.474+0000 7f5fc31d1640 1 -- 172.21.15.31:0/4021979389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fbc1565c0 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.478+0000 7f5fc31d1640 1 -- 172.21.15.31:0/4021979389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fbc0671f0 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.478+0000 7f5fb97fa640 1 -- 172.21.15.31:0/4021979389 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f5fb4014480 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.478+0000 7f5fb97fa640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5fa00667d0 0x7f5fa0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:36.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.478+0000 7f5fb97fa640 1 -- 172.21.15.31:0/4021979389 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5fb4085130 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.478+0000 7f5fc0f6d640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5fa00667d0 0x7f5fa0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:36.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.478+0000 7f5fc0f6d640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5fa00667d0 0x7f5fa0068c90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0000f70 tx=0x7f5fb0007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:36.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.482+0000 7f5fb97fa640 1 -- 172.21.15.31:0/4021979389 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5fb4052080 con 0x7f5fbc0ce5d0 2023-11-26T07:21:36.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:36.670+0000 7f5fc31d1640 1 -- 172.21.15.31:0/4021979389 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5fbc06b460 con 0x7f5fbc0ce5d0 2023-11-26T07:21:37.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.298+0000 7f5fb97fa640 1 -- 172.21.15.31:0/4021979389 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5fb40556a0 con 0x7f5fbc0ce5d0 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 -- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5fa00667d0 msgr2=0x7f5fa0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5fa00667d0 0x7f5fa0068c90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0000f70 tx=0x7f5fb0007b60 comp rx=0 tx=0).stop 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 -- 172.21.15.31:0/4021979389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc0ce5d0 msgr2=0x7f5fbc1f5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc0ce5d0 0x7f5fbc1f5cf0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400efc0 tx=0x7f5fb400c490 comp rx=0 tx=0).stop 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 -- 172.21.15.31:0/4021979389 shutdown_connections 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5fa00667d0 0x7f5fa0068c90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fbc0ce5d0 0x7f5fbc1f5cf0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5fbc0cb9b0 0x7f5fbc1f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 --2- 172.21.15.31:0/4021979389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc06e3f0 0x7f5fbc14d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 -- 172.21.15.31:0/4021979389 >> 172.21.15.31:0/4021979389 conn(0x7f5fbc0b0cb0 msgr2=0x7f5fbc0cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 -- 172.21.15.31:0/4021979389 shutdown_connections 2023-11-26T07:21:37.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.302+0000 7f5fc31d1640 1 -- 172.21.15.31:0/4021979389 wait complete. 2023-11-26T07:21:37.328 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:21:37.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.522+0000 7f31317aa640 1 Processor -- start 2023-11-26T07:21:37.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.522+0000 7f31317aa640 1 -- start start 2023-11-26T07:21:37.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f31317aa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f312c0cdfb0 0x7f312c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:37.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f31317aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f312c0cbd50 0x7f312c0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:37.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f31317aa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312c0b2710 con 0x7f312c15a710 2023-11-26T07:21:37.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f31317aa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312c0cca80 con 0x7f312c0cdfb0 2023-11-26T07:21:37.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f31317aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c0b29f0 con 0x7f312c0cbd50 2023-11-26T07:21:37.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f312affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f312c0cdfb0 0x7f312c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:37.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f312affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f312c0cdfb0 0x7f312c0abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37840/0 (socket says 172.21.15.31:37840) 2023-11-26T07:21:37.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f312affd640 1 -- 172.21.15.31:0/2790439402 learned_addr learned my addr 172.21.15.31:0/2790439402 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:37.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f312b7fe640 1 --2- 172.21.15.31:0/2790439402 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f312c0cbd50 0x7f312c0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:37.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f312affd640 1 -- 172.21.15.31:0/2790439402 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f312c0cbd50 msgr2=0x7f312c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:37.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f312affd640 1 --2- 172.21.15.31:0/2790439402 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f312c0cbd50 0x7f312c0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:37.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f312affd640 1 -- 172.21.15.31:0/2790439402 >> v1:172.21.15.97:6789/0 conn(0x7f312c15a710 legacy=0x7f312c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:37.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.526+0000 7f312affd640 1 -- 172.21.15.31:0/2790439402 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c0b2cd0 con 0x7f312c0cdfb0 2023-11-26T07:21:37.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f312affd640 1 --2- 172.21.15.31:0/2790439402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 0x7f312c0abf90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f311c009870 tx=0x7f311c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6db4e5274ad953 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:37.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f312a7fc640 1 -- 172.21.15.31:0/2790439402 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0a95f0 con 0x7f312c0cdfb0 2023-11-26T07:21:37.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f312a7fc640 1 -- 172.21.15.31:0/2790439402 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c0a9750 con 0x7f312c0cdfb0 2023-11-26T07:21:37.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f312a7fc640 1 -- 172.21.15.31:0/2790439402 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c0a9ae0 con 0x7f312c0cdfb0 2023-11-26T07:21:37.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f31317aa640 1 -- 172.21.15.31:0/2790439402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 msgr2=0x7f312c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:37.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f31317aa640 1 --2- 172.21.15.31:0/2790439402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 0x7f312c0abf90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f311c009870 tx=0x7f311c01c510 comp rx=0 tx=0).stop 2023-11-26T07:21:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f31317aa640 1 -- 172.21.15.31:0/2790439402 shutdown_connections 2023-11-26T07:21:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f31317aa640 1 --2- 172.21.15.31:0/2790439402 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f312c0cbd50 0x7f312c0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f31317aa640 1 --2- 172.21.15.31:0/2790439402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 0x7f312c0abf90 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f31317aa640 1 -- 172.21.15.31:0/2790439402 >> 172.21.15.31:0/2790439402 conn(0x7f312c0b0cb0 msgr2=0x7f312c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f31317aa640 1 -- 172.21.15.31:0/2790439402 shutdown_connections 2023-11-26T07:21:37.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f31317aa640 1 -- 172.21.15.31:0/2790439402 wait complete. 2023-11-26T07:21:37.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f31317aa640 1 Processor -- start 2023-11-26T07:21:37.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.530+0000 7f31317aa640 1 -- start start 2023-11-26T07:21:37.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31317aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f312c0cbd50 0x7f312c09c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:37.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31317aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 0x7f312c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31317aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c15a710 0x7f312c15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31317aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c157c30 con 0x7f312c0cdfb0 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31317aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c157320 con 0x7f312c15a710 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31317aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c09ced0 con 0x7f312c0cbd50 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f3123fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 0x7f312c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f3123fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 0x7f312c15f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37852/0 (socket says 172.21.15.31:37852) 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f3123fff640 1 -- 172.21.15.31:0/2415250367 learned_addr learned my addr 172.21.15.31:0/2415250367 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f312affd640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f312c0cbd50 0x7f312c09c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f312b7fe640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c15a710 0x7f312c15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f3123fff640 1 -- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f312c0cbd50 msgr2=0x7f312c09c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:37.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f3123fff640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f312c0cbd50 0x7f312c09c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f3123fff640 1 -- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c15a710 msgr2=0x7f312c15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f3123fff640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c15a710 0x7f312c15cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f3123fff640 1 -- 172.21.15.31:0/2415250367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c0094d0 con 0x7f312c0cdfb0 2023-11-26T07:21:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f312b7fe640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c15a710 0x7f312c15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:37.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f3123fff640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 0x7f312c15f190 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f311400c390 tx=0x7f311400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31237fe640 1 -- 172.21.15.31:0/2415250367 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31140052d0 con 0x7f312c0cdfb0 2023-11-26T07:21:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31237fe640 1 -- 172.21.15.31:0/2415250367 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3114005430 con 0x7f312c0cdfb0 2023-11-26T07:21:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31317aa640 1 -- 172.21.15.31:0/2415250367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312c1575e0 con 0x7f312c0cdfb0 2023-11-26T07:21:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31237fe640 1 -- 172.21.15.31:0/2415250367 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114003820 con 0x7f312c0cdfb0 2023-11-26T07:21:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31317aa640 1 -- 172.21.15.31:0/2415250367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c099b10 con 0x7f312c0cdfb0 2023-11-26T07:21:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31317aa640 1 -- 172.21.15.31:0/2415250367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312c0671f0 con 0x7f312c0cdfb0 2023-11-26T07:21:37.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.534+0000 7f31237fe640 1 -- 172.21.15.31:0/2415250367 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f3114003980 con 0x7f312c0cdfb0 2023-11-26T07:21:37.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.538+0000 7f31237fe640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f310c0667d0 0x7f310c068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:37.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.538+0000 7f31237fe640 1 -- 172.21.15.31:0/2415250367 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f31140838d0 con 0x7f312c0cdfb0 2023-11-26T07:21:37.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.538+0000 7f312affd640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f310c0667d0 0x7f310c068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:37.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.538+0000 7f312affd640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f310c0667d0 0x7f310c068c90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f311c0ae000 tx=0x7f311c0b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:37.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.542+0000 7f31237fe640 1 -- 172.21.15.31:0/2415250367 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3114050970 con 0x7f312c0cdfb0 2023-11-26T07:21:37.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:37.698+0000 7f31317aa640 1 -- 172.21.15.31:0/2415250367 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f312c06b460 con 0x7f312c0cdfb0 2023-11-26T07:21:38.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.302+0000 7f31237fe640 1 -- 172.21.15.31:0/2415250367 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) v1 ==== 181+0+0 (secure 0 0 0) 0x7f31140170d0 con 0x7f312c0cdfb0 2023-11-26T07:21:38.303 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:21:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 -- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f310c0667d0 msgr2=0x7f310c068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f310c0667d0 0x7f310c068c90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f311c0ae000 tx=0x7f311c0b3040 comp rx=0 tx=0).stop 2023-11-26T07:21:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 -- 172.21.15.31:0/2415250367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 msgr2=0x7f312c15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 0x7f312c15f190 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f311400c390 tx=0x7f311400c850 comp rx=0 tx=0).stop 2023-11-26T07:21:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 -- 172.21.15.31:0/2415250367 shutdown_connections 2023-11-26T07:21:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f310c0667d0 0x7f310c068c90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f312c15a710 0x7f312c15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f312c0cdfb0 0x7f312c15f190 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 --2- 172.21.15.31:0/2415250367 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f312c0cbd50 0x7f312c09c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 -- 172.21.15.31:0/2415250367 >> 172.21.15.31:0/2415250367 conn(0x7f312c0b0cb0 msgr2=0x7f312c15dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:38.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 -- 172.21.15.31:0/2415250367 shutdown_connections 2023-11-26T07:21:38.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.306+0000 7f31317aa640 1 -- 172.21.15.31:0/2415250367 wait complete. 2023-11-26T07:21:38.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:21:38.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaf1ee4640 1 Processor -- start 2023-11-26T07:21:38.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaf1ee4640 1 -- start start 2023-11-26T07:21:38.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaf1ee4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fbaec0b07a0 0x7fbaec1619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaf1ee4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec161f20 0x7fbaec0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaf1ee4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaec161290 con 0x7fbaec0b2870 2023-11-26T07:21:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaf1ee4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaec0b0b80 con 0x7fbaec0b07a0 2023-11-26T07:21:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaf1ee4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec15ebc0 con 0x7fbaec161f20 2023-11-26T07:21:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaebfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbaec0b2870 0x7fbaec0cc9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55188/0 (socket says 172.21.15.31:55188) 2023-11-26T07:21:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaebfff640 1 -- 172.21.15.31:0/2905420060 learned_addr learned my addr 172.21.15.31:0/2905420060 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaeb7fe640 1 --2- 172.21.15.31:0/2905420060 >> v2:172.21.15.97:3300/0 conn(0x7fbaec0b07a0 0x7fbaec1619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516886911 0 0) 0x7fbaec161290 con 0x7fbaec0b2870 2023-11-26T07:21:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbad4003530 con 0x7fbaec0b2870 2023-11-26T07:21:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaeaffd640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec161f20 0x7fbaec0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2759116832 0 0) 0x7fbad4003530 con 0x7fbaec0b2870 2023-11-26T07:21:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec161f20 msgr2=0x7fbaec0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec161f20 0x7fbaec0b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 >> v2:172.21.15.97:3300/0 conn(0x7fbaec0b07a0 msgr2=0x7fbaec1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 --2- 172.21.15.31:0/2905420060 >> v2:172.21.15.97:3300/0 conn(0x7fbaec0b07a0 0x7fbaec1619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec161550 con 0x7fbaec0b2870 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fbae4001a00 con 0x7fbaec0b2870 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 >> v1:172.21.15.31:6789/0 conn(0x7fbaec0b2870 legacy=0x7fbaec0cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec161f20 0x7fbad40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 0x7fbad40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad4004d10 0x7fbad4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec15ebc0 con 0x7fbad4003f30 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad400b990 con 0x7fbad4004d10 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad400bad0 con 0x7fbaec161f20 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaeaffd640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 0x7fbad40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaeb7fe640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec161f20 0x7fbad40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaebfff640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad4004d10 0x7fbad4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaeaffd640 1 -- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec161f20 msgr2=0x7fbad40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaeaffd640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec161f20 0x7fbad40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaeaffd640 1 -- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad4004d10 msgr2=0x7fbad4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.478+0000 7fbaeaffd640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbad4004d10 0x7fbad4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaeaffd640 1 -- 172.21.15.31:0/2905420060 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad4003530 con 0x7fbad4003f30 2023-11-26T07:21:38.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaeaffd640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 0x7fbad40047d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009ab0 tx=0x7fbae001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ea848ae59fd15f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:38.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae00a5510 con 0x7fbad4003f30 2023-11-26T07:21:38.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae00a5670 con 0x7fbad4003f30 2023-11-26T07:21:38.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaea7fc640 1 -- 172.21.15.31:0/2905420060 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae00a5a20 con 0x7fbad4003f30 2023-11-26T07:21:38.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/2905420060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 msgr2=0x7fbad40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaf1ee4640 1 --2- 172.21.15.31:0/2905420060 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 0x7fbad40047d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009ab0 tx=0x7fbae001ca30 comp rx=0 tx=0).stop 2023-11-26T07:21:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaebfff640 1 -- 172.21.15.31:0/2905420060 reap_dead start 2023-11-26T07:21:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/2905420060 shutdown_connections 2023-11-26T07:21:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/2905420060 >> 172.21.15.31:0/2905420060 conn(0x7fbaec0b1d50 msgr2=0x7fbaec0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/2905420060 shutdown_connections 2023-11-26T07:21:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.482+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/2905420060 wait complete. 2023-11-26T07:21:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaf1ee4640 1 Processor -- start 2023-11-26T07:21:38.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaf1ee4640 1 -- start start 2023-11-26T07:21:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaf1ee4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 0x7fbaec151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaf1ee4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec0b2d40 0x7fbaec0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaf1ee4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec15cd10 0x7fbaec15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaf1ee4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec151e40 con 0x7fbad4003f30 2023-11-26T07:21:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaf1ee4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec15d6d0 con 0x7fbaec15cd10 2023-11-26T07:21:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaf1ee4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec0b49f0 con 0x7fbaec0b2d40 2023-11-26T07:21:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaeb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 0x7fbaec151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaeb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 0x7fbaec151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37872/0 (socket says 172.21.15.31:37872) 2023-11-26T07:21:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaeb7fe640 1 -- 172.21.15.31:0/1023041434 learned_addr learned my addr 172.21.15.31:0/1023041434 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaeaffd640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec0b2d40 0x7fbaec0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaebfff640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec15cd10 0x7fbaec15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaeb7fe640 1 -- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec0b2d40 msgr2=0x7fbaec0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaeb7fe640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec0b2d40 0x7fbaec0b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaeb7fe640 1 -- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec15cd10 msgr2=0x7fbaec15d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:38.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaeb7fe640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec15cd10 0x7fbaec15d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:38.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaeb7fe640 1 -- 172.21.15.31:0/1023041434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec0b4cd0 con 0x7fbad4003f30 2023-11-26T07:21:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaeb7fe640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 0x7fbaec151900 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fbadc011660 tx=0x7fbadc011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbad3fff640 1 -- 172.21.15.31:0/1023041434 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc015870 con 0x7fbad4003f30 2023-11-26T07:21:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbad3fff640 1 -- 172.21.15.31:0/1023041434 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc0159d0 con 0x7fbad4003f30 2023-11-26T07:21:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbad3fff640 1 -- 172.21.15.31:0/1023041434 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc015d40 con 0x7fbad4003f30 2023-11-26T07:21:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.486+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/1023041434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaec0b4fb0 con 0x7fbad4003f30 2023-11-26T07:21:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.490+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/1023041434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaec0b5570 con 0x7fbad4003f30 2023-11-26T07:21:38.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.490+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/1023041434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaec04e850 con 0x7fbad4003f30 2023-11-26T07:21:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.490+0000 7fbad3fff640 1 -- 172.21.15.31:0/1023041434 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fbadc0143e0 con 0x7fbad4003f30 2023-11-26T07:21:38.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.490+0000 7fbad3fff640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbad80667d0 0x7fbad8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:38.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.490+0000 7fbad3fff640 1 -- 172.21.15.31:0/1023041434 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fbadc084650 con 0x7fbad4003f30 2023-11-26T07:21:38.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.490+0000 7fbaeaffd640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbad80667d0 0x7fbad8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:38.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.494+0000 7fbaeaffd640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbad80667d0 0x7fbad8068c90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbae00af000 tx=0x7fbae00046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:38.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.498+0000 7fbad3fff640 1 -- 172.21.15.31:0/1023041434 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbadc051820 con 0x7fbad4003f30 2023-11-26T07:21:38.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:38.654+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/1023041434 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbaec0b0320 con 0x7fbad4003f30 2023-11-26T07:21:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.306+0000 7fbad3fff640 1 -- 172.21.15.31:0/1023041434 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbadc054e40 con 0x7fbad4003f30 2023-11-26T07:21:39.310 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-11-26T07:21:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbad80667d0 msgr2=0x7fbad8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbad80667d0 0x7fbad8068c90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbae00af000 tx=0x7fbae00046d0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/1023041434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 msgr2=0x7fbaec151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 0x7fbaec151900 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fbadc011660 tx=0x7fbadc011a60 comp rx=0 tx=0).stop 2023-11-26T07:21:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/1023041434 shutdown_connections 2023-11-26T07:21:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fbad80667d0 0x7fbad8068c90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbaec15cd10 0x7fbaec15d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fbaec0b2d40 0x7fbaec0b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 --2- 172.21.15.31:0/1023041434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbad4003f30 0x7fbaec151900 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/1023041434 >> 172.21.15.31:0/1023041434 conn(0x7fbaec0b1d50 msgr2=0x7fbaec06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:39.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/1023041434 shutdown_connections 2023-11-26T07:21:39.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.310+0000 7fbaf1ee4640 1 -- 172.21.15.31:0/1023041434 wait complete. 2023-11-26T07:21:39.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:21:39.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.490+0000 7f05fa1df640 1 Processor -- start 2023-11-26T07:21:39.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05fa1df640 1 -- start start 2023-11-26T07:21:39.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05fa1df640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05f40cb9b0 0x7f05f406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05fa1df640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f05f40cbd90 0x7f05f415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05fa1df640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f4159630 con 0x7f05f40ce5d0 2023-11-26T07:21:39.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05fa1df640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f415bba0 con 0x7f05f40cb9b0 2023-11-26T07:21:39.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05fa1df640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f40cdd50 con 0x7f05f40cbd90 2023-11-26T07:21:39.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05f37fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05f40cb9b0 0x7f05f406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05f3fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f05f40ce5d0 0x7f05f415bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55196/0 (socket says 172.21.15.31:55196) 2023-11-26T07:21:39.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05f3fff640 1 -- 172.21.15.31:0/287826974 learned_addr learned my addr 172.21.15.31:0/287826974 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:39.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05f2ffd640 1 --2- 172.21.15.31:0/287826974 >> v2:172.21.15.97:3300/0 conn(0x7f05f40cbd90 0x7f05f415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05f37fe640 1 -- 172.21.15.31:0/287826974 >> v2:172.21.15.97:3300/0 conn(0x7f05f40cbd90 msgr2=0x7f05f415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05f37fe640 1 --2- 172.21.15.31:0/287826974 >> v2:172.21.15.97:3300/0 conn(0x7f05f40cbd90 0x7f05f415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05f37fe640 1 -- 172.21.15.31:0/287826974 >> v1:172.21.15.31:6789/0 conn(0x7f05f40ce5d0 legacy=0x7f05f415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05f37fe640 1 -- 172.21.15.31:0/287826974 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f415f6d0 con 0x7f05f40cb9b0 2023-11-26T07:21:39.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.494+0000 7f05f2ffd640 1 --2- 172.21.15.31:0/287826974 >> v2:172.21.15.97:3300/0 conn(0x7f05f40cbd90 0x7f05f415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:39.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05f37fe640 1 --2- 172.21.15.31:0/287826974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40cb9b0 0x7f05f406e3f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f05f40b5200 tx=0x7f05e401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38f34f52f8279c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05f27fc640 1 -- 172.21.15.31:0/287826974 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40ab610 con 0x7f05f40cb9b0 2023-11-26T07:21:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05f27fc640 1 -- 172.21.15.31:0/287826974 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e40ab770 con 0x7f05f40cb9b0 2023-11-26T07:21:39.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05f27fc640 1 -- 172.21.15.31:0/287826974 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40abb00 con 0x7f05f40cb9b0 2023-11-26T07:21:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05fa1df640 1 -- 172.21.15.31:0/287826974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40cb9b0 msgr2=0x7f05f406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05fa1df640 1 --2- 172.21.15.31:0/287826974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40cb9b0 0x7f05f406e3f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f05f40b5200 tx=0x7f05e401c410 comp rx=0 tx=0).stop 2023-11-26T07:21:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05fa1df640 1 -- 172.21.15.31:0/287826974 shutdown_connections 2023-11-26T07:21:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05fa1df640 1 --2- 172.21.15.31:0/287826974 >> v2:172.21.15.97:3300/0 conn(0x7f05f40cbd90 0x7f05f415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05fa1df640 1 --2- 172.21.15.31:0/287826974 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40cb9b0 0x7f05f406e3f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05fa1df640 1 -- 172.21.15.31:0/287826974 >> 172.21.15.31:0/287826974 conn(0x7f05f40b0cb0 msgr2=0x7f05f40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05fa1df640 1 -- 172.21.15.31:0/287826974 shutdown_connections 2023-11-26T07:21:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05fa1df640 1 -- 172.21.15.31:0/287826974 wait complete. 2023-11-26T07:21:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05fa1df640 1 Processor -- start 2023-11-26T07:21:39.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.498+0000 7f05fa1df640 1 -- start start 2023-11-26T07:21:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05fa1df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f40cb9b0 0x7f05f41538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05fa1df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f05f40cbd90 0x7f05f4147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05fa1df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40ce5d0 0x7f05f415de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05fa1df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f41f6570 con 0x7f05f40ce5d0 2023-11-26T07:21:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05fa1df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f4152240 con 0x7f05f40cb9b0 2023-11-26T07:21:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05fa1df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f4153ea0 con 0x7f05f40cbd90 2023-11-26T07:21:39.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40ce5d0 0x7f05f415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40ce5d0 0x7f05f415de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37898/0 (socket says 172.21.15.31:37898) 2023-11-26T07:21:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f3fff640 1 -- 172.21.15.31:0/3966868188 learned_addr learned my addr 172.21.15.31:0/3966868188 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f37fe640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f40cb9b0 0x7f05f41538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f2ffd640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f05f40cbd90 0x7f05f4147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f3fff640 1 -- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f05f40cbd90 msgr2=0x7f05f4147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f3fff640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f05f40cbd90 0x7f05f4147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f3fff640 1 -- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f40cb9b0 msgr2=0x7f05f41538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f3fff640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f40cb9b0 0x7f05f41538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f3fff640 1 -- 172.21.15.31:0/3966868188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4009420 con 0x7f05f40ce5d0 2023-11-26T07:21:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f37fe640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f40cb9b0 0x7f05f41538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05f3fff640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40ce5d0 0x7f05f415de80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00cab0 tx=0x7f05ec00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05dbfff640 1 -- 172.21.15.31:0/3966868188 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec00ac30 con 0x7f05f40ce5d0 2023-11-26T07:21:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05dbfff640 1 -- 172.21.15.31:0/3966868188 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec00ad90 con 0x7f05f40ce5d0 2023-11-26T07:21:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05fa1df640 1 -- 172.21.15.31:0/3966868188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f4152490 con 0x7f05f40ce5d0 2023-11-26T07:21:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05dbfff640 1 -- 172.21.15.31:0/3966868188 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec012600 con 0x7f05f40ce5d0 2023-11-26T07:21:39.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05fa1df640 1 -- 172.21.15.31:0/3966868188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f4152a50 con 0x7f05f40ce5d0 2023-11-26T07:21:39.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.502+0000 7f05fa1df640 1 -- 172.21.15.31:0/3966868188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f40671f0 con 0x7f05f40ce5d0 2023-11-26T07:21:39.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.506+0000 7f05dbfff640 1 -- 172.21.15.31:0/3966868188 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f05ec012760 con 0x7f05f40ce5d0 2023-11-26T07:21:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.506+0000 7f05dbfff640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05e00667d0 0x7f05e0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.506+0000 7f05dbfff640 1 -- 172.21.15.31:0/3966868188 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f05ec083f30 con 0x7f05f40ce5d0 2023-11-26T07:21:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.506+0000 7f05f37fe640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05e00667d0 0x7f05e0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.506+0000 7f05f37fe640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05e00667d0 0x7f05e0068c90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f05e40af000 tx=0x7f05e40a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.510+0000 7f05dbfff640 1 -- 172.21.15.31:0/3966868188 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f05ec051210 con 0x7f05f40ce5d0 2023-11-26T07:21:39.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.666+0000 7f05fa1df640 1 -- 172.21.15.31:0/3966868188 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f05f4152c70 con 0x7f05f40ce5d0 2023-11-26T07:21:39.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05dbfff640 1 -- 172.21.15.31:0/3966868188 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f05ec054830 con 0x7f05f40ce5d0 2023-11-26T07:21:39.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 -- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05e00667d0 msgr2=0x7f05e0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05e00667d0 0x7f05e0068c90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f05e40af000 tx=0x7f05e40a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 -- 172.21.15.31:0/3966868188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40ce5d0 msgr2=0x7f05f415de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40ce5d0 0x7f05f415de80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00cab0 tx=0x7f05ec00cf70 comp rx=0 tx=0).stop 2023-11-26T07:21:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 -- 172.21.15.31:0/3966868188 shutdown_connections 2023-11-26T07:21:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f05e00667d0 0x7f05e0068c90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05f40ce5d0 0x7f05f415de80 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f05f40cbd90 0x7f05f4147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 --2- 172.21.15.31:0/3966868188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f40cb9b0 0x7f05f41538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 -- 172.21.15.31:0/3966868188 >> 172.21.15.31:0/3966868188 conn(0x7f05f40b0cb0 msgr2=0x7f05f40b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 -- 172.21.15.31:0/3966868188 shutdown_connections 2023-11-26T07:21:39.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.674+0000 7f05fa1df640 1 -- 172.21.15.31:0/3966868188 wait complete. 2023-11-26T07:21:39.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2023-11-26T07:21:39.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e1c7c4640 1 Processor -- start 2023-11-26T07:21:39.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e1c7c4640 1 -- start start 2023-11-26T07:21:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e1c7c4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4e140cead0 0x7f4e140d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e1c7c4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140d1eb0 0x7f4e1406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e1c7c4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e140d2300 con 0x7f4e1415d7b0 2023-11-26T07:21:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e1c7c4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e140d17e0 con 0x7f4e140cead0 2023-11-26T07:21:39.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e1c7c4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140cefd0 con 0x7f4e140d1eb0 2023-11-26T07:21:39.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e19d5f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4e140cead0 0x7f4e140d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e1ad61640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140d1eb0 0x7f4e1406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e19d5f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4e140cead0 0x7f4e140d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:40228/0 (socket says 172.21.15.31:40228) 2023-11-26T07:21:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e19d5f640 1 -- 172.21.15.31:0/2879960755 learned_addr learned my addr 172.21.15.31:0/2879960755 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e19d5f640 1 -- 172.21.15.31:0/2879960755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140d1eb0 msgr2=0x7f4e1406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e19d5f640 1 --2- 172.21.15.31:0/2879960755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140d1eb0 0x7f4e1406bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e19d5f640 1 -- 172.21.15.31:0/2879960755 >> v1:172.21.15.97:6789/0 conn(0x7f4e1415d7b0 legacy=0x7f4e140d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e19d5f640 1 -- 172.21.15.31:0/2879960755 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e140cf2b0 con 0x7f4e140cead0 2023-11-26T07:21:39.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.838+0000 7f4e1ad61640 1 --2- 172.21.15.31:0/2879960755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140d1eb0 0x7f4e1406bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e19d5f640 1 --2- 172.21.15.31:0/2879960755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140cead0 0x7f4e140d1970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009870 tx=0x7f4e0401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9b2ca902b5a5189 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1955e640 1 -- 172.21.15.31:0/2879960755 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e040ab5f0 con 0x7f4e140cead0 2023-11-26T07:21:39.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1955e640 1 -- 172.21.15.31:0/2879960755 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e040ab750 con 0x7f4e140cead0 2023-11-26T07:21:39.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1955e640 1 -- 172.21.15.31:0/2879960755 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e040abae0 con 0x7f4e140cead0 2023-11-26T07:21:39.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/2879960755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140cead0 msgr2=0x7f4e140d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1c7c4640 1 --2- 172.21.15.31:0/2879960755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140cead0 0x7f4e140d1970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009870 tx=0x7f4e0401c3f0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/2879960755 shutdown_connections 2023-11-26T07:21:39.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1c7c4640 1 --2- 172.21.15.31:0/2879960755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140d1eb0 0x7f4e1406bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1c7c4640 1 --2- 172.21.15.31:0/2879960755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140cead0 0x7f4e140d1970 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/2879960755 >> 172.21.15.31:0/2879960755 conn(0x7f4e140d33b0 msgr2=0x7f4e140d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:39.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/2879960755 shutdown_connections 2023-11-26T07:21:39.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.842+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/2879960755 wait complete. 2023-11-26T07:21:39.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 Processor -- start 2023-11-26T07:21:39.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 -- start start 2023-11-26T07:21:39.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140cead0 0x7f4e14155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140d1eb0 0x7f4e14153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e1415d7b0 0x7f4e140b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14155d70 con 0x7f4e1415d7b0 2023-11-26T07:21:39.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14154050 con 0x7f4e140d1eb0 2023-11-26T07:21:39.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14154eb0 con 0x7f4e140cead0 2023-11-26T07:21:39.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1ad61640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e1415d7b0 0x7f4e140b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1ad61640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e1415d7b0 0x7f4e140b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37914/0 (socket says 172.21.15.31:37914) 2023-11-26T07:21:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1ad61640 1 -- 172.21.15.31:0/1411313837 learned_addr learned my addr 172.21.15.31:0/1411313837 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e19d5f640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140d1eb0 0x7f4e14153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1a560640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140cead0 0x7f4e14155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1ad61640 1 -- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140cead0 msgr2=0x7f4e14155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1ad61640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140cead0 0x7f4e14155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1ad61640 1 -- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140d1eb0 msgr2=0x7f4e14153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1ad61640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140d1eb0 0x7f4e14153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1ad61640 1 -- 172.21.15.31:0/1411313837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c009a70 con 0x7f4e1415d7b0 2023-11-26T07:21:39.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1ad61640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e1415d7b0 0x7f4e140b1530 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00efc0 tx=0x7f4e0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e0affd640 1 -- 172.21.15.31:0/1411313837 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c009cc0 con 0x7f4e1415d7b0 2023-11-26T07:21:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e0affd640 1 -- 172.21.15.31:0/1411313837 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c00f040 con 0x7f4e1415d7b0 2023-11-26T07:21:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e0affd640 1 -- 172.21.15.31:0/1411313837 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c0039a0 con 0x7f4e1415d7b0 2023-11-26T07:21:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/1411313837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e040094d0 con 0x7f4e1415d7b0 2023-11-26T07:21:39.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/1411313837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e141542a0 con 0x7f4e1415d7b0 2023-11-26T07:21:39.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.846+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/1411313837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e14155020 con 0x7f4e1415d7b0 2023-11-26T07:21:39.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.850+0000 7f4e0affd640 1 -- 172.21.15.31:0/1411313837 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f4e0c014480 con 0x7f4e1415d7b0 2023-11-26T07:21:39.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.850+0000 7f4e0affd640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4df80667d0 0x7f4df8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:39.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.850+0000 7f4e0affd640 1 -- 172.21.15.31:0/1411313837 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4e0c084c60 con 0x7f4e1415d7b0 2023-11-26T07:21:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.850+0000 7f4e1a560640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4df80667d0 0x7f4df8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:39.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.850+0000 7f4e1a560640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4df80667d0 0x7f4df8068c90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4e10007fd0 tx=0x7f4e10009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:39.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:39.854+0000 7f4e0affd640 1 -- 172.21.15.31:0/1411313837 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e0c051f30 con 0x7f4e1415d7b0 2023-11-26T07:21:40.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.074+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/1411313837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) v1 -- 0x7f4e1404e850 con 0x7f4e1415d7b0 2023-11-26T07:21:40.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e0affd640 1 -- 172.21.15.31:0/1411313837 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v221) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4e0c0180e0 con 0x7f4e1415d7b0 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4df80667d0 msgr2=0x7f4df8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4df80667d0 0x7f4df8068c90 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4e10007fd0 tx=0x7f4e10009040 comp rx=0 tx=0).stop 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/1411313837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e1415d7b0 msgr2=0x7f4e140b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e1415d7b0 0x7f4e140b1530 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00efc0 tx=0x7f4e0c00c490 comp rx=0 tx=0).stop 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/1411313837 shutdown_connections 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4df80667d0 0x7f4df8068c90 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e1415d7b0 0x7f4e140b1530 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e140d1eb0 0x7f4e14153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 --2- 172.21.15.31:0/1411313837 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e140cead0 0x7f4e14155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/1411313837 >> 172.21.15.31:0/1411313837 conn(0x7f4e140d33b0 msgr2=0x7f4e140d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:40.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/1411313837 shutdown_connections 2023-11-26T07:21:40.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.310+0000 7f4e1c7c4640 1 -- 172.21.15.31:0/1411313837 wait complete. 2023-11-26T07:21:40.334 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2023-11-26T07:21:40.334 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:21:40.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:40.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.486+0000 7fa7ffb57640 1 Processor -- start 2023-11-26T07:21:40.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.486+0000 7fa7ffb57640 1 -- start start 2023-11-26T07:21:40.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7ffb57640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa7f815a4c0 0x7fa7f80b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7ffb57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cf6e0 0x7fa7f80b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7ffb57640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa7f80b3590 0x7fa7f80ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7ffb57640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f80af9c0 con 0x7fa7f815a4c0 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7ffb57640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f80b05d0 con 0x7fa7f80b3590 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7ffb57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80b1b20 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7fd8f3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa7f815a4c0 0x7fa7f80b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7fd8f3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa7f815a4c0 0x7fa7f80b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37924/0 (socket says 172.21.15.31:37924) 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7fd8f3640 1 -- 172.21.15.31:0/1941099258 learned_addr learned my addr 172.21.15.31:0/1941099258 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7fe0f4640 1 --2- 172.21.15.31:0/1941099258 >> v2:172.21.15.97:3300/0 conn(0x7fa7f80b3590 0x7fa7f80ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7fd0f2640 1 --2- 172.21.15.31:0/1941099258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cf6e0 0x7fa7f80b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7fd8f3640 1 -- 172.21.15.31:0/1941099258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cf6e0 msgr2=0x7fa7f80b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7fd8f3640 1 --2- 172.21.15.31:0/1941099258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cf6e0 0x7fa7f80b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7fd8f3640 1 -- 172.21.15.31:0/1941099258 >> v2:172.21.15.97:3300/0 conn(0x7fa7f80b3590 msgr2=0x7fa7f80ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7fd8f3640 1 --2- 172.21.15.31:0/1941099258 >> v2:172.21.15.97:3300/0 conn(0x7fa7f80b3590 0x7fa7f80ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.490+0000 7fa7fd8f3640 1 -- 172.21.15.31:0/1941099258 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f80b0820 con 0x7fa7f815a4c0 2023-11-26T07:21:40.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7fd8f3640 1 --2- 172.21.15.31:0/1941099258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f815a4c0 0x7fa7f80b15e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80cf490 tx=0x7fa7f401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76abd81d3d455300 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7fc8f1640 1 -- 172.21.15.31:0/1941099258 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40aa5f0 con 0x7fa7f815a4c0 2023-11-26T07:21:40.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7fc8f1640 1 -- 172.21.15.31:0/1941099258 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f40aa750 con 0x7fa7f815a4c0 2023-11-26T07:21:40.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7fc8f1640 1 -- 172.21.15.31:0/1941099258 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40aaae0 con 0x7fa7f815a4c0 2023-11-26T07:21:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 -- 172.21.15.31:0/1941099258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f815a4c0 msgr2=0x7fa7f80b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 --2- 172.21.15.31:0/1941099258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f815a4c0 0x7fa7f80b15e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80cf490 tx=0x7fa7f401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 -- 172.21.15.31:0/1941099258 shutdown_connections 2023-11-26T07:21:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 --2- 172.21.15.31:0/1941099258 >> v2:172.21.15.97:3300/0 conn(0x7fa7f80b3590 0x7fa7f80ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 --2- 172.21.15.31:0/1941099258 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cf6e0 0x7fa7f80b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 --2- 172.21.15.31:0/1941099258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f815a4c0 0x7fa7f80b15e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 -- 172.21.15.31:0/1941099258 >> 172.21.15.31:0/1941099258 conn(0x7fa7f80b5570 msgr2=0x7fa7f80b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 -- 172.21.15.31:0/1941099258 shutdown_connections 2023-11-26T07:21:40.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 -- 172.21.15.31:0/1941099258 wait complete. 2023-11-26T07:21:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 Processor -- start 2023-11-26T07:21:40.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.494+0000 7fa7ffb57640 1 -- start start 2023-11-26T07:21:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ffb57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80b3590 0x7fa7f8147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ffb57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cf6e0 0x7fa7f81491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ffb57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f815a4c0 0x7fa7f8149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ffb57640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f8147aa0 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ffb57640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f814e9b0 con 0x7fa7f815a4c0 2023-11-26T07:21:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ffb57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f814ec90 con 0x7fa7f80b3590 2023-11-26T07:21:40.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fd0f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cf6e0 0x7fa7f81491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fd0f2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cf6e0 0x7fa7f81491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37928/0 (socket says 172.21.15.31:37928) 2023-11-26T07:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fd0f2640 1 -- 172.21.15.31:0/4170351798 learned_addr learned my addr 172.21.15.31:0/4170351798 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fd8f3640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80b3590 0x7fa7f8147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fe0f4640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f815a4c0 0x7fa7f8149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fd0f2640 1 -- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80b3590 msgr2=0x7fa7f8147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fd0f2640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80b3590 0x7fa7f8147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fd0f2640 1 -- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f815a4c0 msgr2=0x7fa7f8149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fd0f2640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f815a4c0 0x7fa7f8149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fd0f2640 1 -- 172.21.15.31:0/4170351798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40094d0 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7fd0f2640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cf6e0 0x7fa7f81491c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800d800 tx=0x7fa7e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ee7fc640 1 -- 172.21.15.31:0/4170351798 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800b6c0 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ee7fc640 1 -- 172.21.15.31:0/4170351798 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e800b820 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ffb57640 1 -- 172.21.15.31:0/4170351798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f814ef70 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ee7fc640 1 -- 172.21.15.31:0/4170351798 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800bb90 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ffb57640 1 -- 172.21.15.31:0/4170351798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f8145bf0 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.498+0000 7fa7ffb57640 1 -- 172.21.15.31:0/4170351798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f804e850 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.502+0000 7fa7ee7fc640 1 -- 172.21.15.31:0/4170351798 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fa7e800bcf0 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.502+0000 7fa7ee7fc640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7dc0667d0 0x7fa7dc068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.502+0000 7fa7ee7fc640 1 -- 172.21.15.31:0/4170351798 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa7e80845e0 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.502+0000 7fa7fd8f3640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7dc0667d0 0x7fa7dc068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.502+0000 7fa7fd8f3640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7dc0667d0 0x7fa7dc068c90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40ae000 tx=0x7fa7f40b5040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.506+0000 7fa7ee7fc640 1 -- 172.21.15.31:0/4170351798 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7e80519c0 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.666+0000 7fa7ffb57640 1 -- 172.21.15.31:0/4170351798 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7f8153530 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.666+0000 7fa7ee7fc640 1 -- 172.21.15.31:0/4170351798 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+12902 (secure 0 0 0) 0x7fa7e8054fe0 con 0x7fa7f80cf6e0 2023-11-26T07:21:40.669 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:40.669 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":60,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:39.304054+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6834/1053315203":"2023-11-27T07:21:36.292699+0000","172.21.15.31:6832/1053315203":"2023-11-27T07:21:36.292699+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 -- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7dc0667d0 msgr2=0x7fa7dc068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7dc0667d0 0x7fa7dc068c90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40ae000 tx=0x7fa7f40b5040 comp rx=0 tx=0).stop 2023-11-26T07:21:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 -- 172.21.15.31:0/4170351798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cf6e0 msgr2=0x7fa7f81491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cf6e0 0x7fa7f81491c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800d800 tx=0x7fa7e800dcc0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 -- 172.21.15.31:0/4170351798 shutdown_connections 2023-11-26T07:21:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa7dc0667d0 0x7fa7dc068c90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f815a4c0 0x7fa7f8149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cf6e0 0x7fa7f81491c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 --2- 172.21.15.31:0/4170351798 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80b3590 0x7fa7f8147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 -- 172.21.15.31:0/4170351798 >> 172.21.15.31:0/4170351798 conn(0x7fa7f80b5570 msgr2=0x7fa7f8153a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:40.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 -- 172.21.15.31:0/4170351798 shutdown_connections 2023-11-26T07:21:40.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.670+0000 7fa7ffb57640 1 -- 172.21.15.31:0/4170351798 wait complete. 2023-11-26T07:21:40.690 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:21:40.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:21:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.834+0000 7f3e951f5640 1 Processor -- start 2023-11-26T07:21:40.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.834+0000 7f3e951f5640 1 -- start start 2023-11-26T07:21:40.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.834+0000 7f3e951f5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e900cb9b0 0x7f3e9006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.834+0000 7f3e951f5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e900cbd90 0x7f3e9015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.834+0000 7f3e951f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900ce5d0 0x7f3e9015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.834+0000 7f3e951f5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9015bba0 con 0x7f3e900cbd90 2023-11-26T07:21:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.834+0000 7f3e951f5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e900cdd50 con 0x7f3e900cb9b0 2023-11-26T07:21:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.834+0000 7f3e951f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90159630 con 0x7f3e900ce5d0 2023-11-26T07:21:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e87fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e900cbd90 0x7f3e9015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e87fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3e900cbd90 0x7f3e9015f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37942/0 (socket says 172.21.15.31:37942) 2023-11-26T07:21:40.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e87fff640 1 -- 172.21.15.31:0/3693164431 learned_addr learned my addr 172.21.15.31:0/3693164431 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e8f59e640 1 --2- 172.21.15.31:0/3693164431 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900ce5d0 0x7f3e9015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e8ed9d640 1 --2- 172.21.15.31:0/3693164431 >> v2:172.21.15.97:3300/0 conn(0x7f3e900cb9b0 0x7f3e9006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e87fff640 1 -- 172.21.15.31:0/3693164431 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900ce5d0 msgr2=0x7f3e9015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e87fff640 1 --2- 172.21.15.31:0/3693164431 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900ce5d0 0x7f3e9015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e87fff640 1 -- 172.21.15.31:0/3693164431 >> v2:172.21.15.97:3300/0 conn(0x7f3e900cb9b0 msgr2=0x7f3e9006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:40.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e87fff640 1 --2- 172.21.15.31:0/3693164431 >> v2:172.21.15.97:3300/0 conn(0x7f3e900cb9b0 0x7f3e9006e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e87fff640 1 -- 172.21.15.31:0/3693164431 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9015f6d0 con 0x7f3e900cbd90 2023-11-26T07:21:40.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e8ed9d640 1 --2- 172.21.15.31:0/3693164431 >> v2:172.21.15.97:3300/0 conn(0x7f3e900cb9b0 0x7f3e9006e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:40.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e8f59e640 1 --2- 172.21.15.31:0/3693164431 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900ce5d0 0x7f3e9015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e87fff640 1 --2- 172.21.15.31:0/3693164431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 0x7f3e9015f190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3e9006c360 tx=0x7f3e7801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9674b303b5593108 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e8e59c640 1 -- 172.21.15.31:0/3693164431 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780ab670 con 0x7f3e900cbd90 2023-11-26T07:21:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e8e59c640 1 -- 172.21.15.31:0/3693164431 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e780ab7d0 con 0x7f3e900cbd90 2023-11-26T07:21:40.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.838+0000 7f3e8e59c640 1 -- 172.21.15.31:0/3693164431 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780abb60 con 0x7f3e900cbd90 2023-11-26T07:21:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 -- 172.21.15.31:0/3693164431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 msgr2=0x7f3e9015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 --2- 172.21.15.31:0/3693164431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 0x7f3e9015f190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f3e9006c360 tx=0x7f3e7801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 -- 172.21.15.31:0/3693164431 shutdown_connections 2023-11-26T07:21:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 --2- 172.21.15.31:0/3693164431 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900ce5d0 0x7f3e9015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 --2- 172.21.15.31:0/3693164431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 0x7f3e9015f190 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 --2- 172.21.15.31:0/3693164431 >> v2:172.21.15.97:3300/0 conn(0x7f3e900cb9b0 0x7f3e9006e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 -- 172.21.15.31:0/3693164431 >> 172.21.15.31:0/3693164431 conn(0x7f3e900b0cb0 msgr2=0x7f3e900b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 -- 172.21.15.31:0/3693164431 shutdown_connections 2023-11-26T07:21:40.844 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 -- 172.21.15.31:0/3693164431 wait complete. 2023-11-26T07:21:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 Processor -- start 2023-11-26T07:21:40.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 -- start start 2023-11-26T07:21:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900cb9b0 0x7f3e90158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 0x7f3e9009c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e900ce5d0 0x7f3e9009c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e901f6670 con 0x7f3e900cbd90 2023-11-26T07:21:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9009cbc0 con 0x7f3e900ce5d0 2023-11-26T07:21:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e951f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e901590d0 con 0x7f3e900cb9b0 2023-11-26T07:21:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e87fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 0x7f3e9009c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e87fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 0x7f3e9009c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37954/0 (socket says 172.21.15.31:37954) 2023-11-26T07:21:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e87fff640 1 -- 172.21.15.31:0/3779891166 learned_addr learned my addr 172.21.15.31:0/3779891166 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e8f59e640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e900ce5d0 0x7f3e9009c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.842+0000 7f3e8ed9d640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900cb9b0 0x7f3e90158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e87fff640 1 -- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900cb9b0 msgr2=0x7f3e90158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e87fff640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900cb9b0 0x7f3e90158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e87fff640 1 -- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e900ce5d0 msgr2=0x7f3e9009c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e87fff640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e900ce5d0 0x7f3e9009c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e87fff640 1 -- 172.21.15.31:0/3779891166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e780094d0 con 0x7f3e900cbd90 2023-11-26T07:21:40.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e8ed9d640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900cb9b0 0x7f3e90158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e87fff640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 0x7f3e9009c140 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3e7801ab80 tx=0x7f3e7801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e877fe640 1 -- 172.21.15.31:0/3779891166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780b48c0 con 0x7f3e900cbd90 2023-11-26T07:21:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e877fe640 1 -- 172.21.15.31:0/3779891166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e780b4a20 con 0x7f3e900cbd90 2023-11-26T07:21:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e877fe640 1 -- 172.21.15.31:0/3779891166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780b4db0 con 0x7f3e900cbd90 2023-11-26T07:21:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e951f5640 1 -- 172.21.15.31:0/3779891166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e900b2130 con 0x7f3e900cbd90 2023-11-26T07:21:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e951f5640 1 -- 172.21.15.31:0/3779891166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e900b26f0 con 0x7f3e900cbd90 2023-11-26T07:21:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e951f5640 1 -- 172.21.15.31:0/3779891166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e900671f0 con 0x7f3e900cbd90 2023-11-26T07:21:40.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e877fe640 1 -- 172.21.15.31:0/3779891166 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f3e780b3380 con 0x7f3e900cbd90 2023-11-26T07:21:40.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e877fe640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e700667d0 0x7f3e70068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:40.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e877fe640 1 -- 172.21.15.31:0/3779891166 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e7811eae0 con 0x7f3e900cbd90 2023-11-26T07:21:40.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.846+0000 7f3e8ed9d640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e700667d0 0x7f3e70068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:40.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.850+0000 7f3e8ed9d640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e700667d0 0x7f3e70068c90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3e80006fd0 tx=0x7f3e80008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:40.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:40.850+0000 7f3e877fe640 1 -- 172.21.15.31:0/3779891166 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e780ebe40 con 0x7f3e900cbd90 2023-11-26T07:21:41.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 -- 172.21.15.31:0/3779891166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e9006b460 con 0x7f3e900cbd90 2023-11-26T07:21:41.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e877fe640 1 -- 172.21.15.31:0/3779891166 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e780c40a0 con 0x7f3e900cbd90 2023-11-26T07:21:41.011 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:21:41.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 -- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e700667d0 msgr2=0x7f3e70068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e700667d0 0x7f3e70068c90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3e80006fd0 tx=0x7f3e80008040 comp rx=0 tx=0).stop 2023-11-26T07:21:41.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 -- 172.21.15.31:0/3779891166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 msgr2=0x7f3e9009c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 0x7f3e9009c140 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3e7801ab80 tx=0x7f3e7801ad10 comp rx=0 tx=0).stop 2023-11-26T07:21:41.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 -- 172.21.15.31:0/3779891166 shutdown_connections 2023-11-26T07:21:41.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3e700667d0 0x7f3e70068c90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e900ce5d0 0x7f3e9009c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e900cbd90 0x7f3e9009c140 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 --2- 172.21.15.31:0/3779891166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e900cb9b0 0x7f3e90158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 -- 172.21.15.31:0/3779891166 >> 172.21.15.31:0/3779891166 conn(0x7f3e900b0cb0 msgr2=0x7f3e90071690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:41.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 -- 172.21.15.31:0/3779891166 shutdown_connections 2023-11-26T07:21:41.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.010+0000 7f3e951f5640 1 -- 172.21.15.31:0/3779891166 wait complete. 2023-11-26T07:21:41.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2023-11-26T07:21:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c1ac68640 1 Processor -- start 2023-11-26T07:21:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c1ac68640 1 -- start start 2023-11-26T07:21:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c1ac68640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c140cead0 0x7f6c140d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c1ac68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c140d1eb0 0x7f6c1406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c1ac68640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c140d2300 con 0x7f6c1415d7b0 2023-11-26T07:21:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c1ac68640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c140d17e0 con 0x7f6c140cead0 2023-11-26T07:21:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c1ac68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c140cefd0 con 0x7f6c140d1eb0 2023-11-26T07:21:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c13fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c140cead0 0x7f6c140d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c13fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c140cead0 0x7f6c140d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37958/0 (socket says 172.21.15.31:37958) 2023-11-26T07:21:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c13fff640 1 -- 172.21.15.31:0/4011545362 learned_addr learned my addr 172.21.15.31:0/4011545362 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c19205640 1 --2- 172.21.15.31:0/4011545362 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c140d1eb0 0x7f6c1406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c13fff640 1 -- 172.21.15.31:0/4011545362 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c140d1eb0 msgr2=0x7f6c1406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c13fff640 1 --2- 172.21.15.31:0/4011545362 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c140d1eb0 0x7f6c1406bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c13fff640 1 -- 172.21.15.31:0/4011545362 >> v1:172.21.15.97:6789/0 conn(0x7f6c1415d7b0 legacy=0x7f6c140d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.174+0000 7f6c13fff640 1 -- 172.21.15.31:0/4011545362 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c140cf2b0 con 0x7f6c140cead0 2023-11-26T07:21:41.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c13fff640 1 --2- 172.21.15.31:0/4011545362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140cead0 0x7f6c140d1970 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009870 tx=0x7f6c0001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a463d08f98966dcb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c137fe640 1 -- 172.21.15.31:0/4011545362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000ab5f0 con 0x7f6c140cead0 2023-11-26T07:21:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c137fe640 1 -- 172.21.15.31:0/4011545362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c000ab750 con 0x7f6c140cead0 2023-11-26T07:21:41.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c137fe640 1 -- 172.21.15.31:0/4011545362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000abae0 con 0x7f6c140cead0 2023-11-26T07:21:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c1ac68640 1 -- 172.21.15.31:0/4011545362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140cead0 msgr2=0x7f6c140d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c1ac68640 1 --2- 172.21.15.31:0/4011545362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140cead0 0x7f6c140d1970 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009870 tx=0x7f6c0001c3f0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c1ac68640 1 -- 172.21.15.31:0/4011545362 shutdown_connections 2023-11-26T07:21:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c1ac68640 1 --2- 172.21.15.31:0/4011545362 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c140d1eb0 0x7f6c1406bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c1ac68640 1 --2- 172.21.15.31:0/4011545362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140cead0 0x7f6c140d1970 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c1ac68640 1 -- 172.21.15.31:0/4011545362 >> 172.21.15.31:0/4011545362 conn(0x7f6c140d33b0 msgr2=0x7f6c140d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c1ac68640 1 -- 172.21.15.31:0/4011545362 shutdown_connections 2023-11-26T07:21:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.178+0000 7f6c1ac68640 1 -- 172.21.15.31:0/4011545362 wait complete. 2023-11-26T07:21:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c1ac68640 1 Processor -- start 2023-11-26T07:21:41.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c1ac68640 1 -- start start 2023-11-26T07:21:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c1ac68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c140cead0 0x7f6c140c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c1ac68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140d1eb0 0x7f6c14154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c1ac68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c1415d7b0 0x7f6c14154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c1ac68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c140b6710 con 0x7f6c140d1eb0 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c1ac68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c140b8e50 con 0x7f6c140cead0 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c1ac68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c140b6430 con 0x7f6c1415d7b0 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140d1eb0 0x7f6c14154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140d1eb0 0x7f6c14154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37964/0 (socket says 172.21.15.31:37964) 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c13fff640 1 -- 172.21.15.31:0/3049257934 learned_addr learned my addr 172.21.15.31:0/3049257934 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c18a04640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c140cead0 0x7f6c140c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c19205640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c1415d7b0 0x7f6c14154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c13fff640 1 -- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c1415d7b0 msgr2=0x7f6c14154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c13fff640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c1415d7b0 0x7f6c14154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c13fff640 1 -- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c140cead0 msgr2=0x7f6c140c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c13fff640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c140cead0 0x7f6c140c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c13fff640 1 -- 172.21.15.31:0/3049257934 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c000094d0 con 0x7f6c140d1eb0 2023-11-26T07:21:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c13fff640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140d1eb0 0x7f6c14154040 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6c000ad040 tx=0x7f6c0001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c117fa640 1 -- 172.21.15.31:0/3049257934 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000be5f0 con 0x7f6c140d1eb0 2023-11-26T07:21:41.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c117fa640 1 -- 172.21.15.31:0/3049257934 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c000be750 con 0x7f6c140d1eb0 2023-11-26T07:21:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c117fa640 1 -- 172.21.15.31:0/3049257934 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000beb00 con 0x7f6c140d1eb0 2023-11-26T07:21:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c1ac68640 1 -- 172.21.15.31:0/3049257934 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c140b8800 con 0x7f6c140d1eb0 2023-11-26T07:21:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.182+0000 7f6c1ac68640 1 -- 172.21.15.31:0/3049257934 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c14155280 con 0x7f6c140d1eb0 2023-11-26T07:21:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.186+0000 7f6c1ac68640 1 -- 172.21.15.31:0/3049257934 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c140b8fe0 con 0x7f6c140d1eb0 2023-11-26T07:21:41.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.186+0000 7f6c117fa640 1 -- 172.21.15.31:0/3049257934 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f6c000b43f0 con 0x7f6c140d1eb0 2023-11-26T07:21:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.186+0000 7f6c117fa640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf8066800 0x7f6bf8068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.186+0000 7f6c117fa640 1 -- 172.21.15.31:0/3049257934 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6c000a8050 con 0x7f6c140d1eb0 2023-11-26T07:21:41.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.186+0000 7f6c18a04640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf8066800 0x7f6bf8068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:41.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.186+0000 7f6c18a04640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf8066800 0x7f6bf8068cc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6c08007fd0 tx=0x7f6c08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:41.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.190+0000 7f6c117fa640 1 -- 172.21.15.31:0/3049257934 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c000ec2b0 con 0x7f6c140d1eb0 2023-11-26T07:21:41.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:41.406+0000 7f6c1ac68640 1 -- 172.21.15.31:0/3049257934 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7f6c140d2fc0 con 0x7f6c140d1eb0 2023-11-26T07:21:42.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c117fa640 1 -- 172.21.15.31:0/3049257934 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v223) v1 ==== 134+0+0 (secure 0 0 0) 0x7f6c000ef8d0 con 0x7f6c140d1eb0 2023-11-26T07:21:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 -- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf8066800 msgr2=0x7f6bf8068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf8066800 0x7f6bf8068cc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6c08007fd0 tx=0x7f6c08009040 comp rx=0 tx=0).stop 2023-11-26T07:21:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 -- 172.21.15.31:0/3049257934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140d1eb0 msgr2=0x7f6c14154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140d1eb0 0x7f6c14154040 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6c000ad040 tx=0x7f6c0001ac30 comp rx=0 tx=0).stop 2023-11-26T07:21:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 -- 172.21.15.31:0/3049257934 shutdown_connections 2023-11-26T07:21:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f6bf8066800 0x7f6bf8068cc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6c1415d7b0 0x7f6c14154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c140d1eb0 0x7f6c14154040 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 --2- 172.21.15.31:0/3049257934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c140cead0 0x7f6c140c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 -- 172.21.15.31:0/3049257934 >> 172.21.15.31:0/3049257934 conn(0x7f6c140d33b0 msgr2=0x7f6c140d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 -- 172.21.15.31:0/3049257934 shutdown_connections 2023-11-26T07:21:42.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.322+0000 7f6c1ac68640 1 -- 172.21.15.31:0/3049257934 wait complete. 2023-11-26T07:21:42.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:21:42.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ef5aa640 1 Processor -- start 2023-11-26T07:21:42.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ef5aa640 1 -- start start 2023-11-26T07:21:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ef5aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50e80cdfb0 0x7f50e80abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ef5aa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e80b2710 con 0x7f50e815a710 2023-11-26T07:21:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ef5aa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e80b29f0 con 0x7f50e80cbd50 2023-11-26T07:21:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ef5aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80cca80 con 0x7f50e80cdfb0 2023-11-26T07:21:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ecb45640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f50e815a710 0x7f50e80ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55204/0 (socket says 172.21.15.31:55204) 2023-11-26T07:21:42.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ecb45640 1 -- 172.21.15.31:0/2925104819 learned_addr learned my addr 172.21.15.31:0/2925104819 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:42.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385981603 0 0) 0x7f50e80b2710 con 0x7f50e815a710 2023-11-26T07:21:42.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d4003530 con 0x7f50e815a710 2023-11-26T07:21:42.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ed346640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50e80cdfb0 0x7f50e80abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1934602692 0 0) 0x7f50d4003530 con 0x7f50e815a710 2023-11-26T07:21:42.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50e80cdfb0 msgr2=0x7f50e80abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:42.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50e80cdfb0 0x7f50e80abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 >> v1:172.21.15.97:6789/0 conn(0x7f50e80cbd50 legacy=0x7f50e80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e80b2cd0 con 0x7f50e815a710 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f50d8001680 con 0x7f50e815a710 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ed346640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50e80cdfb0 0x7f50e80abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 >> v1:172.21.15.31:6789/0 conn(0x7f50e815a710 legacy=0x7f50e80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50d40039b0 0x7f50d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d40047d0 0x7f50d4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50e80cdfb0 0x7f50d40051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e80cca80 con 0x7f50d40047d0 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d400bb20 con 0x7f50d40039b0 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d400bc90 con 0x7f50e80cdfb0 2023-11-26T07:21:42.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ecb45640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d40047d0 0x7f50d4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50ed346640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50d40039b0 0x7f50d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.498+0000 7f50edb47640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50e80cdfb0 0x7f50d40051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50ecb45640 1 -- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50e80cdfb0 msgr2=0x7f50d40051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50ecb45640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50e80cdfb0 0x7f50d40051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50ecb45640 1 -- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50d40039b0 msgr2=0x7f50d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50ecb45640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50d40039b0 0x7f50d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50ecb45640 1 -- 172.21.15.31:0/2925104819 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d4003530 con 0x7f50d40047d0 2023-11-26T07:21:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50ecb45640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d40047d0 0x7f50d4004c50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f50e815ab10 tx=0x7f50d80214a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed1f2ad97e20139 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8005ae0 con 0x7f50d40047d0 2023-11-26T07:21:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d80ad070 con 0x7f50d40047d0 2023-11-26T07:21:42.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50dffff640 1 -- 172.21.15.31:0/2925104819 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d8007070 con 0x7f50d40047d0 2023-11-26T07:21:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50ef5aa640 1 -- 172.21.15.31:0/2925104819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d40047d0 msgr2=0x7f50d4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50ef5aa640 1 --2- 172.21.15.31:0/2925104819 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d40047d0 0x7f50d4004c50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f50e815ab10 tx=0x7f50d80214a0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.502+0000 7f50edb47640 1 -- 172.21.15.31:0/2925104819 reap_dead start 2023-11-26T07:21:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 -- 172.21.15.31:0/2925104819 shutdown_connections 2023-11-26T07:21:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 -- 172.21.15.31:0/2925104819 >> 172.21.15.31:0/2925104819 conn(0x7f50e80b0cb0 msgr2=0x7f50e80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 -- 172.21.15.31:0/2925104819 shutdown_connections 2023-11-26T07:21:42.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 -- 172.21.15.31:0/2925104819 wait complete. 2023-11-26T07:21:42.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 Processor -- start 2023-11-26T07:21:42.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 -- start start 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d40047d0 0x7f50e8150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e815aaf0 0x7f50e81f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e80c9730 0x7f50e80c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e815b060 con 0x7f50e80c9730 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e815b340 con 0x7f50e815aaf0 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ef5aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e815b620 con 0x7f50d40047d0 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50edb47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e80c9730 0x7f50e80c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50edb47640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e80c9730 0x7f50e80c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37982/0 (socket says 172.21.15.31:37982) 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50edb47640 1 -- 172.21.15.31:0/1220081491 learned_addr learned my addr 172.21.15.31:0/1220081491 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ed346640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d40047d0 0x7f50e8150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ecb45640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e815aaf0 0x7f50e81f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50edb47640 1 -- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d40047d0 msgr2=0x7f50e8150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50edb47640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d40047d0 0x7f50e8150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50edb47640 1 -- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e815aaf0 msgr2=0x7f50e81f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50edb47640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e815aaf0 0x7f50e81f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50edb47640 1 -- 172.21.15.31:0/1220081491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e815b900 con 0x7f50e80c9730 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50ecb45640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e815aaf0 0x7f50e81f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.506+0000 7f50edb47640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e80c9730 0x7f50e80c9f90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f50e80d0040 tx=0x7f50e000fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.510+0000 7f50ddffb640 1 -- 172.21.15.31:0/1220081491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0006d70 con 0x7f50e80c9730 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.510+0000 7f50ddffb640 1 -- 172.21.15.31:0/1220081491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0003980 con 0x7f50e80c9730 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.510+0000 7f50ef5aa640 1 -- 172.21.15.31:0/1220081491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e815bbe0 con 0x7f50e80c9730 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.510+0000 7f50ddffb640 1 -- 172.21.15.31:0/1220081491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0003d30 con 0x7f50e80c9730 2023-11-26T07:21:42.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.510+0000 7f50ef5aa640 1 -- 172.21.15.31:0/1220081491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e80c9be0 con 0x7f50e80c9730 2023-11-26T07:21:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.510+0000 7f50ef5aa640 1 -- 172.21.15.31:0/1220081491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e80671f0 con 0x7f50e80c9730 2023-11-26T07:21:42.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.510+0000 7f50ddffb640 1 -- 172.21.15.31:0/1220081491 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f50e0005070 con 0x7f50e80c9730 2023-11-26T07:21:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.510+0000 7f50ddffb640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50cc066800 0x7f50cc068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.510+0000 7f50ddffb640 1 -- 172.21.15.31:0/1220081491 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f50e0083810 con 0x7f50e80c9730 2023-11-26T07:21:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.510+0000 7f50ed346640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50cc066800 0x7f50cc068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:42.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.514+0000 7f50ed346640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50cc066800 0x7f50cc068cc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f50e815e210 tx=0x7f50e4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:42.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.514+0000 7f50ddffb640 1 -- 172.21.15.31:0/1220081491 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50e0050bf0 con 0x7f50e80c9730 2023-11-26T07:21:42.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.674+0000 7f50ef5aa640 1 -- 172.21.15.31:0/1220081491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f50e806b460 con 0x7f50e80c9730 2023-11-26T07:21:42.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.678+0000 7f50ddffb640 1 -- 172.21.15.31:0/1220081491 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f50e0054210 con 0x7f50e80c9730 2023-11-26T07:21:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 -- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50cc066800 msgr2=0x7f50cc068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50cc066800 0x7f50cc068cc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f50e815e210 tx=0x7f50e4008040 comp rx=0 tx=0).stop 2023-11-26T07:21:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 -- 172.21.15.31:0/1220081491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e80c9730 msgr2=0x7f50e80c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e80c9730 0x7f50e80c9f90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f50e80d0040 tx=0x7f50e000fcd0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 -- 172.21.15.31:0/1220081491 shutdown_connections 2023-11-26T07:21:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50cc066800 0x7f50cc068cc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50e80c9730 0x7f50e80c9f90 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50e815aaf0 0x7f50e81f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 --2- 172.21.15.31:0/1220081491 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d40047d0 0x7f50e8150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:42.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 -- 172.21.15.31:0/1220081491 >> 172.21.15.31:0/1220081491 conn(0x7f50e80b0cb0 msgr2=0x7f50e815e760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 -- 172.21.15.31:0/1220081491 shutdown_connections 2023-11-26T07:21:42.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:42.682+0000 7f50ef5aa640 1 -- 172.21.15.31:0/1220081491 wait complete. 2023-11-26T07:21:42.704 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:21:42.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:21:42.757 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:21:42.828 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:21:42.828 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:21:42.828 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:42.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:21:42.837 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:21:42.838 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:42.847 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:21:42.847 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:21:42.847 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:42.886 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:21:42.886 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:21:42.886 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:42.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:21:42.898 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:21:42.898 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:43.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.070+0000 7f9216b80640 1 Processor -- start 2023-11-26T07:21:43.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.070+0000 7f9216b80640 1 -- start start 2023-11-26T07:21:43.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.070+0000 7f9216b80640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9210144c80 0x7f92100abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.070+0000 7f9216b80640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9210070df0 0x7f921006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.070+0000 7f9216b80640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9210145060 con 0x7f92100ac460 2023-11-26T07:21:43.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.070+0000 7f9216b80640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9210071fd0 con 0x7f9210144c80 2023-11-26T07:21:43.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.070+0000 7f9216b80640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92100ac8c0 con 0x7f9210070df0 2023-11-26T07:21:43.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.070+0000 7f921491c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9210144c80 0x7f92100abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.070+0000 7f921491c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9210144c80 0x7f92100abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37998/0 (socket says 172.21.15.31:37998) 2023-11-26T07:21:43.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.070+0000 7f921491c640 1 -- 172.21.15.31:0/3190826871 learned_addr learned my addr 172.21.15.31:0/3190826871 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.074+0000 7f921491c640 1 -- 172.21.15.31:0/3190826871 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9210070df0 msgr2=0x7f921006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.074+0000 7f921511d640 1 --2- 172.21.15.31:0/3190826871 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9210070df0 0x7f921006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.074+0000 7f921491c640 1 --2- 172.21.15.31:0/3190826871 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9210070df0 0x7f921006c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.074+0000 7f921491c640 1 -- 172.21.15.31:0/3190826871 >> v1:172.21.15.97:6789/0 conn(0x7f92100ac460 legacy=0x7f92100706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.074+0000 7f921491c640 1 -- 172.21.15.31:0/3190826871 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9210071450 con 0x7f9210144c80 2023-11-26T07:21:43.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.074+0000 7f921511d640 1 --2- 172.21.15.31:0/3190826871 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9210070df0 0x7f921006c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:43.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.074+0000 7f921491c640 1 --2- 172.21.15.31:0/3190826871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210144c80 0x7f92100abf20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9204009870 tx=0x7f920401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9eb4fd9c7fc170 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.074+0000 7f920f7fe640 1 -- 172.21.15.31:0/3190826871 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92040ac410 con 0x7f9210144c80 2023-11-26T07:21:43.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.074+0000 7f920f7fe640 1 -- 172.21.15.31:0/3190826871 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92040ac570 con 0x7f9210144c80 2023-11-26T07:21:43.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.074+0000 7f920f7fe640 1 -- 172.21.15.31:0/3190826871 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92040ac900 con 0x7f9210144c80 2023-11-26T07:21:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 -- 172.21.15.31:0/3190826871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210144c80 msgr2=0x7f92100abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 --2- 172.21.15.31:0/3190826871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210144c80 0x7f92100abf20 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f9204009870 tx=0x7f920401c500 comp rx=0 tx=0).stop 2023-11-26T07:21:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 -- 172.21.15.31:0/3190826871 shutdown_connections 2023-11-26T07:21:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 --2- 172.21.15.31:0/3190826871 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9210070df0 0x7f921006c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 --2- 172.21.15.31:0/3190826871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210144c80 0x7f92100abf20 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 -- 172.21.15.31:0/3190826871 >> 172.21.15.31:0/3190826871 conn(0x7f92100b0c40 msgr2=0x7f92100b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:43.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 -- 172.21.15.31:0/3190826871 shutdown_connections 2023-11-26T07:21:43.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 -- 172.21.15.31:0/3190826871 wait complete. 2023-11-26T07:21:43.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 Processor -- start 2023-11-26T07:21:43.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 -- start start 2023-11-26T07:21:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210070df0 0x7f92100b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f92100ac460 0x7f921015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9210144c80 0x7f921013f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92100b3ba0 con 0x7f9210070df0 2023-11-26T07:21:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92100b3e80 con 0x7f9210144c80 2023-11-26T07:21:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f9216b80640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9210142650 con 0x7f92100ac460 2023-11-26T07:21:43.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f921491c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210070df0 0x7f92100b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f921511d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9210144c80 0x7f921013f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f921491c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210070df0 0x7f92100b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38000/0 (socket says 172.21.15.31:38000) 2023-11-26T07:21:43.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f921491c640 1 -- 172.21.15.31:0/3572787354 learned_addr learned my addr 172.21.15.31:0/3572787354 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:43.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.078+0000 7f920ffff640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f92100ac460 0x7f921015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f921491c640 1 -- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f92100ac460 msgr2=0x7f921015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f921491c640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f92100ac460 0x7f921015a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f921491c640 1 -- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9210144c80 msgr2=0x7f921013f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f921491c640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9210144c80 0x7f921013f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f921491c640 1 -- 172.21.15.31:0/3572787354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040094d0 con 0x7f9210070df0 2023-11-26T07:21:43.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f921491c640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210070df0 0x7f92100b3590 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f92040af040 tx=0x7f92040192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f920d7fa640 1 -- 172.21.15.31:0/3572787354 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92040ac410 con 0x7f9210070df0 2023-11-26T07:21:43.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f920d7fa640 1 -- 172.21.15.31:0/3572787354 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92040b65f0 con 0x7f9210070df0 2023-11-26T07:21:43.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f920d7fa640 1 -- 172.21.15.31:0/3572787354 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92040b69a0 con 0x7f9210070df0 2023-11-26T07:21:43.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f9216b80640 1 -- 172.21.15.31:0/3572787354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9210142930 con 0x7f9210070df0 2023-11-26T07:21:43.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f9216b80640 1 -- 172.21.15.31:0/3572787354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921015b1a0 con 0x7f9210070df0 2023-11-26T07:21:43.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f9216b80640 1 -- 172.21.15.31:0/3572787354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92100b1ba0 con 0x7f9210070df0 2023-11-26T07:21:43.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f920d7fa640 1 -- 172.21.15.31:0/3572787354 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f92040b6b00 con 0x7f9210070df0 2023-11-26T07:21:43.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f920d7fa640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91f40667d0 0x7f91f4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.082+0000 7f920d7fa640 1 -- 172.21.15.31:0/3572787354 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f920411ea20 con 0x7f9210070df0 2023-11-26T07:21:43.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.086+0000 7f920ffff640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91f40667d0 0x7f91f4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.086+0000 7f920ffff640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91f40667d0 0x7f91f4068c90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f92100b3ad0 tx=0x7f91fc007e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.086+0000 7f920d7fa640 1 -- 172.21.15.31:0/3572787354 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f92040ebf20 con 0x7f9210070df0 2023-11-26T07:21:43.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 -- 172.21.15.31:0/3572787354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f921004e850 con 0x7f9210070df0 2023-11-26T07:21:43.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f920d7fa640 1 -- 172.21.15.31:0/3572787354 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f92040b1020 con 0x7f9210070df0 2023-11-26T07:21:43.284 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:43.284 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/3429205589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3429205589},{"type":"v1","addr":"172.21.15.97:6836","nonce":3429205589}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5959,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3569564413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3569564413},{"type":"v1","addr":"172.21.15.31:6835","nonce":3569564413}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":5965,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/690524463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":690524463},{"type":"v1","addr":"172.21.15.97:6837","nonce":690524463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6528,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/776235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":776235547},{"type":"v1","addr":"172.21.15.31:6834","nonce":776235547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[]} 2023-11-26T07:21:43.284 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 40 2023-11-26T07:21:43.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 -- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91f40667d0 msgr2=0x7f91f4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91f40667d0 0x7f91f4068c90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f92100b3ad0 tx=0x7f91fc007e40 comp rx=0 tx=0).stop 2023-11-26T07:21:43.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 -- 172.21.15.31:0/3572787354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210070df0 msgr2=0x7f92100b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210070df0 0x7f92100b3590 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f92040af040 tx=0x7f92040192a0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 -- 172.21.15.31:0/3572787354 shutdown_connections 2023-11-26T07:21:43.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f91f40667d0 0x7f91f4068c90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9210144c80 0x7f921013f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f92100ac460 0x7f921015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 --2- 172.21.15.31:0/3572787354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9210070df0 0x7f92100b3590 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 -- 172.21.15.31:0/3572787354 >> 172.21.15.31:0/3572787354 conn(0x7f92100b0c40 msgr2=0x7f9210141cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:43.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 -- 172.21.15.31:0/3572787354 shutdown_connections 2023-11-26T07:21:43.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.282+0000 7f9216b80640 1 -- 172.21.15.31:0/3572787354 wait complete. 2023-11-26T07:21:43.306 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:21:43.306 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:21:43.306 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:21:43.306 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:21:43.306 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:21:43.306 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:21:43.307 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:21:43.307 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:21:43.307 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:21:43.307 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:21:43.307 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:21:43.307 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:21:43.307 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:21:43.308 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:21:43.308 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:21:43.308 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:21:43.319 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:21:43.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:21:43.320 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:21:43.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:21:43.325 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:21:43.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:21:43.326 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:21:43.326 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:21:43.365 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:21:43.365 DEBUG:teuthology.parallel:result is None 2023-11-26T07:21:43.367 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:21:43.367 DEBUG:teuthology.parallel:result is None 2023-11-26T07:21:43.373 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:21:43.373 DEBUG:teuthology.parallel:result is None 2023-11-26T07:21:43.375 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:21:43.375 DEBUG:teuthology.parallel:result is None 2023-11-26T07:21:43.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:21:43.456 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-11-26T07:21:43.457 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-11-26T07:21:43.463 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2023-11-26T07:21:43.463 INFO:tasks.ceph.mds.d.smithi097.stdout:starting mds.d at 2023-11-26T07:21:43.465 INFO:tasks.ceph.mds.d.smithi097.stderr:2023-11-26T07:21:43.460+0000 7fbaa90f2640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-26T07:21:43.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.522+0000 7fb2c7c06640 1 Processor -- start 2023-11-26T07:21:43.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.522+0000 7fb2c7c06640 1 -- start start 2023-11-26T07:21:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c7c06640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c015a710 0x7fb2c00abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c7c06640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb2c00cbd50 0x7fb2c00cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c7c06640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c00b2710 con 0x7fb2c00cdfb0 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c7c06640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c00cca80 con 0x7fb2c00cbd50 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c7c06640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c00b29f0 con 0x7fb2c015a710 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c59a2640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb2c00cdfb0 0x7fb2c00ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:55222/0 (socket says 172.21.15.31:55222) 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c59a2640 1 -- 172.21.15.31:0/453023244 learned_addr learned my addr 172.21.15.31:0/453023244 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c61a3640 1 --2- 172.21.15.31:0/453023244 >> v2:172.21.15.31:3300/0 conn(0x7fb2c00cbd50 0x7fb2c00cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c51a1640 1 --2- 172.21.15.31:0/453023244 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c015a710 0x7fb2c00abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c49a0640 1 -- 172.21.15.31:0/453023244 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200226364 0 0) 0x7fb2c00b2710 con 0x7fb2c00cdfb0 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c49a0640 1 -- 172.21.15.31:0/453023244 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2ac003530 con 0x7fb2c00cdfb0 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c61a3640 1 -- 172.21.15.31:0/453023244 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c015a710 msgr2=0x7fb2c00abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:43.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c61a3640 1 --2- 172.21.15.31:0/453023244 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c015a710 0x7fb2c00abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c61a3640 1 -- 172.21.15.31:0/453023244 >> v1:172.21.15.31:6789/0 conn(0x7fb2c00cdfb0 legacy=0x7fb2c00ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c61a3640 1 -- 172.21.15.31:0/453023244 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c00b2cd0 con 0x7fb2c00cbd50 2023-11-26T07:21:43.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.526+0000 7fb2c51a1640 1 --2- 172.21.15.31:0/453023244 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c015a710 0x7fb2c00abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:43.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c61a3640 1 --2- 172.21.15.31:0/453023244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00cbd50 0x7fb2c00cc1e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009d90 tx=0x7fb2b801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fd30080ac6b1c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c49a0640 1 -- 172.21.15.31:0/453023244 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80a5a00 con 0x7fb2c00cbd50 2023-11-26T07:21:43.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c49a0640 1 -- 172.21.15.31:0/453023244 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b80a5b60 con 0x7fb2c00cbd50 2023-11-26T07:21:43.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c49a0640 1 -- 172.21.15.31:0/453023244 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80ad440 con 0x7fb2c00cbd50 2023-11-26T07:21:43.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c7c06640 1 -- 172.21.15.31:0/453023244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00cbd50 msgr2=0x7fb2c00cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c7c06640 1 --2- 172.21.15.31:0/453023244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00cbd50 0x7fb2c00cc1e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8009d90 tx=0x7fb2b801c920 comp rx=0 tx=0).stop 2023-11-26T07:21:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c7c06640 1 -- 172.21.15.31:0/453023244 shutdown_connections 2023-11-26T07:21:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c7c06640 1 --2- 172.21.15.31:0/453023244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c00cbd50 0x7fb2c00cc1e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c7c06640 1 --2- 172.21.15.31:0/453023244 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c015a710 0x7fb2c00abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c7c06640 1 -- 172.21.15.31:0/453023244 >> 172.21.15.31:0/453023244 conn(0x7fb2c00b0cb0 msgr2=0x7fb2c00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c7c06640 1 -- 172.21.15.31:0/453023244 shutdown_connections 2023-11-26T07:21:43.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c7c06640 1 -- 172.21.15.31:0/453023244 wait complete. 2023-11-26T07:21:43.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c7c06640 1 Processor -- start 2023-11-26T07:21:43.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.530+0000 7fb2c7c06640 1 -- start start 2023-11-26T07:21:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c7c06640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00cbd50 0x7fb2c009c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c7c06640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c00cdfb0 0x7fb2c00ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c7c06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c015a710 0x7fb2c0140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c7c06640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c00ad040 con 0x7fb2c015a710 2023-11-26T07:21:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c7c06640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c00acd60 con 0x7fb2c00cbd50 2023-11-26T07:21:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c7c06640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c0154650 con 0x7fb2c00cdfb0 2023-11-26T07:21:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c61a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c015a710 0x7fb2c0140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c61a3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c015a710 0x7fb2c0140850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38032/0 (socket says 172.21.15.31:38032) 2023-11-26T07:21:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c61a3640 1 -- 172.21.15.31:0/3577674788 learned_addr learned my addr 172.21.15.31:0/3577674788 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c51a1640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c00cdfb0 0x7fb2c00ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c59a2640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00cbd50 0x7fb2c009c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c61a3640 1 -- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c00cdfb0 msgr2=0x7fb2c00ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c61a3640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c00cdfb0 0x7fb2c00ac510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c61a3640 1 -- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00cbd50 msgr2=0x7fb2c009c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c61a3640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00cbd50 0x7fb2c009c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c61a3640 1 -- 172.21.15.31:0/3577674788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b0009420 con 0x7fb2c015a710 2023-11-26T07:21:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c61a3640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c015a710 0x7fb2c0140850 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80b0040 tx=0x7fb2b80196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2b67fc640 1 -- 172.21.15.31:0/3577674788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80adce0 con 0x7fb2c015a710 2023-11-26T07:21:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2b67fc640 1 -- 172.21.15.31:0/3577674788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b80ade40 con 0x7fb2c015a710 2023-11-26T07:21:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c7c06640 1 -- 172.21.15.31:0/3577674788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b8009a70 con 0x7fb2c015a710 2023-11-26T07:21:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2b67fc640 1 -- 172.21.15.31:0/3577674788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80b9690 con 0x7fb2c015a710 2023-11-26T07:21:43.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c7c06640 1 -- 172.21.15.31:0/3577674788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c0154c10 con 0x7fb2c015a710 2023-11-26T07:21:43.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2c7c06640 1 -- 172.21.15.31:0/3577674788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c00671f0 con 0x7fb2c015a710 2023-11-26T07:21:43.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.534+0000 7fb2b67fc640 1 -- 172.21.15.31:0/3577674788 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fb2b80b97f0 con 0x7fb2c015a710 2023-11-26T07:21:43.539 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.538+0000 7fb2b67fc640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2a40667d0 0x7fb2a4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.538+0000 7fb2b67fc640 1 -- 172.21.15.31:0/3577674788 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb2b811f6b0 con 0x7fb2c015a710 2023-11-26T07:21:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.538+0000 7fb2c59a2640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2a40667d0 0x7fb2a4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:43.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.538+0000 7fb2c59a2640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2a40667d0 0x7fb2a4068c90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0011f0 tx=0x7fb2bc007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:43.543 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.542+0000 7fb2b67fc640 1 -- 172.21.15.31:0/3577674788 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb2b80ecb20 con 0x7fb2c015a710 2023-11-26T07:21:43.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:43.710+0000 7fb2c7c06640 1 -- 172.21.15.31:0/3577674788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb2c006b460 con 0x7fb2c015a710 2023-11-26T07:21:44.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2b67fc640 1 -- 172.21.15.31:0/3577674788 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb2b8009450 con 0x7fb2c015a710 2023-11-26T07:21:44.347 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-11-26T07:21:44.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 -- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2a40667d0 msgr2=0x7fb2a4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2a40667d0 0x7fb2a4068c90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0011f0 tx=0x7fb2bc007d40 comp rx=0 tx=0).stop 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 -- 172.21.15.31:0/3577674788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c015a710 msgr2=0x7fb2c0140850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c015a710 0x7fb2c0140850 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80b0040 tx=0x7fb2b80196c0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 -- 172.21.15.31:0/3577674788 shutdown_connections 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fb2a40667d0 0x7fb2a4068c90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb2c015a710 0x7fb2c0140850 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb2c00cdfb0 0x7fb2c00ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 --2- 172.21.15.31:0/3577674788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2c00cbd50 0x7fb2c009c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 -- 172.21.15.31:0/3577674788 >> 172.21.15.31:0/3577674788 conn(0x7fb2c00b0cb0 msgr2=0x7fb2c00ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 -- 172.21.15.31:0/3577674788 shutdown_connections 2023-11-26T07:21:44.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.346+0000 7fb2c7c06640 1 -- 172.21.15.31:0/3577674788 wait complete. 2023-11-26T07:21:44.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:21:44.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f4532a32640 1 Processor -- start 2023-11-26T07:21:44.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f4532a32640 1 -- start start 2023-11-26T07:21:44.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f4532a32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 0x7f452c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f4532a32640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c145060 con 0x7f452c070c20 2023-11-26T07:21:44.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f4532a32640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c071fd0 con 0x7f452c144c80 2023-11-26T07:21:44.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f4532a32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c071450 con 0x7f452c0ac630 2023-11-26T07:21:44.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f4530fcf640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f452c070c20 0x7f452c06c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45676/0 (socket says 172.21.15.31:45676) 2023-11-26T07:21:44.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f4530fcf640 1 -- 172.21.15.31:0/1823253621 learned_addr learned my addr 172.21.15.31:0/1823253621 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f452b7fe640 1 --2- 172.21.15.31:0/1823253621 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 0x7f452c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f452affd640 1 -- 172.21.15.31:0/1823253621 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178192489 0 0) 0x7f452c145060 con 0x7f452c070c20 2023-11-26T07:21:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f452affd640 1 -- 172.21.15.31:0/1823253621 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4514003530 con 0x7f452c070c20 2023-11-26T07:21:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f452affd640 1 -- 172.21.15.31:0/1823253621 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468723872 0 0) 0x7f452c071fd0 con 0x7f452c144c80 2023-11-26T07:21:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f452affd640 1 -- 172.21.15.31:0/1823253621 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f452c145060 con 0x7f452c144c80 2023-11-26T07:21:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f452b7fe640 1 -- 172.21.15.31:0/1823253621 >> v1:172.21.15.97:6789/0 conn(0x7f452c144c80 legacy=0x7f452c0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f452b7fe640 1 -- 172.21.15.31:0/1823253621 >> v1:172.21.15.31:6789/0 conn(0x7f452c070c20 legacy=0x7f452c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.518+0000 7f452b7fe640 1 -- 172.21.15.31:0/1823253621 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c071730 con 0x7f452c0ac630 2023-11-26T07:21:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f452b7fe640 1 --2- 172.21.15.31:0/1823253621 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 0x7f452c0706e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f45200097f0 tx=0x7f452001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=849dbcfb0d502a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f452affd640 1 -- 172.21.15.31:0/1823253621 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200aa5f0 con 0x7f452c0ac630 2023-11-26T07:21:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f452affd640 1 -- 172.21.15.31:0/1823253621 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45200aa750 con 0x7f452c0ac630 2023-11-26T07:21:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f452affd640 1 -- 172.21.15.31:0/1823253621 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200aaae0 con 0x7f452c0ac630 2023-11-26T07:21:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f4532a32640 1 -- 172.21.15.31:0/1823253621 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 msgr2=0x7f452c0706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f4532a32640 1 --2- 172.21.15.31:0/1823253621 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 0x7f452c0706e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f45200097f0 tx=0x7f452001c490 comp rx=0 tx=0).stop 2023-11-26T07:21:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f4532a32640 1 -- 172.21.15.31:0/1823253621 shutdown_connections 2023-11-26T07:21:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f4532a32640 1 --2- 172.21.15.31:0/1823253621 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 0x7f452c0706e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f4532a32640 1 -- 172.21.15.31:0/1823253621 >> 172.21.15.31:0/1823253621 conn(0x7f452c0b0c40 msgr2=0x7f452c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:44.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f4532a32640 1 -- 172.21.15.31:0/1823253621 shutdown_connections 2023-11-26T07:21:44.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.522+0000 7f4532a32640 1 -- 172.21.15.31:0/1823253621 wait complete. 2023-11-26T07:21:44.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4532a32640 1 Processor -- start 2023-11-26T07:21:44.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4532a32640 1 -- start start 2023-11-26T07:21:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4532a32640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f452c070c20 0x7f452c09a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4532a32640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 0x7f452c09c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4532a32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c144c80 0x7f452c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4532a32640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c09bca0 con 0x7f452c070c20 2023-11-26T07:21:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4532a32640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c1389f0 con 0x7f452c144c80 2023-11-26T07:21:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4532a32640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c09c990 con 0x7f452c0ac630 2023-11-26T07:21:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f452bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f452c070c20 0x7f452c09a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f452bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f452c070c20 0x7f452c09a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42944/0 (socket says 172.21.15.31:42944) 2023-11-26T07:21:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f452bfff640 1 -- 172.21.15.31:0/2015452279 learned_addr learned my addr 172.21.15.31:0/2015452279 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f452b7fe640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 0x7f452c09c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4530fcf640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c144c80 0x7f452c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f452bfff640 1 -- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 msgr2=0x7f452c09c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f452bfff640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 0x7f452c09c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f452bfff640 1 -- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c144c80 msgr2=0x7f452c15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f452bfff640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c144c80 0x7f452c15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f452bfff640 1 -- 172.21.15.31:0/2015452279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4520009420 con 0x7f452c070c20 2023-11-26T07:21:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f452bfff640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f452c070c20 0x7f452c09a940 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f451c00c490 tx=0x7f451c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4528ff9640 1 -- 172.21.15.31:0/2015452279 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0052d0 con 0x7f452c070c20 2023-11-26T07:21:44.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4528ff9640 1 -- 172.21.15.31:0/2015452279 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c005430 con 0x7f452c070c20 2023-11-26T07:21:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4532a32640 1 -- 172.21.15.31:0/2015452279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c09b020 con 0x7f452c070c20 2023-11-26T07:21:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4528ff9640 1 -- 172.21.15.31:0/2015452279 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c012870 con 0x7f452c070c20 2023-11-26T07:21:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.526+0000 7f4532a32640 1 -- 172.21.15.31:0/2015452279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c09b5e0 con 0x7f452c070c20 2023-11-26T07:21:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.530+0000 7f4532a32640 1 -- 172.21.15.31:0/2015452279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c0b1ba0 con 0x7f452c070c20 2023-11-26T07:21:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.530+0000 7f4528ff9640 1 -- 172.21.15.31:0/2015452279 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f451c0129d0 con 0x7f452c070c20 2023-11-26T07:21:44.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.530+0000 7f4528ff9640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45180667d0 0x7f4518068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.530+0000 7f4528ff9640 1 -- 172.21.15.31:0/2015452279 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f451c083b60 con 0x7f452c070c20 2023-11-26T07:21:44.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.530+0000 7f452b7fe640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45180667d0 0x7f4518068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.534+0000 7f452b7fe640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45180667d0 0x7f4518068c90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45200ae000 tx=0x7f45200a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:44.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.534+0000 7f4528ff9640 1 -- 172.21.15.31:0/2015452279 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f451c050f20 con 0x7f452c070c20 2023-11-26T07:21:44.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 -- 172.21.15.31:0/2015452279 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f452c04e850 con 0x7f452c070c20 2023-11-26T07:21:44.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4528ff9640 1 -- 172.21.15.31:0/2015452279 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5617 (secure 0 0 0) 0x7f451c054540 con 0x7f452c070c20 2023-11-26T07:21:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:44.749 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:21:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-11-26T07:21:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-11-26T07:21:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlRJmWGBAAuUAS68YFt9PaqDV8RFi+6w==", 2023-11-26T07:21:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:21:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:21:44.749 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlIdH1JBAAmPlAQXq78ht4pJf6I7Cd5w==", 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlfuXoHxAAf7vKZr+LVZaN6HR3qiks5w==", 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlWmgkLBAAojAzX06O7kbwmj154qUGog==", 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.752 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDU8GJlc/bqEBAAueOQvSeK6MW+WOT746yQzA==", 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDX8GJlEaW6FxAA8q4rEKSJtIl/Hr2uCNSMUg==", 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:44.753 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDa8GJldV90HxAAQRgcLf8ABSU4Sfb3A1Xa4g==", 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDd8GJl2erPJhAAoC71eDwCOmskb+VCLYQyUw==", 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-11-26T07:21:44.755 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDj8GJllq08LxAAGuz9+Ou/ybWxdkN3PSMaEQ==", 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDm8GJlww4eNxAAjRaBq45nrNwJPlopyOhXNg==", 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:44.756 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDq8GJl2jxiAhAAhu2OULy84YRJYxTmShoVBg==", 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDt8GJld8HNCBAA3deA4HKTqRggbVc73Y8efg==", 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-11-26T07:21:44.758 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA==", 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlEBAQOhAAx5TsMN17L2fNm+j3yqUIOQ==", 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:21:44.759 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl/CN+NhAAeyry8gIOtxwLxCQb0t1eKw==", 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:21:44.760 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDR8GJlA9e7BBAATe3WYA79PgfLc5UHgKGkpw==", 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:21:44.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDP8GJlfT+4NhAACA0t23bQtWvdFLsB9X+X8w==", 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.alice", 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDz8WJlsXRVNRAANIcQYI6xYe6U5js+J39pug==", 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol13010", 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlnh/pExAA6qyPM7So2BLbxbGbquLZZA==", 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlYhb/DxAAx1G3Ve23IdZ7cX9nPb2WhA==", 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:21:44.764 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:21:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:21:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:21:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:44.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:21:44.765 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:21:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 -- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45180667d0 msgr2=0x7f4518068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45180667d0 0x7f4518068c90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45200ae000 tx=0x7f45200a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 -- 172.21.15.31:0/2015452279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f452c070c20 msgr2=0x7f452c09a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f452c070c20 0x7f452c09a940 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f451c00c490 tx=0x7f451c00c950 comp rx=0 tx=0).stop 2023-11-26T07:21:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 -- 172.21.15.31:0/2015452279 shutdown_connections 2023-11-26T07:21:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f45180667d0 0x7f4518068c90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f452c144c80 0x7f452c15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f452c0ac630 0x7f452c09c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 --2- 172.21.15.31:0/2015452279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f452c070c20 0x7f452c09a940 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.746+0000 7f4532a32640 1 -- 172.21.15.31:0/2015452279 >> 172.21.15.31:0/2015452279 conn(0x7f452c0b0c40 msgr2=0x7f452c0aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:44.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.750+0000 7f4532a32640 1 -- 172.21.15.31:0/2015452279 shutdown_connections 2023-11-26T07:21:44.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.750+0000 7f4532a32640 1 -- 172.21.15.31:0/2015452279 wait complete. 2023-11-26T07:21:44.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2023-11-26T07:21:44.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.910+0000 7f4b657f8640 1 Processor -- start 2023-11-26T07:21:44.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.910+0000 7f4b657f8640 1 -- start start 2023-11-26T07:21:44.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b657f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b6015d7c0 0x7f4b600d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b657f8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b60161f20 0x7f4b600cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b657f8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b600d0740 con 0x7f4b600d02f0 2023-11-26T07:21:44.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b657f8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b600d0f90 con 0x7f4b60161f20 2023-11-26T07:21:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b657f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600d11a0 con 0x7f4b6015d7c0 2023-11-26T07:21:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b57fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4b60161f20 0x7f4b600cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b5f7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4b600d02f0 0x7f4b6015dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45678/0 (socket says 172.21.15.31:45678) 2023-11-26T07:21:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b5f7fe640 1 -- 172.21.15.31:0/2072778193 learned_addr learned my addr 172.21.15.31:0/2072778193 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:44.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b5effd640 1 --2- 172.21.15.31:0/2072778193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b6015d7c0 0x7f4b600d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b5e7fc640 1 -- 172.21.15.31:0/2072778193 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891247038 0 0) 0x7f4b580014a0 con 0x7f4b600d02f0 2023-11-26T07:21:44.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b5e7fc640 1 -- 172.21.15.31:0/2072778193 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b4c003530 con 0x7f4b600d02f0 2023-11-26T07:21:44.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b57fff640 1 -- 172.21.15.31:0/2072778193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b6015d7c0 msgr2=0x7f4b600d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:44.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b57fff640 1 --2- 172.21.15.31:0/2072778193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b6015d7c0 0x7f4b600d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b57fff640 1 -- 172.21.15.31:0/2072778193 >> v1:172.21.15.31:6789/0 conn(0x7f4b600d02f0 legacy=0x7f4b6015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.914+0000 7f4b57fff640 1 -- 172.21.15.31:0/2072778193 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600d1480 con 0x7f4b60161f20 2023-11-26T07:21:44.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b57fff640 1 --2- 172.21.15.31:0/2072778193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b60161f20 0x7f4b600cfdb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4b480097a0 tx=0x7f4b4801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354597d382cfd794 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:44.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b5e7fc640 1 -- 172.21.15.31:0/2072778193 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480a7070 con 0x7f4b60161f20 2023-11-26T07:21:44.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b5e7fc640 1 -- 172.21.15.31:0/2072778193 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b480ab430 con 0x7f4b60161f20 2023-11-26T07:21:44.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b5e7fc640 1 -- 172.21.15.31:0/2072778193 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480ab7c0 con 0x7f4b60161f20 2023-11-26T07:21:44.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b657f8640 1 -- 172.21.15.31:0/2072778193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b60161f20 msgr2=0x7f4b600cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b657f8640 1 --2- 172.21.15.31:0/2072778193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b60161f20 0x7f4b600cfdb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4b480097a0 tx=0x7f4b4801c430 comp rx=0 tx=0).stop 2023-11-26T07:21:44.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b657f8640 1 -- 172.21.15.31:0/2072778193 shutdown_connections 2023-11-26T07:21:44.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b657f8640 1 --2- 172.21.15.31:0/2072778193 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b60161f20 0x7f4b600cfdb0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b657f8640 1 --2- 172.21.15.31:0/2072778193 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b6015d7c0 0x7f4b600d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b657f8640 1 -- 172.21.15.31:0/2072778193 >> 172.21.15.31:0/2072778193 conn(0x7f4b6008afc0 msgr2=0x7f4b600d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:44.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b657f8640 1 -- 172.21.15.31:0/2072778193 shutdown_connections 2023-11-26T07:21:44.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b657f8640 1 -- 172.21.15.31:0/2072778193 wait complete. 2023-11-26T07:21:44.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b657f8640 1 Processor -- start 2023-11-26T07:21:44.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.918+0000 7f4b657f8640 1 -- start start 2023-11-26T07:21:44.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b657f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b600d02f0 0x7f4b601573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b657f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6015d7c0 0x7f4b60158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b657f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b600b10b0 0x7f4b600b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b657f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600b62e0 con 0x7f4b600b10b0 2023-11-26T07:21:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b657f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b60159350 con 0x7f4b6015d7c0 2023-11-26T07:21:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b657f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b600b1a80 con 0x7f4b600d02f0 2023-11-26T07:21:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b600b10b0 0x7f4b600b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b600b10b0 0x7f4b600b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42964/0 (socket says 172.21.15.31:42964) 2023-11-26T07:21:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5f7fe640 1 -- 172.21.15.31:0/979670648 learned_addr learned my addr 172.21.15.31:0/979670648 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5effd640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b600d02f0 0x7f4b601573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b57fff640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6015d7c0 0x7f4b60158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5f7fe640 1 -- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b600d02f0 msgr2=0x7f4b601573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5f7fe640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b600d02f0 0x7f4b601573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5f7fe640 1 -- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6015d7c0 msgr2=0x7f4b60158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5f7fe640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6015d7c0 0x7f4b60158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5f7fe640 1 -- 172.21.15.31:0/979670648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b480093b0 con 0x7f4b600b10b0 2023-11-26T07:21:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5effd640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b600d02f0 0x7f4b601573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b57fff640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6015d7c0 0x7f4b60158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b5f7fe640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b600b10b0 0x7f4b600b1540 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800cb40 tx=0x7f4b5800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:44.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b577fe640 1 -- 172.21.15.31:0/979670648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800aca0 con 0x7f4b600b10b0 2023-11-26T07:21:44.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b577fe640 1 -- 172.21.15.31:0/979670648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5800ae00 con 0x7f4b600b10b0 2023-11-26T07:21:44.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b577fe640 1 -- 172.21.15.31:0/979670648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58012620 con 0x7f4b600b10b0 2023-11-26T07:21:44.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b657f8640 1 -- 172.21.15.31:0/979670648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b60157900 con 0x7f4b600b10b0 2023-11-26T07:21:44.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b657f8640 1 -- 172.21.15.31:0/979670648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b601568e0 con 0x7f4b600b10b0 2023-11-26T07:21:44.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b657f8640 1 -- 172.21.15.31:0/979670648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6004e850 con 0x7f4b600b10b0 2023-11-26T07:21:44.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.922+0000 7f4b577fe640 1 -- 172.21.15.31:0/979670648 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f4b58007070 con 0x7f4b600b10b0 2023-11-26T07:21:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.926+0000 7f4b577fe640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4b40066800 0x7f4b40068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.926+0000 7f4b577fe640 1 -- 172.21.15.31:0/979670648 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4b58083ec0 con 0x7f4b600b10b0 2023-11-26T07:21:44.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.926+0000 7f4b5effd640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4b40066800 0x7f4b40068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:44.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.926+0000 7f4b5effd640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4b40066800 0x7f4b40068cc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4b600b3c90 tx=0x7f4b50008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:44.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:44.930+0000 7f4b577fe640 1 -- 172.21.15.31:0/979670648 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4b58051240 con 0x7f4b600b10b0 2023-11-26T07:21:45.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.138+0000 7f4b657f8640 1 -- 172.21.15.31:0/979670648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) v1 -- 0x7f4b600d0740 con 0x7f4b600b10b0 2023-11-26T07:21:45.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.142+0000 7f4b577fe640 1 -- 172.21.15.31:0/979670648 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v19) v1 ==== 82+0+0 (secure 0 0 0) 0x7f4b600d0740 con 0x7f4b600b10b0 2023-11-26T07:21:45.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 -- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4b40066800 msgr2=0x7f4b40068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4b40066800 0x7f4b40068cc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4b600b3c90 tx=0x7f4b50008040 comp rx=0 tx=0).stop 2023-11-26T07:21:45.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 -- 172.21.15.31:0/979670648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b600b10b0 msgr2=0x7f4b600b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b600b10b0 0x7f4b600b1540 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800cb40 tx=0x7f4b5800a060 comp rx=0 tx=0).stop 2023-11-26T07:21:45.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 -- 172.21.15.31:0/979670648 shutdown_connections 2023-11-26T07:21:45.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4b40066800 0x7f4b40068cc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b600b10b0 0x7f4b600b1540 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b6015d7c0 0x7f4b60158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 --2- 172.21.15.31:0/979670648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4b600d02f0 0x7f4b601573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 -- 172.21.15.31:0/979670648 >> 172.21.15.31:0/979670648 conn(0x7f4b6008afc0 msgr2=0x7f4b600cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:45.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 -- 172.21.15.31:0/979670648 shutdown_connections 2023-11-26T07:21:45.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.146+0000 7f4b657f8640 1 -- 172.21.15.31:0/979670648 wait complete. 2023-11-26T07:21:45.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f3511dc4640 1 Processor -- start 2023-11-26T07:21:45.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f3511dc4640 1 -- start start 2023-11-26T07:21:45.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f3511dc4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f350c144c80 0x7f350c0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f3511dc4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f350c0ac460 0x7f350c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f3511dc4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f350c145060 con 0x7f350c070c20 2023-11-26T07:21:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f3511dc4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350c071fd0 con 0x7f350c0ac460 2023-11-26T07:21:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f3511dc4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350c0ac8c0 con 0x7f350c144c80 2023-11-26T07:21:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f350affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f350c0ac460 0x7f350c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f350affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f350c0ac460 0x7f350c0706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42972/0 (socket says 172.21.15.31:42972) 2023-11-26T07:21:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f350bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f350c070c20 0x7f350c15eed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45682/0 (socket says 172.21.15.31:45682) 2023-11-26T07:21:45.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f350affd640 1 -- 172.21.15.31:0/1056004972 learned_addr learned my addr 172.21.15.31:0/1056004972 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f350b7fe640 1 --2- 172.21.15.31:0/1056004972 >> v2:172.21.15.97:3300/0 conn(0x7f350c144c80 0x7f350c0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f350affd640 1 -- 172.21.15.31:0/1056004972 >> v2:172.21.15.97:3300/0 conn(0x7f350c144c80 msgr2=0x7f350c0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f350affd640 1 --2- 172.21.15.31:0/1056004972 >> v2:172.21.15.97:3300/0 conn(0x7f350c144c80 0x7f350c0abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f350affd640 1 -- 172.21.15.31:0/1056004972 >> v1:172.21.15.31:6789/0 conn(0x7f350c070c20 legacy=0x7f350c15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f350a7fc640 1 -- 172.21.15.31:0/1056004972 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067970090 0 0) 0x7f350c145060 con 0x7f350c070c20 2023-11-26T07:21:45.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.310+0000 7f350affd640 1 -- 172.21.15.31:0/1056004972 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c071450 con 0x7f350c0ac460 2023-11-26T07:21:45.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f350affd640 1 --2- 172.21.15.31:0/1056004972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ac460 0x7f350c0706e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f350c06eb80 tx=0x7f350001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e82ea59cfc42cf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:45.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f350a7fc640 1 -- 172.21.15.31:0/1056004972 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000aa430 con 0x7f350c0ac460 2023-11-26T07:21:45.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f350a7fc640 1 -- 172.21.15.31:0/1056004972 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35000aa590 con 0x7f350c0ac460 2023-11-26T07:21:45.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f350a7fc640 1 -- 172.21.15.31:0/1056004972 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000aa920 con 0x7f350c0ac460 2023-11-26T07:21:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f3511dc4640 1 -- 172.21.15.31:0/1056004972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ac460 msgr2=0x7f350c0706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f3511dc4640 1 --2- 172.21.15.31:0/1056004972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ac460 0x7f350c0706e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f350c06eb80 tx=0x7f350001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f3511dc4640 1 -- 172.21.15.31:0/1056004972 shutdown_connections 2023-11-26T07:21:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f3511dc4640 1 --2- 172.21.15.31:0/1056004972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c0ac460 0x7f350c0706e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f3511dc4640 1 --2- 172.21.15.31:0/1056004972 >> v2:172.21.15.97:3300/0 conn(0x7f350c144c80 0x7f350c0abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f3511dc4640 1 -- 172.21.15.31:0/1056004972 >> 172.21.15.31:0/1056004972 conn(0x7f350c0b0c40 msgr2=0x7f350c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:45.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f3511dc4640 1 -- 172.21.15.31:0/1056004972 shutdown_connections 2023-11-26T07:21:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.314+0000 7f3511dc4640 1 -- 172.21.15.31:0/1056004972 wait complete. 2023-11-26T07:21:45.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 Processor -- start 2023-11-26T07:21:45.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 -- start start 2023-11-26T07:21:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f350c070c20 0x7f350c1643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f350c0ac460 0x7f350c1659b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c144c80 0x7f350c0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c13e8b0 con 0x7f350c144c80 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c165ef0 con 0x7f350c070c20 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350c15d2e0 con 0x7f350c0ac460 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c144c80 0x7f350c0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c144c80 0x7f350c0cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42986/0 (socket says 172.21.15.31:42986) 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350bfff640 1 -- 172.21.15.31:0/3428256647 learned_addr learned my addr 172.21.15.31:0/3428256647 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350b7fe640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f350c070c20 0x7f350c1643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350affd640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f350c0ac460 0x7f350c1659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350bfff640 1 -- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f350c0ac460 msgr2=0x7f350c1659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350bfff640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f350c0ac460 0x7f350c1659b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350bfff640 1 -- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f350c070c20 msgr2=0x7f350c1643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350bfff640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f350c070c20 0x7f350c1643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350bfff640 1 -- 172.21.15.31:0/3428256647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35000094d0 con 0x7f350c144c80 2023-11-26T07:21:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350affd640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f350c0ac460 0x7f350c1659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f350bfff640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c144c80 0x7f350c0cf1f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3504001570 tx=0x7f350400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f34f3fff640 1 -- 172.21.15.31:0/3428256647 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350400a640 con 0x7f350c144c80 2023-11-26T07:21:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f34f3fff640 1 -- 172.21.15.31:0/3428256647 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350400a7c0 con 0x7f350c144c80 2023-11-26T07:21:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f34f3fff640 1 -- 172.21.15.31:0/3428256647 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350400ab70 con 0x7f350c144c80 2023-11-26T07:21:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 -- 172.21.15.31:0/3428256647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c164a20 con 0x7f350c144c80 2023-11-26T07:21:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 -- 172.21.15.31:0/3428256647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c164fe0 con 0x7f350c144c80 2023-11-26T07:21:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.318+0000 7f3511dc4640 1 -- 172.21.15.31:0/3428256647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c0b1ba0 con 0x7f350c144c80 2023-11-26T07:21:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.322+0000 7f34f3fff640 1 -- 172.21.15.31:0/3428256647 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f350400acd0 con 0x7f350c144c80 2023-11-26T07:21:45.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.322+0000 7f34f3fff640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34f8066850 0x7f34f8068d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.322+0000 7f34f3fff640 1 -- 172.21.15.31:0/3428256647 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3504083860 con 0x7f350c144c80 2023-11-26T07:21:45.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.322+0000 7f350b7fe640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34f8066850 0x7f34f8068d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.322+0000 7f350b7fe640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34f8066850 0x7f34f8068d10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0079c0 tx=0x7f34fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:45.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.326+0000 7f34f3fff640 1 -- 172.21.15.31:0/3428256647 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3504050da0 con 0x7f350c144c80 2023-11-26T07:21:45.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.486+0000 7f3511dc4640 1 -- 172.21.15.31:0/3428256647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f350c04e850 con 0x7f350c144c80 2023-11-26T07:21:45.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f34f3fff640 1 -- 172.21.15.31:0/3428256647 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+12773 (secure 0 0 0) 0x7f3504018090 con 0x7f350c144c80 2023-11-26T07:21:45.491 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:45.491 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":64,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:44.359166+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:45.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 -- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34f8066850 msgr2=0x7f34f8068d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34f8066850 0x7f34f8068d10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0079c0 tx=0x7f34fc008040 comp rx=0 tx=0).stop 2023-11-26T07:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 -- 172.21.15.31:0/3428256647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c144c80 msgr2=0x7f350c0cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c144c80 0x7f350c0cf1f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3504001570 tx=0x7f350400cca0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 -- 172.21.15.31:0/3428256647 shutdown_connections 2023-11-26T07:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f34f8066850 0x7f34f8068d10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f350c144c80 0x7f350c0cf1f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f350c0ac460 0x7f350c1659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 --2- 172.21.15.31:0/3428256647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f350c070c20 0x7f350c1643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 -- 172.21.15.31:0/3428256647 >> 172.21.15.31:0/3428256647 conn(0x7f350c0b0c40 msgr2=0x7f350c13a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 -- 172.21.15.31:0/3428256647 shutdown_connections 2023-11-26T07:21:45.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.490+0000 7f3511dc4640 1 -- 172.21.15.31:0/3428256647 wait complete. 2023-11-26T07:21:45.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:21:45.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:21:45.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.658+0000 7f505b017640 1 Processor -- start 2023-11-26T07:21:45.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.658+0000 7f505b017640 1 -- start start 2023-11-26T07:21:45.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505b017640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f505415d2a0 0x7f50540d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505b017640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50540b5350 0x7f50540d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505b017640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50540b6250 con 0x7f50540d2c20 2023-11-26T07:21:45.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505b017640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50540ce8c0 con 0x7f505415d2a0 2023-11-26T07:21:45.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505b017640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5054161cf0 con 0x7f50540b5350 2023-11-26T07:21:45.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505a015640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f505415d2a0 0x7f50540d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505a015640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f505415d2a0 0x7f50540d26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42998/0 (socket says 172.21.15.31:42998) 2023-11-26T07:21:45.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505a015640 1 -- 172.21.15.31:0/186920091 learned_addr learned my addr 172.21.15.31:0/186920091 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:45.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505a816640 1 --2- 172.21.15.31:0/186920091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50540b5350 0x7f50540d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505a015640 1 -- 172.21.15.31:0/186920091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50540b5350 msgr2=0x7f50540d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:45.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505a015640 1 --2- 172.21.15.31:0/186920091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50540b5350 0x7f50540d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505a015640 1 -- 172.21.15.31:0/186920091 >> v1:172.21.15.31:6789/0 conn(0x7f50540d2c20 legacy=0x7f50540b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f5059013640 1 -- 172.21.15.31:0/186920091 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529887009 0 0) 0x7f50540b6250 con 0x7f50540d2c20 2023-11-26T07:21:45.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505a015640 1 -- 172.21.15.31:0/186920091 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5054161170 con 0x7f505415d2a0 2023-11-26T07:21:45.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f505a015640 1 --2- 172.21.15.31:0/186920091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505415d2a0 0x7f50540d26e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5050009870 tx=0x7f505001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c677e50db8e17a78 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:45.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.662+0000 7f5059013640 1 -- 172.21.15.31:0/186920091 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500aa670 con 0x7f505415d2a0 2023-11-26T07:21:45.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f5059013640 1 -- 172.21.15.31:0/186920091 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50500aa7d0 con 0x7f505415d2a0 2023-11-26T07:21:45.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f5059013640 1 -- 172.21.15.31:0/186920091 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500aab60 con 0x7f505415d2a0 2023-11-26T07:21:45.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 -- 172.21.15.31:0/186920091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505415d2a0 msgr2=0x7f50540d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 --2- 172.21.15.31:0/186920091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505415d2a0 0x7f50540d26e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5050009870 tx=0x7f505001c510 comp rx=0 tx=0).stop 2023-11-26T07:21:45.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 -- 172.21.15.31:0/186920091 shutdown_connections 2023-11-26T07:21:45.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 --2- 172.21.15.31:0/186920091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50540b5350 0x7f50540d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 --2- 172.21.15.31:0/186920091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f505415d2a0 0x7f50540d26e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 -- 172.21.15.31:0/186920091 >> 172.21.15.31:0/186920091 conn(0x7f50540668a0 msgr2=0x7f50540cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:45.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 -- 172.21.15.31:0/186920091 shutdown_connections 2023-11-26T07:21:45.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 -- 172.21.15.31:0/186920091 wait complete. 2023-11-26T07:21:45.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 Processor -- start 2023-11-26T07:21:45.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 -- start start 2023-11-26T07:21:45.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5350 0x7f5054156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50540d2c20 0x7f50541549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f505415d2a0 0x7f505415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505415a2c0 con 0x7f50540d2c20 2023-11-26T07:21:45.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50540b7bd0 con 0x7f50540b5350 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f505b017640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50540b7eb0 con 0x7f505415d2a0 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.666+0000 7f5059814640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50540d2c20 0x7f50541549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f5059814640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50540d2c20 0x7f50541549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43014/0 (socket says 172.21.15.31:43014) 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f5059814640 1 -- 172.21.15.31:0/1780638926 learned_addr learned my addr 172.21.15.31:0/1780638926 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f5059814640 1 -- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f505415d2a0 msgr2=0x7f505415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f5059814640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f505415d2a0 0x7f505415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f505a015640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5350 0x7f5054156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f5059814640 1 -- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5350 msgr2=0x7f5054156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f5059814640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5350 0x7f5054156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f5059814640 1 -- 172.21.15.31:0/1780638926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500094d0 con 0x7f50540d2c20 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f5059814640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50540d2c20 0x7f50541549f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f504400c3f0 tx=0x7f504400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:45.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f504affd640 1 -- 172.21.15.31:0/1780638926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50440052d0 con 0x7f50540d2c20 2023-11-26T07:21:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f504affd640 1 -- 172.21.15.31:0/1780638926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5044005430 con 0x7f50540d2c20 2023-11-26T07:21:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f505b017640 1 -- 172.21.15.31:0/1780638926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50540b8190 con 0x7f50540d2c20 2023-11-26T07:21:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f504affd640 1 -- 172.21.15.31:0/1780638926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044003860 con 0x7f50540d2c20 2023-11-26T07:21:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f505b017640 1 -- 172.21.15.31:0/1780638926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5054156b40 con 0x7f50540d2c20 2023-11-26T07:21:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f505b017640 1 -- 172.21.15.31:0/1780638926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50540b7a30 con 0x7f50540d2c20 2023-11-26T07:21:45.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.670+0000 7f504affd640 1 -- 172.21.15.31:0/1780638926 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f50440039c0 con 0x7f50540d2c20 2023-11-26T07:21:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.674+0000 7f504affd640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50380667d0 0x7f5038068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.674+0000 7f504affd640 1 -- 172.21.15.31:0/1780638926 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5044083600 con 0x7f50540d2c20 2023-11-26T07:21:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.674+0000 7f505a015640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50380667d0 0x7f5038068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:45.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.674+0000 7f505a015640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50380667d0 0x7f5038068c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f50500ae000 tx=0x7f5050008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:45.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.678+0000 7f504affd640 1 -- 172.21.15.31:0/1780638926 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50540b7a30 con 0x7f50540d2c20 2023-11-26T07:21:45.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.834+0000 7f505b017640 1 -- 172.21.15.31:0/1780638926 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50540b83d0 con 0x7f50540d2c20 2023-11-26T07:21:45.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.834+0000 7f504affd640 1 -- 172.21.15.31:0/1780638926 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5044017020 con 0x7f50540d2c20 2023-11-26T07:21:45.837 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.834+0000 7f505b017640 1 -- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50380667d0 msgr2=0x7f5038068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.834+0000 7f505b017640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50380667d0 0x7f5038068c90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f50500ae000 tx=0x7f5050008040 comp rx=0 tx=0).stop 2023-11-26T07:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.838+0000 7f505b017640 1 -- 172.21.15.31:0/1780638926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50540d2c20 msgr2=0x7f50541549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.838+0000 7f505b017640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50540d2c20 0x7f50541549f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f504400c3f0 tx=0x7f504400c8b0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.838+0000 7f505b017640 1 -- 172.21.15.31:0/1780638926 shutdown_connections 2023-11-26T07:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.838+0000 7f505b017640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f50380667d0 0x7f5038068c90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.838+0000 7f505b017640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f505415d2a0 0x7f505415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.838+0000 7f505b017640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50540d2c20 0x7f50541549f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.838+0000 7f505b017640 1 --2- 172.21.15.31:0/1780638926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50540b5350 0x7f5054156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:45.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.838+0000 7f505b017640 1 -- 172.21.15.31:0/1780638926 >> 172.21.15.31:0/1780638926 conn(0x7f50540668a0 msgr2=0x7f5054167340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:45.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.838+0000 7f505b017640 1 -- 172.21.15.31:0/1780638926 shutdown_connections 2023-11-26T07:21:45.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.838+0000 7f505b017640 1 -- 172.21.15.31:0/1780638926 wait complete. 2023-11-26T07:21:45.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:21:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.998+0000 7fc8e506b640 1 Processor -- start 2023-11-26T07:21:46.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.998+0000 7fc8e506b640 1 -- start start 2023-11-26T07:21:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.998+0000 7fc8e506b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8e0143ae0 0x7fc8e00b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.998+0000 7fc8e506b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8e00ae3d0 0x7fc8e006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:45.998+0000 7fc8e506b640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e00ae7f0 con 0x7fc8e006dfd0 2023-11-26T07:21:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8e506b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e006e980 con 0x7fc8e0143ae0 2023-11-26T07:21:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8e506b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e00b2480 con 0x7fc8e00ae3d0 2023-11-26T07:21:46.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8ded9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8e0143ae0 0x7fc8e00b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8ded9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8e0143ae0 0x7fc8e00b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43020/0 (socket says 172.21.15.31:43020) 2023-11-26T07:21:46.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8ded9d640 1 -- 172.21.15.31:0/2850269445 learned_addr learned my addr 172.21.15.31:0/2850269445 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:46.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8df59e640 1 --2- 172.21.15.31:0/2850269445 >> v2:172.21.15.97:3300/0 conn(0x7fc8e00ae3d0 0x7fc8e006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8ded9d640 1 -- 172.21.15.31:0/2850269445 >> v2:172.21.15.97:3300/0 conn(0x7fc8e00ae3d0 msgr2=0x7fc8e006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:46.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8ded9d640 1 --2- 172.21.15.31:0/2850269445 >> v2:172.21.15.97:3300/0 conn(0x7fc8e00ae3d0 0x7fc8e006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8ded9d640 1 -- 172.21.15.31:0/2850269445 >> v1:172.21.15.97:6789/0 conn(0x7fc8e006dfd0 legacy=0x7fc8e00adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:46.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8ded9d640 1 -- 172.21.15.31:0/2850269445 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e0070670 con 0x7fc8e0143ae0 2023-11-26T07:21:46.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.002+0000 7fc8ded9d640 1 --2- 172.21.15.31:0/2850269445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 0x7fc8e00b1f40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0072ad0 tx=0x7fc8d001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a0f2501e864611 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8de59c640 1 -- 172.21.15.31:0/2850269445 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00a7c30 con 0x7fc8e0143ae0 2023-11-26T07:21:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8de59c640 1 -- 172.21.15.31:0/2850269445 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d00a7d90 con 0x7fc8e0143ae0 2023-11-26T07:21:46.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8de59c640 1 -- 172.21.15.31:0/2850269445 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00af660 con 0x7fc8e0143ae0 2023-11-26T07:21:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 -- 172.21.15.31:0/2850269445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 msgr2=0x7fc8e00b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 --2- 172.21.15.31:0/2850269445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 0x7fc8e00b1f40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0072ad0 tx=0x7fc8d001eb60 comp rx=0 tx=0).stop 2023-11-26T07:21:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 -- 172.21.15.31:0/2850269445 shutdown_connections 2023-11-26T07:21:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 --2- 172.21.15.31:0/2850269445 >> v2:172.21.15.97:3300/0 conn(0x7fc8e00ae3d0 0x7fc8e006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 --2- 172.21.15.31:0/2850269445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 0x7fc8e00b1f40 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 -- 172.21.15.31:0/2850269445 >> 172.21.15.31:0/2850269445 conn(0x7fc8e00b0c40 msgr2=0x7fc8e00b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 -- 172.21.15.31:0/2850269445 shutdown_connections 2023-11-26T07:21:46.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 -- 172.21.15.31:0/2850269445 wait complete. 2023-11-26T07:21:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 Processor -- start 2023-11-26T07:21:46.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 -- start start 2023-11-26T07:21:46.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e006dfd0 0x7fc8e015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8e00ae3d0 0x7fc8e009c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 0x7fc8e01f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e015d820 con 0x7fc8e0143ae0 2023-11-26T07:21:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e01f7330 con 0x7fc8e006dfd0 2023-11-26T07:21:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.006+0000 7fc8e506b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e015d2d0 con 0x7fc8e00ae3d0 2023-11-26T07:21:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8df59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 0x7fc8e01f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8df59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 0x7fc8e01f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43024/0 (socket says 172.21.15.31:43024) 2023-11-26T07:21:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8df59e640 1 -- 172.21.15.31:0/1714232283 learned_addr learned my addr 172.21.15.31:0/1714232283 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8d7fff640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8e00ae3d0 0x7fc8e009c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8ded9d640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e006dfd0 0x7fc8e015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8df59e640 1 -- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8e00ae3d0 msgr2=0x7fc8e009c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8df59e640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8e00ae3d0 0x7fc8e009c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8df59e640 1 -- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e006dfd0 msgr2=0x7fc8e015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8df59e640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e006dfd0 0x7fc8e015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8df59e640 1 -- 172.21.15.31:0/1714232283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d000b770 con 0x7fc8e0143ae0 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8df59e640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 0x7fc8e01f8fe0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8007e70 tx=0x7fc8d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8d77fe640 1 -- 172.21.15.31:0/1714232283 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8014620 con 0x7fc8e0143ae0 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8d77fe640 1 -- 172.21.15.31:0/1714232283 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d80147a0 con 0x7fc8e0143ae0 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8e506b640 1 -- 172.21.15.31:0/1714232283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e01f7000 con 0x7fc8e0143ae0 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8d77fe640 1 -- 172.21.15.31:0/1714232283 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8014b50 con 0x7fc8e0143ae0 2023-11-26T07:21:46.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8e506b640 1 -- 172.21.15.31:0/1714232283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e015d5f0 con 0x7fc8e0143ae0 2023-11-26T07:21:46.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8e506b640 1 -- 172.21.15.31:0/1714232283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e01f95e0 con 0x7fc8e0143ae0 2023-11-26T07:21:46.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8d77fe640 1 -- 172.21.15.31:0/1714232283 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fc8d8014cb0 con 0x7fc8e0143ae0 2023-11-26T07:21:46.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.010+0000 7fc8d77fe640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8c00667d0 0x7fc8c0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.014+0000 7fc8d77fe640 1 -- 172.21.15.31:0/1714232283 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc8d8084a70 con 0x7fc8e0143ae0 2023-11-26T07:21:46.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.014+0000 7fc8ded9d640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8c00667d0 0x7fc8c0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.014+0000 7fc8ded9d640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8c00667d0 0x7fc8c0068c90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b1000 tx=0x7fc8d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.018+0000 7fc8d77fe640 1 -- 172.21.15.31:0/1714232283 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc8d8051df0 con 0x7fc8e0143ae0 2023-11-26T07:21:46.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.174+0000 7fc8e506b640 1 -- 172.21.15.31:0/1714232283 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc8e01f7470 con 0x7fc8e0143ae0 2023-11-26T07:21:46.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.174+0000 7fc8d77fe640 1 -- 172.21.15.31:0/1714232283 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc8e01f7470 con 0x7fc8e0143ae0 2023-11-26T07:21:46.177 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:46.177 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:21:46.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:46.177 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-11-26T07:21:46.177 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-11-26T07:21:46.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:46.177 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.174+0000 7fc8e506b640 1 -- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8c00667d0 msgr2=0x7fc8c0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.174+0000 7fc8e506b640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8c00667d0 0x7fc8c0068c90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00b1000 tx=0x7fc8d0008040 comp rx=0 tx=0).stop 2023-11-26T07:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.174+0000 7fc8e506b640 1 -- 172.21.15.31:0/1714232283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 msgr2=0x7fc8e01f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.174+0000 7fc8e506b640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 0x7fc8e01f8fe0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8007e70 tx=0x7fc8d800e4b0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.178+0000 7fc8e506b640 1 -- 172.21.15.31:0/1714232283 shutdown_connections 2023-11-26T07:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.178+0000 7fc8e506b640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8c00667d0 0x7fc8c0068c90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.178+0000 7fc8e506b640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8e0143ae0 0x7fc8e01f8fe0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.178+0000 7fc8e506b640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8e00ae3d0 0x7fc8e009c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.178+0000 7fc8e506b640 1 --2- 172.21.15.31:0/1714232283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8e006dfd0 0x7fc8e015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.178+0000 7fc8e506b640 1 -- 172.21.15.31:0/1714232283 >> 172.21.15.31:0/1714232283 conn(0x7fc8e00b0c40 msgr2=0x7fc8e006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.178+0000 7fc8e506b640 1 -- 172.21.15.31:0/1714232283 shutdown_connections 2023-11-26T07:21:46.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.178+0000 7fc8e506b640 1 -- 172.21.15.31:0/1714232283 wait complete. 2023-11-26T07:21:46.198 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:21:46.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:21:46.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.342+0000 7f3783d57640 1 Processor -- start 2023-11-26T07:21:46.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.342+0000 7f3783d57640 1 -- start start 2023-11-26T07:21:46.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f3783d57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f377c0cb9b0 0x7f377c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f3783d57640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f377c0cbd90 0x7f377c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f3783d57640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377c159630 con 0x7f377c0ce5d0 2023-11-26T07:21:46.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f3783d57640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377c15bba0 con 0x7f377c0cbd90 2023-11-26T07:21:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f3783d57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377c0cdd50 con 0x7f377c0cb9b0 2023-11-26T07:21:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f37812f2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f377c0cbd90 0x7f377c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f37812f2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f377c0cbd90 0x7f377c15f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43032/0 (socket says 172.21.15.31:43032) 2023-11-26T07:21:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f37812f2640 1 -- 172.21.15.31:0/2742549527 learned_addr learned my addr 172.21.15.31:0/2742549527 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f37822f4640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f377c0ce5d0 0x7f377c15bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45694/0 (socket says 172.21.15.31:45694) 2023-11-26T07:21:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f3781af3640 1 --2- 172.21.15.31:0/2742549527 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f377c0cb9b0 0x7f377c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f37812f2640 1 -- 172.21.15.31:0/2742549527 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f377c0cb9b0 msgr2=0x7f377c06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f37812f2640 1 --2- 172.21.15.31:0/2742549527 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f377c0cb9b0 0x7f377c06e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f37812f2640 1 -- 172.21.15.31:0/2742549527 >> v1:172.21.15.31:6789/0 conn(0x7f377c0ce5d0 legacy=0x7f377c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.346+0000 7f37812f2640 1 -- 172.21.15.31:0/2742549527 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c15f6d0 con 0x7f377c0cbd90 2023-11-26T07:21:46.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f37812f2640 1 --2- 172.21.15.31:0/2742549527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0cbd90 0x7f377c15f190 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f376c009870 tx=0x7f376c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e4e507cd5ea6ea server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f3780af1640 1 -- 172.21.15.31:0/2742549527 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c0aa670 con 0x7f377c0cbd90 2023-11-26T07:21:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f3780af1640 1 -- 172.21.15.31:0/2742549527 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c0aa7d0 con 0x7f377c0cbd90 2023-11-26T07:21:46.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f3780af1640 1 -- 172.21.15.31:0/2742549527 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c0aab60 con 0x7f377c0cbd90 2023-11-26T07:21:46.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f3783d57640 1 -- 172.21.15.31:0/2742549527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0cbd90 msgr2=0x7f377c15f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f3783d57640 1 --2- 172.21.15.31:0/2742549527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0cbd90 0x7f377c15f190 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f376c009870 tx=0x7f376c01c510 comp rx=0 tx=0).stop 2023-11-26T07:21:46.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f3783d57640 1 -- 172.21.15.31:0/2742549527 shutdown_connections 2023-11-26T07:21:46.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f3783d57640 1 --2- 172.21.15.31:0/2742549527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0cbd90 0x7f377c15f190 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f3783d57640 1 --2- 172.21.15.31:0/2742549527 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f377c0cb9b0 0x7f377c06e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f3783d57640 1 -- 172.21.15.31:0/2742549527 >> 172.21.15.31:0/2742549527 conn(0x7f377c0b0cb0 msgr2=0x7f377c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.350+0000 7f3783d57640 1 -- 172.21.15.31:0/2742549527 shutdown_connections 2023-11-26T07:21:46.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f3783d57640 1 -- 172.21.15.31:0/2742549527 wait complete. 2023-11-26T07:21:46.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f3783d57640 1 Processor -- start 2023-11-26T07:21:46.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f3783d57640 1 -- start start 2023-11-26T07:21:46.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f3783d57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c0cb9b0 0x7f377c1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f3783d57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f377c0cbd90 0x7f377c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f3783d57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0ce5d0 0x7f377c152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f3783d57640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c15d180 con 0x7f377c0ce5d0 2023-11-26T07:21:46.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f3783d57640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c1575b0 con 0x7f377c0cb9b0 2023-11-26T07:21:46.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f3783d57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377c157830 con 0x7f377c0cbd90 2023-11-26T07:21:46.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f37822f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0ce5d0 0x7f377c152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f37822f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0ce5d0 0x7f377c152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43034/0 (socket says 172.21.15.31:43034) 2023-11-26T07:21:46.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f37822f4640 1 -- 172.21.15.31:0/1493432559 learned_addr learned my addr 172.21.15.31:0/1493432559 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:46.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f3781af3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c0cb9b0 0x7f377c1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f37812f2640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f377c0cbd90 0x7f377c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f37822f4640 1 -- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f377c0cbd90 msgr2=0x7f377c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f37822f4640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f377c0cbd90 0x7f377c1537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f37822f4640 1 -- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c0cb9b0 msgr2=0x7f377c1585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f37822f4640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c0cb9b0 0x7f377c1585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f37822f4640 1 -- 172.21.15.31:0/1493432559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c0094d0 con 0x7f377c0ce5d0 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.354+0000 7f37822f4640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0ce5d0 0x7f377c152fc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f377400c730 tx=0x7f377400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.358+0000 7f37727fc640 1 -- 172.21.15.31:0/1493432559 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377400ce40 con 0x7f377c0ce5d0 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.358+0000 7f37727fc640 1 -- 172.21.15.31:0/1493432559 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377400a640 con 0x7f377c0ce5d0 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.358+0000 7f3783d57640 1 -- 172.21.15.31:0/1493432559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c157b10 con 0x7f377c0ce5d0 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.358+0000 7f37727fc640 1 -- 172.21.15.31:0/1493432559 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377400a9b0 con 0x7f377c0ce5d0 2023-11-26T07:21:46.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.358+0000 7f3783d57640 1 -- 172.21.15.31:0/1493432559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c15cda0 con 0x7f377c0ce5d0 2023-11-26T07:21:46.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.358+0000 7f3783d57640 1 -- 172.21.15.31:0/1493432559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377c0671f0 con 0x7f377c0ce5d0 2023-11-26T07:21:46.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.358+0000 7f37727fc640 1 -- 172.21.15.31:0/1493432559 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f377400ab10 con 0x7f377c0ce5d0 2023-11-26T07:21:46.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.358+0000 7f37727fc640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f37600667d0 0x7f3760068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:46.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.358+0000 7f37727fc640 1 -- 172.21.15.31:0/1493432559 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f37740837f0 con 0x7f377c0ce5d0 2023-11-26T07:21:46.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.358+0000 7f3781af3640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f37600667d0 0x7f3760068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:46.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.362+0000 7f3781af3640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f37600667d0 0x7f3760068c90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f377c154c40 tx=0x7f3778009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:46.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.362+0000 7f37727fc640 1 -- 172.21.15.31:0/1493432559 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3774050bb0 con 0x7f377c0ce5d0 2023-11-26T07:21:46.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:46.522+0000 7f3783d57640 1 -- 172.21.15.31:0/1493432559 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f377c15d2c0 con 0x7f377c0ce5d0 2023-11-26T07:21:47.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f37727fc640 1 -- 172.21.15.31:0/1493432559 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f37740170b0 con 0x7f377c0ce5d0 2023-11-26T07:21:47.364 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-11-26T07:21:47.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f3783d57640 1 -- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f37600667d0 msgr2=0x7f3760068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f3783d57640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f37600667d0 0x7f3760068c90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f377c154c40 tx=0x7f3778009190 comp rx=0 tx=0).stop 2023-11-26T07:21:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f3783d57640 1 -- 172.21.15.31:0/1493432559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0ce5d0 msgr2=0x7f377c152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f3783d57640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0ce5d0 0x7f377c152fc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f377400c730 tx=0x7f377400cbf0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f3783d57640 1 -- 172.21.15.31:0/1493432559 shutdown_connections 2023-11-26T07:21:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f3783d57640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f37600667d0 0x7f3760068c90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f3783d57640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c0ce5d0 0x7f377c152fc0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f3783d57640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f377c0cbd90 0x7f377c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f3783d57640 1 --2- 172.21.15.31:0/1493432559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f377c0cb9b0 0x7f377c1585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.362+0000 7f3783d57640 1 -- 172.21.15.31:0/1493432559 >> 172.21.15.31:0/1493432559 conn(0x7f377c0b0cb0 msgr2=0x7f377c0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.366+0000 7f3783d57640 1 -- 172.21.15.31:0/1493432559 shutdown_connections 2023-11-26T07:21:47.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.366+0000 7f3783d57640 1 -- 172.21.15.31:0/1493432559 wait complete. 2023-11-26T07:21:47.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:21:47.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.546+0000 7f00c2daa640 1 Processor -- start 2023-11-26T07:21:47.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.546+0000 7f00c2daa640 1 -- start start 2023-11-26T07:21:47.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.546+0000 7f00c2daa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f00bc15d8f0 0x7f00bc15e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.546+0000 7f00c2daa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00bc0ced30 0x7f00bc0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c2daa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc15ecc0 con 0x7f00bc0cf1b0 2023-11-26T07:21:47.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c2daa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00bc0d4420 con 0x7f00bc15d8f0 2023-11-26T07:21:47.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c2daa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0cdfb0 con 0x7f00bc0ced30 2023-11-26T07:21:47.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c0b46640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f00bc15d8f0 0x7f00bc15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c0b46640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f00bc15d8f0 0x7f00bc15e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43038/0 (socket says 172.21.15.31:43038) 2023-11-26T07:21:47.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c0b46640 1 -- 172.21.15.31:0/2152403049 learned_addr learned my addr 172.21.15.31:0/2152403049 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:47.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00bbfff640 1 --2- 172.21.15.31:0/2152403049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00bc0ced30 0x7f00bc0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c0b46640 1 -- 172.21.15.31:0/2152403049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00bc0ced30 msgr2=0x7f00bc0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:47.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c0b46640 1 --2- 172.21.15.31:0/2152403049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00bc0ced30 0x7f00bc0cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c0b46640 1 -- 172.21.15.31:0/2152403049 >> v1:172.21.15.31:6789/0 conn(0x7f00bc0cf1b0 legacy=0x7f00bc0d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c0b46640 1 -- 172.21.15.31:0/2152403049 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc0d4140 con 0x7f00bc15d8f0 2023-11-26T07:21:47.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00bb7fe640 1 -- 172.21.15.31:0/2152403049 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762355058 0 0) 0x7f00bc15ecc0 con 0x7f00bc0cf1b0 2023-11-26T07:21:47.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.550+0000 7f00c0b46640 1 --2- 172.21.15.31:0/2152403049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc15d8f0 0x7f00bc15e780 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009870 tx=0x7f00b001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53424286f933ac06 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:47.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00bb7fe640 1 -- 172.21.15.31:0/2152403049 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b00aa670 con 0x7f00bc15d8f0 2023-11-26T07:21:47.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00bb7fe640 1 -- 172.21.15.31:0/2152403049 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b00aa7d0 con 0x7f00bc15d8f0 2023-11-26T07:21:47.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00bb7fe640 1 -- 172.21.15.31:0/2152403049 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b00aab60 con 0x7f00bc15d8f0 2023-11-26T07:21:47.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 -- 172.21.15.31:0/2152403049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc15d8f0 msgr2=0x7f00bc15e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 --2- 172.21.15.31:0/2152403049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc15d8f0 0x7f00bc15e780 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009870 tx=0x7f00b001c510 comp rx=0 tx=0).stop 2023-11-26T07:21:47.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 -- 172.21.15.31:0/2152403049 shutdown_connections 2023-11-26T07:21:47.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 --2- 172.21.15.31:0/2152403049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00bc0ced30 0x7f00bc0cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 --2- 172.21.15.31:0/2152403049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc15d8f0 0x7f00bc15e780 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 -- 172.21.15.31:0/2152403049 >> 172.21.15.31:0/2152403049 conn(0x7f00bc09c750 msgr2=0x7f00bc0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:47.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 -- 172.21.15.31:0/2152403049 shutdown_connections 2023-11-26T07:21:47.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 -- 172.21.15.31:0/2152403049 wait complete. 2023-11-26T07:21:47.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 Processor -- start 2023-11-26T07:21:47.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 -- start start 2023-11-26T07:21:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc0ced30 0x7f00bc156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00bc0cf1b0 0x7f00bc0b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00bc15d8f0 0x7f00bc157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc1564f0 con 0x7f00bc0ced30 2023-11-26T07:21:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc155bc0 con 0x7f00bc0cf1b0 2023-11-26T07:21:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c2daa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc156e50 con 0x7f00bc15d8f0 2023-11-26T07:21:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.554+0000 7f00c0b46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc0ced30 0x7f00bc156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c0b46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc0ced30 0x7f00bc156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43046/0 (socket says 172.21.15.31:43046) 2023-11-26T07:21:47.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c0b46640 1 -- 172.21.15.31:0/142594909 learned_addr learned my addr 172.21.15.31:0/142594909 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00bbfff640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00bc0cf1b0 0x7f00bc0b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c1347640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00bc15d8f0 0x7f00bc157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c0b46640 1 -- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00bc15d8f0 msgr2=0x7f00bc157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c0b46640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00bc15d8f0 0x7f00bc157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c0b46640 1 -- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00bc0cf1b0 msgr2=0x7f00bc0b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c0b46640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00bc0cf1b0 0x7f00bc0b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c0b46640 1 -- 172.21.15.31:0/142594909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00094d0 con 0x7f00bc0ced30 2023-11-26T07:21:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00bbfff640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00bc0cf1b0 0x7f00bc0b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:47.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c0b46640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc0ced30 0x7f00bc156910 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f00b00ac040 tx=0x7f00b001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:47.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00b97fa640 1 -- 172.21.15.31:0/142594909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b00b3800 con 0x7f00bc0ced30 2023-11-26T07:21:47.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00b97fa640 1 -- 172.21.15.31:0/142594909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b00b3960 con 0x7f00bc0ced30 2023-11-26T07:21:47.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00b97fa640 1 -- 172.21.15.31:0/142594909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b00b3d10 con 0x7f00bc0ced30 2023-11-26T07:21:47.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c2daa640 1 -- 172.21.15.31:0/142594909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00bc155e30 con 0x7f00bc0ced30 2023-11-26T07:21:47.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c2daa640 1 -- 172.21.15.31:0/142594909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc144920 con 0x7f00bc0ced30 2023-11-26T07:21:47.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00c2daa640 1 -- 172.21.15.31:0/142594909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00bc156760 con 0x7f00bc0ced30 2023-11-26T07:21:47.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.558+0000 7f00b97fa640 1 -- 172.21.15.31:0/142594909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f00b00c73a0 con 0x7f00bc0ced30 2023-11-26T07:21:47.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.562+0000 7f00b97fa640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f00a0066800 0x7f00a0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:47.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.562+0000 7f00b97fa640 1 -- 172.21.15.31:0/142594909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f00b011edd0 con 0x7f00bc0ced30 2023-11-26T07:21:47.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.562+0000 7f00bbfff640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f00a0066800 0x7f00a0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:47.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.562+0000 7f00bbfff640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f00a0066800 0x7f00a0068cc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f00bc1fa040 tx=0x7f00a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:47.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.566+0000 7f00b97fa640 1 -- 172.21.15.31:0/142594909 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f00b00ec000 con 0x7f00bc0ced30 2023-11-26T07:21:47.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:47.726+0000 7f00c2daa640 1 -- 172.21.15.31:0/142594909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f00bc04e850 con 0x7f00bc0ced30 2023-11-26T07:21:48.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00b97fa640 1 -- 172.21.15.31:0/142594909 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) v1 ==== 170+0+0 (secure 0 0 0) 0x7f00b00ef620 con 0x7f00bc0ced30 2023-11-26T07:21:48.367 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-11-26T07:21:48.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 -- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f00a0066800 msgr2=0x7f00a0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:48.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f00a0066800 0x7f00a0068cc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f00bc1fa040 tx=0x7f00a8008040 comp rx=0 tx=0).stop 2023-11-26T07:21:48.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 -- 172.21.15.31:0/142594909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc0ced30 msgr2=0x7f00bc156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc0ced30 0x7f00bc156910 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f00b00ac040 tx=0x7f00b001ad50 comp rx=0 tx=0).stop 2023-11-26T07:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 -- 172.21.15.31:0/142594909 shutdown_connections 2023-11-26T07:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f00a0066800 0x7f00a0068cc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f00bc15d8f0 0x7f00bc157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00bc0cf1b0 0x7f00bc0b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 --2- 172.21.15.31:0/142594909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00bc0ced30 0x7f00bc156910 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 -- 172.21.15.31:0/142594909 >> 172.21.15.31:0/142594909 conn(0x7f00bc09c750 msgr2=0x7f00bc15dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 -- 172.21.15.31:0/142594909 shutdown_connections 2023-11-26T07:21:48.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.366+0000 7f00c2daa640 1 -- 172.21.15.31:0/142594909 wait complete. 2023-11-26T07:21:48.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:21:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f90a2101640 1 Processor -- start 2023-11-26T07:21:48.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f90a2101640 1 -- start start 2023-11-26T07:21:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f90a2101640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f909c0b5830 0x7f909c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f90a2101640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 0x7f909c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f90a2101640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c0af630 con 0x7f909c0acd70 2023-11-26T07:21:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f90a2101640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0acb10 con 0x7f909c0b5830 2023-11-26T07:21:48.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f90a2101640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0ae5b0 con 0x7f909c0ad8e0 2023-11-26T07:21:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f909bfff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 0x7f909c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f909b7fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f909c0b5830 0x7f909c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f909bfff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 0x7f909c06d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:44632/0 (socket says 172.21.15.31:44632) 2023-11-26T07:21:48.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f909bfff640 1 -- 172.21.15.31:0/2949699776 learned_addr learned my addr 172.21.15.31:0/2949699776 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f909bfff640 1 -- 172.21.15.31:0/2949699776 >> v2:172.21.15.97:3300/0 conn(0x7f909c0b5830 msgr2=0x7f909c15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f909bfff640 1 --2- 172.21.15.31:0/2949699776 >> v2:172.21.15.97:3300/0 conn(0x7f909c0b5830 0x7f909c15e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f909bfff640 1 -- 172.21.15.31:0/2949699776 >> v1:172.21.15.97:6789/0 conn(0x7f909c0acd70 legacy=0x7f909c0ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:48.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.554+0000 7f909bfff640 1 -- 172.21.15.31:0/2949699776 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0ae870 con 0x7f909c0ad8e0 2023-11-26T07:21:48.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f909bfff640 1 --2- 172.21.15.31:0/2949699776 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 0x7f909c06d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9094009e50 tx=0x7f909401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90717c13d34e797d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:48.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f909a7fc640 1 -- 172.21.15.31:0/2949699776 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940a58e0 con 0x7f909c0ad8e0 2023-11-26T07:21:48.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f909a7fc640 1 -- 172.21.15.31:0/2949699776 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90940a5a40 con 0x7f909c0ad8e0 2023-11-26T07:21:48.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f909a7fc640 1 -- 172.21.15.31:0/2949699776 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940a5dd0 con 0x7f909c0ad8e0 2023-11-26T07:21:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f90a2101640 1 -- 172.21.15.31:0/2949699776 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 msgr2=0x7f909c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f90a2101640 1 --2- 172.21.15.31:0/2949699776 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 0x7f909c06d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9094009e50 tx=0x7f909401ca10 comp rx=0 tx=0).stop 2023-11-26T07:21:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f90a2101640 1 -- 172.21.15.31:0/2949699776 shutdown_connections 2023-11-26T07:21:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f90a2101640 1 --2- 172.21.15.31:0/2949699776 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 0x7f909c06d190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f90a2101640 1 --2- 172.21.15.31:0/2949699776 >> v2:172.21.15.97:3300/0 conn(0x7f909c0b5830 0x7f909c15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f90a2101640 1 -- 172.21.15.31:0/2949699776 >> 172.21.15.31:0/2949699776 conn(0x7f909c071d80 msgr2=0x7f909c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:48.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f90a2101640 1 -- 172.21.15.31:0/2949699776 shutdown_connections 2023-11-26T07:21:48.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.558+0000 7f90a2101640 1 -- 172.21.15.31:0/2949699776 wait complete. 2023-11-26T07:21:48.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f90a2101640 1 Processor -- start 2023-11-26T07:21:48.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f90a2101640 1 -- start start 2023-11-26T07:21:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f90a2101640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c0acd70 0x7f909c13e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f90a2101640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 0x7f909c157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f90a2101640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0b5830 0x7f909c13eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f90a2101640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c162950 con 0x7f909c0b5830 2023-11-26T07:21:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f90a2101640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c141a40 con 0x7f909c0acd70 2023-11-26T07:21:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f90a2101640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c162390 con 0x7f909c0ad8e0 2023-11-26T07:21:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0b5830 0x7f909c13eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.565 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c0acd70 0x7f909c13e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0b5830 0x7f909c13eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43060/0 (socket says 172.21.15.31:43060) 2023-11-26T07:21:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909bfff640 1 -- 172.21.15.31:0/2339609149 learned_addr learned my addr 172.21.15.31:0/2339609149 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c0acd70 0x7f909c13e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:53568/0 (socket says 172.21.15.31:53568) 2023-11-26T07:21:48.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909affd640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 0x7f909c157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909bfff640 1 -- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 msgr2=0x7f909c157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909bfff640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 0x7f909c157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909bfff640 1 -- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c0acd70 msgr2=0x7f909c13e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:48.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909bfff640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c0acd70 0x7f909c13e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909bfff640 1 -- 172.21.15.31:0/2339609149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094009b30 con 0x7f909c0b5830 2023-11-26T07:21:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f909bfff640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0b5830 0x7f909c13eb10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f90940af040 tx=0x7f90940197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f9083fff640 1 -- 172.21.15.31:0/2339609149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940b8640 con 0x7f909c0b5830 2023-11-26T07:21:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f9083fff640 1 -- 172.21.15.31:0/2339609149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90940b87e0 con 0x7f909c0b5830 2023-11-26T07:21:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f9083fff640 1 -- 172.21.15.31:0/2339609149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940b8b90 con 0x7f909c0b5830 2023-11-26T07:21:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.562+0000 7f90a2101640 1 -- 172.21.15.31:0/2339609149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909c162670 con 0x7f909c0b5830 2023-11-26T07:21:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.566+0000 7f90a2101640 1 -- 172.21.15.31:0/2339609149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c13e6f0 con 0x7f909c0b5830 2023-11-26T07:21:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.566+0000 7f90a2101640 1 -- 172.21.15.31:0/2339609149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c156a90 con 0x7f909c0b5830 2023-11-26T07:21:48.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.566+0000 7f9083fff640 1 -- 172.21.15.31:0/2339609149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f90940b8cf0 con 0x7f909c0b5830 2023-11-26T07:21:48.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.566+0000 7f9083fff640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f90880667d0 0x7f9088068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:48.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.566+0000 7f9083fff640 1 -- 172.21.15.31:0/2339609149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f909411f3f0 con 0x7f909c0b5830 2023-11-26T07:21:48.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.566+0000 7f909b7fe640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f90880667d0 0x7f9088068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:48.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.570+0000 7f909b7fe640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f90880667d0 0x7f9088068c90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f908c009b60 tx=0x7f908c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:48.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.574+0000 7f9083fff640 1 -- 172.21.15.31:0/2339609149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f909c156a90 con 0x7f909c0b5830 2023-11-26T07:21:48.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:48.762+0000 7f90a2101640 1 -- 172.21.15.31:0/2339609149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f909c156a90 con 0x7f909c0b5830 2023-11-26T07:21:49.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.390+0000 7f9083fff640 1 -- 172.21.15.31:0/2339609149 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v43) v1 ==== 174+0+0 (secure 0 0 0) 0x7f909c156a90 con 0x7f909c0b5830 2023-11-26T07:21:49.393 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 10 and data pool 11 2023-11-26T07:21:49.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.398+0000 7f90a2101640 1 -- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f90880667d0 msgr2=0x7f9088068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.398+0000 7f90a2101640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f90880667d0 0x7f9088068c90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f908c009b60 tx=0x7f908c009190 comp rx=0 tx=0).stop 2023-11-26T07:21:49.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.398+0000 7f90a2101640 1 -- 172.21.15.31:0/2339609149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0b5830 msgr2=0x7f909c13eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.398+0000 7f90a2101640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0b5830 0x7f909c13eb10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f90940af040 tx=0x7f90940197b0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.398+0000 7f90a2101640 1 -- 172.21.15.31:0/2339609149 shutdown_connections 2023-11-26T07:21:49.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.398+0000 7f90a2101640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f90880667d0 0x7f9088068c90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.398+0000 7f90a2101640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0b5830 0x7f909c13eb10 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.398+0000 7f90a2101640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f909c0ad8e0 0x7f909c157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.398+0000 7f90a2101640 1 --2- 172.21.15.31:0/2339609149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f909c0acd70 0x7f909c13e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.398+0000 7f90a2101640 1 -- 172.21.15.31:0/2339609149 >> 172.21.15.31:0/2339609149 conn(0x7f909c071d80 msgr2=0x7f909c0b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:49.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.402+0000 7f90a2101640 1 -- 172.21.15.31:0/2339609149 shutdown_connections 2023-11-26T07:21:49.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.402+0000 7f90a2101640 1 -- 172.21.15.31:0/2339609149 wait complete. 2023-11-26T07:21:49.433 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:49.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.574+0000 7f0fb9301640 1 Processor -- start 2023-11-26T07:21:49.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.574+0000 7f0fb9301640 1 -- start start 2023-11-26T07:21:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb9301640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0fb4159880 0x7f0fb40b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb9301640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb4071600 0x7f0fb406d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb9301640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb40b41a0 con 0x7f0fb40cb230 2023-11-26T07:21:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb9301640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb40b3be0 con 0x7f0fb4159880 2023-11-26T07:21:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb9301640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb40b3ec0 con 0x7f0fb4071600 2023-11-26T07:21:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fabfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0fb40cb230 0x7f0fb4070ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45710/0 (socket says 172.21.15.31:45710) 2023-11-26T07:21:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fabfff640 1 -- 172.21.15.31:0/1031701666 learned_addr learned my addr 172.21.15.31:0/1031701666 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb37fe640 1 --2- 172.21.15.31:0/1031701666 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb4071600 0x7f0fb406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb2ffd640 1 --2- 172.21.15.31:0/1031701666 >> v2:172.21.15.97:3300/0 conn(0x7f0fb4159880 0x7f0fb40b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb27fc640 1 -- 172.21.15.31:0/1031701666 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817296789 0 0) 0x7f0fb40b41a0 con 0x7f0fb40cb230 2023-11-26T07:21:49.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb27fc640 1 -- 172.21.15.31:0/1031701666 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fa0003530 con 0x7f0fb40cb230 2023-11-26T07:21:49.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb2ffd640 1 -- 172.21.15.31:0/1031701666 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb4071600 msgr2=0x7f0fb406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:49.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb2ffd640 1 --2- 172.21.15.31:0/1031701666 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb4071600 0x7f0fb406d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb2ffd640 1 -- 172.21.15.31:0/1031701666 >> v1:172.21.15.31:6789/0 conn(0x7f0fb40cb230 legacy=0x7f0fb4070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb2ffd640 1 -- 172.21.15.31:0/1031701666 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb406cf60 con 0x7f0fb4159880 2023-11-26T07:21:49.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.578+0000 7f0fb37fe640 1 --2- 172.21.15.31:0/1031701666 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb4071600 0x7f0fb406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb2ffd640 1 --2- 172.21.15.31:0/1031701666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fb4159880 0x7f0fb40b3320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400a960 tx=0x7f0fa401e8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80ea61e5b3db0c9e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb27fc640 1 -- 172.21.15.31:0/1031701666 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40a78d0 con 0x7f0fb4159880 2023-11-26T07:21:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb27fc640 1 -- 172.21.15.31:0/1031701666 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa40a7a30 con 0x7f0fb4159880 2023-11-26T07:21:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb27fc640 1 -- 172.21.15.31:0/1031701666 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40a7dc0 con 0x7f0fb4159880 2023-11-26T07:21:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 -- 172.21.15.31:0/1031701666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fb4159880 msgr2=0x7f0fb40b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 --2- 172.21.15.31:0/1031701666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fb4159880 0x7f0fb40b3320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400a960 tx=0x7f0fa401e8a0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 -- 172.21.15.31:0/1031701666 shutdown_connections 2023-11-26T07:21:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 --2- 172.21.15.31:0/1031701666 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb4071600 0x7f0fb406d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 --2- 172.21.15.31:0/1031701666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fb4159880 0x7f0fb40b3320 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 -- 172.21.15.31:0/1031701666 >> 172.21.15.31:0/1031701666 conn(0x7f0fb40b0c40 msgr2=0x7f0fb40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 -- 172.21.15.31:0/1031701666 shutdown_connections 2023-11-26T07:21:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 -- 172.21.15.31:0/1031701666 wait complete. 2023-11-26T07:21:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 Processor -- start 2023-11-26T07:21:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 -- start start 2023-11-26T07:21:49.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fb4071600 0x7f0fb409afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb40cb230 0x7f0fb40cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fb4159880 0x7f0fb40cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb41590b0 con 0x7f0fb4159880 2023-11-26T07:21:49.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb409b500 con 0x7f0fb4071600 2023-11-26T07:21:49.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.582+0000 7f0fb9301640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb409c540 con 0x7f0fb40cb230 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fb4159880 0x7f0fb40cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fb4159880 0x7f0fb40cdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43064/0 (socket says 172.21.15.31:43064) 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb37fe640 1 -- 172.21.15.31:0/4180366110 learned_addr learned my addr 172.21.15.31:0/4180366110 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fabfff640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb40cb230 0x7f0fb40cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb2ffd640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fb4071600 0x7f0fb409afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb37fe640 1 -- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb40cb230 msgr2=0x7f0fb40cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb37fe640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb40cb230 0x7f0fb40cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb37fe640 1 -- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fb4071600 msgr2=0x7f0fb409afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb37fe640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fb4071600 0x7f0fb409afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb37fe640 1 -- 172.21.15.31:0/4180366110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac009a70 con 0x7f0fb4159880 2023-11-26T07:21:49.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb37fe640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fb4159880 0x7f0fb40cdb50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00efc0 tx=0x7f0fac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:49.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fab7fe640 1 -- 172.21.15.31:0/4180366110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac00cdf0 con 0x7f0fb4159880 2023-11-26T07:21:49.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fab7fe640 1 -- 172.21.15.31:0/4180366110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac00f040 con 0x7f0fb4159880 2023-11-26T07:21:49.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fab7fe640 1 -- 172.21.15.31:0/4180366110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac003be0 con 0x7f0fb4159880 2023-11-26T07:21:49.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb9301640 1 -- 172.21.15.31:0/4180366110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa400a5c0 con 0x7f0fb4159880 2023-11-26T07:21:49.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb9301640 1 -- 172.21.15.31:0/4180366110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb409a7c0 con 0x7f0fb4159880 2023-11-26T07:21:49.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fb9301640 1 -- 172.21.15.31:0/4180366110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb404e850 con 0x7f0fb4159880 2023-11-26T07:21:49.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.586+0000 7f0fab7fe640 1 -- 172.21.15.31:0/4180366110 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f0fac014480 con 0x7f0fb4159880 2023-11-26T07:21:49.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.590+0000 7f0fab7fe640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0f940667d0 0x7f0f94068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.590+0000 7f0fab7fe640 1 -- 172.21.15.31:0/4180366110 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fac0856d0 con 0x7f0fb4159880 2023-11-26T07:21:49.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.590+0000 7f0fb2ffd640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0f940667d0 0x7f0f94068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.590+0000 7f0fb2ffd640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0f940667d0 0x7f0f94068c90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40b1000 tx=0x7f0fa40a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:49.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.594+0000 7f0fab7fe640 1 -- 172.21.15.31:0/4180366110 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0fac052740 con 0x7f0fb4159880 2023-11-26T07:21:49.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.750+0000 7f0fb9301640 1 -- 172.21.15.31:0/4180366110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fb40b24c0 con 0x7f0fb4159880 2023-11-26T07:21:49.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.750+0000 7f0fab7fe640 1 -- 172.21.15.31:0/4180366110 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16497 (secure 0 0 0) 0x7f0fac055d60 con 0x7f0fb4159880 2023-11-26T07:21:49.753 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:49.753 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":67,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:48.765454+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:46.570507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:47.773615+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 -- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0f940667d0 msgr2=0x7f0f94068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0f940667d0 0x7f0f94068c90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40b1000 tx=0x7f0fa40a9070 comp rx=0 tx=0).stop 2023-11-26T07:21:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 -- 172.21.15.31:0/4180366110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fb4159880 msgr2=0x7f0fb40cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fb4159880 0x7f0fb40cdb50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00efc0 tx=0x7f0fac00c490 comp rx=0 tx=0).stop 2023-11-26T07:21:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 -- 172.21.15.31:0/4180366110 shutdown_connections 2023-11-26T07:21:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f0f940667d0 0x7f0f94068c90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fb4159880 0x7f0fb40cdb50 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0fb40cb230 0x7f0fb40cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 --2- 172.21.15.31:0/4180366110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0fb4071600 0x7f0fb409afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 -- 172.21.15.31:0/4180366110 >> 172.21.15.31:0/4180366110 conn(0x7f0fb40b0c40 msgr2=0x7f0fb4154990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 -- 172.21.15.31:0/4180366110 shutdown_connections 2023-11-26T07:21:49.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.754+0000 7f0fb9301640 1 -- 172.21.15.31:0/4180366110 wait complete. 2023-11-26T07:21:49.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.914+0000 7f9dcca16640 1 Processor -- start 2023-11-26T07:21:49.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.914+0000 7f9dcca16640 1 -- start start 2023-11-26T07:21:49.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dcca16640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc815b4d0 0x7f9dc80cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dcca16640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9dc80cda20 0x7f9dc806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dcca16640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc80ad0d0 con 0x7f9dc806aaf0 2023-11-26T07:21:49.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dcca16640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc806af80 con 0x7f9dc80cda20 2023-11-26T07:21:49.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dcca16640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80ad3b0 con 0x7f9dc815b4d0 2023-11-26T07:21:49.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dc6d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9dc806aaf0 0x7f9dc80afbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45724/0 (socket says 172.21.15.31:45724) 2023-11-26T07:21:49.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dc6d9d640 1 -- 172.21.15.31:0/2927186698 learned_addr learned my addr 172.21.15.31:0/2927186698 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:49.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dbffff640 1 --2- 172.21.15.31:0/2927186698 >> v2:172.21.15.97:3300/0 conn(0x7f9dc80cda20 0x7f9dc806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dc659c640 1 --2- 172.21.15.31:0/2927186698 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc815b4d0 0x7f9dc80cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dc5d9b640 1 -- 172.21.15.31:0/2927186698 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054463894 0 0) 0x7f9dc80ad0d0 con 0x7f9dc806aaf0 2023-11-26T07:21:49.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dbffff640 1 -- 172.21.15.31:0/2927186698 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc815b4d0 msgr2=0x7f9dc80cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:49.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dbffff640 1 --2- 172.21.15.31:0/2927186698 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc815b4d0 0x7f9dc80cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dbffff640 1 -- 172.21.15.31:0/2927186698 >> v1:172.21.15.31:6789/0 conn(0x7f9dc806aaf0 legacy=0x7f9dc80afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dbffff640 1 -- 172.21.15.31:0/2927186698 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80ad690 con 0x7f9dc80cda20 2023-11-26T07:21:49.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.918+0000 7f9dc659c640 1 --2- 172.21.15.31:0/2927186698 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc815b4d0 0x7f9dc80cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dbffff640 1 --2- 172.21.15.31:0/2927186698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc80cda20 0x7f9dc806a5b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9db00097c0 tx=0x7f9db001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82ea25ed29198f56 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dc5d9b640 1 -- 172.21.15.31:0/2927186698 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00ab650 con 0x7f9dc80cda20 2023-11-26T07:21:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dc5d9b640 1 -- 172.21.15.31:0/2927186698 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db00ab7b0 con 0x7f9dc80cda20 2023-11-26T07:21:49.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dc5d9b640 1 -- 172.21.15.31:0/2927186698 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00abb40 con 0x7f9dc80cda20 2023-11-26T07:21:49.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dcca16640 1 -- 172.21.15.31:0/2927186698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc80cda20 msgr2=0x7f9dc806a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dcca16640 1 --2- 172.21.15.31:0/2927186698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc80cda20 0x7f9dc806a5b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9db00097c0 tx=0x7f9db001c450 comp rx=0 tx=0).stop 2023-11-26T07:21:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dcca16640 1 -- 172.21.15.31:0/2927186698 shutdown_connections 2023-11-26T07:21:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dcca16640 1 --2- 172.21.15.31:0/2927186698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc80cda20 0x7f9dc806a5b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dcca16640 1 --2- 172.21.15.31:0/2927186698 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc815b4d0 0x7f9dc80cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dcca16640 1 -- 172.21.15.31:0/2927186698 >> 172.21.15.31:0/2927186698 conn(0x7f9dc80b5570 msgr2=0x7f9dc80b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dcca16640 1 -- 172.21.15.31:0/2927186698 shutdown_connections 2023-11-26T07:21:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dcca16640 1 -- 172.21.15.31:0/2927186698 wait complete. 2023-11-26T07:21:49.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dcca16640 1 Processor -- start 2023-11-26T07:21:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.922+0000 7f9dcca16640 1 -- start start 2023-11-26T07:21:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dcca16640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc806aaf0 0x7f9dc8143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dcca16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dc80cda20 0x7f9dc809a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dcca16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc815b4d0 0x7f9dc815a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dcca16640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80cf620 con 0x7f9dc80cda20 2023-11-26T07:21:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dcca16640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80cf900 con 0x7f9dc815b4d0 2023-11-26T07:21:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dcca16640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80cfbe0 con 0x7f9dc806aaf0 2023-11-26T07:21:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dc80cda20 0x7f9dc809a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dc80cda20 0x7f9dc809a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43066/0 (socket says 172.21.15.31:43066) 2023-11-26T07:21:49.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbffff640 1 -- 172.21.15.31:0/1719038035 learned_addr learned my addr 172.21.15.31:0/1719038035 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dc659c640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc806aaf0 0x7f9dc8143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dc6d9d640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc815b4d0 0x7f9dc815a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbffff640 1 -- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc806aaf0 msgr2=0x7f9dc8143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbffff640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc806aaf0 0x7f9dc8143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbffff640 1 -- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc815b4d0 msgr2=0x7f9dc815a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbffff640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc815b4d0 0x7f9dc815a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbffff640 1 -- 172.21.15.31:0/1719038035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db8009420 con 0x7f9dc80cda20 2023-11-26T07:21:49.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dc6d9d640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc815b4d0 0x7f9dc815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbffff640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dc80cda20 0x7f9dc809a1b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9db00ad040 tx=0x7f9db001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbeffd640 1 -- 172.21.15.31:0/1719038035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00b5810 con 0x7f9dc80cda20 2023-11-26T07:21:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbeffd640 1 -- 172.21.15.31:0/1719038035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db00b5990 con 0x7f9dc80cda20 2023-11-26T07:21:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dcca16640 1 -- 172.21.15.31:0/1719038035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db0009420 con 0x7f9dc80cda20 2023-11-26T07:21:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbeffd640 1 -- 172.21.15.31:0/1719038035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00b5d40 con 0x7f9dc80cda20 2023-11-26T07:21:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dcca16640 1 -- 172.21.15.31:0/1719038035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc80d01a0 con 0x7f9dc80cda20 2023-11-26T07:21:49.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.926+0000 7f9dbcff9640 1 -- 172.21.15.31:0/1719038035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da0005260 con 0x7f9dc80cda20 2023-11-26T07:21:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.930+0000 7f9dbeffd640 1 -- 172.21.15.31:0/1719038035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f9db00b4390 con 0x7f9dc80cda20 2023-11-26T07:21:49.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.930+0000 7f9dbeffd640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9da8066800 0x7f9da8068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.930+0000 7f9dbeffd640 1 -- 172.21.15.31:0/1719038035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9db011eff0 con 0x7f9dc80cda20 2023-11-26T07:21:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.930+0000 7f9dc659c640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9da8066800 0x7f9da8068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.930+0000 7f9dc659c640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9da8066800 0x7f9da8068cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009be0 tx=0x7f9db8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:49.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:49.934+0000 7f9dbeffd640 1 -- 172.21.15.31:0/1719038035 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9db00ec030 con 0x7f9dc80cda20 2023-11-26T07:21:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.090+0000 7f9dbcff9640 1 -- 172.21.15.31:0/1719038035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9da00057e0 con 0x7f9dc80cda20 2023-11-26T07:21:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.090+0000 7f9dbeffd640 1 -- 172.21.15.31:0/1719038035 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16497 (secure 0 0 0) 0x7f9db00ab610 con 0x7f9dc80cda20 2023-11-26T07:21:50.094 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:50.094 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":67,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:48.765454+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:46.570507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:47.773615+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 -- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9da8066800 msgr2=0x7f9da8068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9da8066800 0x7f9da8068cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009be0 tx=0x7f9db8008040 comp rx=0 tx=0).stop 2023-11-26T07:21:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 -- 172.21.15.31:0/1719038035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dc80cda20 msgr2=0x7f9dc809a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dc80cda20 0x7f9dc809a1b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9db00ad040 tx=0x7f9db001ac90 comp rx=0 tx=0).stop 2023-11-26T07:21:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 -- 172.21.15.31:0/1719038035 shutdown_connections 2023-11-26T07:21:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f9da8066800 0x7f9da8068cc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc815b4d0 0x7f9dc815a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dc80cda20 0x7f9dc809a1b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 --2- 172.21.15.31:0/1719038035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc806aaf0 0x7f9dc8143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 -- 172.21.15.31:0/1719038035 >> 172.21.15.31:0/1719038035 conn(0x7f9dc80b5570 msgr2=0x7f9dc81595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:50.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 -- 172.21.15.31:0/1719038035 shutdown_connections 2023-11-26T07:21:50.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.094+0000 7f9dbcff9640 1 -- 172.21.15.31:0/1719038035 wait complete. 2023-11-26T07:21:50.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:21:50.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.258+0000 7fcf2e89c640 1 Processor -- start 2023-11-26T07:21:50.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.258+0000 7fcf2e89c640 1 -- start start 2023-11-26T07:21:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf2e89c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf2815d8f0 0x7fcf2815e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf2e89c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2815ecc0 con 0x7fcf280d0510 2023-11-26T07:21:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf2e89c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf280cdfb0 con 0x7fcf280ced30 2023-11-26T07:21:50.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf2e89c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf280d4420 con 0x7fcf2815d8f0 2023-11-26T07:21:50.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf2ce39640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcf280d0510 0x7fcf280d1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45738/0 (socket says 172.21.15.31:45738) 2023-11-26T07:21:50.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf2ce39640 1 -- 172.21.15.31:0/3194782687 learned_addr learned my addr 172.21.15.31:0/3194782687 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf27fff640 1 --2- 172.21.15.31:0/3194782687 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf2815d8f0 0x7fcf2815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf26ffd640 1 -- 172.21.15.31:0/3194782687 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399812254 0 0) 0x7fcf2815ecc0 con 0x7fcf280d0510 2023-11-26T07:21:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf26ffd640 1 -- 172.21.15.31:0/3194782687 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf10003530 con 0x7fcf280d0510 2023-11-26T07:21:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf26ffd640 1 -- 172.21.15.31:0/3194782687 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121044974 0 0) 0x7fcf280cdfb0 con 0x7fcf280ced30 2023-11-26T07:21:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf26ffd640 1 -- 172.21.15.31:0/3194782687 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf2815ecc0 con 0x7fcf280ced30 2023-11-26T07:21:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf27fff640 1 -- 172.21.15.31:0/3194782687 >> v1:172.21.15.97:6789/0 conn(0x7fcf280ced30 legacy=0x7fcf280cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf27fff640 1 -- 172.21.15.31:0/3194782687 >> v1:172.21.15.31:6789/0 conn(0x7fcf280d0510 legacy=0x7fcf280d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.262+0000 7fcf27fff640 1 -- 172.21.15.31:0/3194782687 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf280d4140 con 0x7fcf2815d8f0 2023-11-26T07:21:50.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf27fff640 1 --2- 172.21.15.31:0/3194782687 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf2815d8f0 0x7fcf2815e780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcf180097f0 tx=0x7fcf1801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3e69e6fe3bf900a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:50.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf26ffd640 1 -- 172.21.15.31:0/3194782687 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf180ab650 con 0x7fcf2815d8f0 2023-11-26T07:21:50.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf26ffd640 1 -- 172.21.15.31:0/3194782687 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf180ab7b0 con 0x7fcf2815d8f0 2023-11-26T07:21:50.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf26ffd640 1 -- 172.21.15.31:0/3194782687 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf180abb40 con 0x7fcf2815d8f0 2023-11-26T07:21:50.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf2e89c640 1 -- 172.21.15.31:0/3194782687 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf2815d8f0 msgr2=0x7fcf2815e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf2e89c640 1 --2- 172.21.15.31:0/3194782687 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf2815d8f0 0x7fcf2815e780 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcf180097f0 tx=0x7fcf1801c450 comp rx=0 tx=0).stop 2023-11-26T07:21:50.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf2e89c640 1 -- 172.21.15.31:0/3194782687 shutdown_connections 2023-11-26T07:21:50.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf2e89c640 1 --2- 172.21.15.31:0/3194782687 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf2815d8f0 0x7fcf2815e780 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf2e89c640 1 -- 172.21.15.31:0/3194782687 >> 172.21.15.31:0/3194782687 conn(0x7fcf2809c750 msgr2=0x7fcf280d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:50.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf2e89c640 1 -- 172.21.15.31:0/3194782687 shutdown_connections 2023-11-26T07:21:50.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.266+0000 7fcf2e89c640 1 -- 172.21.15.31:0/3194782687 wait complete. 2023-11-26T07:21:50.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 Processor -- start 2023-11-26T07:21:50.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 -- start start 2023-11-26T07:21:50.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ced30 0x7fcf28158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280d0510 0x7fcf281579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2815d8f0 0x7fcf28157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf280b7900 con 0x7fcf280d0510 2023-11-26T07:21:50.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28145e50 con 0x7fcf2815d8f0 2023-11-26T07:21:50.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28143080 con 0x7fcf280ced30 2023-11-26T07:21:50.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280d0510 0x7fcf281579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf277fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280d0510 0x7fcf281579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43080/0 (socket says 172.21.15.31:43080) 2023-11-26T07:21:50.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf277fe640 1 -- 172.21.15.31:0/2190684715 learned_addr learned my addr 172.21.15.31:0/2190684715 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:50.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf27fff640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ced30 0x7fcf28158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2ce39640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2815d8f0 0x7fcf28157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf277fe640 1 -- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ced30 msgr2=0x7fcf28158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf277fe640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ced30 0x7fcf28158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf277fe640 1 -- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2815d8f0 msgr2=0x7fcf28157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:50.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf277fe640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2815d8f0 0x7fcf28157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:50.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf277fe640 1 -- 172.21.15.31:0/2190684715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf18009420 con 0x7fcf280d0510 2023-11-26T07:21:50.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf277fe640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280d0510 0x7fcf281579f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c520 tx=0x7fcf1c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:50.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf24ff9640 1 -- 172.21.15.31:0/2190684715 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c005480 con 0x7fcf280d0510 2023-11-26T07:21:50.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf24ff9640 1 -- 172.21.15.31:0/2190684715 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c003600 con 0x7fcf280d0510 2023-11-26T07:21:50.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 -- 172.21.15.31:0/2190684715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf28143360 con 0x7fcf280d0510 2023-11-26T07:21:50.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf24ff9640 1 -- 172.21.15.31:0/2190684715 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0039d0 con 0x7fcf280d0510 2023-11-26T07:21:50.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 -- 172.21.15.31:0/2190684715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf28143920 con 0x7fcf280d0510 2023-11-26T07:21:50.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.270+0000 7fcf2e89c640 1 -- 172.21.15.31:0/2190684715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf28143b40 con 0x7fcf280d0510 2023-11-26T07:21:50.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.274+0000 7fcf24ff9640 1 -- 172.21.15.31:0/2190684715 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fcf1c003b30 con 0x7fcf280d0510 2023-11-26T07:21:50.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.274+0000 7fcf24ff9640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcf140667d0 0x7fcf14068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:50.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.274+0000 7fcf24ff9640 1 -- 172.21.15.31:0/2190684715 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf1c0839b0 con 0x7fcf280d0510 2023-11-26T07:21:50.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.274+0000 7fcf27fff640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcf140667d0 0x7fcf14068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:50.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.274+0000 7fcf27fff640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcf140667d0 0x7fcf14068c90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcf180af000 tx=0x7fcf180b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:50.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.278+0000 7fcf24ff9640 1 -- 172.21.15.31:0/2190684715 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcf1c050aa0 con 0x7fcf280d0510 2023-11-26T07:21:50.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:50.474+0000 7fcf2e89c640 1 -- 172.21.15.31:0/2190684715 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcf2804e850 con 0x7fcf280d0510 2023-11-26T07:21:51.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf24ff9640 1 -- 172.21.15.31:0/2190684715 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcf1c017020 con 0x7fcf280d0510 2023-11-26T07:21:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 -- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcf140667d0 msgr2=0x7fcf14068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcf140667d0 0x7fcf14068c90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcf180af000 tx=0x7fcf180b6040 comp rx=0 tx=0).stop 2023-11-26T07:21:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 -- 172.21.15.31:0/2190684715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280d0510 msgr2=0x7fcf281579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280d0510 0x7fcf281579f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c520 tx=0x7fcf1c00c9e0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 -- 172.21.15.31:0/2190684715 shutdown_connections 2023-11-26T07:21:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcf140667d0 0x7fcf14068c90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2815d8f0 0x7fcf28157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280d0510 0x7fcf281579f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 --2- 172.21.15.31:0/2190684715 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ced30 0x7fcf28158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 -- 172.21.15.31:0/2190684715 >> 172.21.15.31:0/2190684715 conn(0x7fcf2809c750 msgr2=0x7fcf280c9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 -- 172.21.15.31:0/2190684715 shutdown_connections 2023-11-26T07:21:51.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.421+0000 7fcf2e89c640 1 -- 172.21.15.31:0/2190684715 wait complete. 2023-11-26T07:21:51.448 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:21:51.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:21:51.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.597+0000 7f3983279640 1 Processor -- start 2023-11-26T07:21:51.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.597+0000 7f3983279640 1 -- start start 2023-11-26T07:21:51.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.597+0000 7f3983279640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c06e3f0 0x7f397c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.597+0000 7f3983279640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c159630 con 0x7f397c0cb9b0 2023-11-26T07:21:51.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.597+0000 7f3983279640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397c15f6d0 con 0x7f397c0ce5d0 2023-11-26T07:21:51.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.597+0000 7f3983279640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c15bba0 con 0x7f397c06e3f0 2023-11-26T07:21:51.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3981015640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f397c0cb9b0 0x7f397c0cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45742/0 (socket says 172.21.15.31:45742) 2023-11-26T07:21:51.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3981015640 1 -- 172.21.15.31:0/1889080340 learned_addr learned my addr 172.21.15.31:0/1889080340 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:51.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336703746 0 0) 0x7f397c159630 con 0x7f397c0cb9b0 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3980814640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c06e3f0 0x7f397c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3968003530 con 0x7f397c0cb9b0 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762668863 0 0) 0x7f397c15f6d0 con 0x7f397c0ce5d0 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f397c159630 con 0x7f397c0ce5d0 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3584564615 0 0) 0x7f3968003530 con 0x7f397c0cb9b0 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c06e3f0 msgr2=0x7f397c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c06e3f0 0x7f397c15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 >> v1:172.21.15.97:6789/0 conn(0x7f397c0ce5d0 legacy=0x7f397c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0cd1d0 con 0x7f397c0cb9b0 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f3978001680 con 0x7f397c0cb9b0 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 >> v1:172.21.15.31:6789/0 conn(0x7f397c0cb9b0 legacy=0x7f397c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3968003b40 0x7f3968003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39680047d0 0x7f3968004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c06e3f0 0x7f3968005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396800ba40 con 0x7f39680047d0 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396800bb80 con 0x7f3968003b40 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396800bd40 con 0x7f397c06e3f0 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3980814640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39680047d0 0x7f3968004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3981015640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3968003b40 0x7f3968003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3981816640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c06e3f0 0x7f3968005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3980814640 1 -- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c06e3f0 msgr2=0x7f3968005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3980814640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c06e3f0 0x7f3968005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3980814640 1 -- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3968003b40 msgr2=0x7f3968003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:51.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3980814640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3968003b40 0x7f3968003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.601+0000 7f3980814640 1 -- 172.21.15.31:0/1889080340 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3968003530 con 0x7f39680047d0 2023-11-26T07:21:51.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3980814640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39680047d0 0x7f3968004c50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f396c009ae0 tx=0x7f396c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e155621953e4df7f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:51.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0a5640 con 0x7f39680047d0 2023-11-26T07:21:51.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396c0a57a0 con 0x7f39680047d0 2023-11-26T07:21:51.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3973fff640 1 -- 172.21.15.31:0/1889080340 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0a5b50 con 0x7f39680047d0 2023-11-26T07:21:51.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3983279640 1 -- 172.21.15.31:0/1889080340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39680047d0 msgr2=0x7f3968004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3983279640 1 --2- 172.21.15.31:0/1889080340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39680047d0 0x7f3968004c50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f396c009ae0 tx=0x7f396c01ca50 comp rx=0 tx=0).stop 2023-11-26T07:21:51.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3981816640 1 -- 172.21.15.31:0/1889080340 reap_dead start 2023-11-26T07:21:51.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3983279640 1 -- 172.21.15.31:0/1889080340 shutdown_connections 2023-11-26T07:21:51.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3983279640 1 -- 172.21.15.31:0/1889080340 >> 172.21.15.31:0/1889080340 conn(0x7f397c0b0cb0 msgr2=0x7f397c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:51.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3983279640 1 -- 172.21.15.31:0/1889080340 shutdown_connections 2023-11-26T07:21:51.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3983279640 1 -- 172.21.15.31:0/1889080340 wait complete. 2023-11-26T07:21:51.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3983279640 1 Processor -- start 2023-11-26T07:21:51.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.605+0000 7f3983279640 1 -- start start 2023-11-26T07:21:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3983279640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39680047d0 0x7f397c158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3983279640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c1f0df0 0x7f397c153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3983279640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f397c155b80 0x7f397c156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3983279640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c0abf90 con 0x7f397c155b80 2023-11-26T07:21:51.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3983279640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397c158a90 con 0x7f39680047d0 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3983279640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397c1f13c0 con 0x7f397c1f0df0 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981816640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f397c155b80 0x7f397c156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981816640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f397c155b80 0x7f397c156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43102/0 (socket says 172.21.15.31:43102) 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981816640 1 -- 172.21.15.31:0/959479102 learned_addr learned my addr 172.21.15.31:0/959479102 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981015640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39680047d0 0x7f397c158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3980814640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c1f0df0 0x7f397c153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981816640 1 -- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c1f0df0 msgr2=0x7f397c153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981816640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c1f0df0 0x7f397c153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981816640 1 -- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39680047d0 msgr2=0x7f397c158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981816640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39680047d0 0x7f397c158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:51.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981816640 1 -- 172.21.15.31:0/959479102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c1f16a0 con 0x7f397c155b80 2023-11-26T07:21:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981015640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39680047d0 0x7f397c158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3981816640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f397c155b80 0x7f397c156000 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f397c1602b0 tx=0x7f397400f170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3971ffb640 1 -- 172.21.15.31:0/959479102 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397400fd50 con 0x7f397c155b80 2023-11-26T07:21:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3971ffb640 1 -- 172.21.15.31:0/959479102 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397400fed0 con 0x7f397c155b80 2023-11-26T07:21:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3971ffb640 1 -- 172.21.15.31:0/959479102 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974015650 con 0x7f397c155b80 2023-11-26T07:21:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3983279640 1 -- 172.21.15.31:0/959479102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397c1f1980 con 0x7f397c155b80 2023-11-26T07:21:51.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3983279640 1 -- 172.21.15.31:0/959479102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397c156540 con 0x7f397c155b80 2023-11-26T07:21:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3983279640 1 -- 172.21.15.31:0/959479102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f397c0671f0 con 0x7f397c155b80 2023-11-26T07:21:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.609+0000 7f3971ffb640 1 -- 172.21.15.31:0/959479102 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f39740157b0 con 0x7f397c155b80 2023-11-26T07:21:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.613+0000 7f3971ffb640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3960066800 0x7f3960068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.613+0000 7f3971ffb640 1 -- 172.21.15.31:0/959479102 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f39740877d0 con 0x7f397c155b80 2023-11-26T07:21:51.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.613+0000 7f3981015640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3960066800 0x7f3960068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:51.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.613+0000 7f3981015640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3960066800 0x7f3960068cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3978001ec0 tx=0x7f3978009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:51.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.617+0000 7f3971ffb640 1 -- 172.21.15.31:0/959479102 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f39740582d0 con 0x7f397c155b80 2023-11-26T07:21:51.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:51.809+0000 7f3983279640 1 -- 172.21.15.31:0/959479102 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f397c06e000 con 0x7f397c155b80 2023-11-26T07:21:52.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.425+0000 7f3971ffb640 1 -- 172.21.15.31:0/959479102 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3974057e60 con 0x7f397c155b80 2023-11-26T07:21:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 -- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3960066800 msgr2=0x7f3960068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3960066800 0x7f3960068cc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3978001ec0 tx=0x7f3978009040 comp rx=0 tx=0).stop 2023-11-26T07:21:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 -- 172.21.15.31:0/959479102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f397c155b80 msgr2=0x7f397c156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f397c155b80 0x7f397c156000 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f397c1602b0 tx=0x7f397400f170 comp rx=0 tx=0).stop 2023-11-26T07:21:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 -- 172.21.15.31:0/959479102 shutdown_connections 2023-11-26T07:21:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3960066800 0x7f3960068cc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f397c155b80 0x7f397c156000 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f397c1f0df0 0x7f397c153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 --2- 172.21.15.31:0/959479102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39680047d0 0x7f397c158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 -- 172.21.15.31:0/959479102 >> 172.21.15.31:0/959479102 conn(0x7f397c0b0cb0 msgr2=0x7f397c15f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 -- 172.21.15.31:0/959479102 shutdown_connections 2023-11-26T07:21:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.429+0000 7f3983279640 1 -- 172.21.15.31:0/959479102 wait complete. 2023-11-26T07:21:52.454 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:21:52.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:52.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.597+0000 7f58f95d1640 1 Processor -- start 2023-11-26T07:21:52.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.597+0000 7f58f95d1640 1 -- start start 2023-11-26T07:21:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.597+0000 7f58f95d1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58f40cb230 0x7f58f4070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.597+0000 7f58f95d1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f4071880 con 0x7f58f4071430 2023-11-26T07:21:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.597+0000 7f58f95d1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f40b41a0 con 0x7f58f4159880 2023-11-26T07:21:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.597+0000 7f58f95d1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f40b3be0 con 0x7f58f40cb230 2023-11-26T07:21:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.597+0000 7f58f37fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f58f4071430 0x7f58f406d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45744/0 (socket says 172.21.15.31:45744) 2023-11-26T07:21:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.597+0000 7f58f37fe640 1 -- 172.21.15.31:0/2071982630 learned_addr learned my addr 172.21.15.31:0/2071982630 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:52.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.597+0000 7f58ebfff640 1 --2- 172.21.15.31:0/2071982630 >> v2:172.21.15.97:3300/0 conn(0x7f58f40cb230 0x7f58f4070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.601+0000 7f58f27fc640 1 -- 172.21.15.31:0/2071982630 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428336064 0 0) 0x7f58ec0014a0 con 0x7f58f4071430 2023-11-26T07:21:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.601+0000 7f58f27fc640 1 -- 172.21.15.31:0/2071982630 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e0003530 con 0x7f58f4071430 2023-11-26T07:21:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.601+0000 7f58ebfff640 1 -- 172.21.15.31:0/2071982630 >> v1:172.21.15.97:6789/0 conn(0x7f58f4159880 legacy=0x7f58f40b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.601+0000 7f58ebfff640 1 -- 172.21.15.31:0/2071982630 >> v1:172.21.15.31:6789/0 conn(0x7f58f4071430 legacy=0x7f58f406d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.601+0000 7f58f27fc640 1 -- 172.21.15.31:0/2071982630 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2425541756 0 0) 0x7f58e0003530 con 0x7f58f4071430 2023-11-26T07:21:52.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.601+0000 7f58ebfff640 1 -- 172.21.15.31:0/2071982630 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f40b3ec0 con 0x7f58f40cb230 2023-11-26T07:21:52.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.601+0000 7f58ebfff640 1 --2- 172.21.15.31:0/2071982630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f40cb230 0x7f58f4070ef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0097c0 tx=0x7f58dc01c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4690aa3f6cf103d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.601+0000 7f58f27fc640 1 -- 172.21.15.31:0/2071982630 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec001b90 con 0x7f58f40cb230 2023-11-26T07:21:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.601+0000 7f58f27fc640 1 -- 172.21.15.31:0/2071982630 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc0a7050 con 0x7f58f40cb230 2023-11-26T07:21:52.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.601+0000 7f58f27fc640 1 -- 172.21.15.31:0/2071982630 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0ab620 con 0x7f58f40cb230 2023-11-26T07:21:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 -- 172.21.15.31:0/2071982630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f40cb230 msgr2=0x7f58f4070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 --2- 172.21.15.31:0/2071982630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f40cb230 0x7f58f4070ef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0097c0 tx=0x7f58dc01c310 comp rx=0 tx=0).stop 2023-11-26T07:21:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 -- 172.21.15.31:0/2071982630 shutdown_connections 2023-11-26T07:21:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 --2- 172.21.15.31:0/2071982630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f40cb230 0x7f58f4070ef0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 -- 172.21.15.31:0/2071982630 >> 172.21.15.31:0/2071982630 conn(0x7f58f40b0c40 msgr2=0x7f58f40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 -- 172.21.15.31:0/2071982630 shutdown_connections 2023-11-26T07:21:52.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 -- 172.21.15.31:0/2071982630 wait complete. 2023-11-26T07:21:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 Processor -- start 2023-11-26T07:21:52.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 -- start start 2023-11-26T07:21:52.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58f4071430 0x7f58f414b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58f40cb230 0x7f58f40ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f4159880 0x7f58f40cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40cbbd0 con 0x7f58f40cb230 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40cbeb0 con 0x7f58f4159880 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f95d1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40cc190 con 0x7f58f4071430 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58ebfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58f40cb230 0x7f58f40ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58ebfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58f40cb230 0x7f58f40ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43114/0 (socket says 172.21.15.31:43114) 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58ebfff640 1 -- 172.21.15.31:0/3338257656 learned_addr learned my addr 172.21.15.31:0/3338257656 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f2ffd640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58f4071430 0x7f58f414b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58f37fe640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f4159880 0x7f58f40cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58ebfff640 1 -- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58f4071430 msgr2=0x7f58f414b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58ebfff640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58f4071430 0x7f58f414b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58ebfff640 1 -- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f4159880 msgr2=0x7f58f40cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58ebfff640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f4159880 0x7f58f40cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.605+0000 7f58ebfff640 1 -- 172.21.15.31:0/3338257656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc009420 con 0x7f58f40cb230 2023-11-26T07:21:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58ebfff640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58f40cb230 0x7f58f40ccc70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0ae040 tx=0x7f58dc01ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58eb7fe640 1 -- 172.21.15.31:0/3338257656 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc0abda0 con 0x7f58f40cb230 2023-11-26T07:21:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58eb7fe640 1 -- 172.21.15.31:0/3338257656 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc004020 con 0x7f58f40cb230 2023-11-26T07:21:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58f95d1640 1 -- 172.21.15.31:0/3338257656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f40cc470 con 0x7f58f40cb230 2023-11-26T07:21:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58eb7fe640 1 -- 172.21.15.31:0/3338257656 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc004390 con 0x7f58f40cb230 2023-11-26T07:21:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58f95d1640 1 -- 172.21.15.31:0/3338257656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f40cb880 con 0x7f58f40cb230 2023-11-26T07:21:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58f95d1640 1 -- 172.21.15.31:0/3338257656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f404e850 con 0x7f58f40cb230 2023-11-26T07:21:52.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58eb7fe640 1 -- 172.21.15.31:0/3338257656 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f58dc003620 con 0x7f58f40cb230 2023-11-26T07:21:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58eb7fe640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f58d40667d0 0x7f58d4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58eb7fe640 1 -- 172.21.15.31:0/3338257656 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58dc11ea40 con 0x7f58f40cb230 2023-11-26T07:21:52.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.609+0000 7f58f2ffd640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f58d40667d0 0x7f58d4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.613+0000 7f58f2ffd640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f58d40667d0 0x7f58d4068c90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f58e4000f70 tx=0x7f58e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.613+0000 7f58eb7fe640 1 -- 172.21.15.31:0/3338257656 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f58dc0ebac0 con 0x7f58f40cb230 2023-11-26T07:21:52.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 -- 172.21.15.31:0/3338257656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58f40cb010 con 0x7f58f40cb230 2023-11-26T07:21:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58eb7fe640 1 -- 172.21.15.31:0/3338257656 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f58dc0ef0e0 con 0x7f58f40cb230 2023-11-26T07:21:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:52.815 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2851182363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2851182363},{"type":"v1","addr":"172.21.15.31:6835","nonce":2851182363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2056572548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2056572548},{"type":"v1","addr":"172.21.15.97:6837","nonce":2056572548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1012539369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1012539369},{"type":"v1","addr":"172.21.15.97:6836","nonce":1012539369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:48.765793+0000","modified":"2023-11-26T07:21:51.814230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6597},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6597":{"gid":6597,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/242493352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":242493352},{"type":"v1","addr":"172.21.15.31:6834","nonce":242493352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:21:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 47 2023-11-26T07:21:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 -- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f58d40667d0 msgr2=0x7f58d4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f58d40667d0 0x7f58d4068c90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f58e4000f70 tx=0x7f58e4009040 comp rx=0 tx=0).stop 2023-11-26T07:21:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 -- 172.21.15.31:0/3338257656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58f40cb230 msgr2=0x7f58f40ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58f40cb230 0x7f58f40ccc70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0ae040 tx=0x7f58dc01ab50 comp rx=0 tx=0).stop 2023-11-26T07:21:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 -- 172.21.15.31:0/3338257656 shutdown_connections 2023-11-26T07:21:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f58d40667d0 0x7f58d4068c90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f4159880 0x7f58f40cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58f40cb230 0x7f58f40ccc70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 --2- 172.21.15.31:0/3338257656 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58f4071430 0x7f58f414b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 -- 172.21.15.31:0/3338257656 >> 172.21.15.31:0/3338257656 conn(0x7f58f40b0c40 msgr2=0x7f58f414fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 -- 172.21.15.31:0/3338257656 shutdown_connections 2023-11-26T07:21:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.809+0000 7f58f95d1640 1 -- 172.21.15.31:0/3338257656 wait complete. 2023-11-26T07:21:52.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:52.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.973+0000 7ff102e16640 1 Processor -- start 2023-11-26T07:21:52.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.973+0000 7ff102e16640 1 -- start start 2023-11-26T07:21:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff102e16640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0fc071cc0 0x7ff0fc0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff102e16640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff0fc0b2ee0 0x7ff0fc0adee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff102e16640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0fc0ad740 con 0x7ff0fc06e410 2023-11-26T07:21:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff102e16640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0fc0b3e20 con 0x7ff0fc071cc0 2023-11-26T07:21:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff102e16640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0fc15dde0 con 0x7ff0fc0b2ee0 2023-11-26T07:21:52.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff0fbfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0fc071cc0 0x7ff0fc0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff0fbfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0fc071cc0 0x7ff0fc0b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43126/0 (socket says 172.21.15.31:43126) 2023-11-26T07:21:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff0fbfff640 1 -- 172.21.15.31:0/1514780313 learned_addr learned my addr 172.21.15.31:0/1514780313 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff1013b3640 1 --2- 172.21.15.31:0/1514780313 >> v2:172.21.15.97:3300/0 conn(0x7ff0fc0b2ee0 0x7ff0fc0adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff0fbfff640 1 -- 172.21.15.31:0/1514780313 >> v2:172.21.15.97:3300/0 conn(0x7ff0fc0b2ee0 msgr2=0x7ff0fc0adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff0fbfff640 1 --2- 172.21.15.31:0/1514780313 >> v2:172.21.15.97:3300/0 conn(0x7ff0fc0b2ee0 0x7ff0fc0adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff0fbfff640 1 -- 172.21.15.31:0/1514780313 >> v1:172.21.15.97:6789/0 conn(0x7ff0fc06e410 legacy=0x7ff0fc0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff0fbfff640 1 -- 172.21.15.31:0/1514780313 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc0b3580 con 0x7ff0fc071cc0 2023-11-26T07:21:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.977+0000 7ff1013b3640 1 --2- 172.21.15.31:0/1514780313 >> v2:172.21.15.97:3300/0 conn(0x7ff0fc0b2ee0 0x7ff0fc0adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:52.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff0fbfff640 1 --2- 172.21.15.31:0/1514780313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 0x7ff0fc0b29a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0af5e0 tx=0x7ff0e801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b087771fd9209184 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff0fb7fe640 1 -- 172.21.15.31:0/1514780313 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e80ac410 con 0x7ff0fc071cc0 2023-11-26T07:21:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff0fb7fe640 1 -- 172.21.15.31:0/1514780313 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e80ac570 con 0x7ff0fc071cc0 2023-11-26T07:21:52.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff0fb7fe640 1 -- 172.21.15.31:0/1514780313 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e80ac900 con 0x7ff0fc071cc0 2023-11-26T07:21:52.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff102e16640 1 -- 172.21.15.31:0/1514780313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 msgr2=0x7ff0fc0b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff102e16640 1 --2- 172.21.15.31:0/1514780313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 0x7ff0fc0b29a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0af5e0 tx=0x7ff0e801c4c0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff102e16640 1 -- 172.21.15.31:0/1514780313 shutdown_connections 2023-11-26T07:21:52.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff102e16640 1 --2- 172.21.15.31:0/1514780313 >> v2:172.21.15.97:3300/0 conn(0x7ff0fc0b2ee0 0x7ff0fc0adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff102e16640 1 --2- 172.21.15.31:0/1514780313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 0x7ff0fc0b29a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff102e16640 1 -- 172.21.15.31:0/1514780313 >> 172.21.15.31:0/1514780313 conn(0x7ff0fc0b1e40 msgr2=0x7ff0fc070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:52.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff102e16640 1 -- 172.21.15.31:0/1514780313 shutdown_connections 2023-11-26T07:21:52.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff102e16640 1 -- 172.21.15.31:0/1514780313 wait complete. 2023-11-26T07:21:52.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff102e16640 1 Processor -- start 2023-11-26T07:21:52.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.981+0000 7ff102e16640 1 -- start start 2023-11-26T07:21:52.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff102e16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc06e410 0x7ff0fc148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff102e16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 0x7ff0fc15f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff102e16640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0fc0b2ee0 0x7ff0fc0cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff102e16640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc148a70 con 0x7ff0fc071cc0 2023-11-26T07:21:52.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff102e16640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc0ccf60 con 0x7ff0fc06e410 2023-11-26T07:21:52.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff102e16640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc0cd240 con 0x7ff0fc0b2ee0 2023-11-26T07:21:52.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 0x7ff0fc15f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 0x7ff0fc15f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43142/0 (socket says 172.21.15.31:43142) 2023-11-26T07:21:52.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0fbfff640 1 -- 172.21.15.31:0/1510653723 learned_addr learned my addr 172.21.15.31:0/1510653723 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:52.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0fbfff640 1 -- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0fc0b2ee0 msgr2=0x7ff0fc0cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0fbfff640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0fc0b2ee0 0x7ff0fc0cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff100bb2640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc06e410 0x7ff0fc148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0fbfff640 1 -- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc06e410 msgr2=0x7ff0fc148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:52.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0fbfff640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc06e410 0x7ff0fc148170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:52.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0fbfff640 1 -- 172.21.15.31:0/1510653723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e80094d0 con 0x7ff0fc071cc0 2023-11-26T07:21:52.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff100bb2640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc06e410 0x7ff0fc148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:21:52.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0fbfff640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 0x7ff0fc15f070 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80af040 tx=0x7ff0e8019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0f97fa640 1 -- 172.21.15.31:0/1510653723 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e80acea0 con 0x7ff0fc071cc0 2023-11-26T07:21:52.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0f97fa640 1 -- 172.21.15.31:0/1510653723 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e80b5630 con 0x7ff0fc071cc0 2023-11-26T07:21:52.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff0f97fa640 1 -- 172.21.15.31:0/1510653723 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e80ac410 con 0x7ff0fc071cc0 2023-11-26T07:21:52.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff102e16640 1 -- 172.21.15.31:0/1510653723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc0cd520 con 0x7ff0fc071cc0 2023-11-26T07:21:52.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff102e16640 1 -- 172.21.15.31:0/1510653723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc1486b0 con 0x7ff0fc071cc0 2023-11-26T07:21:52.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.985+0000 7ff102e16640 1 -- 172.21.15.31:0/1510653723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc143690 con 0x7ff0fc071cc0 2023-11-26T07:21:52.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.989+0000 7ff0f97fa640 1 -- 172.21.15.31:0/1510653723 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7ff0e80b3070 con 0x7ff0fc071cc0 2023-11-26T07:21:52.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.989+0000 7ff0f97fa640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff0e0066800 0x7ff0e0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:52.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.989+0000 7ff0f97fa640 1 -- 172.21.15.31:0/1510653723 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0e811eba0 con 0x7ff0fc071cc0 2023-11-26T07:21:52.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.989+0000 7ff100bb2640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff0e0066800 0x7ff0e0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:52.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.989+0000 7ff100bb2640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff0e0066800 0x7ff0e0068cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc15f000 tx=0x7ff0f0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:52.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:52.993+0000 7ff0f97fa640 1 -- 172.21.15.31:0/1510653723 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0e80ebb50 con 0x7ff0fc071cc0 2023-11-26T07:21:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.149+0000 7ff102e16640 1 -- 172.21.15.31:0/1510653723 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0fc04e850 con 0x7ff0fc071cc0 2023-11-26T07:21:53.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff0f97fa640 1 -- 172.21.15.31:0/1510653723 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16484 (secure 0 0 0) 0x7ff0e80b1090 con 0x7ff0fc071cc0 2023-11-26T07:21:53.156 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:53.156 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":68,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:50.369840+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:46.570507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:47.773615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 -- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff0e0066800 msgr2=0x7ff0e0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff0e0066800 0x7ff0e0068cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc15f000 tx=0x7ff0f0009040 comp rx=0 tx=0).stop 2023-11-26T07:21:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 -- 172.21.15.31:0/1510653723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 msgr2=0x7ff0fc15f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 0x7ff0fc15f070 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff0e80af040 tx=0x7ff0e8019260 comp rx=0 tx=0).stop 2023-11-26T07:21:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 -- 172.21.15.31:0/1510653723 shutdown_connections 2023-11-26T07:21:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff0e0066800 0x7ff0e0068cc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0fc0b2ee0 0x7ff0fc0cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0fc071cc0 0x7ff0fc15f070 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 --2- 172.21.15.31:0/1510653723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0fc06e410 0x7ff0fc148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 -- 172.21.15.31:0/1510653723 >> 172.21.15.31:0/1510653723 conn(0x7ff0fc0b1e40 msgr2=0x7ff0fc0b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 -- 172.21.15.31:0/1510653723 shutdown_connections 2023-11-26T07:21:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.153+0000 7ff102e16640 1 -- 172.21.15.31:0/1510653723 wait complete. 2023-11-26T07:21:53.178 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:21:53.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:21:53.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fc38d640 1 Processor -- start 2023-11-26T07:21:53.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fc38d640 1 -- start start 2023-11-26T07:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fc38d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8f4144c80 0x7fc8f40abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fc38d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc8f40ac460 0x7fc8f40706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fc38d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8f4070c20 0x7fc8f415eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fc38d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f4145060 con 0x7fc8f4144c80 2023-11-26T07:21:53.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fc38d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f4071fd0 con 0x7fc8f40ac460 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fc38d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40ac8c0 con 0x7fc8f4070c20 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fa129640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8f4144c80 0x7fc8f40abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fa129640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc8f4144c80 0x7fc8f40abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43156/0 (socket says 172.21.15.31:43156) 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fa129640 1 -- 172.21.15.31:0/3889641175 learned_addr learned my addr 172.21.15.31:0/3889641175 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8f9928640 1 --2- 172.21.15.31:0/3889641175 >> v2:172.21.15.97:3300/0 conn(0x7fc8f40ac460 0x7fc8f40706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fa92a640 1 --2- 172.21.15.31:0/3889641175 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8f4070c20 0x7fc8f415eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fa129640 1 -- 172.21.15.31:0/3889641175 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8f4070c20 msgr2=0x7fc8f415eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fa129640 1 --2- 172.21.15.31:0/3889641175 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8f4070c20 0x7fc8f415eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fa129640 1 -- 172.21.15.31:0/3889641175 >> v2:172.21.15.97:3300/0 conn(0x7fc8f40ac460 msgr2=0x7fc8f40706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fa129640 1 --2- 172.21.15.31:0/3889641175 >> v2:172.21.15.97:3300/0 conn(0x7fc8f40ac460 0x7fc8f40706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.321+0000 7fc8fa129640 1 -- 172.21.15.31:0/3889641175 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f4071450 con 0x7fc8f4144c80 2023-11-26T07:21:53.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8fa129640 1 --2- 172.21.15.31:0/3889641175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 0x7fc8f40abf20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4072d10 tx=0x7fc8f001c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72485c28104f10 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:53.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8f9127640 1 -- 172.21.15.31:0/3889641175 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00ab600 con 0x7fc8f4144c80 2023-11-26T07:21:53.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8f9127640 1 -- 172.21.15.31:0/3889641175 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f00a7050 con 0x7fc8f4144c80 2023-11-26T07:21:53.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8f9127640 1 -- 172.21.15.31:0/3889641175 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00ab980 con 0x7fc8f4144c80 2023-11-26T07:21:53.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8fc38d640 1 -- 172.21.15.31:0/3889641175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 msgr2=0x7fc8f40abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8fc38d640 1 --2- 172.21.15.31:0/3889641175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 0x7fc8f40abf20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4072d10 tx=0x7fc8f001c210 comp rx=0 tx=0).stop 2023-11-26T07:21:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8fc38d640 1 -- 172.21.15.31:0/3889641175 shutdown_connections 2023-11-26T07:21:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8fc38d640 1 --2- 172.21.15.31:0/3889641175 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8f4070c20 0x7fc8f415eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8fc38d640 1 --2- 172.21.15.31:0/3889641175 >> v2:172.21.15.97:3300/0 conn(0x7fc8f40ac460 0x7fc8f40706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8fc38d640 1 --2- 172.21.15.31:0/3889641175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 0x7fc8f40abf20 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8fc38d640 1 -- 172.21.15.31:0/3889641175 >> 172.21.15.31:0/3889641175 conn(0x7fc8f40b0c40 msgr2=0x7fc8f40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.325+0000 7fc8fc38d640 1 -- 172.21.15.31:0/3889641175 shutdown_connections 2023-11-26T07:21:53.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fc38d640 1 -- 172.21.15.31:0/3889641175 wait complete. 2023-11-26T07:21:53.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fc38d640 1 Processor -- start 2023-11-26T07:21:53.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fc38d640 1 -- start start 2023-11-26T07:21:53.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fc38d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f4070c20 0x7fc8f415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fc38d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8f40ac460 0x7fc8f413a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fc38d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 0x7fc8f413ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fc38d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f4099a30 con 0x7fc8f4144c80 2023-11-26T07:21:53.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fc38d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f4099ff0 con 0x7fc8f4070c20 2023-11-26T07:21:53.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fc38d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f415a5c0 con 0x7fc8f40ac460 2023-11-26T07:21:53.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fa92a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 0x7fc8f413ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fa92a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 0x7fc8f413ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43162/0 (socket says 172.21.15.31:43162) 2023-11-26T07:21:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fa92a640 1 -- 172.21.15.31:0/459776347 learned_addr learned my addr 172.21.15.31:0/459776347 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fa129640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f4070c20 0x7fc8f415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8f9928640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8f40ac460 0x7fc8f413a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fa92a640 1 -- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8f40ac460 msgr2=0x7fc8f413a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fa92a640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8f40ac460 0x7fc8f413a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fa92a640 1 -- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f4070c20 msgr2=0x7fc8f415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fa92a640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f4070c20 0x7fc8f415a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fa92a640 1 -- 172.21.15.31:0/459776347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f00094d0 con 0x7fc8f4144c80 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.329+0000 7fc8fa92a640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 0x7fc8f413ad60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec007e60 tx=0x7fc8ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.333+0000 7fc8eaffd640 1 -- 172.21.15.31:0/459776347 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec014620 con 0x7fc8f4144c80 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.333+0000 7fc8eaffd640 1 -- 172.21.15.31:0/459776347 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ec00ee90 con 0x7fc8f4144c80 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.333+0000 7fc8eaffd640 1 -- 172.21.15.31:0/459776347 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec0149e0 con 0x7fc8f4144c80 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.333+0000 7fc8fc38d640 1 -- 172.21.15.31:0/459776347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f415e670 con 0x7fc8f4144c80 2023-11-26T07:21:53.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.333+0000 7fc8fc38d640 1 -- 172.21.15.31:0/459776347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f4099c80 con 0x7fc8f4144c80 2023-11-26T07:21:53.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.333+0000 7fc8fc38d640 1 -- 172.21.15.31:0/459776347 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f40b1ba0 con 0x7fc8f4144c80 2023-11-26T07:21:53.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.333+0000 7fc8eaffd640 1 -- 172.21.15.31:0/459776347 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fc8ec014b40 con 0x7fc8f4144c80 2023-11-26T07:21:53.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.333+0000 7fc8eaffd640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8d8066800 0x7fc8d8068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.333+0000 7fc8eaffd640 1 -- 172.21.15.31:0/459776347 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8ec084b80 con 0x7fc8f4144c80 2023-11-26T07:21:53.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.333+0000 7fc8fa129640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8d8066800 0x7fc8d8068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.337+0000 7fc8fa129640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8d8066800 0x7fc8d8068cc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00ad000 tx=0x7fc8f00a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:53.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.337+0000 7fc8eaffd640 1 -- 172.21.15.31:0/459776347 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc8ec051ca0 con 0x7fc8f4144c80 2023-11-26T07:21:53.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.497+0000 7fc8fc38d640 1 -- 172.21.15.31:0/459776347 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8f406c910 con 0x7fc8d8066800 2023-11-26T07:21:53.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.505+0000 7fc8eaffd640 1 -- 172.21.15.31:0/459776347 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288737 (secure 0 0 0) 0x7fc8f00ab5a0 con 0x7fc8d8066800 2023-11-26T07:21:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:53.513 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:21:53.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 -- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8d8066800 msgr2=0x7fc8d8068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8d8066800 0x7fc8d8068cc0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00ad000 tx=0x7fc8f00a6000 comp rx=0 tx=0).stop 2023-11-26T07:21:53.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 -- 172.21.15.31:0/459776347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 msgr2=0x7fc8f413ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 0x7fc8f413ad60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec007e60 tx=0x7fc8ec00e4b0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 -- 172.21.15.31:0/459776347 shutdown_connections 2023-11-26T07:21:53.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fc8d8066800 0x7fc8d8068cc0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc8f4144c80 0x7fc8f413ad60 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc8f40ac460 0x7fc8f413a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 --2- 172.21.15.31:0/459776347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc8f4070c20 0x7fc8f415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 -- 172.21.15.31:0/459776347 >> 172.21.15.31:0/459776347 conn(0x7fc8f40b0c40 msgr2=0x7fc8f40af090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:53.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 -- 172.21.15.31:0/459776347 shutdown_connections 2023-11-26T07:21:53.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.509+0000 7fc8fc38d640 1 -- 172.21.15.31:0/459776347 wait complete. 2023-11-26T07:21:53.531 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2023-11-26T07:21:52.371614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257052,"kb_used_data":7588,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717668,"statfs":{"total":773094113280,"available":772830892032,"internally_reserved":0,"allocated":7770112,"data_stored":3919769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.004499"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403752+0000","last_change":"2023-11-26T07:21:49.396807+0000","last_active":"2023-11-26T07:21:49.403752+0000","last_peered":"2023-11-26T07:21:49.403752+0000","last_clean":"2023-11-26T07:21:49.403752+0000","last_became_active":"2023-11-26T07:21:49.396478+0000","last_became_peered":"2023-11-26T07:21:49.396478+0000","last_unstale":"2023-11-26T07:21:49.403752+0000","last_undegraded":"2023-11-26T07:21:49.403752+0000","last_fullsized":"2023-11-26T07:21:49.403752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:50.985519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.395252+0000","last_change":"2023-11-26T07:21:49.385627+0000","last_active":"2023-11-26T07:21:49.395252+0000","last_peered":"2023-11-26T07:21:49.395252+0000","last_clean":"2023-11-26T07:21:49.395252+0000","last_became_active":"2023-11-26T07:21:49.381419+0000","last_became_peered":"2023-11-26T07:21:49.381419+0000","last_unstale":"2023-11-26T07:21:49.395252+0000","last_undegraded":"2023-11-26T07:21:49.395252+0000","last_fullsized":"2023-11-26T07:21:49.395252+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:44.940845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.399209+0000","last_change":"2023-11-26T07:21:49.388149+0000","last_active":"2023-11-26T07:21:49.399209+0000","last_peered":"2023-11-26T07:21:49.399209+0000","last_clean":"2023-11-26T07:21:49.399209+0000","last_became_active":"2023-11-26T07:21:49.387843+0000","last_became_peered":"2023-11-26T07:21:49.387843+0000","last_unstale":"2023-11-26T07:21:49.399209+0000","last_undegraded":"2023-11-26T07:21:49.399209+0000","last_fullsized":"2023-11-26T07:21:49.399209+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:02.563169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400729+0000","last_change":"2023-11-26T07:21:49.396963+0000","last_active":"2023-11-26T07:21:49.400729+0000","last_peered":"2023-11-26T07:21:49.400729+0000","last_clean":"2023-11-26T07:21:49.400729+0000","last_became_active":"2023-11-26T07:21:49.396682+0000","last_became_peered":"2023-11-26T07:21:49.396682+0000","last_unstale":"2023-11-26T07:21:49.400729+0000","last_undegraded":"2023-11-26T07:21:49.400729+0000","last_fullsized":"2023-11-26T07:21:49.400729+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:53:24.881150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.401950+0000","last_change":"2023-11-26T07:21:49.397512+0000","last_active":"2023-11-26T07:21:49.401950+0000","last_peered":"2023-11-26T07:21:49.401950+0000","last_clean":"2023-11-26T07:21:49.401950+0000","last_became_active":"2023-11-26T07:21:49.397131+0000","last_became_peered":"2023-11-26T07:21:49.397131+0000","last_unstale":"2023-11-26T07:21:49.401950+0000","last_undegraded":"2023-11-26T07:21:49.401950+0000","last_fullsized":"2023-11-26T07:21:49.401950+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:28:48.891145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.384035+0000","last_change":"2023-11-26T07:21:49.382277+0000","last_active":"2023-11-26T07:21:49.384035+0000","last_peered":"2023-11-26T07:21:49.384035+0000","last_clean":"2023-11-26T07:21:49.384035+0000","last_became_active":"2023-11-26T07:21:49.381517+0000","last_became_peered":"2023-11-26T07:21:49.381517+0000","last_unstale":"2023-11-26T07:21:49.384035+0000","last_undegraded":"2023-11-26T07:21:49.384035+0000","last_fullsized":"2023-11-26T07:21:49.384035+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:51:43.081093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.404449+0000","last_change":"2023-11-26T07:21:49.400222+0000","last_active":"2023-11-26T07:21:49.404449+0000","last_peered":"2023-11-26T07:21:49.404449+0000","last_clean":"2023-11-26T07:21:49.404449+0000","last_became_active":"2023-11-26T07:21:49.399942+0000","last_became_peered":"2023-11-26T07:21:49.399942+0000","last_unstale":"2023-11-26T07:21:49.404449+0000","last_undegraded":"2023-11-26T07:21:49.404449+0000","last_fullsized":"2023-11-26T07:21:49.404449+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:41.096161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.397083+0000","last_change":"2023-11-26T07:21:49.389216+0000","last_active":"2023-11-26T07:21:49.397083+0000","last_peered":"2023-11-26T07:21:49.397083+0000","last_clean":"2023-11-26T07:21:49.397083+0000","last_became_active":"2023-11-26T07:21:49.388920+0000","last_became_peered":"2023-11-26T07:21:49.388920+0000","last_unstale":"2023-11-26T07:21:49.397083+0000","last_undegraded":"2023-11-26T07:21:49.397083+0000","last_fullsized":"2023-11-26T07:21:49.397083+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:21.895779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.402437+0000","last_change":"2023-11-26T07:21:49.397940+0000","last_active":"2023-11-26T07:21:49.402437+0000","last_peered":"2023-11-26T07:21:49.402437+0000","last_clean":"2023-11-26T07:21:49.402437+0000","last_became_active":"2023-11-26T07:21:49.397426+0000","last_became_peered":"2023-11-26T07:21:49.397426+0000","last_unstale":"2023-11-26T07:21:49.402437+0000","last_undegraded":"2023-11-26T07:21:49.402437+0000","last_fullsized":"2023-11-26T07:21:49.402437+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:10:22.554778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.392989+0000","last_change":"2023-11-26T07:21:49.387190+0000","last_active":"2023-11-26T07:21:49.392989+0000","last_peered":"2023-11-26T07:21:49.392989+0000","last_clean":"2023-11-26T07:21:49.392989+0000","last_became_active":"2023-11-26T07:21:49.386912+0000","last_became_peered":"2023-11-26T07:21:49.386912+0000","last_unstale":"2023-11-26T07:21:49.392989+0000","last_undegraded":"2023-11-26T07:21:49.392989+0000","last_fullsized":"2023-11-26T07:21:49.392989+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:40.570012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.411253+0000","last_change":"2023-11-26T07:21:49.403218+0000","last_active":"2023-11-26T07:21:49.411253+0000","last_peered":"2023-11-26T07:21:49.411253+0000","last_clean":"2023-11-26T07:21:49.411253+0000","last_became_active":"2023-11-26T07:21:49.402211+0000","last_became_peered":"2023-11-26T07:21:49.402211+0000","last_unstale":"2023-11-26T07:21:49.411253+0000","last_undegraded":"2023-11-26T07:21:49.411253+0000","last_fullsized":"2023-11-26T07:21:49.411253+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:33.115731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408081+0000","last_change":"2023-11-26T07:21:49.401167+0000","last_active":"2023-11-26T07:21:49.408081+0000","last_peered":"2023-11-26T07:21:49.408081+0000","last_clean":"2023-11-26T07:21:49.408081+0000","last_became_active":"2023-11-26T07:21:49.399604+0000","last_became_peered":"2023-11-26T07:21:49.399604+0000","last_unstale":"2023-11-26T07:21:49.408081+0000","last_undegraded":"2023-11-26T07:21:49.408081+0000","last_fullsized":"2023-11-26T07:21:49.408081+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:47.172012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.398331+0000","last_change":"2023-11-26T07:21:49.392234+0000","last_active":"2023-11-26T07:21:49.398331+0000","last_peered":"2023-11-26T07:21:49.398331+0000","last_clean":"2023-11-26T07:21:49.398331+0000","last_became_active":"2023-11-26T07:21:49.391905+0000","last_became_peered":"2023-11-26T07:21:49.391905+0000","last_unstale":"2023-11-26T07:21:49.398331+0000","last_undegraded":"2023-11-26T07:21:49.398331+0000","last_fullsized":"2023-11-26T07:21:49.398331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:15:43.249996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.395841+0000","last_change":"2023-11-26T07:21:49.388501+0000","last_active":"2023-11-26T07:21:49.395841+0000","last_peered":"2023-11-26T07:21:49.395841+0000","last_clean":"2023-11-26T07:21:49.395841+0000","last_became_active":"2023-11-26T07:21:49.388188+0000","last_became_peered":"2023-11-26T07:21:49.388188+0000","last_unstale":"2023-11-26T07:21:49.395841+0000","last_undegraded":"2023-11-26T07:21:49.395841+0000","last_fullsized":"2023-11-26T07:21:49.395841+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:37:10.533229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.397435+0000","last_change":"2023-11-26T07:21:49.391890+0000","last_active":"2023-11-26T07:21:49.397435+0000","last_peered":"2023-11-26T07:21:49.397435+0000","last_clean":"2023-11-26T07:21:49.397435+0000","last_became_active":"2023-11-26T07:21:49.391565+0000","last_became_peered":"2023-11-26T07:21:49.391565+0000","last_unstale":"2023-11-26T07:21:49.397435+0000","last_undegraded":"2023-11-26T07:21:49.397435+0000","last_fullsized":"2023-11-26T07:21:49.397435+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:34:44.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.392937+0000","last_change":"2023-11-26T07:21:49.382719+0000","last_active":"2023-11-26T07:21:49.392937+0000","last_peered":"2023-11-26T07:21:49.392937+0000","last_clean":"2023-11-26T07:21:49.392937+0000","last_became_active":"2023-11-26T07:21:49.382421+0000","last_became_peered":"2023-11-26T07:21:49.382421+0000","last_unstale":"2023-11-26T07:21:49.392937+0000","last_undegraded":"2023-11-26T07:21:49.392937+0000","last_fullsized":"2023-11-26T07:21:49.392937+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:48.862601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.407840+0000","last_change":"2023-11-26T07:21:49.403064+0000","last_active":"2023-11-26T07:21:49.407840+0000","last_peered":"2023-11-26T07:21:49.407840+0000","last_clean":"2023-11-26T07:21:49.407840+0000","last_became_active":"2023-11-26T07:21:49.402768+0000","last_became_peered":"2023-11-26T07:21:49.402768+0000","last_unstale":"2023-11-26T07:21:49.407840+0000","last_undegraded":"2023-11-26T07:21:49.407840+0000","last_fullsized":"2023-11-26T07:21:49.407840+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:52:24.065676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400461+0000","last_change":"2023-11-26T07:21:49.398692+0000","last_active":"2023-11-26T07:21:49.400461+0000","last_peered":"2023-11-26T07:21:49.400461+0000","last_clean":"2023-11-26T07:21:49.400461+0000","last_became_active":"2023-11-26T07:21:49.398331+0000","last_became_peered":"2023-11-26T07:21:49.398331+0000","last_unstale":"2023-11-26T07:21:49.400461+0000","last_undegraded":"2023-11-26T07:21:49.400461+0000","last_fullsized":"2023-11-26T07:21:49.400461+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:31:43.225955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.396693+0000","last_change":"2023-11-26T07:21:49.388716+0000","last_active":"2023-11-26T07:21:49.396693+0000","last_peered":"2023-11-26T07:21:49.396693+0000","last_clean":"2023-11-26T07:21:49.396693+0000","last_became_active":"2023-11-26T07:21:49.388182+0000","last_became_peered":"2023-11-26T07:21:49.388182+0000","last_unstale":"2023-11-26T07:21:49.396693+0000","last_undegraded":"2023-11-26T07:21:49.396693+0000","last_fullsized":"2023-11-26T07:21:49.396693+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:31:43.225955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.393284+0000","last_change":"2023-11-26T07:21:49.386891+0000","last_active":"2023-11-26T07:21:49.393284+0000","last_peered":"2023-11-26T07:21:49.393284+0000","last_clean":"2023-11-26T07:21:49.393284+0000","last_became_active":"2023-11-26T07:21:49.386563+0000","last_became_peered":"2023-11-26T07:21:49.386563+0000","last_unstale":"2023-11-26T07:21:49.393284+0000","last_undegraded":"2023-11-26T07:21:49.393284+0000","last_fullsized":"2023-11-26T07:21:49.393284+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:11.355579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.393090+0000","last_change":"2023-11-26T07:21:49.384756+0000","last_active":"2023-11-26T07:21:49.393090+0000","last_peered":"2023-11-26T07:21:49.393090+0000","last_clean":"2023-11-26T07:21:49.393090+0000","last_became_active":"2023-11-26T07:21:49.384336+0000","last_became_peered":"2023-11-26T07:21:49.384336+0000","last_unstale":"2023-11-26T07:21:49.393090+0000","last_undegraded":"2023-11-26T07:21:49.393090+0000","last_fullsized":"2023-11-26T07:21:49.393090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:00:05.668971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.399425+0000","last_change":"2023-11-26T07:21:49.388554+0000","last_active":"2023-11-26T07:21:49.399425+0000","last_peered":"2023-11-26T07:21:49.399425+0000","last_clean":"2023-11-26T07:21:49.399425+0000","last_became_active":"2023-11-26T07:21:49.388207+0000","last_became_peered":"2023-11-26T07:21:49.388207+0000","last_unstale":"2023-11-26T07:21:49.399425+0000","last_undegraded":"2023-11-26T07:21:49.399425+0000","last_fullsized":"2023-11-26T07:21:49.399425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:43.608330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.399025+0000","last_change":"2023-11-26T07:21:49.392681+0000","last_active":"2023-11-26T07:21:49.399025+0000","last_peered":"2023-11-26T07:21:49.399025+0000","last_clean":"2023-11-26T07:21:49.399025+0000","last_became_active":"2023-11-26T07:21:49.392410+0000","last_became_peered":"2023-11-26T07:21:49.392410+0000","last_unstale":"2023-11-26T07:21:49.399025+0000","last_undegraded":"2023-11-26T07:21:49.399025+0000","last_fullsized":"2023-11-26T07:21:49.399025+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:41.096161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.411027+0000","last_change":"2023-11-26T07:21:49.403067+0000","last_active":"2023-11-26T07:21:49.411027+0000","last_peered":"2023-11-26T07:21:49.411027+0000","last_clean":"2023-11-26T07:21:49.411027+0000","last_became_active":"2023-11-26T07:21:49.402635+0000","last_became_peered":"2023-11-26T07:21:49.402635+0000","last_unstale":"2023-11-26T07:21:49.411027+0000","last_undegraded":"2023-11-26T07:21:49.411027+0000","last_fullsized":"2023-11-26T07:21:49.411027+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:54:00.928815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400165+0000","last_change":"2023-11-26T07:21:49.394488+0000","last_active":"2023-11-26T07:21:49.400165+0000","last_peered":"2023-11-26T07:21:49.400165+0000","last_clean":"2023-11-26T07:21:49.400165+0000","last_became_active":"2023-11-26T07:21:49.394162+0000","last_became_peered":"2023-11-26T07:21:49.394162+0000","last_unstale":"2023-11-26T07:21:49.400165+0000","last_undegraded":"2023-11-26T07:21:49.400165+0000","last_fullsized":"2023-11-26T07:21:49.400165+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:41.370780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403104+0000","last_change":"2023-11-26T07:21:49.396545+0000","last_active":"2023-11-26T07:21:49.403104+0000","last_peered":"2023-11-26T07:21:49.403104+0000","last_clean":"2023-11-26T07:21:49.403104+0000","last_became_active":"2023-11-26T07:21:49.396212+0000","last_became_peered":"2023-11-26T07:21:49.396212+0000","last_unstale":"2023-11-26T07:21:49.403104+0000","last_undegraded":"2023-11-26T07:21:49.403104+0000","last_fullsized":"2023-11-26T07:21:49.403104+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:35.979200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.401515+0000","last_change":"2023-11-26T07:21:49.396517+0000","last_active":"2023-11-26T07:21:49.401515+0000","last_peered":"2023-11-26T07:21:49.401515+0000","last_clean":"2023-11-26T07:21:49.401515+0000","last_became_active":"2023-11-26T07:21:49.396193+0000","last_became_peered":"2023-11-26T07:21:49.396193+0000","last_unstale":"2023-11-26T07:21:49.401515+0000","last_undegraded":"2023-11-26T07:21:49.401515+0000","last_fullsized":"2023-11-26T07:21:49.401515+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:59:52.804916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.404633+0000","last_change":"2023-11-26T07:21:49.400059+0000","last_active":"2023-11-26T07:21:49.404633+0000","last_peered":"2023-11-26T07:21:49.404633+0000","last_clean":"2023-11-26T07:21:49.404633+0000","last_became_active":"2023-11-26T07:21:49.399750+0000","last_became_peered":"2023-11-26T07:21:49.399750+0000","last_unstale":"2023-11-26T07:21:49.404633+0000","last_undegraded":"2023-11-26T07:21:49.404633+0000","last_fullsized":"2023-11-26T07:21:49.404633+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:14:05.777652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.401410+0000","last_change":"2023-11-26T07:21:49.388013+0000","last_active":"2023-11-26T07:21:49.401410+0000","last_peered":"2023-11-26T07:21:49.401410+0000","last_clean":"2023-11-26T07:21:49.401410+0000","last_became_active":"2023-11-26T07:21:49.387712+0000","last_became_peered":"2023-11-26T07:21:49.387712+0000","last_unstale":"2023-11-26T07:21:49.401410+0000","last_undegraded":"2023-11-26T07:21:49.401410+0000","last_fullsized":"2023-11-26T07:21:49.401410+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:53:12.402635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408050+0000","last_change":"2023-11-26T07:21:49.402177+0000","last_active":"2023-11-26T07:21:49.408050+0000","last_peered":"2023-11-26T07:21:49.408050+0000","last_clean":"2023-11-26T07:21:49.408050+0000","last_became_active":"2023-11-26T07:21:49.401727+0000","last_became_peered":"2023-11-26T07:21:49.401727+0000","last_unstale":"2023-11-26T07:21:49.408050+0000","last_undegraded":"2023-11-26T07:21:49.408050+0000","last_fullsized":"2023-11-26T07:21:49.408050+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:54:00.928815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403657+0000","last_change":"2023-11-26T07:21:49.391443+0000","last_active":"2023-11-26T07:21:49.403657+0000","last_peered":"2023-11-26T07:21:49.403657+0000","last_clean":"2023-11-26T07:21:49.403657+0000","last_became_active":"2023-11-26T07:21:49.391082+0000","last_became_peered":"2023-11-26T07:21:49.391082+0000","last_unstale":"2023-11-26T07:21:49.403657+0000","last_undegraded":"2023-11-26T07:21:49.403657+0000","last_fullsized":"2023-11-26T07:21:49.403657+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:34:44.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.395651+0000","last_change":"2023-11-26T07:21:49.387843+0000","last_active":"2023-11-26T07:21:49.395651+0000","last_peered":"2023-11-26T07:21:49.395651+0000","last_clean":"2023-11-26T07:21:49.395651+0000","last_became_active":"2023-11-26T07:21:49.386986+0000","last_became_peered":"2023-11-26T07:21:49.386986+0000","last_unstale":"2023-11-26T07:21:49.395651+0000","last_undegraded":"2023-11-26T07:21:49.395651+0000","last_fullsized":"2023-11-26T07:21:49.395651+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:47.172012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.399080+0000","last_change":"2023-11-26T07:21:49.393020+0000","last_active":"2023-11-26T07:21:49.399080+0000","last_peered":"2023-11-26T07:21:49.399080+0000","last_clean":"2023-11-26T07:21:49.399080+0000","last_became_active":"2023-11-26T07:21:49.391427+0000","last_became_peered":"2023-11-26T07:21:49.391427+0000","last_unstale":"2023-11-26T07:21:49.399080+0000","last_undegraded":"2023-11-26T07:21:49.399080+0000","last_fullsized":"2023-11-26T07:21:49.399080+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:11.355579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"67'4","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.415391+0000","last_change":"2023-11-26T07:21:48.398703+0000","last_active":"2023-11-26T07:21:49.415391+0000","last_peered":"2023-11-26T07:21:49.415391+0000","last_clean":"2023-11-26T07:21:49.415391+0000","last_became_active":"2023-11-26T07:21:48.398368+0000","last_became_peered":"2023-11-26T07:21:48.398368+0000","last_unstale":"2023-11-26T07:21:49.415391+0000","last_undegraded":"2023-11-26T07:21:49.415391+0000","last_fullsized":"2023-11-26T07:21:49.415391+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:42:26.965755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.402018+0000","last_change":"2023-11-26T07:21:49.397748+0000","last_active":"2023-11-26T07:21:49.402018+0000","last_peered":"2023-11-26T07:21:49.402018+0000","last_clean":"2023-11-26T07:21:49.402018+0000","last_became_active":"2023-11-26T07:21:49.396946+0000","last_became_peered":"2023-11-26T07:21:49.396946+0000","last_unstale":"2023-11-26T07:21:49.402018+0000","last_undegraded":"2023-11-26T07:21:49.402018+0000","last_fullsized":"2023-11-26T07:21:49.402018+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:56:41.517579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"67'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.412055+0000","last_change":"2023-11-26T07:21:48.394808+0000","last_active":"2023-11-26T07:21:49.412055+0000","last_peered":"2023-11-26T07:21:49.412055+0000","last_clean":"2023-11-26T07:21:49.412055+0000","last_became_active":"2023-11-26T07:21:48.394448+0000","last_became_peered":"2023-11-26T07:21:48.394448+0000","last_unstale":"2023-11-26T07:21:49.412055+0000","last_undegraded":"2023-11-26T07:21:49.412055+0000","last_fullsized":"2023-11-26T07:21:49.412055+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:49.973476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400352+0000","last_change":"2023-11-26T07:21:49.392982+0000","last_active":"2023-11-26T07:21:49.400352+0000","last_peered":"2023-11-26T07:21:49.400352+0000","last_clean":"2023-11-26T07:21:49.400352+0000","last_became_active":"2023-11-26T07:21:49.392682+0000","last_became_peered":"2023-11-26T07:21:49.392682+0000","last_unstale":"2023-11-26T07:21:49.400352+0000","last_undegraded":"2023-11-26T07:21:49.400352+0000","last_fullsized":"2023-11-26T07:21:49.400352+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:53:24.881150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.417326+0000","last_change":"2023-11-26T07:21:48.394872+0000","last_active":"2023-11-26T07:21:49.417326+0000","last_peered":"2023-11-26T07:21:49.417326+0000","last_clean":"2023-11-26T07:21:49.417326+0000","last_became_active":"2023-11-26T07:21:48.394493+0000","last_became_peered":"2023-11-26T07:21:48.394493+0000","last_unstale":"2023-11-26T07:21:49.417326+0000","last_undegraded":"2023-11-26T07:21:49.417326+0000","last_fullsized":"2023-11-26T07:21:49.417326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:38:58.619515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408637+0000","last_change":"2023-11-26T07:21:49.399480+0000","last_active":"2023-11-26T07:21:49.408637+0000","last_peered":"2023-11-26T07:21:49.408637+0000","last_clean":"2023-11-26T07:21:49.408637+0000","last_became_active":"2023-11-26T07:21:49.399154+0000","last_became_peered":"2023-11-26T07:21:49.399154+0000","last_unstale":"2023-11-26T07:21:49.408637+0000","last_undegraded":"2023-11-26T07:21:49.408637+0000","last_fullsized":"2023-11-26T07:21:49.408637+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:58:27.502782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.413847+0000","last_change":"2023-11-26T07:21:48.385903+0000","last_active":"2023-11-26T07:21:49.413847+0000","last_peered":"2023-11-26T07:21:49.413847+0000","last_clean":"2023-11-26T07:21:49.413847+0000","last_became_active":"2023-11-26T07:21:48.385507+0000","last_became_peered":"2023-11-26T07:21:48.385507+0000","last_unstale":"2023-11-26T07:21:49.413847+0000","last_undegraded":"2023-11-26T07:21:49.413847+0000","last_fullsized":"2023-11-26T07:21:49.413847+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:47.850557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.404445+0000","last_change":"2023-11-26T07:21:49.397875+0000","last_active":"2023-11-26T07:21:49.404445+0000","last_peered":"2023-11-26T07:21:49.404445+0000","last_clean":"2023-11-26T07:21:49.404445+0000","last_became_active":"2023-11-26T07:21:49.397545+0000","last_became_peered":"2023-11-26T07:21:49.397545+0000","last_unstale":"2023-11-26T07:21:49.404445+0000","last_undegraded":"2023-11-26T07:21:49.404445+0000","last_fullsized":"2023-11-26T07:21:49.404445+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:11.355579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"67'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.414555+0000","last_change":"2023-11-26T07:21:48.398816+0000","last_active":"2023-11-26T07:21:49.414555+0000","last_peered":"2023-11-26T07:21:49.414555+0000","last_clean":"2023-11-26T07:21:49.414555+0000","last_became_active":"2023-11-26T07:21:48.398412+0000","last_became_peered":"2023-11-26T07:21:48.398412+0000","last_unstale":"2023-11-26T07:21:49.414555+0000","last_undegraded":"2023-11-26T07:21:49.414555+0000","last_fullsized":"2023-11-26T07:21:49.414555+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:49.166458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400385+0000","last_change":"2023-11-26T07:21:49.392972+0000","last_active":"2023-11-26T07:21:49.400385+0000","last_peered":"2023-11-26T07:21:49.400385+0000","last_clean":"2023-11-26T07:21:49.400385+0000","last_became_active":"2023-11-26T07:21:49.392682+0000","last_became_peered":"2023-11-26T07:21:49.392682+0000","last_unstale":"2023-11-26T07:21:49.400385+0000","last_undegraded":"2023-11-26T07:21:49.400385+0000","last_fullsized":"2023-11-26T07:21:49.400385+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:55:23.508125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.413341+0000","last_change":"2023-11-26T07:21:48.392436+0000","last_active":"2023-11-26T07:21:49.413341+0000","last_peered":"2023-11-26T07:21:49.413341+0000","last_clean":"2023-11-26T07:21:49.413341+0000","last_became_active":"2023-11-26T07:21:48.391047+0000","last_became_peered":"2023-11-26T07:21:48.391047+0000","last_unstale":"2023-11-26T07:21:49.413341+0000","last_undegraded":"2023-11-26T07:21:49.413341+0000","last_fullsized":"2023-11-26T07:21:49.413341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:42.596286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.401746+0000","last_change":"2023-11-26T07:21:49.394754+0000","last_active":"2023-11-26T07:21:49.401746+0000","last_peered":"2023-11-26T07:21:49.401746+0000","last_clean":"2023-11-26T07:21:49.401746+0000","last_became_active":"2023-11-26T07:21:49.394486+0000","last_became_peered":"2023-11-26T07:21:49.394486+0000","last_unstale":"2023-11-26T07:21:49.401746+0000","last_undegraded":"2023-11-26T07:21:49.401746+0000","last_fullsized":"2023-11-26T07:21:49.401746+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:27:27.588608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.387669+0000","last_change":"2023-11-26T07:21:48.393086+0000","last_active":"2023-11-26T07:21:49.387669+0000","last_peered":"2023-11-26T07:21:49.387669+0000","last_clean":"2023-11-26T07:21:49.387669+0000","last_became_active":"2023-11-26T07:21:48.392743+0000","last_became_peered":"2023-11-26T07:21:48.392743+0000","last_unstale":"2023-11-26T07:21:49.387669+0000","last_undegraded":"2023-11-26T07:21:49.387669+0000","last_fullsized":"2023-11-26T07:21:49.387669+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:50.658713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.398425+0000","last_change":"2023-11-26T07:21:49.393868+0000","last_active":"2023-11-26T07:21:49.398425+0000","last_peered":"2023-11-26T07:21:49.398425+0000","last_clean":"2023-11-26T07:21:49.398425+0000","last_became_active":"2023-11-26T07:21:49.393597+0000","last_became_peered":"2023-11-26T07:21:49.393597+0000","last_unstale":"2023-11-26T07:21:49.398425+0000","last_undegraded":"2023-11-26T07:21:49.398425+0000","last_fullsized":"2023-11-26T07:21:49.398425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:23.963334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.379572+0000","last_change":"2023-11-26T07:21:48.381170+0000","last_active":"2023-11-26T07:21:49.379572+0000","last_peered":"2023-11-26T07:21:49.379572+0000","last_clean":"2023-11-26T07:21:49.379572+0000","last_became_active":"2023-11-26T07:21:48.380878+0000","last_became_peered":"2023-11-26T07:21:48.380878+0000","last_unstale":"2023-11-26T07:21:49.379572+0000","last_undegraded":"2023-11-26T07:21:49.379572+0000","last_fullsized":"2023-11-26T07:21:49.379572+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:32:55.075524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403084+0000","last_change":"2023-11-26T07:21:49.394081+0000","last_active":"2023-11-26T07:21:49.403084+0000","last_peered":"2023-11-26T07:21:49.403084+0000","last_clean":"2023-11-26T07:21:49.403084+0000","last_became_active":"2023-11-26T07:21:49.393766+0000","last_became_peered":"2023-11-26T07:21:49.393766+0000","last_unstale":"2023-11-26T07:21:49.403084+0000","last_undegraded":"2023-11-26T07:21:49.403084+0000","last_fullsized":"2023-11-26T07:21:49.403084+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:23.963334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.376564+0000","last_change":"2023-11-26T07:21:48.396547+0000","last_active":"2023-11-26T07:21:49.376564+0000","last_peered":"2023-11-26T07:21:49.376564+0000","last_clean":"2023-11-26T07:21:49.376564+0000","last_became_active":"2023-11-26T07:21:48.394575+0000","last_became_peered":"2023-11-26T07:21:48.394575+0000","last_unstale":"2023-11-26T07:21:49.376564+0000","last_undegraded":"2023-11-26T07:21:49.376564+0000","last_fullsized":"2023-11-26T07:21:49.376564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:07:00.249022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408887+0000","last_change":"2023-11-26T07:21:49.402868+0000","last_active":"2023-11-26T07:21:49.408887+0000","last_peered":"2023-11-26T07:21:49.408887+0000","last_clean":"2023-11-26T07:21:49.408887+0000","last_became_active":"2023-11-26T07:21:49.402556+0000","last_became_peered":"2023-11-26T07:21:49.402556+0000","last_unstale":"2023-11-26T07:21:49.408887+0000","last_undegraded":"2023-11-26T07:21:49.408887+0000","last_fullsized":"2023-11-26T07:21:49.408887+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:55:30.809651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.378200+0000","last_change":"2023-11-26T07:21:48.388629+0000","last_active":"2023-11-26T07:21:49.378200+0000","last_peered":"2023-11-26T07:21:49.378200+0000","last_clean":"2023-11-26T07:21:49.378200+0000","last_became_active":"2023-11-26T07:21:48.388348+0000","last_became_peered":"2023-11-26T07:21:48.388348+0000","last_unstale":"2023-11-26T07:21:49.378200+0000","last_undegraded":"2023-11-26T07:21:49.378200+0000","last_fullsized":"2023-11-26T07:21:49.378200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:39.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.391193+0000","last_change":"2023-11-26T07:21:49.383791+0000","last_active":"2023-11-26T07:21:49.391193+0000","last_peered":"2023-11-26T07:21:49.391193+0000","last_clean":"2023-11-26T07:21:49.391193+0000","last_became_active":"2023-11-26T07:21:49.383503+0000","last_became_peered":"2023-11-26T07:21:49.383503+0000","last_unstale":"2023-11-26T07:21:49.391193+0000","last_undegraded":"2023-11-26T07:21:49.391193+0000","last_fullsized":"2023-11-26T07:21:49.391193+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:51.670757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"67'3","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.416207+0000","last_change":"2023-11-26T07:21:48.393905+0000","last_active":"2023-11-26T07:21:49.416207+0000","last_peered":"2023-11-26T07:21:49.416207+0000","last_clean":"2023-11-26T07:21:49.416207+0000","last_became_active":"2023-11-26T07:21:48.393594+0000","last_became_peered":"2023-11-26T07:21:48.393594+0000","last_unstale":"2023-11-26T07:21:49.416207+0000","last_undegraded":"2023-11-26T07:21:49.416207+0000","last_fullsized":"2023-11-26T07:21:49.416207+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:04.690030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.405697+0000","last_change":"2023-11-26T07:21:49.398185+0000","last_active":"2023-11-26T07:21:49.405697+0000","last_peered":"2023-11-26T07:21:49.405697+0000","last_clean":"2023-11-26T07:21:49.405697+0000","last_became_active":"2023-11-26T07:21:49.397672+0000","last_became_peered":"2023-11-26T07:21:49.397672+0000","last_unstale":"2023-11-26T07:21:49.405697+0000","last_undegraded":"2023-11-26T07:21:49.405697+0000","last_fullsized":"2023-11-26T07:21:49.405697+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:23.156268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"67'6","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.419664+0000","last_change":"2023-11-26T07:21:48.387605+0000","last_active":"2023-11-26T07:21:49.419664+0000","last_peered":"2023-11-26T07:21:49.419664+0000","last_clean":"2023-11-26T07:21:49.419664+0000","last_became_active":"2023-11-26T07:21:48.386842+0000","last_became_peered":"2023-11-26T07:21:48.386842+0000","last_unstale":"2023-11-26T07:21:49.419664+0000","last_undegraded":"2023-11-26T07:21:49.419664+0000","last_fullsized":"2023-11-26T07:21:49.419664+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:04:34.750837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403784+0000","last_change":"2023-11-26T07:21:49.397364+0000","last_active":"2023-11-26T07:21:49.403784+0000","last_peered":"2023-11-26T07:21:49.403784+0000","last_clean":"2023-11-26T07:21:49.403784+0000","last_became_active":"2023-11-26T07:21:49.396515+0000","last_became_peered":"2023-11-26T07:21:49.396515+0000","last_unstale":"2023-11-26T07:21:49.403784+0000","last_undegraded":"2023-11-26T07:21:49.403784+0000","last_fullsized":"2023-11-26T07:21:49.403784+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:15:43.249996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.417812+0000","last_change":"2023-11-26T07:21:48.393070+0000","last_active":"2023-11-26T07:21:49.417812+0000","last_peered":"2023-11-26T07:21:49.417812+0000","last_clean":"2023-11-26T07:21:49.417812+0000","last_became_active":"2023-11-26T07:21:48.392740+0000","last_became_peered":"2023-11-26T07:21:48.392740+0000","last_unstale":"2023-11-26T07:21:49.417812+0000","last_undegraded":"2023-11-26T07:21:49.417812+0000","last_fullsized":"2023-11-26T07:21:49.417812+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:51:42.069049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.404413+0000","last_change":"2023-11-26T07:21:49.398053+0000","last_active":"2023-11-26T07:21:49.404413+0000","last_peered":"2023-11-26T07:21:49.404413+0000","last_clean":"2023-11-26T07:21:49.404413+0000","last_became_active":"2023-11-26T07:21:49.397697+0000","last_became_peered":"2023-11-26T07:21:49.397697+0000","last_unstale":"2023-11-26T07:21:49.404413+0000","last_undegraded":"2023-11-26T07:21:49.404413+0000","last_fullsized":"2023-11-26T07:21:49.404413+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:07:01.261066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.377591+0000","last_change":"2023-11-26T07:21:48.394119+0000","last_active":"2023-11-26T07:21:49.377591+0000","last_peered":"2023-11-26T07:21:49.377591+0000","last_clean":"2023-11-26T07:21:49.377591+0000","last_became_active":"2023-11-26T07:21:48.393783+0000","last_became_peered":"2023-11-26T07:21:48.393783+0000","last_unstale":"2023-11-26T07:21:49.377591+0000","last_undegraded":"2023-11-26T07:21:49.377591+0000","last_fullsized":"2023-11-26T07:21:49.377591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:14:04.765608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.394060+0000","last_change":"2023-11-26T07:21:49.389159+0000","last_active":"2023-11-26T07:21:49.394060+0000","last_peered":"2023-11-26T07:21:49.394060+0000","last_clean":"2023-11-26T07:21:49.394060+0000","last_became_active":"2023-11-26T07:21:49.388206+0000","last_became_peered":"2023-11-26T07:21:49.388206+0000","last_unstale":"2023-11-26T07:21:49.394060+0000","last_undegraded":"2023-11-26T07:21:49.394060+0000","last_fullsized":"2023-11-26T07:21:49.394060+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:41.096161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.382101+0000","last_change":"2023-11-26T07:21:48.392636+0000","last_active":"2023-11-26T07:21:49.382101+0000","last_peered":"2023-11-26T07:21:49.382101+0000","last_clean":"2023-11-26T07:21:49.382101+0000","last_became_active":"2023-11-26T07:21:48.392359+0000","last_became_peered":"2023-11-26T07:21:48.392359+0000","last_unstale":"2023-11-26T07:21:49.382101+0000","last_undegraded":"2023-11-26T07:21:49.382101+0000","last_fullsized":"2023-11-26T07:21:49.382101+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:24:39.610517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.392586+0000","last_change":"2023-11-26T07:21:49.387238+0000","last_active":"2023-11-26T07:21:49.392586+0000","last_peered":"2023-11-26T07:21:49.392586+0000","last_clean":"2023-11-26T07:21:49.392586+0000","last_became_active":"2023-11-26T07:21:49.385793+0000","last_became_peered":"2023-11-26T07:21:49.385793+0000","last_unstale":"2023-11-26T07:21:49.392586+0000","last_undegraded":"2023-11-26T07:21:49.392586+0000","last_fullsized":"2023-11-26T07:21:49.392586+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:06:52.285644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"67'8","reported_seq":19,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.418263+0000","last_change":"2023-11-26T07:21:48.396212+0000","last_active":"2023-11-26T07:21:49.418263+0000","last_peered":"2023-11-26T07:21:49.418263+0000","last_clean":"2023-11-26T07:21:49.418263+0000","last_became_active":"2023-11-26T07:21:48.395180+0000","last_became_peered":"2023-11-26T07:21:48.395180+0000","last_unstale":"2023-11-26T07:21:49.418263+0000","last_undegraded":"2023-11-26T07:21:49.418263+0000","last_fullsized":"2023-11-26T07:21:49.418263+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:23.373412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400648+0000","last_change":"2023-11-26T07:21:49.394799+0000","last_active":"2023-11-26T07:21:49.400648+0000","last_peered":"2023-11-26T07:21:49.400648+0000","last_clean":"2023-11-26T07:21:49.400648+0000","last_became_active":"2023-11-26T07:21:49.394528+0000","last_became_peered":"2023-11-26T07:21:49.394528+0000","last_unstale":"2023-11-26T07:21:49.400648+0000","last_undegraded":"2023-11-26T07:21:49.400648+0000","last_fullsized":"2023-11-26T07:21:49.400648+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:30:16.661796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.414586+0000","last_change":"2023-11-26T07:21:48.393181+0000","last_active":"2023-11-26T07:21:49.414586+0000","last_peered":"2023-11-26T07:21:49.414586+0000","last_clean":"2023-11-26T07:21:49.414586+0000","last_became_active":"2023-11-26T07:21:48.392844+0000","last_became_peered":"2023-11-26T07:21:48.392844+0000","last_unstale":"2023-11-26T07:21:49.414586+0000","last_undegraded":"2023-11-26T07:21:49.414586+0000","last_fullsized":"2023-11-26T07:21:49.414586+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:23.373412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408549+0000","last_change":"2023-11-26T07:21:49.399317+0000","last_active":"2023-11-26T07:21:49.408549+0000","last_peered":"2023-11-26T07:21:49.408549+0000","last_clean":"2023-11-26T07:21:49.408549+0000","last_became_active":"2023-11-26T07:21:49.399029+0000","last_became_peered":"2023-11-26T07:21:49.399029+0000","last_unstale":"2023-11-26T07:21:49.408549+0000","last_undegraded":"2023-11-26T07:21:49.408549+0000","last_fullsized":"2023-11-26T07:21:49.408549+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:14.154472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.414438+0000","last_change":"2023-11-26T07:21:48.395462+0000","last_active":"2023-11-26T07:21:49.414438+0000","last_peered":"2023-11-26T07:21:49.414438+0000","last_clean":"2023-11-26T07:21:49.414438+0000","last_became_active":"2023-11-26T07:21:48.395219+0000","last_became_peered":"2023-11-26T07:21:48.395219+0000","last_unstale":"2023-11-26T07:21:49.414438+0000","last_undegraded":"2023-11-26T07:21:49.414438+0000","last_fullsized":"2023-11-26T07:21:49.414438+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:34.967156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408102+0000","last_change":"2023-11-26T07:21:49.401344+0000","last_active":"2023-11-26T07:21:49.408102+0000","last_peered":"2023-11-26T07:21:49.408102+0000","last_clean":"2023-11-26T07:21:49.408102+0000","last_became_active":"2023-11-26T07:21:49.400912+0000","last_became_peered":"2023-11-26T07:21:49.400912+0000","last_unstale":"2023-11-26T07:21:49.408102+0000","last_undegraded":"2023-11-26T07:21:49.408102+0000","last_fullsized":"2023-11-26T07:21:49.408102+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:21.895779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408244+0000","last_change":"2023-11-26T07:21:48.392365+0000","last_active":"2023-11-26T07:21:49.408244+0000","last_peered":"2023-11-26T07:21:49.408244+0000","last_clean":"2023-11-26T07:21:49.408244+0000","last_became_active":"2023-11-26T07:21:48.392071+0000","last_became_peered":"2023-11-26T07:21:48.392071+0000","last_unstale":"2023-11-26T07:21:49.408244+0000","last_undegraded":"2023-11-26T07:21:49.408244+0000","last_fullsized":"2023-11-26T07:21:49.408244+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:40.358736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.405677+0000","last_change":"2023-11-26T07:21:49.397997+0000","last_active":"2023-11-26T07:21:49.405677+0000","last_peered":"2023-11-26T07:21:49.405677+0000","last_clean":"2023-11-26T07:21:49.405677+0000","last_became_active":"2023-11-26T07:21:49.397225+0000","last_became_peered":"2023-11-26T07:21:49.397225+0000","last_unstale":"2023-11-26T07:21:49.405677+0000","last_undegraded":"2023-11-26T07:21:49.405677+0000","last_fullsized":"2023-11-26T07:21:49.405677+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:05.170257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.386090+0000","last_change":"2023-11-26T07:21:48.398779+0000","last_active":"2023-11-26T07:21:49.386090+0000","last_peered":"2023-11-26T07:21:49.386090+0000","last_clean":"2023-11-26T07:21:49.386090+0000","last_became_active":"2023-11-26T07:21:48.398174+0000","last_became_peered":"2023-11-26T07:21:48.398174+0000","last_unstale":"2023-11-26T07:21:49.386090+0000","last_undegraded":"2023-11-26T07:21:49.386090+0000","last_fullsized":"2023-11-26T07:21:49.386090+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:24:39.610517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.401430+0000","last_change":"2023-11-26T07:21:49.395666+0000","last_active":"2023-11-26T07:21:49.401430+0000","last_peered":"2023-11-26T07:21:49.401430+0000","last_clean":"2023-11-26T07:21:49.401430+0000","last_became_active":"2023-11-26T07:21:49.395321+0000","last_became_peered":"2023-11-26T07:21:49.395321+0000","last_unstale":"2023-11-26T07:21:49.401430+0000","last_undegraded":"2023-11-26T07:21:49.401430+0000","last_fullsized":"2023-11-26T07:21:49.401430+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:46:35.365406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.377733+0000","last_change":"2023-11-26T07:21:48.392226+0000","last_active":"2023-11-26T07:21:49.377733+0000","last_peered":"2023-11-26T07:21:49.377733+0000","last_clean":"2023-11-26T07:21:49.377733+0000","last_became_active":"2023-11-26T07:21:48.390698+0000","last_became_peered":"2023-11-26T07:21:48.390698+0000","last_unstale":"2023-11-26T07:21:49.377733+0000","last_undegraded":"2023-11-26T07:21:49.377733+0000","last_fullsized":"2023-11-26T07:21:49.377733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:30:15.649752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.396402+0000","last_change":"2023-11-26T07:21:49.386769+0000","last_active":"2023-11-26T07:21:49.396402+0000","last_peered":"2023-11-26T07:21:49.396402+0000","last_clean":"2023-11-26T07:21:49.396402+0000","last_became_active":"2023-11-26T07:21:49.386449+0000","last_became_peered":"2023-11-26T07:21:49.386449+0000","last_unstale":"2023-11-26T07:21:49.396402+0000","last_undegraded":"2023-11-26T07:21:49.396402+0000","last_fullsized":"2023-11-26T07:21:49.396402+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:16.783367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.384733+0000","last_change":"2023-11-26T07:21:48.390565+0000","last_active":"2023-11-26T07:21:49.384733+0000","last_peered":"2023-11-26T07:21:49.384733+0000","last_clean":"2023-11-26T07:21:49.384733+0000","last_became_active":"2023-11-26T07:21:48.390300+0000","last_became_peered":"2023-11-26T07:21:48.390300+0000","last_unstale":"2023-11-26T07:21:49.384733+0000","last_undegraded":"2023-11-26T07:21:49.384733+0000","last_fullsized":"2023-11-26T07:21:49.384733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:49.973476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.396734+0000","last_change":"2023-11-26T07:21:49.387988+0000","last_active":"2023-11-26T07:21:49.396734+0000","last_peered":"2023-11-26T07:21:49.396734+0000","last_clean":"2023-11-26T07:21:49.396734+0000","last_became_active":"2023-11-26T07:21:49.386702+0000","last_became_peered":"2023-11-26T07:21:49.386702+0000","last_unstale":"2023-11-26T07:21:49.396734+0000","last_undegraded":"2023-11-26T07:21:49.396734+0000","last_fullsized":"2023-11-26T07:21:49.396734+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:08.928818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.380271+0000","last_change":"2023-11-26T07:21:48.393566+0000","last_active":"2023-11-26T07:21:49.380271+0000","last_peered":"2023-11-26T07:21:49.380271+0000","last_clean":"2023-11-26T07:21:49.380271+0000","last_became_active":"2023-11-26T07:21:48.393280+0000","last_became_peered":"2023-11-26T07:21:48.393280+0000","last_unstale":"2023-11-26T07:21:49.380271+0000","last_undegraded":"2023-11-26T07:21:49.380271+0000","last_fullsized":"2023-11-26T07:21:49.380271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:33.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.398366+0000","last_change":"2023-11-26T07:21:49.392245+0000","last_active":"2023-11-26T07:21:49.398366+0000","last_peered":"2023-11-26T07:21:49.398366+0000","last_clean":"2023-11-26T07:21:49.398366+0000","last_became_active":"2023-11-26T07:21:49.391954+0000","last_became_peered":"2023-11-26T07:21:49.391954+0000","last_unstale":"2023-11-26T07:21:49.398366+0000","last_undegraded":"2023-11-26T07:21:49.398366+0000","last_fullsized":"2023-11-26T07:21:49.398366+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:54:00.928815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.413342+0000","last_change":"2023-11-26T07:21:48.392636+0000","last_active":"2023-11-26T07:21:49.413342+0000","last_peered":"2023-11-26T07:21:49.413342+0000","last_clean":"2023-11-26T07:21:49.413342+0000","last_became_active":"2023-11-26T07:21:48.392335+0000","last_became_peered":"2023-11-26T07:21:48.392335+0000","last_unstale":"2023-11-26T07:21:49.413342+0000","last_undegraded":"2023-11-26T07:21:49.413342+0000","last_fullsized":"2023-11-26T07:21:49.413342+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:19:14.549626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403214+0000","last_change":"2023-11-26T07:21:49.396214+0000","last_active":"2023-11-26T07:21:49.403214+0000","last_peered":"2023-11-26T07:21:49.403214+0000","last_clean":"2023-11-26T07:21:49.403214+0000","last_became_active":"2023-11-26T07:21:49.395836+0000","last_became_peered":"2023-11-26T07:21:49.395836+0000","last_unstale":"2023-11-26T07:21:49.403214+0000","last_undegraded":"2023-11-26T07:21:49.403214+0000","last_fullsized":"2023-11-26T07:21:49.403214+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:15:43.249996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.381885+0000","last_change":"2023-11-26T07:21:48.392000+0000","last_active":"2023-11-26T07:21:49.381885+0000","last_peered":"2023-11-26T07:21:49.381885+0000","last_clean":"2023-11-26T07:21:49.381885+0000","last_became_active":"2023-11-26T07:21:48.391748+0000","last_became_peered":"2023-11-26T07:21:48.391748+0000","last_unstale":"2023-11-26T07:21:49.381885+0000","last_undegraded":"2023-11-26T07:21:49.381885+0000","last_fullsized":"2023-11-26T07:21:49.381885+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:33.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408869+0000","last_change":"2023-11-26T07:21:49.400924+0000","last_active":"2023-11-26T07:21:49.408869+0000","last_peered":"2023-11-26T07:21:49.408869+0000","last_clean":"2023-11-26T07:21:49.408869+0000","last_became_active":"2023-11-26T07:21:49.400635+0000","last_became_peered":"2023-11-26T07:21:49.400635+0000","last_unstale":"2023-11-26T07:21:49.408869+0000","last_undegraded":"2023-11-26T07:21:49.408869+0000","last_fullsized":"2023-11-26T07:21:49.408869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:34.307607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.380870+0000","last_change":"2023-11-26T07:21:48.390952+0000","last_active":"2023-11-26T07:21:49.380870+0000","last_peered":"2023-11-26T07:21:49.380870+0000","last_clean":"2023-11-26T07:21:49.380870+0000","last_became_active":"2023-11-26T07:21:48.390649+0000","last_became_peered":"2023-11-26T07:21:48.390649+0000","last_unstale":"2023-11-26T07:21:49.380870+0000","last_undegraded":"2023-11-26T07:21:49.380870+0000","last_fullsized":"2023-11-26T07:21:49.380870+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:47:24.227453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.411276+0000","last_change":"2023-11-26T07:21:49.403406+0000","last_active":"2023-11-26T07:21:49.411276+0000","last_peered":"2023-11-26T07:21:49.411276+0000","last_clean":"2023-11-26T07:21:49.411276+0000","last_became_active":"2023-11-26T07:21:49.401830+0000","last_became_peered":"2023-11-26T07:21:49.401830+0000","last_unstale":"2023-11-26T07:21:49.411276+0000","last_undegraded":"2023-11-26T07:21:49.411276+0000","last_fullsized":"2023-11-26T07:21:49.411276+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:01.553446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.380658+0000","last_change":"2023-11-26T07:21:48.396049+0000","last_active":"2023-11-26T07:21:49.380658+0000","last_peered":"2023-11-26T07:21:49.380658+0000","last_clean":"2023-11-26T07:21:49.380658+0000","last_became_active":"2023-11-26T07:21:48.395756+0000","last_became_peered":"2023-11-26T07:21:48.395756+0000","last_unstale":"2023-11-26T07:21:49.380658+0000","last_undegraded":"2023-11-26T07:21:49.380658+0000","last_fullsized":"2023-11-26T07:21:49.380658+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:24:39.610517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.410477+0000","last_change":"2023-11-26T07:21:49.399941+0000","last_active":"2023-11-26T07:21:49.410477+0000","last_peered":"2023-11-26T07:21:49.410477+0000","last_clean":"2023-11-26T07:21:49.410477+0000","last_became_active":"2023-11-26T07:21:49.399610+0000","last_became_peered":"2023-11-26T07:21:49.399610+0000","last_unstale":"2023-11-26T07:21:49.410477+0000","last_undegraded":"2023-11-26T07:21:49.410477+0000","last_fullsized":"2023-11-26T07:21:49.410477+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:40.976535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.379828+0000","last_change":"2023-11-26T07:21:48.383780+0000","last_active":"2023-11-26T07:21:49.379828+0000","last_peered":"2023-11-26T07:21:49.379828+0000","last_clean":"2023-11-26T07:21:49.379828+0000","last_became_active":"2023-11-26T07:21:48.383535+0000","last_became_peered":"2023-11-26T07:21:48.383535+0000","last_unstale":"2023-11-26T07:21:49.379828+0000","last_undegraded":"2023-11-26T07:21:49.379828+0000","last_fullsized":"2023-11-26T07:21:49.379828+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:16:06.413411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400414+0000","last_change":"2023-11-26T07:21:49.393305+0000","last_active":"2023-11-26T07:21:49.400414+0000","last_peered":"2023-11-26T07:21:49.400414+0000","last_clean":"2023-11-26T07:21:49.400414+0000","last_became_active":"2023-11-26T07:21:49.393042+0000","last_became_peered":"2023-11-26T07:21:49.393042+0000","last_unstale":"2023-11-26T07:21:49.400414+0000","last_undegraded":"2023-11-26T07:21:49.400414+0000","last_fullsized":"2023-11-26T07:21:49.400414+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:04:35.762881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.388267+0000","last_change":"2023-11-26T07:21:48.391398+0000","last_active":"2023-11-26T07:21:49.388267+0000","last_peered":"2023-11-26T07:21:49.388267+0000","last_clean":"2023-11-26T07:21:49.388267+0000","last_became_active":"2023-11-26T07:21:48.391125+0000","last_became_peered":"2023-11-26T07:21:48.391125+0000","last_unstale":"2023-11-26T07:21:49.388267+0000","last_undegraded":"2023-11-26T07:21:49.388267+0000","last_fullsized":"2023-11-26T07:21:49.388267+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:43.928801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.405814+0000","last_change":"2023-11-26T07:21:49.400041+0000","last_active":"2023-11-26T07:21:49.405814+0000","last_peered":"2023-11-26T07:21:49.405814+0000","last_clean":"2023-11-26T07:21:49.405814+0000","last_became_active":"2023-11-26T07:21:49.399776+0000","last_became_peered":"2023-11-26T07:21:49.399776+0000","last_unstale":"2023-11-26T07:21:49.405814+0000","last_undegraded":"2023-11-26T07:21:49.405814+0000","last_fullsized":"2023-11-26T07:21:49.405814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:18:58.253246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.375830+0000","last_change":"2023-11-26T07:21:48.395694+0000","last_active":"2023-11-26T07:21:49.375830+0000","last_peered":"2023-11-26T07:21:49.375830+0000","last_clean":"2023-11-26T07:21:49.375830+0000","last_became_active":"2023-11-26T07:21:48.395436+0000","last_became_peered":"2023-11-26T07:21:48.395436+0000","last_unstale":"2023-11-26T07:21:49.375830+0000","last_undegraded":"2023-11-26T07:21:49.375830+0000","last_fullsized":"2023-11-26T07:21:49.375830+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:19:14.549626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400366+0000","last_change":"2023-11-26T07:21:49.391773+0000","last_active":"2023-11-26T07:21:49.400366+0000","last_peered":"2023-11-26T07:21:49.400366+0000","last_clean":"2023-11-26T07:21:49.400366+0000","last_became_active":"2023-11-26T07:21:49.391519+0000","last_became_peered":"2023-11-26T07:21:49.391519+0000","last_unstale":"2023-11-26T07:21:49.400366+0000","last_undegraded":"2023-11-26T07:21:49.400366+0000","last_fullsized":"2023-11-26T07:21:49.400366+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:00:05.668971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.377153+0000","last_change":"2023-11-26T07:21:48.387268+0000","last_active":"2023-11-26T07:21:49.377153+0000","last_peered":"2023-11-26T07:21:49.377153+0000","last_clean":"2023-11-26T07:21:49.377153+0000","last_became_active":"2023-11-26T07:21:48.386321+0000","last_became_peered":"2023-11-26T07:21:48.386321+0000","last_unstale":"2023-11-26T07:21:49.377153+0000","last_undegraded":"2023-11-26T07:21:49.377153+0000","last_fullsized":"2023-11-26T07:21:49.377153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:24:29.883454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.396612+0000","last_change":"2023-11-26T07:21:49.387981+0000","last_active":"2023-11-26T07:21:49.396612+0000","last_peered":"2023-11-26T07:21:49.396612+0000","last_clean":"2023-11-26T07:21:49.396612+0000","last_became_active":"2023-11-26T07:21:49.387676+0000","last_became_peered":"2023-11-26T07:21:49.387676+0000","last_unstale":"2023-11-26T07:21:49.396612+0000","last_undegraded":"2023-11-26T07:21:49.396612+0000","last_fullsized":"2023-11-26T07:21:49.396612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:01.134963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.414018+0000","last_change":"2023-11-26T07:21:48.387491+0000","last_active":"2023-11-26T07:21:49.414018+0000","last_peered":"2023-11-26T07:21:49.414018+0000","last_clean":"2023-11-26T07:21:49.414018+0000","last_became_active":"2023-11-26T07:21:48.386596+0000","last_became_peered":"2023-11-26T07:21:48.386596+0000","last_unstale":"2023-11-26T07:21:49.414018+0000","last_undegraded":"2023-11-26T07:21:49.414018+0000","last_fullsized":"2023-11-26T07:21:49.414018+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:55:22.496081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":203,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-26T07:21:48.388739+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:21:48.388739+0000","last_peered":"2023-11-26T07:21:48.388739+0000","last_clean":"2023-11-26T07:21:48.388739+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:21:48.388739+0000","last_undegraded":"2023-11-26T07:21:48.388739+0000","last_fullsized":"2023-11-26T07:21:48.388739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1417216,"data_stored":932888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.56599999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":0.995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84799999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61799999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.48499999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.92500000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71399999999999997},"min":{"1min":0.56899999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84499999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.58999999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92500000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.50600000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":1.016,"15min":1.016},"last":0.53100000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.85099999999999998,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":0.81899999999999995},"min":{"1min":0.68100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93500000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.71099999999999997},"min":{"1min":0.47199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73299999999999998,"15min":0.77500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95899999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.84899999999999998},"min":{"1min":0.58699999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91500000000000004,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.499,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93400000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.82999999999999996},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.012,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.84999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":341996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":0.86699999999999999,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.222000000000001,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":3.4020000000000001,"5min":1.246,"15min":0.91500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":54.737000000000002,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.499}]},{"osd":1,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.40400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.1880000000000002,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.6689999999999996,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.72499999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.439,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.8479999999999999,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.72499999999999998,"15min":0.68899999999999995},"min":{"1min":0.57099999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0170000000000003,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.76200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.81299999999999994,"15min":0.751},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.4870000000000001,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.601,"5min":0.94399999999999995,"15min":0.874},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":18.77,"5min":18.77,"15min":18.77},"last":0.81899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.53100000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.121,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.65300000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67400000000000004,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.77900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.71199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":819200,"data_stored":341994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.626,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.64400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.87,"5min":1.073,"15min":1.073},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.96499999999999997,"5min":1.036,"15min":1.036},"last":0.628}]},{"osd":2,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.012},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.49099999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79000000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.67500000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.71299999999999997},"min":{"1min":0.495,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.028,"5min":1.097,"15min":1.097},"last":0.752},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.88700000000000001,"5min":1.071,"15min":1.071},"last":0.69899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65200000000000002,"15min":0.68600000000000005},"min":{"1min":0.42699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.137,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.77000000000000002},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.0029999999999999,"5min":1.206,"15min":1.206},"last":0.73599999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.62,"15min":0.67000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93100000000000005,"5min":1.143,"15min":1.143},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.73199999999999998},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.1020000000000001,"5min":1.228,"15min":1.228},"last":0.71099999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0580000000000001,"5min":1.264,"15min":1.264},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.077,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.80500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":341974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84899999999999998,"5min":1.016,"15min":1.016},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93100000000000005,"5min":1.26,"15min":1.26},"last":0.72599999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90200000000000002,"5min":1.077,"15min":1.077},"last":0.871}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.302,"15min":1.302},"last":0.752},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.753},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":1.181,"15min":1.181},"last":0.83199999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.747},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97999999999999998,"5min":1.702,"15min":1.702},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.78700000000000003},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0409999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.79400000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.81100000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.72999999999999998},"min":{"1min":0.42199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97099999999999997,"5min":1.129,"15min":1.129},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.78800000000000003},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.012,"15min":1.012},"last":0.89800000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88300000000000001,"5min":1.016,"15min":1.016},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.81200000000000006},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.0980000000000001,"5min":1.242,"15min":1.242},"last":0.91300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":823296,"data_stored":342016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.68999999999999995,"5min":1.294,"15min":1.294},"last":0.57999999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":1.456,"15min":1.456},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.52900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.212,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.77800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2649999999999999,"5min":1.395,"15min":1.395},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1080000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.64700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.333,"15min":1.333},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.228,"5min":1.347,"15min":1.347},"last":0.749}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.397,"15min":1.397},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1859999999999999,"5min":1.363,"15min":1.363},"last":0.72899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":827392,"data_stored":343516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56000000000000005,"15min":0.64800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65800000000000003,"5min":3.931,"15min":3.931},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.71099999999999997},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.47199999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60199999999999998,"15min":0.69199999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95499999999999996,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.69299999999999995},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.7650000000000001,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.40100000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.751,"15min":0.90300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98899999999999999,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.85199999999999998,"15min":0.92900000000000005},"min":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1240000000000001,"5min":4.431,"15min":4.431},"last":0.873},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.274,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.71099999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.61199999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89400000000000002,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8069999999999999,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.77600000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.98399999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.222,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.79600000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.93899999999999995,"15min":0.995},"min":{"1min":0.621,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.4740000000000002,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.052,"5min":1.609,"15min":1.609},"last":0.88800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":344132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.57499999999999996,"5min":1.034,"15min":1.034},"last":0.378},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69999999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66900000000000004,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.499,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.34499999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95599999999999996,"5min":1.143,"15min":1.143},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.216,"15min":1.216},"last":0.69099999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.58999999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97399999999999998,"5min":1.262,"15min":1.262},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.998,"5min":1.54,"15min":1.54},"last":0.77800000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.61599999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.054,"5min":1.145,"15min":1.145},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.91600000000000004,"5min":1.333,"15min":1.333},"last":0.71599999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95699999999999996,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0189999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1404928,"data_stored":931253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.45300000000000001},"min":{"1min":0.29799999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.68300000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.48499999999999999,"15min":0.47799999999999998},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.78600000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39600000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79700000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.497}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.38400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83599999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.437}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.81499999999999995},"min":{"1min":0.59499999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.67800000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.88500000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.63400000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.874}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.64800000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.61899999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1180000000000001,"5min":1.123,"15min":1.123},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.83099999999999996,"15min":0.81899999999999995},"min":{"1min":0.69399999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.95399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:21:53.537 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:21:53.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f79142e3640 1 Processor -- start 2023-11-26T07:21:53.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f79142e3640 1 -- start start 2023-11-26T07:21:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f79142e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0d2080 0x7f790c168160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f79142e3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c0cefd0 con 0x7f790c0cead0 2023-11-26T07:21:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f79142e3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790c0cf2b0 con 0x7f790c15d7b0 2023-11-26T07:21:53.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f79142e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c0d17e0 con 0x7f790c0d2080 2023-11-26T07:21:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791187e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f790c0cead0 0x7f790c0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53848/0 (socket says 172.21.15.31:53848) 2023-11-26T07:21:53.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791187e640 1 -- 172.21.15.31:0/709298278 learned_addr learned my addr 172.21.15.31:0/709298278 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:53.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f7912880640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0d2080 0x7f790c168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015380488 0 0) 0x7f790c0cefd0 con 0x7f790c0cead0 2023-11-26T07:21:53.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78f8003530 con 0x7f790c0cead0 2023-11-26T07:21:53.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995044937 0 0) 0x7f790c0cf2b0 con 0x7f790c15d7b0 2023-11-26T07:21:53.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f790c0cefd0 con 0x7f790c15d7b0 2023-11-26T07:21:53.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2898626583 0 0) 0x7f78f8003530 con 0x7f790c0cead0 2023-11-26T07:21:53.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0d2080 msgr2=0x7f790c168160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:53.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0d2080 0x7f790c168160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 >> v1:172.21.15.97:6789/0 conn(0x7f790c15d7b0 legacy=0x7f790c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c06b3f0 con 0x7f790c0cead0 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f78fc001680 con 0x7f790c0cead0 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 >> v1:172.21.15.31:6789/0 conn(0x7f790c0cead0 legacy=0x7f790c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0d2080 0x7f78f8003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 0x7f78f80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f8004d10 0x7f78f8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c0d17e0 con 0x7f78f80040a0 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f800bba0 con 0x7f78f8004d10 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.717+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f800bd60 con 0x7f790c0d2080 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791187e640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 0x7f78f80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791207f640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0d2080 0x7f78f8003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f7912880640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f8004d10 0x7f78f8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791187e640 1 -- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0d2080 msgr2=0x7f78f8003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791187e640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0d2080 0x7f78f8003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791187e640 1 -- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f8004d10 msgr2=0x7f78f8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791187e640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78f8004d10 0x7f78f8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791187e640 1 -- 172.21.15.31:0/709298278 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f8003530 con 0x7f78f80040a0 2023-11-26T07:21:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791187e640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 0x7f78f80047d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f790c0cef00 tx=0x7f78fc01cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a506bf09a7b7cbb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0058e0 con 0x7f78f80040a0 2023-11-26T07:21:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc005a60 con 0x7f78f80040a0 2023-11-26T07:21:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.721+0000 7f791107d640 1 -- 172.21.15.31:0/709298278 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc005e50 con 0x7f78f80040a0 2023-11-26T07:21:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 -- 172.21.15.31:0/709298278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 msgr2=0x7f78f80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 --2- 172.21.15.31:0/709298278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 0x7f78f80047d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f790c0cef00 tx=0x7f78fc01cb50 comp rx=0 tx=0).stop 2023-11-26T07:21:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f7912880640 1 -- 172.21.15.31:0/709298278 reap_dead start 2023-11-26T07:21:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 -- 172.21.15.31:0/709298278 shutdown_connections 2023-11-26T07:21:53.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 -- 172.21.15.31:0/709298278 >> 172.21.15.31:0/709298278 conn(0x7f790c0d33b0 msgr2=0x7f790c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 -- 172.21.15.31:0/709298278 shutdown_connections 2023-11-26T07:21:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 -- 172.21.15.31:0/709298278 wait complete. 2023-11-26T07:21:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 Processor -- start 2023-11-26T07:21:53.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 -- start start 2023-11-26T07:21:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 0x7f790c0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f790c1550c0 0x7f790c155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0cf4f0 0x7f790c0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c156920 con 0x7f78f80040a0 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790c155a80 con 0x7f790c1550c0 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f79142e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f790c0ccfa0 con 0x7f790c0cf4f0 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f791207f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 0x7f790c0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f791207f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 0x7f790c0b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50472/0 (socket says 172.21.15.31:50472) 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f791207f640 1 -- 172.21.15.31:0/1173583538 learned_addr learned my addr 172.21.15.31:0/1173583538 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f791187e640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f790c1550c0 0x7f790c155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f7912880640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0cf4f0 0x7f790c0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f791207f640 1 -- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0cf4f0 msgr2=0x7f790c0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f791207f640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0cf4f0 0x7f790c0cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f791207f640 1 -- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f790c1550c0 msgr2=0x7f790c155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f791207f640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f790c1550c0 0x7f790c155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f791207f640 1 -- 172.21.15.31:0/1173583538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790c156d40 con 0x7f78f80040a0 2023-11-26T07:21:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.725+0000 7f791187e640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f790c1550c0 0x7f790c155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f791207f640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 0x7f790c0b0dc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f790800f7d0 tx=0x7f790800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f7902ffd640 1 -- 172.21.15.31:0/1173583538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790800fe80 con 0x7f78f80040a0 2023-11-26T07:21:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f7902ffd640 1 -- 172.21.15.31:0/1173583538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79080052e0 con 0x7f78f80040a0 2023-11-26T07:21:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f7902ffd640 1 -- 172.21.15.31:0/1173583538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f790800d910 con 0x7f78f80040a0 2023-11-26T07:21:53.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f79142e3640 1 -- 172.21.15.31:0/1173583538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790c155ca0 con 0x7f78f80040a0 2023-11-26T07:21:53.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f79142e3640 1 -- 172.21.15.31:0/1173583538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790c156260 con 0x7f78f80040a0 2023-11-26T07:21:53.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f79142e3640 1 -- 172.21.15.31:0/1173583538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790c156a60 con 0x7f78f80040a0 2023-11-26T07:21:53.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f7902ffd640 1 -- 172.21.15.31:0/1173583538 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f790800da70 con 0x7f78f80040a0 2023-11-26T07:21:53.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f7902ffd640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f00667d0 0x7f78f0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:53.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f7902ffd640 1 -- 172.21.15.31:0/1173583538 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79080842f0 con 0x7f78f80040a0 2023-11-26T07:21:53.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.729+0000 7f791187e640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f00667d0 0x7f78f0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:53.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.733+0000 7f791187e640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f00667d0 0x7f78f0068c90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0aa000 tx=0x7f78fc001a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:53.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.733+0000 7f7902ffd640 1 -- 172.21.15.31:0/1173583538 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f790c156a60 con 0x7f78f80040a0 2023-11-26T07:21:53.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.885+0000 7f79142e3640 1 -- 172.21.15.31:0/1173583538 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f790c156a60 con 0x7f78f00667d0 2023-11-26T07:21:53.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.893+0000 7f7902ffd640 1 -- 172.21.15.31:0/1173583538 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288737 (secure 0 0 0) 0x7f790c156a60 con 0x7f78f00667d0 2023-11-26T07:21:53.899 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:53.902 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:21:53.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 -- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f00667d0 msgr2=0x7f78f0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f00667d0 0x7f78f0068c90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f78fc0aa000 tx=0x7f78fc001a00 comp rx=0 tx=0).stop 2023-11-26T07:21:53.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 -- 172.21.15.31:0/1173583538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 msgr2=0x7f790c0b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:53.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 0x7f790c0b0dc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f790800f7d0 tx=0x7f790800fc90 comp rx=0 tx=0).stop 2023-11-26T07:21:53.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 -- 172.21.15.31:0/1173583538 shutdown_connections 2023-11-26T07:21:53.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f78f00667d0 0x7f78f0068c90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f790c0cf4f0 0x7f790c0cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f790c1550c0 0x7f790c155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 --2- 172.21.15.31:0/1173583538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78f80040a0 0x7f790c0b0dc0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:53.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 -- 172.21.15.31:0/1173583538 >> 172.21.15.31:0/1173583538 conn(0x7f790c0d33b0 msgr2=0x7f790c0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 -- 172.21.15.31:0/1173583538 shutdown_connections 2023-11-26T07:21:53.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:53.897+0000 7f79142e3640 1 -- 172.21.15.31:0/1173583538 wait complete. 2023-11-26T07:21:53.921 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2023-11-26T07:21:52.371614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257052,"kb_used_data":7588,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717668,"statfs":{"total":773094113280,"available":772830892032,"internally_reserved":0,"allocated":7770112,"data_stored":3919769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.004499"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403752+0000","last_change":"2023-11-26T07:21:49.396807+0000","last_active":"2023-11-26T07:21:49.403752+0000","last_peered":"2023-11-26T07:21:49.403752+0000","last_clean":"2023-11-26T07:21:49.403752+0000","last_became_active":"2023-11-26T07:21:49.396478+0000","last_became_peered":"2023-11-26T07:21:49.396478+0000","last_unstale":"2023-11-26T07:21:49.403752+0000","last_undegraded":"2023-11-26T07:21:49.403752+0000","last_fullsized":"2023-11-26T07:21:49.403752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:50.985519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.395252+0000","last_change":"2023-11-26T07:21:49.385627+0000","last_active":"2023-11-26T07:21:49.395252+0000","last_peered":"2023-11-26T07:21:49.395252+0000","last_clean":"2023-11-26T07:21:49.395252+0000","last_became_active":"2023-11-26T07:21:49.381419+0000","last_became_peered":"2023-11-26T07:21:49.381419+0000","last_unstale":"2023-11-26T07:21:49.395252+0000","last_undegraded":"2023-11-26T07:21:49.395252+0000","last_fullsized":"2023-11-26T07:21:49.395252+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:44.940845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.399209+0000","last_change":"2023-11-26T07:21:49.388149+0000","last_active":"2023-11-26T07:21:49.399209+0000","last_peered":"2023-11-26T07:21:49.399209+0000","last_clean":"2023-11-26T07:21:49.399209+0000","last_became_active":"2023-11-26T07:21:49.387843+0000","last_became_peered":"2023-11-26T07:21:49.387843+0000","last_unstale":"2023-11-26T07:21:49.399209+0000","last_undegraded":"2023-11-26T07:21:49.399209+0000","last_fullsized":"2023-11-26T07:21:49.399209+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:02.563169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400729+0000","last_change":"2023-11-26T07:21:49.396963+0000","last_active":"2023-11-26T07:21:49.400729+0000","last_peered":"2023-11-26T07:21:49.400729+0000","last_clean":"2023-11-26T07:21:49.400729+0000","last_became_active":"2023-11-26T07:21:49.396682+0000","last_became_peered":"2023-11-26T07:21:49.396682+0000","last_unstale":"2023-11-26T07:21:49.400729+0000","last_undegraded":"2023-11-26T07:21:49.400729+0000","last_fullsized":"2023-11-26T07:21:49.400729+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:53:24.881150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.401950+0000","last_change":"2023-11-26T07:21:49.397512+0000","last_active":"2023-11-26T07:21:49.401950+0000","last_peered":"2023-11-26T07:21:49.401950+0000","last_clean":"2023-11-26T07:21:49.401950+0000","last_became_active":"2023-11-26T07:21:49.397131+0000","last_became_peered":"2023-11-26T07:21:49.397131+0000","last_unstale":"2023-11-26T07:21:49.401950+0000","last_undegraded":"2023-11-26T07:21:49.401950+0000","last_fullsized":"2023-11-26T07:21:49.401950+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:28:48.891145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.384035+0000","last_change":"2023-11-26T07:21:49.382277+0000","last_active":"2023-11-26T07:21:49.384035+0000","last_peered":"2023-11-26T07:21:49.384035+0000","last_clean":"2023-11-26T07:21:49.384035+0000","last_became_active":"2023-11-26T07:21:49.381517+0000","last_became_peered":"2023-11-26T07:21:49.381517+0000","last_unstale":"2023-11-26T07:21:49.384035+0000","last_undegraded":"2023-11-26T07:21:49.384035+0000","last_fullsized":"2023-11-26T07:21:49.384035+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:51:43.081093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.404449+0000","last_change":"2023-11-26T07:21:49.400222+0000","last_active":"2023-11-26T07:21:49.404449+0000","last_peered":"2023-11-26T07:21:49.404449+0000","last_clean":"2023-11-26T07:21:49.404449+0000","last_became_active":"2023-11-26T07:21:49.399942+0000","last_became_peered":"2023-11-26T07:21:49.399942+0000","last_unstale":"2023-11-26T07:21:49.404449+0000","last_undegraded":"2023-11-26T07:21:49.404449+0000","last_fullsized":"2023-11-26T07:21:49.404449+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:41.096161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.397083+0000","last_change":"2023-11-26T07:21:49.389216+0000","last_active":"2023-11-26T07:21:49.397083+0000","last_peered":"2023-11-26T07:21:49.397083+0000","last_clean":"2023-11-26T07:21:49.397083+0000","last_became_active":"2023-11-26T07:21:49.388920+0000","last_became_peered":"2023-11-26T07:21:49.388920+0000","last_unstale":"2023-11-26T07:21:49.397083+0000","last_undegraded":"2023-11-26T07:21:49.397083+0000","last_fullsized":"2023-11-26T07:21:49.397083+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:21.895779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.402437+0000","last_change":"2023-11-26T07:21:49.397940+0000","last_active":"2023-11-26T07:21:49.402437+0000","last_peered":"2023-11-26T07:21:49.402437+0000","last_clean":"2023-11-26T07:21:49.402437+0000","last_became_active":"2023-11-26T07:21:49.397426+0000","last_became_peered":"2023-11-26T07:21:49.397426+0000","last_unstale":"2023-11-26T07:21:49.402437+0000","last_undegraded":"2023-11-26T07:21:49.402437+0000","last_fullsized":"2023-11-26T07:21:49.402437+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:10:22.554778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.392989+0000","last_change":"2023-11-26T07:21:49.387190+0000","last_active":"2023-11-26T07:21:49.392989+0000","last_peered":"2023-11-26T07:21:49.392989+0000","last_clean":"2023-11-26T07:21:49.392989+0000","last_became_active":"2023-11-26T07:21:49.386912+0000","last_became_peered":"2023-11-26T07:21:49.386912+0000","last_unstale":"2023-11-26T07:21:49.392989+0000","last_undegraded":"2023-11-26T07:21:49.392989+0000","last_fullsized":"2023-11-26T07:21:49.392989+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:40.570012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.411253+0000","last_change":"2023-11-26T07:21:49.403218+0000","last_active":"2023-11-26T07:21:49.411253+0000","last_peered":"2023-11-26T07:21:49.411253+0000","last_clean":"2023-11-26T07:21:49.411253+0000","last_became_active":"2023-11-26T07:21:49.402211+0000","last_became_peered":"2023-11-26T07:21:49.402211+0000","last_unstale":"2023-11-26T07:21:49.411253+0000","last_undegraded":"2023-11-26T07:21:49.411253+0000","last_fullsized":"2023-11-26T07:21:49.411253+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:32:33.115731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408081+0000","last_change":"2023-11-26T07:21:49.401167+0000","last_active":"2023-11-26T07:21:49.408081+0000","last_peered":"2023-11-26T07:21:49.408081+0000","last_clean":"2023-11-26T07:21:49.408081+0000","last_became_active":"2023-11-26T07:21:49.399604+0000","last_became_peered":"2023-11-26T07:21:49.399604+0000","last_unstale":"2023-11-26T07:21:49.408081+0000","last_undegraded":"2023-11-26T07:21:49.408081+0000","last_fullsized":"2023-11-26T07:21:49.408081+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:47.172012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.398331+0000","last_change":"2023-11-26T07:21:49.392234+0000","last_active":"2023-11-26T07:21:49.398331+0000","last_peered":"2023-11-26T07:21:49.398331+0000","last_clean":"2023-11-26T07:21:49.398331+0000","last_became_active":"2023-11-26T07:21:49.391905+0000","last_became_peered":"2023-11-26T07:21:49.391905+0000","last_unstale":"2023-11-26T07:21:49.398331+0000","last_undegraded":"2023-11-26T07:21:49.398331+0000","last_fullsized":"2023-11-26T07:21:49.398331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:15:43.249996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.395841+0000","last_change":"2023-11-26T07:21:49.388501+0000","last_active":"2023-11-26T07:21:49.395841+0000","last_peered":"2023-11-26T07:21:49.395841+0000","last_clean":"2023-11-26T07:21:49.395841+0000","last_became_active":"2023-11-26T07:21:49.388188+0000","last_became_peered":"2023-11-26T07:21:49.388188+0000","last_unstale":"2023-11-26T07:21:49.395841+0000","last_undegraded":"2023-11-26T07:21:49.395841+0000","last_fullsized":"2023-11-26T07:21:49.395841+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:37:10.533229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.397435+0000","last_change":"2023-11-26T07:21:49.391890+0000","last_active":"2023-11-26T07:21:49.397435+0000","last_peered":"2023-11-26T07:21:49.397435+0000","last_clean":"2023-11-26T07:21:49.397435+0000","last_became_active":"2023-11-26T07:21:49.391565+0000","last_became_peered":"2023-11-26T07:21:49.391565+0000","last_unstale":"2023-11-26T07:21:49.397435+0000","last_undegraded":"2023-11-26T07:21:49.397435+0000","last_fullsized":"2023-11-26T07:21:49.397435+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:34:44.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.392937+0000","last_change":"2023-11-26T07:21:49.382719+0000","last_active":"2023-11-26T07:21:49.392937+0000","last_peered":"2023-11-26T07:21:49.392937+0000","last_clean":"2023-11-26T07:21:49.392937+0000","last_became_active":"2023-11-26T07:21:49.382421+0000","last_became_peered":"2023-11-26T07:21:49.382421+0000","last_unstale":"2023-11-26T07:21:49.392937+0000","last_undegraded":"2023-11-26T07:21:49.392937+0000","last_fullsized":"2023-11-26T07:21:49.392937+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:48.862601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.407840+0000","last_change":"2023-11-26T07:21:49.403064+0000","last_active":"2023-11-26T07:21:49.407840+0000","last_peered":"2023-11-26T07:21:49.407840+0000","last_clean":"2023-11-26T07:21:49.407840+0000","last_became_active":"2023-11-26T07:21:49.402768+0000","last_became_peered":"2023-11-26T07:21:49.402768+0000","last_unstale":"2023-11-26T07:21:49.407840+0000","last_undegraded":"2023-11-26T07:21:49.407840+0000","last_fullsized":"2023-11-26T07:21:49.407840+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:52:24.065676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400461+0000","last_change":"2023-11-26T07:21:49.398692+0000","last_active":"2023-11-26T07:21:49.400461+0000","last_peered":"2023-11-26T07:21:49.400461+0000","last_clean":"2023-11-26T07:21:49.400461+0000","last_became_active":"2023-11-26T07:21:49.398331+0000","last_became_peered":"2023-11-26T07:21:49.398331+0000","last_unstale":"2023-11-26T07:21:49.400461+0000","last_undegraded":"2023-11-26T07:21:49.400461+0000","last_fullsized":"2023-11-26T07:21:49.400461+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:31:43.225955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.396693+0000","last_change":"2023-11-26T07:21:49.388716+0000","last_active":"2023-11-26T07:21:49.396693+0000","last_peered":"2023-11-26T07:21:49.396693+0000","last_clean":"2023-11-26T07:21:49.396693+0000","last_became_active":"2023-11-26T07:21:49.388182+0000","last_became_peered":"2023-11-26T07:21:49.388182+0000","last_unstale":"2023-11-26T07:21:49.396693+0000","last_undegraded":"2023-11-26T07:21:49.396693+0000","last_fullsized":"2023-11-26T07:21:49.396693+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:31:43.225955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.393284+0000","last_change":"2023-11-26T07:21:49.386891+0000","last_active":"2023-11-26T07:21:49.393284+0000","last_peered":"2023-11-26T07:21:49.393284+0000","last_clean":"2023-11-26T07:21:49.393284+0000","last_became_active":"2023-11-26T07:21:49.386563+0000","last_became_peered":"2023-11-26T07:21:49.386563+0000","last_unstale":"2023-11-26T07:21:49.393284+0000","last_undegraded":"2023-11-26T07:21:49.393284+0000","last_fullsized":"2023-11-26T07:21:49.393284+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:11.355579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.393090+0000","last_change":"2023-11-26T07:21:49.384756+0000","last_active":"2023-11-26T07:21:49.393090+0000","last_peered":"2023-11-26T07:21:49.393090+0000","last_clean":"2023-11-26T07:21:49.393090+0000","last_became_active":"2023-11-26T07:21:49.384336+0000","last_became_peered":"2023-11-26T07:21:49.384336+0000","last_unstale":"2023-11-26T07:21:49.393090+0000","last_undegraded":"2023-11-26T07:21:49.393090+0000","last_fullsized":"2023-11-26T07:21:49.393090+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:00:05.668971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.399425+0000","last_change":"2023-11-26T07:21:49.388554+0000","last_active":"2023-11-26T07:21:49.399425+0000","last_peered":"2023-11-26T07:21:49.399425+0000","last_clean":"2023-11-26T07:21:49.399425+0000","last_became_active":"2023-11-26T07:21:49.388207+0000","last_became_peered":"2023-11-26T07:21:49.388207+0000","last_unstale":"2023-11-26T07:21:49.399425+0000","last_undegraded":"2023-11-26T07:21:49.399425+0000","last_fullsized":"2023-11-26T07:21:49.399425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:43.608330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.399025+0000","last_change":"2023-11-26T07:21:49.392681+0000","last_active":"2023-11-26T07:21:49.399025+0000","last_peered":"2023-11-26T07:21:49.399025+0000","last_clean":"2023-11-26T07:21:49.399025+0000","last_became_active":"2023-11-26T07:21:49.392410+0000","last_became_peered":"2023-11-26T07:21:49.392410+0000","last_unstale":"2023-11-26T07:21:49.399025+0000","last_undegraded":"2023-11-26T07:21:49.399025+0000","last_fullsized":"2023-11-26T07:21:49.399025+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:41.096161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.411027+0000","last_change":"2023-11-26T07:21:49.403067+0000","last_active":"2023-11-26T07:21:49.411027+0000","last_peered":"2023-11-26T07:21:49.411027+0000","last_clean":"2023-11-26T07:21:49.411027+0000","last_became_active":"2023-11-26T07:21:49.402635+0000","last_became_peered":"2023-11-26T07:21:49.402635+0000","last_unstale":"2023-11-26T07:21:49.411027+0000","last_undegraded":"2023-11-26T07:21:49.411027+0000","last_fullsized":"2023-11-26T07:21:49.411027+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:54:00.928815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400165+0000","last_change":"2023-11-26T07:21:49.394488+0000","last_active":"2023-11-26T07:21:49.400165+0000","last_peered":"2023-11-26T07:21:49.400165+0000","last_clean":"2023-11-26T07:21:49.400165+0000","last_became_active":"2023-11-26T07:21:49.394162+0000","last_became_peered":"2023-11-26T07:21:49.394162+0000","last_unstale":"2023-11-26T07:21:49.400165+0000","last_undegraded":"2023-11-26T07:21:49.400165+0000","last_fullsized":"2023-11-26T07:21:49.400165+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:41.370780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403104+0000","last_change":"2023-11-26T07:21:49.396545+0000","last_active":"2023-11-26T07:21:49.403104+0000","last_peered":"2023-11-26T07:21:49.403104+0000","last_clean":"2023-11-26T07:21:49.403104+0000","last_became_active":"2023-11-26T07:21:49.396212+0000","last_became_peered":"2023-11-26T07:21:49.396212+0000","last_unstale":"2023-11-26T07:21:49.403104+0000","last_undegraded":"2023-11-26T07:21:49.403104+0000","last_fullsized":"2023-11-26T07:21:49.403104+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:35.979200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.401515+0000","last_change":"2023-11-26T07:21:49.396517+0000","last_active":"2023-11-26T07:21:49.401515+0000","last_peered":"2023-11-26T07:21:49.401515+0000","last_clean":"2023-11-26T07:21:49.401515+0000","last_became_active":"2023-11-26T07:21:49.396193+0000","last_became_peered":"2023-11-26T07:21:49.396193+0000","last_unstale":"2023-11-26T07:21:49.401515+0000","last_undegraded":"2023-11-26T07:21:49.401515+0000","last_fullsized":"2023-11-26T07:21:49.401515+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:59:52.804916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.404633+0000","last_change":"2023-11-26T07:21:49.400059+0000","last_active":"2023-11-26T07:21:49.404633+0000","last_peered":"2023-11-26T07:21:49.404633+0000","last_clean":"2023-11-26T07:21:49.404633+0000","last_became_active":"2023-11-26T07:21:49.399750+0000","last_became_peered":"2023-11-26T07:21:49.399750+0000","last_unstale":"2023-11-26T07:21:49.404633+0000","last_undegraded":"2023-11-26T07:21:49.404633+0000","last_fullsized":"2023-11-26T07:21:49.404633+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:14:05.777652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.401410+0000","last_change":"2023-11-26T07:21:49.388013+0000","last_active":"2023-11-26T07:21:49.401410+0000","last_peered":"2023-11-26T07:21:49.401410+0000","last_clean":"2023-11-26T07:21:49.401410+0000","last_became_active":"2023-11-26T07:21:49.387712+0000","last_became_peered":"2023-11-26T07:21:49.387712+0000","last_unstale":"2023-11-26T07:21:49.401410+0000","last_undegraded":"2023-11-26T07:21:49.401410+0000","last_fullsized":"2023-11-26T07:21:49.401410+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:53:12.402635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408050+0000","last_change":"2023-11-26T07:21:49.402177+0000","last_active":"2023-11-26T07:21:49.408050+0000","last_peered":"2023-11-26T07:21:49.408050+0000","last_clean":"2023-11-26T07:21:49.408050+0000","last_became_active":"2023-11-26T07:21:49.401727+0000","last_became_peered":"2023-11-26T07:21:49.401727+0000","last_unstale":"2023-11-26T07:21:49.408050+0000","last_undegraded":"2023-11-26T07:21:49.408050+0000","last_fullsized":"2023-11-26T07:21:49.408050+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:54:00.928815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403657+0000","last_change":"2023-11-26T07:21:49.391443+0000","last_active":"2023-11-26T07:21:49.403657+0000","last_peered":"2023-11-26T07:21:49.403657+0000","last_clean":"2023-11-26T07:21:49.403657+0000","last_became_active":"2023-11-26T07:21:49.391082+0000","last_became_peered":"2023-11-26T07:21:49.391082+0000","last_unstale":"2023-11-26T07:21:49.403657+0000","last_undegraded":"2023-11-26T07:21:49.403657+0000","last_fullsized":"2023-11-26T07:21:49.403657+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:34:44.760381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.395651+0000","last_change":"2023-11-26T07:21:49.387843+0000","last_active":"2023-11-26T07:21:49.395651+0000","last_peered":"2023-11-26T07:21:49.395651+0000","last_clean":"2023-11-26T07:21:49.395651+0000","last_became_active":"2023-11-26T07:21:49.386986+0000","last_became_peered":"2023-11-26T07:21:49.386986+0000","last_unstale":"2023-11-26T07:21:49.395651+0000","last_undegraded":"2023-11-26T07:21:49.395651+0000","last_fullsized":"2023-11-26T07:21:49.395651+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:18:47.172012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.399080+0000","last_change":"2023-11-26T07:21:49.393020+0000","last_active":"2023-11-26T07:21:49.399080+0000","last_peered":"2023-11-26T07:21:49.399080+0000","last_clean":"2023-11-26T07:21:49.399080+0000","last_became_active":"2023-11-26T07:21:49.391427+0000","last_became_peered":"2023-11-26T07:21:49.391427+0000","last_unstale":"2023-11-26T07:21:49.399080+0000","last_undegraded":"2023-11-26T07:21:49.399080+0000","last_fullsized":"2023-11-26T07:21:49.399080+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:11.355579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"67'4","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.415391+0000","last_change":"2023-11-26T07:21:48.398703+0000","last_active":"2023-11-26T07:21:49.415391+0000","last_peered":"2023-11-26T07:21:49.415391+0000","last_clean":"2023-11-26T07:21:49.415391+0000","last_became_active":"2023-11-26T07:21:48.398368+0000","last_became_peered":"2023-11-26T07:21:48.398368+0000","last_unstale":"2023-11-26T07:21:49.415391+0000","last_undegraded":"2023-11-26T07:21:49.415391+0000","last_fullsized":"2023-11-26T07:21:49.415391+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:42:26.965755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.402018+0000","last_change":"2023-11-26T07:21:49.397748+0000","last_active":"2023-11-26T07:21:49.402018+0000","last_peered":"2023-11-26T07:21:49.402018+0000","last_clean":"2023-11-26T07:21:49.402018+0000","last_became_active":"2023-11-26T07:21:49.396946+0000","last_became_peered":"2023-11-26T07:21:49.396946+0000","last_unstale":"2023-11-26T07:21:49.402018+0000","last_undegraded":"2023-11-26T07:21:49.402018+0000","last_fullsized":"2023-11-26T07:21:49.402018+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:56:41.517579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"67'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.412055+0000","last_change":"2023-11-26T07:21:48.394808+0000","last_active":"2023-11-26T07:21:49.412055+0000","last_peered":"2023-11-26T07:21:49.412055+0000","last_clean":"2023-11-26T07:21:49.412055+0000","last_became_active":"2023-11-26T07:21:48.394448+0000","last_became_peered":"2023-11-26T07:21:48.394448+0000","last_unstale":"2023-11-26T07:21:49.412055+0000","last_undegraded":"2023-11-26T07:21:49.412055+0000","last_fullsized":"2023-11-26T07:21:49.412055+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:49.973476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400352+0000","last_change":"2023-11-26T07:21:49.392982+0000","last_active":"2023-11-26T07:21:49.400352+0000","last_peered":"2023-11-26T07:21:49.400352+0000","last_clean":"2023-11-26T07:21:49.400352+0000","last_became_active":"2023-11-26T07:21:49.392682+0000","last_became_peered":"2023-11-26T07:21:49.392682+0000","last_unstale":"2023-11-26T07:21:49.400352+0000","last_undegraded":"2023-11-26T07:21:49.400352+0000","last_fullsized":"2023-11-26T07:21:49.400352+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:53:24.881150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.417326+0000","last_change":"2023-11-26T07:21:48.394872+0000","last_active":"2023-11-26T07:21:49.417326+0000","last_peered":"2023-11-26T07:21:49.417326+0000","last_clean":"2023-11-26T07:21:49.417326+0000","last_became_active":"2023-11-26T07:21:48.394493+0000","last_became_peered":"2023-11-26T07:21:48.394493+0000","last_unstale":"2023-11-26T07:21:49.417326+0000","last_undegraded":"2023-11-26T07:21:49.417326+0000","last_fullsized":"2023-11-26T07:21:49.417326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:38:58.619515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408637+0000","last_change":"2023-11-26T07:21:49.399480+0000","last_active":"2023-11-26T07:21:49.408637+0000","last_peered":"2023-11-26T07:21:49.408637+0000","last_clean":"2023-11-26T07:21:49.408637+0000","last_became_active":"2023-11-26T07:21:49.399154+0000","last_became_peered":"2023-11-26T07:21:49.399154+0000","last_unstale":"2023-11-26T07:21:49.408637+0000","last_undegraded":"2023-11-26T07:21:49.408637+0000","last_fullsized":"2023-11-26T07:21:49.408637+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:58:27.502782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.413847+0000","last_change":"2023-11-26T07:21:48.385903+0000","last_active":"2023-11-26T07:21:49.413847+0000","last_peered":"2023-11-26T07:21:49.413847+0000","last_clean":"2023-11-26T07:21:49.413847+0000","last_became_active":"2023-11-26T07:21:48.385507+0000","last_became_peered":"2023-11-26T07:21:48.385507+0000","last_unstale":"2023-11-26T07:21:49.413847+0000","last_undegraded":"2023-11-26T07:21:49.413847+0000","last_fullsized":"2023-11-26T07:21:49.413847+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:47.850557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.404445+0000","last_change":"2023-11-26T07:21:49.397875+0000","last_active":"2023-11-26T07:21:49.404445+0000","last_peered":"2023-11-26T07:21:49.404445+0000","last_clean":"2023-11-26T07:21:49.404445+0000","last_became_active":"2023-11-26T07:21:49.397545+0000","last_became_peered":"2023-11-26T07:21:49.397545+0000","last_unstale":"2023-11-26T07:21:49.404445+0000","last_undegraded":"2023-11-26T07:21:49.404445+0000","last_fullsized":"2023-11-26T07:21:49.404445+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:11.355579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"67'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.414555+0000","last_change":"2023-11-26T07:21:48.398816+0000","last_active":"2023-11-26T07:21:49.414555+0000","last_peered":"2023-11-26T07:21:49.414555+0000","last_clean":"2023-11-26T07:21:49.414555+0000","last_became_active":"2023-11-26T07:21:48.398412+0000","last_became_peered":"2023-11-26T07:21:48.398412+0000","last_unstale":"2023-11-26T07:21:49.414555+0000","last_undegraded":"2023-11-26T07:21:49.414555+0000","last_fullsized":"2023-11-26T07:21:49.414555+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:49.166458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400385+0000","last_change":"2023-11-26T07:21:49.392972+0000","last_active":"2023-11-26T07:21:49.400385+0000","last_peered":"2023-11-26T07:21:49.400385+0000","last_clean":"2023-11-26T07:21:49.400385+0000","last_became_active":"2023-11-26T07:21:49.392682+0000","last_became_peered":"2023-11-26T07:21:49.392682+0000","last_unstale":"2023-11-26T07:21:49.400385+0000","last_undegraded":"2023-11-26T07:21:49.400385+0000","last_fullsized":"2023-11-26T07:21:49.400385+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:55:23.508125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.413341+0000","last_change":"2023-11-26T07:21:48.392436+0000","last_active":"2023-11-26T07:21:49.413341+0000","last_peered":"2023-11-26T07:21:49.413341+0000","last_clean":"2023-11-26T07:21:49.413341+0000","last_became_active":"2023-11-26T07:21:48.391047+0000","last_became_peered":"2023-11-26T07:21:48.391047+0000","last_unstale":"2023-11-26T07:21:49.413341+0000","last_undegraded":"2023-11-26T07:21:49.413341+0000","last_fullsized":"2023-11-26T07:21:49.413341+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:58:42.596286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.401746+0000","last_change":"2023-11-26T07:21:49.394754+0000","last_active":"2023-11-26T07:21:49.401746+0000","last_peered":"2023-11-26T07:21:49.401746+0000","last_clean":"2023-11-26T07:21:49.401746+0000","last_became_active":"2023-11-26T07:21:49.394486+0000","last_became_peered":"2023-11-26T07:21:49.394486+0000","last_unstale":"2023-11-26T07:21:49.401746+0000","last_undegraded":"2023-11-26T07:21:49.401746+0000","last_fullsized":"2023-11-26T07:21:49.401746+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:27:27.588608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.387669+0000","last_change":"2023-11-26T07:21:48.393086+0000","last_active":"2023-11-26T07:21:49.387669+0000","last_peered":"2023-11-26T07:21:49.387669+0000","last_clean":"2023-11-26T07:21:49.387669+0000","last_became_active":"2023-11-26T07:21:48.392743+0000","last_became_peered":"2023-11-26T07:21:48.392743+0000","last_unstale":"2023-11-26T07:21:49.387669+0000","last_undegraded":"2023-11-26T07:21:49.387669+0000","last_fullsized":"2023-11-26T07:21:49.387669+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:50.658713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.398425+0000","last_change":"2023-11-26T07:21:49.393868+0000","last_active":"2023-11-26T07:21:49.398425+0000","last_peered":"2023-11-26T07:21:49.398425+0000","last_clean":"2023-11-26T07:21:49.398425+0000","last_became_active":"2023-11-26T07:21:49.393597+0000","last_became_peered":"2023-11-26T07:21:49.393597+0000","last_unstale":"2023-11-26T07:21:49.398425+0000","last_undegraded":"2023-11-26T07:21:49.398425+0000","last_fullsized":"2023-11-26T07:21:49.398425+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:23.963334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.379572+0000","last_change":"2023-11-26T07:21:48.381170+0000","last_active":"2023-11-26T07:21:49.379572+0000","last_peered":"2023-11-26T07:21:49.379572+0000","last_clean":"2023-11-26T07:21:49.379572+0000","last_became_active":"2023-11-26T07:21:48.380878+0000","last_became_peered":"2023-11-26T07:21:48.380878+0000","last_unstale":"2023-11-26T07:21:49.379572+0000","last_undegraded":"2023-11-26T07:21:49.379572+0000","last_fullsized":"2023-11-26T07:21:49.379572+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:32:55.075524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403084+0000","last_change":"2023-11-26T07:21:49.394081+0000","last_active":"2023-11-26T07:21:49.403084+0000","last_peered":"2023-11-26T07:21:49.403084+0000","last_clean":"2023-11-26T07:21:49.403084+0000","last_became_active":"2023-11-26T07:21:49.393766+0000","last_became_peered":"2023-11-26T07:21:49.393766+0000","last_unstale":"2023-11-26T07:21:49.403084+0000","last_undegraded":"2023-11-26T07:21:49.403084+0000","last_fullsized":"2023-11-26T07:21:49.403084+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:23.963334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.376564+0000","last_change":"2023-11-26T07:21:48.396547+0000","last_active":"2023-11-26T07:21:49.376564+0000","last_peered":"2023-11-26T07:21:49.376564+0000","last_clean":"2023-11-26T07:21:49.376564+0000","last_became_active":"2023-11-26T07:21:48.394575+0000","last_became_peered":"2023-11-26T07:21:48.394575+0000","last_unstale":"2023-11-26T07:21:49.376564+0000","last_undegraded":"2023-11-26T07:21:49.376564+0000","last_fullsized":"2023-11-26T07:21:49.376564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:07:00.249022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408887+0000","last_change":"2023-11-26T07:21:49.402868+0000","last_active":"2023-11-26T07:21:49.408887+0000","last_peered":"2023-11-26T07:21:49.408887+0000","last_clean":"2023-11-26T07:21:49.408887+0000","last_became_active":"2023-11-26T07:21:49.402556+0000","last_became_peered":"2023-11-26T07:21:49.402556+0000","last_unstale":"2023-11-26T07:21:49.408887+0000","last_undegraded":"2023-11-26T07:21:49.408887+0000","last_fullsized":"2023-11-26T07:21:49.408887+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:55:30.809651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.378200+0000","last_change":"2023-11-26T07:21:48.388629+0000","last_active":"2023-11-26T07:21:49.378200+0000","last_peered":"2023-11-26T07:21:49.378200+0000","last_clean":"2023-11-26T07:21:49.378200+0000","last_became_active":"2023-11-26T07:21:48.388348+0000","last_became_peered":"2023-11-26T07:21:48.388348+0000","last_unstale":"2023-11-26T07:21:49.378200+0000","last_undegraded":"2023-11-26T07:21:49.378200+0000","last_fullsized":"2023-11-26T07:21:49.378200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:44:39.557968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.391193+0000","last_change":"2023-11-26T07:21:49.383791+0000","last_active":"2023-11-26T07:21:49.391193+0000","last_peered":"2023-11-26T07:21:49.391193+0000","last_clean":"2023-11-26T07:21:49.391193+0000","last_became_active":"2023-11-26T07:21:49.383503+0000","last_became_peered":"2023-11-26T07:21:49.383503+0000","last_unstale":"2023-11-26T07:21:49.391193+0000","last_undegraded":"2023-11-26T07:21:49.391193+0000","last_fullsized":"2023-11-26T07:21:49.391193+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:18:51.670757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"67'3","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.416207+0000","last_change":"2023-11-26T07:21:48.393905+0000","last_active":"2023-11-26T07:21:49.416207+0000","last_peered":"2023-11-26T07:21:49.416207+0000","last_clean":"2023-11-26T07:21:49.416207+0000","last_became_active":"2023-11-26T07:21:48.393594+0000","last_became_peered":"2023-11-26T07:21:48.393594+0000","last_unstale":"2023-11-26T07:21:49.416207+0000","last_undegraded":"2023-11-26T07:21:49.416207+0000","last_fullsized":"2023-11-26T07:21:49.416207+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:04.690030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.405697+0000","last_change":"2023-11-26T07:21:49.398185+0000","last_active":"2023-11-26T07:21:49.405697+0000","last_peered":"2023-11-26T07:21:49.405697+0000","last_clean":"2023-11-26T07:21:49.405697+0000","last_became_active":"2023-11-26T07:21:49.397672+0000","last_became_peered":"2023-11-26T07:21:49.397672+0000","last_unstale":"2023-11-26T07:21:49.405697+0000","last_undegraded":"2023-11-26T07:21:49.405697+0000","last_fullsized":"2023-11-26T07:21:49.405697+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:52:23.156268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"67'6","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.419664+0000","last_change":"2023-11-26T07:21:48.387605+0000","last_active":"2023-11-26T07:21:49.419664+0000","last_peered":"2023-11-26T07:21:49.419664+0000","last_clean":"2023-11-26T07:21:49.419664+0000","last_became_active":"2023-11-26T07:21:48.386842+0000","last_became_peered":"2023-11-26T07:21:48.386842+0000","last_unstale":"2023-11-26T07:21:49.419664+0000","last_undegraded":"2023-11-26T07:21:49.419664+0000","last_fullsized":"2023-11-26T07:21:49.419664+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:04:34.750837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403784+0000","last_change":"2023-11-26T07:21:49.397364+0000","last_active":"2023-11-26T07:21:49.403784+0000","last_peered":"2023-11-26T07:21:49.403784+0000","last_clean":"2023-11-26T07:21:49.403784+0000","last_became_active":"2023-11-26T07:21:49.396515+0000","last_became_peered":"2023-11-26T07:21:49.396515+0000","last_unstale":"2023-11-26T07:21:49.403784+0000","last_undegraded":"2023-11-26T07:21:49.403784+0000","last_fullsized":"2023-11-26T07:21:49.403784+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:15:43.249996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.417812+0000","last_change":"2023-11-26T07:21:48.393070+0000","last_active":"2023-11-26T07:21:49.417812+0000","last_peered":"2023-11-26T07:21:49.417812+0000","last_clean":"2023-11-26T07:21:49.417812+0000","last_became_active":"2023-11-26T07:21:48.392740+0000","last_became_peered":"2023-11-26T07:21:48.392740+0000","last_unstale":"2023-11-26T07:21:49.417812+0000","last_undegraded":"2023-11-26T07:21:49.417812+0000","last_fullsized":"2023-11-26T07:21:49.417812+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:51:42.069049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.404413+0000","last_change":"2023-11-26T07:21:49.398053+0000","last_active":"2023-11-26T07:21:49.404413+0000","last_peered":"2023-11-26T07:21:49.404413+0000","last_clean":"2023-11-26T07:21:49.404413+0000","last_became_active":"2023-11-26T07:21:49.397697+0000","last_became_peered":"2023-11-26T07:21:49.397697+0000","last_unstale":"2023-11-26T07:21:49.404413+0000","last_undegraded":"2023-11-26T07:21:49.404413+0000","last_fullsized":"2023-11-26T07:21:49.404413+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:07:01.261066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.377591+0000","last_change":"2023-11-26T07:21:48.394119+0000","last_active":"2023-11-26T07:21:49.377591+0000","last_peered":"2023-11-26T07:21:49.377591+0000","last_clean":"2023-11-26T07:21:49.377591+0000","last_became_active":"2023-11-26T07:21:48.393783+0000","last_became_peered":"2023-11-26T07:21:48.393783+0000","last_unstale":"2023-11-26T07:21:49.377591+0000","last_undegraded":"2023-11-26T07:21:49.377591+0000","last_fullsized":"2023-11-26T07:21:49.377591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:14:04.765608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.394060+0000","last_change":"2023-11-26T07:21:49.389159+0000","last_active":"2023-11-26T07:21:49.394060+0000","last_peered":"2023-11-26T07:21:49.394060+0000","last_clean":"2023-11-26T07:21:49.394060+0000","last_became_active":"2023-11-26T07:21:49.388206+0000","last_became_peered":"2023-11-26T07:21:49.388206+0000","last_unstale":"2023-11-26T07:21:49.394060+0000","last_undegraded":"2023-11-26T07:21:49.394060+0000","last_fullsized":"2023-11-26T07:21:49.394060+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:41.096161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.382101+0000","last_change":"2023-11-26T07:21:48.392636+0000","last_active":"2023-11-26T07:21:49.382101+0000","last_peered":"2023-11-26T07:21:49.382101+0000","last_clean":"2023-11-26T07:21:49.382101+0000","last_became_active":"2023-11-26T07:21:48.392359+0000","last_became_peered":"2023-11-26T07:21:48.392359+0000","last_unstale":"2023-11-26T07:21:49.382101+0000","last_undegraded":"2023-11-26T07:21:49.382101+0000","last_fullsized":"2023-11-26T07:21:49.382101+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:24:39.610517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.392586+0000","last_change":"2023-11-26T07:21:49.387238+0000","last_active":"2023-11-26T07:21:49.392586+0000","last_peered":"2023-11-26T07:21:49.392586+0000","last_clean":"2023-11-26T07:21:49.392586+0000","last_became_active":"2023-11-26T07:21:49.385793+0000","last_became_peered":"2023-11-26T07:21:49.385793+0000","last_unstale":"2023-11-26T07:21:49.392586+0000","last_undegraded":"2023-11-26T07:21:49.392586+0000","last_fullsized":"2023-11-26T07:21:49.392586+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:06:52.285644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"67'8","reported_seq":19,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.418263+0000","last_change":"2023-11-26T07:21:48.396212+0000","last_active":"2023-11-26T07:21:49.418263+0000","last_peered":"2023-11-26T07:21:49.418263+0000","last_clean":"2023-11-26T07:21:49.418263+0000","last_became_active":"2023-11-26T07:21:48.395180+0000","last_became_peered":"2023-11-26T07:21:48.395180+0000","last_unstale":"2023-11-26T07:21:49.418263+0000","last_undegraded":"2023-11-26T07:21:49.418263+0000","last_fullsized":"2023-11-26T07:21:49.418263+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:23.373412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400648+0000","last_change":"2023-11-26T07:21:49.394799+0000","last_active":"2023-11-26T07:21:49.400648+0000","last_peered":"2023-11-26T07:21:49.400648+0000","last_clean":"2023-11-26T07:21:49.400648+0000","last_became_active":"2023-11-26T07:21:49.394528+0000","last_became_peered":"2023-11-26T07:21:49.394528+0000","last_unstale":"2023-11-26T07:21:49.400648+0000","last_undegraded":"2023-11-26T07:21:49.400648+0000","last_fullsized":"2023-11-26T07:21:49.400648+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:30:16.661796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.414586+0000","last_change":"2023-11-26T07:21:48.393181+0000","last_active":"2023-11-26T07:21:49.414586+0000","last_peered":"2023-11-26T07:21:49.414586+0000","last_clean":"2023-11-26T07:21:49.414586+0000","last_became_active":"2023-11-26T07:21:48.392844+0000","last_became_peered":"2023-11-26T07:21:48.392844+0000","last_unstale":"2023-11-26T07:21:49.414586+0000","last_undegraded":"2023-11-26T07:21:49.414586+0000","last_fullsized":"2023-11-26T07:21:49.414586+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:41:23.373412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408549+0000","last_change":"2023-11-26T07:21:49.399317+0000","last_active":"2023-11-26T07:21:49.408549+0000","last_peered":"2023-11-26T07:21:49.408549+0000","last_clean":"2023-11-26T07:21:49.408549+0000","last_became_active":"2023-11-26T07:21:49.399029+0000","last_became_peered":"2023-11-26T07:21:49.399029+0000","last_unstale":"2023-11-26T07:21:49.408549+0000","last_undegraded":"2023-11-26T07:21:49.408549+0000","last_fullsized":"2023-11-26T07:21:49.408549+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:31:14.154472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.414438+0000","last_change":"2023-11-26T07:21:48.395462+0000","last_active":"2023-11-26T07:21:49.414438+0000","last_peered":"2023-11-26T07:21:49.414438+0000","last_clean":"2023-11-26T07:21:49.414438+0000","last_became_active":"2023-11-26T07:21:48.395219+0000","last_became_peered":"2023-11-26T07:21:48.395219+0000","last_unstale":"2023-11-26T07:21:49.414438+0000","last_undegraded":"2023-11-26T07:21:49.414438+0000","last_fullsized":"2023-11-26T07:21:49.414438+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:46:34.967156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408102+0000","last_change":"2023-11-26T07:21:49.401344+0000","last_active":"2023-11-26T07:21:49.408102+0000","last_peered":"2023-11-26T07:21:49.408102+0000","last_clean":"2023-11-26T07:21:49.408102+0000","last_became_active":"2023-11-26T07:21:49.400912+0000","last_became_peered":"2023-11-26T07:21:49.400912+0000","last_unstale":"2023-11-26T07:21:49.408102+0000","last_undegraded":"2023-11-26T07:21:49.408102+0000","last_fullsized":"2023-11-26T07:21:49.408102+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:28:21.895779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408244+0000","last_change":"2023-11-26T07:21:48.392365+0000","last_active":"2023-11-26T07:21:49.408244+0000","last_peered":"2023-11-26T07:21:49.408244+0000","last_clean":"2023-11-26T07:21:49.408244+0000","last_became_active":"2023-11-26T07:21:48.392071+0000","last_became_peered":"2023-11-26T07:21:48.392071+0000","last_unstale":"2023-11-26T07:21:49.408244+0000","last_undegraded":"2023-11-26T07:21:49.408244+0000","last_fullsized":"2023-11-26T07:21:49.408244+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:30:40.358736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.405677+0000","last_change":"2023-11-26T07:21:49.397997+0000","last_active":"2023-11-26T07:21:49.405677+0000","last_peered":"2023-11-26T07:21:49.405677+0000","last_clean":"2023-11-26T07:21:49.405677+0000","last_became_active":"2023-11-26T07:21:49.397225+0000","last_became_peered":"2023-11-26T07:21:49.397225+0000","last_unstale":"2023-11-26T07:21:49.405677+0000","last_undegraded":"2023-11-26T07:21:49.405677+0000","last_fullsized":"2023-11-26T07:21:49.405677+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:05.170257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.386090+0000","last_change":"2023-11-26T07:21:48.398779+0000","last_active":"2023-11-26T07:21:49.386090+0000","last_peered":"2023-11-26T07:21:49.386090+0000","last_clean":"2023-11-26T07:21:49.386090+0000","last_became_active":"2023-11-26T07:21:48.398174+0000","last_became_peered":"2023-11-26T07:21:48.398174+0000","last_unstale":"2023-11-26T07:21:49.386090+0000","last_undegraded":"2023-11-26T07:21:49.386090+0000","last_fullsized":"2023-11-26T07:21:49.386090+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:24:39.610517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.401430+0000","last_change":"2023-11-26T07:21:49.395666+0000","last_active":"2023-11-26T07:21:49.401430+0000","last_peered":"2023-11-26T07:21:49.401430+0000","last_clean":"2023-11-26T07:21:49.401430+0000","last_became_active":"2023-11-26T07:21:49.395321+0000","last_became_peered":"2023-11-26T07:21:49.395321+0000","last_unstale":"2023-11-26T07:21:49.401430+0000","last_undegraded":"2023-11-26T07:21:49.401430+0000","last_fullsized":"2023-11-26T07:21:49.401430+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:46:35.365406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.377733+0000","last_change":"2023-11-26T07:21:48.392226+0000","last_active":"2023-11-26T07:21:49.377733+0000","last_peered":"2023-11-26T07:21:49.377733+0000","last_clean":"2023-11-26T07:21:49.377733+0000","last_became_active":"2023-11-26T07:21:48.390698+0000","last_became_peered":"2023-11-26T07:21:48.390698+0000","last_unstale":"2023-11-26T07:21:49.377733+0000","last_undegraded":"2023-11-26T07:21:49.377733+0000","last_fullsized":"2023-11-26T07:21:49.377733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:30:15.649752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.396402+0000","last_change":"2023-11-26T07:21:49.386769+0000","last_active":"2023-11-26T07:21:49.396402+0000","last_peered":"2023-11-26T07:21:49.396402+0000","last_clean":"2023-11-26T07:21:49.396402+0000","last_became_active":"2023-11-26T07:21:49.386449+0000","last_became_peered":"2023-11-26T07:21:49.386449+0000","last_unstale":"2023-11-26T07:21:49.396402+0000","last_undegraded":"2023-11-26T07:21:49.396402+0000","last_fullsized":"2023-11-26T07:21:49.396402+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:16.783367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.384733+0000","last_change":"2023-11-26T07:21:48.390565+0000","last_active":"2023-11-26T07:21:49.384733+0000","last_peered":"2023-11-26T07:21:49.384733+0000","last_clean":"2023-11-26T07:21:49.384733+0000","last_became_active":"2023-11-26T07:21:48.390300+0000","last_became_peered":"2023-11-26T07:21:48.390300+0000","last_unstale":"2023-11-26T07:21:49.384733+0000","last_undegraded":"2023-11-26T07:21:49.384733+0000","last_fullsized":"2023-11-26T07:21:49.384733+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:26:49.973476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.396734+0000","last_change":"2023-11-26T07:21:49.387988+0000","last_active":"2023-11-26T07:21:49.396734+0000","last_peered":"2023-11-26T07:21:49.396734+0000","last_clean":"2023-11-26T07:21:49.396734+0000","last_became_active":"2023-11-26T07:21:49.386702+0000","last_became_peered":"2023-11-26T07:21:49.386702+0000","last_unstale":"2023-11-26T07:21:49.396734+0000","last_undegraded":"2023-11-26T07:21:49.396734+0000","last_fullsized":"2023-11-26T07:21:49.396734+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:08.928818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.380271+0000","last_change":"2023-11-26T07:21:48.393566+0000","last_active":"2023-11-26T07:21:49.380271+0000","last_peered":"2023-11-26T07:21:49.380271+0000","last_clean":"2023-11-26T07:21:49.380271+0000","last_became_active":"2023-11-26T07:21:48.393280+0000","last_became_peered":"2023-11-26T07:21:48.393280+0000","last_unstale":"2023-11-26T07:21:49.380271+0000","last_undegraded":"2023-11-26T07:21:49.380271+0000","last_fullsized":"2023-11-26T07:21:49.380271+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:33.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.398366+0000","last_change":"2023-11-26T07:21:49.392245+0000","last_active":"2023-11-26T07:21:49.398366+0000","last_peered":"2023-11-26T07:21:49.398366+0000","last_clean":"2023-11-26T07:21:49.398366+0000","last_became_active":"2023-11-26T07:21:49.391954+0000","last_became_peered":"2023-11-26T07:21:49.391954+0000","last_unstale":"2023-11-26T07:21:49.398366+0000","last_undegraded":"2023-11-26T07:21:49.398366+0000","last_fullsized":"2023-11-26T07:21:49.398366+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:54:00.928815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.413342+0000","last_change":"2023-11-26T07:21:48.392636+0000","last_active":"2023-11-26T07:21:49.413342+0000","last_peered":"2023-11-26T07:21:49.413342+0000","last_clean":"2023-11-26T07:21:49.413342+0000","last_became_active":"2023-11-26T07:21:48.392335+0000","last_became_peered":"2023-11-26T07:21:48.392335+0000","last_unstale":"2023-11-26T07:21:49.413342+0000","last_undegraded":"2023-11-26T07:21:49.413342+0000","last_fullsized":"2023-11-26T07:21:49.413342+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:19:14.549626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.403214+0000","last_change":"2023-11-26T07:21:49.396214+0000","last_active":"2023-11-26T07:21:49.403214+0000","last_peered":"2023-11-26T07:21:49.403214+0000","last_clean":"2023-11-26T07:21:49.403214+0000","last_became_active":"2023-11-26T07:21:49.395836+0000","last_became_peered":"2023-11-26T07:21:49.395836+0000","last_unstale":"2023-11-26T07:21:49.403214+0000","last_undegraded":"2023-11-26T07:21:49.403214+0000","last_fullsized":"2023-11-26T07:21:49.403214+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:15:43.249996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.381885+0000","last_change":"2023-11-26T07:21:48.392000+0000","last_active":"2023-11-26T07:21:49.381885+0000","last_peered":"2023-11-26T07:21:49.381885+0000","last_clean":"2023-11-26T07:21:49.381885+0000","last_became_active":"2023-11-26T07:21:48.391748+0000","last_became_peered":"2023-11-26T07:21:48.391748+0000","last_unstale":"2023-11-26T07:21:49.381885+0000","last_undegraded":"2023-11-26T07:21:49.381885+0000","last_fullsized":"2023-11-26T07:21:49.381885+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:33.295564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.408869+0000","last_change":"2023-11-26T07:21:49.400924+0000","last_active":"2023-11-26T07:21:49.408869+0000","last_peered":"2023-11-26T07:21:49.408869+0000","last_clean":"2023-11-26T07:21:49.408869+0000","last_became_active":"2023-11-26T07:21:49.400635+0000","last_became_peered":"2023-11-26T07:21:49.400635+0000","last_unstale":"2023-11-26T07:21:49.408869+0000","last_undegraded":"2023-11-26T07:21:49.408869+0000","last_fullsized":"2023-11-26T07:21:49.408869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:34.307607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.380870+0000","last_change":"2023-11-26T07:21:48.390952+0000","last_active":"2023-11-26T07:21:49.380870+0000","last_peered":"2023-11-26T07:21:49.380870+0000","last_clean":"2023-11-26T07:21:49.380870+0000","last_became_active":"2023-11-26T07:21:48.390649+0000","last_became_peered":"2023-11-26T07:21:48.390649+0000","last_unstale":"2023-11-26T07:21:49.380870+0000","last_undegraded":"2023-11-26T07:21:49.380870+0000","last_fullsized":"2023-11-26T07:21:49.380870+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:47:24.227453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.411276+0000","last_change":"2023-11-26T07:21:49.403406+0000","last_active":"2023-11-26T07:21:49.411276+0000","last_peered":"2023-11-26T07:21:49.411276+0000","last_clean":"2023-11-26T07:21:49.411276+0000","last_became_active":"2023-11-26T07:21:49.401830+0000","last_became_peered":"2023-11-26T07:21:49.401830+0000","last_unstale":"2023-11-26T07:21:49.411276+0000","last_undegraded":"2023-11-26T07:21:49.411276+0000","last_fullsized":"2023-11-26T07:21:49.411276+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:01.553446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.380658+0000","last_change":"2023-11-26T07:21:48.396049+0000","last_active":"2023-11-26T07:21:49.380658+0000","last_peered":"2023-11-26T07:21:49.380658+0000","last_clean":"2023-11-26T07:21:49.380658+0000","last_became_active":"2023-11-26T07:21:48.395756+0000","last_became_peered":"2023-11-26T07:21:48.395756+0000","last_unstale":"2023-11-26T07:21:49.380658+0000","last_undegraded":"2023-11-26T07:21:49.380658+0000","last_fullsized":"2023-11-26T07:21:49.380658+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:24:39.610517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.410477+0000","last_change":"2023-11-26T07:21:49.399941+0000","last_active":"2023-11-26T07:21:49.410477+0000","last_peered":"2023-11-26T07:21:49.410477+0000","last_clean":"2023-11-26T07:21:49.410477+0000","last_became_active":"2023-11-26T07:21:49.399610+0000","last_became_peered":"2023-11-26T07:21:49.399610+0000","last_unstale":"2023-11-26T07:21:49.410477+0000","last_undegraded":"2023-11-26T07:21:49.410477+0000","last_fullsized":"2023-11-26T07:21:49.410477+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:52:40.976535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.379828+0000","last_change":"2023-11-26T07:21:48.383780+0000","last_active":"2023-11-26T07:21:49.379828+0000","last_peered":"2023-11-26T07:21:49.379828+0000","last_clean":"2023-11-26T07:21:49.379828+0000","last_became_active":"2023-11-26T07:21:48.383535+0000","last_became_peered":"2023-11-26T07:21:48.383535+0000","last_unstale":"2023-11-26T07:21:49.379828+0000","last_undegraded":"2023-11-26T07:21:49.379828+0000","last_fullsized":"2023-11-26T07:21:49.379828+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:16:06.413411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400414+0000","last_change":"2023-11-26T07:21:49.393305+0000","last_active":"2023-11-26T07:21:49.400414+0000","last_peered":"2023-11-26T07:21:49.400414+0000","last_clean":"2023-11-26T07:21:49.400414+0000","last_became_active":"2023-11-26T07:21:49.393042+0000","last_became_peered":"2023-11-26T07:21:49.393042+0000","last_unstale":"2023-11-26T07:21:49.400414+0000","last_undegraded":"2023-11-26T07:21:49.400414+0000","last_fullsized":"2023-11-26T07:21:49.400414+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:04:35.762881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.388267+0000","last_change":"2023-11-26T07:21:48.391398+0000","last_active":"2023-11-26T07:21:49.388267+0000","last_peered":"2023-11-26T07:21:49.388267+0000","last_clean":"2023-11-26T07:21:49.388267+0000","last_became_active":"2023-11-26T07:21:48.391125+0000","last_became_peered":"2023-11-26T07:21:48.391125+0000","last_unstale":"2023-11-26T07:21:49.388267+0000","last_undegraded":"2023-11-26T07:21:49.388267+0000","last_fullsized":"2023-11-26T07:21:49.388267+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:07:43.928801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.405814+0000","last_change":"2023-11-26T07:21:49.400041+0000","last_active":"2023-11-26T07:21:49.405814+0000","last_peered":"2023-11-26T07:21:49.405814+0000","last_clean":"2023-11-26T07:21:49.405814+0000","last_became_active":"2023-11-26T07:21:49.399776+0000","last_became_peered":"2023-11-26T07:21:49.399776+0000","last_unstale":"2023-11-26T07:21:49.405814+0000","last_undegraded":"2023-11-26T07:21:49.405814+0000","last_fullsized":"2023-11-26T07:21:49.405814+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:18:58.253246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.375830+0000","last_change":"2023-11-26T07:21:48.395694+0000","last_active":"2023-11-26T07:21:49.375830+0000","last_peered":"2023-11-26T07:21:49.375830+0000","last_clean":"2023-11-26T07:21:49.375830+0000","last_became_active":"2023-11-26T07:21:48.395436+0000","last_became_peered":"2023-11-26T07:21:48.395436+0000","last_unstale":"2023-11-26T07:21:49.375830+0000","last_undegraded":"2023-11-26T07:21:49.375830+0000","last_fullsized":"2023-11-26T07:21:49.375830+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:19:14.549626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.400366+0000","last_change":"2023-11-26T07:21:49.391773+0000","last_active":"2023-11-26T07:21:49.400366+0000","last_peered":"2023-11-26T07:21:49.400366+0000","last_clean":"2023-11-26T07:21:49.400366+0000","last_became_active":"2023-11-26T07:21:49.391519+0000","last_became_peered":"2023-11-26T07:21:49.391519+0000","last_unstale":"2023-11-26T07:21:49.400366+0000","last_undegraded":"2023-11-26T07:21:49.400366+0000","last_fullsized":"2023-11-26T07:21:49.400366+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:00:05.668971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.377153+0000","last_change":"2023-11-26T07:21:48.387268+0000","last_active":"2023-11-26T07:21:49.377153+0000","last_peered":"2023-11-26T07:21:49.377153+0000","last_clean":"2023-11-26T07:21:49.377153+0000","last_became_active":"2023-11-26T07:21:48.386321+0000","last_became_peered":"2023-11-26T07:21:48.386321+0000","last_unstale":"2023-11-26T07:21:49.377153+0000","last_undegraded":"2023-11-26T07:21:49.377153+0000","last_fullsized":"2023-11-26T07:21:49.377153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:24:29.883454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.396612+0000","last_change":"2023-11-26T07:21:49.387981+0000","last_active":"2023-11-26T07:21:49.396612+0000","last_peered":"2023-11-26T07:21:49.396612+0000","last_clean":"2023-11-26T07:21:49.396612+0000","last_became_active":"2023-11-26T07:21:49.387676+0000","last_became_peered":"2023-11-26T07:21:49.387676+0000","last_unstale":"2023-11-26T07:21:49.396612+0000","last_undegraded":"2023-11-26T07:21:49.396612+0000","last_fullsized":"2023-11-26T07:21:49.396612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:48.363887+0000","last_clean_scrub_stamp":"2023-11-26T07:21:48.363887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:01.134963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-11-26T07:21:49.414018+0000","last_change":"2023-11-26T07:21:48.387491+0000","last_active":"2023-11-26T07:21:49.414018+0000","last_peered":"2023-11-26T07:21:49.414018+0000","last_clean":"2023-11-26T07:21:49.414018+0000","last_became_active":"2023-11-26T07:21:48.386596+0000","last_became_peered":"2023-11-26T07:21:48.386596+0000","last_unstale":"2023-11-26T07:21:49.414018+0000","last_undegraded":"2023-11-26T07:21:49.414018+0000","last_fullsized":"2023-11-26T07:21:49.414018+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:21:47.351844+0000","last_clean_scrub_stamp":"2023-11-26T07:21:47.351844+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:55:22.496081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":203,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-26T07:21:48.388739+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:21:48.388739+0000","last_peered":"2023-11-26T07:21:48.388739+0000","last_clean":"2023-11-26T07:21:48.388739+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:21:48.388739+0000","last_undegraded":"2023-11-26T07:21:48.388739+0000","last_fullsized":"2023-11-26T07:21:48.388739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1417216,"data_stored":932888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.56599999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":0.995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84799999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61799999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.48499999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.92500000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71399999999999997},"min":{"1min":0.56899999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84499999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.58999999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92500000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.50600000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":1.016,"15min":1.016},"last":0.53100000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.85099999999999998,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":0.81899999999999995},"min":{"1min":0.68100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93500000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.71199999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.71099999999999997},"min":{"1min":0.47199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73299999999999998,"15min":0.77500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95899999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.84899999999999998},"min":{"1min":0.58699999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91500000000000004,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.499,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93400000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.82999999999999996},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.012,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.84999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":341996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":0.86699999999999999,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.222000000000001,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":3.4020000000000001,"5min":1.246,"15min":0.91500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":54.737000000000002,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.499}]},{"osd":1,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.40400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.1880000000000002,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.6689999999999996,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.72499999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.439,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.8479999999999999,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.72499999999999998,"15min":0.68899999999999995},"min":{"1min":0.57099999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0170000000000003,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.76200000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.81299999999999994,"15min":0.751},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.4870000000000001,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.601,"5min":0.94399999999999995,"15min":0.874},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":18.77,"5min":18.77,"15min":18.77},"last":0.81899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.53100000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.121,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.65300000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67400000000000004,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.77900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.71199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574905,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":819200,"data_stored":341994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.626,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.64400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.58199999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.87,"5min":1.073,"15min":1.073},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.96499999999999997,"5min":1.036,"15min":1.036},"last":0.628}]},{"osd":2,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.012},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.49099999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79000000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.67500000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.71299999999999997},"min":{"1min":0.495,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.028,"5min":1.097,"15min":1.097},"last":0.752},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.88700000000000001,"5min":1.071,"15min":1.071},"last":0.69899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65200000000000002,"15min":0.68600000000000005},"min":{"1min":0.42699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.137,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.77000000000000002},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.0029999999999999,"5min":1.206,"15min":1.206},"last":0.73599999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.62,"15min":0.67000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93100000000000005,"5min":1.143,"15min":1.143},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.73199999999999998},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.1020000000000001,"5min":1.228,"15min":1.228},"last":0.71099999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0580000000000001,"5min":1.264,"15min":1.264},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.077,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.80500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":819200,"data_stored":341974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84899999999999998,"5min":1.016,"15min":1.016},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93100000000000005,"5min":1.26,"15min":1.26},"last":0.72599999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90200000000000002,"5min":1.077,"15min":1.077},"last":0.871}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.302,"15min":1.302},"last":0.752},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.753},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":1.181,"15min":1.181},"last":0.83199999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.747},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97999999999999998,"5min":1.702,"15min":1.702},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.78700000000000003},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0409999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.79400000000000004}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.81100000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.72999999999999998},"min":{"1min":0.42199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97099999999999997,"5min":1.129,"15min":1.129},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.78800000000000003},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.012,"15min":1.012},"last":0.89800000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88300000000000001,"5min":1.016,"15min":1.016},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.81200000000000006},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.0980000000000001,"5min":1.242,"15min":1.242},"last":0.91300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":823296,"data_stored":342016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.68999999999999995,"5min":1.294,"15min":1.294},"last":0.57999999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53000000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":1.456,"15min":1.456},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.52900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.212,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.77800000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2649999999999999,"5min":1.395,"15min":1.395},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1080000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.64700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.333,"15min":1.333},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.228,"5min":1.347,"15min":1.347},"last":0.749}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.397,"15min":1.397},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1859999999999999,"5min":1.363,"15min":1.363},"last":0.72899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":827392,"data_stored":343516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56000000000000005,"15min":0.64800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65800000000000003,"5min":3.931,"15min":3.931},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.71099999999999997},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.47199999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60199999999999998,"15min":0.69199999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95499999999999996,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.69299999999999995},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.7650000000000001,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.40100000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.751,"15min":0.90300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98899999999999999,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.85199999999999998,"15min":0.92900000000000005},"min":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1240000000000001,"5min":4.431,"15min":4.431},"last":0.873},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.274,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.71099999999999997}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.61199999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89400000000000002,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8069999999999999,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.77600000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.98399999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.222,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.79600000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.93899999999999995,"15min":0.995},"min":{"1min":0.621,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.4740000000000002,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.052,"5min":1.609,"15min":1.609},"last":0.88800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":835584,"data_stored":344132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.57499999999999996,"5min":1.034,"15min":1.034},"last":0.378},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69999999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66900000000000004,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.504,"5min":0.499,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.34499999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95599999999999996,"5min":1.143,"15min":1.143},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.216,"15min":1.216},"last":0.69099999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.58999999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97399999999999998,"5min":1.262,"15min":1.262},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.998,"5min":1.54,"15min":1.54},"last":0.77800000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.61599999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.054,"5min":1.145,"15min":1.145},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.91600000000000004,"5min":1.333,"15min":1.333},"last":0.71599999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95699999999999996,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0189999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32552,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339288,"statfs":{"total":96636764160,"available":96603430912,"internally_reserved":0,"allocated":1404928,"data_stored":931253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.45300000000000001},"min":{"1min":0.29799999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.68300000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.48499999999999999,"15min":0.47799999999999998},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.78600000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39600000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79700000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.497}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.38400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83599999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.437}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.81499999999999995},"min":{"1min":0.59499999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.67800000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.88500000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.63400000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.874}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.64800000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.61899999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.85299999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1180000000000001,"5min":1.123,"15min":1.123},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.83099999999999996,"15min":0.81899999999999995},"min":{"1min":0.69399999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.95399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:21:53.926 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:21:53.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:21:54.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1e92a640 1 Processor -- start 2023-11-26T07:21:54.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1e92a640 1 -- start start 2023-11-26T07:21:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1e92a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3f180cb9b0 0x7f3f1806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1e92a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f180ce5d0 0x7f3f1815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1e92a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f18159630 con 0x7f3f180cbd90 2023-11-26T07:21:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1e92a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1815bba0 con 0x7f3f180ce5d0 2023-11-26T07:21:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1e92a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f180cdd50 con 0x7f3f180cb9b0 2023-11-26T07:21:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1cec7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f180ce5d0 0x7f3f1815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1cec7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f180ce5d0 0x7f3f1815bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50482/0 (socket says 172.21.15.31:50482) 2023-11-26T07:21:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1cec7640 1 -- 172.21.15.31:0/543524491 learned_addr learned my addr 172.21.15.31:0/543524491 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1cec7640 1 -- 172.21.15.31:0/543524491 >> v2:172.21.15.97:3300/0 conn(0x7f3f180cb9b0 msgr2=0x7f3f1806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1cec7640 1 --2- 172.21.15.31:0/543524491 >> v2:172.21.15.97:3300/0 conn(0x7f3f180cb9b0 0x7f3f1806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1cec7640 1 -- 172.21.15.31:0/543524491 >> v1:172.21.15.97:6789/0 conn(0x7f3f180cbd90 legacy=0x7f3f1815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.109+0000 7f3f1cec7640 1 -- 172.21.15.31:0/543524491 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f180cd1d0 con 0x7f3f180ce5d0 2023-11-26T07:21:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f1cec7640 1 --2- 172.21.15.31:0/543524491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180ce5d0 0x7f3f1815bfd0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3f1806c360 tx=0x7f3f1001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a464c9c5186f9d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f16ffd640 1 -- 172.21.15.31:0/543524491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f100a5800 con 0x7f3f180ce5d0 2023-11-26T07:21:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f16ffd640 1 -- 172.21.15.31:0/543524491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f100a5960 con 0x7f3f180ce5d0 2023-11-26T07:21:54.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f16ffd640 1 -- 172.21.15.31:0/543524491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f100a5cf0 con 0x7f3f180ce5d0 2023-11-26T07:21:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f1e92a640 1 -- 172.21.15.31:0/543524491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180ce5d0 msgr2=0x7f3f1815bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f1e92a640 1 --2- 172.21.15.31:0/543524491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180ce5d0 0x7f3f1815bfd0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3f1806c360 tx=0x7f3f1001c960 comp rx=0 tx=0).stop 2023-11-26T07:21:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f1e92a640 1 -- 172.21.15.31:0/543524491 shutdown_connections 2023-11-26T07:21:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f1e92a640 1 --2- 172.21.15.31:0/543524491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180ce5d0 0x7f3f1815bfd0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f1e92a640 1 --2- 172.21.15.31:0/543524491 >> v2:172.21.15.97:3300/0 conn(0x7f3f180cb9b0 0x7f3f1806e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f1e92a640 1 -- 172.21.15.31:0/543524491 >> 172.21.15.31:0/543524491 conn(0x7f3f180b0cb0 msgr2=0x7f3f180b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f1e92a640 1 -- 172.21.15.31:0/543524491 shutdown_connections 2023-11-26T07:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.113+0000 7f3f1e92a640 1 -- 172.21.15.31:0/543524491 wait complete. 2023-11-26T07:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1e92a640 1 Processor -- start 2023-11-26T07:21:54.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1e92a640 1 -- start start 2023-11-26T07:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1e92a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180cb9b0 0x7f3f18153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1e92a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f180cbd90 0x7f3f18165220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1e92a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f180ce5d0 0x7f3f18152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1e92a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18165760 con 0x7f3f180cb9b0 2023-11-26T07:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1e92a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f181659e0 con 0x7f3f180ce5d0 2023-11-26T07:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1e92a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18165cc0 con 0x7f3f180cbd90 2023-11-26T07:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f17fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180cb9b0 0x7f3f18153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f17fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180cb9b0 0x7f3f18153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50496/0 (socket says 172.21.15.31:50496) 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f17fff640 1 -- 172.21.15.31:0/831808725 learned_addr learned my addr 172.21.15.31:0/831808725 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1cec7640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f180ce5d0 0x7f3f18152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f177fe640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f180cbd90 0x7f3f18165220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f17fff640 1 -- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f180cbd90 msgr2=0x7f3f18165220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f17fff640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f180cbd90 0x7f3f18165220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f17fff640 1 -- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f180ce5d0 msgr2=0x7f3f18152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f17fff640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f180ce5d0 0x7f3f18152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f17fff640 1 -- 172.21.15.31:0/831808725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f10009b30 con 0x7f3f180cb9b0 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1cec7640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f180ce5d0 0x7f3f18152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f17fff640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180cb9b0 0x7f3f18153640 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800b550 tx=0x7f3f0800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f14ff9640 1 -- 172.21.15.31:0/831808725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08010870 con 0x7f3f180cb9b0 2023-11-26T07:21:54.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f14ff9640 1 -- 172.21.15.31:0/831808725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f080109f0 con 0x7f3f180cb9b0 2023-11-26T07:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f14ff9640 1 -- 172.21.15.31:0/831808725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f08010da0 con 0x7f3f180cb9b0 2023-11-26T07:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1e92a640 1 -- 172.21.15.31:0/831808725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f18165fa0 con 0x7f3f180cb9b0 2023-11-26T07:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.117+0000 7f3f1e92a640 1 -- 172.21.15.31:0/831808725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f18166280 con 0x7f3f180cb9b0 2023-11-26T07:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.121+0000 7f3f1e92a640 1 -- 172.21.15.31:0/831808725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f180671f0 con 0x7f3f180cb9b0 2023-11-26T07:21:54.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.121+0000 7f3f14ff9640 1 -- 172.21.15.31:0/831808725 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f3f0800f3c0 con 0x7f3f180cb9b0 2023-11-26T07:21:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.121+0000 7f3f14ff9640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3f04066800 0x7f3f04068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.121+0000 7f3f14ff9640 1 -- 172.21.15.31:0/831808725 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f08084560 con 0x7f3f180cb9b0 2023-11-26T07:21:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.121+0000 7f3f177fe640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3f04066800 0x7f3f04068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.125+0000 7f3f177fe640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3f04066800 0x7f3f04068cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3f180b41b0 tx=0x7f3f0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:54.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.125+0000 7f3f14ff9640 1 -- 172.21.15.31:0/831808725 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3f08051440 con 0x7f3f180cb9b0 2023-11-26T07:21:54.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.337+0000 7f3f1e92a640 1 -- 172.21.15.31:0/831808725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3f1806b460 con 0x7f3f180cb9b0 2023-11-26T07:21:54.343 INFO:tasks.ceph.mds.a.smithi031.stderr:2023-11-26T07:21:54.341+0000 7f9f5f15e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:21:54.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.341+0000 7f3f14ff9640 1 -- 172.21.15.31:0/831808725 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3f080140b0 con 0x7f3f180cb9b0 2023-11-26T07:21:54.344 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-11-26T07:21:54.345 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA== 2023-11-26T07:21:54.345 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2023-11-26T07:21:54.345 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2023-11-26T07:21:54.345 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:21:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-11-26T07:21:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.341+0000 7f3f1e92a640 1 -- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3f04066800 msgr2=0x7f3f04068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.341+0000 7f3f1e92a640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3f04066800 0x7f3f04068cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3f180b41b0 tx=0x7f3f0c009040 comp rx=0 tx=0).stop 2023-11-26T07:21:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.345+0000 7f3f1e92a640 1 -- 172.21.15.31:0/831808725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180cb9b0 msgr2=0x7f3f18153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.345+0000 7f3f1e92a640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180cb9b0 0x7f3f18153640 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800b550 tx=0x7f3f0800ba10 comp rx=0 tx=0).stop 2023-11-26T07:21:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.345+0000 7f3f1e92a640 1 -- 172.21.15.31:0/831808725 shutdown_connections 2023-11-26T07:21:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.345+0000 7f3f1e92a640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3f04066800 0x7f3f04068cc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.345+0000 7f3f1e92a640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f180ce5d0 0x7f3f18152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.345+0000 7f3f1e92a640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3f180cbd90 0x7f3f18165220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.345+0000 7f3f1e92a640 1 --2- 172.21.15.31:0/831808725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f180cb9b0 0x7f3f18153640 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.345+0000 7f3f1e92a640 1 -- 172.21.15.31:0/831808725 >> 172.21.15.31:0/831808725 conn(0x7f3f180b0cb0 msgr2=0x7f3f1806e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.345+0000 7f3f1e92a640 1 -- 172.21.15.31:0/831808725 shutdown_connections 2023-11-26T07:21:54.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.345+0000 7f3f1e92a640 1 -- 172.21.15.31:0/831808725 wait complete. 2023-11-26T07:21:54.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:21:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.517+0000 7f6748806640 1 Processor -- start 2023-11-26T07:21:54.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.517+0000 7f6748806640 1 -- start start 2023-11-26T07:21:54.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.517+0000 7f6748806640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f67440ce5c0 0x7f67440cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.517+0000 7f6748806640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f674406a5b0 0x7f674406aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.517+0000 7f6748806640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67440b0a70 con 0x7f674415d8b0 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.517+0000 7f6748806640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67440cddb0 con 0x7f674406a5b0 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.517+0000 7f6748806640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67440b4100 con 0x7f67440ce5c0 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.517+0000 7f6742d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f674406a5b0 0x7f674406aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.517+0000 7f6742d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f674406a5b0 0x7f674406aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50502/0 (socket says 172.21.15.31:50502) 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.517+0000 7f6742d9d640 1 -- 172.21.15.31:0/3439045389 learned_addr learned my addr 172.21.15.31:0/3439045389 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f673bfff640 1 --2- 172.21.15.31:0/3439045389 >> v2:172.21.15.97:3300/0 conn(0x7f67440ce5c0 0x7f67440cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f6742d9d640 1 -- 172.21.15.31:0/3439045389 >> v2:172.21.15.97:3300/0 conn(0x7f67440ce5c0 msgr2=0x7f67440cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f6742d9d640 1 --2- 172.21.15.31:0/3439045389 >> v2:172.21.15.97:3300/0 conn(0x7f67440ce5c0 0x7f67440cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f6742d9d640 1 -- 172.21.15.31:0/3439045389 >> v1:172.21.15.97:6789/0 conn(0x7f674415d8b0 legacy=0x7f67440b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f6742d9d640 1 -- 172.21.15.31:0/3439045389 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440b43e0 con 0x7f674406a5b0 2023-11-26T07:21:54.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f6742d9d640 1 --2- 172.21.15.31:0/3439045389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674406a5b0 0x7f674406aa20 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f67440b1780 tx=0x7f673c01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ceb185e696f2f32 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:54.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f6741d9b640 1 -- 172.21.15.31:0/3439045389 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0af400 con 0x7f674406a5b0 2023-11-26T07:21:54.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f6741d9b640 1 -- 172.21.15.31:0/3439045389 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c0ab050 con 0x7f674406a5b0 2023-11-26T07:21:54.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f6741d9b640 1 -- 172.21.15.31:0/3439045389 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0af780 con 0x7f674406a5b0 2023-11-26T07:21:54.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f6748806640 1 -- 172.21.15.31:0/3439045389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674406a5b0 msgr2=0x7f674406aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.521+0000 7f6748806640 1 --2- 172.21.15.31:0/3439045389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674406a5b0 0x7f674406aa20 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f67440b1780 tx=0x7f673c01f1c0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 -- 172.21.15.31:0/3439045389 shutdown_connections 2023-11-26T07:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 --2- 172.21.15.31:0/3439045389 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f674406a5b0 0x7f674406aa20 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 --2- 172.21.15.31:0/3439045389 >> v2:172.21.15.97:3300/0 conn(0x7f67440ce5c0 0x7f67440cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 -- 172.21.15.31:0/3439045389 >> 172.21.15.31:0/3439045389 conn(0x7f67440b5570 msgr2=0x7f67440b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 -- 172.21.15.31:0/3439045389 shutdown_connections 2023-11-26T07:21:54.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 -- 172.21.15.31:0/3439045389 wait complete. 2023-11-26T07:21:54.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 Processor -- start 2023-11-26T07:21:54.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 -- start start 2023-11-26T07:21:54.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674406a5b0 0x7f674406f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67440ce5c0 0x7f67441470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674415d8b0 0x7f67440b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674415a0c0 con 0x7f67440ce5c0 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744148390 con 0x7f674406a5b0 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6748806640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6744147780 con 0x7f674415d8b0 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f673bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67440ce5c0 0x7f67441470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f673bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67440ce5c0 0x7f67441470c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50504/0 (socket says 172.21.15.31:50504) 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f673bfff640 1 -- 172.21.15.31:0/2429715648 learned_addr learned my addr 172.21.15.31:0/2429715648 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f674259c640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674406a5b0 0x7f674406f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f6742d9d640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674415d8b0 0x7f67440b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f673bfff640 1 -- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674415d8b0 msgr2=0x7f67440b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f673bfff640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674415d8b0 0x7f67440b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f673bfff640 1 -- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674406a5b0 msgr2=0x7f674406f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f673bfff640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674406a5b0 0x7f674406f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f673bfff640 1 -- 172.21.15.31:0/2429715648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c009b30 con 0x7f67440ce5c0 2023-11-26T07:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.525+0000 7f673bfff640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67440ce5c0 0x7f67441470c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f672c00d800 tx=0x7f672c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.529+0000 7f673affd640 1 -- 172.21.15.31:0/2429715648 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00b6c0 con 0x7f67440ce5c0 2023-11-26T07:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.529+0000 7f673affd640 1 -- 172.21.15.31:0/2429715648 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c00b820 con 0x7f67440ce5c0 2023-11-26T07:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.529+0000 7f6748806640 1 -- 172.21.15.31:0/2429715648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6744147a60 con 0x7f67440ce5c0 2023-11-26T07:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.529+0000 7f673affd640 1 -- 172.21.15.31:0/2429715648 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00bb90 con 0x7f67440ce5c0 2023-11-26T07:21:54.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.529+0000 7f6748806640 1 -- 172.21.15.31:0/2429715648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6744148020 con 0x7f67440ce5c0 2023-11-26T07:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.529+0000 7f6748806640 1 -- 172.21.15.31:0/2429715648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6744148540 con 0x7f67440ce5c0 2023-11-26T07:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.529+0000 7f673affd640 1 -- 172.21.15.31:0/2429715648 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f672c00bcf0 con 0x7f67440ce5c0 2023-11-26T07:21:54.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.529+0000 7f673affd640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67240667d0 0x7f6724068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:54.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.529+0000 7f673affd640 1 -- 172.21.15.31:0/2429715648 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f672c084750 con 0x7f67440ce5c0 2023-11-26T07:21:54.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.529+0000 7f674259c640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67240667d0 0x7f6724068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:54.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.533+0000 7f674259c640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67240667d0 0x7f6724068c90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6734007fd0 tx=0x7f6734009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:54.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.533+0000 7f673affd640 1 -- 172.21.15.31:0/2429715648 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6744148540 con 0x7f67440ce5c0 2023-11-26T07:21:54.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.725+0000 7f6748806640 1 -- 172.21.15.31:0/2429715648 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6744148540 con 0x7f67440ce5c0 2023-11-26T07:21:54.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.725+0000 7f673affd640 1 -- 172.21.15.31:0/2429715648 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f6744148540 con 0x7f67440ce5c0 2023-11-26T07:21:54.730 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:54.730 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2851182363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2851182363},{"type":"v1","addr":"172.21.15.31:6835","nonce":2851182363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2056572548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2056572548},{"type":"v1","addr":"172.21.15.97:6837","nonce":2056572548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1012539369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1012539369},{"type":"v1","addr":"172.21.15.97:6836","nonce":1012539369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:48.765793+0000","modified":"2023-11-26T07:21:51.814230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6597},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6597":{"gid":6597,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/242493352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":242493352},{"type":"v1","addr":"172.21.15.31:6834","nonce":242493352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:21:54.730 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 47 2023-11-26T07:21:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 -- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67240667d0 msgr2=0x7f6724068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67240667d0 0x7f6724068c90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6734007fd0 tx=0x7f6734009040 comp rx=0 tx=0).stop 2023-11-26T07:21:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 -- 172.21.15.31:0/2429715648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67440ce5c0 msgr2=0x7f67441470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67440ce5c0 0x7f67441470c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f672c00d800 tx=0x7f672c00dcc0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 -- 172.21.15.31:0/2429715648 shutdown_connections 2023-11-26T07:21:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f67240667d0 0x7f6724068c90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f674415d8b0 0x7f67440b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67440ce5c0 0x7f67441470c0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 --2- 172.21.15.31:0/2429715648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f674406a5b0 0x7f674406f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 -- 172.21.15.31:0/2429715648 >> 172.21.15.31:0/2429715648 conn(0x7f67440b5570 msgr2=0x7f674406e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 -- 172.21.15.31:0/2429715648 shutdown_connections 2023-11-26T07:21:54.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:54.729+0000 7f6748806640 1 -- 172.21.15.31:0/2429715648 wait complete. 2023-11-26T07:21:55.154 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:21:48.765793+0000', 'modified': '2023-11-26T07:21:51.814230+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6597}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6597': {'gid': 6597, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6834/242493352', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6832', 'nonce': 242493352}, {'type': 'v1', 'addr': '172.21.15.31:6834', 'nonce': 242493352}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:21:55.154 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:21:55.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-26T07:21:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.297+0000 7f4da3aea640 1 Processor -- start 2023-11-26T07:21:55.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da3aea640 1 -- start start 2023-11-26T07:21:55.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da3aea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d9c15e690 0x7f4d9c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da3aea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d9c06d190 0x7f4d9c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da3aea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c0af630 con 0x7f4d9c0b5830 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da3aea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9c0acb10 con 0x7f4d9c15e690 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da3aea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c15eaf0 con 0x7f4d9c06d190 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da1085640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d9c15e690 0x7f4d9c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da1085640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d9c15e690 0x7f4d9c0ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51640/0 (socket says 172.21.15.31:51640) 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da1085640 1 -- 172.21.15.31:0/1847838089 learned_addr learned my addr 172.21.15.31:0/1847838089 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da2087640 1 --2- 172.21.15.31:0/1847838089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d9c06d190 0x7f4d9c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da1085640 1 -- 172.21.15.31:0/1847838089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d9c06d190 msgr2=0x7f4d9c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da1085640 1 --2- 172.21.15.31:0/1847838089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d9c06d190 0x7f4d9c06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da1085640 1 -- 172.21.15.31:0/1847838089 >> v1:172.21.15.97:6789/0 conn(0x7f4d9c0b5830 legacy=0x7f4d9c0acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.301+0000 7f4da1085640 1 -- 172.21.15.31:0/1847838089 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c0ae5b0 con 0x7f4d9c15e690 2023-11-26T07:21:55.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da1085640 1 --2- 172.21.15.31:0/1847838089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d9c15e690 0x7f4d9c0ad480 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009870 tx=0x7f4d8c01c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09cba26a0923491 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da0884640 1 -- 172.21.15.31:0/1847838089 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0aa670 con 0x7f4d9c15e690 2023-11-26T07:21:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da0884640 1 -- 172.21.15.31:0/1847838089 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c06e350 con 0x7f4d9c15e690 2023-11-26T07:21:55.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da0884640 1 -- 172.21.15.31:0/1847838089 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0aa9f0 con 0x7f4d9c15e690 2023-11-26T07:21:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da3aea640 1 -- 172.21.15.31:0/1847838089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d9c15e690 msgr2=0x7f4d9c0ad480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da3aea640 1 --2- 172.21.15.31:0/1847838089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d9c15e690 0x7f4d9c0ad480 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009870 tx=0x7f4d8c01c510 comp rx=0 tx=0).stop 2023-11-26T07:21:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da3aea640 1 -- 172.21.15.31:0/1847838089 shutdown_connections 2023-11-26T07:21:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da3aea640 1 --2- 172.21.15.31:0/1847838089 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d9c06d190 0x7f4d9c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da3aea640 1 --2- 172.21.15.31:0/1847838089 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d9c15e690 0x7f4d9c0ad480 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da3aea640 1 -- 172.21.15.31:0/1847838089 >> 172.21.15.31:0/1847838089 conn(0x7f4d9c071d80 msgr2=0x7f4d9c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:55.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da3aea640 1 -- 172.21.15.31:0/1847838089 shutdown_connections 2023-11-26T07:21:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.305+0000 7f4da3aea640 1 -- 172.21.15.31:0/1847838089 wait complete. 2023-11-26T07:21:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da3aea640 1 Processor -- start 2023-11-26T07:21:55.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da3aea640 1 -- start start 2023-11-26T07:21:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da3aea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d9c06d190 0x7f4d9c149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da3aea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d9c0b5830 0x7f4d9c14a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da3aea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d9c15e690 0x7f4d9c1525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da3aea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c1575e0 con 0x7f4d9c06d190 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da3aea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c156a60 con 0x7f4d9c15e690 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da3aea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c156d40 con 0x7f4d9c0b5830 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da1886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d9c06d190 0x7f4d9c149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da1886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d9c06d190 0x7f4d9c149ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50516/0 (socket says 172.21.15.31:50516) 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da1886640 1 -- 172.21.15.31:0/149180436 learned_addr learned my addr 172.21.15.31:0/149180436 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da2087640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d9c15e690 0x7f4d9c1525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da1085640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d9c0b5830 0x7f4d9c14a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da1886640 1 -- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d9c0b5830 msgr2=0x7f4d9c14a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da1886640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d9c0b5830 0x7f4d9c14a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da1886640 1 -- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d9c15e690 msgr2=0x7f4d9c1525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da1886640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d9c15e690 0x7f4d9c1525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da1886640 1 -- 172.21.15.31:0/149180436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c0094d0 con 0x7f4d9c06d190 2023-11-26T07:21:55.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da1886640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d9c06d190 0x7f4d9c149ae0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800c500 tx=0x7f4d9800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4d927fc640 1 -- 172.21.15.31:0/149180436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98005220 con 0x7f4d9c06d190 2023-11-26T07:21:55.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4d927fc640 1 -- 172.21.15.31:0/149180436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98005380 con 0x7f4d9c06d190 2023-11-26T07:21:55.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4d927fc640 1 -- 172.21.15.31:0/149180436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98007270 con 0x7f4d9c06d190 2023-11-26T07:21:55.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da3aea640 1 -- 172.21.15.31:0/149180436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c157020 con 0x7f4d9c06d190 2023-11-26T07:21:55.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.309+0000 7f4da3aea640 1 -- 172.21.15.31:0/149180436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c1549c0 con 0x7f4d9c06d190 2023-11-26T07:21:55.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4d927fc640 1 -- 172.21.15.31:0/149180436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f4d980073d0 con 0x7f4d9c06d190 2023-11-26T07:21:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4d927fc640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d800667d0 0x7f4d80068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4d927fc640 1 -- 172.21.15.31:0/149180436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d98083a70 con 0x7f4d9c06d190 2023-11-26T07:21:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da1085640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d800667d0 0x7f4d80068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da6e53000 1 Processor -- start 2023-11-26T07:21:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da6e53000 1 -- start start 2023-11-26T07:21:55.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da1085640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d800667d0 0x7f4d80068c90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0ae000 tx=0x7f4d8c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac99c90 0x561a2ac9a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2ac4fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac50320 0x561a2ac50700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da6e53000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2aaba130 con 0x561a2ac99c90 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da6e53000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2aaa79a0 con 0x561a2ac50320 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da6e53000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a2aaa4ed0 con 0x561a2ac9a5b0 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da1886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac99c90 0x561a2ac9a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da1886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac99c90 0x561a2ac9a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50522/0 (socket says 172.21.15.31:50522) 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da1886640 1 -- 172.21.15.31:0/654456769 learned_addr learned my addr 172.21.15.31:0/654456769 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da2087640 1 --2- 172.21.15.31:0/654456769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac50320 0x561a2ac50700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.313+0000 7f4da1085640 1 --2- 172.21.15.31:0/654456769 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2ac4fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1886640 1 -- 172.21.15.31:0/654456769 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 msgr2=0x561a2ac4fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1886640 1 --2- 172.21.15.31:0/654456769 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2ac4fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1886640 1 -- 172.21.15.31:0/654456769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac50320 msgr2=0x561a2ac50700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1886640 1 --2- 172.21.15.31:0/654456769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac50320 0x561a2ac50700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1886640 1 -- 172.21.15.31:0/654456769 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a2a798890 con 0x561a2ac99c90 2023-11-26T07:21:55.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1886640 1 --2- 172.21.15.31:0/654456769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac99c90 0x561a2ac9a070 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4d98014000 tx=0x7f4d98054240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f77f8729d04e9a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4d7f7fe640 1 -- 172.21.15.31:0/654456769 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d980588c0 con 0x561a2ac99c90 2023-11-26T07:21:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4d7f7fe640 1 -- 172.21.15.31:0/654456769 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9805ca70 con 0x561a2ac99c90 2023-11-26T07:21:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4d7f7fe640 1 -- 172.21.15.31:0/654456769 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9805c5e0 con 0x561a2ac99c90 2023-11-26T07:21:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 -- 172.21.15.31:0/654456769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac99c90 msgr2=0x561a2ac9a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 --2- 172.21.15.31:0/654456769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac99c90 0x561a2ac9a070 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4d98014000 tx=0x7f4d98054240 comp rx=0 tx=0).stop 2023-11-26T07:21:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 -- 172.21.15.31:0/654456769 shutdown_connections 2023-11-26T07:21:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 --2- 172.21.15.31:0/654456769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac50320 0x561a2ac50700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 --2- 172.21.15.31:0/654456769 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2ac4fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 --2- 172.21.15.31:0/654456769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac99c90 0x561a2ac9a070 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 -- 172.21.15.31:0/654456769 >> 172.21.15.31:0/654456769 conn(0x561a2ac4c2d0 msgr2=0x561a2ac4e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 -- 172.21.15.31:0/654456769 shutdown_connections 2023-11-26T07:21:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 -- 172.21.15.31:0/654456769 wait complete. 2023-11-26T07:21:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 -- start start 2023-11-26T07:21:55.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 Processor -- start 2023-11-26T07:21:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac54d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac55280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2ac68910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2aaa07c0 con 0x561a2ac50320 2023-11-26T07:21:55.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2aa58bf0 con 0x561a2ac99c90 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a2aa5ea90 con 0x561a2ac9a5b0 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da6e53000 10 client.0 _ll_register_callbacks cb 0x561a2ac3b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac54d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1085640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac55280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da2087640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2ac68910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1085640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac55280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51652/0 (socket says 172.21.15.31:51652) 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac54d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50526/0 (socket says 172.21.15.31:50526) 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.317+0000 7f4da1085640 1 -- 172.21.15.31:0/5406364 learned_addr learned my addr 172.21.15.31:0/5406364 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da1085640 1 -- 172.21.15.31:0/5406364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 msgr2=0x561a2ac68910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da1085640 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2ac68910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da1085640 1 -- 172.21.15.31:0/5406364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 msgr2=0x561a2ac54d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da1085640 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac54d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da1085640 1 -- 172.21.15.31:0/5406364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c1549c0 con 0x561a2ac99c90 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da1886640 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac54d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da1085640 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac55280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c01abc0 tx=0x7f4d8c01ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a47886533a75fe server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4d7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4d7cff9640 1 -- 172.21.15.31:0/5406364 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0c4070 con 0x561a2ac99c90 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4d7cff9640 1 -- 172.21.15.31:0/5406364 <== mon.1 v2:172.21.15.97:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d8c0b5a60 con 0x561a2ac99c90 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 -- 172.21.15.31:0/5406364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a2aa62470 con 0x561a2ac99c90 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4d7cff9640 1 -- 172.21.15.31:0/5406364 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d8c0b5c20 con 0x561a2ac99c90 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 10 client.6472 fetch_fsmap learned FSMap version 47 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 -- 172.21.15.31:0/5406364 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x561a2aa61190 con 0x561a2ac99c90 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4d7cff9640 1 -- 172.21.15.31:0/5406364 <== mon.1 v2:172.21.15.97:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561a2aa62470 con 0x561a2ac99c90 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4d7cff9640 1 -- 172.21.15.31:0/5406364 <== mon.1 v2:172.21.15.97:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d8c0a8050 con 0x561a2ac99c90 2023-11-26T07:21:55.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 10 client.6472 fetch_fsmap finished waiting for FSMap version 47 2023-11-26T07:21:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 10 client.6472 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:21:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 20 client.6472 populate_metadata read hostname 'smithi031' 2023-11-26T07:21:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.31:6832/242493352,v1:172.21.15.31:6834/242493352] conn(0x561a2ac69320 0x561a2acabf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 4 client.6472 mds_command: new command op to 6597 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:21:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 -- 172.21.15.31:0/5406364 --> [v2:172.21.15.31:6832/242493352,v1:172.21.15.31:6834/242493352] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a2aa65b50 con 0x561a2ac69320 2023-11-26T07:21:55.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da1886640 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.31:6832/242493352,v1:172.21.15.31:6834/242493352] conn(0x561a2ac69320 0x561a2acabf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da1886640 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.31:6832/242493352,v1:172.21.15.31:6834/242493352] conn(0x561a2ac69320 0x561a2acabf50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba189976e1a70d7a server_cookie=8e54405cc14c4dc6 in_seq=0 out_seq=0 2023-11-26T07:21:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4d7cff9640 10 client.6472 ms_handle_connect on v2:172.21.15.31:6832/242493352 2023-11-26T07:21:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4d7cff9640 1 -- 172.21.15.31:0/5406364 <== mds.0 v2:172.21.15.31:6832/242493352 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a2aa65b50 con 0x561a2ac69320 2023-11-26T07:21:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4d7cff9640 10 client.6472 handle_command_reply: tid=0 2023-11-26T07:21:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 client.6472 shutdown 2023-11-26T07:21:55.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 -- 172.21.15.31:0/5406364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 msgr2=0x561a2ac55280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac55280 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c01abc0 tx=0x7f4d8c01ad50 comp rx=0 tx=0).stop 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 -- 172.21.15.31:0/5406364 shutdown_connections 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.31:6832/242493352,v1:172.21.15.31:6834/242493352] conn(0x561a2ac69320 0x561a2acabf50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2ac68910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac55280 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 --2- 172.21.15.31:0/5406364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac54d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4d7cff9640 0 client.6472 ms_handle_reset on v2:172.21.15.31:6832/242493352 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 -- 172.21.15.31:0/5406364 >> 172.21.15.31:0/5406364 conn(0x561a2ac4c2d0 msgr2=0x561a2ac4e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 -- 172.21.15.31:0/5406364 shutdown_connections 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 1 -- 172.21.15.31:0/5406364 wait complete. 2023-11-26T07:21:55.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.321+0000 7f4da6e53000 20 client.6472 trim_cache size 0 max 16384 2023-11-26T07:21:55.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.337+0000 7f4da6e53000 1 Processor -- start 2023-11-26T07:21:55.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.337+0000 7f4da6e53000 1 -- start start 2023-11-26T07:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac50320 0x561a2ac98a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac97b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac9a5b0 0x561a2ac98070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2a759940 con 0x561a2ac9a5b0 2023-11-26T07:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2aaa3320 con 0x561a2ac99c90 2023-11-26T07:21:55.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a2a7991b0 con 0x561a2ac50320 2023-11-26T07:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da2087640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac9a5b0 0x561a2ac98070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da2087640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac9a5b0 0x561a2ac98070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50542/0 (socket says 172.21.15.31:50542) 2023-11-26T07:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da2087640 1 -- 172.21.15.31:0/3557345982 learned_addr learned my addr 172.21.15.31:0/3557345982 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da2087640 1 -- 172.21.15.31:0/3557345982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac50320 msgr2=0x561a2ac98a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da2087640 1 --2- 172.21.15.31:0/3557345982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac50320 0x561a2ac98a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da2087640 1 -- 172.21.15.31:0/3557345982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 msgr2=0x561a2ac97b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da2087640 1 --2- 172.21.15.31:0/3557345982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac97b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da2087640 1 -- 172.21.15.31:0/3557345982 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a2aa47490 con 0x561a2ac9a5b0 2023-11-26T07:21:55.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da2087640 1 --2- 172.21.15.31:0/3557345982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac9a5b0 0x561a2ac98070 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4d94017850 tx=0x7f4d94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaf9bcc537f7b4c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4d7ffff640 1 -- 172.21.15.31:0/3557345982 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9400e040 con 0x561a2ac9a5b0 2023-11-26T07:21:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4d7ffff640 1 -- 172.21.15.31:0/3557345982 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9400f040 con 0x561a2ac9a5b0 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4d7ffff640 1 -- 172.21.15.31:0/3557345982 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94015cd0 con 0x561a2ac9a5b0 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 -- 172.21.15.31:0/3557345982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac9a5b0 msgr2=0x561a2ac98070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 --2- 172.21.15.31:0/3557345982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac9a5b0 0x561a2ac98070 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4d94017850 tx=0x7f4d94009750 comp rx=0 tx=0).stop 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 -- 172.21.15.31:0/3557345982 shutdown_connections 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 --2- 172.21.15.31:0/3557345982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac9a5b0 0x561a2ac98070 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 --2- 172.21.15.31:0/3557345982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac97b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 --2- 172.21.15.31:0/3557345982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac50320 0x561a2ac98a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 -- 172.21.15.31:0/3557345982 >> 172.21.15.31:0/3557345982 conn(0x561a2ac4c2d0 msgr2=0x561a2ac4e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 -- 172.21.15.31:0/3557345982 shutdown_connections 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 -- 172.21.15.31:0/3557345982 wait complete. 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 -- start start 2023-11-26T07:21:55.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.341+0000 7f4da6e53000 1 Processor -- start 2023-11-26T07:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac7c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac7cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2acabf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2ac688c0 con 0x561a2ac50320 2023-11-26T07:21:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a2aa4b3b0 con 0x561a2ac99c90 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x561a2aa6a2e0 con 0x561a2ac9a5b0 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 10 client.0 _ll_register_callbacks cb 0x561a2aaeec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac7c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1886640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac7c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50546/0 (socket says 172.21.15.31:50546) 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1886640 1 -- 172.21.15.31:0/1811347259 learned_addr learned my addr 172.21.15.31:0/1811347259 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1085640 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac7cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da2087640 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2acabf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1886640 1 -- 172.21.15.31:0/1811347259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 msgr2=0x561a2acabf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1886640 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2acabf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1886640 1 -- 172.21.15.31:0/1811347259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 msgr2=0x561a2ac7cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1886640 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac7cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1886640 1 -- 172.21.15.31:0/1811347259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d94017530 con 0x561a2ac50320 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1085640 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac7cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da2087640 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2acabf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:55.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1886640 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac7c7e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4d98054520 tx=0x7f4d98000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bfee679e7efa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4d7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:21:55.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4d7cff9640 1 -- 172.21.15.31:0/1811347259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98071030 con 0x561a2ac50320 2023-11-26T07:21:55.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4d7cff9640 1 -- 172.21.15.31:0/1811347259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d9805ca70 con 0x561a2ac50320 2023-11-26T07:21:55.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 -- 172.21.15.31:0/1811347259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a2aa9f110 con 0x561a2ac50320 2023-11-26T07:21:55.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4d7cff9640 1 -- 172.21.15.31:0/1811347259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d980545f0 con 0x561a2ac50320 2023-11-26T07:21:55.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 10 client.6750 fetch_fsmap learned FSMap version 47 2023-11-26T07:21:55.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 -- 172.21.15.31:0/1811347259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x561a2a798cc0 con 0x561a2ac50320 2023-11-26T07:21:55.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4d7cff9640 1 -- 172.21.15.31:0/1811347259 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561a2aa9f110 con 0x561a2ac50320 2023-11-26T07:21:55.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4d7cff9640 1 -- 172.21.15.31:0/1811347259 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d98019030 con 0x561a2ac50320 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 10 client.6750 fetch_fsmap finished waiting for FSMap version 47 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 10 client.6750 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 20 client.6750 populate_metadata read hostname 'smithi031' 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.31:6832/242493352,v1:172.21.15.31:6834/242493352] conn(0x561a2acac490 0x561a2acac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 4 client.6750 mds_command: new command op to 6597 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 -- 172.21.15.31:0/1811347259 --> [v2:172.21.15.31:6832/242493352,v1:172.21.15.31:6834/242493352] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561a2aace9a0 con 0x561a2acac490 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1085640 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.31:6832/242493352,v1:172.21.15.31:6834/242493352] conn(0x561a2acac490 0x561a2acac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da1085640 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.31:6832/242493352,v1:172.21.15.31:6834/242493352] conn(0x561a2acac490 0x561a2acac870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ae79a322add892 server_cookie=5f015b3e2816c39e in_seq=0 out_seq=0 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4d7cff9640 10 client.6750 ms_handle_connect on v2:172.21.15.31:6832/242493352 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4d7cff9640 1 -- 172.21.15.31:0/1811347259 <== mds.0 v2:172.21.15.31:6832/242493352 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561a2aace9a0 con 0x561a2acac490 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4d7cff9640 10 client.6750 handle_command_reply: tid=0 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.345+0000 7f4da6e53000 1 client.6750 shutdown 2023-11-26T07:21:55.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 1 -- 172.21.15.31:0/1811347259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 msgr2=0x561a2ac7c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac7c7e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4d98054520 tx=0x7f4d98000ee0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 1 -- 172.21.15.31:0/1811347259 shutdown_connections 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.31:6832/242493352,v1:172.21.15.31:6834/242493352] conn(0x561a2acac490 0x561a2acac870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x561a2ac9a5b0 0x561a2acabf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a2ac99c90 0x561a2ac7cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4d7cff9640 0 client.6750 ms_handle_reset on v2:172.21.15.31:6832/242493352 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 1 --2- 172.21.15.31:0/1811347259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x561a2ac50320 0x561a2ac7c7e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 1 -- 172.21.15.31:0/1811347259 >> 172.21.15.31:0/1811347259 conn(0x561a2ac4c2d0 msgr2=0x561a2aca70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 1 -- 172.21.15.31:0/1811347259 shutdown_connections 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 1 -- 172.21.15.31:0/1811347259 wait complete. 2023-11-26T07:21:55.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da6e53000 20 client.6750 trim_cache size 0 max 16384 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "4c2b4df1-71b1-4841-8807-af144d2610e7", 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 6597, 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 5.9580870800000003, 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 47, 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 68, 2023-11-26T07:21:55.355 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 68, 2023-11-26T07:21:55.356 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 11.891096138 2023-11-26T07:21:55.356 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:21:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 -- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d800667d0 msgr2=0x7f4d80068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d800667d0 0x7f4d80068c90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0ae000 tx=0x7f4d8c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 -- 172.21.15.31:0/149180436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d9c06d190 msgr2=0x7f4d9c149ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d9c06d190 0x7f4d9c149ae0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4d9800c500 tx=0x7f4d9800c9c0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 -- 172.21.15.31:0/149180436 shutdown_connections 2023-11-26T07:21:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4d800667d0 0x7f4d80068c90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d9c15e690 0x7f4d9c1525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d9c0b5830 0x7f4d9c14a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 --2- 172.21.15.31:0/149180436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d9c06d190 0x7f4d9c149ae0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 -- 172.21.15.31:0/149180436 >> 172.21.15.31:0/149180436 conn(0x7f4d9c071d80 msgr2=0x7f4d9c0acd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 -- 172.21.15.31:0/149180436 shutdown_connections 2023-11-26T07:21:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.349+0000 7f4da3aea640 1 -- 172.21.15.31:0/149180436 wait complete. 2023-11-26T07:21:55.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:21:55.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.509+0000 7fcfc2b01640 1 Processor -- start 2023-11-26T07:21:55.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.509+0000 7fcfc2b01640 1 -- start start 2023-11-26T07:21:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfc2b01640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcfbc0cead0 0x7fcfbc0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfc2b01640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfbc0d1eb0 0x7fcfbc06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfc2b01640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfbc0d2300 con 0x7fcfbc15d7b0 2023-11-26T07:21:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfc2b01640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfbc0d17e0 con 0x7fcfbc0cead0 2023-11-26T07:21:55.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfc2b01640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0cefd0 con 0x7fcfbc0d1eb0 2023-11-26T07:21:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfbbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcfbc0cead0 0x7fcfbc0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfc109e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfbc0d1eb0 0x7fcfbc06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfbbfff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcfbc0cead0 0x7fcfbc0d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51676/0 (socket says 172.21.15.31:51676) 2023-11-26T07:21:55.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfbbfff640 1 -- 172.21.15.31:0/1742615166 learned_addr learned my addr 172.21.15.31:0/1742615166 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfbbfff640 1 -- 172.21.15.31:0/1742615166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfbc0d1eb0 msgr2=0x7fcfbc06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfbbfff640 1 --2- 172.21.15.31:0/1742615166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfbc0d1eb0 0x7fcfbc06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfbbfff640 1 -- 172.21.15.31:0/1742615166 >> v1:172.21.15.97:6789/0 conn(0x7fcfbc15d7b0 legacy=0x7fcfbc0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.513+0000 7fcfbbfff640 1 -- 172.21.15.31:0/1742615166 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0cf2b0 con 0x7fcfbc0cead0 2023-11-26T07:21:55.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfbbfff640 1 --2- 172.21.15.31:0/1742615166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfbc0cead0 0x7fcfbc0d1970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009870 tx=0x7fcfa801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ec07809a1119b27 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfbb7fe640 1 -- 172.21.15.31:0/1742615166 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80ab5f0 con 0x7fcfbc0cead0 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfbb7fe640 1 -- 172.21.15.31:0/1742615166 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa80ab750 con 0x7fcfbc0cead0 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfbb7fe640 1 -- 172.21.15.31:0/1742615166 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa80abae0 con 0x7fcfbc0cead0 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfc2b01640 1 -- 172.21.15.31:0/1742615166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfbc0cead0 msgr2=0x7fcfbc0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfc2b01640 1 --2- 172.21.15.31:0/1742615166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfbc0cead0 0x7fcfbc0d1970 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009870 tx=0x7fcfa801c3f0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfc2b01640 1 -- 172.21.15.31:0/1742615166 shutdown_connections 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfc2b01640 1 --2- 172.21.15.31:0/1742615166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfbc0d1eb0 0x7fcfbc06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfc2b01640 1 --2- 172.21.15.31:0/1742615166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfbc0cead0 0x7fcfbc0d1970 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfc2b01640 1 -- 172.21.15.31:0/1742615166 >> 172.21.15.31:0/1742615166 conn(0x7fcfbc0d33b0 msgr2=0x7fcfbc0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfc2b01640 1 -- 172.21.15.31:0/1742615166 shutdown_connections 2023-11-26T07:21:55.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfc2b01640 1 -- 172.21.15.31:0/1742615166 wait complete. 2023-11-26T07:21:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfc2b01640 1 Processor -- start 2023-11-26T07:21:55.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.517+0000 7fcfc2b01640 1 -- start start 2023-11-26T07:21:55.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc2b01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfbc0cead0 0x7fcfbc0c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc2b01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfbc0d1eb0 0x7fcfbc154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc2b01640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfbc15d7b0 0x7fcfbc154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc2b01640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0b6710 con 0x7fcfbc0cead0 2023-11-26T07:21:55.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc2b01640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0b8e50 con 0x7fcfbc0d1eb0 2023-11-26T07:21:55.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc2b01640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0b6430 con 0x7fcfbc15d7b0 2023-11-26T07:21:55.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc089d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfbc0cead0 0x7fcfbc0c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc089d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfbc0cead0 0x7fcfbc0c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50562/0 (socket says 172.21.15.31:50562) 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc089d640 1 -- 172.21.15.31:0/931469700 learned_addr learned my addr 172.21.15.31:0/931469700 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfbbfff640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfbc0d1eb0 0x7fcfbc154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc109e640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfbc15d7b0 0x7fcfbc154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc089d640 1 -- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfbc15d7b0 msgr2=0x7fcfbc154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc089d640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfbc15d7b0 0x7fcfbc154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc089d640 1 -- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfbc0d1eb0 msgr2=0x7fcfbc154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc089d640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfbc0d1eb0 0x7fcfbc154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc089d640 1 -- 172.21.15.31:0/931469700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa80094d0 con 0x7fcfbc0cead0 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc089d640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfbc0cead0 0x7fcfbc0c9520 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c3f0 tx=0x7fcfb000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfb97fa640 1 -- 172.21.15.31:0/931469700 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00052d0 con 0x7fcfbc0cead0 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfb97fa640 1 -- 172.21.15.31:0/931469700 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb0005430 con 0x7fcfbc0cead0 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfb97fa640 1 -- 172.21.15.31:0/931469700 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0003840 con 0x7fcfbc0cead0 2023-11-26T07:21:55.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc2b01640 1 -- 172.21.15.31:0/931469700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc0b8800 con 0x7fcfbc0cead0 2023-11-26T07:21:55.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc2b01640 1 -- 172.21.15.31:0/931469700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc155280 con 0x7fcfbc0cead0 2023-11-26T07:21:55.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfc2b01640 1 -- 172.21.15.31:0/931469700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfbc04e850 con 0x7fcfbc0cead0 2023-11-26T07:21:55.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.521+0000 7fcfb97fa640 1 -- 172.21.15.31:0/931469700 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fcfb00039a0 con 0x7fcfbc0cead0 2023-11-26T07:21:55.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.525+0000 7fcfb97fa640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfa0066770 0x7fcfa0068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:55.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.525+0000 7fcfb97fa640 1 -- 172.21.15.31:0/931469700 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfb0083600 con 0x7fcfbc0cead0 2023-11-26T07:21:55.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.525+0000 7fcfbbfff640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfa0066770 0x7fcfa0068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:55.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.525+0000 7fcfbbfff640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfa0066770 0x7fcfa0068c30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80af000 tx=0x7fcfa80a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:55.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.529+0000 7fcfb97fa640 1 -- 172.21.15.31:0/931469700 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcfb00507e0 con 0x7fcfbc0cead0 2023-11-26T07:21:55.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.721+0000 7fcfc2b01640 1 -- 172.21.15.31:0/931469700 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcfbc0d2300 con 0x7fcfbc0cead0 2023-11-26T07:21:55.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.721+0000 7fcfb97fa640 1 -- 172.21.15.31:0/931469700 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcfbc0d2300 con 0x7fcfbc0cead0 2023-11-26T07:21:55.725 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:21:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 -- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfa0066770 msgr2=0x7fcfa0068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfa0066770 0x7fcfa0068c30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80af000 tx=0x7fcfa80a7000 comp rx=0 tx=0).stop 2023-11-26T07:21:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 -- 172.21.15.31:0/931469700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfbc0cead0 msgr2=0x7fcfbc0c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfbc0cead0 0x7fcfbc0c9520 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000c3f0 tx=0x7fcfb000c8b0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 -- 172.21.15.31:0/931469700 shutdown_connections 2023-11-26T07:21:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcfa0066770 0x7fcfa0068c30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfbc15d7b0 0x7fcfbc154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfbc0d1eb0 0x7fcfbc154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 --2- 172.21.15.31:0/931469700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfbc0cead0 0x7fcfbc0c9520 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:55.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 -- 172.21.15.31:0/931469700 >> 172.21.15.31:0/931469700 conn(0x7fcfbc0d33b0 msgr2=0x7fcfbc0d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 -- 172.21.15.31:0/931469700 shutdown_connections 2023-11-26T07:21:55.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:55.725+0000 7fcfc2b01640 1 -- 172.21.15.31:0/931469700 wait complete. 2023-11-26T07:21:55.746 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:21:55.746 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:21:55.746 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-11-26T07:21:55.746 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:21:55.746 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:21:55.746 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:21:55.746 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:21:55.747 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:21:55.747 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:21:55.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:55.747 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-11-26T07:21:55.757 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:21:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:21:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:21:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-11-26T07:21:55.757 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:55.757 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42124sec preferred_lft 42124sec 2023-11-26T07:21:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 02:42:1b:e4:a9:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:21:55.759 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:21:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:55.760 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:21:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:21:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:21:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:55.760 INFO:teuthology.orchestra.run.smithi031.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:21:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:21:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:21:55.760 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:55.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:55.761 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:21:55.817 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-11-26T07:21:55.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:55.819 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-11-26T07:21:55.875 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-11-26T07:21:55.875 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:21:55.876 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-26T07:21:55.876 INFO:teuthology.orchestra.run.smithi031.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:21:55.876 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-26T07:21:55.876 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:21:55.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:55.876 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:21:55.876 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:21:55.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-26T07:21:55.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-26T07:21:55.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-26T07:21:55.877 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:21:55.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:55.970 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:21:55.979 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:21:55.979 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:21:55.980 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:21:55.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:55.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-11-26T07:21:56.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:21:56.043 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:21:56.094 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:21:56.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:21:56.095 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:21:56.145 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:21:56.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:56.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:56.238 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:56.238 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:21:56.251 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:56.251 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:21:56.251 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:21:56.251 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:21:56.251 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:21:56.255 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:21:56.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:56.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:21:56.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:56.277 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:21:56.337 INFO:teuthology.orchestra.run.smithi031.stdout:client6487 2023-11-26T07:21:56.337 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:21:56.337 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:56.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:21:56.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.569+0000 7f3fe749c640 1 Processor -- start 2023-11-26T07:21:56.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.569+0000 7f3fe749c640 1 -- start start 2023-11-26T07:21:56.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.569+0000 7f3fe749c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3fe00b39c0 0x7f3fe006c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.569+0000 7f3fe749c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3fe006cb70 0x7f3fe0070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.569+0000 7f3fe749c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe006cf90 con 0x7f3fe015f540 2023-11-26T07:21:56.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.569+0000 7f3fe749c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00b2650 con 0x7f3fe006cb70 2023-11-26T07:21:56.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.569+0000 7f3fe749c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00b2920 con 0x7f3fe00b39c0 2023-11-26T07:21:56.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.569+0000 7f3fe5a39640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3fe006cb70 0x7f3fe0070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.569+0000 7f3fe5a39640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3fe006cb70 0x7f3fe0070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50566/0 (socket says 172.21.15.31:50566) 2023-11-26T07:21:56.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.569+0000 7f3fe5a39640 1 -- 172.21.15.31:0/3763945926 learned_addr learned my addr 172.21.15.31:0/3763945926 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:56.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.573+0000 7f3fe4a37640 1 --2- 172.21.15.31:0/3763945926 >> v2:172.21.15.97:3300/0 conn(0x7f3fe00b39c0 0x7f3fe006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.573+0000 7f3fe5a39640 1 -- 172.21.15.31:0/3763945926 >> v2:172.21.15.97:3300/0 conn(0x7f3fe00b39c0 msgr2=0x7f3fe006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.573+0000 7f3fe5a39640 1 --2- 172.21.15.31:0/3763945926 >> v2:172.21.15.97:3300/0 conn(0x7f3fe00b39c0 0x7f3fe006c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.573+0000 7f3fe5a39640 1 -- 172.21.15.31:0/3763945926 >> v1:172.21.15.97:6789/0 conn(0x7f3fe015f540 legacy=0x7f3fe00b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.573+0000 7f3fe5a39640 1 -- 172.21.15.31:0/3763945926 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00b2c00 con 0x7f3fe006cb70 2023-11-26T07:21:56.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.573+0000 7f3fe4a37640 1 --2- 172.21.15.31:0/3763945926 >> v2:172.21.15.97:3300/0 conn(0x7f3fe00b39c0 0x7f3fe006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:56.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.573+0000 7f3fe5a39640 1 --2- 172.21.15.31:0/3763945926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe006cb70 0x7f3fe0070880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3fe006eb50 tx=0x7f3fd801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee040b42df286c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:56.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.573+0000 7f3fd7fff640 1 -- 172.21.15.31:0/3763945926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80a57d0 con 0x7f3fe006cb70 2023-11-26T07:21:56.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.573+0000 7f3fd7fff640 1 -- 172.21.15.31:0/3763945926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd80a5930 con 0x7f3fe006cb70 2023-11-26T07:21:56.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.573+0000 7f3fd7fff640 1 -- 172.21.15.31:0/3763945926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80a5cc0 con 0x7f3fe006cb70 2023-11-26T07:21:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 -- 172.21.15.31:0/3763945926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe006cb70 msgr2=0x7f3fe0070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 --2- 172.21.15.31:0/3763945926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe006cb70 0x7f3fe0070880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3fe006eb50 tx=0x7f3fd801c910 comp rx=0 tx=0).stop 2023-11-26T07:21:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 -- 172.21.15.31:0/3763945926 shutdown_connections 2023-11-26T07:21:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 --2- 172.21.15.31:0/3763945926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe006cb70 0x7f3fe0070880 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 --2- 172.21.15.31:0/3763945926 >> v2:172.21.15.97:3300/0 conn(0x7f3fe00b39c0 0x7f3fe006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 -- 172.21.15.31:0/3763945926 >> 172.21.15.31:0/3763945926 conn(0x7f3fe00b0c40 msgr2=0x7f3fe00b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 -- 172.21.15.31:0/3763945926 shutdown_connections 2023-11-26T07:21:56.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 -- 172.21.15.31:0/3763945926 wait complete. 2023-11-26T07:21:56.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 Processor -- start 2023-11-26T07:21:56.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 -- start start 2023-11-26T07:21:56.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3fe006cb70 0x7f3fe00cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe00b39c0 0x7f3fe00cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fe015f540 0x7f3fe0159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00cd240 con 0x7f3fe00b39c0 2023-11-26T07:21:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00ccf60 con 0x7f3fe015f540 2023-11-26T07:21:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe749c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00cb330 con 0x7f3fe006cb70 2023-11-26T07:21:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe4a37640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe00b39c0 0x7f3fe00cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe4a37640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe00b39c0 0x7f3fe00cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50572/0 (socket says 172.21.15.31:50572) 2023-11-26T07:21:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe4a37640 1 -- 172.21.15.31:0/107862826 learned_addr learned my addr 172.21.15.31:0/107862826 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe5a39640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fe015f540 0x7f3fe0159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe5238640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3fe006cb70 0x7f3fe00cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe4a37640 1 -- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3fe006cb70 msgr2=0x7f3fe00cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe4a37640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3fe006cb70 0x7f3fe00cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe4a37640 1 -- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fe015f540 msgr2=0x7f3fe0159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe4a37640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fe015f540 0x7f3fe0159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.577+0000 7f3fe4a37640 1 -- 172.21.15.31:0/107862826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd8009b30 con 0x7f3fe00b39c0 2023-11-26T07:21:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fe4a37640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe00b39c0 0x7f3fe00cc610 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000d800 tx=0x7f3fd000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fd5ffb640 1 -- 172.21.15.31:0/107862826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000b6c0 con 0x7f3fe00b39c0 2023-11-26T07:21:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fd5ffb640 1 -- 172.21.15.31:0/107862826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd000b820 con 0x7f3fe00b39c0 2023-11-26T07:21:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fe749c640 1 -- 172.21.15.31:0/107862826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe00cb610 con 0x7f3fe00b39c0 2023-11-26T07:21:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fd5ffb640 1 -- 172.21.15.31:0/107862826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd000bbd0 con 0x7f3fe00b39c0 2023-11-26T07:21:56.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fe749c640 1 -- 172.21.15.31:0/107862826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe00cc350 con 0x7f3fe00b39c0 2023-11-26T07:21:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fe749c640 1 -- 172.21.15.31:0/107862826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe004e850 con 0x7f3fe00b39c0 2023-11-26T07:21:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fd5ffb640 1 -- 172.21.15.31:0/107862826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f3fd000bd30 con 0x7f3fe00b39c0 2023-11-26T07:21:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fd5ffb640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3fc4066800 0x7f3fc4068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:56.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fd5ffb640 1 -- 172.21.15.31:0/107862826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fd00847d0 con 0x7f3fe00b39c0 2023-11-26T07:21:56.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.581+0000 7f3fe5238640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3fc4066800 0x7f3fc4068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:56.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.585+0000 7f3fe5238640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3fc4066800 0x7f3fc4068cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc007fd0 tx=0x7f3fdc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:56.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.585+0000 7f3fd5ffb640 1 -- 172.21.15.31:0/107862826 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3fd0051860 con 0x7f3fe00b39c0 2023-11-26T07:21:56.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.777+0000 7f3fe749c640 1 -- 172.21.15.31:0/107862826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3fe00ced70 con 0x7f3fe00b39c0 2023-11-26T07:21:56.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.777+0000 7f3fd5ffb640 1 -- 172.21.15.31:0/107862826 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3fd0054e80 con 0x7f3fe00b39c0 2023-11-26T07:21:56.781 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:21:56.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 -- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3fc4066800 msgr2=0x7f3fc4068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3fc4066800 0x7f3fc4068cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc007fd0 tx=0x7f3fdc009040 comp rx=0 tx=0).stop 2023-11-26T07:21:56.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 -- 172.21.15.31:0/107862826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe00b39c0 msgr2=0x7f3fe00cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:56.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe00b39c0 0x7f3fe00cc610 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000d800 tx=0x7f3fd000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 -- 172.21.15.31:0/107862826 shutdown_connections 2023-11-26T07:21:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f3fc4066800 0x7f3fc4068cc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3fe015f540 0x7f3fe0159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fe00b39c0 0x7f3fe00cc610 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 --2- 172.21.15.31:0/107862826 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3fe006cb70 0x7f3fe00cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 -- 172.21.15.31:0/107862826 >> 172.21.15.31:0/107862826 conn(0x7f3fe00b0c40 msgr2=0x7f3fe0163010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 -- 172.21.15.31:0/107862826 shutdown_connections 2023-11-26T07:21:56.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:56.781+0000 7f3fe749c640 1 -- 172.21.15.31:0/107862826 wait complete. 2023-11-26T07:21:56.802 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:21:56.802 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:21:56.802 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-26T07:21:56.802 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:21:56.802 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:21:56.802 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:21:56.802 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:21:56.803 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:21:56.803 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:21:56.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:56.803 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-26T07:21:56.813 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:21:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:21:56.813 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:21:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-26T07:21:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:56.814 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-11-26T07:21:56.814 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:56.814 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-11-26T07:21:56.814 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42146sec preferred_lft 42146sec 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:61:bc:5b:3d brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:21:56.815 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:21:56.816 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:21:56.817 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:21:56.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:56.818 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:21:56.886 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-26T07:21:56.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:56.888 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-26T07:21:56.941 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-26T07:21:56.941 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:21:56.942 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:21:56.942 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:21:56.942 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:21:56.942 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:21:56.942 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:21:56.942 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:21:56.942 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:21:56.942 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:21:56.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:56.943 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:21:56.943 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:21:56.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:21:56.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:21:56.943 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:21:56.943 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:21:57.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:57.041 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:21:57.049 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:21:57.049 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:21:57.050 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:21:57.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:57.050 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-26T07:21:57.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:21:57.119 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:21:57.170 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:21:57.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:21:57.170 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:21:57.221 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:21:57.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:57.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:57.319 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:57.319 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:21:57.332 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:21:57.332 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:21:57.333 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:21:57.333 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 1@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:21:57.333 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:21:57.336 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:21:57.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:57.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:21:57.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:21:57.357 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:21:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:client5942 2023-11-26T07:21:57.413 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:21:57.413 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:57.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:21:57.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7c016640 1 Processor -- start 2023-11-26T07:21:57.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7c016640 1 -- start start 2023-11-26T07:21:57.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7c016640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd740b4210 0x7fdd740b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7c016640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd7415db40 0x7fdd7415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7c016640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd74147190 con 0x7fdd74146820 2023-11-26T07:21:57.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7c016640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd74146c00 con 0x7fdd7415db40 2023-11-26T07:21:57.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7c016640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7406bf90 con 0x7fdd740b4210 2023-11-26T07:21:57.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7a5b3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd7415db40 0x7fdd7415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7a5b3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdd7415db40 0x7fdd7415df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50578/0 (socket says 172.21.15.31:50578) 2023-11-26T07:21:57.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7a5b3640 1 -- 172.21.15.31:0/718164697 learned_addr learned my addr 172.21.15.31:0/718164697 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:57.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd79db2640 1 --1- 172.21.15.31:0/718164697 >> v1:172.21.15.31:6789/0 conn(0x7fdd74146820 0x7fdd74070a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53854/0 (socket says 172.21.15.31:53854) 2023-11-26T07:21:57.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd795b1640 1 --2- 172.21.15.31:0/718164697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd740b4210 0x7fdd740b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7a5b3640 1 -- 172.21.15.31:0/718164697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd740b4210 msgr2=0x7fdd740b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:57.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7a5b3640 1 --2- 172.21.15.31:0/718164697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd740b4210 0x7fdd740b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7a5b3640 1 -- 172.21.15.31:0/718164697 >> v1:172.21.15.31:6789/0 conn(0x7fdd74146820 legacy=0x7fdd74070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:57.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd7a5b3640 1 -- 172.21.15.31:0/718164697 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd740b4bb0 con 0x7fdd7415db40 2023-11-26T07:21:57.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.609+0000 7fdd78db0640 1 -- 172.21.15.31:0/718164697 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969324337 0 0) 0x7fdd74147190 con 0x7fdd74146820 2023-11-26T07:21:57.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd7a5b3640 1 --2- 172.21.15.31:0/718164697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd7415db40 0x7fdd7415df90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009e50 tx=0x7fdd6c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66a2d846ffeadb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:57.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd78db0640 1 -- 172.21.15.31:0/718164697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0a58a0 con 0x7fdd7415db40 2023-11-26T07:21:57.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd78db0640 1 -- 172.21.15.31:0/718164697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c0a5a00 con 0x7fdd7415db40 2023-11-26T07:21:57.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd78db0640 1 -- 172.21.15.31:0/718164697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0a5d90 con 0x7fdd7415db40 2023-11-26T07:21:57.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd7c016640 1 -- 172.21.15.31:0/718164697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd7415db40 msgr2=0x7fdd7415df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:57.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd7c016640 1 --2- 172.21.15.31:0/718164697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd7415db40 0x7fdd7415df90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009e50 tx=0x7fdd6c01c9e0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd7c016640 1 -- 172.21.15.31:0/718164697 shutdown_connections 2023-11-26T07:21:57.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd7c016640 1 --2- 172.21.15.31:0/718164697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd7415db40 0x7fdd7415df90 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd7c016640 1 --2- 172.21.15.31:0/718164697 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd740b4210 0x7fdd740b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd7c016640 1 -- 172.21.15.31:0/718164697 >> 172.21.15.31:0/718164697 conn(0x7fdd740b2f10 msgr2=0x7fdd740b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:57.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd7c016640 1 -- 172.21.15.31:0/718164697 shutdown_connections 2023-11-26T07:21:57.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.613+0000 7fdd7c016640 1 -- 172.21.15.31:0/718164697 wait complete. 2023-11-26T07:21:57.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7c016640 1 Processor -- start 2023-11-26T07:21:57.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7c016640 1 -- start start 2023-11-26T07:21:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7c016640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd740b4210 0x7fdd7415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7c016640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd74146820 0x7fdd740b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7c016640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd7415db40 0x7fdd740cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7c016640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740ccaa0 con 0x7fdd740b4210 2023-11-26T07:21:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7c016640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740ccd80 con 0x7fdd7415db40 2023-11-26T07:21:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7c016640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740cd060 con 0x7fdd74146820 2023-11-26T07:21:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd79db2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd740b4210 0x7fdd7415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd79db2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd740b4210 0x7fdd7415cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50590/0 (socket says 172.21.15.31:50590) 2023-11-26T07:21:57.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd79db2640 1 -- 172.21.15.31:0/2510185593 learned_addr learned my addr 172.21.15.31:0/2510185593 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:57.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7a5b3640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd7415db40 0x7fdd740cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd795b1640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd74146820 0x7fdd740b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd79db2640 1 -- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd74146820 msgr2=0x7fdd740b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:57.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd79db2640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd74146820 0x7fdd740b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd79db2640 1 -- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd7415db40 msgr2=0x7fdd740cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:57.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd79db2640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd7415db40 0x7fdd740cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:57.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd79db2640 1 -- 172.21.15.31:0/2510185593 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c009b30 con 0x7fdd740b4210 2023-11-26T07:21:57.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd79db2640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd740b4210 0x7fdd7415cf80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fdd700011f0 tx=0x7fdd7000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:57.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd6a7fc640 1 -- 172.21.15.31:0/2510185593 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd700074e0 con 0x7fdd740b4210 2023-11-26T07:21:57.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd6a7fc640 1 -- 172.21.15.31:0/2510185593 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70007640 con 0x7fdd740b4210 2023-11-26T07:21:57.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7c016640 1 -- 172.21.15.31:0/2510185593 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7406d030 con 0x7fdd740b4210 2023-11-26T07:21:57.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd6a7fc640 1 -- 172.21.15.31:0/2510185593 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70004a10 con 0x7fdd740b4210 2023-11-26T07:21:57.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.617+0000 7fdd7c016640 1 -- 172.21.15.31:0/2510185593 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd740cd620 con 0x7fdd740b4210 2023-11-26T07:21:57.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.621+0000 7fdd7c016640 1 -- 172.21.15.31:0/2510185593 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd740cd840 con 0x7fdd740b4210 2023-11-26T07:21:57.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.621+0000 7fdd6a7fc640 1 -- 172.21.15.31:0/2510185593 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fdd70004b70 con 0x7fdd740b4210 2023-11-26T07:21:57.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.621+0000 7fdd6a7fc640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd580667d0 0x7fdd58068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:57.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.621+0000 7fdd6a7fc640 1 -- 172.21.15.31:0/2510185593 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd700839e0 con 0x7fdd740b4210 2023-11-26T07:21:57.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.621+0000 7fdd795b1640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd580667d0 0x7fdd58068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:57.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.621+0000 7fdd795b1640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd580667d0 0x7fdd58068c90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdd740cfc90 tx=0x7fdd64008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:57.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.625+0000 7fdd6a7fc640 1 -- 172.21.15.31:0/2510185593 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdd700509c0 con 0x7fdd740b4210 2023-11-26T07:21:57.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:57.817+0000 7fdd7c016640 1 -- 172.21.15.31:0/2510185593 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdd7404e850 con 0x7fdd740b4210 2023-11-26T07:21:58.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd6a7fc640 1 -- 172.21.15.31:0/2510185593 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 136+0+0 (secure 0 0 0) 0x7fdd700180c0 con 0x7fdd740b4210 2023-11-26T07:21:58.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 -- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd580667d0 msgr2=0x7fdd58068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd580667d0 0x7fdd58068c90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdd740cfc90 tx=0x7fdd64008040 comp rx=0 tx=0).stop 2023-11-26T07:21:58.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 -- 172.21.15.31:0/2510185593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd740b4210 msgr2=0x7fdd7415cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd740b4210 0x7fdd7415cf80 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fdd700011f0 tx=0x7fdd7000c9b0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 -- 172.21.15.31:0/2510185593 shutdown_connections 2023-11-26T07:21:58.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdd580667d0 0x7fdd58068c90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd7415db40 0x7fdd740cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdd74146820 0x7fdd740b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 --2- 172.21.15.31:0/2510185593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd740b4210 0x7fdd7415cf80 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 -- 172.21.15.31:0/2510185593 >> 172.21.15.31:0/2510185593 conn(0x7fdd740b2f10 msgr2=0x7fdd741408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:58.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 -- 172.21.15.31:0/2510185593 shutdown_connections 2023-11-26T07:21:58.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.449+0000 7fdd7c016640 1 -- 172.21.15.31:0/2510185593 wait complete. 2023-11-26T07:21:58.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:21:58.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.621+0000 7f42582fe640 1 Processor -- start 2023-11-26T07:21:58.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.621+0000 7f42582fe640 1 -- start start 2023-11-26T07:21:58.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f42582fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42500cb9b0 0x7f425006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f42582fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42500cbd90 0x7f425015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f42582fe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4250159630 con 0x7f42500ce5d0 2023-11-26T07:21:58.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f42582fe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425015bba0 con 0x7f42500cbd90 2023-11-26T07:21:58.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f42582fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42500cdd50 con 0x7f42500cb9b0 2023-11-26T07:21:58.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f4255899640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42500cbd90 0x7f425015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:58.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f4255899640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f42500cbd90 0x7f425015f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50604/0 (socket says 172.21.15.31:50604) 2023-11-26T07:21:58.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f4255899640 1 -- 172.21.15.31:0/616903390 learned_addr learned my addr 172.21.15.31:0/616903390 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:58.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f425609a640 1 --2- 172.21.15.31:0/616903390 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42500cb9b0 0x7f425006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:58.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f4255899640 1 -- 172.21.15.31:0/616903390 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42500cb9b0 msgr2=0x7f425006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:58.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f4255899640 1 --2- 172.21.15.31:0/616903390 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42500cb9b0 0x7f425006e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f4255899640 1 -- 172.21.15.31:0/616903390 >> v1:172.21.15.31:6789/0 conn(0x7f42500ce5d0 legacy=0x7f425015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f4255899640 1 -- 172.21.15.31:0/616903390 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425015f6d0 con 0x7f42500cbd90 2023-11-26T07:21:58.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.625+0000 7f4255098640 1 -- 172.21.15.31:0/616903390 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805758518 0 0) 0x7f4250159630 con 0x7f42500ce5d0 2023-11-26T07:21:58.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f4255899640 1 --2- 172.21.15.31:0/616903390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cbd90 0x7f425015f190 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4240009870 tx=0x7f424001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526eeedc184cfabf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:58.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f4255098640 1 -- 172.21.15.31:0/616903390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42400aa670 con 0x7f42500cbd90 2023-11-26T07:21:58.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f4255098640 1 -- 172.21.15.31:0/616903390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42400aa7d0 con 0x7f42500cbd90 2023-11-26T07:21:58.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f4255098640 1 -- 172.21.15.31:0/616903390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42400aab60 con 0x7f42500cbd90 2023-11-26T07:21:58.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 -- 172.21.15.31:0/616903390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cbd90 msgr2=0x7f425015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 --2- 172.21.15.31:0/616903390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cbd90 0x7f425015f190 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4240009870 tx=0x7f424001c510 comp rx=0 tx=0).stop 2023-11-26T07:21:58.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 -- 172.21.15.31:0/616903390 shutdown_connections 2023-11-26T07:21:58.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 --2- 172.21.15.31:0/616903390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cbd90 0x7f425015f190 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 --2- 172.21.15.31:0/616903390 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42500cb9b0 0x7f425006e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 -- 172.21.15.31:0/616903390 >> 172.21.15.31:0/616903390 conn(0x7f42500b0cb0 msgr2=0x7f42500b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:58.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 -- 172.21.15.31:0/616903390 shutdown_connections 2023-11-26T07:21:58.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 -- 172.21.15.31:0/616903390 wait complete. 2023-11-26T07:21:58.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 Processor -- start 2023-11-26T07:21:58.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 -- start start 2023-11-26T07:21:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.629+0000 7f42582fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cb9b0 0x7f42501fae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f42582fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42500cbd90 0x7f42500afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f42582fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42500ce5d0 0x7f4250153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f42582fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425015e380 con 0x7f42500cb9b0 2023-11-26T07:21:58.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f42582fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42501fa9f0 con 0x7f42500ce5d0 2023-11-26T07:21:58.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f42582fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425015e840 con 0x7f42500cbd90 2023-11-26T07:21:58.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f425609a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cb9b0 0x7f42501fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:58.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f425609a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cb9b0 0x7f42501fae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50620/0 (socket says 172.21.15.31:50620) 2023-11-26T07:21:58.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f425609a640 1 -- 172.21.15.31:0/4168886504 learned_addr learned my addr 172.21.15.31:0/4168886504 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:58.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f425609a640 1 -- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42500cbd90 msgr2=0x7f42500afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f425609a640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42500cbd90 0x7f42500afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f425609a640 1 -- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42500ce5d0 msgr2=0x7f4250153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f425609a640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42500ce5d0 0x7f4250153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f425609a640 1 -- 172.21.15.31:0/4168886504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400094d0 con 0x7f42500cb9b0 2023-11-26T07:21:58.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f425609a640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cb9b0 0x7f42501fae10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f424c00e930 tx=0x7f424c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:58.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f4246ffd640 1 -- 172.21.15.31:0/4168886504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c009690 con 0x7f42500cb9b0 2023-11-26T07:21:58.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f4246ffd640 1 -- 172.21.15.31:0/4168886504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c00ca00 con 0x7f42500cb9b0 2023-11-26T07:21:58.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f4246ffd640 1 -- 172.21.15.31:0/4168886504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c00cd70 con 0x7f42500cb9b0 2023-11-26T07:21:58.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f42582fe640 1 -- 172.21.15.31:0/4168886504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42501fa650 con 0x7f42500cb9b0 2023-11-26T07:21:58.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f42582fe640 1 -- 172.21.15.31:0/4168886504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42501fb350 con 0x7f42500cb9b0 2023-11-26T07:21:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f42582fe640 1 -- 172.21.15.31:0/4168886504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42501fac60 con 0x7f42500cb9b0 2023-11-26T07:21:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.633+0000 7f4246ffd640 1 -- 172.21.15.31:0/4168886504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f424c013470 con 0x7f42500cb9b0 2023-11-26T07:21:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.637+0000 7f4246ffd640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42340667d0 0x7f4234068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:58.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.637+0000 7f4246ffd640 1 -- 172.21.15.31:0/4168886504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f424c0845c0 con 0x7f42500cb9b0 2023-11-26T07:21:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.637+0000 7f4255899640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42340667d0 0x7f4234068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:58.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.637+0000 7f4255899640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42340667d0 0x7f4234068c90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f42400ae000 tx=0x7f4240008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:58.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.641+0000 7f4246ffd640 1 -- 172.21.15.31:0/4168886504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f424c051740 con 0x7f42500cb9b0 2023-11-26T07:21:58.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.825+0000 7f42582fe640 1 -- 172.21.15.31:0/4168886504 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f425015caf0 con 0x7f42340667d0 2023-11-26T07:21:58.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f4246ffd640 1 -- 172.21.15.31:0/4168886504 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f42400aa610 con 0x7f42340667d0 2023-11-26T07:21:58.831 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:21:58.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:21:58.832 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-11-26T07:21:58.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:21:58.832 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:21:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 -- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42340667d0 msgr2=0x7f4234068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42340667d0 0x7f4234068c90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f42400ae000 tx=0x7f4240008040 comp rx=0 tx=0).stop 2023-11-26T07:21:58.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 -- 172.21.15.31:0/4168886504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cb9b0 msgr2=0x7f42501fae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cb9b0 0x7f42501fae10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f424c00e930 tx=0x7f424c00edf0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 -- 172.21.15.31:0/4168886504 shutdown_connections 2023-11-26T07:21:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f42340667d0 0x7f4234068c90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42500ce5d0 0x7f4250153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f42500cbd90 0x7f42500afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 --2- 172.21.15.31:0/4168886504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42500cb9b0 0x7f42501fae10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 -- 172.21.15.31:0/4168886504 >> 172.21.15.31:0/4168886504 conn(0x7f42500b0cb0 msgr2=0x7f42501581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 -- 172.21.15.31:0/4168886504 shutdown_connections 2023-11-26T07:21:58.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.829+0000 7f42582fe640 1 -- 172.21.15.31:0/4168886504 wait complete. 2023-11-26T07:21:58.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:21:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3d15f5640 1 Processor -- start 2023-11-26T07:21:58.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3d15f5640 1 -- start start 2023-11-26T07:21:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3d15f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 0x7fa3cc15df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3d15f5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa3cc15e4d0 0x7fa3cc0ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3d15f5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc0cd870 con 0x7fa3cc154940 2023-11-26T07:21:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3d15f5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc154d20 con 0x7fa3cc15e4d0 2023-11-26T07:21:59.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3d15f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15ecc0 con 0x7fa3cc06e120 2023-11-26T07:21:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3caffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa3cc154940 0x7fa3cc06da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53872/0 (socket says 172.21.15.31:53872) 2023-11-26T07:21:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3caffd640 1 -- 172.21.15.31:0/3464899015 learned_addr learned my addr 172.21.15.31:0/3464899015 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3cb7fe640 1 --2- 172.21.15.31:0/3464899015 >> v2:172.21.15.97:3300/0 conn(0x7fa3cc15e4d0 0x7fa3cc0ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3c3fff640 1 --2- 172.21.15.31:0/3464899015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 0x7fa3cc15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3ca7fc640 1 -- 172.21.15.31:0/3464899015 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163944766 0 0) 0x7fa3cc0cd870 con 0x7fa3cc154940 2023-11-26T07:21:59.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3ca7fc640 1 -- 172.21.15.31:0/3464899015 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3b8003530 con 0x7fa3cc154940 2023-11-26T07:21:59.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3c3fff640 1 -- 172.21.15.31:0/3464899015 >> v2:172.21.15.97:3300/0 conn(0x7fa3cc15e4d0 msgr2=0x7fa3cc0ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:59.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3c3fff640 1 --2- 172.21.15.31:0/3464899015 >> v2:172.21.15.97:3300/0 conn(0x7fa3cc15e4d0 0x7fa3cc0ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3c3fff640 1 -- 172.21.15.31:0/3464899015 >> v1:172.21.15.31:6789/0 conn(0x7fa3cc154940 legacy=0x7fa3cc06da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:58.997+0000 7fa3c3fff640 1 -- 172.21.15.31:0/3464899015 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc0ccf40 con 0x7fa3cc06e120 2023-11-26T07:21:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3c3fff640 1 --2- 172.21.15.31:0/3464899015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 0x7fa3cc15df90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40097c0 tx=0x7fa3b401c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba83cbb15b5deb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3ca7fc640 1 -- 172.21.15.31:0/3464899015 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b40ab600 con 0x7fa3cc06e120 2023-11-26T07:21:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3ca7fc640 1 -- 172.21.15.31:0/3464899015 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b40a7050 con 0x7fa3cc06e120 2023-11-26T07:21:59.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3ca7fc640 1 -- 172.21.15.31:0/3464899015 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b40ab980 con 0x7fa3cc06e120 2023-11-26T07:21:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3d15f5640 1 -- 172.21.15.31:0/3464899015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 msgr2=0x7fa3cc15df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3d15f5640 1 --2- 172.21.15.31:0/3464899015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 0x7fa3cc15df90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40097c0 tx=0x7fa3b401c270 comp rx=0 tx=0).stop 2023-11-26T07:21:59.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3d15f5640 1 -- 172.21.15.31:0/3464899015 shutdown_connections 2023-11-26T07:21:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3d15f5640 1 --2- 172.21.15.31:0/3464899015 >> v2:172.21.15.97:3300/0 conn(0x7fa3cc15e4d0 0x7fa3cc0ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3d15f5640 1 --2- 172.21.15.31:0/3464899015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 0x7fa3cc15df90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3d15f5640 1 -- 172.21.15.31:0/3464899015 >> 172.21.15.31:0/3464899015 conn(0x7fa3cc0b0cb0 msgr2=0x7fa3cc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3d15f5640 1 -- 172.21.15.31:0/3464899015 shutdown_connections 2023-11-26T07:21:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.001+0000 7fa3d15f5640 1 -- 172.21.15.31:0/3464899015 wait complete. 2023-11-26T07:21:59.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3d15f5640 1 Processor -- start 2023-11-26T07:21:59.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3d15f5640 1 -- start start 2023-11-26T07:21:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3d15f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 0x7fa3cc13ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3d15f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3cc154940 0x7fa3cc14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3d15f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc15e4d0 0x7fa3cc14f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3d15f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc156760 con 0x7fa3cc154940 2023-11-26T07:21:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3d15f5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc155870 con 0x7fa3cc15e4d0 2023-11-26T07:21:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3d15f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc155b50 con 0x7fa3cc06e120 2023-11-26T07:21:59.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3cc154940 0x7fa3cc14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3cc154940 0x7fa3cc14ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50632/0 (socket says 172.21.15.31:50632) 2023-11-26T07:21:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c3fff640 1 -- 172.21.15.31:0/2867990840 learned_addr learned my addr 172.21.15.31:0/2867990840 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3caffd640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 0x7fa3cc13ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3cb7fe640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc15e4d0 0x7fa3cc14f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c3fff640 1 -- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 msgr2=0x7fa3cc13ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c3fff640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 0x7fa3cc13ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c3fff640 1 -- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc15e4d0 msgr2=0x7fa3cc14f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c3fff640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc15e4d0 0x7fa3cc14f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c3fff640 1 -- 172.21.15.31:0/2867990840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b4009420 con 0x7fa3cc154940 2023-11-26T07:21:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3caffd640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 0x7fa3cc13ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3cb7fe640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc15e4d0 0x7fa3cc14f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:59.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c3fff640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3cc154940 0x7fa3cc14ebf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4018e80 tx=0x7fa3b4019010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c37fe640 1 -- 172.21.15.31:0/2867990840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b40b4400 con 0x7fa3cc154940 2023-11-26T07:21:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c37fe640 1 -- 172.21.15.31:0/2867990840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b40b4560 con 0x7fa3cc154940 2023-11-26T07:21:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3d15f5640 1 -- 172.21.15.31:0/2867990840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc155e30 con 0x7fa3cc154940 2023-11-26T07:21:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3c37fe640 1 -- 172.21.15.31:0/2867990840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b40b4910 con 0x7fa3cc154940 2023-11-26T07:21:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.005+0000 7fa3d15f5640 1 -- 172.21.15.31:0/2867990840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc1563f0 con 0x7fa3cc154940 2023-11-26T07:21:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.009+0000 7fa3d15f5640 1 -- 172.21.15.31:0/2867990840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3cc0671f0 con 0x7fa3cc154940 2023-11-26T07:21:59.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.009+0000 7fa3c37fe640 1 -- 172.21.15.31:0/2867990840 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fa3b40b2040 con 0x7fa3cc154940 2023-11-26T07:21:59.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.009+0000 7fa3c37fe640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa3ac0667d0 0x7fa3ac068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.009+0000 7fa3c37fe640 1 -- 172.21.15.31:0/2867990840 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3b411e0f0 con 0x7fa3cc154940 2023-11-26T07:21:59.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.009+0000 7fa3caffd640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa3ac0667d0 0x7fa3ac068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.009+0000 7fa3caffd640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa3ac0667d0 0x7fa3ac068c90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc005890 tx=0x7fa3bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.013+0000 7fa3c37fe640 1 -- 172.21.15.31:0/2867990840 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa3b40eb050 con 0x7fa3cc154940 2023-11-26T07:21:59.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.169+0000 7fa3d15f5640 1 -- 172.21.15.31:0/2867990840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa3cc06b460 con 0x7fa3cc154940 2023-11-26T07:21:59.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.169+0000 7fa3c37fe640 1 -- 172.21.15.31:0/2867990840 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa3b40ee670 con 0x7fa3cc154940 2023-11-26T07:21:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 -- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa3ac0667d0 msgr2=0x7fa3ac068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa3ac0667d0 0x7fa3ac068c90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc005890 tx=0x7fa3bc009040 comp rx=0 tx=0).stop 2023-11-26T07:21:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 -- 172.21.15.31:0/2867990840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3cc154940 msgr2=0x7fa3cc14ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3cc154940 0x7fa3cc14ebf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4018e80 tx=0x7fa3b4019010 comp rx=0 tx=0).stop 2023-11-26T07:21:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 -- 172.21.15.31:0/2867990840 shutdown_connections 2023-11-26T07:21:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa3ac0667d0 0x7fa3ac068c90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3cc15e4d0 0x7fa3cc14f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3cc154940 0x7fa3cc14ebf0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 --2- 172.21.15.31:0/2867990840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa3cc06e120 0x7fa3cc13ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 -- 172.21.15.31:0/2867990840 >> 172.21.15.31:0/2867990840 conn(0x7fa3cc0b0cb0 msgr2=0x7fa3cc0b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 -- 172.21.15.31:0/2867990840 shutdown_connections 2023-11-26T07:21:59.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.173+0000 7fa3d15f5640 1 -- 172.21.15.31:0/2867990840 wait complete. 2023-11-26T07:21:59.195 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:21:59.195 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:59.205 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:21:59.205 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:21:59.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:21:59.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.441+0000 7f74c886b640 1 Processor -- start 2023-11-26T07:21:59.445 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.441+0000 7f74c886b640 1 -- start start 2023-11-26T07:21:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.441+0000 7f74c886b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74c4144c80 0x7f74c40abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.441+0000 7f74c886b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74c4070df0 0x7f74c406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.441+0000 7f74c886b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c4145060 con 0x7f74c40ac460 2023-11-26T07:21:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.441+0000 7f74c886b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c4071fd0 con 0x7f74c4144c80 2023-11-26T07:21:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.441+0000 7f74c886b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c40ac8c0 con 0x7f74c4070df0 2023-11-26T07:21:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.441+0000 7f74c259c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74c4144c80 0x7f74c40abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.445+0000 7f74c259c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f74c4144c80 0x7f74c40abf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50640/0 (socket says 172.21.15.31:50640) 2023-11-26T07:21:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.445+0000 7f74c259c640 1 -- 172.21.15.31:0/4136877387 learned_addr learned my addr 172.21.15.31:0/4136877387 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.445+0000 7f74c259c640 1 -- 172.21.15.31:0/4136877387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74c4070df0 msgr2=0x7f74c406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.445+0000 7f74c259c640 1 --2- 172.21.15.31:0/4136877387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74c4070df0 0x7f74c406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.445+0000 7f74c259c640 1 -- 172.21.15.31:0/4136877387 >> v1:172.21.15.31:6789/0 conn(0x7f74c40ac460 legacy=0x7f74c40706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.445+0000 7f74c259c640 1 -- 172.21.15.31:0/4136877387 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c4071450 con 0x7f74c4144c80 2023-11-26T07:21:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.445+0000 7f74c259c640 1 --2- 172.21.15.31:0/4136877387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c4144c80 0x7f74c40abf20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009870 tx=0x7f74b401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83c8b579cd0c78d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.445+0000 7f74c1d9b640 1 -- 172.21.15.31:0/4136877387 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b40a7070 con 0x7f74c4144c80 2023-11-26T07:21:59.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.445+0000 7f74c1d9b640 1 -- 172.21.15.31:0/4136877387 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b40ab430 con 0x7f74c4144c80 2023-11-26T07:21:59.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.445+0000 7f74c1d9b640 1 -- 172.21.15.31:0/4136877387 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b40ab7c0 con 0x7f74c4144c80 2023-11-26T07:21:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 -- 172.21.15.31:0/4136877387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c4144c80 msgr2=0x7f74c40abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 --2- 172.21.15.31:0/4136877387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c4144c80 0x7f74c40abf20 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009870 tx=0x7f74b401c540 comp rx=0 tx=0).stop 2023-11-26T07:21:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 -- 172.21.15.31:0/4136877387 shutdown_connections 2023-11-26T07:21:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 --2- 172.21.15.31:0/4136877387 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74c4070df0 0x7f74c406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 --2- 172.21.15.31:0/4136877387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c4144c80 0x7f74c40abf20 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 -- 172.21.15.31:0/4136877387 >> 172.21.15.31:0/4136877387 conn(0x7f74c40b0c40 msgr2=0x7f74c40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 -- 172.21.15.31:0/4136877387 shutdown_connections 2023-11-26T07:21:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 -- 172.21.15.31:0/4136877387 wait complete. 2023-11-26T07:21:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 Processor -- start 2023-11-26T07:21:59.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 -- start start 2023-11-26T07:21:59.454 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74c4070df0 0x7f74c413a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c40ac460 0x7f74c415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c4144c80 0x7f74c414b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c41592c0 con 0x7f74c40ac460 2023-11-26T07:21:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4140530 con 0x7f74c4144c80 2023-11-26T07:21:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c886b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4141360 con 0x7f74c4070df0 2023-11-26T07:21:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c40ac460 0x7f74c415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c40ac460 0x7f74c415ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50646/0 (socket says 172.21.15.31:50646) 2023-11-26T07:21:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74bbfff640 1 -- 172.21.15.31:0/3904594059 learned_addr learned my addr 172.21.15.31:0/3904594059 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c2d9d640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c4144c80 0x7f74c414b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.449+0000 7f74c259c640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74c4070df0 0x7f74c413a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74bbfff640 1 -- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74c4070df0 msgr2=0x7f74c413a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74bbfff640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74c4070df0 0x7f74c413a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74bbfff640 1 -- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c4144c80 msgr2=0x7f74c414b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74bbfff640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c4144c80 0x7f74c414b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74bbfff640 1 -- 172.21.15.31:0/3904594059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b40094d0 con 0x7f74c40ac460 2023-11-26T07:21:59.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74c259c640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74c4070df0 0x7f74c413a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:59.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74bbfff640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c40ac460 0x7f74c415ac10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00c500 tx=0x7f74ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74baffd640 1 -- 172.21.15.31:0/3904594059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac005220 con 0x7f74c40ac460 2023-11-26T07:21:59.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74baffd640 1 -- 172.21.15.31:0/3904594059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac005380 con 0x7f74c40ac460 2023-11-26T07:21:59.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74c886b640 1 -- 172.21.15.31:0/3904594059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c41407e0 con 0x7f74c40ac460 2023-11-26T07:21:59.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74baffd640 1 -- 172.21.15.31:0/3904594059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac007270 con 0x7f74c40ac460 2023-11-26T07:21:59.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74c886b640 1 -- 172.21.15.31:0/3904594059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c4140da0 con 0x7f74c40ac460 2023-11-26T07:21:59.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74c886b640 1 -- 172.21.15.31:0/3904594059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c40b1ba0 con 0x7f74c40ac460 2023-11-26T07:21:59.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74baffd640 1 -- 172.21.15.31:0/3904594059 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f74ac0073d0 con 0x7f74c40ac460 2023-11-26T07:21:59.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74baffd640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f74a40667d0 0x7f74a4068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.453+0000 7f74baffd640 1 -- 172.21.15.31:0/3904594059 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74ac083c60 con 0x7f74c40ac460 2023-11-26T07:21:59.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.457+0000 7f74c259c640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f74a40667d0 0x7f74a4068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.457+0000 7f74c259c640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f74a40667d0 0x7f74a4068c90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f74b40af000 tx=0x7f74b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.457+0000 7f74baffd640 1 -- 172.21.15.31:0/3904594059 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f74ac050ca0 con 0x7f74c40ac460 2023-11-26T07:21:59.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 -- 172.21.15.31:0/3904594059 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74c404e850 con 0x7f74c40ac460 2023-11-26T07:21:59.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74baffd640 1 -- 172.21.15.31:0/3904594059 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16484 (secure 0 0 0) 0x7f74ac0542c0 con 0x7f74c40ac460 2023-11-26T07:21:59.624 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:21:59.624 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":68,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:50.369840+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:46.570507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:47.773615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:21:59.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 -- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f74a40667d0 msgr2=0x7f74a4068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f74a40667d0 0x7f74a4068c90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f74b40af000 tx=0x7f74b4008040 comp rx=0 tx=0).stop 2023-11-26T07:21:59.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 -- 172.21.15.31:0/3904594059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c40ac460 msgr2=0x7f74c415ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c40ac460 0x7f74c415ac10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00c500 tx=0x7f74ac00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 -- 172.21.15.31:0/3904594059 shutdown_connections 2023-11-26T07:21:59.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f74a40667d0 0x7f74a4068c90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74c4144c80 0x7f74c414b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74c40ac460 0x7f74c415ac10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 --2- 172.21.15.31:0/3904594059 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f74c4070df0 0x7f74c413a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 -- 172.21.15.31:0/3904594059 >> 172.21.15.31:0/3904594059 conn(0x7f74c40b0c40 msgr2=0x7f74c40afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:59.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 -- 172.21.15.31:0/3904594059 shutdown_connections 2023-11-26T07:21:59.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.621+0000 7f74c886b640 1 -- 172.21.15.31:0/3904594059 wait complete. 2023-11-26T07:21:59.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:21:46.570507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:21:47.773615+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:21:59.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:21:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe5849f8640 1 Processor -- start 2023-11-26T07:21:59.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe5849f8640 1 -- start start 2023-11-26T07:21:59.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe5849f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe5800cb9b0 0x7fe58006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe5849f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 0x7fe58015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe5849f8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe580159630 con 0x7fe5800ce5d0 2023-11-26T07:21:59.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe5849f8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58015bba0 con 0x7fe5800cb9b0 2023-11-26T07:21:59.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe5849f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800cdd50 con 0x7fe5800cbd90 2023-11-26T07:21:59.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe57ed9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe5800ce5d0 0x7fe58015bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53892/0 (socket says 172.21.15.31:53892) 2023-11-26T07:21:59.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe57ed9d640 1 -- 172.21.15.31:0/2416015379 learned_addr learned my addr 172.21.15.31:0/2416015379 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:21:59.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe577fff640 1 --2- 172.21.15.31:0/2416015379 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 0x7fe58015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe57e59c640 1 --2- 172.21.15.31:0/2416015379 >> v2:172.21.15.97:3300/0 conn(0x7fe5800cb9b0 0x7fe58006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe57dd9b640 1 -- 172.21.15.31:0/2416015379 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645050759 0 0) 0x7fe580159630 con 0x7fe5800ce5d0 2023-11-26T07:21:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.789+0000 7fe57dd9b640 1 -- 172.21.15.31:0/2416015379 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe56c003530 con 0x7fe5800ce5d0 2023-11-26T07:21:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.793+0000 7fe57dd9b640 1 -- 172.21.15.31:0/2416015379 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3629965757 0 0) 0x7fe56c003530 con 0x7fe5800ce5d0 2023-11-26T07:21:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.793+0000 7fe577fff640 1 -- 172.21.15.31:0/2416015379 >> v2:172.21.15.97:3300/0 conn(0x7fe5800cb9b0 msgr2=0x7fe58006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:21:59.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.793+0000 7fe577fff640 1 --2- 172.21.15.31:0/2416015379 >> v2:172.21.15.97:3300/0 conn(0x7fe5800cb9b0 0x7fe58006e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.793+0000 7fe577fff640 1 -- 172.21.15.31:0/2416015379 >> v1:172.21.15.31:6789/0 conn(0x7fe5800ce5d0 legacy=0x7fe58015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.793+0000 7fe577fff640 1 -- 172.21.15.31:0/2416015379 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58015f6d0 con 0x7fe5800cbd90 2023-11-26T07:21:59.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.793+0000 7fe577fff640 1 --2- 172.21.15.31:0/2416015379 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 0x7fe58015f190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5680097c0 tx=0x7fe56801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88cfc13bdbf1a593 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.793+0000 7fe57dd9b640 1 -- 172.21.15.31:0/2416015379 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578001a00 con 0x7fe5800cbd90 2023-11-26T07:21:59.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.793+0000 7fe57dd9b640 1 -- 172.21.15.31:0/2416015379 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5680a7050 con 0x7fe5800cbd90 2023-11-26T07:21:59.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.793+0000 7fe57dd9b640 1 -- 172.21.15.31:0/2416015379 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680ab620 con 0x7fe5800cbd90 2023-11-26T07:21:59.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 -- 172.21.15.31:0/2416015379 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 msgr2=0x7fe58015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 --2- 172.21.15.31:0/2416015379 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 0x7fe58015f190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe5680097c0 tx=0x7fe56801c370 comp rx=0 tx=0).stop 2023-11-26T07:21:59.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 -- 172.21.15.31:0/2416015379 shutdown_connections 2023-11-26T07:21:59.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 --2- 172.21.15.31:0/2416015379 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 0x7fe58015f190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 --2- 172.21.15.31:0/2416015379 >> v2:172.21.15.97:3300/0 conn(0x7fe5800cb9b0 0x7fe58006e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 -- 172.21.15.31:0/2416015379 >> 172.21.15.31:0/2416015379 conn(0x7fe5800b0cb0 msgr2=0x7fe5800b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:59.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 -- 172.21.15.31:0/2416015379 shutdown_connections 2023-11-26T07:21:59.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 -- 172.21.15.31:0/2416015379 wait complete. 2023-11-26T07:21:59.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 Processor -- start 2023-11-26T07:21:59.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 -- start start 2023-11-26T07:21:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5800cb9b0 0x7fe58009ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 0x7fe58009a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5800ce5d0 0x7fe58009a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800b03c0 con 0x7fe5800ce5d0 2023-11-26T07:21:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800af740 con 0x7fe5800cb9b0 2023-11-26T07:21:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.797+0000 7fe5849f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800afa20 con 0x7fe5800cbd90 2023-11-26T07:21:59.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5800ce5d0 0x7fe58009a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5800ce5d0 0x7fe58009a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50662/0 (socket says 172.21.15.31:50662) 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57ed9d640 1 -- 172.21.15.31:0/50758877 learned_addr learned my addr 172.21.15.31:0/50758877 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe577fff640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 0x7fe58009a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57e59c640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5800cb9b0 0x7fe58009ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57ed9d640 1 -- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 msgr2=0x7fe58009a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57ed9d640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 0x7fe58009a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57ed9d640 1 -- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5800cb9b0 msgr2=0x7fe58009ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57ed9d640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5800cb9b0 0x7fe58009ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57ed9d640 1 -- 172.21.15.31:0/50758877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe568009420 con 0x7fe5800ce5d0 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57e59c640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5800cb9b0 0x7fe58009ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:21:59.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe57ed9d640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5800ce5d0 0x7fe58009a920 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe57800cc70 tx=0x7fe57800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe576ffd640 1 -- 172.21.15.31:0/50758877 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57800ce80 con 0x7fe5800ce5d0 2023-11-26T07:21:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe576ffd640 1 -- 172.21.15.31:0/50758877 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578012400 con 0x7fe5800ce5d0 2023-11-26T07:21:59.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe5849f8640 1 -- 172.21.15.31:0/50758877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5800afd00 con 0x7fe5800ce5d0 2023-11-26T07:21:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe576ffd640 1 -- 172.21.15.31:0/50758877 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5780127b0 con 0x7fe5800ce5d0 2023-11-26T07:21:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe5849f8640 1 -- 172.21.15.31:0/50758877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58009cf40 con 0x7fe5800ce5d0 2023-11-26T07:21:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe5849f8640 1 -- 172.21.15.31:0/50758877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5800671f0 con 0x7fe5800ce5d0 2023-11-26T07:21:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe576ffd640 1 -- 172.21.15.31:0/50758877 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fe578012910 con 0x7fe5800ce5d0 2023-11-26T07:21:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.801+0000 7fe576ffd640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe5600667d0 0x7fe560068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:21:59.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.805+0000 7fe576ffd640 1 -- 172.21.15.31:0/50758877 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe578084ae0 con 0x7fe5800ce5d0 2023-11-26T07:21:59.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.805+0000 7fe57e59c640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe5600667d0 0x7fe560068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:21:59.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.805+0000 7fe57e59c640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe5600667d0 0x7fe560068c90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe570009bb0 tx=0x7fe570008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:21:59.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.805+0000 7fe576ffd640 1 -- 172.21.15.31:0/50758877 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe578051aa0 con 0x7fe5800ce5d0 2023-11-26T07:21:59.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 -- 172.21.15.31:0/50758877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5800b0500 con 0x7fe5800ce5d0 2023-11-26T07:21:59.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe576ffd640 1 -- 172.21.15.31:0/50758877 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5780550c0 con 0x7fe5800ce5d0 2023-11-26T07:21:59.967 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:21:59.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 -- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe5600667d0 msgr2=0x7fe560068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe5600667d0 0x7fe560068c90 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe570009bb0 tx=0x7fe570008040 comp rx=0 tx=0).stop 2023-11-26T07:21:59.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 -- 172.21.15.31:0/50758877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5800ce5d0 msgr2=0x7fe58009a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:21:59.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5800ce5d0 0x7fe58009a920 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe57800cc70 tx=0x7fe57800a430 comp rx=0 tx=0).stop 2023-11-26T07:21:59.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 -- 172.21.15.31:0/50758877 shutdown_connections 2023-11-26T07:21:59.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fe5600667d0 0x7fe560068c90 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5800ce5d0 0x7fe58009a920 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe5800cbd90 0x7fe58009a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 --2- 172.21.15.31:0/50758877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5800cb9b0 0x7fe58009ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:21:59.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 -- 172.21.15.31:0/50758877 >> 172.21.15.31:0/50758877 conn(0x7fe5800b0cb0 msgr2=0x7fe580165710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:21:59.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 -- 172.21.15.31:0/50758877 shutdown_connections 2023-11-26T07:21:59.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:21:59.965+0000 7fe5849f8640 1 -- 172.21.15.31:0/50758877 wait complete. 2023-11-26T07:21:59.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:00.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.129+0000 7fdf767df640 1 Processor -- start 2023-11-26T07:22:00.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.129+0000 7fdf767df640 1 -- start start 2023-11-26T07:22:00.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf767df640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf7006e3f0 0x7fdf7015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf767df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 0x7fdf7015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf767df640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf70159630 con 0x7fdf700cb9b0 2023-11-26T07:22:00.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf767df640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7006e860 con 0x7fdf7006e3f0 2023-11-26T07:22:00.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf767df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7015bba0 con 0x7fdf700ce5d0 2023-11-26T07:22:00.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf6ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdf700cb9b0 0x7fdf700cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53902/0 (socket says 172.21.15.31:53902) 2023-11-26T07:22:00.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf6ffff640 1 -- 172.21.15.31:0/3603902913 learned_addr learned my addr 172.21.15.31:0/3603902913 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:00.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf74d7c640 1 --2- 172.21.15.31:0/3603902913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 0x7fdf7015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf6f7fe640 1 --2- 172.21.15.31:0/3603902913 >> v2:172.21.15.97:3300/0 conn(0x7fdf7006e3f0 0x7fdf7015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf6effd640 1 -- 172.21.15.31:0/3603902913 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233849054 0 0) 0x7fdf70159630 con 0x7fdf700cb9b0 2023-11-26T07:22:00.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf6effd640 1 -- 172.21.15.31:0/3603902913 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf58003530 con 0x7fdf700cb9b0 2023-11-26T07:22:00.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf74d7c640 1 -- 172.21.15.31:0/3603902913 >> v2:172.21.15.97:3300/0 conn(0x7fdf7006e3f0 msgr2=0x7fdf7015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:00.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf74d7c640 1 --2- 172.21.15.31:0/3603902913 >> v2:172.21.15.97:3300/0 conn(0x7fdf7006e3f0 0x7fdf7015f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf74d7c640 1 -- 172.21.15.31:0/3603902913 >> v1:172.21.15.31:6789/0 conn(0x7fdf700cb9b0 legacy=0x7fdf700cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf74d7c640 1 -- 172.21.15.31:0/3603902913 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7015f6d0 con 0x7fdf700ce5d0 2023-11-26T07:22:00.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.133+0000 7fdf6f7fe640 1 --2- 172.21.15.31:0/3603902913 >> v2:172.21.15.97:3300/0 conn(0x7fdf7006e3f0 0x7fdf7015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:00.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf74d7c640 1 --2- 172.21.15.31:0/3603902913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 0x7fdf7015bfd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009d90 tx=0x7fdf6801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2c875b62231e35e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf6effd640 1 -- 172.21.15.31:0/3603902913 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf680a58e0 con 0x7fdf700ce5d0 2023-11-26T07:22:00.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf6effd640 1 -- 172.21.15.31:0/3603902913 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf680a5a40 con 0x7fdf700ce5d0 2023-11-26T07:22:00.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf6effd640 1 -- 172.21.15.31:0/3603902913 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf680a5dd0 con 0x7fdf700ce5d0 2023-11-26T07:22:00.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf767df640 1 -- 172.21.15.31:0/3603902913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 msgr2=0x7fdf7015bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf767df640 1 --2- 172.21.15.31:0/3603902913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 0x7fdf7015bfd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009d90 tx=0x7fdf6801c950 comp rx=0 tx=0).stop 2023-11-26T07:22:00.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf767df640 1 -- 172.21.15.31:0/3603902913 shutdown_connections 2023-11-26T07:22:00.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf767df640 1 --2- 172.21.15.31:0/3603902913 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 0x7fdf7015bfd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf767df640 1 --2- 172.21.15.31:0/3603902913 >> v2:172.21.15.97:3300/0 conn(0x7fdf7006e3f0 0x7fdf7015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf767df640 1 -- 172.21.15.31:0/3603902913 >> 172.21.15.31:0/3603902913 conn(0x7fdf700b0cb0 msgr2=0x7fdf700b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf767df640 1 -- 172.21.15.31:0/3603902913 shutdown_connections 2023-11-26T07:22:00.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf767df640 1 -- 172.21.15.31:0/3603902913 wait complete. 2023-11-26T07:22:00.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf767df640 1 Processor -- start 2023-11-26T07:22:00.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.137+0000 7fdf767df640 1 -- start start 2023-11-26T07:22:00.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf767df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf7006e3f0 0x7fdf701517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf767df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf700cb9b0 0x7fdf70156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf767df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 0x7fdf7014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf767df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7015b670 con 0x7fdf7006e3f0 2023-11-26T07:22:00.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf767df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7014dac0 con 0x7fdf700cb9b0 2023-11-26T07:22:00.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf767df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf70156630 con 0x7fdf700ce5d0 2023-11-26T07:22:00.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf7006e3f0 0x7fdf701517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf7006e3f0 0x7fdf701517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50674/0 (socket says 172.21.15.31:50674) 2023-11-26T07:22:00.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6ffff640 1 -- 172.21.15.31:0/913379050 learned_addr learned my addr 172.21.15.31:0/913379050 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:00.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6f7fe640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf700cb9b0 0x7fdf70156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf74d7c640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 0x7fdf7014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6ffff640 1 -- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 msgr2=0x7fdf7014b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6ffff640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 0x7fdf7014b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6ffff640 1 -- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf700cb9b0 msgr2=0x7fdf70156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6ffff640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf700cb9b0 0x7fdf70156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6ffff640 1 -- 172.21.15.31:0/913379050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64009420 con 0x7fdf7006e3f0 2023-11-26T07:22:00.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6f7fe640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf700cb9b0 0x7fdf70156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:00.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6ffff640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf7006e3f0 0x7fdf701517d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c450 tx=0x7fdf6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6cff9640 1 -- 172.21.15.31:0/913379050 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60005cf0 con 0x7fdf7006e3f0 2023-11-26T07:22:00.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6cff9640 1 -- 172.21.15.31:0/913379050 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf60005e50 con 0x7fdf7006e3f0 2023-11-26T07:22:00.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf6cff9640 1 -- 172.21.15.31:0/913379050 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60007290 con 0x7fdf7006e3f0 2023-11-26T07:22:00.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf767df640 1 -- 172.21.15.31:0/913379050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf68009a70 con 0x7fdf7006e3f0 2023-11-26T07:22:00.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf767df640 1 -- 172.21.15.31:0/913379050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf70156350 con 0x7fdf7006e3f0 2023-11-26T07:22:00.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.141+0000 7fdf767df640 1 -- 172.21.15.31:0/913379050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf700671f0 con 0x7fdf7006e3f0 2023-11-26T07:22:00.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.145+0000 7fdf6cff9640 1 -- 172.21.15.31:0/913379050 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fdf600073f0 con 0x7fdf7006e3f0 2023-11-26T07:22:00.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.145+0000 7fdf6cff9640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdf5c066800 0x7fdf5c068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.145+0000 7fdf6cff9640 1 -- 172.21.15.31:0/913379050 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf60083800 con 0x7fdf7006e3f0 2023-11-26T07:22:00.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.145+0000 7fdf6f7fe640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdf5c066800 0x7fdf5c068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.145+0000 7fdf6f7fe640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdf5c066800 0x7fdf5c068cc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdf7015f5c0 tx=0x7fdf64009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.149+0000 7fdf6cff9640 1 -- 172.21.15.31:0/913379050 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdf600509b0 con 0x7fdf7006e3f0 2023-11-26T07:22:00.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.305+0000 7fdf767df640 1 -- 172.21.15.31:0/913379050 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf7006b460 con 0x7fdf7006e3f0 2023-11-26T07:22:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.305+0000 7fdf6cff9640 1 -- 172.21.15.31:0/913379050 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf600170a0 con 0x7fdf7006e3f0 2023-11-26T07:22:00.309 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:22:00.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 -- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdf5c066800 msgr2=0x7fdf5c068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdf5c066800 0x7fdf5c068cc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdf7015f5c0 tx=0x7fdf64009190 comp rx=0 tx=0).stop 2023-11-26T07:22:00.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 -- 172.21.15.31:0/913379050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf7006e3f0 msgr2=0x7fdf701517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf7006e3f0 0x7fdf701517d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000c450 tx=0x7fdf6000c910 comp rx=0 tx=0).stop 2023-11-26T07:22:00.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 -- 172.21.15.31:0/913379050 shutdown_connections 2023-11-26T07:22:00.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fdf5c066800 0x7fdf5c068cc0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf700ce5d0 0x7fdf7014b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf700cb9b0 0x7fdf70156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 --2- 172.21.15.31:0/913379050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf7006e3f0 0x7fdf701517d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 -- 172.21.15.31:0/913379050 >> 172.21.15.31:0/913379050 conn(0x7fdf700b0cb0 msgr2=0x7fdf700b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 -- 172.21.15.31:0/913379050 shutdown_connections 2023-11-26T07:22:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.309+0000 7fdf767df640 1 -- 172.21.15.31:0/913379050 wait complete. 2023-11-26T07:22:00.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:00.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.473+0000 7f867a5a4640 1 Processor -- start 2023-11-26T07:22:00.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.473+0000 7f867a5a4640 1 -- start start 2023-11-26T07:22:00.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f867a5a4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f867415d2a0 0x7f86740d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f867a5a4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740d2c20 0x7f86740b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f867a5a4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86740b6250 con 0x7f86740b5180 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f867a5a4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86740ce8c0 con 0x7f867415d2a0 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f867a5a4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8674161cf0 con 0x7f86740d2c20 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f86795a2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f867415d2a0 0x7f86740d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f86795a2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f867415d2a0 0x7f86740d26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50682/0 (socket says 172.21.15.31:50682) 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f86795a2640 1 -- 172.21.15.31:0/4242524502 learned_addr learned my addr 172.21.15.31:0/4242524502 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f8678da1640 1 --2- 172.21.15.31:0/4242524502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740d2c20 0x7f86740b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f86795a2640 1 -- 172.21.15.31:0/4242524502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740d2c20 msgr2=0x7f86740b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f86795a2640 1 --2- 172.21.15.31:0/4242524502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740d2c20 0x7f86740b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f86795a2640 1 -- 172.21.15.31:0/4242524502 >> v1:172.21.15.97:6789/0 conn(0x7f86740b5180 legacy=0x7f867406b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f86795a2640 1 -- 172.21.15.31:0/4242524502 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86740b5590 con 0x7f867415d2a0 2023-11-26T07:22:00.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.477+0000 7f8678da1640 1 --2- 172.21.15.31:0/4242524502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740d2c20 0x7f86740b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:00.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f86795a2640 1 --2- 172.21.15.31:0/4242524502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867415d2a0 0x7f86740d26e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8670009870 tx=0x7f867001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad1017ed42fdfe0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f866bfff640 1 -- 172.21.15.31:0/4242524502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700a7070 con 0x7f867415d2a0 2023-11-26T07:22:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f866bfff640 1 -- 172.21.15.31:0/4242524502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86700ab430 con 0x7f867415d2a0 2023-11-26T07:22:00.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f866bfff640 1 -- 172.21.15.31:0/4242524502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700ab7c0 con 0x7f867415d2a0 2023-11-26T07:22:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 -- 172.21.15.31:0/4242524502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867415d2a0 msgr2=0x7f86740d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 --2- 172.21.15.31:0/4242524502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867415d2a0 0x7f86740d26e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8670009870 tx=0x7f867001c510 comp rx=0 tx=0).stop 2023-11-26T07:22:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 -- 172.21.15.31:0/4242524502 shutdown_connections 2023-11-26T07:22:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 --2- 172.21.15.31:0/4242524502 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740d2c20 0x7f86740b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 --2- 172.21.15.31:0/4242524502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867415d2a0 0x7f86740d26e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 -- 172.21.15.31:0/4242524502 >> 172.21.15.31:0/4242524502 conn(0x7f86740668a0 msgr2=0x7f86740cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 -- 172.21.15.31:0/4242524502 shutdown_connections 2023-11-26T07:22:00.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 -- 172.21.15.31:0/4242524502 wait complete. 2023-11-26T07:22:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 Processor -- start 2023-11-26T07:22:00.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 -- start start 2023-11-26T07:22:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740b5180 0x7f86741527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.481+0000 7f867a5a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86740d2c20 0x7f86741600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f867a5a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867415d2a0 0x7f86740d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f867a5a4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86741548d0 con 0x7f86740d2c20 2023-11-26T07:22:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f867a5a4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8674154380 con 0x7f867415d2a0 2023-11-26T07:22:00.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f867a5a4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8674153000 con 0x7f86740b5180 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8678da1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86740d2c20 0x7f86741600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f86795a2640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740b5180 0x7f86741527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8678da1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86740d2c20 0x7f86741600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50690/0 (socket says 172.21.15.31:50690) 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8678da1640 1 -- 172.21.15.31:0/2500417725 learned_addr learned my addr 172.21.15.31:0/2500417725 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8679da3640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867415d2a0 0x7f86740d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8678da1640 1 -- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740b5180 msgr2=0x7f86741527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8678da1640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740b5180 0x7f86741527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8678da1640 1 -- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867415d2a0 msgr2=0x7f86740d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8678da1640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867415d2a0 0x7f86740d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8678da1640 1 -- 172.21.15.31:0/2500417725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86700094d0 con 0x7f86740d2c20 2023-11-26T07:22:00.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8678da1640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86740d2c20 0x7f86741600c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f866400e810 tx=0x7f866400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8669ffb640 1 -- 172.21.15.31:0/2500417725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866400c8d0 con 0x7f86740d2c20 2023-11-26T07:22:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8669ffb640 1 -- 172.21.15.31:0/2500417725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866400ca30 con 0x7f86740d2c20 2023-11-26T07:22:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f867a5a4640 1 -- 172.21.15.31:0/2500417725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86741532e0 con 0x7f86740d2c20 2023-11-26T07:22:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8669ffb640 1 -- 172.21.15.31:0/2500417725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866400cdc0 con 0x7f86740d2c20 2023-11-26T07:22:00.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f867a5a4640 1 -- 172.21.15.31:0/2500417725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86741538a0 con 0x7f86740d2c20 2023-11-26T07:22:00.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f867a5a4640 1 -- 172.21.15.31:0/2500417725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8674074990 con 0x7f86740d2c20 2023-11-26T07:22:00.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.485+0000 7f8669ffb640 1 -- 172.21.15.31:0/2500417725 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f8664013410 con 0x7f86740d2c20 2023-11-26T07:22:00.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.489+0000 7f8669ffb640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86580667d0 0x7f8658068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.489+0000 7f8669ffb640 1 -- 172.21.15.31:0/2500417725 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8664084250 con 0x7f86740d2c20 2023-11-26T07:22:00.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.489+0000 7f86795a2640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86580667d0 0x7f8658068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.489+0000 7f86795a2640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86580667d0 0x7f8658068c90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f86700af000 tx=0x7f8670008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.493+0000 7f8669ffb640 1 -- 172.21.15.31:0/2500417725 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8664051220 con 0x7f86740d2c20 2023-11-26T07:22:00.601 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:22:00.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:22:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.649+0000 7f867a5a4640 1 -- 172.21.15.31:0/2500417725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86740b5f60 con 0x7f86740d2c20 2023-11-26T07:22:00.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.649+0000 7f8669ffb640 1 -- 172.21.15.31:0/2500417725 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8664054840 con 0x7f86740d2c20 2023-11-26T07:22:00.653 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:22:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 -- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86580667d0 msgr2=0x7f8658068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86580667d0 0x7f8658068c90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f86700af000 tx=0x7f8670008040 comp rx=0 tx=0).stop 2023-11-26T07:22:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 -- 172.21.15.31:0/2500417725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86740d2c20 msgr2=0x7f86741600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86740d2c20 0x7f86741600c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f866400e810 tx=0x7f866400ecd0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 -- 172.21.15.31:0/2500417725 shutdown_connections 2023-11-26T07:22:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f86580667d0 0x7f8658068c90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f867415d2a0 0x7f86740d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86740d2c20 0x7f86741600c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 --2- 172.21.15.31:0/2500417725 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f86740b5180 0x7f86741527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 -- 172.21.15.31:0/2500417725 >> 172.21.15.31:0/2500417725 conn(0x7f86740668a0 msgr2=0x7f86740b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 -- 172.21.15.31:0/2500417725 shutdown_connections 2023-11-26T07:22:00.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.653+0000 7f867a5a4640 1 -- 172.21.15.31:0/2500417725 wait complete. 2023-11-26T07:22:00.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:22:00.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.857+0000 7f5e43fa1640 1 Processor -- start 2023-11-26T07:22:00.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.857+0000 7f5e43fa1640 1 -- start start 2023-11-26T07:22:00.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e43fa1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e3c0b3c60 0x7f5e3c0b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e43fa1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5e3c162190 0x7f5e3c1625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e43fa1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0b2590 con 0x7f5e3c0b4040 2023-11-26T07:22:00.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e43fa1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b2870 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e43fa1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b2b50 con 0x7f5e3c162190 2023-11-26T07:22:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e41d3d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e3c0b3c60 0x7f5e3c0b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e41d3d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e3c0b3c60 0x7f5e3c0b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50696/0 (socket says 172.21.15.31:50696) 2023-11-26T07:22:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e41d3d640 1 -- 172.21.15.31:0/4207284529 learned_addr learned my addr 172.21.15.31:0/4207284529 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e4253e640 1 --2- 172.21.15.31:0/4207284529 >> v2:172.21.15.97:3300/0 conn(0x7f5e3c162190 0x7f5e3c1625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e41d3d640 1 -- 172.21.15.31:0/4207284529 >> v2:172.21.15.97:3300/0 conn(0x7f5e3c162190 msgr2=0x7f5e3c1625a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e41d3d640 1 --2- 172.21.15.31:0/4207284529 >> v2:172.21.15.97:3300/0 conn(0x7f5e3c162190 0x7f5e3c1625a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e41d3d640 1 -- 172.21.15.31:0/4207284529 >> v1:172.21.15.97:6789/0 conn(0x7f5e3c0b4040 legacy=0x7f5e3c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e41d3d640 1 -- 172.21.15.31:0/4207284529 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c0b2e30 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.861+0000 7f5e41d3d640 1 --2- 172.21.15.31:0/4207284529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 0x7f5e3c0b1fb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0b1cd0 tx=0x7f5e3801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b65f2a8f6a6fa62 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e40d3b640 1 -- 172.21.15.31:0/4207284529 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380aa650 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e40d3b640 1 -- 172.21.15.31:0/4207284529 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e380aa7b0 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e40d3b640 1 -- 172.21.15.31:0/4207284529 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380aab40 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 -- 172.21.15.31:0/4207284529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 msgr2=0x7f5e3c0b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 --2- 172.21.15.31:0/4207284529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 0x7f5e3c0b1fb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0b1cd0 tx=0x7f5e3801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 -- 172.21.15.31:0/4207284529 shutdown_connections 2023-11-26T07:22:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 --2- 172.21.15.31:0/4207284529 >> v2:172.21.15.97:3300/0 conn(0x7f5e3c162190 0x7f5e3c1625a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 --2- 172.21.15.31:0/4207284529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 0x7f5e3c0b1fb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 -- 172.21.15.31:0/4207284529 >> 172.21.15.31:0/4207284529 conn(0x7f5e3c0b0cb0 msgr2=0x7f5e3c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 -- 172.21.15.31:0/4207284529 shutdown_connections 2023-11-26T07:22:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 -- 172.21.15.31:0/4207284529 wait complete. 2023-11-26T07:22:00.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 Processor -- start 2023-11-26T07:22:00.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 -- start start 2023-11-26T07:22:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 0x7f5e3c09baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3c162190 0x7f5e3c15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5e3c15caf0 0x7f5e3c0afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c15b880 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c15d300 con 0x7f5e3c162190 2023-11-26T07:22:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e43fa1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c15d5e0 con 0x7f5e3c15caf0 2023-11-26T07:22:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.865+0000 7f5e41d3d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 0x7f5e3c09baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e41d3d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 0x7f5e3c09baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50710/0 (socket says 172.21.15.31:50710) 2023-11-26T07:22:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e41d3d640 1 -- 172.21.15.31:0/1695578086 learned_addr learned my addr 172.21.15.31:0/1695578086 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:00.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e4253e640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5e3c15caf0 0x7f5e3c0afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e4153c640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3c162190 0x7f5e3c15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e41d3d640 1 -- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5e3c15caf0 msgr2=0x7f5e3c0afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e41d3d640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5e3c15caf0 0x7f5e3c0afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e41d3d640 1 -- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3c162190 msgr2=0x7f5e3c15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e41d3d640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3c162190 0x7f5e3c15b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e41d3d640 1 -- 172.21.15.31:0/1695578086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e380094d0 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e41d3d640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 0x7f5e3c09baa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5e380ac040 tx=0x7f5e3801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e327fc640 1 -- 172.21.15.31:0/1695578086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380a98c0 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e327fc640 1 -- 172.21.15.31:0/1695578086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e380a9a20 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e43fa1640 1 -- 172.21.15.31:0/1695578086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c15d8c0 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e327fc640 1 -- 172.21.15.31:0/1695578086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380a9dd0 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e43fa1640 1 -- 172.21.15.31:0/1695578086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c15cfa0 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e43fa1640 1 -- 172.21.15.31:0/1695578086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c0671f0 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e327fc640 1 -- 172.21.15.31:0/1695578086 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f5e380b33f0 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.869+0000 7f5e327fc640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5e20066800 0x7f5e20068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.873+0000 7f5e327fc640 1 -- 172.21.15.31:0/1695578086 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e3811ee40 con 0x7f5e3c0b3c60 2023-11-26T07:22:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.873+0000 7f5e4153c640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5e20066800 0x7f5e20068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.873+0000 7f5e4153c640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5e20066800 0x7f5e20068cc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0aec40 tx=0x7f5e2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:00.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:00.873+0000 7f5e327fc640 1 -- 172.21.15.31:0/1695578086 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5e380ebe50 con 0x7f5e3c0b3c60 2023-11-26T07:22:01.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.045+0000 7f5e43fa1640 1 -- 172.21.15.31:0/1695578086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5e3c06b460 con 0x7f5e3c0b3c60 2023-11-26T07:22:01.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.045+0000 7f5e327fc640 1 -- 172.21.15.31:0/1695578086 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5e380c2070 con 0x7f5e3c0b3c60 2023-11-26T07:22:01.050 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:22:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 -- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5e20066800 msgr2=0x7f5e20068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5e20066800 0x7f5e20068cc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0aec40 tx=0x7f5e2c009040 comp rx=0 tx=0).stop 2023-11-26T07:22:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 -- 172.21.15.31:0/1695578086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 msgr2=0x7f5e3c09baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 0x7f5e3c09baa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5e380ac040 tx=0x7f5e3801ad10 comp rx=0 tx=0).stop 2023-11-26T07:22:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 -- 172.21.15.31:0/1695578086 shutdown_connections 2023-11-26T07:22:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f5e20066800 0x7f5e20068cc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5e3c15caf0 0x7f5e3c0afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e3c162190 0x7f5e3c15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 --2- 172.21.15.31:0/1695578086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e3c0b3c60 0x7f5e3c09baa0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 -- 172.21.15.31:0/1695578086 >> 172.21.15.31:0/1695578086 conn(0x7f5e3c0b0cb0 msgr2=0x7f5e3c070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 -- 172.21.15.31:0/1695578086 shutdown_connections 2023-11-26T07:22:01.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.049+0000 7f5e43fa1640 1 -- 172.21.15.31:0/1695578086 wait complete. 2023-11-26T07:22:01.071 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:22:01.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:01.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:01.099 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:22:01.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:01.100 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:22:01.155 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:01.155 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:01.164 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:01.164 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:01.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.353+0000 7f2d8949c640 1 Processor -- start 2023-11-26T07:22:01.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.353+0000 7f2d8949c640 1 -- start start 2023-11-26T07:22:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.353+0000 7f2d8949c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d840ac630 0x7f2d840706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.353+0000 7f2d8949c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d84145060 con 0x7f2d84144c80 2023-11-26T07:22:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.353+0000 7f2d8949c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d84071fd0 con 0x7f2d84070c20 2023-11-26T07:22:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.353+0000 7f2d8949c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d84071450 con 0x7f2d840ac630 2023-11-26T07:22:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d7bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d840ac630 0x7f2d840706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d7bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d840ac630 0x7f2d840706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50726/0 (socket says 172.21.15.31:50726) 2023-11-26T07:22:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d7bfff640 1 -- 172.21.15.31:0/3487065897 learned_addr learned my addr 172.21.15.31:0/3487065897 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:01.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d82ffd640 1 --1- 172.21.15.31:0/3487065897 >> v1:172.21.15.31:6789/0 conn(0x7f2d84144c80 0x7f2d840abf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53906/0 (socket says 172.21.15.31:53906) 2023-11-26T07:22:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d7bfff640 1 -- 172.21.15.31:0/3487065897 >> v1:172.21.15.97:6789/0 conn(0x7f2d84070c20 legacy=0x7f2d8406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d7bfff640 1 -- 172.21.15.31:0/3487065897 >> v1:172.21.15.31:6789/0 conn(0x7f2d84144c80 legacy=0x7f2d840abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d7bfff640 1 -- 172.21.15.31:0/3487065897 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84071730 con 0x7f2d840ac630 2023-11-26T07:22:01.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d7bfff640 1 --2- 172.21.15.31:0/3487065897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 0x7f2d840706e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009870 tx=0x7f2d6c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c81bc7dc127a8e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d827fc640 1 -- 172.21.15.31:0/3487065897 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0aa670 con 0x7f2d840ac630 2023-11-26T07:22:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d827fc640 1 -- 172.21.15.31:0/3487065897 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c0aa7d0 con 0x7f2d840ac630 2023-11-26T07:22:01.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.357+0000 7f2d827fc640 1 -- 172.21.15.31:0/3487065897 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0aab60 con 0x7f2d840ac630 2023-11-26T07:22:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 -- 172.21.15.31:0/3487065897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 msgr2=0x7f2d840706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 --2- 172.21.15.31:0/3487065897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 0x7f2d840706e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009870 tx=0x7f2d6c01c510 comp rx=0 tx=0).stop 2023-11-26T07:22:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 -- 172.21.15.31:0/3487065897 shutdown_connections 2023-11-26T07:22:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 --2- 172.21.15.31:0/3487065897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 0x7f2d840706e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 -- 172.21.15.31:0/3487065897 >> 172.21.15.31:0/3487065897 conn(0x7f2d840b0c40 msgr2=0x7f2d840b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 -- 172.21.15.31:0/3487065897 shutdown_connections 2023-11-26T07:22:01.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 -- 172.21.15.31:0/3487065897 wait complete. 2023-11-26T07:22:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 Processor -- start 2023-11-26T07:22:01.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 -- start start 2023-11-26T07:22:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84070c20 0x7f2d8415e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 0x7f2d8413c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d84144c80 0x7f2d8415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8413d500 con 0x7f2d840ac630 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8413e950 con 0x7f2d84070c20 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d8949c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8413cf40 con 0x7f2d84144c80 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 0x7f2d8413c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 0x7f2d8413c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50734/0 (socket says 172.21.15.31:50734) 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d7bfff640 1 -- 172.21.15.31:0/2280508091 learned_addr learned my addr 172.21.15.31:0/2280508091 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d82ffd640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84070c20 0x7f2d8415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d837fe640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d84144c80 0x7f2d8415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d7bfff640 1 -- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d84144c80 msgr2=0x7f2d8415a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.361+0000 7f2d7bfff640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d84144c80 0x7f2d8415a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d7bfff640 1 -- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84070c20 msgr2=0x7f2d8415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d7bfff640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84070c20 0x7f2d8415e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d7bfff640 1 -- 172.21.15.31:0/2280508091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c0094d0 con 0x7f2d840ac630 2023-11-26T07:22:01.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d7bfff640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 0x7f2d8413c660 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0ac040 tx=0x7f2d6c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d7b7fe640 1 -- 172.21.15.31:0/2280508091 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0b38c0 con 0x7f2d840ac630 2023-11-26T07:22:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d7b7fe640 1 -- 172.21.15.31:0/2280508091 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c0b3a40 con 0x7f2d840ac630 2023-11-26T07:22:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d8949c640 1 -- 172.21.15.31:0/2280508091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8413d220 con 0x7f2d840ac630 2023-11-26T07:22:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d7b7fe640 1 -- 172.21.15.31:0/2280508091 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0b3df0 con 0x7f2d840ac630 2023-11-26T07:22:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d8949c640 1 -- 172.21.15.31:0/2280508091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8413cba0 con 0x7f2d840ac630 2023-11-26T07:22:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d8949c640 1 -- 172.21.15.31:0/2280508091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8404e850 con 0x7f2d840ac630 2023-11-26T07:22:01.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d7b7fe640 1 -- 172.21.15.31:0/2280508091 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f2d6c0b2380 con 0x7f2d840ac630 2023-11-26T07:22:01.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d7b7fe640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d640667d0 0x7f2d64068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d7b7fe640 1 -- 172.21.15.31:0/2280508091 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d6c11e760 con 0x7f2d840ac630 2023-11-26T07:22:01.370 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.365+0000 7f2d82ffd640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d640667d0 0x7f2d64068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.369+0000 7f2d82ffd640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d640667d0 0x7f2d64068c90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d74000f70 tx=0x7f2d74007e50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.369+0000 7f2d7b7fe640 1 -- 172.21.15.31:0/2280508091 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d6c0eb7e0 con 0x7f2d840ac630 2023-11-26T07:22:01.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 -- 172.21.15.31:0/2280508091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d8413edc0 con 0x7f2d840ac630 2023-11-26T07:22:01.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d7b7fe640 1 -- 172.21.15.31:0/2280508091 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16484 (secure 0 0 0) 0x7f2d6c0eee00 con 0x7f2d840ac630 2023-11-26T07:22:01.532 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:01.532 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":68,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:21:50.369840+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:46.570507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:47.773615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 -- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d640667d0 msgr2=0x7f2d64068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d640667d0 0x7f2d64068c90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d74000f70 tx=0x7f2d74007e50 comp rx=0 tx=0).stop 2023-11-26T07:22:01.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 -- 172.21.15.31:0/2280508091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 msgr2=0x7f2d8413c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 0x7f2d8413c660 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0ac040 tx=0x7f2d6c01ad50 comp rx=0 tx=0).stop 2023-11-26T07:22:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 -- 172.21.15.31:0/2280508091 shutdown_connections 2023-11-26T07:22:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f2d640667d0 0x7f2d64068c90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d84144c80 0x7f2d8415a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d840ac630 0x7f2d8413c660 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 --2- 172.21.15.31:0/2280508091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d84070c20 0x7f2d8415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 -- 172.21.15.31:0/2280508091 >> 172.21.15.31:0/2280508091 conn(0x7f2d840b0c40 msgr2=0x7f2d840b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 -- 172.21.15.31:0/2280508091 shutdown_connections 2023-11-26T07:22:01.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.529+0000 7f2d8949c640 1 -- 172.21.15.31:0/2280508091 wait complete. 2023-11-26T07:22:01.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:21:46.570507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:21:47.773615+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:01.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47aa9df640 1 Processor -- start 2023-11-26T07:22:01.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47aa9df640 1 -- start start 2023-11-26T07:22:01.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47aa9df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a4166580 0x7f47a4168970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47aa9df640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a40af630 con 0x7f47a415e690 2023-11-26T07:22:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47aa9df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a40acb10 con 0x7f47a40b5830 2023-11-26T07:22:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47aa9df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a40adb90 con 0x7f47a4166580 2023-11-26T07:22:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47a37fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f47a415e690 0x7f47a40ad480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53918/0 (socket says 172.21.15.31:53918) 2023-11-26T07:22:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47a37fe640 1 -- 172.21.15.31:0/4196669983 learned_addr learned my addr 172.21.15.31:0/4196669983 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:01.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47a8f7c640 1 --2- 172.21.15.31:0/4196669983 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a4166580 0x7f47a4168970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47a2ffd640 1 -- 172.21.15.31:0/4196669983 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480770443 0 0) 0x7f4798001120 con 0x7f47a415e690 2023-11-26T07:22:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47a2ffd640 1 -- 172.21.15.31:0/4196669983 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f478c003530 con 0x7f47a415e690 2023-11-26T07:22:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.697+0000 7f47a2ffd640 1 -- 172.21.15.31:0/4196669983 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518917673 0 0) 0x7f47a40acb10 con 0x7f47a40b5830 2023-11-26T07:22:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47a2ffd640 1 -- 172.21.15.31:0/4196669983 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4798001120 con 0x7f47a40b5830 2023-11-26T07:22:01.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47a8f7c640 1 -- 172.21.15.31:0/4196669983 >> v1:172.21.15.97:6789/0 conn(0x7f47a40b5830 legacy=0x7f47a40acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47a8f7c640 1 -- 172.21.15.31:0/4196669983 >> v1:172.21.15.31:6789/0 conn(0x7f47a415e690 legacy=0x7f47a40ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47a8f7c640 1 -- 172.21.15.31:0/4196669983 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a415eaf0 con 0x7f47a4166580 2023-11-26T07:22:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47a8f7c640 1 --2- 172.21.15.31:0/4196669983 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a4166580 0x7f47a4168970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f479c009dc0 tx=0x7f479c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c68560f3a3444c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47a2ffd640 1 -- 172.21.15.31:0/4196669983 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0a5a30 con 0x7f47a4166580 2023-11-26T07:22:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47a2ffd640 1 -- 172.21.15.31:0/4196669983 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c0a5b90 con 0x7f47a4166580 2023-11-26T07:22:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47a2ffd640 1 -- 172.21.15.31:0/4196669983 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0ad440 con 0x7f47a4166580 2023-11-26T07:22:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47aa9df640 1 -- 172.21.15.31:0/4196669983 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a4166580 msgr2=0x7f47a4168970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47aa9df640 1 --2- 172.21.15.31:0/4196669983 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a4166580 0x7f47a4168970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f479c009dc0 tx=0x7f479c01c970 comp rx=0 tx=0).stop 2023-11-26T07:22:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47aa9df640 1 -- 172.21.15.31:0/4196669983 shutdown_connections 2023-11-26T07:22:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47aa9df640 1 --2- 172.21.15.31:0/4196669983 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a4166580 0x7f47a4168970 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.701+0000 7f47aa9df640 1 -- 172.21.15.31:0/4196669983 >> 172.21.15.31:0/4196669983 conn(0x7f47a4071d80 msgr2=0x7f47a406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47aa9df640 1 -- 172.21.15.31:0/4196669983 shutdown_connections 2023-11-26T07:22:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47aa9df640 1 -- 172.21.15.31:0/4196669983 wait complete. 2023-11-26T07:22:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47aa9df640 1 Processor -- start 2023-11-26T07:22:01.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47aa9df640 1 -- start start 2023-11-26T07:22:01.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47aa9df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a40b5830 0x7f47a40ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47aa9df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a415e690 0x7f47a40cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47aa9df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4166580 0x7f47a4151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47aa9df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a415a630 con 0x7f47a40b5830 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47aa9df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a415a910 con 0x7f47a4166580 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47aa9df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a415abf0 con 0x7f47a415e690 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a40b5830 0x7f47a40ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a40b5830 0x7f47a40ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50742/0 (socket says 172.21.15.31:50742) 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a3fff640 1 -- 172.21.15.31:0/1253434130 learned_addr learned my addr 172.21.15.31:0/1253434130 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a37fe640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a415e690 0x7f47a40cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a8f7c640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4166580 0x7f47a4151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a3fff640 1 -- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a415e690 msgr2=0x7f47a40cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a3fff640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a415e690 0x7f47a40cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a3fff640 1 -- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4166580 msgr2=0x7f47a4151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a3fff640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4166580 0x7f47a4151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a3fff640 1 -- 172.21.15.31:0/1253434130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c009a70 con 0x7f47a40b5830 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a37fe640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a415e690 0x7f47a40cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a8f7c640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4166580 0x7f47a4151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.705+0000 7f47a3fff640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a40b5830 0x7f47a40ce7c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f479400c430 tx=0x7f479400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.709+0000 7f47a0ff9640 1 -- 172.21.15.31:0/1253434130 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47940052d0 con 0x7f47a40b5830 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.709+0000 7f47a0ff9640 1 -- 172.21.15.31:0/1253434130 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4794005450 con 0x7f47a40b5830 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.709+0000 7f47a0ff9640 1 -- 172.21.15.31:0/1253434130 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794003840 con 0x7f47a40b5830 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.709+0000 7f47aa9df640 1 -- 172.21.15.31:0/1253434130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a415aed0 con 0x7f47a40b5830 2023-11-26T07:22:01.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.709+0000 7f47aa9df640 1 -- 172.21.15.31:0/1253434130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a415a3d0 con 0x7f47a40b5830 2023-11-26T07:22:01.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.709+0000 7f47aa9df640 1 -- 172.21.15.31:0/1253434130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a415b0c0 con 0x7f47a40b5830 2023-11-26T07:22:01.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.709+0000 7f47a0ff9640 1 -- 172.21.15.31:0/1253434130 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f47940039a0 con 0x7f47a40b5830 2023-11-26T07:22:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.709+0000 7f47a0ff9640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4790066800 0x7f4790068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.709+0000 7f47a0ff9640 1 -- 172.21.15.31:0/1253434130 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4794083930 con 0x7f47a40b5830 2023-11-26T07:22:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.709+0000 7f47a37fe640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4790066800 0x7f4790068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:01.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.713+0000 7f47a37fe640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4790066800 0x7f4790068cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f47a40adb60 tx=0x7f4798009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:01.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.713+0000 7f47a0ff9640 1 -- 172.21.15.31:0/1253434130 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4794050a30 con 0x7f47a40b5830 2023-11-26T07:22:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 -- 172.21.15.31:0/1253434130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47a404e850 con 0x7f47a40b5830 2023-11-26T07:22:01.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47a0ff9640 1 -- 172.21.15.31:0/1253434130 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47940170a0 con 0x7f47a40b5830 2023-11-26T07:22:01.875 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:22:01.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 -- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4790066800 msgr2=0x7f4790068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4790066800 0x7f4790068cc0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f47a40adb60 tx=0x7f4798009040 comp rx=0 tx=0).stop 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 -- 172.21.15.31:0/1253434130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a40b5830 msgr2=0x7f47a40ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a40b5830 0x7f47a40ce7c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f479400c430 tx=0x7f479400c8f0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 -- 172.21.15.31:0/1253434130 shutdown_connections 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f4790066800 0x7f4790068cc0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a4166580 0x7f47a4151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47a415e690 0x7f47a40cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 --2- 172.21.15.31:0/1253434130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a40b5830 0x7f47a40ce7c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 -- 172.21.15.31:0/1253434130 >> 172.21.15.31:0/1253434130 conn(0x7f47a4071d80 msgr2=0x7f47a4160350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 -- 172.21.15.31:0/1253434130 shutdown_connections 2023-11-26T07:22:01.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:01.873+0000 7f47aa9df640 1 -- 172.21.15.31:0/1253434130 wait complete. 2023-11-26T07:22:01.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:02.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.037+0000 7f06cc432640 1 Processor -- start 2023-11-26T07:22:02.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.037+0000 7f06cc432640 1 -- start start 2023-11-26T07:22:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06cc432640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f06c415a710 0x7f06c40abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06cc432640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06c40cbd50 0x7f06c40cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06cc432640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c40b2710 con 0x7f06c40cdfb0 2023-11-26T07:22:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06cc432640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c40cca80 con 0x7f06c40cbd50 2023-11-26T07:22:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06cc432640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c40b29f0 con 0x7f06c415a710 2023-11-26T07:22:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06ca9cf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06c40cbd50 0x7f06c40cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06ca9cf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f06c40cbd50 0x7f06c40cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50754/0 (socket says 172.21.15.31:50754) 2023-11-26T07:22:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06ca9cf640 1 -- 172.21.15.31:0/3953425914 learned_addr learned my addr 172.21.15.31:0/3953425914 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:02.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06c99cd640 1 --2- 172.21.15.31:0/3953425914 >> v2:172.21.15.97:3300/0 conn(0x7f06c415a710 0x7f06c40abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06ca9cf640 1 -- 172.21.15.31:0/3953425914 >> v2:172.21.15.97:3300/0 conn(0x7f06c415a710 msgr2=0x7f06c40abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06ca9cf640 1 --2- 172.21.15.31:0/3953425914 >> v2:172.21.15.97:3300/0 conn(0x7f06c415a710 0x7f06c40abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06ca9cf640 1 -- 172.21.15.31:0/3953425914 >> v1:172.21.15.97:6789/0 conn(0x7f06c40cdfb0 legacy=0x7f06c40ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.041+0000 7f06ca9cf640 1 -- 172.21.15.31:0/3953425914 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c40b2cd0 con 0x7f06c40cbd50 2023-11-26T07:22:02.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06ca9cf640 1 --2- 172.21.15.31:0/3953425914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 0x7f06c40cc1e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f06c40d0040 tx=0x7f06bc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935974695e718fa server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06c91cc640 1 -- 172.21.15.31:0/3953425914 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0a5840 con 0x7f06c40cbd50 2023-11-26T07:22:02.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06c91cc640 1 -- 172.21.15.31:0/3953425914 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc0a59a0 con 0x7f06c40cbd50 2023-11-26T07:22:02.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06c91cc640 1 -- 172.21.15.31:0/3953425914 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0a5d30 con 0x7f06c40cbd50 2023-11-26T07:22:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06cc432640 1 -- 172.21.15.31:0/3953425914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 msgr2=0x7f06c40cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06cc432640 1 --2- 172.21.15.31:0/3953425914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 0x7f06c40cc1e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f06c40d0040 tx=0x7f06bc01c960 comp rx=0 tx=0).stop 2023-11-26T07:22:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06cc432640 1 -- 172.21.15.31:0/3953425914 shutdown_connections 2023-11-26T07:22:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06cc432640 1 --2- 172.21.15.31:0/3953425914 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 0x7f06c40cc1e0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06cc432640 1 --2- 172.21.15.31:0/3953425914 >> v2:172.21.15.97:3300/0 conn(0x7f06c415a710 0x7f06c40abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06cc432640 1 -- 172.21.15.31:0/3953425914 >> 172.21.15.31:0/3953425914 conn(0x7f06c40b0cb0 msgr2=0x7f06c40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06cc432640 1 -- 172.21.15.31:0/3953425914 shutdown_connections 2023-11-26T07:22:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06cc432640 1 -- 172.21.15.31:0/3953425914 wait complete. 2023-11-26T07:22:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06cc432640 1 Processor -- start 2023-11-26T07:22:02.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.045+0000 7f06cc432640 1 -- start start 2023-11-26T07:22:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06cc432640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 0x7f06c4154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06cc432640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06c40cdfb0 0x7f06c4070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06cc432640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c415a710 0x7f06c415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06cc432640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c4157bb0 con 0x7f06c40cbd50 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06cc432640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c41575d0 con 0x7f06c415a710 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06cc432640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c4157840 con 0x7f06c40cdfb0 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06ca1ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 0x7f06c4154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06ca1ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 0x7f06c4154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50768/0 (socket says 172.21.15.31:50768) 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06ca1ce640 1 -- 172.21.15.31:0/1406645784 learned_addr learned my addr 172.21.15.31:0/1406645784 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06c99cd640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06c40cdfb0 0x7f06c4070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06ca9cf640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c415a710 0x7f06c415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06ca1ce640 1 -- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06c40cdfb0 msgr2=0x7f06c4070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06ca1ce640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06c40cdfb0 0x7f06c4070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06ca1ce640 1 -- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c415a710 msgr2=0x7f06c415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06ca1ce640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c415a710 0x7f06c415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06ca1ce640 1 -- 172.21.15.31:0/1406645784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc009b30 con 0x7f06c40cbd50 2023-11-26T07:22:02.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06ca1ce640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 0x7f06c4154910 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f06c000c4b0 tx=0x7f06c000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06baffd640 1 -- 172.21.15.31:0/1406645784 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0012830 con 0x7f06c40cbd50 2023-11-26T07:22:02.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06baffd640 1 -- 172.21.15.31:0/1406645784 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c00129b0 con 0x7f06c40cbd50 2023-11-26T07:22:02.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06baffd640 1 -- 172.21.15.31:0/1406645784 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0012d60 con 0x7f06c40cbd50 2023-11-26T07:22:02.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06cc432640 1 -- 172.21.15.31:0/1406645784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c4159350 con 0x7f06c40cbd50 2023-11-26T07:22:02.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06cc432640 1 -- 172.21.15.31:0/1406645784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c4070bd0 con 0x7f06c40cbd50 2023-11-26T07:22:02.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06cc432640 1 -- 172.21.15.31:0/1406645784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c40671f0 con 0x7f06c40cbd50 2023-11-26T07:22:02.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.049+0000 7f06baffd640 1 -- 172.21.15.31:0/1406645784 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f06c001c460 con 0x7f06c40cbd50 2023-11-26T07:22:02.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.053+0000 7f06baffd640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06a80667d0 0x7f06a8068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.053+0000 7f06baffd640 1 -- 172.21.15.31:0/1406645784 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06c0083fe0 con 0x7f06c40cbd50 2023-11-26T07:22:02.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.053+0000 7f06c99cd640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06a80667d0 0x7f06a8068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.053+0000 7f06c99cd640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06a80667d0 0x7f06a8068c90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f06c415dc60 tx=0x7f06b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.057+0000 7f06baffd640 1 -- 172.21.15.31:0/1406645784 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f06c0051150 con 0x7f06c40cbd50 2023-11-26T07:22:02.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 -- 172.21.15.31:0/1406645784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06c4157cf0 con 0x7f06c40cbd50 2023-11-26T07:22:02.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06baffd640 1 -- 172.21.15.31:0/1406645784 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06c4157cf0 con 0x7f06c40cbd50 2023-11-26T07:22:02.215 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:22:02.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 -- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06a80667d0 msgr2=0x7f06a8068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06a80667d0 0x7f06a8068c90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f06c415dc60 tx=0x7f06b4008040 comp rx=0 tx=0).stop 2023-11-26T07:22:02.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 -- 172.21.15.31:0/1406645784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 msgr2=0x7f06c4154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 0x7f06c4154910 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f06c000c4b0 tx=0x7f06c000c970 comp rx=0 tx=0).stop 2023-11-26T07:22:02.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 -- 172.21.15.31:0/1406645784 shutdown_connections 2023-11-26T07:22:02.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f06a80667d0 0x7f06a8068c90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06c415a710 0x7f06c415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06c40cdfb0 0x7f06c4070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 --2- 172.21.15.31:0/1406645784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06c40cbd50 0x7f06c4154910 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 -- 172.21.15.31:0/1406645784 >> 172.21.15.31:0/1406645784 conn(0x7f06c40b0cb0 msgr2=0x7f06c4159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 -- 172.21.15.31:0/1406645784 shutdown_connections 2023-11-26T07:22:02.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.213+0000 7f06cc432640 1 -- 172.21.15.31:0/1406645784 wait complete. 2023-11-26T07:22:02.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:02.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.377+0000 7efe0a735640 1 Processor -- start 2023-11-26T07:22:02.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.377+0000 7efe0a735640 1 -- start start 2023-11-26T07:22:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe0a735640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe040cc460 0x7efe040abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe0a735640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe040b2710 con 0x7efe040cdfb0 2023-11-26T07:22:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe0a735640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe040b29f0 con 0x7efe0415a710 2023-11-26T07:22:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe0a735640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe040cca80 con 0x7efe040cc460 2023-11-26T07:22:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe08cd2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe040cc460 0x7efe040abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe03fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efe040cdfb0 0x7efe040ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53932/0 (socket says 172.21.15.31:53932) 2023-11-26T07:22:02.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe03fff640 1 -- 172.21.15.31:0/3206827581 learned_addr learned my addr 172.21.15.31:0/3206827581 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe08cd2640 1 -- 172.21.15.31:0/3206827581 >> v1:172.21.15.97:6789/0 conn(0x7efe0415a710 legacy=0x7efe040cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe08cd2640 1 -- 172.21.15.31:0/3206827581 >> v1:172.21.15.31:6789/0 conn(0x7efe040cdfb0 legacy=0x7efe040ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe08cd2640 1 -- 172.21.15.31:0/3206827581 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe040b2cd0 con 0x7efe040cc460 2023-11-26T07:22:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe02ffd640 1 -- 172.21.15.31:0/3206827581 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279258286 0 0) 0x7efe040b2710 con 0x7efe040cdfb0 2023-11-26T07:22:02.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.381+0000 7efe08cd2640 1 --2- 172.21.15.31:0/3206827581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cc460 0x7efe040abf90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009d90 tx=0x7efdfc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdcb9fde8fc88d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe02ffd640 1 -- 172.21.15.31:0/3206827581 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0a59c0 con 0x7efe040cc460 2023-11-26T07:22:02.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe02ffd640 1 -- 172.21.15.31:0/3206827581 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc0a5b20 con 0x7efe040cc460 2023-11-26T07:22:02.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe02ffd640 1 -- 172.21.15.31:0/3206827581 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0ad440 con 0x7efe040cc460 2023-11-26T07:22:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 -- 172.21.15.31:0/3206827581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cc460 msgr2=0x7efe040abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 --2- 172.21.15.31:0/3206827581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cc460 0x7efe040abf90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009d90 tx=0x7efdfc01c8e0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 -- 172.21.15.31:0/3206827581 shutdown_connections 2023-11-26T07:22:02.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 --2- 172.21.15.31:0/3206827581 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe040cc460 0x7efe040abf90 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 -- 172.21.15.31:0/3206827581 >> 172.21.15.31:0/3206827581 conn(0x7efe040b0cb0 msgr2=0x7efe040b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 -- 172.21.15.31:0/3206827581 shutdown_connections 2023-11-26T07:22:02.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 -- 172.21.15.31:0/3206827581 wait complete. 2023-11-26T07:22:02.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 Processor -- start 2023-11-26T07:22:02.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 -- start start 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe040cc460 0x7efe0414f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efe040cdfb0 0x7efe0415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0415a710 0x7efe04150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe04153bc0 con 0x7efe0415a710 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe04153e60 con 0x7efe040cc460 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.385+0000 7efe0a735640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe04154140 con 0x7efe040cdfb0 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe08cd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0415a710 0x7efe04150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe08cd2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0415a710 0x7efe04150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50784/0 (socket says 172.21.15.31:50784) 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe08cd2640 1 -- 172.21.15.31:0/2740590207 learned_addr learned my addr 172.21.15.31:0/2740590207 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe03fff640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe040cc460 0x7efe0414f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe037fe640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efe040cdfb0 0x7efe0415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe08cd2640 1 -- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efe040cdfb0 msgr2=0x7efe0415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe08cd2640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efe040cdfb0 0x7efe0415b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe08cd2640 1 -- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe040cc460 msgr2=0x7efe0414f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe08cd2640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe040cc460 0x7efe0414f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe08cd2640 1 -- 172.21.15.31:0/2740590207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc009a70 con 0x7efe0415a710 2023-11-26T07:22:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe08cd2640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0415a710 0x7efe04150e80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7efdfc01a510 tx=0x7efdfc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe00ff9640 1 -- 172.21.15.31:0/2740590207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0b9650 con 0x7efe0415a710 2023-11-26T07:22:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe00ff9640 1 -- 172.21.15.31:0/2740590207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc0b97b0 con 0x7efe0415a710 2023-11-26T07:22:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe00ff9640 1 -- 172.21.15.31:0/2740590207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0b9b40 con 0x7efe0415a710 2023-11-26T07:22:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe0a735640 1 -- 172.21.15.31:0/2740590207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe04154420 con 0x7efe0415a710 2023-11-26T07:22:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe0a735640 1 -- 172.21.15.31:0/2740590207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe041549e0 con 0x7efe0415a710 2023-11-26T07:22:02.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe0a735640 1 -- 172.21.15.31:0/2740590207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe040671f0 con 0x7efe0415a710 2023-11-26T07:22:02.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.389+0000 7efe00ff9640 1 -- 172.21.15.31:0/2740590207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7efdfc0b9ca0 con 0x7efe0415a710 2023-11-26T07:22:02.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.393+0000 7efe00ff9640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efdf00667d0 0x7efdf0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.393+0000 7efe00ff9640 1 -- 172.21.15.31:0/2740590207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdfc120040 con 0x7efe0415a710 2023-11-26T07:22:02.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.393+0000 7efe03fff640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efdf00667d0 0x7efdf0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.393+0000 7efe03fff640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efdf00667d0 0x7efdf0068c90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efdf4000f70 tx=0x7efdf4007b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.393+0000 7efe00ff9640 1 -- 172.21.15.31:0/2740590207 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efdfc0ed0a0 con 0x7efe0415a710 2023-11-26T07:22:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.553+0000 7efe0a735640 1 -- 172.21.15.31:0/2740590207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe040671f0 con 0x7efe0415a710 2023-11-26T07:22:02.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.553+0000 7efe00ff9640 1 -- 172.21.15.31:0/2740590207 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7efdfc0b2020 con 0x7efe0415a710 2023-11-26T07:22:02.557 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:22:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 -- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efdf00667d0 msgr2=0x7efdf0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efdf00667d0 0x7efdf0068c90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efdf4000f70 tx=0x7efdf4007b60 comp rx=0 tx=0).stop 2023-11-26T07:22:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 -- 172.21.15.31:0/2740590207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0415a710 msgr2=0x7efe04150e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0415a710 0x7efe04150e80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7efdfc01a510 tx=0x7efdfc019680 comp rx=0 tx=0).stop 2023-11-26T07:22:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 -- 172.21.15.31:0/2740590207 shutdown_connections 2023-11-26T07:22:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7efdf00667d0 0x7efdf0068c90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0415a710 0x7efe04150e80 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efe040cdfb0 0x7efe0415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 --2- 172.21.15.31:0/2740590207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe040cc460 0x7efe0414f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 -- 172.21.15.31:0/2740590207 >> 172.21.15.31:0/2740590207 conn(0x7efe040b0cb0 msgr2=0x7efe040ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 -- 172.21.15.31:0/2740590207 shutdown_connections 2023-11-26T07:22:02.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.557+0000 7efe0a735640 1 -- 172.21.15.31:0/2740590207 wait complete. 2023-11-26T07:22:02.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:22:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.717+0000 7fcadece3640 1 Processor -- start 2023-11-26T07:22:02.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.717+0000 7fcadece3640 1 -- start start 2023-11-26T07:22:02.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadece3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcad80cdfb0 0x7fcad80abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadece3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad80b2710 con 0x7fcad815a710 2023-11-26T07:22:02.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadece3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad80b29f0 con 0x7fcad80cbd50 2023-11-26T07:22:02.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadece3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80cca80 con 0x7fcad80cdfb0 2023-11-26T07:22:02.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad7fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcad815a710 0x7fcad80ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:53946/0 (socket says 172.21.15.31:53946) 2023-11-26T07:22:02.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad7fff640 1 -- 172.21.15.31:0/2893798767 learned_addr learned my addr 172.21.15.31:0/2893798767 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224406672 0 0) 0x7fcad80b2710 con 0x7fcad815a710 2023-11-26T07:22:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcac8003530 con 0x7fcad815a710 2023-11-26T07:22:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadca7f640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcad80cdfb0 0x7fcad80abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2569396209 0 0) 0x7fcac8003530 con 0x7fcad815a710 2023-11-26T07:22:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcad80cdfb0 msgr2=0x7fcad80abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcad80cdfb0 0x7fcad80abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 >> v1:172.21.15.97:6789/0 conn(0x7fcad80cbd50 legacy=0x7fcad80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80b2cd0 con 0x7fcad815a710 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fcac4001680 con 0x7fcad815a710 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 >> v1:172.21.15.31:6789/0 conn(0x7fcad815a710 legacy=0x7fcad80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcad80cdfb0 0x7fcac80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac8003f80 0x7fcac80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcac8004d10 0x7fcac8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80cca80 con 0x7fcac8004d10 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac800dce0 con 0x7fcac8003f80 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac800dea0 con 0x7fcad80cdfb0 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadd280640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcac8004d10 0x7fcac8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadca7f640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcad80cdfb0 0x7fcac80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad7fff640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac8003f80 0x7fcac80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadd280640 1 -- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcad80cdfb0 msgr2=0x7fcac80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadd280640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcad80cdfb0 0x7fcac80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadd280640 1 -- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac8003f80 msgr2=0x7fcac80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:02.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadd280640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac8003f80 0x7fcac80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcadd280640 1 -- 172.21.15.31:0/2893798767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac8003530 con 0x7fcac8004d10 2023-11-26T07:22:02.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.721+0000 7fcad7fff640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcac8003f80 0x7fcac80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:02.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcadd280640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcac8004d10 0x7fcac8005190 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fcad0001e70 tx=0x7fcad001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc74041717a4a7ea server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0003b80 con 0x7fcac8004d10 2023-11-26T07:22:02.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad0003ce0 con 0x7fcac8004d10 2023-11-26T07:22:02.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcad77fe640 1 -- 172.21.15.31:0/2893798767 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00052c0 con 0x7fcac8004d10 2023-11-26T07:22:02.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcadece3640 1 -- 172.21.15.31:0/2893798767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcac8004d10 msgr2=0x7fcac8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcadece3640 1 --2- 172.21.15.31:0/2893798767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcac8004d10 0x7fcac8005190 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fcad0001e70 tx=0x7fcad001c760 comp rx=0 tx=0).stop 2023-11-26T07:22:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcadd280640 1 -- 172.21.15.31:0/2893798767 reap_dead start 2023-11-26T07:22:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcadece3640 1 -- 172.21.15.31:0/2893798767 shutdown_connections 2023-11-26T07:22:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcadece3640 1 -- 172.21.15.31:0/2893798767 >> 172.21.15.31:0/2893798767 conn(0x7fcad80b0cb0 msgr2=0x7fcad80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcadece3640 1 -- 172.21.15.31:0/2893798767 shutdown_connections 2023-11-26T07:22:02.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.725+0000 7fcadece3640 1 -- 172.21.15.31:0/2893798767 wait complete. 2023-11-26T07:22:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadece3640 1 Processor -- start 2023-11-26T07:22:02.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadece3640 1 -- start start 2023-11-26T07:22:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadece3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcac8004d10 0x7fcad80aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadece3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcad814a8c0 0x7fcad814ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadece3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad8099a30 0x7fcad8099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadece3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad814c390 con 0x7fcad814a8c0 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadece3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad809a760 con 0x7fcad8099a30 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadece3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad814b5d0 con 0x7fcac8004d10 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcad814a8c0 0x7fcad814ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcad814a8c0 0x7fcad814ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50794/0 (socket says 172.21.15.31:50794) 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad7fff640 1 -- 172.21.15.31:0/2422684494 learned_addr learned my addr 172.21.15.31:0/2422684494 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadd280640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad8099a30 0x7fcad8099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadca7f640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcac8004d10 0x7fcad80aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad7fff640 1 -- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcac8004d10 msgr2=0x7fcad80aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad7fff640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcac8004d10 0x7fcad80aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad7fff640 1 -- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad8099a30 msgr2=0x7fcad8099ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad7fff640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad8099a30 0x7fcad8099ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad7fff640 1 -- 172.21.15.31:0/2422684494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad814b8b0 con 0x7fcad814a8c0 2023-11-26T07:22:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadd280640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad8099a30 0x7fcad8099ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad7fff640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcad814a8c0 0x7fcad814ad40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fcac4001ec0 tx=0x7fcac400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad57fa640 1 -- 172.21.15.31:0/2422684494 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400c8a0 con 0x7fcad814a8c0 2023-11-26T07:22:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad57fa640 1 -- 172.21.15.31:0/2422684494 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac400ca00 con 0x7fcad814a8c0 2023-11-26T07:22:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcad57fa640 1 -- 172.21.15.31:0/2422684494 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400cd70 con 0x7fcad814a8c0 2023-11-26T07:22:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadece3640 1 -- 172.21.15.31:0/2422684494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad814bb90 con 0x7fcad814a8c0 2023-11-26T07:22:02.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.729+0000 7fcadece3640 1 -- 172.21.15.31:0/2422684494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad814b280 con 0x7fcad814a8c0 2023-11-26T07:22:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.733+0000 7fcadece3640 1 -- 172.21.15.31:0/2422684494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad80671f0 con 0x7fcad814a8c0 2023-11-26T07:22:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.733+0000 7fcad57fa640 1 -- 172.21.15.31:0/2422684494 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fcac401d460 con 0x7fcad814a8c0 2023-11-26T07:22:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.733+0000 7fcad57fa640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcabc0667d0 0x7fcabc068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:02.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.733+0000 7fcad57fa640 1 -- 172.21.15.31:0/2422684494 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcac40843d0 con 0x7fcad814a8c0 2023-11-26T07:22:02.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.733+0000 7fcadca7f640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcabc0667d0 0x7fcabc068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:02.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.733+0000 7fcadca7f640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcabc0667d0 0x7fcabc068c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009b20 tx=0x7fcacc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:02.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.737+0000 7fcad57fa640 1 -- 172.21.15.31:0/2422684494 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcac4051490 con 0x7fcad814a8c0 2023-11-26T07:22:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 -- 172.21.15.31:0/2422684494 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcad806b460 con 0x7fcad814a8c0 2023-11-26T07:22:02.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcad57fa640 1 -- 172.21.15.31:0/2422684494 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcac4054ab0 con 0x7fcad814a8c0 2023-11-26T07:22:02.908 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:22:02.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 -- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcabc0667d0 msgr2=0x7fcabc068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcabc0667d0 0x7fcabc068c90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009b20 tx=0x7fcacc008040 comp rx=0 tx=0).stop 2023-11-26T07:22:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 -- 172.21.15.31:0/2422684494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcad814a8c0 msgr2=0x7fcad814ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcad814a8c0 0x7fcad814ad40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fcac4001ec0 tx=0x7fcac400eca0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 -- 172.21.15.31:0/2422684494 shutdown_connections 2023-11-26T07:22:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fcabc0667d0 0x7fcabc068c90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcad8099a30 0x7fcad8099ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcad814a8c0 0x7fcad814ad40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 --2- 172.21.15.31:0/2422684494 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcac8004d10 0x7fcad80aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 -- 172.21.15.31:0/2422684494 >> 172.21.15.31:0/2422684494 conn(0x7fcad80b0cb0 msgr2=0x7fcad80cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 -- 172.21.15.31:0/2422684494 shutdown_connections 2023-11-26T07:22:02.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:02.905+0000 7fcadece3640 1 -- 172.21.15.31:0/2422684494 wait complete. 2023-11-26T07:22:02.929 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:22:02.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:02.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:02.958 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:22:02.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:02.959 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:22:03.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac9a9c640 1 Processor -- start 2023-11-26T07:22:03.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac9a9c640 1 -- start start 2023-11-26T07:22:03.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac9a9c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feac415d880 0x7feac40b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac9a9c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7feac40b6c70 0x7feac40d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac9a9c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40d3ca0 0x7feac40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac9a9c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac40d4110 con 0x7feac415d880 2023-11-26T07:22:03.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac9a9c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac40d15f0 con 0x7feac40b6c70 2023-11-26T07:22:03.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac9a9c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac40ce000 con 0x7feac40d3ca0 2023-11-26T07:22:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac8a9a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feac415d880 0x7feac40b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac8a9a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7feac415d880 0x7feac40b6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50798/0 (socket says 172.21.15.31:50798) 2023-11-26T07:22:03.157 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac8a9a640 1 -- 172.21.15.31:0/3642715615 learned_addr learned my addr 172.21.15.31:0/3642715615 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac929b640 1 --2- 172.21.15.31:0/3642715615 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40d3ca0 0x7feac40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac8a9a640 1 -- 172.21.15.31:0/3642715615 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40d3ca0 msgr2=0x7feac40cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feabbfff640 1 --2- 172.21.15.31:0/3642715615 >> v2:172.21.15.97:3300/0 conn(0x7feac40b6c70 0x7feac40d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac8a9a640 1 --2- 172.21.15.31:0/3642715615 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40d3ca0 0x7feac40cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac8a9a640 1 -- 172.21.15.31:0/3642715615 >> v2:172.21.15.97:3300/0 conn(0x7feac40b6c70 msgr2=0x7feac40d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac8a9a640 1 --2- 172.21.15.31:0/3642715615 >> v2:172.21.15.97:3300/0 conn(0x7feac40b6c70 0x7feac40d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac8a9a640 1 -- 172.21.15.31:0/3642715615 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac40d0630 con 0x7feac415d880 2023-11-26T07:22:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feac929b640 1 --2- 172.21.15.31:0/3642715615 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40d3ca0 0x7feac40cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:03.158 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.153+0000 7feabbfff640 1 --2- 172.21.15.31:0/3642715615 >> v2:172.21.15.97:3300/0 conn(0x7feac40b6c70 0x7feac40d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:03.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feac8a9a640 1 --2- 172.21.15.31:0/3642715615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac415d880 0x7feac40b6730 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7feac40cd850 tx=0x7feac001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3dc3e60f1d078f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:03.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feabb7fe640 1 -- 172.21.15.31:0/3642715615 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00aa640 con 0x7feac415d880 2023-11-26T07:22:03.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feabb7fe640 1 -- 172.21.15.31:0/3642715615 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac00aa7a0 con 0x7feac415d880 2023-11-26T07:22:03.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feabb7fe640 1 -- 172.21.15.31:0/3642715615 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac00aab30 con 0x7feac415d880 2023-11-26T07:22:03.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feac9a9c640 1 -- 172.21.15.31:0/3642715615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac415d880 msgr2=0x7feac40b6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feac9a9c640 1 --2- 172.21.15.31:0/3642715615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac415d880 0x7feac40b6730 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7feac40cd850 tx=0x7feac001c500 comp rx=0 tx=0).stop 2023-11-26T07:22:03.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feac9a9c640 1 -- 172.21.15.31:0/3642715615 shutdown_connections 2023-11-26T07:22:03.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feac9a9c640 1 --2- 172.21.15.31:0/3642715615 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40d3ca0 0x7feac40cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feac9a9c640 1 --2- 172.21.15.31:0/3642715615 >> v2:172.21.15.97:3300/0 conn(0x7feac40b6c70 0x7feac40d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feac9a9c640 1 --2- 172.21.15.31:0/3642715615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac415d880 0x7feac40b6730 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feac9a9c640 1 -- 172.21.15.31:0/3642715615 >> 172.21.15.31:0/3642715615 conn(0x7feac406b9f0 msgr2=0x7feac406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:03.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feac9a9c640 1 -- 172.21.15.31:0/3642715615 shutdown_connections 2023-11-26T07:22:03.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.157+0000 7feac9a9c640 1 -- 172.21.15.31:0/3642715615 wait complete. 2023-11-26T07:22:03.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 Processor -- start 2023-11-26T07:22:03.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 -- start start 2023-11-26T07:22:03.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40b6c70 0x7feac41567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac40d3ca0 0x7feac40b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac415d880 0x7feac4158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac4156d10 con 0x7feac40d3ca0 2023-11-26T07:22:03.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac4155a50 con 0x7feac415d880 2023-11-26T07:22:03.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac4155cf0 con 0x7feac40b6c70 2023-11-26T07:22:03.165 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feabbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac40d3ca0 0x7feac40b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feabbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac40d3ca0 0x7feac40b7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50812/0 (socket says 172.21.15.31:50812) 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feabbfff640 1 -- 172.21.15.31:0/3442145243 learned_addr learned my addr 172.21.15.31:0/3442145243 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac929b640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac415d880 0x7feac4158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac8a9a640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40b6c70 0x7feac41567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feabbfff640 1 -- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40b6c70 msgr2=0x7feac41567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feabbfff640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40b6c70 0x7feac41567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feabbfff640 1 -- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac415d880 msgr2=0x7feac4158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feabbfff640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac415d880 0x7feac4158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feabbfff640 1 -- 172.21.15.31:0/3442145243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac00094d0 con 0x7feac40d3ca0 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac8a9a640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40b6c70 0x7feac41567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac929b640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac415d880 0x7feac4158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:03.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feabbfff640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac40d3ca0 0x7feac40b7790 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7feab400c7c0 tx=0x7feab400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feab97fa640 1 -- 172.21.15.31:0/3442145243 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4007a30 con 0x7feac40d3ca0 2023-11-26T07:22:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feab97fa640 1 -- 172.21.15.31:0/3442145243 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab4007b90 con 0x7feac40d3ca0 2023-11-26T07:22:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feab97fa640 1 -- 172.21.15.31:0/3442145243 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab400ce70 con 0x7feac40d3ca0 2023-11-26T07:22:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 -- 172.21.15.31:0/3442145243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac4155fd0 con 0x7feac40d3ca0 2023-11-26T07:22:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 -- 172.21.15.31:0/3442145243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac41571c0 con 0x7feac40d3ca0 2023-11-26T07:22:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.161+0000 7feac9a9c640 1 -- 172.21.15.31:0/3442145243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac4074a00 con 0x7feac40d3ca0 2023-11-26T07:22:03.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.165+0000 7feab97fa640 1 -- 172.21.15.31:0/3442145243 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7feab40113f0 con 0x7feac40d3ca0 2023-11-26T07:22:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.165+0000 7feab97fa640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feab00667d0 0x7feab0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.165+0000 7feab97fa640 1 -- 172.21.15.31:0/3442145243 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feab40845e0 con 0x7feac40d3ca0 2023-11-26T07:22:03.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.165+0000 7feac8a9a640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feab00667d0 0x7feab0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.165+0000 7feac8a9a640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feab00667d0 0x7feab0068c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feac00ae000 tx=0x7feac0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:03.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.169+0000 7feab97fa640 1 -- 172.21.15.31:0/3442145243 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feab4051540 con 0x7feac40d3ca0 2023-11-26T07:22:03.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.357+0000 7feac9a9c640 1 -- 172.21.15.31:0/3442145243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feac40d11a0 con 0x7feac40d3ca0 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.357+0000 7feab97fa640 1 -- 172.21.15.31:0/3442145243 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4912 (secure 0 0 0) 0x7feab4054b60 con 0x7feac40d3ca0 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2851182363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2851182363},{"type":"v1","addr":"172.21.15.31:6835","nonce":2851182363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2056572548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2056572548},{"type":"v1","addr":"172.21.15.97:6837","nonce":2056572548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1012539369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1012539369},{"type":"v1","addr":"172.21.15.97:6836","nonce":1012539369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:48.765793+0000","modified":"2023-11-26T07:21:51.814230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6597},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6597":{"gid":6597,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/242493352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":242493352},{"type":"v1","addr":"172.21.15.31:6834","nonce":242493352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:22:03.362 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 48 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 -- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feab00667d0 msgr2=0x7feab0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feab00667d0 0x7feab0068c90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feac00ae000 tx=0x7feac0008040 comp rx=0 tx=0).stop 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 -- 172.21.15.31:0/3442145243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac40d3ca0 msgr2=0x7feac40b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac40d3ca0 0x7feac40b7790 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7feab400c7c0 tx=0x7feab400cc80 comp rx=0 tx=0).stop 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 -- 172.21.15.31:0/3442145243 shutdown_connections 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7feab00667d0 0x7feab0068c90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feac415d880 0x7feac4158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feac40d3ca0 0x7feac40b7790 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 --2- 172.21.15.31:0/3442145243 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7feac40b6c70 0x7feac41567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 -- 172.21.15.31:0/3442145243 >> 172.21.15.31:0/3442145243 conn(0x7feac406b9f0 msgr2=0x7feac415dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 -- 172.21.15.31:0/3442145243 shutdown_connections 2023-11-26T07:22:03.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.361+0000 7feac9a9c640 1 -- 172.21.15.31:0/3442145243 wait complete. 2023-11-26T07:22:03.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-26T07:22:03.546 INFO:teuthology.orchestra.run.smithi031.stdout:[] 2023-11-26T07:22:03.546 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2023-11-26T07:22:03.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol54170 2023-11-26T07:22:03.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f7989b2a640 1 Processor -- start 2023-11-26T07:22:03.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f7989b2a640 1 -- start start 2023-11-26T07:22:03.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f7989b2a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79840ae680 0x7f79840cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f7989b2a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7984070890 0x7f7984070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f7989b2a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7984072950 con 0x7f7984159f00 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f7989b2a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798406e2b0 con 0x7f79840ae680 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f7989b2a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798406c6f0 con 0x7f7984070890 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f7982ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79840ae680 0x7f79840cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f7982ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79840ae680 0x7f79840cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58228/0 (socket says 172.21.15.31:58228) 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f7982ffd640 1 -- 172.21.15.31:0/4145251175 learned_addr learned my addr 172.21.15.31:0/4145251175 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.689+0000 7f79837fe640 1 --1- 172.21.15.31:0/4145251175 >> v1:172.21.15.31:6789/0 conn(0x7f7984159f00 0x7f79840ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45858/0 (socket says 172.21.15.31:45858) 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f7983fff640 1 --2- 172.21.15.31:0/4145251175 >> v2:172.21.15.97:3300/0 conn(0x7f7984070890 0x7f7984070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f7982ffd640 1 -- 172.21.15.31:0/4145251175 >> v2:172.21.15.97:3300/0 conn(0x7f7984070890 msgr2=0x7f7984070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:03.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f7982ffd640 1 --2- 172.21.15.31:0/4145251175 >> v2:172.21.15.97:3300/0 conn(0x7f7984070890 0x7f7984070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f7982ffd640 1 -- 172.21.15.31:0/4145251175 >> v1:172.21.15.31:6789/0 conn(0x7f7984159f00 legacy=0x7f79840ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f7982ffd640 1 -- 172.21.15.31:0/4145251175 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798406e550 con 0x7f79840ae680 2023-11-26T07:22:03.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f7983fff640 1 --2- 172.21.15.31:0/4145251175 >> v2:172.21.15.97:3300/0 conn(0x7f7984070890 0x7f7984070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:03.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f7982ffd640 1 --2- 172.21.15.31:0/4145251175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 0x7f79840cd5b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7984070360 tx=0x7f797801d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa207cdecd4caf83 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:03.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f79827fc640 1 -- 172.21.15.31:0/4145251175 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780a68d0 con 0x7f79840ae680 2023-11-26T07:22:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f79827fc640 1 -- 172.21.15.31:0/4145251175 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79780a6a30 con 0x7f79840ae680 2023-11-26T07:22:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f79827fc640 1 -- 172.21.15.31:0/4145251175 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780a6dc0 con 0x7f79840ae680 2023-11-26T07:22:03.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f7989b2a640 1 -- 172.21.15.31:0/4145251175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 msgr2=0x7f79840cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.693+0000 7f7989b2a640 1 --2- 172.21.15.31:0/4145251175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 0x7f79840cd5b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7984070360 tx=0x7f797801d8a0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 -- 172.21.15.31:0/4145251175 shutdown_connections 2023-11-26T07:22:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 --2- 172.21.15.31:0/4145251175 >> v2:172.21.15.97:3300/0 conn(0x7f7984070890 0x7f7984070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 --2- 172.21.15.31:0/4145251175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 0x7f79840cd5b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 -- 172.21.15.31:0/4145251175 >> 172.21.15.31:0/4145251175 conn(0x7f79840b0cb0 msgr2=0x7f79840b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 -- 172.21.15.31:0/4145251175 shutdown_connections 2023-11-26T07:22:03.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 -- 172.21.15.31:0/4145251175 wait complete. 2023-11-26T07:22:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 Processor -- start 2023-11-26T07:22:03.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 -- start start 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984070890 0x7f798414c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 0x7f798415ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f798415b7a0 0x7f79841561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798415b1d0 con 0x7f79840ae680 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798415cd40 con 0x7f7984070890 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7989b2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798415d1d0 con 0x7f798415b7a0 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7982ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 0x7f798415ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7982ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 0x7f798415ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58238/0 (socket says 172.21.15.31:58238) 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7982ffd640 1 -- 172.21.15.31:0/3765031623 learned_addr learned my addr 172.21.15.31:0/3765031623 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f79837fe640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984070890 0x7f798414c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7983fff640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f798415b7a0 0x7f79841561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7982ffd640 1 -- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f798415b7a0 msgr2=0x7f79841561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7982ffd640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f798415b7a0 0x7f79841561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7982ffd640 1 -- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984070890 msgr2=0x7f798414c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7982ffd640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984070890 0x7f798414c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7982ffd640 1 -- 172.21.15.31:0/3765031623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797800a670 con 0x7f79840ae680 2023-11-26T07:22:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.697+0000 7f7983fff640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f798415b7a0 0x7f79841561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f7982ffd640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 0x7f798415ac90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f797800a1c0 tx=0x7f797801c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f796bfff640 1 -- 172.21.15.31:0/3765031623 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780aea50 con 0x7f79840ae680 2023-11-26T07:22:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f796bfff640 1 -- 172.21.15.31:0/3765031623 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79780aebb0 con 0x7f79840ae680 2023-11-26T07:22:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f7989b2a640 1 -- 172.21.15.31:0/3765031623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798414e460 con 0x7f79840ae680 2023-11-26T07:22:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f796bfff640 1 -- 172.21.15.31:0/3765031623 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780b9450 con 0x7f79840ae680 2023-11-26T07:22:03.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f7989b2a640 1 -- 172.21.15.31:0/3765031623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798415c6f0 con 0x7f79840ae680 2023-11-26T07:22:03.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f7989b2a640 1 -- 172.21.15.31:0/3765031623 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79840671f0 con 0x7f79840ae680 2023-11-26T07:22:03.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f796bfff640 1 -- 172.21.15.31:0/3765031623 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7f79780b5070 con 0x7f79840ae680 2023-11-26T07:22:03.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f796bfff640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79700667d0 0x7f7970068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:03.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f796bfff640 1 -- 172.21.15.31:0/3765031623 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f797811f250 con 0x7f79840ae680 2023-11-26T07:22:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.701+0000 7f79837fe640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79700667d0 0x7f7970068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:03.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.705+0000 7f79837fe640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79700667d0 0x7f7970068c90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7974000f70 tx=0x7f7974007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:03.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.705+0000 7f796bfff640 1 -- 172.21.15.31:0/3765031623 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79780ec350 con 0x7f79840ae680 2023-11-26T07:22:03.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.889+0000 7f7989b2a640 1 -- 172.21.15.31:0/3765031623 --> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol54170", "target": ["mon-mgr", ""]}) v1 -- 0x7f7984071230 con 0x7f79700667d0 2023-11-26T07:22:03.899 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:03.896+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:03.899 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:03.896+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:03.899 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:03.896+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:03.899 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:03.896+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:03.899 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:03.896+0000 7f2400996640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:03.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.929+0000 7f796bfff640 1 -- 172.21.15.31:0/3765031623 <== mgr.4100 v2:172.21.15.97:6832/3008114855 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7984071230 con 0x7f79700667d0 2023-11-26T07:22:03.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 -- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79700667d0 msgr2=0x7f7970068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79700667d0 0x7f7970068c90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7974000f70 tx=0x7f7974007c90 comp rx=0 tx=0).stop 2023-11-26T07:22:03.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 -- 172.21.15.31:0/3765031623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 msgr2=0x7f798415ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:03.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 0x7f798415ac90 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f797800a1c0 tx=0x7f797801c0e0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 -- 172.21.15.31:0/3765031623 shutdown_connections 2023-11-26T07:22:03.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7f79700667d0 0x7f7970068c90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f798415b7a0 0x7f79841561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79840ae680 0x7f798415ac90 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 --2- 172.21.15.31:0/3765031623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7984070890 0x7f798414c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:03.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 -- 172.21.15.31:0/3765031623 >> 172.21.15.31:0/3765031623 conn(0x7f79840b0cb0 msgr2=0x7f79840ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:03.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 -- 172.21.15.31:0/3765031623 shutdown_connections 2023-11-26T07:22:03.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:03.933+0000 7f7989b2a640 1 -- 172.21.15.31:0/3765031623 wait complete. 2023-11-26T07:22:03.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:04.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.093+0000 7fddf4fa1640 1 Processor -- start 2023-11-26T07:22:04.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.093+0000 7fddf4fa1640 1 -- start start 2023-11-26T07:22:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.093+0000 7fddf4fa1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddf0146c60 0x7fddf0070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.093+0000 7fddf4fa1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fddf0147040 0x7fddf0070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.093+0000 7fddf4fa1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf015dba0 con 0x7fddf006bb10 2023-11-26T07:22:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.093+0000 7fddf4fa1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf006c670 con 0x7fddf0146c60 2023-11-26T07:22:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.093+0000 7fddf4fa1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf006d100 con 0x7fddf0147040 2023-11-26T07:22:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.093+0000 7fddee59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddf0146c60 0x7fddf0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.093+0000 7fddee59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fddf0146c60 0x7fddf0070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58244/0 (socket says 172.21.15.31:58244) 2023-11-26T07:22:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.093+0000 7fddee59c640 1 -- 172.21.15.31:0/1563314804 learned_addr learned my addr 172.21.15.31:0/1563314804 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.097+0000 7fdde7fff640 1 --2- 172.21.15.31:0/1563314804 >> v2:172.21.15.97:3300/0 conn(0x7fddf0147040 0x7fddf0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.097+0000 7fddee59c640 1 -- 172.21.15.31:0/1563314804 >> v2:172.21.15.97:3300/0 conn(0x7fddf0147040 msgr2=0x7fddf0070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.097+0000 7fddee59c640 1 --2- 172.21.15.31:0/1563314804 >> v2:172.21.15.97:3300/0 conn(0x7fddf0147040 0x7fddf0070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.097+0000 7fddee59c640 1 -- 172.21.15.31:0/1563314804 >> v1:172.21.15.97:6789/0 conn(0x7fddf006bb10 legacy=0x7fddf006bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:04.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.097+0000 7fddee59c640 1 -- 172.21.15.31:0/1563314804 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf006c860 con 0x7fddf0146c60 2023-11-26T07:22:04.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.097+0000 7fddee59c640 1 --2- 172.21.15.31:0/1563314804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 0x7fddf0070020 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fddf006e1f0 tx=0x7fdde001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7f81837261d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:04.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.097+0000 7fddedd9b640 1 -- 172.21.15.31:0/1563314804 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde00ab600 con 0x7fddf0146c60 2023-11-26T07:22:04.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.097+0000 7fddedd9b640 1 -- 172.21.15.31:0/1563314804 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde00ab760 con 0x7fddf0146c60 2023-11-26T07:22:04.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.097+0000 7fddedd9b640 1 -- 172.21.15.31:0/1563314804 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde00abaf0 con 0x7fddf0146c60 2023-11-26T07:22:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 -- 172.21.15.31:0/1563314804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 msgr2=0x7fddf0070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 --2- 172.21.15.31:0/1563314804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 0x7fddf0070020 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fddf006e1f0 tx=0x7fdde001c400 comp rx=0 tx=0).stop 2023-11-26T07:22:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 -- 172.21.15.31:0/1563314804 shutdown_connections 2023-11-26T07:22:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 --2- 172.21.15.31:0/1563314804 >> v2:172.21.15.97:3300/0 conn(0x7fddf0147040 0x7fddf0070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 --2- 172.21.15.31:0/1563314804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 0x7fddf0070020 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 -- 172.21.15.31:0/1563314804 >> 172.21.15.31:0/1563314804 conn(0x7fddf00b2ea0 msgr2=0x7fddf00b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 -- 172.21.15.31:0/1563314804 shutdown_connections 2023-11-26T07:22:04.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 -- 172.21.15.31:0/1563314804 wait complete. 2023-11-26T07:22:04.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 Processor -- start 2023-11-26T07:22:04.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 -- start start 2023-11-26T07:22:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddf006bb10 0x7fddf013b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 0x7fddf013e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf0147040 0x7fddf015c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf0155800 con 0x7fddf0146c60 2023-11-26T07:22:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf013c390 con 0x7fddf0147040 2023-11-26T07:22:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddf4fa1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf013c670 con 0x7fddf006bb10 2023-11-26T07:22:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fdde7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 0x7fddf013e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fdde7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 0x7fddf013e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58248/0 (socket says 172.21.15.31:58248) 2023-11-26T07:22:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fdde7fff640 1 -- 172.21.15.31:0/3987934424 learned_addr learned my addr 172.21.15.31:0/3987934424 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:04.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddee59c640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddf006bb10 0x7fddf013b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddeed9d640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf0147040 0x7fddf015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fdde7fff640 1 -- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddf006bb10 msgr2=0x7fddf013b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fdde7fff640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddf006bb10 0x7fddf013b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fdde7fff640 1 -- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf0147040 msgr2=0x7fddf015c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fdde7fff640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf0147040 0x7fddf015c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fdde7fff640 1 -- 172.21.15.31:0/3987934424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00094d0 con 0x7fddf0146c60 2023-11-26T07:22:04.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.101+0000 7fddeed9d640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf0147040 0x7fddf015c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:04.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fdde7fff640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 0x7fddf013e150 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fddd800d5f0 tx=0x7fddd800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:04.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fdde6ffd640 1 -- 172.21.15.31:0/3987934424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd80125f0 con 0x7fddf0146c60 2023-11-26T07:22:04.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fdde6ffd640 1 -- 172.21.15.31:0/3987934424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd8012750 con 0x7fddf0146c60 2023-11-26T07:22:04.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fdde6ffd640 1 -- 172.21.15.31:0/3987934424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8012ae0 con 0x7fddf0146c60 2023-11-26T07:22:04.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fddf4fa1640 1 -- 172.21.15.31:0/3987934424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf013c950 con 0x7fddf0146c60 2023-11-26T07:22:04.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fddf4fa1640 1 -- 172.21.15.31:0/3987934424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf0140050 con 0x7fddf0146c60 2023-11-26T07:22:04.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fddf4fa1640 1 -- 172.21.15.31:0/3987934424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf0140210 con 0x7fddf0146c60 2023-11-26T07:22:04.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fdde6ffd640 1 -- 172.21.15.31:0/3987934424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 10) v1 ==== 86158+0+0 (secure 0 0 0) 0x7fddd8012c40 con 0x7fddf0146c60 2023-11-26T07:22:04.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fdde6ffd640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddd00667d0 0x7fddd0068c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fdde6ffd640 1 -- 172.21.15.31:0/3987934424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddd8084840 con 0x7fddf0146c60 2023-11-26T07:22:04.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.105+0000 7fddee59c640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddd00667d0 0x7fddd0068c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.109+0000 7fddee59c640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddd00667d0 0x7fddd0068c90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdde00af000 tx=0x7fdde00b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:04.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.109+0000 7fdde6ffd640 1 -- 172.21.15.31:0/3987934424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fddd8051870 con 0x7fddf0146c60 2023-11-26T07:22:04.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 -- 172.21.15.31:0/3987934424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddf01682c0 con 0x7fddf0146c60 2023-11-26T07:22:04.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fdde6ffd640 1 -- 172.21.15.31:0/3987934424 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4912 (secure 0 0 0) 0x7fddd8007380 con 0x7fddf0146c60 2023-11-26T07:22:04.303 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:04.304 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2851182363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2851182363},{"type":"v1","addr":"172.21.15.31:6835","nonce":2851182363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2056572548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2056572548},{"type":"v1","addr":"172.21.15.97:6837","nonce":2056572548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1012539369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1012539369},{"type":"v1","addr":"172.21.15.97:6836","nonce":1012539369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:48.765793+0000","modified":"2023-11-26T07:21:51.814230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6597},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6597":{"gid":6597,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/242493352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":242493352},{"type":"v1","addr":"172.21.15.31:6834","nonce":242493352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:22:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 48 2023-11-26T07:22:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 -- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddd00667d0 msgr2=0x7fddd0068c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddd00667d0 0x7fddd0068c90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdde00af000 tx=0x7fdde00b6040 comp rx=0 tx=0).stop 2023-11-26T07:22:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 -- 172.21.15.31:0/3987934424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 msgr2=0x7fddf013e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 0x7fddf013e150 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fddd800d5f0 tx=0x7fddd800dab0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 -- 172.21.15.31:0/3987934424 shutdown_connections 2023-11-26T07:22:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fddd00667d0 0x7fddd0068c90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddf0147040 0x7fddf015c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddf0146c60 0x7fddf013e150 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 --2- 172.21.15.31:0/3987934424 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddf006bb10 0x7fddf013b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 -- 172.21.15.31:0/3987934424 >> 172.21.15.31:0/3987934424 conn(0x7fddf00b2ea0 msgr2=0x7fddf0145bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 -- 172.21.15.31:0/3987934424 shutdown_connections 2023-11-26T07:22:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.301+0000 7fddf4fa1640 1 -- 172.21.15.31:0/3987934424 wait complete. 2023-11-26T07:22:04.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-26T07:22:04.492 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":6005,"entity":{"name":{"type":"client","num":6005},"addr":{"type":"any","addr":"172.21.15.97:0","nonce":1921595339}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.56799439200000001,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.97170112647574,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6005 172.21.15.97:0/1921595339","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"x","hostname":"smithi097","pid":"38054","root":"/"}}] 2023-11-26T07:22:04.493 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi097", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "38054", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.97:0", "nonce": 1921595339, "type": "any" }, "name": { "num": 6005, "type": "client" } }, "id": 6005, "inst": "client.6005 172.21.15.97:0/1921595339", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.97170112647574 }, "state": "open", "uptime": 0.567994392 } ] 2023-11-26T07:22:04.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-26T07:22:04.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.633+0000 7ff51549a640 1 Processor -- start 2023-11-26T07:22:04.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.633+0000 7ff51549a640 1 -- start start 2023-11-26T07:22:04.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.633+0000 7ff51549a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510159880 0x7ff5100b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.633+0000 7ff51549a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5100b41a0 con 0x7ff510071600 2023-11-26T07:22:04.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.633+0000 7ff51549a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5100b3be0 con 0x7ff5100cb230 2023-11-26T07:22:04.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.633+0000 7ff51549a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100b3ec0 con 0x7ff510159880 2023-11-26T07:22:04.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.633+0000 7ff50f7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff510071600 0x7ff51006d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45868/0 (socket says 172.21.15.31:45868) 2023-11-26T07:22:04.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.633+0000 7ff50f7fe640 1 -- 172.21.15.31:0/3289484465 learned_addr learned my addr 172.21.15.31:0/3289484465 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:04.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50effd640 1 --2- 172.21.15.31:0/3289484465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510159880 0x7ff5100b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50e7fc640 1 -- 172.21.15.31:0/3289484465 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799304694 0 0) 0x7ff5100b41a0 con 0x7ff510071600 2023-11-26T07:22:04.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50e7fc640 1 -- 172.21.15.31:0/3289484465 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4fc003450 con 0x7ff510071600 2023-11-26T07:22:04.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50e7fc640 1 -- 172.21.15.31:0/3289484465 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184242490 0 0) 0x7ff5100b3be0 con 0x7ff5100cb230 2023-11-26T07:22:04.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50e7fc640 1 -- 172.21.15.31:0/3289484465 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5100b41a0 con 0x7ff5100cb230 2023-11-26T07:22:04.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50effd640 1 -- 172.21.15.31:0/3289484465 >> v1:172.21.15.97:6789/0 conn(0x7ff5100cb230 legacy=0x7ff510070ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50e7fc640 1 -- 172.21.15.31:0/3289484465 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2863942573 0 0) 0x7ff4fc003450 con 0x7ff510071600 2023-11-26T07:22:04.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50effd640 1 -- 172.21.15.31:0/3289484465 >> v1:172.21.15.31:6789/0 conn(0x7ff510071600 legacy=0x7ff51006d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50effd640 1 -- 172.21.15.31:0/3289484465 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51006cf60 con 0x7ff510159880 2023-11-26T07:22:04.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50effd640 1 --2- 172.21.15.31:0/3289484465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510159880 0x7ff5100b3320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff5000097f0 tx=0x7ff50001c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd07d57366814119 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:04.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.637+0000 7ff50e7fc640 1 -- 172.21.15.31:0/3289484465 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508001a00 con 0x7ff510159880 2023-11-26T07:22:04.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff50e7fc640 1 -- 172.21.15.31:0/3289484465 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5000a7050 con 0x7ff510159880 2023-11-26T07:22:04.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff50e7fc640 1 -- 172.21.15.31:0/3289484465 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5000ab600 con 0x7ff510159880 2023-11-26T07:22:04.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 -- 172.21.15.31:0/3289484465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510159880 msgr2=0x7ff5100b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 --2- 172.21.15.31:0/3289484465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510159880 0x7ff5100b3320 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff5000097f0 tx=0x7ff50001c320 comp rx=0 tx=0).stop 2023-11-26T07:22:04.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 -- 172.21.15.31:0/3289484465 shutdown_connections 2023-11-26T07:22:04.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 --2- 172.21.15.31:0/3289484465 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510159880 0x7ff5100b3320 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 -- 172.21.15.31:0/3289484465 >> 172.21.15.31:0/3289484465 conn(0x7ff5100b0c40 msgr2=0x7ff5100b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:04.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 -- 172.21.15.31:0/3289484465 shutdown_connections 2023-11-26T07:22:04.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 -- 172.21.15.31:0/3289484465 wait complete. 2023-11-26T07:22:04.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 Processor -- start 2023-11-26T07:22:04.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 -- start start 2023-11-26T07:22:04.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510071600 0x7ff510143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5100cb230 0x7ff510099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510159880 0x7ff51014f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100ce4f0 con 0x7ff510159880 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff510143eb0 con 0x7ff5100cb230 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff51549a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51014e4c0 con 0x7ff510071600 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff50f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510159880 0x7ff51014f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff50f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510159880 0x7ff51014f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58250/0 (socket says 172.21.15.31:58250) 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.641+0000 7ff50f7fe640 1 -- 172.21.15.31:0/1149098337 learned_addr learned my addr 172.21.15.31:0/1149098337 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff50effd640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510071600 0x7ff510143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff507fff640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5100cb230 0x7ff510099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff50f7fe640 1 -- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510071600 msgr2=0x7ff510143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff50f7fe640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510071600 0x7ff510143970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff50f7fe640 1 -- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5100cb230 msgr2=0x7ff510099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff50f7fe640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5100cb230 0x7ff510099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff50f7fe640 1 -- 172.21.15.31:0/1149098337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff500009420 con 0x7ff510159880 2023-11-26T07:22:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff50f7fe640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510159880 0x7ff51014f560 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff508001f80 tx=0x7ff50800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff5077fe640 1 -- 172.21.15.31:0/1149098337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508012400 con 0x7ff510159880 2023-11-26T07:22:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff5077fe640 1 -- 172.21.15.31:0/1149098337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff508012560 con 0x7ff510159880 2023-11-26T07:22:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff51549a640 1 -- 172.21.15.31:0/1149098337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51014e7a0 con 0x7ff510159880 2023-11-26T07:22:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff5077fe640 1 -- 172.21.15.31:0/1149098337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508012950 con 0x7ff510159880 2023-11-26T07:22:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff51549a640 1 -- 172.21.15.31:0/1149098337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51014ed60 con 0x7ff510159880 2023-11-26T07:22:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff51549a640 1 -- 172.21.15.31:0/1149098337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51004e850 con 0x7ff510159880 2023-11-26T07:22:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.645+0000 7ff5077fe640 1 -- 172.21.15.31:0/1149098337 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 86209+0+0 (secure 0 0 0) 0x7ff508007070 con 0x7ff510159880 2023-11-26T07:22:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.649+0000 7ff5077fe640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff4f0066860 0x7ff4f0068d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:04.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.649+0000 7ff5077fe640 1 -- 172.21.15.31:0/1149098337 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff508084630 con 0x7ff510159880 2023-11-26T07:22:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.649+0000 7ff50effd640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff4f0066860 0x7ff4f0068d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:04.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.649+0000 7ff50effd640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff4f0066860 0x7ff4f0068d20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff5000ad000 tx=0x7ff500008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:04.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.653+0000 7ff5077fe640 1 -- 172.21.15.31:0/1149098337 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff508051550 con 0x7ff510159880 2023-11-26T07:22:04.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.849+0000 7ff51549a640 1 -- 172.21.15.31:0/1149098337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7ff5100b24c0 con 0x7ff510159880 2023-11-26T07:22:04.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.853+0000 7ff5077fe640 1 -- 172.21.15.31:0/1149098337 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) v1 ==== 81+0+448255 (secure 0 0 0) 0x7ff508054b70 con 0x7ff510159880 2023-11-26T07:22:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:04.862 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:22:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 11, 2023-11-26T07:22:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "active_gid": 4100, 2023-11-26T07:22:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "active_name": "x", 2023-11-26T07:22:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addrs": { 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:6832", 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3008114855 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:6833", 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3008114855 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "active_addr": "172.21.15.97:6833/3008114855", 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "active_change": "2023-11-26T07:17:12.293862+0000", 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "active_mgr_features": 4540138322906710015, 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "available": true, 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "standbys": [ 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 4110, 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "y", 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_features": 4540138322906710015, 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-11-26T07:22:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.867 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.870 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.871 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.872 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.873 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.874 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-11-26T07:22:04.875 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.876 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.877 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-11-26T07:22:04.878 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.879 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.880 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-11-26T07:22:04.881 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.882 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.883 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.884 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-11-26T07:22:04.885 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-11-26T07:22:04.886 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-26T07:22:04.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.889 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-11-26T07:22:04.890 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.891 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.892 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.893 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.895 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.896 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.897 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-11-26T07:22:04.898 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-11-26T07:22:04.899 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.902 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.903 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:04.904 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.906 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.907 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.908 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.909 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-11-26T07:22:04.910 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.911 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2023-11-26T07:22:04.912 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.913 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.915 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-11-26T07:22:04.916 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.919 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.920 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-26T07:22:04.921 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.922 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2023-11-26T07:22:04.923 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2023-11-26T07:22:04.924 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.925 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-11-26T07:22:04.927 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.928 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-11-26T07:22:04.929 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-11-26T07:22:04.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.932 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-11-26T07:22:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.935 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-11-26T07:22:04.936 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.937 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-11-26T07:22:04.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-11-26T07:22:04.939 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.941 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.942 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.943 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.944 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-26T07:22:04.946 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.947 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:04.948 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.951 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-11-26T07:22:04.952 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-11-26T07:22:04.954 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.956 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-11-26T07:22:04.957 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.958 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.959 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-11-26T07:22:04.960 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.961 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.962 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-26T07:22:04.963 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.964 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2023-11-26T07:22:04.965 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.966 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-11-26T07:22:04.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.968 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.969 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-11-26T07:22:04.970 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.971 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.972 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:04.973 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:04.974 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:04.975 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.976 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:04.977 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.978 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.979 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.980 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-26T07:22:04.981 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-26T07:22:04.982 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.985 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.986 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.987 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.988 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.989 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-26T07:22:04.990 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.991 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.992 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.993 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.995 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.997 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:04.998 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:04.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.001 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.002 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.003 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.004 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-26T07:22:05.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.006 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.007 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.008 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.009 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.010 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-26T07:22:05.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.013 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.014 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.015 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.016 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.017 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.018 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-26T07:22:05.019 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.021 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.022 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-11-26T07:22:05.023 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-26T07:22:05.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.026 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.027 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.028 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.029 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-11-26T07:22:05.030 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-26T07:22:05.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.033 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.035 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-11-26T07:22:05.037 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.040 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.043 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.044 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-11-26T07:22:05.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.047 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.048 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.049 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.050 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-11-26T07:22:05.051 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.052 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.053 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-11-26T07:22:05.056 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.057 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.058 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.059 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.060 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.061 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-11-26T07:22:05.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-11-26T07:22:05.064 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.065 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-11-26T07:22:05.066 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.067 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.068 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.069 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.070 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.072 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.074 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.075 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.076 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.077 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.078 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.079 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.080 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.082 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.083 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-11-26T07:22:05.084 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.087 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.090 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.091 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.094 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.095 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.096 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.097 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-11-26T07:22:05.098 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.101 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.102 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.103 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.104 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.105 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.106 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.107 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.109 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.110 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.113 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.114 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.115 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.116 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.118 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.119 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.120 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-11-26T07:22:05.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.122 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.123 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.124 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.125 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-11-26T07:22:05.126 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.127 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.128 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.130 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.131 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.134 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.135 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.136 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.138 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.139 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.143 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.144 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.145 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.146 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-11-26T07:22:05.147 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.148 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.151 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.152 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.153 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.154 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.155 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.157 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.158 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.160 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.161 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.162 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.163 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.164 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.165 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-11-26T07:22:05.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.168 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.170 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-11-26T07:22:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.173 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.175 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.176 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-11-26T07:22:05.178 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-11-26T07:22:05.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.182 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-26T07:22:05.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.184 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.185 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.186 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.187 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.192 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-11-26T07:22:05.193 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-11-26T07:22:05.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.196 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.197 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-11-26T07:22:05.198 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.199 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.200 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.201 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.203 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.205 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.206 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.208 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.209 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.211 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.212 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.215 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.216 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.217 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.218 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.219 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.220 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.221 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.223 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.224 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-11-26T07:22:05.226 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.229 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.230 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-11-26T07:22:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-11-26T07:22:05.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.234 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.235 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.237 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-11-26T07:22:05.238 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-11-26T07:22:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.243 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.244 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.245 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.247 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.248 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.250 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.251 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.252 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.254 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.255 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-11-26T07:22:05.256 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.257 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.258 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.259 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.260 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.263 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-11-26T07:22:05.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.265 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.266 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.267 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.268 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.269 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.272 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.273 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-11-26T07:22:05.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-11-26T07:22:05.275 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.278 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.279 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.280 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-11-26T07:22:05.282 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.283 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.284 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.285 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.286 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.287 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.288 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.289 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.290 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.291 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.293 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.294 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-26T07:22:05.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.296 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.297 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.300 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.301 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.303 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.304 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.305 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.307 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.308 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-11-26T07:22:05.309 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.310 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-11-26T07:22:05.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.313 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-11-26T07:22:05.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-11-26T07:22:05.315 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.317 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.320 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.321 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.322 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.323 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.325 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: "modules": [ 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: "iostat", 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: "nfs", 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: "restful" 2023-11-26T07:22:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "available_modules": [ 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "alerts", 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-11-26T07:22:05.327 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How frequently to reexamine health status", 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.328 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.329 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.330 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.331 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.333 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_destination": { 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_destination", 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.334 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email address to send alerts to", 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_from_name": { 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_from_name", 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "Ceph", 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Email From: name", 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_host": { 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_host", 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.336 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP server", 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_password": { 2023-11-26T07:22:05.337 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_password", 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Password to authenticate with", 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_port": { 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_port", 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "465", 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP port", 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_sender": { 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_sender", 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.340 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SMTP envelope sender", 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_ssl": { 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_ssl", 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.341 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: "smtp_user": { 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "smtp_user", 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "User to authenticate as", 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.343 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "balancer", 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "active": { 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "active", 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.344 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically balance PGs across cluster", 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_time": { 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_time", 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.345 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0000", 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "beginning time of day to automatically balance", 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "begin_weekday": { 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "begin_weekday", 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-11-26T07:22:05.346 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_max_iterations": { 2023-11-26T07:22:05.347 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_max_iterations", 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "25", 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "250", 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_metrics": { 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_metrics", 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "pgs,objects,bytes", 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-26T07:22:05.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_compat_step": { 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crush_compat_step", 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.5", 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0.001", 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "0.999", 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.350 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "aggressiveness of optimization", 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "end_time": { 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_time", 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2359", 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.351 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ending time of day to automatically balance", 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "end_weekday": { 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "end_weekday", 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-11-26T07:22:05.352 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "0", 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "6", 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.354 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.355 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.356 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.358 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min_score": { 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_score", 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.359 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "upmap", 2023-11-26T07:22:05.360 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-compat", 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: "none", 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap" 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Balancer mode", 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.361 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_ids": { 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_ids", 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.362 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-26T07:22:05.363 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_deviation": { 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_deviation", 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.364 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "upmap_max_optimizations": { 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "upmap_max_optimizations", 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:05.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.366 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm", 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_down_multiplier": { 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_down_multiplier", 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.367 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_refresh_rate": { 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_refresh_rate", 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "20", 2023-11-26T07:22:05.368 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "agent_starting_port": { 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "agent_starting_port", 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.369 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4721", 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_ptrace": { 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_ptrace", 2023-11-26T07:22:05.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_interval": { 2023-11-26T07:22:05.371 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_interval", 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "autotune_memory_target_ratio": { 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "autotune_memory_target_ratio", 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0.7", 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_log_destination": { 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephadm_log_destination", 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.374 INFO:teuthology.orchestra.run.smithi031.stdout: "file", 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: "file,syslog", 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: "syslog" 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: "cgroups_split": { 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cgroups_split", 2023-11-26T07:22:05.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.376 INFO:teuthology.orchestra.run.smithi031.stdout: "config_checks_enabled": { 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_checks_enabled", 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.377 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "config_dashboard": { 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "config_dashboard", 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-26T07:22:05.378 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_alertmanager": { 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_alertmanager", 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.379 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_base": { 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_base", 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.380 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Container image name, without the tag", 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_elasticsearch": { 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_elasticsearch", 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.381 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "elasticsearch container image", 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_grafana": { 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_grafana", 2023-11-26T07:22:05.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.383 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_haproxy": { 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_haproxy", 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "HAproxy container image", 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.384 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_agent": { 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_agent", 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger agent container image", 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_collector": { 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_collector", 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.386 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger collector container image", 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_jaeger_query": { 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_jaeger_query", 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.387 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Jaeger query container image", 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_keepalived": { 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_keepalived", 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-26T07:22:05.388 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Keepalived container image", 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_loki": { 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_loki", 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Loki container image", 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_node_exporter": { 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_node_exporter", 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.390 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_nvmeof": { 2023-11-26T07:22:05.391 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_nvmeof", 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Nvme-of container image", 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.392 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_prometheus": { 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_prometheus", 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Prometheus container image", 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.393 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_promtail": { 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_promtail", 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Promtail container image", 2023-11-26T07:22:05.394 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image_snmp_gateway": { 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_image_snmp_gateway", 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.395 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "SNMP Gateway container image", 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "container_init": { 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "container_init", 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.396 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Run podman/docker with `--init`", 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "daemon_cache_timeout": { 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "daemon_cache_timeout", 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "default_cephadm_command_timeout": { 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_cephadm_command_timeout", 2023-11-26T07:22:05.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "900", 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.399 INFO:teuthology.orchestra.run.smithi031.stdout: "default_registry": { 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "default_registry", 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "docker.io", 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.400 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "device_cache_timeout": { 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_cache_timeout", 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1800", 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache device inventory", 2023-11-26T07:22:05.401 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: "device_enhanced_scan": { 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_enhanced_scan", 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.402 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "facts_cache_timeout": { 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "facts_cache_timeout", 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.403 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "seconds to cache host facts data", 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "host_check_interval": { 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "host_check_interval", 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.404 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to perform a host check", 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "inventory_list_all": { 2023-11-26T07:22:05.405 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "inventory_list_all", 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.406 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.407 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "log_refresh_metadata": { 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_refresh_metadata", 2023-11-26T07:22:05.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.409 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.411 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.412 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-26T07:22:05.414 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "*", 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.415 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max_count_per_host": { 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_count_per_host", 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.416 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of daemons per service per host", 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd_draining_count": { 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_osd_draining_count", 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "migration_current": { 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "migration_current", 2023-11-26T07:22:05.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "internal - do not modify", 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": { 2023-11-26T07:22:05.419 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mode", 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "root", 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm-package", 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "root" 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "mode for remote execution of cephadm", 2023-11-26T07:22:05.420 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "prometheus_alerts_path": { 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus_alerts_path", 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.421 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_insecure": { 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_insecure", 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.422 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_password": { 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_password", 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_url": { 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_url", 2023-11-26T07:22:05.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "registry_username": { 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "registry_username", 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.426 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "secure_monitoring_stack": { 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "secure_monitoring_stack", 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-26T07:22:05.427 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "service_discovery_port": { 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "service_discovery_port", 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8765", 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.428 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "cephadm service discovery port", 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ssh_config_file": { 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssh_config_file", 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.429 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "use_agent": { 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_agent", 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.430 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "use_repo_digest": { 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "use_repo_digest", 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.431 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_failed_host_check": { 2023-11-26T07:22:05.432 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_failed_host_check", 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if the host check fails", 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_daemons": { 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_daemons", 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.434 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_on_stray_hosts": { 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_on_stray_hosts", 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.435 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crash", 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.436 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.437 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.439 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.440 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "retain_interval": { 2023-11-26T07:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retain_interval", 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "31536000", 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_recent_interval": { 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_recent_interval", 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1209600", 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dashboard", 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.446 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_HOST": { 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.447 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.448 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_ENABLED": { 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_ENABLED", 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.449 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.450 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "ENABLE_BROWSABLE_API": { 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.452 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-26T07:22:05.453 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.456 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.457 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.458 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.459 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-26T07:22:05.460 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.461 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_PASSWORD": { 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.463 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_URL": { 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_URL", 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.464 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_API_USERNAME": { 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-11-26T07:22:05.465 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-26T07:22:05.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-26T07:22:05.468 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_HOST": { 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.471 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.472 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.473 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-26T07:22:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.476 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.477 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.479 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-26T07:22:05.480 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_ENABLED": { 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.482 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.483 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.484 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.485 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8", 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.486 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "45", 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ACCESS_KEY": { 2023-11-26T07:22:05.487 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.488 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "admin", 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SECRET_KEY": { 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.490 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "RGW_API_SSL_VERIFY": { 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.491 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "UNSAFE_TLS_v1_2": { 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "UNSAFE_TLS_v1_2", 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.492 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-11-26T07:22:05.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.494 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-26T07:22:05.495 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.496 INFO:teuthology.orchestra.run.smithi031.stdout: "cross_origin_url": { 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cross_origin_url", 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.497 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "crt_file": { 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "crt_file", 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.498 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "debug": { 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "debug", 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.499 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Enable/disable debug options", 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "jwt_token_ttl": { 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "jwt_token_ttl", 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "28800", 2023-11-26T07:22:05.500 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.503 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.504 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.507 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "motd": { 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "motd", 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.508 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "The message of the day", 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect_resolve_ip_addr": { 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "redirect_resolve_ip_addr", 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-11-26T07:22:05.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.511 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8080", 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl_server_port": { 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl_server_port", 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8443", 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.514 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "redirect", 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.515 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "redirect" 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.516 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "url_prefix": { 2023-11-26T07:22:05.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url_prefix", 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_monitoring": { 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_monitoring", 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.519 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "monitor device health metrics", 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.520 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.521 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.522 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.523 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.524 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.525 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "mark_out_threshold": { 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mark_out_threshold", 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "2419200", 2023-11-26T07:22:05.526 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": { 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pool_name", 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "device_health_metrics", 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "retention_period": { 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "retention_period", 2023-11-26T07:22:05.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15552000", 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long to retain device health metrics", 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.529 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_frequency": { 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_frequency", 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.530 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "self_heal": { 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "self_heal", 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-26T07:22:05.531 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.532 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how frequently to wake up and check device health", 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "warn_threshold": { 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "warn_threshold", 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "7257600", 2023-11-26T07:22:05.533 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "diskprediction_local", 2023-11-26T07:22:05.534 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.535 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.536 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.538 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.540 INFO:teuthology.orchestra.run.smithi031.stdout: "predict_interval": { 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predict_interval", 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "86400", 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.541 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "predictor_model": { 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "predictor_model", 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "prophetstor", 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.542 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "600", 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.543 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "influx", 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": false, 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "influxdb python module not found", 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.544 INFO:teuthology.orchestra.run.smithi031.stdout: "batch_size": { 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "batch_size", 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5000", 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.545 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "database": { 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "database", 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "ceph", 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-26T07:22:05.546 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": { 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "hostname", 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.547 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server hostname", 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.548 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "30", 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "5", 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.550 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.551 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.553 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.554 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "password": { 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "password", 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.555 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "password of InfluxDB server user", 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "port": { 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "port", 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "8086", 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "InfluxDB server port", 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "ssl": { 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "ssl", 2023-11-26T07:22:05.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "false", 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.558 INFO:teuthology.orchestra.run.smithi031.stdout: "threads": { 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threads", 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "32", 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.559 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "username": { 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "username", 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "username of InfluxDB server user", 2023-11-26T07:22:05.560 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "verify_ssl": { 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "verify_ssl", 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "true", 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.561 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "insights", 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.562 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.563 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "iostat", 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "localpool", 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "failure_domain": { 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "failure_domain", 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "host", 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "failure domain for any created local pool", 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": { 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "min_size", 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default min_size for any created local pool", 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "num_rep": { 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "num_rep", 2023-11-26T07:22:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3", 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default replica count for any created local pool", 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": { 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_num", 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "128", 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "default pg_num for any created local pool", 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "prefix": { 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prefix", 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "name prefix for any created local pool", 2023-11-26T07:22:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "subtree": { 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subtree", 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "rack", 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirroring", 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "nfs", 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "fail_fs": { 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "fail_fs", 2023-11-26T07:22:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator": { 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "orchestrator", 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm", 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "rook", 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "test_orchestrator" 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Orchestrator backend", 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_perf_query", 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.611 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "osd_support", 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "pg_autoscaler", 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "threshold": { 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "threshold", 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "3.0", 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1.0", 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "scaling threshold", 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "progress", 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_pg_recovery_event": { 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_pg_recovery_event", 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max_completed_events": { 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_completed_events", 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "50", 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "number of past completed events to remember", 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "sleep_interval": { 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "sleep_interval", 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "5", 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "how long the module is going to sleep", 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "prometheus", 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "cache": { 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cache", 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "exclude_perf_counters": { 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "exclude_perf_counters", 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools": { 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools", 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "300", 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "scrape_interval": { 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "scrape_interval", 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15.0", 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "::", 2023-11-26T07:22:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "9283", 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "stale_cache_strategy": { 2023-11-26T07:22:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stale_cache_strategy", 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "log", 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_behaviour": { 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_behaviour", 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "default", 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "default", 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "error" 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "standby_error_status_code": { 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "standby_error_status_code", 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "500", 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "400", 2023-11-26T07:22:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "599", 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-11-26T07:22:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_snap_create": { 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_snap_create", 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10", 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mirror_snapshot_schedule": { 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mirror_snapshot_schedule", 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "trash_purge_schedule": { 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "trash_purge_schedule", 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "restful", 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_auth": { 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enable_auth", 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "key_file": { 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "key_file", 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "server_addr": { 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_addr", 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "server_port": { 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "server_port", 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "selftest", 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "roption1": { 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption1", 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "roption2": { 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "roption2", 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "xyz", 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption1": { 2023-11-26T07:22:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption1", 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption2": { 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption2", 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption3": { 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption3", 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "float", 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption4": { 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption4", 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption5": { 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption5", 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption6": { 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption6", 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "rwoption7": { 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rwoption7", 2023-11-26T07:22:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "1", 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "42", 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "testkey": { 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testkey", 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "testlkey": { 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testlkey", 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "testnewline": { 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "testnewline", 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snap_schedule", 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_m_granularity": { 2023-11-26T07:22:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "allow_m_granularity", 2023-11-26T07:22:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "allow minute scheduled snapshots", 2023-11-26T07:22:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "dump_on_update": { 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "dump_on_update", 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "dump database to debug log on update", 2023-11-26T07:22:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "stats", 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "status", 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telegraf", 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "address": { 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "address", 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "15", 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "telemetry", 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_basic": { 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_basic", 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_crash": { 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_crash", 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_device": { 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_device", 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "True", 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_ident": { 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_ident", 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "channel_perf": { 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "channel_perf", 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "contact": { 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "contact", 2023-11-26T07:22:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "description": { 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "description", 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.739 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "device_url": { 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "device_url", 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enabled": { 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "enabled", 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.741 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "24", 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "8", 2023-11-26T07:22:05.742 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "last_opt_revision": { 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "last_opt_revision", 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.743 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "1", 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard": { 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard", 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "leaderboard_description": { 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "leaderboard_description", 2023-11-26T07:22:05.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.746 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.747 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.748 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.749 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.750 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "organization": { 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "organization", 2023-11-26T07:22:05.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "proxy": { 2023-11-26T07:22:05.753 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "proxy", 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "url": { 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "url", 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "test_orchestrator", 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.757 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.758 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.760 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.761 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.762 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.763 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.764 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.766 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.768 INFO:teuthology.orchestra.run.smithi031.stdout: "max_concurrent_clones": { 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "max_concurrent_clones", 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "4", 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "periodic_async_work": { 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "periodic_async_work", 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Periodically check for async work", 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot_clone_delay": { 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "snapshot_clone_delay", 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "0", 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.771 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix", 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: "can_run": true, 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: "error_string": "", 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: "module_options": { 2023-11-26T07:22:05.772 INFO:teuthology.orchestra.run.smithi031.stdout: "discovery_interval": { 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "discovery_interval", 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "uint", 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "100", 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.773 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "identifier": { 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "identifier", 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.774 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "interval": { 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "interval", 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "secs", 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "60", 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.775 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "log_level": { 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_level", 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.776 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.777 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster": { 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster", 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.778 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_cluster_level": { 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_cluster_level", 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "info", 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [ 2023-11-26T07:22:05.779 INFO:teuthology.orchestra.run.smithi031.stdout: "", 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: "critical", 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: "debug", 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: "error", 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: "info", 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: "warning" 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.780 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": { 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "log_to_file", 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "bool", 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "False", 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.781 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_host": { 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_host", 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "", 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.782 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_port": { 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_port", 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "int", 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "10051", 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.783 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "zabbix_sender": { 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "zabbix_sender", 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "str", 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "level": "advanced", 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 0, 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-26T07:22:05.784 INFO:teuthology.orchestra.run.smithi031.stdout: "min": "", 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: "max": "", 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: "enum_allowed": [], 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: "desc": "", 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: "long_desc": "", 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tags": [], 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: "see_also": [] 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: "services": {}, 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: "always_on_modules": { 2023-11-26T07:22:05.785 INFO:teuthology.orchestra.run.smithi031.stdout: "octopus": [ 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.786 INFO:teuthology.orchestra.run.smithi031.stdout: "pacific": [ 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.787 INFO:teuthology.orchestra.run.smithi031.stdout: "quincy": [ 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:22:05.788 INFO:teuthology.orchestra.run.smithi031.stdout: "reef": [ 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: "balancer", 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: "devicehealth", 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: "orchestrator", 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscaler", 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: "progress", 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_support", 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: "status", 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: "telemetry", 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: "volumes" 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: "last_failure_osd_epoch": 0, 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: "active_clients": [ 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "devicehealth", 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 4191921753 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "libcephsqlite", 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 4223698495 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "rbd_support", 2023-11-26T07:22:05.791 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3784615915 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3104042147 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 62351307 2023-11-26T07:22:05.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3164817620 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2385439334 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.795 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1033952263 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 770009225 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.797 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 160948415 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "volumes", 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.97:0", 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1921595339 2023-11-26T07:22:05.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:05.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:05.799 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:22:05.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 -- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff4f0066860 msgr2=0x7ff4f0068d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:05.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff4f0066860 0x7ff4f0068d20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff5000ad000 tx=0x7ff500008040 comp rx=0 tx=0).stop 2023-11-26T07:22:05.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 -- 172.21.15.31:0/1149098337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510159880 msgr2=0x7ff51014f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510159880 0x7ff51014f560 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff508001f80 tx=0x7ff50800a430 comp rx=0 tx=0).stop 2023-11-26T07:22:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 -- 172.21.15.31:0/1149098337 shutdown_connections 2023-11-26T07:22:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7ff4f0066860 0x7ff4f0068d20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff510159880 0x7ff51014f560 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5100cb230 0x7ff510099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 --2- 172.21.15.31:0/1149098337 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff510071600 0x7ff510143970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 -- 172.21.15.31:0/1149098337 >> 172.21.15.31:0/1149098337 conn(0x7ff5100b0c40 msgr2=0x7ff5101504c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.857+0000 7ff51549a640 1 -- 172.21.15.31:0/1149098337 shutdown_connections 2023-11-26T07:22:05.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:04.861+0000 7ff51549a640 1 -- 172.21.15.31:0/1149098337 wait complete. 2023-11-26T07:22:05.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail x 2023-11-26T07:22:05.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.953+0000 7fa489832640 1 Processor -- start 2023-11-26T07:22:05.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.953+0000 7fa489832640 1 -- start start 2023-11-26T07:22:05.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.953+0000 7fa489832640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4840cdfb0 0x7fa4840abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.953+0000 7fa489832640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa48415a710 0x7fa4840ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.953+0000 7fa489832640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4840ce8d0 con 0x7fa4840cbd50 2023-11-26T07:22:05.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.953+0000 7fa489832640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4840cca80 con 0x7fa4840cdfb0 2023-11-26T07:22:05.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.953+0000 7fa489832640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840b2710 con 0x7fa48415a710 2023-11-26T07:22:05.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.957+0000 7fa482ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4840cdfb0 0x7fa4840abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.957+0000 7fa482ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa4840cdfb0 0x7fa4840abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58254/0 (socket says 172.21.15.31:58254) 2023-11-26T07:22:05.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.957+0000 7fa482ffd640 1 -- 172.21.15.31:0/1605298952 learned_addr learned my addr 172.21.15.31:0/1605298952 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.957+0000 7fa47bfff640 1 --2- 172.21.15.31:0/1605298952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa48415a710 0x7fa4840ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.957+0000 7fa482ffd640 1 -- 172.21.15.31:0/1605298952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa48415a710 msgr2=0x7fa4840ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.957+0000 7fa482ffd640 1 --2- 172.21.15.31:0/1605298952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa48415a710 0x7fa4840ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.957+0000 7fa482ffd640 1 -- 172.21.15.31:0/1605298952 >> v1:172.21.15.97:6789/0 conn(0x7fa4840cbd50 legacy=0x7fa4840cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.957+0000 7fa482ffd640 1 -- 172.21.15.31:0/1605298952 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4840b29f0 con 0x7fa4840cdfb0 2023-11-26T07:22:05.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.957+0000 7fa482ffd640 1 --2- 172.21.15.31:0/1605298952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 0x7fa4840abf90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa474009870 tx=0x7fa47401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2485d9748cf821f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:05.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.957+0000 7fa4827fc640 1 -- 172.21.15.31:0/1605298952 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740aa670 con 0x7fa4840cdfb0 2023-11-26T07:22:05.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa4827fc640 1 -- 172.21.15.31:0/1605298952 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4740aa7d0 con 0x7fa4840cdfb0 2023-11-26T07:22:05.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa4827fc640 1 -- 172.21.15.31:0/1605298952 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740aab60 con 0x7fa4840cdfb0 2023-11-26T07:22:05.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 -- 172.21.15.31:0/1605298952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 msgr2=0x7fa4840abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:05.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 --2- 172.21.15.31:0/1605298952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 0x7fa4840abf90 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa474009870 tx=0x7fa47401c510 comp rx=0 tx=0).stop 2023-11-26T07:22:05.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 -- 172.21.15.31:0/1605298952 shutdown_connections 2023-11-26T07:22:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 --2- 172.21.15.31:0/1605298952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa48415a710 0x7fa4840ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 --2- 172.21.15.31:0/1605298952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 0x7fa4840abf90 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 -- 172.21.15.31:0/1605298952 >> 172.21.15.31:0/1605298952 conn(0x7fa4840b0cb0 msgr2=0x7fa4840b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 -- 172.21.15.31:0/1605298952 shutdown_connections 2023-11-26T07:22:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 -- 172.21.15.31:0/1605298952 wait complete. 2023-11-26T07:22:05.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 Processor -- start 2023-11-26T07:22:05.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 -- start start 2023-11-26T07:22:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4840cbd50 0x7fa484070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 0x7fa484072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa48415a710 0x7fa4841f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840717f0 con 0x7fa4840cdfb0 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48415c680 con 0x7fa48415a710 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa489832640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840728a0 con 0x7fa4840cbd50 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa47bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 0x7fa484072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa47bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 0x7fa484072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58264/0 (socket says 172.21.15.31:58264) 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa47bfff640 1 -- 172.21.15.31:0/2864954794 learned_addr learned my addr 172.21.15.31:0/2864954794 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa4837fe640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa48415a710 0x7fa4841f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.961+0000 7fa482ffd640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4840cbd50 0x7fa484070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa47bfff640 1 -- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4840cbd50 msgr2=0x7fa484070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa47bfff640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4840cbd50 0x7fa484070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa47bfff640 1 -- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa48415a710 msgr2=0x7fa4841f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:05.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa47bfff640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa48415a710 0x7fa4841f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:05.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa47bfff640 1 -- 172.21.15.31:0/2864954794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740094d0 con 0x7fa4840cdfb0 2023-11-26T07:22:05.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa4837fe640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa48415a710 0x7fa4841f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:05.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa482ffd640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4840cbd50 0x7fa484070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:05.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa47bfff640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 0x7fa484072360 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa46c006e90 tx=0x7fa46c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa480ff9640 1 -- 172.21.15.31:0/2864954794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00b6c0 con 0x7fa4840cdfb0 2023-11-26T07:22:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa480ff9640 1 -- 172.21.15.31:0/2864954794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c00b840 con 0x7fa4840cdfb0 2023-11-26T07:22:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa480ff9640 1 -- 172.21.15.31:0/2864954794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00bbf0 con 0x7fa4840cdfb0 2023-11-26T07:22:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa489832640 1 -- 172.21.15.31:0/2864954794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa484070c70 con 0x7fa4840cdfb0 2023-11-26T07:22:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa489832640 1 -- 172.21.15.31:0/2864954794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa484071230 con 0x7fa4840cdfb0 2023-11-26T07:22:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa489832640 1 -- 172.21.15.31:0/2864954794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4840671f0 con 0x7fa4840cdfb0 2023-11-26T07:22:05.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa480ff9640 1 -- 172.21.15.31:0/2864954794 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 86209+0+0 (secure 0 0 0) 0x7fa46c00bd50 con 0x7fa4840cdfb0 2023-11-26T07:22:05.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.965+0000 7fa480ff9640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa464066860 0x7fa464068d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:05.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.969+0000 7fa480ff9640 1 -- 172.21.15.31:0/2864954794 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa46c084a40 con 0x7fa4840cdfb0 2023-11-26T07:22:05.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.969+0000 7fa482ffd640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa464066860 0x7fa464068d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:05.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.969+0000 7fa482ffd640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa464066860 0x7fa464068d20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa4740ae000 tx=0x7fa4740a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:05.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:05.973+0000 7fa480ff9640 1 -- 172.21.15.31:0/2864954794 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa46c051970 con 0x7fa4840cdfb0 2023-11-26T07:22:06.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.169+0000 7fa489832640 1 -- 172.21.15.31:0/2864954794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr fail", "who": "x"} v 0) v1 -- 0x7fa48406b460 con 0x7fa4840cdfb0 2023-11-26T07:22:06.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.173+0000 7fa480ff9640 1 -- 172.21.15.31:0/2864954794 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "x"}]=0 v12) v1 ==== 68+0+0 (secure 0 0 0) 0x7fa46c054f90 con 0x7fa4840cdfb0 2023-11-26T07:22:06.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa480ff9640 1 -- 172.21.15.31:0/2864954794 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mgrmap(e 12) v1 ==== 42894+0+0 (secure 0 0 0) 0x7fa46c051d40 con 0x7fa4840cdfb0 2023-11-26T07:22:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa480ff9640 1 -- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa464066860 msgr2=0x7fa464068d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa480ff9640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa464066860 0x7fa464068d20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa4740ae000 tx=0x7fa4740a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:06.180 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:06.176+0000 7f2476317640 -1 mgr handle_mgr_map I was active but no longer am 2023-11-26T07:22:06.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa489832640 1 -- 172.21.15.31:0/2864954794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 msgr2=0x7fa484072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa489832640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 0x7fa484072360 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa46c006e90 tx=0x7fa46c00dc60 comp rx=0 tx=0).stop 2023-11-26T07:22:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa489832640 1 -- 172.21.15.31:0/2864954794 shutdown_connections 2023-11-26T07:22:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa489832640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:6832/3008114855,v1:172.21.15.97:6833/3008114855] conn(0x7fa464066860 0x7fa464068d20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa489832640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa48415a710 0x7fa4841f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa489832640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4840cdfb0 0x7fa484072360 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.181 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa489832640 1 --2- 172.21.15.31:0/2864954794 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa4840cbd50 0x7fa484070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa489832640 1 -- 172.21.15.31:0/2864954794 >> 172.21.15.31:0/2864954794 conn(0x7fa4840b0cb0 msgr2=0x7fa484163eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa489832640 1 -- 172.21.15.31:0/2864954794 shutdown_connections 2023-11-26T07:22:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.177+0000 7fa489832640 1 -- 172.21.15.31:0/2864954794 wait complete. 2023-11-26T07:22:06.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:06.292 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:22:06.289+0000 7f1cb58bb640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:06.292 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:22:06.289+0000 7f1cb58bb640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:06.292 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:22:06.289+0000 7f1cb58bb640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:06.292 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:22:06.289+0000 7f1cb58bb640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:06.292 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:22:06.289+0000 7f1cb58bb640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:06.293 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:22:06.289+0000 7f1cb28b5640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:06.293 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:22:06.289+0000 7f1cb28b5640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:06.293 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:22:06.289+0000 7f1cb28b5640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:06.293 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:22:06.289+0000 7f1cb28b5640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:06.293 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:22:06.289+0000 7f1cb28b5640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:22:06.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08d6491640 1 Processor -- start 2023-11-26T07:22:06.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08d6491640 1 -- start start 2023-11-26T07:22:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08d6491640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08d00cfdb0 0x7f08d00d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08d6491640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d00d12e0 con 0x7f08d015d530 2023-11-26T07:22:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08d6491640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d00d15c0 con 0x7f08d01654c0 2023-11-26T07:22:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08d6491640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d00d0760 con 0x7f08d00cfdb0 2023-11-26T07:22:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08cffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f08d015d530 0x7f08d00cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45876/0 (socket says 172.21.15.31:45876) 2023-11-26T07:22:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08cffff640 1 -- 172.21.15.31:0/1916622326 learned_addr learned my addr 172.21.15.31:0/1916622326 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:06.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08cf7fe640 1 --2- 172.21.15.31:0/1916622326 >> v2:172.21.15.97:3300/0 conn(0x7f08d00cfdb0 0x7f08d00d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08ceffd640 1 -- 172.21.15.31:0/1916622326 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578049364 0 0) 0x7f08d00d12e0 con 0x7f08d015d530 2023-11-26T07:22:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08ceffd640 1 -- 172.21.15.31:0/1916622326 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08b8003530 con 0x7f08d015d530 2023-11-26T07:22:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08cf7fe640 1 -- 172.21.15.31:0/1916622326 >> v1:172.21.15.97:6789/0 conn(0x7f08d01654c0 legacy=0x7f08d01678b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08cf7fe640 1 -- 172.21.15.31:0/1916622326 >> v1:172.21.15.31:6789/0 conn(0x7f08d015d530 legacy=0x7f08d00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.389+0000 7f08cf7fe640 1 -- 172.21.15.31:0/1916622326 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d00d18a0 con 0x7f08d00cfdb0 2023-11-26T07:22:06.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08cf7fe640 1 --2- 172.21.15.31:0/1916622326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d00cfdb0 0x7f08d00d0220 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f08c40097f0 tx=0x7f08c401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e5b52372179a257 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:06.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08ceffd640 1 -- 172.21.15.31:0/1916622326 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c40a8970 con 0x7f08d00cfdb0 2023-11-26T07:22:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08ceffd640 1 -- 172.21.15.31:0/1916622326 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c40a8ad0 con 0x7f08d00cfdb0 2023-11-26T07:22:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08ceffd640 1 -- 172.21.15.31:0/1916622326 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c40a8e60 con 0x7f08d00cfdb0 2023-11-26T07:22:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08d6491640 1 -- 172.21.15.31:0/1916622326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d00cfdb0 msgr2=0x7f08d00d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08d6491640 1 --2- 172.21.15.31:0/1916622326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d00cfdb0 0x7f08d00d0220 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f08c40097f0 tx=0x7f08c401ec70 comp rx=0 tx=0).stop 2023-11-26T07:22:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08d6491640 1 -- 172.21.15.31:0/1916622326 shutdown_connections 2023-11-26T07:22:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08d6491640 1 --2- 172.21.15.31:0/1916622326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d00cfdb0 0x7f08d00d0220 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08d6491640 1 -- 172.21.15.31:0/1916622326 >> 172.21.15.31:0/1916622326 conn(0x7f08d009c6d0 msgr2=0x7f08d00d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08d6491640 1 -- 172.21.15.31:0/1916622326 shutdown_connections 2023-11-26T07:22:06.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.393+0000 7f08d6491640 1 -- 172.21.15.31:0/1916622326 wait complete. 2023-11-26T07:22:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d6491640 1 Processor -- start 2023-11-26T07:22:06.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d6491640 1 -- start start 2023-11-26T07:22:06.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d6491640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d00cfdb0 0x7f08d014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d6491640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d015d530 0x7f08d00b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d6491640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d01654c0 0x7f08d00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d6491640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d015e610 con 0x7f08d01654c0 2023-11-26T07:22:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d6491640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d015e8f0 con 0x7f08d00cfdb0 2023-11-26T07:22:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d6491640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d015ebd0 con 0x7f08d015d530 2023-11-26T07:22:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d4a2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d01654c0 0x7f08d00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08cf7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d015d530 0x7f08d00b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d4a2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d01654c0 0x7f08d00b8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58386/0 (socket says 172.21.15.31:58386) 2023-11-26T07:22:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d4a2e640 1 -- 172.21.15.31:0/1256169489 learned_addr learned my addr 172.21.15.31:0/1256169489 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:06.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08cffff640 1 --2- 172.21.15.31:0/1256169489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d00cfdb0 0x7f08d014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:06.402 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:06.396+0000 7f6aafb7e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-26T07:22:06.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d4a2e640 1 -- 172.21.15.31:0/1256169489 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d015d530 msgr2=0x7f08d00b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d4a2e640 1 --2- 172.21.15.31:0/1256169489 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d015d530 0x7f08d00b8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d4a2e640 1 -- 172.21.15.31:0/1256169489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d00cfdb0 msgr2=0x7f08d014f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d4a2e640 1 --2- 172.21.15.31:0/1256169489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d00cfdb0 0x7f08d014f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d4a2e640 1 -- 172.21.15.31:0/1256169489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c4009420 con 0x7f08d01654c0 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08cf7fe640 1 --2- 172.21.15.31:0/1256169489 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d015d530 0x7f08d00b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d4a2e640 1 --2- 172.21.15.31:0/1256169489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d01654c0 0x7f08d00b8bb0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f08c800c730 tx=0x7f08c800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08ccff9640 1 -- 172.21.15.31:0/1256169489 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8003b80 con 0x7f08d01654c0 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08ccff9640 1 -- 172.21.15.31:0/1256169489 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c8003ce0 con 0x7f08d01654c0 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d6491640 1 -- 172.21.15.31:0/1256169489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d015eeb0 con 0x7f08d01654c0 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08ccff9640 1 -- 172.21.15.31:0/1256169489 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8005310 con 0x7f08d01654c0 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.397+0000 7f08d6491640 1 -- 172.21.15.31:0/1256169489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d014fbb0 con 0x7f08d01654c0 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.401+0000 7f08ccff9640 1 -- 172.21.15.31:0/1256169489 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 42894+0+0 (secure 0 0 0) 0x7f08c8005680 con 0x7f08d01654c0 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.401+0000 7f08d6491640 1 -- 172.21.15.31:0/1256169489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d0067a40 con 0x7f08d01654c0 2023-11-26T07:22:06.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.401+0000 7f08ccff9640 1 -- 172.21.15.31:0/1256169489 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f08c8047bf0 con 0x7f08d01654c0 2023-11-26T07:22:06.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.405+0000 7f08ccff9640 1 -- 172.21.15.31:0/1256169489 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f08c8012420 con 0x7f08d01654c0 2023-11-26T07:22:06.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.593+0000 7f08d6491640 1 -- 172.21.15.31:0/1256169489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08d004e850 con 0x7f08d01654c0 2023-11-26T07:22:06.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.593+0000 7f08ccff9640 1 -- 172.21.15.31:0/1256169489 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f08c801a360 con 0x7f08d01654c0 2023-11-26T07:22:06.598 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:06.599 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2851182363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2851182363},{"type":"v1","addr":"172.21.15.31:6835","nonce":2851182363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2056572548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2056572548},{"type":"v1","addr":"172.21.15.97:6837","nonce":2056572548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1012539369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1012539369},{"type":"v1","addr":"172.21.15.97:6836","nonce":1012539369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:48.765793+0000","modified":"2023-11-26T07:21:51.814230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6597},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6597":{"gid":6597,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/242493352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":242493352},{"type":"v1","addr":"172.21.15.31:6834","nonce":242493352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:22:06.599 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 48 2023-11-26T07:22:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.597+0000 7f08d6491640 1 -- 172.21.15.31:0/1256169489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d01654c0 msgr2=0x7f08d00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.597+0000 7f08d6491640 1 --2- 172.21.15.31:0/1256169489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d01654c0 0x7f08d00b8bb0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f08c800c730 tx=0x7f08c800cbf0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.597+0000 7f08d6491640 1 -- 172.21.15.31:0/1256169489 shutdown_connections 2023-11-26T07:22:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.597+0000 7f08d6491640 1 --2- 172.21.15.31:0/1256169489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d01654c0 0x7f08d00b8bb0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.597+0000 7f08d6491640 1 --2- 172.21.15.31:0/1256169489 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08d015d530 0x7f08d00b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.597+0000 7f08d6491640 1 --2- 172.21.15.31:0/1256169489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08d00cfdb0 0x7f08d014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.597+0000 7f08d6491640 1 -- 172.21.15.31:0/1256169489 >> 172.21.15.31:0/1256169489 conn(0x7f08d009c6d0 msgr2=0x7f08d0161130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.597+0000 7f08d6491640 1 -- 172.21.15.31:0/1256169489 shutdown_connections 2023-11-26T07:22:06.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:06.597+0000 7f08d6491640 1 -- 172.21.15.31:0/1256169489 wait complete. 2023-11-26T07:22:06.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2023-11-26T07:22:06.783 INFO:teuthology.orchestra.run.smithi031.stdout:[{"id":6918,"entity":{"name":{"type":"client","num":6918},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":3339234760}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.47199534300000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978478028495499,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6918 172.21.15.31:0/3339234760","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi031","pid":"37986","root":"/"}},{"id":6604,"entity":{"name":{"type":"client","num":6604},"addr":{"type":"any","addr":"172.21.15.31:0","nonce":2976267327}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.47199534300000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978385709224171,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6604 172.21.15.31:0/2976267327","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"44c2237fe77713214148c1fd622c872f346f1b99","ceph_version":"ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)","entity_id":"y","hostname":"smithi031","pid":"37986","root":"/"}}] 2023-11-26T07:22:06.783 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37986", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 3339234760, "type": "any" }, "name": { "num": 6918, "type": "client" } }, "id": 6918, "inst": "client.6918 172.21.15.31:0/3339234760", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978478028495499 }, "state": "open", "uptime": 0.471995343 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "44c2237fe77713214148c1fd622c872f346f1b99", "ceph_version": "ceph version 18.0.0-7484-g44c2237f (44c2237fe77713214148c1fd622c872f346f1b99) reef (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi031", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "37986", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.31:0", "nonce": 2976267327, "type": "any" }, "name": { "num": 6604, "type": "client" } }, "id": 6604, "inst": "client.6604 172.21.15.31:0/2976267327", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978385709224171 }, "state": "open", "uptime": 0.471995343 } ] 2023-11-26T07:22:06.783 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-26T07:22:06.784 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:22:06.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:06.819 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:06.816+0000 7f6aafb7e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-26T07:22:06.864 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:06.936 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:22:06.936 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:22:06.936 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:06.945 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:22:06.946 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:06.946 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:06.955 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:22:06.955 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:22:06.955 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:06.968 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:06.964+0000 7f6aafb7e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-26T07:22:06.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:22:06.994 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:06.994 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:07.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:22:07.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:07.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.177+0000 7febb98bb640 1 Processor -- start 2023-11-26T07:22:07.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb98bb640 1 -- start start 2023-11-26T07:22:07.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb98bb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7febb40d3760 0x7febb40d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb98bb640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb40d15f0 con 0x7febb415d880 2023-11-26T07:22:07.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb98bb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb40ce000 con 0x7febb40cfa20 2023-11-26T07:22:07.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb98bb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb40d4110 con 0x7febb40d3760 2023-11-26T07:22:07.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb88b9640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7febb415d880 0x7febb40b6730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45886/0 (socket says 172.21.15.31:45886) 2023-11-26T07:22:07.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb88b9640 1 -- 172.21.15.31:0/2894693493 learned_addr learned my addr 172.21.15.31:0/2894693493 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:07.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb3fff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7febb40d3760 0x7febb40d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb3fff640 1 -- 172.21.15.31:0/2894693493 >> v1:172.21.15.97:6789/0 conn(0x7febb40cfa20 legacy=0x7febb40cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb3fff640 1 -- 172.21.15.31:0/2894693493 >> v1:172.21.15.31:6789/0 conn(0x7febb415d880 legacy=0x7febb40b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.181+0000 7febb3fff640 1 -- 172.21.15.31:0/2894693493 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb40d0630 con 0x7febb40d3760 2023-11-26T07:22:07.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.185+0000 7febb3fff640 1 --2- 172.21.15.31:0/2894693493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb40d3760 0x7febb40d3bd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7feba00097c0 tx=0x7feba001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf0f758423be286 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.185+0000 7febb37fe640 1 -- 172.21.15.31:0/2894693493 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba00ab680 con 0x7febb40d3760 2023-11-26T07:22:07.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.185+0000 7febb37fe640 1 -- 172.21.15.31:0/2894693493 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba00ab7e0 con 0x7febb40d3760 2023-11-26T07:22:07.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.185+0000 7febb37fe640 1 -- 172.21.15.31:0/2894693493 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba00abb70 con 0x7febb40d3760 2023-11-26T07:22:07.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.189+0000 7febb98bb640 1 -- 172.21.15.31:0/2894693493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb40d3760 msgr2=0x7febb40d3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.189+0000 7febb98bb640 1 --2- 172.21.15.31:0/2894693493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb40d3760 0x7febb40d3bd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7feba00097c0 tx=0x7feba001c480 comp rx=0 tx=0).stop 2023-11-26T07:22:07.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.189+0000 7febb98bb640 1 -- 172.21.15.31:0/2894693493 shutdown_connections 2023-11-26T07:22:07.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.189+0000 7febb98bb640 1 --2- 172.21.15.31:0/2894693493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb40d3760 0x7febb40d3bd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.189+0000 7febb98bb640 1 -- 172.21.15.31:0/2894693493 >> 172.21.15.31:0/2894693493 conn(0x7febb406b9f0 msgr2=0x7febb406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:07.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.189+0000 7febb98bb640 1 -- 172.21.15.31:0/2894693493 shutdown_connections 2023-11-26T07:22:07.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.189+0000 7febb98bb640 1 -- 172.21.15.31:0/2894693493 wait complete. 2023-11-26T07:22:07.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.189+0000 7febb98bb640 1 Processor -- start 2023-11-26T07:22:07.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.189+0000 7febb98bb640 1 -- start start 2023-11-26T07:22:07.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb98bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb40cfa20 0x7febb4154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb98bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb40d3760 0x7febb4156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb98bb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7febb415d880 0x7febb40b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb98bb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb4155870 con 0x7febb40cfa20 2023-11-26T07:22:07.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb98bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb4154c60 con 0x7febb40d3760 2023-11-26T07:22:07.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb98bb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb4154f40 con 0x7febb415d880 2023-11-26T07:22:07.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb88b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb40cfa20 0x7febb4154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb88b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb40cfa20 0x7febb4154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58396/0 (socket says 172.21.15.31:58396) 2023-11-26T07:22:07.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb88b9640 1 -- 172.21.15.31:0/884140221 learned_addr learned my addr 172.21.15.31:0/884140221 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:07.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb3fff640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb40d3760 0x7febb4156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb90ba640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7febb415d880 0x7febb40b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb88b9640 1 -- 172.21.15.31:0/884140221 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7febb415d880 msgr2=0x7febb40b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb88b9640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7febb415d880 0x7febb40b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb88b9640 1 -- 172.21.15.31:0/884140221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb40d3760 msgr2=0x7febb4156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb88b9640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb40d3760 0x7febb4156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb88b9640 1 -- 172.21.15.31:0/884140221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0009420 con 0x7febb40cfa20 2023-11-26T07:22:07.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb88b9640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb40cfa20 0x7febb4154360 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7feba800c400 tx=0x7feba800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb17fa640 1 -- 172.21.15.31:0/884140221 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8005cf0 con 0x7febb40cfa20 2023-11-26T07:22:07.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb17fa640 1 -- 172.21.15.31:0/884140221 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba8005e50 con 0x7febb40cfa20 2023-11-26T07:22:07.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb98bb640 1 -- 172.21.15.31:0/884140221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb4155220 con 0x7febb40cfa20 2023-11-26T07:22:07.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb17fa640 1 -- 172.21.15.31:0/884140221 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba80072d0 con 0x7febb40cfa20 2023-11-26T07:22:07.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.193+0000 7febb98bb640 1 -- 172.21.15.31:0/884140221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb41548a0 con 0x7febb40cfa20 2023-11-26T07:22:07.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.197+0000 7febb98bb640 1 -- 172.21.15.31:0/884140221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb4074a00 con 0x7febb40cfa20 2023-11-26T07:22:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.197+0000 7febb17fa640 1 -- 172.21.15.31:0/884140221 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43229+0+0 (secure 0 0 0) 0x7feba8007430 con 0x7febb40cfa20 2023-11-26T07:22:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.197+0000 7febb17fa640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7feb980354e0 0x7feb980379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.197+0000 7febb17fa640 1 -- 172.21.15.31:0/884140221 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7feba8047700 con 0x7febb40cfa20 2023-11-26T07:22:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.197+0000 7febb3fff640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7feb980354e0 0x7feb980379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.197+0000 7febb3fff640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7feb980354e0 0x7feb980379a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feba00af000 tx=0x7feba00a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.201+0000 7febb17fa640 1 -- 172.21.15.31:0/884140221 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feba8012400 con 0x7febb40cfa20 2023-11-26T07:22:07.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.393+0000 7febb98bb640 1 -- 172.21.15.31:0/884140221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febb40d11a0 con 0x7febb40cfa20 2023-11-26T07:22:07.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.393+0000 7febb17fa640 1 -- 172.21.15.31:0/884140221 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4912 (secure 0 0 0) 0x7feba8017020 con 0x7febb40cfa20 2023-11-26T07:22:07.396 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:07.396 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2851182363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2851182363},{"type":"v1","addr":"172.21.15.31:6835","nonce":2851182363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2056572548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2056572548},{"type":"v1","addr":"172.21.15.97:6837","nonce":2056572548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1012539369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1012539369},{"type":"v1","addr":"172.21.15.97:6836","nonce":1012539369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:48.765793+0000","modified":"2023-11-26T07:21:51.814230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6597},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6597":{"gid":6597,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/242493352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":242493352},{"type":"v1","addr":"172.21.15.31:6834","nonce":242493352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:22:07.397 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 48 2023-11-26T07:22:07.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.393+0000 7febb98bb640 1 -- 172.21.15.31:0/884140221 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7feb980354e0 msgr2=0x7feb980379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.393+0000 7febb98bb640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7feb980354e0 0x7feb980379a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feba00af000 tx=0x7feba00a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.393+0000 7febb98bb640 1 -- 172.21.15.31:0/884140221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb40cfa20 msgr2=0x7febb4154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.393+0000 7febb98bb640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb40cfa20 0x7febb4154360 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7feba800c400 tx=0x7feba800c8c0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.397+0000 7febb98bb640 1 -- 172.21.15.31:0/884140221 shutdown_connections 2023-11-26T07:22:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.397+0000 7febb98bb640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7feb980354e0 0x7feb980379a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.397+0000 7febb98bb640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7febb415d880 0x7febb40b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.397+0000 7febb98bb640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febb40d3760 0x7febb4156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.397+0000 7febb98bb640 1 --2- 172.21.15.31:0/884140221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febb40cfa20 0x7febb4154360 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.397+0000 7febb98bb640 1 -- 172.21.15.31:0/884140221 >> 172.21.15.31:0/884140221 conn(0x7febb406b9f0 msgr2=0x7febb406bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.397+0000 7febb98bb640 1 -- 172.21.15.31:0/884140221 shutdown_connections 2023-11-26T07:22:07.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.397+0000 7febb98bb640 1 -- 172.21.15.31:0/884140221 wait complete. 2023-11-26T07:22:07.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:07.461 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:07.456+0000 7f6aafb7e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-26T07:22:07.522 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:07.520+0000 7f6aafb7e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-26T07:22:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.557+0000 7f8f1b59e640 1 Processor -- start 2023-11-26T07:22:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.557+0000 7f8f1b59e640 1 -- start start 2023-11-26T07:22:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f1b59e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8f1c072100 0x7f8f1c15e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f1b59e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1c15ea40 con 0x7f8f1c13fff0 2023-11-26T07:22:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f1b59e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1c15a7c0 con 0x7f8f1c0724e0 2023-11-26T07:22:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f1b59e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1c15b440 con 0x7f8f1c072100 2023-11-26T07:22:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f1ad9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8f1c13fff0 0x7f8f1c0ce380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45898/0 (socket says 172.21.15.31:45898) 2023-11-26T07:22:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f1ad9d640 1 -- 172.21.15.31:0/579080509 learned_addr learned my addr 172.21.15.31:0/579080509 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f1a59c640 1 --2- 172.21.15.31:0/579080509 >> v2:172.21.15.97:3300/0 conn(0x7f8f1c072100 0x7f8f1c15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f19d9b640 1 -- 172.21.15.31:0/579080509 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898047728 0 0) 0x7f8f1c15ea40 con 0x7f8f1c13fff0 2023-11-26T07:22:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f19d9b640 1 -- 172.21.15.31:0/579080509 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f08005740 con 0x7f8f1c13fff0 2023-11-26T07:22:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f19d9b640 1 -- 172.21.15.31:0/579080509 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529566674 0 0) 0x7f8f1c15a7c0 con 0x7f8f1c0724e0 2023-11-26T07:22:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f19d9b640 1 -- 172.21.15.31:0/579080509 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f1c15ea40 con 0x7f8f1c0724e0 2023-11-26T07:22:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f1a59c640 1 -- 172.21.15.31:0/579080509 >> v1:172.21.15.97:6789/0 conn(0x7f8f1c0724e0 legacy=0x7f8f1c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f1a59c640 1 -- 172.21.15.31:0/579080509 >> v1:172.21.15.31:6789/0 conn(0x7f8f1c13fff0 legacy=0x7f8f1c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.561+0000 7f8f1a59c640 1 -- 172.21.15.31:0/579080509 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c15aaa0 con 0x7f8f1c072100 2023-11-26T07:22:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1a59c640 1 --2- 172.21.15.31:0/579080509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1c072100 0x7f8f1c15e500 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0097c0 tx=0x7f8f0c01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7803faa4ba6aa60c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f19d9b640 1 -- 172.21.15.31:0/579080509 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0ab600 con 0x7f8f1c072100 2023-11-26T07:22:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f19d9b640 1 -- 172.21.15.31:0/579080509 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0a7050 con 0x7f8f1c072100 2023-11-26T07:22:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f19d9b640 1 -- 172.21.15.31:0/579080509 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0ab980 con 0x7f8f1c072100 2023-11-26T07:22:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 -- 172.21.15.31:0/579080509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1c072100 msgr2=0x7f8f1c15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.567 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 --2- 172.21.15.31:0/579080509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1c072100 0x7f8f1c15e500 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0097c0 tx=0x7f8f0c01c3f0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 -- 172.21.15.31:0/579080509 shutdown_connections 2023-11-26T07:22:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 --2- 172.21.15.31:0/579080509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1c072100 0x7f8f1c15e500 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 -- 172.21.15.31:0/579080509 >> 172.21.15.31:0/579080509 conn(0x7f8f1c0b0e00 msgr2=0x7f8f1c0b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 -- 172.21.15.31:0/579080509 shutdown_connections 2023-11-26T07:22:07.568 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 -- 172.21.15.31:0/579080509 wait complete. 2023-11-26T07:22:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 Processor -- start 2023-11-26T07:22:07.569 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 -- start start 2023-11-26T07:22:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f1c072100 0x7f8f1c0b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.565+0000 7f8f1b59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1c0724e0 0x7f8f1c13b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c13fff0 0x7f8f1c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1b59e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c13ba50 con 0x7f8f1c13fff0 2023-11-26T07:22:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1b59e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c0b3020 con 0x7f8f1c0724e0 2023-11-26T07:22:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1b59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1c141540 con 0x7f8f1c072100 2023-11-26T07:22:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c13fff0 0x7f8f1c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f13fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1c0724e0 0x7f8f1c13b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1ad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c13fff0 0x7f8f1c0b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58408/0 (socket says 172.21.15.31:58408) 2023-11-26T07:22:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1ad9d640 1 -- 172.21.15.31:0/3869297876 learned_addr learned my addr 172.21.15.31:0/3869297876 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:07.571 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1a59c640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f1c072100 0x7f8f1c0b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.572 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1ad9d640 1 -- 172.21.15.31:0/3869297876 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f1c072100 msgr2=0x7f8f1c0b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1ad9d640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f1c072100 0x7f8f1c0b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1ad9d640 1 -- 172.21.15.31:0/3869297876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1c0724e0 msgr2=0x7f8f1c13b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1ad9d640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1c0724e0 0x7f8f1c13b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1ad9d640 1 -- 172.21.15.31:0/3869297876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c009420 con 0x7f8f1c13fff0 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1ad9d640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c13fff0 0x7f8f1c0b2ae0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400cb40 tx=0x7f8f1400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f12ffd640 1 -- 172.21.15.31:0/3869297876 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400aca0 con 0x7f8f1c13fff0 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f12ffd640 1 -- 172.21.15.31:0/3869297876 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1400ae00 con 0x7f8f1c13fff0 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f12ffd640 1 -- 172.21.15.31:0/3869297876 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14012600 con 0x7f8f1c13fff0 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1b59e640 1 -- 172.21.15.31:0/3869297876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1c140580 con 0x7f8f1c13fff0 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1b59e640 1 -- 172.21.15.31:0/3869297876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1c0b3510 con 0x7f8f1c13fff0 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1b59e640 1 -- 172.21.15.31:0/3869297876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f1c0b3730 con 0x7f8f1c13fff0 2023-11-26T07:22:07.573 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f12ffd640 1 -- 172.21.15.31:0/3869297876 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43229+0+0 (secure 0 0 0) 0x7f8f14007070 con 0x7f8f1c13fff0 2023-11-26T07:22:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f12ffd640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8efc0354e0 0x7f8efc0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f12ffd640 1 -- 172.21.15.31:0/3869297876 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f8f14048140 con 0x7f8f1c13fff0 2023-11-26T07:22:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1a59c640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8efc0354e0 0x7f8efc0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.569+0000 7f8f1a59c640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8efc0354e0 0x7f8efc0379a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0ad000 tx=0x7f8f0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.573+0000 7f8f12ffd640 1 -- 172.21.15.31:0/3869297876 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8f140192c0 con 0x7f8f1c13fff0 2023-11-26T07:22:07.579 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:07.576+0000 7f6aafb7e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-26T07:22:07.665 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:07.660+0000 7f6aafb7e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-26T07:22:07.722 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:07.720+0000 7f6aafb7e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-26T07:22:07.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 -- 172.21.15.31:0/3869297876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f1c070940 con 0x7f8f1c13fff0 2023-11-26T07:22:07.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f12ffd640 1 -- 172.21.15.31:0/3869297876 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+17291 (secure 0 0 0) 0x7f8f14017020 con 0x7f8f1c13fff0 2023-11-26T07:22:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:07.736 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":69,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:06.173856+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:46.570507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:47.773615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3784615915":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/62351307":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/160948415":"2023-11-27T07:22:06.173818+0000","172.21.15.97:6833/3008114855":"2023-11-27T07:22:06.173818+0000","172.21.15.97:6832/3008114855":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/1921595339":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/3104042147":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/770009225":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/2385439334":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/4191921753":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/3164817620":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/4223698495":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/1033952263":"2023-11-27T07:22:06.173818+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:07.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 -- 172.21.15.31:0/3869297876 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8efc0354e0 msgr2=0x7f8efc0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8efc0354e0 0x7f8efc0379a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0ad000 tx=0x7f8f0c008040 comp rx=0 tx=0).stop 2023-11-26T07:22:07.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 -- 172.21.15.31:0/3869297876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c13fff0 msgr2=0x7f8f1c0b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c13fff0 0x7f8f1c0b2ae0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400cb40 tx=0x7f8f1400a060 comp rx=0 tx=0).stop 2023-11-26T07:22:07.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 -- 172.21.15.31:0/3869297876 shutdown_connections 2023-11-26T07:22:07.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8efc0354e0 0x7f8efc0379a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f1c13fff0 0x7f8f1c0b2ae0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f1c0724e0 0x7f8f1c13b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 --2- 172.21.15.31:0/3869297876 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f1c072100 0x7f8f1c0b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 -- 172.21.15.31:0/3869297876 >> 172.21.15.31:0/3869297876 conn(0x7f8f1c0b0e00 msgr2=0x7f8f1c13e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:07.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 -- 172.21.15.31:0/3869297876 shutdown_connections 2023-11-26T07:22:07.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.733+0000 7f8f1b59e640 1 -- 172.21.15.31:0/3869297876 wait complete. 2023-11-26T07:22:07.757 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:21:46.570507+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:21:47.773615+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:07.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:07.806 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:07.804+0000 7f6aafb7e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-26T07:22:07.863 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:07.860+0000 7f6aafb7e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-26T07:22:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe8816640 1 Processor -- start 2023-11-26T07:22:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe8816640 1 -- start start 2023-11-26T07:22:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe8816640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8fe40b2870 0x7f8fe40b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe8816640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe4161290 con 0x7f8fe406afc0 2023-11-26T07:22:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe8816640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe4161550 con 0x7f8fe40b07a0 2023-11-26T07:22:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe8816640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fe40b0b80 con 0x7f8fe40b2870 2023-11-26T07:22:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe1d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8fe40b2870 0x7f8fe40b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe2d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8fe406afc0 0x7f8fe40cc9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45914/0 (socket says 172.21.15.31:45914) 2023-11-26T07:22:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe2d9d640 1 -- 172.21.15.31:0/3019535654 learned_addr learned my addr 172.21.15.31:0/3019535654 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe1d9b640 1 -- 172.21.15.31:0/3019535654 >> v1:172.21.15.97:6789/0 conn(0x7f8fe40b07a0 legacy=0x7f8fe41619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe1d9b640 1 -- 172.21.15.31:0/3019535654 >> v1:172.21.15.31:6789/0 conn(0x7f8fe406afc0 legacy=0x7f8fe40cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.901+0000 7f8fe1d9b640 1 -- 172.21.15.31:0/3019535654 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe40c65a0 con 0x7f8fe40b2870 2023-11-26T07:22:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe1d9b640 1 --2- 172.21.15.31:0/3019535654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b2870 0x7f8fe40b0ee0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80097c0 tx=0x7f8fd801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15d749497ae738c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe159a640 1 -- 172.21.15.31:0/3019535654 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80ab600 con 0x7f8fe40b2870 2023-11-26T07:22:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe159a640 1 -- 172.21.15.31:0/3019535654 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd80a7050 con 0x7f8fe40b2870 2023-11-26T07:22:07.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe159a640 1 -- 172.21.15.31:0/3019535654 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80ab980 con 0x7f8fe40b2870 2023-11-26T07:22:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe8816640 1 -- 172.21.15.31:0/3019535654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b2870 msgr2=0x7f8fe40b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe8816640 1 --2- 172.21.15.31:0/3019535654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b2870 0x7f8fe40b0ee0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80097c0 tx=0x7f8fd801c480 comp rx=0 tx=0).stop 2023-11-26T07:22:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe8816640 1 -- 172.21.15.31:0/3019535654 shutdown_connections 2023-11-26T07:22:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe8816640 1 --2- 172.21.15.31:0/3019535654 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b2870 0x7f8fe40b0ee0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe8816640 1 -- 172.21.15.31:0/3019535654 >> 172.21.15.31:0/3019535654 conn(0x7f8fe40b1d50 msgr2=0x7f8fe40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe8816640 1 -- 172.21.15.31:0/3019535654 shutdown_connections 2023-11-26T07:22:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.905+0000 7f8fe8816640 1 -- 172.21.15.31:0/3019535654 wait complete. 2023-11-26T07:22:07.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe8816640 1 Processor -- start 2023-11-26T07:22:07.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe8816640 1 -- start start 2023-11-26T07:22:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe8816640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe406afc0 0x7f8fe414ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe8816640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b07a0 0x7f8fe415bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe8816640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe40b2870 0x7f8fe4160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe8816640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe414d360 con 0x7f8fe40b07a0 2023-11-26T07:22:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe8816640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe415f1b0 con 0x7f8fe406afc0 2023-11-26T07:22:07.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe8816640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe415f490 con 0x7f8fe40b2870 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b07a0 0x7f8fe415bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b07a0 0x7f8fe415bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58418/0 (socket says 172.21.15.31:58418) 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe1d9b640 1 -- 172.21.15.31:0/3120818596 learned_addr learned my addr 172.21.15.31:0/3120818596 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe259c640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe406afc0 0x7f8fe414ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe2d9d640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe40b2870 0x7f8fe4160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe1d9b640 1 -- 172.21.15.31:0/3120818596 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe40b2870 msgr2=0x7f8fe4160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe1d9b640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe40b2870 0x7f8fe4160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe1d9b640 1 -- 172.21.15.31:0/3120818596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe406afc0 msgr2=0x7f8fe414ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe1d9b640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe406afc0 0x7f8fe414ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe1d9b640 1 -- 172.21.15.31:0/3120818596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd8009420 con 0x7f8fe40b07a0 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe1d9b640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b07a0 0x7f8fe415bfa0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8019090 tx=0x7f8fd8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.915 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fd2ffd640 1 -- 172.21.15.31:0/3120818596 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80b4600 con 0x7f8fe40b07a0 2023-11-26T07:22:07.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fd2ffd640 1 -- 172.21.15.31:0/3120818596 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd80b4780 con 0x7f8fe40b07a0 2023-11-26T07:22:07.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fd2ffd640 1 -- 172.21.15.31:0/3120818596 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80b4b30 con 0x7f8fe40b07a0 2023-11-26T07:22:07.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.909+0000 7f8fe8816640 1 -- 172.21.15.31:0/3120818596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe415f770 con 0x7f8fe40b07a0 2023-11-26T07:22:07.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.913+0000 7f8fe8816640 1 -- 172.21.15.31:0/3120818596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe415fd30 con 0x7f8fe40b07a0 2023-11-26T07:22:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.913+0000 7f8fd2ffd640 1 -- 172.21.15.31:0/3120818596 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43229+0+0 (secure 0 0 0) 0x7f8fd80b2040 con 0x7f8fe40b07a0 2023-11-26T07:22:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.913+0000 7f8fe8816640 1 -- 172.21.15.31:0/3120818596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe404e850 con 0x7f8fe40b07a0 2023-11-26T07:22:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.913+0000 7f8fd2ffd640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8fc40354e0 0x7f8fc40379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.913+0000 7f8fd2ffd640 1 -- 172.21.15.31:0/3120818596 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f8fd80e23e0 con 0x7f8fe40b07a0 2023-11-26T07:22:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.913+0000 7f8fe259c640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8fc40354e0 0x7f8fc40379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:07.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.913+0000 7f8fe259c640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8fc40354e0 0x7f8fc40379a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8fe414dbd0 tx=0x7f8fd4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:07.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:07.917+0000 7f8fd2ffd640 1 -- 172.21.15.31:0/3120818596 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8fd80bf310 con 0x7f8fe40b07a0 2023-11-26T07:22:08.008 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:08.004+0000 7f6aafb7e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-26T07:22:08.066 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:08.064+0000 7f6aafb7e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-26T07:22:08.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 -- 172.21.15.31:0/3120818596 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8fe415ff20 con 0x7f8fe40b07a0 2023-11-26T07:22:08.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fd2ffd640 1 -- 172.21.15.31:0/3120818596 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8fd80b0020 con 0x7f8fe40b07a0 2023-11-26T07:22:08.079 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:22:08.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 -- 172.21.15.31:0/3120818596 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8fc40354e0 msgr2=0x7f8fc40379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8fc40354e0 0x7f8fc40379a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8fe414dbd0 tx=0x7f8fd4009040 comp rx=0 tx=0).stop 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 -- 172.21.15.31:0/3120818596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b07a0 msgr2=0x7f8fe415bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b07a0 0x7f8fe415bfa0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8019090 tx=0x7f8fd8019220 comp rx=0 tx=0).stop 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 -- 172.21.15.31:0/3120818596 shutdown_connections 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8fc40354e0 0x7f8fc40379a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8fe40b2870 0x7f8fe4160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fe40b07a0 0x7f8fe415bfa0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 --2- 172.21.15.31:0/3120818596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fe406afc0 0x7f8fe414ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 -- 172.21.15.31:0/3120818596 >> 172.21.15.31:0/3120818596 conn(0x7f8fe40b1d50 msgr2=0x7f8fe406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 -- 172.21.15.31:0/3120818596 shutdown_connections 2023-11-26T07:22:08.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.077+0000 7f8fe8816640 1 -- 172.21.15.31:0/3120818596 wait complete. 2023-11-26T07:22:08.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:08.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.241+0000 7f8ebebe4640 1 Processor -- start 2023-11-26T07:22:08.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.241+0000 7f8ebebe4640 1 -- start start 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebebe4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8eb80b4c40 0x7f8eb80b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebebe4640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8eb80d10e0 0x7f8eb80d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebebe4640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb80b55d0 con 0x7f8eb815d2a0 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebebe4640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb80ce8c0 con 0x7f8eb80d10e0 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebebe4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb80b6250 con 0x7f8eb80b4c40 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebe3e3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8eb80d10e0 0x7f8eb80d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebe3e3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8eb80d10e0 0x7f8eb80d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58428/0 (socket says 172.21.15.31:58428) 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebe3e3640 1 -- 172.21.15.31:0/2193905649 learned_addr learned my addr 172.21.15.31:0/2193905649 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebd3e1640 1 --2- 172.21.15.31:0/2193905649 >> v2:172.21.15.97:3300/0 conn(0x7f8eb80b4c40 0x7f8eb80b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebe3e3640 1 -- 172.21.15.31:0/2193905649 >> v2:172.21.15.97:3300/0 conn(0x7f8eb80b4c40 msgr2=0x7f8eb80b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebe3e3640 1 --2- 172.21.15.31:0/2193905649 >> v2:172.21.15.97:3300/0 conn(0x7f8eb80b4c40 0x7f8eb80b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebe3e3640 1 -- 172.21.15.31:0/2193905649 >> v1:172.21.15.97:6789/0 conn(0x7f8eb815d2a0 legacy=0x7f8eb80d26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:08.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.245+0000 7f8ebe3e3640 1 -- 172.21.15.31:0/2193905649 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb8161cf0 con 0x7f8eb80d10e0 2023-11-26T07:22:08.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebe3e3640 1 --2- 172.21.15.31:0/2193905649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d10e0 0x7f8eb80d1520 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80cd8a0 tx=0x7f8eb001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3044910b0a0b42fd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebcbe0640 1 -- 172.21.15.31:0/2193905649 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00a5820 con 0x7f8eb80d10e0 2023-11-26T07:22:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebcbe0640 1 -- 172.21.15.31:0/2193905649 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb00a5980 con 0x7f8eb80d10e0 2023-11-26T07:22:08.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebcbe0640 1 -- 172.21.15.31:0/2193905649 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00a5d10 con 0x7f8eb80d10e0 2023-11-26T07:22:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebebe4640 1 -- 172.21.15.31:0/2193905649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d10e0 msgr2=0x7f8eb80d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebebe4640 1 --2- 172.21.15.31:0/2193905649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d10e0 0x7f8eb80d1520 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80cd8a0 tx=0x7f8eb001c960 comp rx=0 tx=0).stop 2023-11-26T07:22:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebebe4640 1 -- 172.21.15.31:0/2193905649 shutdown_connections 2023-11-26T07:22:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebebe4640 1 --2- 172.21.15.31:0/2193905649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb80d10e0 0x7f8eb80d1520 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebebe4640 1 --2- 172.21.15.31:0/2193905649 >> v2:172.21.15.97:3300/0 conn(0x7f8eb80b4c40 0x7f8eb80b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebebe4640 1 -- 172.21.15.31:0/2193905649 >> 172.21.15.31:0/2193905649 conn(0x7f8eb80668a0 msgr2=0x7f8eb80cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebebe4640 1 -- 172.21.15.31:0/2193905649 shutdown_connections 2023-11-26T07:22:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebebe4640 1 -- 172.21.15.31:0/2193905649 wait complete. 2023-11-26T07:22:08.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebebe4640 1 Processor -- start 2023-11-26T07:22:08.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.249+0000 7f8ebebe4640 1 -- start start 2023-11-26T07:22:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebebe4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb80b4c40 0x7f8eb8155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebebe4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80d10e0 0x7f8eb80d3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebebe4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb815d2a0 0x7f8eb81f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebebe4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb8156ef0 con 0x7f8eb815d2a0 2023-11-26T07:22:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebebe4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb815a5c0 con 0x7f8eb80b4c40 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebebe4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb815a000 con 0x7f8eb80d10e0 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebe3e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb815d2a0 0x7f8eb81f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebe3e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb815d2a0 0x7f8eb81f91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58440/0 (socket says 172.21.15.31:58440) 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebe3e3640 1 -- 172.21.15.31:0/3149263152 learned_addr learned my addr 172.21.15.31:0/3149263152 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebd3e1640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80d10e0 0x7f8eb80d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebdbe2640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb80b4c40 0x7f8eb8155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebe3e3640 1 -- 172.21.15.31:0/3149263152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80d10e0 msgr2=0x7f8eb80d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebe3e3640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80d10e0 0x7f8eb80d3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebe3e3640 1 -- 172.21.15.31:0/3149263152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb80b4c40 msgr2=0x7f8eb8155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebe3e3640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb80b4c40 0x7f8eb8155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebe3e3640 1 -- 172.21.15.31:0/3149263152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb0009b30 con 0x7f8eb815d2a0 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebe3e3640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb815d2a0 0x7f8eb81f91f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8eb001a590 tx=0x7f8eb0019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8eae7fc640 1 -- 172.21.15.31:0/3149263152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00b8670 con 0x7f8eb815d2a0 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8eae7fc640 1 -- 172.21.15.31:0/3149263152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb00b87d0 con 0x7f8eb815d2a0 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8eae7fc640 1 -- 172.21.15.31:0/3149263152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00b8b80 con 0x7f8eb815d2a0 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebebe4640 1 -- 172.21.15.31:0/3149263152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb815a2e0 con 0x7f8eb815d2a0 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebebe4640 1 -- 172.21.15.31:0/3149263152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb80cd0d0 con 0x7f8eb815d2a0 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8eae7fc640 1 -- 172.21.15.31:0/3149263152 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43229+0+0 (secure 0 0 0) 0x7f8eb00b8ce0 con 0x7f8eb815d2a0 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebebe4640 1 -- 172.21.15.31:0/3149263152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb8074990 con 0x7f8eb815d2a0 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8eae7fc640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8e9c0354e0 0x7f8e9c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8eae7fc640 1 -- 172.21.15.31:0/3149263152 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f8eb00e3770 con 0x7f8eb815d2a0 2023-11-26T07:22:08.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.253+0000 7f8ebdbe2640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8e9c0354e0 0x7f8e9c0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.257+0000 7f8ebdbe2640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8e9c0354e0 0x7f8e9c0379a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400b810 tx=0x7f8eb4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:08.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.261+0000 7f8eae7fc640 1 -- 172.21.15.31:0/3149263152 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8eb00c0d80 con 0x7f8eb815d2a0 2023-11-26T07:22:08.310 INFO:tasks.ceph.mgr.x.smithi097.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-11-26T07:22:08.310 INFO:tasks.ceph.mgr.x.smithi097.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-11-26T07:22:08.310 INFO:tasks.ceph.mgr.x.smithi097.stderr: from numpy import show_config as show_numpy_config 2023-11-26T07:22:08.319 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:08.316+0000 7f6aafb7e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-26T07:22:08.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.429+0000 7f8ebebe4640 1 -- 172.21.15.31:0/3149263152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8eb80b55d0 con 0x7f8eb815d2a0 2023-11-26T07:22:08.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.429+0000 7f8eae7fc640 1 -- 172.21.15.31:0/3149263152 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8eb80b55d0 con 0x7f8eb815d2a0 2023-11-26T07:22:08.434 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:22:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 -- 172.21.15.31:0/3149263152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8e9c0354e0 msgr2=0x7f8e9c0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8e9c0354e0 0x7f8e9c0379a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400b810 tx=0x7f8eb4008040 comp rx=0 tx=0).stop 2023-11-26T07:22:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 -- 172.21.15.31:0/3149263152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb815d2a0 msgr2=0x7f8eb81f91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb815d2a0 0x7f8eb81f91f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f8eb001a590 tx=0x7f8eb0019700 comp rx=0 tx=0).stop 2023-11-26T07:22:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 -- 172.21.15.31:0/3149263152 shutdown_connections 2023-11-26T07:22:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8e9c0354e0 0x7f8e9c0379a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8eb815d2a0 0x7f8eb81f91f0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8eb80d10e0 0x7f8eb80d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 --2- 172.21.15.31:0/3149263152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb80b4c40 0x7f8eb8155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 -- 172.21.15.31:0/3149263152 >> 172.21.15.31:0/3149263152 conn(0x7f8eb80668a0 msgr2=0x7f8eb815c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:08.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 -- 172.21.15.31:0/3149263152 shutdown_connections 2023-11-26T07:22:08.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.433+0000 7f8ebebe4640 1 -- 172.21.15.31:0/3149263152 wait complete. 2023-11-26T07:22:08.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:08.456 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:08.452+0000 7f6aafb7e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:22:08.583 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:08.580+0000 7f6aafb7e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-26T07:22:08.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.597+0000 7f45fce3a640 1 Processor -- start 2023-11-26T07:22:08.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.597+0000 7f45fce3a640 1 -- start start 2023-11-26T07:22:08.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45fce3a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45f815b2d0 0x7f45f80ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45fce3a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45f80d38d0 0x7f45f80b5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45fce3a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f815b6b0 con 0x7f45f80b5250 2023-11-26T07:22:08.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45fce3a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f80b5c40 con 0x7f45f80d38d0 2023-11-26T07:22:08.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45fce3a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f815ebd0 con 0x7f45f815b2d0 2023-11-26T07:22:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45f6ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f45f80b5250 0x7f45f80d31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45926/0 (socket says 172.21.15.31:45926) 2023-11-26T07:22:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45f7fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45f80d38d0 0x7f45f80b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45f6ffd640 1 -- 172.21.15.31:0/3038604040 learned_addr learned my addr 172.21.15.31:0/3038604040 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45f77fe640 1 --2- 172.21.15.31:0/3038604040 >> v2:172.21.15.97:3300/0 conn(0x7f45f815b2d0 0x7f45f80ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45f7fff640 1 -- 172.21.15.31:0/3038604040 >> v2:172.21.15.97:3300/0 conn(0x7f45f815b2d0 msgr2=0x7f45f80ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45f7fff640 1 --2- 172.21.15.31:0/3038604040 >> v2:172.21.15.97:3300/0 conn(0x7f45f815b2d0 0x7f45f80ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45f7fff640 1 -- 172.21.15.31:0/3038604040 >> v1:172.21.15.31:6789/0 conn(0x7f45f80b5250 legacy=0x7f45f80d31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.601+0000 7f45f7fff640 1 -- 172.21.15.31:0/3038604040 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f80cda70 con 0x7f45f80d38d0 2023-11-26T07:22:08.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45f7fff640 1 --2- 172.21.15.31:0/3038604040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80d38d0 0x7f45f80b5700 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f45f815ed10 tx=0x7f45f001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a384f982503d132 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45f67fc640 1 -- 172.21.15.31:0/3038604040 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f00a58c0 con 0x7f45f80d38d0 2023-11-26T07:22:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45f67fc640 1 -- 172.21.15.31:0/3038604040 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f00a5a20 con 0x7f45f80d38d0 2023-11-26T07:22:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45f67fc640 1 -- 172.21.15.31:0/3038604040 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f00a5db0 con 0x7f45f80d38d0 2023-11-26T07:22:08.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 -- 172.21.15.31:0/3038604040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80d38d0 msgr2=0x7f45f80b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 --2- 172.21.15.31:0/3038604040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80d38d0 0x7f45f80b5700 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f45f815ed10 tx=0x7f45f001c910 comp rx=0 tx=0).stop 2023-11-26T07:22:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 -- 172.21.15.31:0/3038604040 shutdown_connections 2023-11-26T07:22:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 --2- 172.21.15.31:0/3038604040 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80d38d0 0x7f45f80b5700 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 --2- 172.21.15.31:0/3038604040 >> v2:172.21.15.97:3300/0 conn(0x7f45f815b2d0 0x7f45f80ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 -- 172.21.15.31:0/3038604040 >> 172.21.15.31:0/3038604040 conn(0x7f45f80666a0 msgr2=0x7f45f8066ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 -- 172.21.15.31:0/3038604040 shutdown_connections 2023-11-26T07:22:08.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 -- 172.21.15.31:0/3038604040 wait complete. 2023-11-26T07:22:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 Processor -- start 2023-11-26T07:22:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 -- start start 2023-11-26T07:22:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80b5250 0x7f45f8154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45f80d38d0 0x7f45f80c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.605+0000 7f45fce3a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815b2d0 0x7f45f80cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45fce3a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f8154c60 con 0x7f45f80b5250 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45fce3a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f8154f40 con 0x7f45f815b2d0 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45fce3a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f8155220 con 0x7f45f80d38d0 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80b5250 0x7f45f8154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80b5250 0x7f45f8154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58480/0 (socket says 172.21.15.31:58480) 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f77fe640 1 -- 172.21.15.31:0/860096561 learned_addr learned my addr 172.21.15.31:0/860096561 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f6ffd640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45f80d38d0 0x7f45f80c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f7fff640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815b2d0 0x7f45f80cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f77fe640 1 -- 172.21.15.31:0/860096561 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45f80d38d0 msgr2=0x7f45f80c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f77fe640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45f80d38d0 0x7f45f80c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f77fe640 1 -- 172.21.15.31:0/860096561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815b2d0 msgr2=0x7f45f80cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f77fe640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815b2d0 0x7f45f80cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f77fe640 1 -- 172.21.15.31:0/860096561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0009a70 con 0x7f45f80b5250 2023-11-26T07:22:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f77fe640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80b5250 0x7f45f8154580 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f45e800e870 tx=0x7f45e800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45dffff640 1 -- 172.21.15.31:0/860096561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8009510 con 0x7f45f80b5250 2023-11-26T07:22:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45dffff640 1 -- 172.21.15.31:0/860096561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e800eec0 con 0x7f45f80b5250 2023-11-26T07:22:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45fce3a640 1 -- 172.21.15.31:0/860096561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f8155500 con 0x7f45f80b5250 2023-11-26T07:22:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45dffff640 1 -- 172.21.15.31:0/860096561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e800cb40 con 0x7f45f80b5250 2023-11-26T07:22:08.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45fce3a640 1 -- 172.21.15.31:0/860096561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f8155d00 con 0x7f45f80b5250 2023-11-26T07:22:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45fce3a640 1 -- 172.21.15.31:0/860096561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f8155a20 con 0x7f45f80b5250 2023-11-26T07:22:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45dffff640 1 -- 172.21.15.31:0/860096561 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43229+0+0 (secure 0 0 0) 0x7f45e800cca0 con 0x7f45f80b5250 2023-11-26T07:22:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45dffff640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45e40354e0 0x7f45e40379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45dffff640 1 -- 172.21.15.31:0/860096561 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f45e8048300 con 0x7f45f80b5250 2023-11-26T07:22:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f6ffd640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45e40354e0 0x7f45e40379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.609+0000 7f45f6ffd640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45e40354e0 0x7f45e40379a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f45f815ca30 tx=0x7f45ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:08.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.613+0000 7f45dffff640 1 -- 172.21.15.31:0/860096561 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f45f8155a20 con 0x7f45f80b5250 2023-11-26T07:22:08.655 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:08.652+0000 7f6aafb7e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-26T07:22:08.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.773+0000 7f45fce3a640 1 -- 172.21.15.31:0/860096561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45f8155a20 con 0x7f45f80b5250 2023-11-26T07:22:08.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.773+0000 7f45dffff640 1 -- 172.21.15.31:0/860096561 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45e801bd40 con 0x7f45f80b5250 2023-11-26T07:22:08.778 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:22:08.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 -- 172.21.15.31:0/860096561 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45e40354e0 msgr2=0x7f45e40379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45e40354e0 0x7f45e40379a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f45f815ca30 tx=0x7f45ec009040 comp rx=0 tx=0).stop 2023-11-26T07:22:08.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 -- 172.21.15.31:0/860096561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80b5250 msgr2=0x7f45f8154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.780 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80b5250 0x7f45f8154580 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f45e800e870 tx=0x7f45e800ec70 comp rx=0 tx=0).stop 2023-11-26T07:22:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 -- 172.21.15.31:0/860096561 shutdown_connections 2023-11-26T07:22:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45e40354e0 0x7f45e40379a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45f815b2d0 0x7f45f80cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45f80d38d0 0x7f45f80c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 --2- 172.21.15.31:0/860096561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45f80b5250 0x7f45f8154580 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 -- 172.21.15.31:0/860096561 >> 172.21.15.31:0/860096561 conn(0x7f45f80666a0 msgr2=0x7f45f815f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 -- 172.21.15.31:0/860096561 shutdown_connections 2023-11-26T07:22:08.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.777+0000 7f45fce3a640 1 -- 172.21.15.31:0/860096561 wait complete. 2023-11-26T07:22:08.792 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:08.788+0000 7f6aafb7e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-26T07:22:08.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.937+0000 7efff851d640 1 Processor -- start 2023-11-26T07:22:08.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.937+0000 7efff851d640 1 -- start start 2023-11-26T07:22:08.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff851d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efff006dfd0 0x7efff006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff851d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff00adcc0 0x7efff00ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff851d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff006e980 con 0x7efff0143ae0 2023-11-26T07:22:08.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff851d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff0070670 con 0x7efff006dfd0 2023-11-26T07:22:08.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff851d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff0070950 con 0x7efff00adcc0 2023-11-26T07:22:08.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff5ab8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efff006dfd0 0x7efff006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff62b9640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7efff0143ae0 0x7efff00b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45934/0 (socket says 172.21.15.31:45934) 2023-11-26T07:22:08.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff62b9640 1 -- 172.21.15.31:0/1695338899 learned_addr learned my addr 172.21.15.31:0/1695338899 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:08.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff6aba640 1 --2- 172.21.15.31:0/1695338899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff00adcc0 0x7efff00ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff5ab8640 1 -- 172.21.15.31:0/1695338899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff00adcc0 msgr2=0x7efff00ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:08.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff5ab8640 1 --2- 172.21.15.31:0/1695338899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff00adcc0 0x7efff00ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff5ab8640 1 -- 172.21.15.31:0/1695338899 >> v1:172.21.15.31:6789/0 conn(0x7efff0143ae0 legacy=0x7efff00b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff5ab8640 1 -- 172.21.15.31:0/1695338899 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff0070c30 con 0x7efff006dfd0 2023-11-26T07:22:08.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.941+0000 7efff6aba640 1 --2- 172.21.15.31:0/1695338899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff00adcc0 0x7efff00ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:08.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff5ab8640 1 --2- 172.21.15.31:0/1695338899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff006dfd0 0x7efff006e440 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7effe00097c0 tx=0x7effe001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae20dc232644804 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:08.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff52b7640 1 -- 172.21.15.31:0/1695338899 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe00ab650 con 0x7efff006dfd0 2023-11-26T07:22:08.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff52b7640 1 -- 172.21.15.31:0/1695338899 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe00ab7b0 con 0x7efff006dfd0 2023-11-26T07:22:08.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff52b7640 1 -- 172.21.15.31:0/1695338899 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe00abb40 con 0x7efff006dfd0 2023-11-26T07:22:08.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff851d640 1 -- 172.21.15.31:0/1695338899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff006dfd0 msgr2=0x7efff006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff851d640 1 --2- 172.21.15.31:0/1695338899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff006dfd0 0x7efff006e440 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7effe00097c0 tx=0x7effe001c450 comp rx=0 tx=0).stop 2023-11-26T07:22:08.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff851d640 1 -- 172.21.15.31:0/1695338899 shutdown_connections 2023-11-26T07:22:08.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff851d640 1 --2- 172.21.15.31:0/1695338899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff00adcc0 0x7efff00ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff851d640 1 --2- 172.21.15.31:0/1695338899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff006dfd0 0x7efff006e440 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff851d640 1 -- 172.21.15.31:0/1695338899 >> 172.21.15.31:0/1695338899 conn(0x7efff00b0c40 msgr2=0x7efff00b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:08.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff851d640 1 -- 172.21.15.31:0/1695338899 shutdown_connections 2023-11-26T07:22:08.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff851d640 1 -- 172.21.15.31:0/1695338899 wait complete. 2023-11-26T07:22:08.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff851d640 1 Processor -- start 2023-11-26T07:22:08.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.945+0000 7efff851d640 1 -- start start 2023-11-26T07:22:08.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff851d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff006dfd0 0x7efff009b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff851d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff00adcc0 0x7efff01f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff851d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff0143ae0 0x7efff015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff851d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff015cce0 con 0x7efff0143ae0 2023-11-26T07:22:08.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff851d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff01f9610 con 0x7efff00adcc0 2023-11-26T07:22:08.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff851d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff015c0d0 con 0x7efff006dfd0 2023-11-26T07:22:08.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff6aba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff0143ae0 0x7efff015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff6aba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff0143ae0 0x7efff015b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58486/0 (socket says 172.21.15.31:58486) 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff6aba640 1 -- 172.21.15.31:0/3644299845 learned_addr learned my addr 172.21.15.31:0/3644299845 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff62b9640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff006dfd0 0x7efff009b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff5ab8640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff00adcc0 0x7efff01f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff6aba640 1 -- 172.21.15.31:0/3644299845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff006dfd0 msgr2=0x7efff009b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff6aba640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff006dfd0 0x7efff009b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff6aba640 1 -- 172.21.15.31:0/3644299845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff00adcc0 msgr2=0x7efff01f90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff6aba640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff00adcc0 0x7efff01f90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff6aba640 1 -- 172.21.15.31:0/3644299845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe0009420 con 0x7efff0143ae0 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff6aba640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff0143ae0 0x7efff015b880 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7effe8009ea0 tx=0x7effe8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7effe6ffd640 1 -- 172.21.15.31:0/3644299845 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8010070 con 0x7efff0143ae0 2023-11-26T07:22:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7effe6ffd640 1 -- 172.21.15.31:0/3644299845 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe8008c70 con 0x7efff0143ae0 2023-11-26T07:22:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7effe6ffd640 1 -- 172.21.15.31:0/3644299845 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe80039a0 con 0x7efff0143ae0 2023-11-26T07:22:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff851d640 1 -- 172.21.15.31:0/3644299845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff015c3b0 con 0x7efff0143ae0 2023-11-26T07:22:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff851d640 1 -- 172.21.15.31:0/3644299845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff015c970 con 0x7efff0143ae0 2023-11-26T07:22:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff851d640 1 -- 172.21.15.31:0/3644299845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff004e850 con 0x7efff0143ae0 2023-11-26T07:22:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7effe6ffd640 1 -- 172.21.15.31:0/3644299845 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43229+0+0 (secure 0 0 0) 0x7effe8014070 con 0x7efff0143ae0 2023-11-26T07:22:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7effe6ffd640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7effd40354e0 0x7effd40379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7effe6ffd640 1 -- 172.21.15.31:0/3644299845 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7effe8047bd0 con 0x7efff0143ae0 2023-11-26T07:22:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.949+0000 7efff62b9640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7effd40354e0 0x7effd40379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.953+0000 7efff62b9640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7effd40354e0 0x7effd40379a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7effec005890 tx=0x7effec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:08.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:08.957+0000 7effe6ffd640 1 -- 172.21.15.31:0/3644299845 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7effe8004030 con 0x7efff0143ae0 2023-11-26T07:22:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 -- 172.21.15.31:0/3644299845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efff006cc40 con 0x7efff0143ae0 2023-11-26T07:22:09.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7effe6ffd640 1 -- 172.21.15.31:0/3644299845 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4912 (secure 0 0 0) 0x7effe801db00 con 0x7efff0143ae0 2023-11-26T07:22:09.148 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:09.148 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2851182363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2851182363},{"type":"v1","addr":"172.21.15.31:6835","nonce":2851182363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2056572548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2056572548},{"type":"v1","addr":"172.21.15.97:6837","nonce":2056572548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1012539369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1012539369},{"type":"v1","addr":"172.21.15.97:6836","nonce":1012539369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:48.765793+0000","modified":"2023-11-26T07:21:51.814230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6597},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6597":{"gid":6597,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/242493352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":242493352},{"type":"v1","addr":"172.21.15.31:6834","nonce":242493352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:22:09.148 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 48 2023-11-26T07:22:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 -- 172.21.15.31:0/3644299845 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7effd40354e0 msgr2=0x7effd40379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7effd40354e0 0x7effd40379a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7effec005890 tx=0x7effec009040 comp rx=0 tx=0).stop 2023-11-26T07:22:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 -- 172.21.15.31:0/3644299845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff0143ae0 msgr2=0x7efff015b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff0143ae0 0x7efff015b880 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7effe8009ea0 tx=0x7effe8009f90 comp rx=0 tx=0).stop 2023-11-26T07:22:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 -- 172.21.15.31:0/3644299845 shutdown_connections 2023-11-26T07:22:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7effd40354e0 0x7effd40379a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efff0143ae0 0x7efff015b880 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efff00adcc0 0x7efff01f90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 --2- 172.21.15.31:0/3644299845 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7efff006dfd0 0x7efff009b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 -- 172.21.15.31:0/3644299845 >> 172.21.15.31:0/3644299845 conn(0x7efff00b0c40 msgr2=0x7efff00b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 -- 172.21.15.31:0/3644299845 shutdown_connections 2023-11-26T07:22:09.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.145+0000 7efff851d640 1 -- 172.21.15.31:0/3644299845 wait complete. 2023-11-26T07:22:09.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:09.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.309+0000 7f8bcefb8640 1 Processor -- start 2023-11-26T07:22:09.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.309+0000 7f8bcefb8640 1 -- start start 2023-11-26T07:22:09.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.309+0000 7f8bcefb8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8bc806dfd0 0x7f8bc806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.309+0000 7f8bcefb8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc806e980 con 0x7f8bc8143ae0 2023-11-26T07:22:09.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.309+0000 7f8bcefb8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc80ae7f0 con 0x7f8bc80adcc0 2023-11-26T07:22:09.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.309+0000 7f8bcefb8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc8070670 con 0x7f8bc806dfd0 2023-11-26T07:22:09.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.309+0000 7f8bccd54640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8bc8143ae0 0x7f8bc80b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45950/0 (socket says 172.21.15.31:45950) 2023-11-26T07:22:09.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.309+0000 7f8bccd54640 1 -- 172.21.15.31:0/1370448088 learned_addr learned my addr 172.21.15.31:0/1370448088 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:09.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.309+0000 7f8bc7fff640 1 --2- 172.21.15.31:0/1370448088 >> v2:172.21.15.97:3300/0 conn(0x7f8bc806dfd0 0x7f8bc806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.313+0000 7f8bc77fe640 1 -- 172.21.15.31:0/1370448088 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442897472 0 0) 0x7f8bc806e980 con 0x7f8bc8143ae0 2023-11-26T07:22:09.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.313+0000 7f8bc77fe640 1 -- 172.21.15.31:0/1370448088 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb8003530 con 0x7f8bc8143ae0 2023-11-26T07:22:09.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.313+0000 7f8bc7fff640 1 -- 172.21.15.31:0/1370448088 >> v1:172.21.15.97:6789/0 conn(0x7f8bc80adcc0 legacy=0x7f8bc80ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.313+0000 7f8bc77fe640 1 -- 172.21.15.31:0/1370448088 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745262943 0 0) 0x7f8bc80ae7f0 con 0x7f8bc80adcc0 2023-11-26T07:22:09.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.313+0000 7f8bc7fff640 1 -- 172.21.15.31:0/1370448088 >> v1:172.21.15.31:6789/0 conn(0x7f8bc8143ae0 legacy=0x7f8bc80b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.313+0000 7f8bc7fff640 1 -- 172.21.15.31:0/1370448088 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc8070950 con 0x7f8bc806dfd0 2023-11-26T07:22:09.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.313+0000 7f8bc7fff640 1 --2- 172.21.15.31:0/1370448088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc806dfd0 0x7f8bc806e440 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40097c0 tx=0x7f8bb401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5f6267c0edc6246 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:09.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.313+0000 7f8bc77fe640 1 -- 172.21.15.31:0/1370448088 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40ab650 con 0x7f8bc806dfd0 2023-11-26T07:22:09.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.313+0000 7f8bc77fe640 1 -- 172.21.15.31:0/1370448088 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb40ab7b0 con 0x7f8bc806dfd0 2023-11-26T07:22:09.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.313+0000 7f8bc77fe640 1 -- 172.21.15.31:0/1370448088 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40abb40 con 0x7f8bc806dfd0 2023-11-26T07:22:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 -- 172.21.15.31:0/1370448088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc806dfd0 msgr2=0x7f8bc806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 --2- 172.21.15.31:0/1370448088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc806dfd0 0x7f8bc806e440 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40097c0 tx=0x7f8bb401c450 comp rx=0 tx=0).stop 2023-11-26T07:22:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 -- 172.21.15.31:0/1370448088 shutdown_connections 2023-11-26T07:22:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 --2- 172.21.15.31:0/1370448088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc806dfd0 0x7f8bc806e440 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 -- 172.21.15.31:0/1370448088 >> 172.21.15.31:0/1370448088 conn(0x7f8bc80b0c40 msgr2=0x7f8bc80b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 -- 172.21.15.31:0/1370448088 shutdown_connections 2023-11-26T07:22:09.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 -- 172.21.15.31:0/1370448088 wait complete. 2023-11-26T07:22:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 Processor -- start 2023-11-26T07:22:09.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 -- start start 2023-11-26T07:22:09.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8bc806dfd0 0x7f8bc8139d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bc80adcc0 0x7f8bc813ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc8143ae0 0x7f8bc813b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc815db10 con 0x7f8bc80adcc0 2023-11-26T07:22:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8158b70 con 0x7f8bc8143ae0 2023-11-26T07:22:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcefb8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc813a3a0 con 0x7f8bc806dfd0 2023-11-26T07:22:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bc80adcc0 0x7f8bc813ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bc80adcc0 0x7f8bc813ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58500/0 (socket says 172.21.15.31:58500) 2023-11-26T07:22:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bc7fff640 1 -- 172.21.15.31:0/706955756 learned_addr learned my addr 172.21.15.31:0/706955756 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bccd54640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8bc806dfd0 0x7f8bc8139d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bcd555640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc8143ae0 0x7f8bc813b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bc7fff640 1 -- 172.21.15.31:0/706955756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8bc806dfd0 msgr2=0x7f8bc8139d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bc7fff640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8bc806dfd0 0x7f8bc8139d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bc7fff640 1 -- 172.21.15.31:0/706955756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc8143ae0 msgr2=0x7f8bc813b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bc7fff640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc8143ae0 0x7f8bc813b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.317+0000 7f8bc7fff640 1 -- 172.21.15.31:0/706955756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb4009420 con 0x7f8bc80adcc0 2023-11-26T07:22:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bc7fff640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bc80adcc0 0x7f8bc813ab10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40097c0 tx=0x7f8bb401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bc57fa640 1 -- 172.21.15.31:0/706955756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40bf600 con 0x7f8bc80adcc0 2023-11-26T07:22:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bc57fa640 1 -- 172.21.15.31:0/706955756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb40bf760 con 0x7f8bc80adcc0 2023-11-26T07:22:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bcefb8640 1 -- 172.21.15.31:0/706955756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc815aa10 con 0x7f8bc80adcc0 2023-11-26T07:22:09.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bc57fa640 1 -- 172.21.15.31:0/706955756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40bfb10 con 0x7f8bc80adcc0 2023-11-26T07:22:09.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bcefb8640 1 -- 172.21.15.31:0/706955756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc813a5f0 con 0x7f8bc80adcc0 2023-11-26T07:22:09.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bcefb8640 1 -- 172.21.15.31:0/706955756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc804e850 con 0x7f8bc80adcc0 2023-11-26T07:22:09.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bc57fa640 1 -- 172.21.15.31:0/706955756 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43229+0+0 (secure 0 0 0) 0x7f8bb40b5400 con 0x7f8bc80adcc0 2023-11-26T07:22:09.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bc57fa640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8bac0354e0 0x7f8bac0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bc57fa640 1 -- 172.21.15.31:0/706955756 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f8bb40e3500 con 0x7f8bc80adcc0 2023-11-26T07:22:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bccd54640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8bac0354e0 0x7f8bac0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.321+0000 7f8bccd54640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8bac0354e0 0x7f8bac0379a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc007fd0 tx=0x7f8bbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:09.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.325+0000 7f8bc57fa640 1 -- 172.21.15.31:0/706955756 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8bb40be3b0 con 0x7f8bc80adcc0 2023-11-26T07:22:09.335 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:09.332+0000 7f6aafb7e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-26T07:22:09.419 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:09.416+0000 7f6aafb7e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-26T07:22:09.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.481+0000 7f8bc57fa640 1 -- 172.21.15.31:0/706955756 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 14) v1 ==== 43285+0+0 (secure 0 0 0) 0x7f8bb40b3390 con 0x7f8bc80adcc0 2023-11-26T07:22:09.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.485+0000 7f8bcefb8640 1 -- 172.21.15.31:0/706955756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bc806cc40 con 0x7f8bc80adcc0 2023-11-26T07:22:09.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.485+0000 7f8bc57fa640 1 -- 172.21.15.31:0/706955756 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+17291 (secure 0 0 0) 0x7f8bb40be3b0 con 0x7f8bc80adcc0 2023-11-26T07:22:09.489 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:09.489 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":69,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:06.173856+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:46.570507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:47.773615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3784615915":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/62351307":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/160948415":"2023-11-27T07:22:06.173818+0000","172.21.15.97:6833/3008114855":"2023-11-27T07:22:06.173818+0000","172.21.15.97:6832/3008114855":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/1921595339":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/3104042147":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/770009225":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/2385439334":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/4191921753":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/3164817620":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/4223698495":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/1033952263":"2023-11-27T07:22:06.173818+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:09.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 -- 172.21.15.31:0/706955756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8bac0354e0 msgr2=0x7f8bac0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8bac0354e0 0x7f8bac0379a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc007fd0 tx=0x7f8bbc009040 comp rx=0 tx=0).stop 2023-11-26T07:22:09.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 -- 172.21.15.31:0/706955756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bc80adcc0 msgr2=0x7f8bc813ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bc80adcc0 0x7f8bc813ab10 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40097c0 tx=0x7f8bb401ac90 comp rx=0 tx=0).stop 2023-11-26T07:22:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 -- 172.21.15.31:0/706955756 shutdown_connections 2023-11-26T07:22:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8bac0354e0 0x7f8bac0379a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8bc8143ae0 0x7f8bc813b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8bc80adcc0 0x7f8bc813ab10 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 --2- 172.21.15.31:0/706955756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8bc806dfd0 0x7f8bc8139d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 -- 172.21.15.31:0/706955756 >> 172.21.15.31:0/706955756 conn(0x7f8bc80b0c40 msgr2=0x7f8bc80b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 -- 172.21.15.31:0/706955756 shutdown_connections 2023-11-26T07:22:09.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.489+0000 7f8bcefb8640 1 -- 172.21.15.31:0/706955756 wait complete. 2023-11-26T07:22:09.511 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:22:09.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:09.571 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:09.568+0000 7f6aafb7e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-26T07:22:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.649+0000 7f07024e3640 1 Processor -- start 2023-11-26T07:22:09.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.649+0000 7f07024e3640 1 -- start start 2023-11-26T07:22:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f07024e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 0x7f06fc0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f07024e3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f06fc072640 0x7f06fc06ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f07024e3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc070920 con 0x7f06fc06c630 2023-11-26T07:22:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f07024e3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc071be0 con 0x7f06fc072640 2023-11-26T07:22:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f07024e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc15d7c0 con 0x7f06fc15d3e0 2023-11-26T07:22:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f06fb7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f06fc06c630 0x7f06fc071f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45960/0 (socket says 172.21.15.31:45960) 2023-11-26T07:22:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f06fb7fe640 1 -- 172.21.15.31:0/4127359771 learned_addr learned my addr 172.21.15.31:0/4127359771 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f06fbfff640 1 --2- 172.21.15.31:0/4127359771 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 0x7f06fc0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f0700a80640 1 --2- 172.21.15.31:0/4127359771 >> v2:172.21.15.97:3300/0 conn(0x7f06fc072640 0x7f06fc06ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.656 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:09.652+0000 7f6aafb7e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-26T07:22:09.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f06faffd640 1 -- 172.21.15.31:0/4127359771 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869225767 0 0) 0x7f06fc070920 con 0x7f06fc06c630 2023-11-26T07:22:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f06fbfff640 1 -- 172.21.15.31:0/4127359771 >> v2:172.21.15.97:3300/0 conn(0x7f06fc072640 msgr2=0x7f06fc06ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f06fbfff640 1 --2- 172.21.15.31:0/4127359771 >> v2:172.21.15.97:3300/0 conn(0x7f06fc072640 0x7f06fc06ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f06fbfff640 1 -- 172.21.15.31:0/4127359771 >> v1:172.21.15.31:6789/0 conn(0x7f06fc06c630 legacy=0x7f06fc071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f06fbfff640 1 -- 172.21.15.31:0/4127359771 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc070bc0 con 0x7f06fc15d3e0 2023-11-26T07:22:09.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.653+0000 7f0700a80640 1 --2- 172.21.15.31:0/4127359771 >> v2:172.21.15.97:3300/0 conn(0x7f06fc072640 0x7f06fc06ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:09.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f06fbfff640 1 --2- 172.21.15.31:0/4127359771 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 0x7f06fc0716a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0097f0 tx=0x7f06ec01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebadc7fb0d950370 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:09.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f06faffd640 1 -- 172.21.15.31:0/4127359771 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0ab650 con 0x7f06fc15d3e0 2023-11-26T07:22:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f06faffd640 1 -- 172.21.15.31:0/4127359771 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ec0ab7b0 con 0x7f06fc15d3e0 2023-11-26T07:22:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f06faffd640 1 -- 172.21.15.31:0/4127359771 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0abb40 con 0x7f06fc15d3e0 2023-11-26T07:22:09.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 -- 172.21.15.31:0/4127359771 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 msgr2=0x7f06fc0716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 --2- 172.21.15.31:0/4127359771 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 0x7f06fc0716a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0097f0 tx=0x7f06ec01c450 comp rx=0 tx=0).stop 2023-11-26T07:22:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 -- 172.21.15.31:0/4127359771 shutdown_connections 2023-11-26T07:22:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 --2- 172.21.15.31:0/4127359771 >> v2:172.21.15.97:3300/0 conn(0x7f06fc072640 0x7f06fc06ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 --2- 172.21.15.31:0/4127359771 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 0x7f06fc0716a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 -- 172.21.15.31:0/4127359771 >> 172.21.15.31:0/4127359771 conn(0x7f06fc0b0c40 msgr2=0x7f06fc0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 -- 172.21.15.31:0/4127359771 shutdown_connections 2023-11-26T07:22:09.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 -- 172.21.15.31:0/4127359771 wait complete. 2023-11-26T07:22:09.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 Processor -- start 2023-11-26T07:22:09.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 -- start start 2023-11-26T07:22:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.657+0000 7f07024e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc06c630 0x7f06fc154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f07024e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc072640 0x7f06fc1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f07024e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 0x7f06fc14d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f07024e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc159000 con 0x7f06fc06c630 2023-11-26T07:22:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f07024e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc156eb0 con 0x7f06fc072640 2023-11-26T07:22:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f07024e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc155fe0 con 0x7f06fc15d3e0 2023-11-26T07:22:09.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc06c630 0x7f06fc154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc06c630 0x7f06fc154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58512/0 (socket says 172.21.15.31:58512) 2023-11-26T07:22:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fbfff640 1 -- 172.21.15.31:0/3954094965 learned_addr learned my addr 172.21.15.31:0/3954094965 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f0700a80640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 0x7f06fc14d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fb7fe640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc072640 0x7f06fc1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fbfff640 1 -- 172.21.15.31:0/3954094965 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 msgr2=0x7f06fc14d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fbfff640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 0x7f06fc14d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fbfff640 1 -- 172.21.15.31:0/3954094965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc072640 msgr2=0x7f06fc1558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fbfff640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc072640 0x7f06fc1558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fbfff640 1 -- 172.21.15.31:0/3954094965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f4009a70 con 0x7f06fc06c630 2023-11-26T07:22:09.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fbfff640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc06c630 0x7f06fc154ed0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0ad040 tx=0x7f06ec0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:09.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06f8ff9640 1 -- 172.21.15.31:0/3954094965 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0ab650 con 0x7f06fc06c630 2023-11-26T07:22:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06f8ff9640 1 -- 172.21.15.31:0/3954094965 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ec004220 con 0x7f06fc06c630 2023-11-26T07:22:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f07024e3640 1 -- 172.21.15.31:0/3954094965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ec009420 con 0x7f06fc06c630 2023-11-26T07:22:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06f8ff9640 1 -- 172.21.15.31:0/3954094965 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec0ab920 con 0x7f06fc06c630 2023-11-26T07:22:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f07024e3640 1 -- 172.21.15.31:0/3954094965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc158940 con 0x7f06fc06c630 2023-11-26T07:22:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f07024e3640 1 -- 172.21.15.31:0/3954094965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc04e850 con 0x7f06fc06c630 2023-11-26T07:22:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06f8ff9640 1 -- 172.21.15.31:0/3954094965 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 43285+0+0 (secure 0 0 0) 0x7f06ec0bc070 con 0x7f06fc06c630 2023-11-26T07:22:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06f8ff9640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f06e8035570 0x7f06e8037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06f8ff9640 1 -- 172.21.15.31:0/3954094965 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f06ec0e1cb0 con 0x7f06fc06c630 2023-11-26T07:22:09.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fb7fe640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f06e8035570 0x7f06e8037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:09.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.661+0000 7f06fb7fe640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f06e8035570 0x7f06e8037a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06fc155410 tx=0x7f06f0007d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:09.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.665+0000 7f06f8ff9640 1 -- 172.21.15.31:0/3954094965 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f06ec0e8080 con 0x7f06fc06c630 2023-11-26T07:22:09.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 -- 172.21.15.31:0/3954094965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06fc0aed50 con 0x7f06fc06c630 2023-11-26T07:22:09.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f06f8ff9640 1 -- 172.21.15.31:0/3954094965 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f06ec0aba80 con 0x7f06fc06c630 2023-11-26T07:22:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:09.864 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2851182363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2851182363},{"type":"v1","addr":"172.21.15.31:6835","nonce":2851182363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2056572548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2056572548},{"type":"v1","addr":"172.21.15.97:6837","nonce":2056572548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1012539369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1012539369},{"type":"v1","addr":"172.21.15.97:6836","nonce":1012539369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:48.765793+0000","modified":"2023-11-26T07:21:51.814230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6597},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6597":{"gid":6597,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/242493352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":242493352},{"type":"v1","addr":"172.21.15.31:6834","nonce":242493352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:22:09.864 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 48 2023-11-26T07:22:09.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 -- 172.21.15.31:0/3954094965 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f06e8035570 msgr2=0x7f06e8037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f06e8035570 0x7f06e8037a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06fc155410 tx=0x7f06f0007d60 comp rx=0 tx=0).stop 2023-11-26T07:22:09.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 -- 172.21.15.31:0/3954094965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc06c630 msgr2=0x7f06fc154ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc06c630 0x7f06fc154ed0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0ad040 tx=0x7f06ec0191f0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 -- 172.21.15.31:0/3954094965 shutdown_connections 2023-11-26T07:22:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f06e8035570 0x7f06e8037a30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f06fc15d3e0 0x7f06fc14d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06fc072640 0x7f06fc1558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 --2- 172.21.15.31:0/3954094965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06fc06c630 0x7f06fc154ed0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 -- 172.21.15.31:0/3954094965 >> 172.21.15.31:0/3954094965 conn(0x7f06fc0b0c40 msgr2=0x7f06fc160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 -- 172.21.15.31:0/3954094965 shutdown_connections 2023-11-26T07:22:09.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:09.861+0000 7f07024e3640 1 -- 172.21.15.31:0/3954094965 wait complete. 2023-11-26T07:22:09.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:09.971 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:09.968+0000 7f6aafb7e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-26T07:22:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f81164fe640 1 Processor -- start 2023-11-26T07:22:10.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f81164fe640 1 -- start start 2023-11-26T07:22:10.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f81164fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f811015d7b0 0x7f81100d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f81164fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f81100cead0 0x7f81100d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f81164fe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81100d22c0 con 0x7f81100d1eb0 2023-11-26T07:22:10.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f81164fe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81100cefd0 con 0x7f811015d7b0 2023-11-26T07:22:10.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f81164fe640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81100cf2b0 con 0x7f81100cead0 2023-11-26T07:22:10.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f810ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f811015d7b0 0x7f81100d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f810ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f811015d7b0 0x7f81100d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58514/0 (socket says 172.21.15.31:58514) 2023-11-26T07:22:10.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f810ffff640 1 -- 172.21.15.31:0/2569521 learned_addr learned my addr 172.21.15.31:0/2569521 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:10.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f810f7fe640 1 --2- 172.21.15.31:0/2569521 >> v2:172.21.15.97:3300/0 conn(0x7f81100cead0 0x7f81100d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f810ffff640 1 -- 172.21.15.31:0/2569521 >> v2:172.21.15.97:3300/0 conn(0x7f81100cead0 msgr2=0x7f81100d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:10.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f810ffff640 1 --2- 172.21.15.31:0/2569521 >> v2:172.21.15.97:3300/0 conn(0x7f81100cead0 0x7f81100d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f810ffff640 1 -- 172.21.15.31:0/2569521 >> v1:172.21.15.97:6789/0 conn(0x7f81100d1eb0 legacy=0x7f81100cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.025+0000 7f810ffff640 1 -- 172.21.15.31:0/2569521 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811006b3f0 con 0x7f811015d7b0 2023-11-26T07:22:10.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f810ffff640 1 --2- 172.21.15.31:0/2569521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 0x7f81100d10d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f811008ba60 tx=0x7f810001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfcd7181928255b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f810effd640 1 -- 172.21.15.31:0/2569521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81000ac410 con 0x7f811015d7b0 2023-11-26T07:22:10.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f810effd640 1 -- 172.21.15.31:0/2569521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81000ac570 con 0x7f811015d7b0 2023-11-26T07:22:10.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f810effd640 1 -- 172.21.15.31:0/2569521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81000ac900 con 0x7f811015d7b0 2023-11-26T07:22:10.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f81164fe640 1 -- 172.21.15.31:0/2569521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 msgr2=0x7f81100d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f81164fe640 1 --2- 172.21.15.31:0/2569521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 0x7f81100d10d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f811008ba60 tx=0x7f810001c3b0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f81164fe640 1 -- 172.21.15.31:0/2569521 shutdown_connections 2023-11-26T07:22:10.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f81164fe640 1 --2- 172.21.15.31:0/2569521 >> v2:172.21.15.97:3300/0 conn(0x7f81100cead0 0x7f81100d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f81164fe640 1 --2- 172.21.15.31:0/2569521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 0x7f81100d10d0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f81164fe640 1 -- 172.21.15.31:0/2569521 >> 172.21.15.31:0/2569521 conn(0x7f81100d33b0 msgr2=0x7f81100d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:10.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f81164fe640 1 -- 172.21.15.31:0/2569521 shutdown_connections 2023-11-26T07:22:10.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.029+0000 7f81164fe640 1 -- 172.21.15.31:0/2569521 wait complete. 2023-11-26T07:22:10.034 INFO:tasks.ceph.mgr.x.smithi097.stderr:2023-11-26T07:22:10.032+0000 7f6aafb7e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-26T07:22:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f81164fe640 1 Processor -- start 2023-11-26T07:22:10.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f81164fe640 1 -- start start 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f81164fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81100cead0 0x7f81100c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f81164fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81100d1eb0 0x7f81100cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f81164fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 0x7f81101548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f81164fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100cc130 con 0x7f811015d7b0 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f81164fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100c9410 con 0x7f81100cead0 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f81164fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8110154e00 con 0x7f81100d1eb0 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f8114a9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 0x7f81101548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f810ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81100cead0 0x7f81100c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f8114a9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 0x7f81101548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58524/0 (socket says 172.21.15.31:58524) 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f810f7fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81100d1eb0 0x7f81100cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f810ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81100cead0 0x7f81100c9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:43308/0 (socket says 172.21.15.31:43308) 2023-11-26T07:22:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f8114a9b640 1 -- 172.21.15.31:0/4123646883 learned_addr learned my addr 172.21.15.31:0/4123646883 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:10.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f8114a9b640 1 -- 172.21.15.31:0/4123646883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81100d1eb0 msgr2=0x7f81100cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f8114a9b640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81100d1eb0 0x7f81100cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f8114a9b640 1 -- 172.21.15.31:0/4123646883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81100cead0 msgr2=0x7f81100c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f8114a9b640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81100cead0 0x7f81100c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f8114a9b640 1 -- 172.21.15.31:0/4123646883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81000094d0 con 0x7f811015d7b0 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f810ffff640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81100cead0 0x7f81100c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f8114a9b640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 0x7f81101548c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f810800c730 tx=0x7f810800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f810cff9640 1 -- 172.21.15.31:0/4123646883 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108003b80 con 0x7f811015d7b0 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f810cff9640 1 -- 172.21.15.31:0/4123646883 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8108003ce0 con 0x7f811015d7b0 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f810cff9640 1 -- 172.21.15.31:0/4123646883 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108005310 con 0x7f811015d7b0 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f81164fe640 1 -- 172.21.15.31:0/4123646883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81100cb520 con 0x7f811015d7b0 2023-11-26T07:22:10.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.033+0000 7f81164fe640 1 -- 172.21.15.31:0/4123646883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81100cbae0 con 0x7f811015d7b0 2023-11-26T07:22:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.037+0000 7f810cff9640 1 -- 172.21.15.31:0/4123646883 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 43285+0+0 (secure 0 0 0) 0x7f8108005680 con 0x7f811015d7b0 2023-11-26T07:22:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.037+0000 7f81164fe640 1 -- 172.21.15.31:0/4123646883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81100cc2e0 con 0x7f811015d7b0 2023-11-26T07:22:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.037+0000 7f810cff9640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80fc035570 0x7f80fc037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.037+0000 7f810cff9640 1 -- 172.21.15.31:0/4123646883 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f8108048180 con 0x7f811015d7b0 2023-11-26T07:22:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.037+0000 7f810ffff640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80fc035570 0x7f80fc037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.037+0000 7f810ffff640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80fc035570 0x7f80fc037a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f81000ae000 tx=0x7f8100009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.041+0000 7f810cff9640 1 -- 172.21.15.31:0/4123646883 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f81100cc2e0 con 0x7f811015d7b0 2023-11-26T07:22:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.229+0000 7f81164fe640 1 -- 172.21.15.31:0/4123646883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81100cc2e0 con 0x7f811015d7b0 2023-11-26T07:22:10.233 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.229+0000 7f810cff9640 1 -- 172.21.15.31:0/4123646883 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f81100cc2e0 con 0x7f811015d7b0 2023-11-26T07:22:10.233 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:10.233 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2851182363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2851182363},{"type":"v1","addr":"172.21.15.31:6835","nonce":2851182363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2056572548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2056572548},{"type":"v1","addr":"172.21.15.97:6837","nonce":2056572548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1012539369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1012539369},{"type":"v1","addr":"172.21.15.97:6836","nonce":1012539369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:21:48.765793+0000","modified":"2023-11-26T07:21:51.814230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6597},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6597":{"gid":6597,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6834/242493352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":242493352},{"type":"v1","addr":"172.21.15.31:6834","nonce":242493352}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-26T07:22:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 48 2023-11-26T07:22:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 -- 172.21.15.31:0/4123646883 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80fc035570 msgr2=0x7f80fc037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80fc035570 0x7f80fc037a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f81000ae000 tx=0x7f8100009040 comp rx=0 tx=0).stop 2023-11-26T07:22:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 -- 172.21.15.31:0/4123646883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 msgr2=0x7f81101548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 0x7f81101548c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f810800c730 tx=0x7f810800cbf0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 -- 172.21.15.31:0/4123646883 shutdown_connections 2023-11-26T07:22:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80fc035570 0x7f80fc037a30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f811015d7b0 0x7f81101548c0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f81100d1eb0 0x7f81100cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 --2- 172.21.15.31:0/4123646883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81100cead0 0x7f81100c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 -- 172.21.15.31:0/4123646883 >> 172.21.15.31:0/4123646883 conn(0x7f81100d33b0 msgr2=0x7f81100b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 -- 172.21.15.31:0/4123646883 shutdown_connections 2023-11-26T07:22:10.236 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.233+0000 7f81164fe640 1 -- 172.21.15.31:0/4123646883 wait complete. 2023-11-26T07:22:10.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:10.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.393+0000 7f581cec9640 1 Processor -- start 2023-11-26T07:22:10.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.393+0000 7f581cec9640 1 -- start start 2023-11-26T07:22:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.393+0000 7f581cec9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5818143ae0 0x7f58180b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.393+0000 7f581cec9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f581806dfd0 0x7f581806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.393+0000 7f581cec9640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581806e980 con 0x7f58180adcc0 2023-11-26T07:22:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.393+0000 7f581cec9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58180ae7f0 con 0x7f5818143ae0 2023-11-26T07:22:10.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.393+0000 7f581cec9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58180b2480 con 0x7f581806dfd0 2023-11-26T07:22:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.393+0000 7f581659c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5818143ae0 0x7f58180b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f5816d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f58180adcc0 0x7f58180ae0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45962/0 (socket says 172.21.15.31:45962) 2023-11-26T07:22:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f5816d9d640 1 -- 172.21.15.31:0/865926168 learned_addr learned my addr 172.21.15.31:0/865926168 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f5815d9b640 1 --2- 172.21.15.31:0/865926168 >> v2:172.21.15.97:3300/0 conn(0x7f581806dfd0 0x7f581806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f581659c640 1 -- 172.21.15.31:0/865926168 >> v2:172.21.15.97:3300/0 conn(0x7f581806dfd0 msgr2=0x7f581806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f581659c640 1 --2- 172.21.15.31:0/865926168 >> v2:172.21.15.97:3300/0 conn(0x7f581806dfd0 0x7f581806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f581659c640 1 -- 172.21.15.31:0/865926168 >> v1:172.21.15.31:6789/0 conn(0x7f58180adcc0 legacy=0x7f58180ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f581659c640 1 -- 172.21.15.31:0/865926168 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5818070670 con 0x7f5818143ae0 2023-11-26T07:22:10.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f581659c640 1 --2- 172.21.15.31:0/865926168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5818143ae0 0x7f58180b1f40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f581806cbe0 tx=0x7f580801e640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ea310aa2cd534c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f581559a640 1 -- 172.21.15.31:0/865926168 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080a8610 con 0x7f5818143ae0 2023-11-26T07:22:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f581559a640 1 -- 172.21.15.31:0/865926168 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58080a8770 con 0x7f5818143ae0 2023-11-26T07:22:10.402 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.397+0000 7f581559a640 1 -- 172.21.15.31:0/865926168 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080a8b00 con 0x7f5818143ae0 2023-11-26T07:22:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 -- 172.21.15.31:0/865926168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5818143ae0 msgr2=0x7f58180b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 --2- 172.21.15.31:0/865926168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5818143ae0 0x7f58180b1f40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f581806cbe0 tx=0x7f580801e640 comp rx=0 tx=0).stop 2023-11-26T07:22:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 -- 172.21.15.31:0/865926168 shutdown_connections 2023-11-26T07:22:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 --2- 172.21.15.31:0/865926168 >> v2:172.21.15.97:3300/0 conn(0x7f581806dfd0 0x7f581806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 --2- 172.21.15.31:0/865926168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5818143ae0 0x7f58180b1f40 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 -- 172.21.15.31:0/865926168 >> 172.21.15.31:0/865926168 conn(0x7f58180b0c40 msgr2=0x7f58180b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:10.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 -- 172.21.15.31:0/865926168 shutdown_connections 2023-11-26T07:22:10.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 -- 172.21.15.31:0/865926168 wait complete. 2023-11-26T07:22:10.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 Processor -- start 2023-11-26T07:22:10.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 -- start start 2023-11-26T07:22:10.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f581806dfd0 0x7f5818159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58180adcc0 0x7f5818159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818143ae0 0x7f5818130600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581815a0b0 con 0x7f58180adcc0 2023-11-26T07:22:10.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581809ca90 con 0x7f5818143ae0 2023-11-26T07:22:10.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581cec9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581809cd70 con 0x7f581806dfd0 2023-11-26T07:22:10.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f5815d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58180adcc0 0x7f5818159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f5815d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58180adcc0 0x7f5818159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58552/0 (socket says 172.21.15.31:58552) 2023-11-26T07:22:10.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f5815d9b640 1 -- 172.21.15.31:0/2986556750 learned_addr learned my addr 172.21.15.31:0/2986556750 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:10.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f581659c640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f581806dfd0 0x7f5818159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f5816d9d640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818143ae0 0x7f5818130600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f5815d9b640 1 -- 172.21.15.31:0/2986556750 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f581806dfd0 msgr2=0x7f5818159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f5815d9b640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f581806dfd0 0x7f5818159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f5815d9b640 1 -- 172.21.15.31:0/2986556750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818143ae0 msgr2=0x7f5818130600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f5815d9b640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818143ae0 0x7f5818130600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.401+0000 7f5815d9b640 1 -- 172.21.15.31:0/2986556750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5808009420 con 0x7f58180adcc0 2023-11-26T07:22:10.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f5815d9b640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58180adcc0 0x7f5818159b70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f580c00e930 tx=0x7f580c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f5806ffd640 1 -- 172.21.15.31:0/2986556750 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c009690 con 0x7f58180adcc0 2023-11-26T07:22:10.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f5806ffd640 1 -- 172.21.15.31:0/2986556750 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c00ca00 con 0x7f58180adcc0 2023-11-26T07:22:10.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f581cec9640 1 -- 172.21.15.31:0/2986556750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581812e4f0 con 0x7f58180adcc0 2023-11-26T07:22:10.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f5806ffd640 1 -- 172.21.15.31:0/2986556750 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00cd70 con 0x7f58180adcc0 2023-11-26T07:22:10.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f581cec9640 1 -- 172.21.15.31:0/2986556750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581809c750 con 0x7f58180adcc0 2023-11-26T07:22:10.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f581cec9640 1 -- 172.21.15.31:0/2986556750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581809cf90 con 0x7f58180adcc0 2023-11-26T07:22:10.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f5806ffd640 1 -- 172.21.15.31:0/2986556750 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 43285+0+0 (secure 0 0 0) 0x7f580c013470 con 0x7f58180adcc0 2023-11-26T07:22:10.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f5806ffd640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f57f8035570 0x7f57f8037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f5806ffd640 1 -- 172.21.15.31:0/2986556750 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f580c0485e0 con 0x7f58180adcc0 2023-11-26T07:22:10.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f581659c640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f57f8035570 0x7f57f8037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.405+0000 7f581659c640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f57f8035570 0x7f57f8037a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f58080b1000 tx=0x7f58080b4040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.409+0000 7f5806ffd640 1 -- 172.21.15.31:0/2986556750 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f580c01b590 con 0x7f58180adcc0 2023-11-26T07:22:10.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.497+0000 7f5806ffd640 1 -- 172.21.15.31:0/2986556750 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f580c013720 con 0x7f58180adcc0 2023-11-26T07:22:10.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.573+0000 7f581cec9640 1 -- 172.21.15.31:0/2986556750 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f581804e850 con 0x7f58180adcc0 2023-11-26T07:22:10.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.573+0000 7f5806ffd640 1 -- 172.21.15.31:0/2986556750 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+17291 (secure 0 0 0) 0x7f580c028a00 con 0x7f58180adcc0 2023-11-26T07:22:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:10.578 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":69,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:06.173856+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:21:46.570507+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-26T07:21:47.773615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3784615915":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/62351307":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/160948415":"2023-11-27T07:22:06.173818+0000","172.21.15.97:6833/3008114855":"2023-11-27T07:22:06.173818+0000","172.21.15.97:6832/3008114855":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/1921595339":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/3104042147":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/770009225":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/2385439334":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/4191921753":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/3164817620":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/4223698495":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/1033952263":"2023-11-27T07:22:06.173818+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:10.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 -- 172.21.15.31:0/2986556750 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f57f8035570 msgr2=0x7f57f8037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f57f8035570 0x7f57f8037a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f58080b1000 tx=0x7f58080b4040 comp rx=0 tx=0).stop 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 -- 172.21.15.31:0/2986556750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58180adcc0 msgr2=0x7f5818159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58180adcc0 0x7f5818159b70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f580c00e930 tx=0x7f580c00edf0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 -- 172.21.15.31:0/2986556750 shutdown_connections 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f57f8035570 0x7f57f8037a30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818143ae0 0x7f5818130600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58180adcc0 0x7f5818159b70 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 --2- 172.21.15.31:0/2986556750 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f581806dfd0 0x7f5818159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 -- 172.21.15.31:0/2986556750 >> 172.21.15.31:0/2986556750 conn(0x7f58180b0c40 msgr2=0x7f581806ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:10.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 -- 172.21.15.31:0/2986556750 shutdown_connections 2023-11-26T07:22:10.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.577+0000 7f581cec9640 1 -- 172.21.15.31:0/2986556750 wait complete. 2023-11-26T07:22:10.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:22:10.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b3301640 1 Processor -- start 2023-11-26T07:22:10.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b3301640 1 -- start start 2023-11-26T07:22:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b3301640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa9ac15a710 0x7fa9ac0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b3301640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9ac0cbd50 0x7fa9ac0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b3301640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ac0b2710 con 0x7fa9ac0cdfb0 2023-11-26T07:22:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b3301640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ac0cca80 con 0x7fa9ac15a710 2023-11-26T07:22:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b3301640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ac0b29f0 con 0x7fa9ac0cbd50 2023-11-26T07:22:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b109d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa9ac0cdfb0 0x7fa9ac0ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45978/0 (socket says 172.21.15.31:45978) 2023-11-26T07:22:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b109d640 1 -- 172.21.15.31:0/2946925152 learned_addr learned my addr 172.21.15.31:0/2946925152 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:10.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b089c640 1 --2- 172.21.15.31:0/2946925152 >> v2:172.21.15.31:3300/0 conn(0x7fa9ac15a710 0x7fa9ac0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b189e640 1 --2- 172.21.15.31:0/2946925152 >> v2:172.21.15.97:3300/0 conn(0x7fa9ac0cbd50 0x7fa9ac0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b089c640 1 -- 172.21.15.31:0/2946925152 >> v2:172.21.15.97:3300/0 conn(0x7fa9ac0cbd50 msgr2=0x7fa9ac0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:10.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b089c640 1 --2- 172.21.15.31:0/2946925152 >> v2:172.21.15.97:3300/0 conn(0x7fa9ac0cbd50 0x7fa9ac0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b089c640 1 -- 172.21.15.31:0/2946925152 >> v1:172.21.15.31:6789/0 conn(0x7fa9ac0cdfb0 legacy=0x7fa9ac0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9b089c640 1 -- 172.21.15.31:0/2946925152 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac0b2cd0 con 0x7fa9ac15a710 2023-11-26T07:22:10.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.741+0000 7fa9a3fff640 1 -- 172.21.15.31:0/2946925152 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654066276 0 0) 0x7fa9ac0b2710 con 0x7fa9ac0cdfb0 2023-11-26T07:22:10.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9b089c640 1 --2- 172.21.15.31:0/2946925152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 0x7fa9ac0abf90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0aead0 tx=0x7fa99c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de6edd85076ea68 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9a3fff640 1 -- 172.21.15.31:0/2946925152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c0ab650 con 0x7fa9ac15a710 2023-11-26T07:22:10.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9a3fff640 1 -- 172.21.15.31:0/2946925152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99c0ab7b0 con 0x7fa9ac15a710 2023-11-26T07:22:10.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9a3fff640 1 -- 172.21.15.31:0/2946925152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99c0abb40 con 0x7fa9ac15a710 2023-11-26T07:22:10.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9b3301640 1 -- 172.21.15.31:0/2946925152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 msgr2=0x7fa9ac0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9b3301640 1 --2- 172.21.15.31:0/2946925152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 0x7fa9ac0abf90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0aead0 tx=0x7fa99c01c430 comp rx=0 tx=0).stop 2023-11-26T07:22:10.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9b3301640 1 -- 172.21.15.31:0/2946925152 shutdown_connections 2023-11-26T07:22:10.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9b3301640 1 --2- 172.21.15.31:0/2946925152 >> v2:172.21.15.97:3300/0 conn(0x7fa9ac0cbd50 0x7fa9ac0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9b3301640 1 --2- 172.21.15.31:0/2946925152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 0x7fa9ac0abf90 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9b3301640 1 -- 172.21.15.31:0/2946925152 >> 172.21.15.31:0/2946925152 conn(0x7fa9ac0b0cb0 msgr2=0x7fa9ac0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:10.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9b3301640 1 -- 172.21.15.31:0/2946925152 shutdown_connections 2023-11-26T07:22:10.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.745+0000 7fa9b3301640 1 -- 172.21.15.31:0/2946925152 wait complete. 2023-11-26T07:22:10.750 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b3301640 1 Processor -- start 2023-11-26T07:22:10.751 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b3301640 1 -- start start 2023-11-26T07:22:10.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b3301640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9ac0cbd50 0x7fa9ac15d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.752 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b3301640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9ac0cdfb0 0x7fa9ac159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b3301640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 0x7fa9ac070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b3301640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac154950 con 0x7fa9ac15a710 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b3301640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac153aa0 con 0x7fa9ac0cbd50 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b3301640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac154de0 con 0x7fa9ac0cdfb0 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b189e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 0x7fa9ac070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b189e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 0x7fa9ac070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58564/0 (socket says 172.21.15.31:58564) 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b189e640 1 -- 172.21.15.31:0/3591387262 learned_addr learned my addr 172.21.15.31:0/3591387262 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b089c640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9ac0cdfb0 0x7fa9ac159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b109d640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9ac0cbd50 0x7fa9ac15d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b189e640 1 -- 172.21.15.31:0/3591387262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9ac0cdfb0 msgr2=0x7fa9ac159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.753 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b189e640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9ac0cdfb0 0x7fa9ac159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b189e640 1 -- 172.21.15.31:0/3591387262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9ac0cbd50 msgr2=0x7fa9ac15d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b189e640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9ac0cbd50 0x7fa9ac15d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b189e640 1 -- 172.21.15.31:0/3591387262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c009420 con 0x7fa9ac15a710 2023-11-26T07:22:10.754 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b189e640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 0x7fa9ac070690 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4007e40 tx=0x7fa9a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9a1ffb640 1 -- 172.21.15.31:0/3591387262 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4014620 con 0x7fa9ac15a710 2023-11-26T07:22:10.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9a1ffb640 1 -- 172.21.15.31:0/3591387262 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a400ee90 con 0x7fa9ac15a710 2023-11-26T07:22:10.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b3301640 1 -- 172.21.15.31:0/3591387262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ac153d40 con 0x7fa9ac15a710 2023-11-26T07:22:10.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9a1ffb640 1 -- 172.21.15.31:0/3591387262 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a40149e0 con 0x7fa9ac15a710 2023-11-26T07:22:10.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.749+0000 7fa9b3301640 1 -- 172.21.15.31:0/3591387262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac154300 con 0x7fa9ac15a710 2023-11-26T07:22:10.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.753+0000 7fa9b3301640 1 -- 172.21.15.31:0/3591387262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ac0671f0 con 0x7fa9ac15a710 2023-11-26T07:22:10.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.753+0000 7fa9a1ffb640 1 -- 172.21.15.31:0/3591387262 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fa9a4014b40 con 0x7fa9ac15a710 2023-11-26T07:22:10.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.753+0000 7fa9a1ffb640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa9900665e0 0x7fa990068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:10.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.753+0000 7fa9a1ffb640 1 -- 172.21.15.31:0/3591387262 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fa9a4084af0 con 0x7fa9ac15a710 2023-11-26T07:22:10.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.753+0000 7fa9b109d640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa9900665e0 0x7fa990068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:10.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.753+0000 7fa9b109d640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa9900665e0 0x7fa990068aa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8007fd0 tx=0x7fa9a8007d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.757+0000 7fa9a1ffb640 1 -- 172.21.15.31:0/3591387262 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa9a4051cc0 con 0x7fa9ac15a710 2023-11-26T07:22:10.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.945+0000 7fa9b3301640 1 -- 172.21.15.31:0/3591387262 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa9ac06b460 con 0x7fa9ac15a710 2023-11-26T07:22:10.949 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-11-26T07:22:10.945+0000 7fe86499d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:22:10.952 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.949+0000 7fa9a1ffb640 1 -- 172.21.15.31:0/3591387262 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa9a40552e0 con 0x7fa9ac15a710 2023-11-26T07:22:10.952 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:22:10.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 -- 172.21.15.31:0/3591387262 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa9900665e0 msgr2=0x7fa990068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa9900665e0 0x7fa990068aa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8007fd0 tx=0x7fa9a8007d60 comp rx=0 tx=0).stop 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 -- 172.21.15.31:0/3591387262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 msgr2=0x7fa9ac070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 0x7fa9ac070690 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4007e40 tx=0x7fa9a400e4b0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 -- 172.21.15.31:0/3591387262 shutdown_connections 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa9900665e0 0x7fa990068aa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9ac15a710 0x7fa9ac070690 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa9ac0cdfb0 0x7fa9ac159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 --2- 172.21.15.31:0/3591387262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9ac0cbd50 0x7fa9ac15d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 -- 172.21.15.31:0/3591387262 >> 172.21.15.31:0/3591387262 conn(0x7fa9ac0b0cb0 msgr2=0x7fa9ac0ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 -- 172.21.15.31:0/3591387262 shutdown_connections 2023-11-26T07:22:10.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:10.953+0000 7fa9b3301640 1 -- 172.21.15.31:0/3591387262 wait complete. 2023-11-26T07:22:10.971 INFO:tasks.ceph.mds.a.smithi031.stderr: -7065> 2023-11-26T07:21:54.341+0000 7f9f5f15e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:22:10.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:22:11.025 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-11-26T07:22:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4f4be7640 1 Processor -- start 2023-11-26T07:22:11.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4f4be7640 1 -- start start 2023-11-26T07:22:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4f4be7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4f0146820 0x7fc4f0070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4f4be7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc4f00b4210 0x7fc4f00b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4f4be7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 0x7fc4f01615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4f4be7640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f00b4be0 con 0x7fc4f0146820 2023-11-26T07:22:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4f4be7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f0147190 con 0x7fc4f00b4210 2023-11-26T07:22:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4f4be7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f006d030 con 0x7fc4f0161180 2023-11-26T07:22:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4ee59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4f0146820 0x7fc4f0070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4ee59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc4f0146820 0x7fc4f0070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58592/0 (socket says 172.21.15.31:58592) 2023-11-26T07:22:11.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4ee59c640 1 -- 172.21.15.31:0/3765459014 learned_addr learned my addr 172.21.15.31:0/3765459014 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4eed9d640 1 --2- 172.21.15.31:0/3765459014 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 0x7fc4f01615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4edd9b640 1 --2- 172.21.15.31:0/3765459014 >> v2:172.21.15.97:3300/0 conn(0x7fc4f00b4210 0x7fc4f00b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4ee59c640 1 -- 172.21.15.31:0/3765459014 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 msgr2=0x7fc4f01615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4ee59c640 1 --2- 172.21.15.31:0/3765459014 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 0x7fc4f01615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4ee59c640 1 -- 172.21.15.31:0/3765459014 >> v2:172.21.15.97:3300/0 conn(0x7fc4f00b4210 msgr2=0x7fc4f00b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4ee59c640 1 --2- 172.21.15.31:0/3765459014 >> v2:172.21.15.97:3300/0 conn(0x7fc4f00b4210 0x7fc4f00b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4ee59c640 1 -- 172.21.15.31:0/3765459014 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f0146c00 con 0x7fc4f0146820 2023-11-26T07:22:11.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.129+0000 7fc4edd9b640 1 --2- 172.21.15.31:0/3765459014 >> v2:172.21.15.97:3300/0 conn(0x7fc4f00b4210 0x7fc4f00b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:11.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4ee59c640 1 --2- 172.21.15.31:0/3765459014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f0146820 0x7fc4f0070a50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00b5ad0 tx=0x7fc4e001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21c69c52659b98b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:11.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4ed59a640 1 -- 172.21.15.31:0/3765459014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00aa5f0 con 0x7fc4f0146820 2023-11-26T07:22:11.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4ed59a640 1 -- 172.21.15.31:0/3765459014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e00aa750 con 0x7fc4f0146820 2023-11-26T07:22:11.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4ed59a640 1 -- 172.21.15.31:0/3765459014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00aaae0 con 0x7fc4f0146820 2023-11-26T07:22:11.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4f4be7640 1 -- 172.21.15.31:0/3765459014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f0146820 msgr2=0x7fc4f0070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:11.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4f4be7640 1 --2- 172.21.15.31:0/3765459014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f0146820 0x7fc4f0070a50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00b5ad0 tx=0x7fc4e001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4f4be7640 1 -- 172.21.15.31:0/3765459014 shutdown_connections 2023-11-26T07:22:11.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4f4be7640 1 --2- 172.21.15.31:0/3765459014 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 0x7fc4f01615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4f4be7640 1 --2- 172.21.15.31:0/3765459014 >> v2:172.21.15.97:3300/0 conn(0x7fc4f00b4210 0x7fc4f00b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4f4be7640 1 --2- 172.21.15.31:0/3765459014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f0146820 0x7fc4f0070a50 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4f4be7640 1 -- 172.21.15.31:0/3765459014 >> 172.21.15.31:0/3765459014 conn(0x7fc4f00b2f10 msgr2=0x7fc4f00b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:11.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4f4be7640 1 -- 172.21.15.31:0/3765459014 shutdown_connections 2023-11-26T07:22:11.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.133+0000 7fc4f4be7640 1 -- 172.21.15.31:0/3765459014 wait complete. 2023-11-26T07:22:11.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4f4be7640 1 Processor -- start 2023-11-26T07:22:11.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4f4be7640 1 -- start start 2023-11-26T07:22:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4f4be7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f00b4210 0x7fc4f013e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4f4be7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f0146820 0x7fc4f013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4f4be7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 0x7fc4f00b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4f4be7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f01f6950 con 0x7fc4f00b4210 2023-11-26T07:22:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4f4be7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f013ff90 con 0x7fc4f0146820 2023-11-26T07:22:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4f4be7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0152890 con 0x7fc4f0161180 2023-11-26T07:22:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4ee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f00b4210 0x7fc4f013e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4ee59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f00b4210 0x7fc4f013e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58606/0 (socket says 172.21.15.31:58606) 2023-11-26T07:22:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4ee59c640 1 -- 172.21.15.31:0/2047099166 learned_addr learned my addr 172.21.15.31:0/2047099166 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4edd9b640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f0146820 0x7fc4f013ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4eed9d640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 0x7fc4f00b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4ee59c640 1 -- 172.21.15.31:0/2047099166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 msgr2=0x7fc4f00b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4ee59c640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 0x7fc4f00b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4ee59c640 1 -- 172.21.15.31:0/2047099166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f0146820 msgr2=0x7fc4f013ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4ee59c640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f0146820 0x7fc4f013ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4ee59c640 1 -- 172.21.15.31:0/2047099166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e00094d0 con 0x7fc4f00b4210 2023-11-26T07:22:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4eed9d640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 0x7fc4f00b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4edd9b640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f0146820 0x7fc4f013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4ee59c640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f00b4210 0x7fc4f013e480 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc4e001ab80 tx=0x7fc4e001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.137+0000 7fc4deffd640 1 -- 172.21.15.31:0/2047099166 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00b3800 con 0x7fc4f00b4210 2023-11-26T07:22:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.141+0000 7fc4deffd640 1 -- 172.21.15.31:0/2047099166 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e00b3960 con 0x7fc4f00b4210 2023-11-26T07:22:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.141+0000 7fc4deffd640 1 -- 172.21.15.31:0/2047099166 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00b3d10 con 0x7fc4f00b4210 2023-11-26T07:22:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.141+0000 7fc4f4be7640 1 -- 172.21.15.31:0/2047099166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f013f6f0 con 0x7fc4f00b4210 2023-11-26T07:22:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.141+0000 7fc4f4be7640 1 -- 172.21.15.31:0/2047099166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f013fcb0 con 0x7fc4f00b4210 2023-11-26T07:22:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.141+0000 7fc4f4be7640 1 -- 172.21.15.31:0/2047099166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f004e850 con 0x7fc4f00b4210 2023-11-26T07:22:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.141+0000 7fc4deffd640 1 -- 172.21.15.31:0/2047099166 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fc4e00b2380 con 0x7fc4f00b4210 2023-11-26T07:22:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.141+0000 7fc4deffd640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc4d0066590 0x7fc4d0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.141+0000 7fc4deffd640 1 -- 172.21.15.31:0/2047099166 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fc4e00a7070 con 0x7fc4f00b4210 2023-11-26T07:22:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.141+0000 7fc4edd9b640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc4d0066590 0x7fc4d0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:11.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.141+0000 7fc4edd9b640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc4d0066590 0x7fc4d0068a50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc4f006d730 tx=0x7fc4e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:11.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.145+0000 7fc4deffd640 1 -- 172.21.15.31:0/2047099166 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc4e00eba90 con 0x7fc4f00b4210 2023-11-26T07:22:11.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.337+0000 7fc4f4be7640 1 -- 172.21.15.31:0/2047099166 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc4f00d1f00 con 0x7fc4f00b4210 2023-11-26T07:22:11.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4deffd640 1 -- 172.21.15.31:0/2047099166 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc4e00ef0b0 con 0x7fc4f00b4210 2023-11-26T07:22:11.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4f4be7640 1 -- 172.21.15.31:0/2047099166 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc4d0066590 msgr2=0x7fc4d0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4f4be7640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc4d0066590 0x7fc4d0068a50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc4f006d730 tx=0x7fc4e4008040 comp rx=0 tx=0).stop 2023-11-26T07:22:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4f4be7640 1 -- 172.21.15.31:0/2047099166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f00b4210 msgr2=0x7fc4f013e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4f4be7640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f00b4210 0x7fc4f013e480 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc4e001ab80 tx=0x7fc4e001ad10 comp rx=0 tx=0).stop 2023-11-26T07:22:11.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4f4be7640 1 -- 172.21.15.31:0/2047099166 shutdown_connections 2023-11-26T07:22:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4f4be7640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc4d0066590 0x7fc4d0068a50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4f4be7640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc4f0161180 0x7fc4f00b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4f4be7640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4f0146820 0x7fc4f013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4f4be7640 1 --2- 172.21.15.31:0/2047099166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f00b4210 0x7fc4f013e480 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.953+0000 7fc4f4be7640 1 -- 172.21.15.31:0/2047099166 >> 172.21.15.31:0/2047099166 conn(0x7fc4f00b2f10 msgr2=0x7fc4f00b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.957+0000 7fc4f4be7640 1 -- 172.21.15.31:0/2047099166 shutdown_connections 2023-11-26T07:22:11.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:11.957+0000 7fc4f4be7640 1 -- 172.21.15.31:0/2047099166 wait complete. 2023-11-26T07:22:11.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:22:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.173+0000 7f298d613640 1 Processor -- start 2023-11-26T07:22:12.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.173+0000 7f298d613640 1 -- start start 2023-11-26T07:22:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.173+0000 7f298d613640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2988146820 0x7f2988070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f298d613640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f298815eb50 0x7f29880b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f298d613640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2988147190 con 0x7f29880b4210 2023-11-26T07:22:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f298d613640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29880b4be0 con 0x7f2988146820 2023-11-26T07:22:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f298d613640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2988146c00 con 0x7f298815eb50 2023-11-26T07:22:12.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f297ffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f29880b4210 0x7f2988161180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:45986/0 (socket says 172.21.15.31:45986) 2023-11-26T07:22:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f297ffff640 1 -- 172.21.15.31:0/247605466 learned_addr learned my addr 172.21.15.31:0/247605466 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f29877fe640 1 --2- 172.21.15.31:0/247605466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f298815eb50 0x7f29880b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f2986ffd640 1 --2- 172.21.15.31:0/247605466 >> v2:172.21.15.97:3300/0 conn(0x7f2988146820 0x7f2988070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f29867fc640 1 -- 172.21.15.31:0/247605466 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319105143 0 0) 0x7f2988147190 con 0x7f29880b4210 2023-11-26T07:22:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f29867fc640 1 -- 172.21.15.31:0/247605466 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2974003530 con 0x7f29880b4210 2023-11-26T07:22:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f29877fe640 1 -- 172.21.15.31:0/247605466 >> v2:172.21.15.97:3300/0 conn(0x7f2988146820 msgr2=0x7f2988070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f29877fe640 1 --2- 172.21.15.31:0/247605466 >> v2:172.21.15.97:3300/0 conn(0x7f2988146820 0x7f2988070a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f29877fe640 1 -- 172.21.15.31:0/247605466 >> v1:172.21.15.31:6789/0 conn(0x7f29880b4210 legacy=0x7f2988161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:12.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f29877fe640 1 -- 172.21.15.31:0/247605466 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298806d030 con 0x7f298815eb50 2023-11-26T07:22:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.177+0000 7f29877fe640 1 --2- 172.21.15.31:0/247605466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f298815eb50 0x7f29880b46a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2980009d90 tx=0x7f298001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9221183f2d2673f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f29867fc640 1 -- 172.21.15.31:0/247605466 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800a5900 con 0x7f298815eb50 2023-11-26T07:22:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f29867fc640 1 -- 172.21.15.31:0/247605466 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29800a5a60 con 0x7f298815eb50 2023-11-26T07:22:12.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f29867fc640 1 -- 172.21.15.31:0/247605466 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800a5df0 con 0x7f298815eb50 2023-11-26T07:22:12.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 -- 172.21.15.31:0/247605466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f298815eb50 msgr2=0x7f29880b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:12.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 --2- 172.21.15.31:0/247605466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f298815eb50 0x7f29880b46a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2980009d90 tx=0x7f298001c920 comp rx=0 tx=0).stop 2023-11-26T07:22:12.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 -- 172.21.15.31:0/247605466 shutdown_connections 2023-11-26T07:22:12.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 --2- 172.21.15.31:0/247605466 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f298815eb50 0x7f29880b46a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:12.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 --2- 172.21.15.31:0/247605466 >> v2:172.21.15.97:3300/0 conn(0x7f2988146820 0x7f2988070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:12.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 -- 172.21.15.31:0/247605466 >> 172.21.15.31:0/247605466 conn(0x7f29880b2f10 msgr2=0x7f29880b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:12.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 -- 172.21.15.31:0/247605466 shutdown_connections 2023-11-26T07:22:12.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 -- 172.21.15.31:0/247605466 wait complete. 2023-11-26T07:22:12.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 Processor -- start 2023-11-26T07:22:12.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 -- start start 2023-11-26T07:22:12.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29880b4210 0x7f298815ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2988146820 0x7f298813a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298815eb50 0x7f298815c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298815f7b0 con 0x7f29880b4210 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2988140660 con 0x7f298815eb50 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f298d613640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29881400a0 con 0x7f2988146820 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f2986ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29880b4210 0x7f298815ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f2986ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29880b4210 0x7f298815ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58622/0 (socket says 172.21.15.31:58622) 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.181+0000 7f2986ffd640 1 -- 172.21.15.31:0/2868408587 learned_addr learned my addr 172.21.15.31:0/2868408587 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f297ffff640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2988146820 0x7f298813a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f29877fe640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298815eb50 0x7f298815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f2986ffd640 1 -- 172.21.15.31:0/2868408587 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2988146820 msgr2=0x7f298813a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:12.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f2986ffd640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2988146820 0x7f298813a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f2986ffd640 1 -- 172.21.15.31:0/2868408587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298815eb50 msgr2=0x7f298815c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f2986ffd640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298815eb50 0x7f298815c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f2986ffd640 1 -- 172.21.15.31:0/2868408587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980009a70 con 0x7f29880b4210 2023-11-26T07:22:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f2986ffd640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29880b4210 0x7f298815ff70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f297800e930 tx=0x7f297800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:12.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f297f7fe640 1 -- 172.21.15.31:0/2868408587 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2978009690 con 0x7f29880b4210 2023-11-26T07:22:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f297f7fe640 1 -- 172.21.15.31:0/2868408587 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297800ca00 con 0x7f29880b4210 2023-11-26T07:22:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f298d613640 1 -- 172.21.15.31:0/2868408587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2988140380 con 0x7f29880b4210 2023-11-26T07:22:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f297f7fe640 1 -- 172.21.15.31:0/2868408587 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800cd70 con 0x7f29880b4210 2023-11-26T07:22:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f298d613640 1 -- 172.21.15.31:0/2868408587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298815d540 con 0x7f29880b4210 2023-11-26T07:22:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f298d613640 1 -- 172.21.15.31:0/2868408587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298804e850 con 0x7f29880b4210 2023-11-26T07:22:12.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.185+0000 7f297f7fe640 1 -- 172.21.15.31:0/2868408587 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f2978013470 con 0x7f29880b4210 2023-11-26T07:22:12.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.189+0000 7f297f7fe640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2968066590 0x7f2968068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:12.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.189+0000 7f297f7fe640 1 -- 172.21.15.31:0/2868408587 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f2978084430 con 0x7f29880b4210 2023-11-26T07:22:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.189+0000 7f297ffff640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2968066590 0x7f2968068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:12.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.189+0000 7f297ffff640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2968066590 0x7f2968068a50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29880b0ef0 tx=0x7f2970009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:12.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.193+0000 7f297f7fe640 1 -- 172.21.15.31:0/2868408587 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2978051680 con 0x7f29880b4210 2023-11-26T07:22:12.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:12.353+0000 7f298d613640 1 -- 172.21.15.31:0/2868408587 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f298806d8d0 con 0x7f29880b4210 2023-11-26T07:22:13.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.061+0000 7f297f7fe640 1 -- 172.21.15.31:0/2868408587 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v71) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2978054ca0 con 0x7f29880b4210 2023-11-26T07:22:13.065 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:22:13.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 -- 172.21.15.31:0/2868408587 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2968066590 msgr2=0x7f2968068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2968066590 0x7f2968068a50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f29880b0ef0 tx=0x7f2970009040 comp rx=0 tx=0).stop 2023-11-26T07:22:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 -- 172.21.15.31:0/2868408587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29880b4210 msgr2=0x7f298815ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29880b4210 0x7f298815ff70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f297800e930 tx=0x7f297800edf0 comp rx=0 tx=0).stop 2023-11-26T07:22:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 -- 172.21.15.31:0/2868408587 shutdown_connections 2023-11-26T07:22:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2968066590 0x7f2968068a50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f298815eb50 0x7f298815c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2988146820 0x7f298813a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 --2- 172.21.15.31:0/2868408587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29880b4210 0x7f298815ff70 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 -- 172.21.15.31:0/2868408587 >> 172.21.15.31:0/2868408587 conn(0x7f29880b2f10 msgr2=0x7f29880b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:13.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 -- 172.21.15.31:0/2868408587 shutdown_connections 2023-11-26T07:22:13.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.065+0000 7f298d613640 1 -- 172.21.15.31:0/2868408587 wait complete. 2023-11-26T07:22:13.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:22:13.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b45a68640 1 Processor -- start 2023-11-26T07:22:13.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b45a68640 1 -- start start 2023-11-26T07:22:13.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b45a68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b40147060 0x7f2b400b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:13.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b45a68640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2b400ad000 0x7f2b4006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:13.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b45a68640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b400b2cb0 con 0x7f2b40070a50 2023-11-26T07:22:13.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b45a68640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b400b2f90 con 0x7f2b400ad000 2023-11-26T07:22:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b45a68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400b3270 con 0x7f2b40147060 2023-11-26T07:22:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b3effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2b400ad000 0x7f2b4006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b3effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2b400ad000 0x7f2b4006cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58638/0 (socket says 172.21.15.31:58638) 2023-11-26T07:22:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b3effd640 1 -- 172.21.15.31:0/217196740 learned_addr learned my addr 172.21.15.31:0/217196740 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b3f7fe640 1 --2- 172.21.15.31:0/217196740 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b40147060 0x7f2b400b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b3effd640 1 -- 172.21.15.31:0/217196740 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b40147060 msgr2=0x7f2b400b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b3effd640 1 --2- 172.21.15.31:0/217196740 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b40147060 0x7f2b400b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b3effd640 1 -- 172.21.15.31:0/217196740 >> v1:172.21.15.97:6789/0 conn(0x7f2b40070a50 legacy=0x7f2b400b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.245+0000 7f2b3effd640 1 -- 172.21.15.31:0/217196740 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b400b3550 con 0x7f2b400ad000 2023-11-26T07:22:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b3effd640 1 --2- 172.21.15.31:0/217196740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 0x7f2b4006cde0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009870 tx=0x7f2b3401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eece2a92877a977 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b3e7fc640 1 -- 172.21.15.31:0/217196740 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b340ab5f0 con 0x7f2b400ad000 2023-11-26T07:22:13.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b3e7fc640 1 -- 172.21.15.31:0/217196740 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b340ab750 con 0x7f2b400ad000 2023-11-26T07:22:13.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b3e7fc640 1 -- 172.21.15.31:0/217196740 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b340abae0 con 0x7f2b400ad000 2023-11-26T07:22:13.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b45a68640 1 -- 172.21.15.31:0/217196740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 msgr2=0x7f2b4006cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:13.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b45a68640 1 --2- 172.21.15.31:0/217196740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 0x7f2b4006cde0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009870 tx=0x7f2b3401c540 comp rx=0 tx=0).stop 2023-11-26T07:22:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b45a68640 1 -- 172.21.15.31:0/217196740 shutdown_connections 2023-11-26T07:22:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b45a68640 1 --2- 172.21.15.31:0/217196740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 0x7f2b4006cde0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b45a68640 1 --2- 172.21.15.31:0/217196740 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b40147060 0x7f2b400b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b45a68640 1 -- 172.21.15.31:0/217196740 >> 172.21.15.31:0/217196740 conn(0x7f2b400cf3b0 msgr2=0x7f2b400cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b45a68640 1 -- 172.21.15.31:0/217196740 shutdown_connections 2023-11-26T07:22:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.249+0000 7f2b45a68640 1 -- 172.21.15.31:0/217196740 wait complete. 2023-11-26T07:22:13.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b45a68640 1 Processor -- start 2023-11-26T07:22:13.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b45a68640 1 -- start start 2023-11-26T07:22:13.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b45a68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b40070a50 0x7f2b4013e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:13.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b45a68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 0x7f2b4013ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:13.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b45a68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b40147060 0x7f2b40162170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:13.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b45a68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40164280 con 0x7f2b400ad000 2023-11-26T07:22:13.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b45a68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400cc8f0 con 0x7f2b40070a50 2023-11-26T07:22:13.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b45a68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400ccbd0 con 0x7f2b40147060 2023-11-26T07:22:13.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 0x7f2b4013ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:13.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b40070a50 0x7f2b4013e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:13.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 0x7f2b4013ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58640/0 (socket says 172.21.15.31:58640) 2023-11-26T07:22:13.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3effd640 1 -- 172.21.15.31:0/2568820418 learned_addr learned my addr 172.21.15.31:0/2568820418 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:13.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3ffff640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b40147060 0x7f2b40162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:13.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3effd640 1 -- 172.21.15.31:0/2568820418 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b40147060 msgr2=0x7f2b40162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:13.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3effd640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b40147060 0x7f2b40162170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3effd640 1 -- 172.21.15.31:0/2568820418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b40070a50 msgr2=0x7f2b4013e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3effd640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b40070a50 0x7f2b4013e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3effd640 1 -- 172.21.15.31:0/2568820418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340094d0 con 0x7f2b400ad000 2023-11-26T07:22:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b3effd640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 0x7f2b4013ff30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2b340ad040 tx=0x7f2b3401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b27fff640 1 -- 172.21.15.31:0/2568820418 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b340be5f0 con 0x7f2b400ad000 2023-11-26T07:22:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b27fff640 1 -- 172.21.15.31:0/2568820418 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b340be770 con 0x7f2b400ad000 2023-11-26T07:22:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b27fff640 1 -- 172.21.15.31:0/2568820418 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b340beb20 con 0x7f2b400ad000 2023-11-26T07:22:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b45a68640 1 -- 172.21.15.31:0/2568820418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b400cceb0 con 0x7f2b400ad000 2023-11-26T07:22:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.253+0000 7f2b45a68640 1 -- 172.21.15.31:0/2568820418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b400cd470 con 0x7f2b400ad000 2023-11-26T07:22:13.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.257+0000 7f2b45a68640 1 -- 172.21.15.31:0/2568820418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4004e850 con 0x7f2b400ad000 2023-11-26T07:22:13.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.257+0000 7f2b27fff640 1 -- 172.21.15.31:0/2568820418 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f2b340b43f0 con 0x7f2b400ad000 2023-11-26T07:22:13.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.257+0000 7f2b27fff640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2b2c066590 0x7f2b2c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:13.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.257+0000 7f2b27fff640 1 -- 172.21.15.31:0/2568820418 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5916+0+0 (secure 0 0 0) 0x7f2b340a8050 con 0x7f2b400ad000 2023-11-26T07:22:13.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.257+0000 7f2b3f7fe640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2b2c066590 0x7f2b2c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:13.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.257+0000 7f2b3f7fe640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2b2c066590 0x7f2b2c068a50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b300079c0 tx=0x7f2b30008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:13.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.261+0000 7f2b27fff640 1 -- 172.21.15.31:0/2568820418 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2b340ebf50 con 0x7f2b400ad000 2023-11-26T07:22:13.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:13.425+0000 7f2b45a68640 1 -- 172.21.15.31:0/2568820418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2b400af1a0 con 0x7f2b400ad000 2023-11-26T07:22:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.065+0000 7f2b27fff640 1 -- 172.21.15.31:0/2568820418 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v72) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2b340c40d0 con 0x7f2b400ad000 2023-11-26T07:22:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-11-26T07:22:14.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.065+0000 7f2b45a68640 1 -- 172.21.15.31:0/2568820418 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2b2c066590 msgr2=0x7f2b2c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.065+0000 7f2b45a68640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2b2c066590 0x7f2b2c068a50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2b300079c0 tx=0x7f2b30008040 comp rx=0 tx=0).stop 2023-11-26T07:22:14.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.065+0000 7f2b45a68640 1 -- 172.21.15.31:0/2568820418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 msgr2=0x7f2b4013ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.065+0000 7f2b45a68640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 0x7f2b4013ff30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2b340ad040 tx=0x7f2b3401ad80 comp rx=0 tx=0).stop 2023-11-26T07:22:14.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.069+0000 7f2b45a68640 1 -- 172.21.15.31:0/2568820418 shutdown_connections 2023-11-26T07:22:14.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.069+0000 7f2b45a68640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2b2c066590 0x7f2b2c068a50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.069+0000 7f2b45a68640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2b40147060 0x7f2b40162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.069+0000 7f2b45a68640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b400ad000 0x7f2b4013ff30 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.069+0000 7f2b45a68640 1 --2- 172.21.15.31:0/2568820418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b40070a50 0x7f2b4013e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.069+0000 7f2b45a68640 1 -- 172.21.15.31:0/2568820418 >> 172.21.15.31:0/2568820418 conn(0x7f2b400cf3b0 msgr2=0x7f2b40144300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:14.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.069+0000 7f2b45a68640 1 -- 172.21.15.31:0/2568820418 shutdown_connections 2023-11-26T07:22:14.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.073+0000 7f2b45a68640 1 -- 172.21.15.31:0/2568820418 wait complete. 2023-11-26T07:22:14.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:22:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.249+0000 7f571989a640 1 Processor -- start 2023-11-26T07:22:14.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.249+0000 7f571989a640 1 -- start start 2023-11-26T07:22:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f571989a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f571406cc30 0x7f57140caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f571989a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57140cbfe0 con 0x7f57140ce4b0 2023-11-26T07:22:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f571989a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571406b970 con 0x7f571415d610 2023-11-26T07:22:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f571989a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571406bc50 con 0x7f571406cc30 2023-11-26T07:22:14.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f570bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f57140ce4b0 0x7f571406c520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36582/0 (socket says 172.21.15.31:36582) 2023-11-26T07:22:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f570bfff640 1 -- 172.21.15.31:0/1607219453 learned_addr learned my addr 172.21.15.31:0/1607219453 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f57137fe640 1 --2- 172.21.15.31:0/1607219453 >> v2:172.21.15.97:3300/0 conn(0x7f571406cc30 0x7f57140caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f57127fc640 1 -- 172.21.15.31:0/1607219453 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384525339 0 0) 0x7f57140cbfe0 con 0x7f57140ce4b0 2023-11-26T07:22:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f57127fc640 1 -- 172.21.15.31:0/1607219453 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5700003530 con 0x7f57140ce4b0 2023-11-26T07:22:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f57127fc640 1 -- 172.21.15.31:0/1607219453 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723853960 0 0) 0x7f571406b970 con 0x7f571415d610 2023-11-26T07:22:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f57127fc640 1 -- 172.21.15.31:0/1607219453 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57140cbfe0 con 0x7f571415d610 2023-11-26T07:22:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f57127fc640 1 -- 172.21.15.31:0/1607219453 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2643879458 0 0) 0x7f5700003530 con 0x7f57140ce4b0 2023-11-26T07:22:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f57137fe640 1 -- 172.21.15.31:0/1607219453 >> v1:172.21.15.97:6789/0 conn(0x7f571415d610 legacy=0x7f57140cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f57137fe640 1 -- 172.21.15.31:0/1607219453 >> v1:172.21.15.31:6789/0 conn(0x7f57140ce4b0 legacy=0x7f571406c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.253+0000 7f57137fe640 1 -- 172.21.15.31:0/1607219453 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57140cfe60 con 0x7f571406cc30 2023-11-26T07:22:14.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f57137fe640 1 --2- 172.21.15.31:0/1607219453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f571406cc30 0x7f57140caa60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f570c009d90 tx=0x7f570c01c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=638ded36676ae1f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f57127fc640 1 -- 172.21.15.31:0/1607219453 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc001680 con 0x7f571406cc30 2023-11-26T07:22:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f57127fc640 1 -- 172.21.15.31:0/1607219453 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c0a6480 con 0x7f571406cc30 2023-11-26T07:22:14.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f57127fc640 1 -- 172.21.15.31:0/1607219453 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c0a67f0 con 0x7f571406cc30 2023-11-26T07:22:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f571989a640 1 -- 172.21.15.31:0/1607219453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f571406cc30 msgr2=0x7f57140caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f571989a640 1 --2- 172.21.15.31:0/1607219453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f571406cc30 0x7f57140caa60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f570c009d90 tx=0x7f570c01c870 comp rx=0 tx=0).stop 2023-11-26T07:22:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f571989a640 1 -- 172.21.15.31:0/1607219453 shutdown_connections 2023-11-26T07:22:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f571989a640 1 --2- 172.21.15.31:0/1607219453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f571406cc30 0x7f57140caa60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f571989a640 1 -- 172.21.15.31:0/1607219453 >> 172.21.15.31:0/1607219453 conn(0x7f57140b2f10 msgr2=0x7f57140b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f571989a640 1 -- 172.21.15.31:0/1607219453 shutdown_connections 2023-11-26T07:22:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f571989a640 1 -- 172.21.15.31:0/1607219453 wait complete. 2023-11-26T07:22:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f571989a640 1 Processor -- start 2023-11-26T07:22:14.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.257+0000 7f571989a640 1 -- start start 2023-11-26T07:22:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f571989a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f571406cc30 0x7f5714148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f571989a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f57140ce4b0 0x7f5714145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f571989a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571415d610 0x7f5714151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f571989a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571414f600 con 0x7f571415d610 2023-11-26T07:22:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f571989a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5714146b00 con 0x7f571406cc30 2023-11-26T07:22:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f571989a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571414f040 con 0x7f57140ce4b0 2023-11-26T07:22:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f57137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571415d610 0x7f5714151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f57137fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571415d610 0x7f5714151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49304/0 (socket says 172.21.15.31:49304) 2023-11-26T07:22:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f57137fe640 1 -- 172.21.15.31:0/1770328135 learned_addr learned my addr 172.21.15.31:0/1770328135 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f5712ffd640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f571406cc30 0x7f5714148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f570bfff640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f57140ce4b0 0x7f5714145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f57137fe640 1 -- 172.21.15.31:0/1770328135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f57140ce4b0 msgr2=0x7f5714145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f57137fe640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f57140ce4b0 0x7f5714145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f57137fe640 1 -- 172.21.15.31:0/1770328135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f571406cc30 msgr2=0x7f5714148b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f57137fe640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f571406cc30 0x7f5714148b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f57137fe640 1 -- 172.21.15.31:0/1770328135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c009a70 con 0x7f571415d610 2023-11-26T07:22:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f57137fe640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571415d610 0x7f5714151580 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f570c009d90 tx=0x7f570c019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f5710ff9640 1 -- 172.21.15.31:0/1770328135 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c0b4040 con 0x7f571415d610 2023-11-26T07:22:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f5710ff9640 1 -- 172.21.15.31:0/1770328135 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c0bc680 con 0x7f571415d610 2023-11-26T07:22:14.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f5710ff9640 1 -- 172.21.15.31:0/1770328135 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c007000 con 0x7f571415d610 2023-11-26T07:22:14.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f571989a640 1 -- 172.21.15.31:0/1770328135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571414f320 con 0x7f571415d610 2023-11-26T07:22:14.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f571989a640 1 -- 172.21.15.31:0/1770328135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5714146320 con 0x7f571415d610 2023-11-26T07:22:14.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f571989a640 1 -- 172.21.15.31:0/1770328135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571404e850 con 0x7f571415d610 2023-11-26T07:22:14.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.261+0000 7f5710ff9640 1 -- 172.21.15.31:0/1770328135 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f570c0bca30 con 0x7f571415d610 2023-11-26T07:22:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.265+0000 7f5710ff9640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f56f4066590 0x7f56f4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.265+0000 7f5710ff9640 1 -- 172.21.15.31:0/1770328135 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f570c11edb0 con 0x7f571415d610 2023-11-26T07:22:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.265+0000 7f5712ffd640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f56f4066590 0x7f56f4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.265+0000 7f5712ffd640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f56f4066590 0x7f56f4068a50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5704001720 tx=0x7f5704009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:14.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.269+0000 7f5710ff9640 1 -- 172.21.15.31:0/1770328135 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f570c0ec220 con 0x7f571415d610 2023-11-26T07:22:14.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.425+0000 7f571989a640 1 -- 172.21.15.31:0/1770328135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5714152c60 con 0x7f571415d610 2023-11-26T07:22:14.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.429+0000 7f5710ff9640 1 -- 172.21.15.31:0/1770328135 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f570c009450 con 0x7f571415d610 2023-11-26T07:22:14.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 -- 172.21.15.31:0/1770328135 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f56f4066590 msgr2=0x7f56f4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f56f4066590 0x7f56f4068a50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5704001720 tx=0x7f5704009040 comp rx=0 tx=0).stop 2023-11-26T07:22:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 -- 172.21.15.31:0/1770328135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571415d610 msgr2=0x7f5714151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571415d610 0x7f5714151580 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f570c009d90 tx=0x7f570c019610 comp rx=0 tx=0).stop 2023-11-26T07:22:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 -- 172.21.15.31:0/1770328135 shutdown_connections 2023-11-26T07:22:14.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f56f4066590 0x7f56f4068a50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571415d610 0x7f5714151580 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f57140ce4b0 0x7f5714145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 --2- 172.21.15.31:0/1770328135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f571406cc30 0x7f5714148b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 -- 172.21.15.31:0/1770328135 >> 172.21.15.31:0/1770328135 conn(0x7f57140b2f10 msgr2=0x7f57140b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 -- 172.21.15.31:0/1770328135 shutdown_connections 2023-11-26T07:22:14.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.433+0000 7f571989a640 1 -- 172.21.15.31:0/1770328135 wait complete. 2023-11-26T07:22:14.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2023-11-26T07:22:14.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87c12bb640 1 Processor -- start 2023-11-26T07:22:14.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87c12bb640 1 -- start start 2023-11-26T07:22:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87c12bb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f87bc15d570 0x7f87bc0b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87c12bb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f87bc0d1da0 0x7f87bc0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87c12bb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87bc0d23c0 con 0x7f87bc0b8720 2023-11-26T07:22:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87c12bb640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87bc0d21d0 con 0x7f87bc0d1da0 2023-11-26T07:22:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87c12bb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87bc0d26a0 con 0x7f87bc15d570 2023-11-26T07:22:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87bb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f87bc0d1da0 0x7f87bc0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87bb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f87bc0d1da0 0x7f87bc0b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49312/0 (socket says 172.21.15.31:49312) 2023-11-26T07:22:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87bb7fe640 1 -- 172.21.15.31:0/379610683 learned_addr learned my addr 172.21.15.31:0/379610683 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87baffd640 1 --2- 172.21.15.31:0/379610683 >> v2:172.21.15.97:3300/0 conn(0x7f87bc15d570 0x7f87bc0b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87bb7fe640 1 -- 172.21.15.31:0/379610683 >> v2:172.21.15.97:3300/0 conn(0x7f87bc15d570 msgr2=0x7f87bc0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87bb7fe640 1 --2- 172.21.15.31:0/379610683 >> v2:172.21.15.97:3300/0 conn(0x7f87bc15d570 0x7f87bc0b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87bb7fe640 1 -- 172.21.15.31:0/379610683 >> v1:172.21.15.97:6789/0 conn(0x7f87bc0b8720 legacy=0x7f87bc0d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87bb7fe640 1 -- 172.21.15.31:0/379610683 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0d2980 con 0x7f87bc0d1da0 2023-11-26T07:22:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.597+0000 7f87baffd640 1 --2- 172.21.15.31:0/379610683 >> v2:172.21.15.97:3300/0 conn(0x7f87bc15d570 0x7f87bc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:14.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87bb7fe640 1 --2- 172.21.15.31:0/379610683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0d1da0 0x7f87bc0b5130 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f87bc08b720 tx=0x7f87b401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df15fe2fbd717f32 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87b9ffb640 1 -- 172.21.15.31:0/379610683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40a5670 con 0x7f87bc0d1da0 2023-11-26T07:22:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87b9ffb640 1 -- 172.21.15.31:0/379610683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b40a57d0 con 0x7f87bc0d1da0 2023-11-26T07:22:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87b9ffb640 1 -- 172.21.15.31:0/379610683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40a5b60 con 0x7f87bc0d1da0 2023-11-26T07:22:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87c12bb640 1 -- 172.21.15.31:0/379610683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0d1da0 msgr2=0x7f87bc0b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87c12bb640 1 --2- 172.21.15.31:0/379610683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0d1da0 0x7f87bc0b5130 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f87bc08b720 tx=0x7f87b401c900 comp rx=0 tx=0).stop 2023-11-26T07:22:14.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87c12bb640 1 -- 172.21.15.31:0/379610683 shutdown_connections 2023-11-26T07:22:14.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87c12bb640 1 --2- 172.21.15.31:0/379610683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0d1da0 0x7f87bc0b5130 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87c12bb640 1 --2- 172.21.15.31:0/379610683 >> v2:172.21.15.97:3300/0 conn(0x7f87bc15d570 0x7f87bc0b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87c12bb640 1 -- 172.21.15.31:0/379610683 >> 172.21.15.31:0/379610683 conn(0x7f87bc0d3340 msgr2=0x7f87bc0d3750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:14.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87c12bb640 1 -- 172.21.15.31:0/379610683 shutdown_connections 2023-11-26T07:22:14.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87c12bb640 1 -- 172.21.15.31:0/379610683 wait complete. 2023-11-26T07:22:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.601+0000 7f87c12bb640 1 Processor -- start 2023-11-26T07:22:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87c12bb640 1 -- start start 2023-11-26T07:22:14.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87c12bb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87bc0b8720 0x7f87bc0cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87c12bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0d1da0 0x7f87bc155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87c12bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc15d570 0x7f87bc156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87c12bb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0cb9b0 con 0x7f87bc15d570 2023-11-26T07:22:14.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87c12bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0cabc0 con 0x7f87bc0d1da0 2023-11-26T07:22:14.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87c12bb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc0caea0 con 0x7f87bc0b8720 2023-11-26T07:22:14.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc15d570 0x7f87bc156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc15d570 0x7f87bc156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49328/0 (socket says 172.21.15.31:49328) 2023-11-26T07:22:14.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87bb7fe640 1 -- 172.21.15.31:0/891883015 learned_addr learned my addr 172.21.15.31:0/891883015 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87baffd640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87bc0b8720 0x7f87bc0cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87ba7fc640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0d1da0 0x7f87bc155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87bb7fe640 1 -- 172.21.15.31:0/891883015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87bc0b8720 msgr2=0x7f87bc0cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87bb7fe640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87bc0b8720 0x7f87bc0cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87bb7fe640 1 -- 172.21.15.31:0/891883015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0d1da0 msgr2=0x7f87bc155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87bb7fe640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0d1da0 0x7f87bc155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87bb7fe640 1 -- 172.21.15.31:0/891883015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b4009b30 con 0x7f87bc15d570 2023-11-26T07:22:14.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87ba7fc640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0d1da0 0x7f87bc155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87bb7fe640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc15d570 0x7f87bc156400 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f87b40af040 tx=0x7f87b40196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:14.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87a37fe640 1 -- 172.21.15.31:0/891883015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40b8670 con 0x7f87bc15d570 2023-11-26T07:22:14.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87a37fe640 1 -- 172.21.15.31:0/891883015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b40b87d0 con 0x7f87bc15d570 2023-11-26T07:22:14.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87a37fe640 1 -- 172.21.15.31:0/891883015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40b8b60 con 0x7f87bc15d570 2023-11-26T07:22:14.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87c12bb640 1 -- 172.21.15.31:0/891883015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc0cbbc0 con 0x7f87bc15d570 2023-11-26T07:22:14.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87c12bb640 1 -- 172.21.15.31:0/891883015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc0cc180 con 0x7f87bc15d570 2023-11-26T07:22:14.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.605+0000 7f87c12bb640 1 -- 172.21.15.31:0/891883015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc074a00 con 0x7f87bc15d570 2023-11-26T07:22:14.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.609+0000 7f87a37fe640 1 -- 172.21.15.31:0/891883015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f87b40b8cc0 con 0x7f87bc15d570 2023-11-26T07:22:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.609+0000 7f87a37fe640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f87a8066590 0x7f87a8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.609+0000 7f87a37fe640 1 -- 172.21.15.31:0/891883015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f87b411f470 con 0x7f87bc15d570 2023-11-26T07:22:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.609+0000 7f87baffd640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f87a8066590 0x7f87a8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.609+0000 7f87baffd640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f87a8066590 0x7f87a8068a50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0d1d70 tx=0x7f87ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:14.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.613+0000 7f87a37fe640 1 -- 172.21.15.31:0/891883015 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f87b40eca80 con 0x7f87bc15d570 2023-11-26T07:22:14.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:14.833+0000 7f87c12bb640 1 -- 172.21.15.31:0/891883015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) v1 -- 0x7f87bc156c40 con 0x7f87bc15d570 2023-11-26T07:22:15.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.081+0000 7f87a37fe640 1 -- 172.21.15.31:0/891883015 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v255) v1 ==== 131+0+0 (secure 0 0 0) 0x7f87b40f00a0 con 0x7f87bc15d570 2023-11-26T07:22:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 -- 172.21.15.31:0/891883015 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f87a8066590 msgr2=0x7f87a8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f87a8066590 0x7f87a8068a50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0d1d70 tx=0x7f87ac008040 comp rx=0 tx=0).stop 2023-11-26T07:22:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 -- 172.21.15.31:0/891883015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc15d570 msgr2=0x7f87bc156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc15d570 0x7f87bc156400 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f87b40af040 tx=0x7f87b40196a0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 -- 172.21.15.31:0/891883015 shutdown_connections 2023-11-26T07:22:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f87a8066590 0x7f87a8068a50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc15d570 0x7f87bc156400 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0d1da0 0x7f87bc155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 --2- 172.21.15.31:0/891883015 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f87bc0b8720 0x7f87bc0cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 -- 172.21.15.31:0/891883015 >> 172.21.15.31:0/891883015 conn(0x7f87bc0d3340 msgr2=0x7f87bc0b7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 -- 172.21.15.31:0/891883015 shutdown_connections 2023-11-26T07:22:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.085+0000 7f87c12bb640 1 -- 172.21.15.31:0/891883015 wait complete. 2023-11-26T07:22:15.107 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2023-11-26T07:22:15.107 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2023-11-26T07:22:15.107 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:22:15.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:15.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5198af3640 1 Processor -- start 2023-11-26T07:22:15.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5198af3640 1 -- start start 2023-11-26T07:22:15.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5198af3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f519406bc90 0x7f51940cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5198af3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51940b2850 0x7f51940b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5198af3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519406d4f0 con 0x7f5194160980 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5198af3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51940d2070 con 0x7f51940b2850 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5198af3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519406c150 con 0x7f519406bc90 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f51937fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5194160980 0x7f519406b580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36586/0 (socket says 172.21.15.31:36586) 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f51937fe640 1 -- 172.21.15.31:0/4201561524 learned_addr learned my addr 172.21.15.31:0/4201561524 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5193fff640 1 --2- 172.21.15.31:0/4201561524 >> v2:172.21.15.31:3300/0 conn(0x7f51940b2850 0x7f51940b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5192ffd640 1 --2- 172.21.15.31:0/4201561524 >> v2:172.21.15.97:3300/0 conn(0x7f519406bc90 0x7f51940cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f51927fc640 1 -- 172.21.15.31:0/4201561524 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293016933 0 0) 0x7f519406d4f0 con 0x7f5194160980 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f51927fc640 1 -- 172.21.15.31:0/4201561524 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f517c003530 con 0x7f5194160980 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5193fff640 1 -- 172.21.15.31:0/4201561524 >> v2:172.21.15.97:3300/0 conn(0x7f519406bc90 msgr2=0x7f51940cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5193fff640 1 --2- 172.21.15.31:0/4201561524 >> v2:172.21.15.97:3300/0 conn(0x7f519406bc90 0x7f51940cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5193fff640 1 -- 172.21.15.31:0/4201561524 >> v1:172.21.15.31:6789/0 conn(0x7f5194160980 legacy=0x7f519406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.249+0000 7f5193fff640 1 -- 172.21.15.31:0/4201561524 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519406c430 con 0x7f51940b2850 2023-11-26T07:22:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f5193fff640 1 --2- 172.21.15.31:0/4201561524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51940b2850 0x7f51940b18f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f51940b92f0 tx=0x7f518c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7d652e1b088978 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f51927fc640 1 -- 172.21.15.31:0/4201561524 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0a59a0 con 0x7f51940b2850 2023-11-26T07:22:15.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f51927fc640 1 -- 172.21.15.31:0/4201561524 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c0a5b00 con 0x7f51940b2850 2023-11-26T07:22:15.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f51927fc640 1 -- 172.21.15.31:0/4201561524 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0a5e70 con 0x7f51940b2850 2023-11-26T07:22:15.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f5198af3640 1 -- 172.21.15.31:0/4201561524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51940b2850 msgr2=0x7f51940b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f5198af3640 1 --2- 172.21.15.31:0/4201561524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51940b2850 0x7f51940b18f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f51940b92f0 tx=0x7f518c01c8e0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f5198af3640 1 -- 172.21.15.31:0/4201561524 shutdown_connections 2023-11-26T07:22:15.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f5198af3640 1 --2- 172.21.15.31:0/4201561524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51940b2850 0x7f51940b18f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f5198af3640 1 --2- 172.21.15.31:0/4201561524 >> v2:172.21.15.97:3300/0 conn(0x7f519406bc90 0x7f51940cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f5198af3640 1 -- 172.21.15.31:0/4201561524 >> 172.21.15.31:0/4201561524 conn(0x7f519409c5c0 msgr2=0x7f51940d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:15.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f5198af3640 1 -- 172.21.15.31:0/4201561524 shutdown_connections 2023-11-26T07:22:15.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f5198af3640 1 -- 172.21.15.31:0/4201561524 wait complete. 2023-11-26T07:22:15.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.253+0000 7f5198af3640 1 Processor -- start 2023-11-26T07:22:15.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5198af3640 1 -- start start 2023-11-26T07:22:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5198af3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f519406bc90 0x7f5194156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5198af3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51940b2850 0x7f51940ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5198af3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5194160980 0x7f51940b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5198af3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51940b5fe0 con 0x7f5194160980 2023-11-26T07:22:15.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5198af3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51940b62c0 con 0x7f51940b2850 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5198af3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51940b65a0 con 0x7f519406bc90 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5193fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5194160980 0x7f51940b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5193fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5194160980 0x7f51940b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49350/0 (socket says 172.21.15.31:49350) 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5193fff640 1 -- 172.21.15.31:0/1568081823 learned_addr learned my addr 172.21.15.31:0/1568081823 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f51937fe640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f519406bc90 0x7f5194156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5192ffd640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51940b2850 0x7f51940ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5193fff640 1 -- 172.21.15.31:0/1568081823 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f519406bc90 msgr2=0x7f5194156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5193fff640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f519406bc90 0x7f5194156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5193fff640 1 -- 172.21.15.31:0/1568081823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51940b2850 msgr2=0x7f51940ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5193fff640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51940b2850 0x7f51940ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5193fff640 1 -- 172.21.15.31:0/1568081823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c009a70 con 0x7f5194160980 2023-11-26T07:22:15.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5193fff640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5194160980 0x7f51940b06e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f518c0194f0 tx=0x7f518c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f517bfff640 1 -- 172.21.15.31:0/1568081823 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0aed30 con 0x7f5194160980 2023-11-26T07:22:15.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f517bfff640 1 -- 172.21.15.31:0/1568081823 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c0aee90 con 0x7f5194160980 2023-11-26T07:22:15.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f517bfff640 1 -- 172.21.15.31:0/1568081823 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0b7610 con 0x7f5194160980 2023-11-26T07:22:15.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5198af3640 1 -- 172.21.15.31:0/1568081823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51940b6880 con 0x7f5194160980 2023-11-26T07:22:15.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5198af3640 1 -- 172.21.15.31:0/1568081823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5194156f90 con 0x7f5194160980 2023-11-26T07:22:15.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.257+0000 7f5198af3640 1 -- 172.21.15.31:0/1568081823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5194074a00 con 0x7f5194160980 2023-11-26T07:22:15.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.261+0000 7f517bfff640 1 -- 172.21.15.31:0/1568081823 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f518c0b4050 con 0x7f5194160980 2023-11-26T07:22:15.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.261+0000 7f517bfff640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5180066590 0x7f5180068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.261+0000 7f517bfff640 1 -- 172.21.15.31:0/1568081823 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f518c11f280 con 0x7f5194160980 2023-11-26T07:22:15.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.261+0000 7f51937fe640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5180066590 0x7f5180068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.261+0000 7f51937fe640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5180066590 0x7f5180068a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5184005890 tx=0x7f5184009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.265+0000 7f517bfff640 1 -- 172.21.15.31:0/1568081823 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f518c0ec870 con 0x7f5194160980 2023-11-26T07:22:15.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 -- 172.21.15.31:0/1568081823 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51940b6d30 con 0x7f5194160980 2023-11-26T07:22:15.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f517bfff640 1 -- 172.21.15.31:0/1568081823 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+13709 (secure 0 0 0) 0x7f518c009450 con 0x7f5194160980 2023-11-26T07:22:15.427 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:15.428 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:14.065037+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/4223698495":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/3164817620":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/4191921753":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/1033952263":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/2385439334":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/770009225":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/3104042147":"2023-11-27T07:22:06.173818+0000","172.21.15.31:6834/242493352":"2023-11-27T07:22:10.950126+0000","172.21.15.97:0/1921595339":"2023-11-27T07:22:06.173818+0000","172.21.15.97:6832/3008114855":"2023-11-27T07:22:06.173818+0000","172.21.15.31:6832/242493352":"2023-11-27T07:22:10.950126+0000","172.21.15.97:6833/3008114855":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/160948415":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/62351307":"2023-11-27T07:22:06.173818+0000","172.21.15.97:0/3784615915":"2023-11-27T07:22:06.173818+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:15.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 -- 172.21.15.31:0/1568081823 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5180066590 msgr2=0x7f5180068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5180066590 0x7f5180068a50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5184005890 tx=0x7f5184009040 comp rx=0 tx=0).stop 2023-11-26T07:22:15.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 -- 172.21.15.31:0/1568081823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5194160980 msgr2=0x7f51940b06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5194160980 0x7f51940b06e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f518c0194f0 tx=0x7f518c019680 comp rx=0 tx=0).stop 2023-11-26T07:22:15.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 -- 172.21.15.31:0/1568081823 shutdown_connections 2023-11-26T07:22:15.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5180066590 0x7f5180068a50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5194160980 0x7f51940b06e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51940b2850 0x7f51940ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 --2- 172.21.15.31:0/1568081823 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f519406bc90 0x7f5194156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 -- 172.21.15.31:0/1568081823 >> 172.21.15.31:0/1568081823 conn(0x7f519409c5c0 msgr2=0x7f519415b380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 -- 172.21.15.31:0/1568081823 shutdown_connections 2023-11-26T07:22:15.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.425+0000 7f5198af3640 1 -- 172.21.15.31:0/1568081823 wait complete. 2023-11-26T07:22:15.448 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:15.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:15.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.589+0000 7f1a776c9640 1 Processor -- start 2023-11-26T07:22:15.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a776c9640 1 -- start start 2023-11-26T07:22:15.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a776c9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70146820 0x7f1a70070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a776c9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a700b4210 0x7f1a700b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a776c9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a70161180 0x7f1a701615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a776c9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a70147190 con 0x7f1a700b4210 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a776c9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a7006d030 con 0x7f1a70161180 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a776c9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700b4bb0 con 0x7f1a70146820 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a74c64640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a700b4210 0x7f1a700b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a74c64640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a700b4210 0x7f1a700b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49352/0 (socket says 172.21.15.31:49352) 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a74c64640 1 -- 172.21.15.31:0/2278541343 learned_addr learned my addr 172.21.15.31:0/2278541343 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a75c66640 1 --2- 172.21.15.31:0/2278541343 >> v2:172.21.15.97:3300/0 conn(0x7f1a70161180 0x7f1a701615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a75465640 1 --2- 172.21.15.31:0/2278541343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70146820 0x7f1a70070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a74c64640 1 -- 172.21.15.31:0/2278541343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70146820 msgr2=0x7f1a70070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a74c64640 1 --2- 172.21.15.31:0/2278541343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70146820 0x7f1a70070a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a74c64640 1 -- 172.21.15.31:0/2278541343 >> v2:172.21.15.97:3300/0 conn(0x7f1a70161180 msgr2=0x7f1a701615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:15.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a74c64640 1 --2- 172.21.15.31:0/2278541343 >> v2:172.21.15.97:3300/0 conn(0x7f1a70161180 0x7f1a701615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.593+0000 7f1a74c64640 1 -- 172.21.15.31:0/2278541343 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70146c00 con 0x7f1a700b4210 2023-11-26T07:22:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a74c64640 1 --2- 172.21.15.31:0/2278541343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 0x7f1a700b4670 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1a700b0e10 tx=0x7f1a6001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc29fc4e20428c84 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a67fff640 1 -- 172.21.15.31:0/2278541343 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a600aa5f0 con 0x7f1a700b4210 2023-11-26T07:22:15.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a67fff640 1 -- 172.21.15.31:0/2278541343 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a600aa750 con 0x7f1a700b4210 2023-11-26T07:22:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a67fff640 1 -- 172.21.15.31:0/2278541343 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a600aaae0 con 0x7f1a700b4210 2023-11-26T07:22:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 -- 172.21.15.31:0/2278541343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 msgr2=0x7f1a700b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 --2- 172.21.15.31:0/2278541343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 0x7f1a700b4670 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1a700b0e10 tx=0x7f1a6001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 -- 172.21.15.31:0/2278541343 shutdown_connections 2023-11-26T07:22:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 --2- 172.21.15.31:0/2278541343 >> v2:172.21.15.97:3300/0 conn(0x7f1a70161180 0x7f1a701615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 --2- 172.21.15.31:0/2278541343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 0x7f1a700b4670 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 --2- 172.21.15.31:0/2278541343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70146820 0x7f1a70070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 -- 172.21.15.31:0/2278541343 >> 172.21.15.31:0/2278541343 conn(0x7f1a700b2f10 msgr2=0x7f1a700b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 -- 172.21.15.31:0/2278541343 shutdown_connections 2023-11-26T07:22:15.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 -- 172.21.15.31:0/2278541343 wait complete. 2023-11-26T07:22:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 Processor -- start 2023-11-26T07:22:15.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.597+0000 7f1a776c9640 1 -- start start 2023-11-26T07:22:15.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a776c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 0x7f1a7013c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a776c9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70146820 0x7f1a70141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a776c9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70161180 0x7f1a7015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a776c9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7013cc30 con 0x7f1a700b4210 2023-11-26T07:22:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a776c9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7013d860 con 0x7f1a70146820 2023-11-26T07:22:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a776c9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7013db00 con 0x7f1a70161180 2023-11-26T07:22:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75465640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 0x7f1a7013c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75465640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 0x7f1a7013c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49368/0 (socket says 172.21.15.31:49368) 2023-11-26T07:22:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75465640 1 -- 172.21.15.31:0/88523984 learned_addr learned my addr 172.21.15.31:0/88523984 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:15.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a74c64640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70146820 0x7f1a70141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75465640 1 -- 172.21.15.31:0/88523984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70161180 msgr2=0x7f1a7015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75c66640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70161180 0x7f1a7015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75465640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70161180 0x7f1a7015cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75465640 1 -- 172.21.15.31:0/88523984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70146820 msgr2=0x7f1a70141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75465640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70146820 0x7f1a70141170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75465640 1 -- 172.21.15.31:0/88523984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600094d0 con 0x7f1a700b4210 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a74c64640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70146820 0x7f1a70141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75c66640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70161180 0x7f1a7015cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a75465640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 0x7f1a7013c6f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00e8b0 tx=0x7f1a6c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a65ffb640 1 -- 172.21.15.31:0/88523984 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00c940 con 0x7f1a700b4210 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a65ffb640 1 -- 172.21.15.31:0/88523984 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6c00caa0 con 0x7f1a700b4210 2023-11-26T07:22:15.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a65ffb640 1 -- 172.21.15.31:0/88523984 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00ce30 con 0x7f1a700b4210 2023-11-26T07:22:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a776c9640 1 -- 172.21.15.31:0/88523984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7013dde0 con 0x7f1a700b4210 2023-11-26T07:22:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a776c9640 1 -- 172.21.15.31:0/88523984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7013e0c0 con 0x7f1a700b4210 2023-11-26T07:22:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.601+0000 7f1a776c9640 1 -- 172.21.15.31:0/88523984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a70159d00 con 0x7f1a700b4210 2023-11-26T07:22:15.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.605+0000 7f1a65ffb640 1 -- 172.21.15.31:0/88523984 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f1a6c013470 con 0x7f1a700b4210 2023-11-26T07:22:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.605+0000 7f1a65ffb640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a54066590 0x7f1a54068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.605+0000 7f1a65ffb640 1 -- 172.21.15.31:0/88523984 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f1a6c084160 con 0x7f1a700b4210 2023-11-26T07:22:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.605+0000 7f1a74c64640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a54066590 0x7f1a54068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.605+0000 7f1a74c64640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a54066590 0x7f1a54068a50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a600ae000 tx=0x7f1a60008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.609+0000 7f1a65ffb640 1 -- 172.21.15.31:0/88523984 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1a6c051510 con 0x7f1a700b4210 2023-11-26T07:22:15.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.769+0000 7f1a776c9640 1 -- 172.21.15.31:0/88523984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1a700712e0 con 0x7f1a700b4210 2023-11-26T07:22:15.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.769+0000 7f1a65ffb640 1 -- 172.21.15.31:0/88523984 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1a6c054b30 con 0x7f1a700b4210 2023-11-26T07:22:15.773 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:22:15.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 -- 172.21.15.31:0/88523984 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a54066590 msgr2=0x7f1a54068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a54066590 0x7f1a54068a50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1a600ae000 tx=0x7f1a60008040 comp rx=0 tx=0).stop 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 -- 172.21.15.31:0/88523984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 msgr2=0x7f1a7013c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 0x7f1a7013c6f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00e8b0 tx=0x7f1a6c00ed70 comp rx=0 tx=0).stop 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 -- 172.21.15.31:0/88523984 shutdown_connections 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a54066590 0x7f1a54068a50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a70161180 0x7f1a7015cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a70146820 0x7f1a70141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 --2- 172.21.15.31:0/88523984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a700b4210 0x7f1a7013c6f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 -- 172.21.15.31:0/88523984 >> 172.21.15.31:0/88523984 conn(0x7f1a700b2f10 msgr2=0x7f1a7006e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 -- 172.21.15.31:0/88523984 shutdown_connections 2023-11-26T07:22:15.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.773+0000 7f1a776c9640 1 -- 172.21.15.31:0/88523984 wait complete. 2023-11-26T07:22:15.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2023-11-26T07:22:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f5452a640 1 Processor -- start 2023-11-26T07:22:15.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f5452a640 1 -- start start 2023-11-26T07:22:15.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f5452a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f4c0ac630 0x7f8f4c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f5452a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8f4c070c20 0x7f8f4c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f5452a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c145060 con 0x7f8f4c144c80 2023-11-26T07:22:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f5452a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4c071fd0 con 0x7f8f4c070c20 2023-11-26T07:22:15.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f5452a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c071450 con 0x7f8f4c0ac630 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f52ac7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8f4c070c20 0x7f8f4c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f522c6640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8f4c144c80 0x7f8f4c0abf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36588/0 (socket says 172.21.15.31:36588) 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f522c6640 1 -- 172.21.15.31:0/317639059 learned_addr learned my addr 172.21.15.31:0/317639059 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f52ac7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8f4c070c20 0x7f8f4c06c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49370/0 (socket says 172.21.15.31:49370) 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.933+0000 7f8f51ac5640 1 --2- 172.21.15.31:0/317639059 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f4c0ac630 0x7f8f4c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f52ac7640 1 -- 172.21.15.31:0/317639059 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f4c0ac630 msgr2=0x7f8f4c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f52ac7640 1 --2- 172.21.15.31:0/317639059 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f4c0ac630 0x7f8f4c0706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f52ac7640 1 -- 172.21.15.31:0/317639059 >> v1:172.21.15.31:6789/0 conn(0x7f8f4c144c80 legacy=0x7f8f4c0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f52ac7640 1 -- 172.21.15.31:0/317639059 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c071730 con 0x7f8f4c070c20 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f512c4640 1 -- 172.21.15.31:0/317639059 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471407139 0 0) 0x7f8f4c145060 con 0x7f8f4c144c80 2023-11-26T07:22:15.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f51ac5640 1 --2- 172.21.15.31:0/317639059 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f4c0ac630 0x7f8f4c0706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f52ac7640 1 --2- 172.21.15.31:0/317639059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c070c20 0x7f8f4c06c910 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009d90 tx=0x7f8f4401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=603dbcd6e97c0477 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f512c4640 1 -- 172.21.15.31:0/317639059 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f440a59e0 con 0x7f8f4c070c20 2023-11-26T07:22:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f512c4640 1 -- 172.21.15.31:0/317639059 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f440a5b40 con 0x7f8f4c070c20 2023-11-26T07:22:15.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f512c4640 1 -- 172.21.15.31:0/317639059 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f440ad440 con 0x7f8f4c070c20 2023-11-26T07:22:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f5452a640 1 -- 172.21.15.31:0/317639059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c070c20 msgr2=0x7f8f4c06c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f5452a640 1 --2- 172.21.15.31:0/317639059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c070c20 0x7f8f4c06c910 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009d90 tx=0x7f8f4401c920 comp rx=0 tx=0).stop 2023-11-26T07:22:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f5452a640 1 -- 172.21.15.31:0/317639059 shutdown_connections 2023-11-26T07:22:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f5452a640 1 --2- 172.21.15.31:0/317639059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c070c20 0x7f8f4c06c910 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f5452a640 1 --2- 172.21.15.31:0/317639059 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f4c0ac630 0x7f8f4c0706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.937+0000 7f8f5452a640 1 -- 172.21.15.31:0/317639059 >> 172.21.15.31:0/317639059 conn(0x7f8f4c0b0c40 msgr2=0x7f8f4c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f5452a640 1 -- 172.21.15.31:0/317639059 shutdown_connections 2023-11-26T07:22:15.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f5452a640 1 -- 172.21.15.31:0/317639059 wait complete. 2023-11-26T07:22:15.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f5452a640 1 Processor -- start 2023-11-26T07:22:15.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f5452a640 1 -- start start 2023-11-26T07:22:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f5452a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4c070c20 0x7f8f4c15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f5452a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f4c0ac630 0x7f8f4c0b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f5452a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c144c80 0x7f8f4c13e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f5452a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c144550 con 0x7f8f4c144c80 2023-11-26T07:22:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f5452a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c09ba20 con 0x7f8f4c070c20 2023-11-26T07:22:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f5452a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c09bcc0 con 0x7f8f4c0ac630 2023-11-26T07:22:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f52ac7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c144c80 0x7f8f4c13e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f52ac7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c144c80 0x7f8f4c13e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49384/0 (socket says 172.21.15.31:49384) 2023-11-26T07:22:15.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f52ac7640 1 -- 172.21.15.31:0/2016374756 learned_addr learned my addr 172.21.15.31:0/2016374756 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:15.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f52ac7640 1 -- 172.21.15.31:0/2016374756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f4c0ac630 msgr2=0x7f8f4c0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f52ac7640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f4c0ac630 0x7f8f4c0b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f52ac7640 1 -- 172.21.15.31:0/2016374756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4c070c20 msgr2=0x7f8f4c15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:15.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f522c6640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4c070c20 0x7f8f4c15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f52ac7640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4c070c20 0x7f8f4c15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:15.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f52ac7640 1 -- 172.21.15.31:0/2016374756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f44009a70 con 0x7f8f4c144c80 2023-11-26T07:22:15.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f522c6640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4c070c20 0x7f8f4c15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:15.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f52ac7640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c144c80 0x7f8f4c13e310 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8f440b0040 tx=0x7f8f440196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.941+0000 7f8f42ffd640 1 -- 172.21.15.31:0/2016374756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f440ade40 con 0x7f8f4c144c80 2023-11-26T07:22:15.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.945+0000 7f8f42ffd640 1 -- 172.21.15.31:0/2016374756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f44009450 con 0x7f8f4c144c80 2023-11-26T07:22:15.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.945+0000 7f8f42ffd640 1 -- 172.21.15.31:0/2016374756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f440b9690 con 0x7f8f4c144c80 2023-11-26T07:22:15.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.945+0000 7f8f5452a640 1 -- 172.21.15.31:0/2016374756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c09bf60 con 0x7f8f4c144c80 2023-11-26T07:22:15.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.945+0000 7f8f5452a640 1 -- 172.21.15.31:0/2016374756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c09c520 con 0x7f8f4c144c80 2023-11-26T07:22:15.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.945+0000 7f8f5452a640 1 -- 172.21.15.31:0/2016374756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4c09c710 con 0x7f8f4c144c80 2023-11-26T07:22:15.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.945+0000 7f8f42ffd640 1 -- 172.21.15.31:0/2016374756 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f8f440b97f0 con 0x7f8f4c144c80 2023-11-26T07:22:15.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.945+0000 7f8f42ffd640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8f30066590 0x7f8f30068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:15.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.945+0000 7f8f42ffd640 1 -- 172.21.15.31:0/2016374756 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f8f4411f7f0 con 0x7f8f4c144c80 2023-11-26T07:22:15.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.945+0000 7f8f522c6640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8f30066590 0x7f8f30068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:15.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.945+0000 7f8f522c6640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8f30066590 0x7f8f30068a50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f48000f70 tx=0x7f8f48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:15.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:15.949+0000 7f8f42ffd640 1 -- 172.21.15.31:0/2016374756 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8f440eccb0 con 0x7f8f4c144c80 2023-11-26T07:22:16.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:16.165+0000 7f8f5452a640 1 -- 172.21.15.31:0/2016374756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7f8f4c04e850 con 0x7f8f4c144c80 2023-11-26T07:22:17.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.089+0000 7f8f42ffd640 1 -- 172.21.15.31:0/2016374756 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v257) v1 ==== 149+0+0 (secure 0 0 0) 0x7f8f4411f020 con 0x7f8f4c144c80 2023-11-26T07:22:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 -- 172.21.15.31:0/2016374756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8f30066590 msgr2=0x7f8f30068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8f30066590 0x7f8f30068a50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f48000f70 tx=0x7f8f48009040 comp rx=0 tx=0).stop 2023-11-26T07:22:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 -- 172.21.15.31:0/2016374756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c144c80 msgr2=0x7f8f4c13e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c144c80 0x7f8f4c13e310 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f8f440b0040 tx=0x7f8f440196c0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 -- 172.21.15.31:0/2016374756 shutdown_connections 2023-11-26T07:22:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8f30066590 0x7f8f30068a50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f4c144c80 0x7f8f4c13e310 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8f4c0ac630 0x7f8f4c0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 --2- 172.21.15.31:0/2016374756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f4c070c20 0x7f8f4c15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 -- 172.21.15.31:0/2016374756 >> 172.21.15.31:0/2016374756 conn(0x7f8f4c0b0c40 msgr2=0x7f8f4c13c190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 -- 172.21.15.31:0/2016374756 shutdown_connections 2023-11-26T07:22:17.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.093+0000 7f8f5452a640 1 -- 172.21.15.31:0/2016374756 wait complete. 2023-11-26T07:22:17.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:22:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.265+0000 7f2fa545a640 1 Processor -- start 2023-11-26T07:22:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.265+0000 7f2fa545a640 1 -- start start 2023-11-26T07:22:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2fa545a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2fa00cdfb0 0x7f2fa00abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2fa545a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fa00cbd50 0x7f2fa00cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2fa545a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa00b2710 con 0x7f2fa015a710 2023-11-26T07:22:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2fa545a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa00cca80 con 0x7f2fa00cbd50 2023-11-26T07:22:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2fa545a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa00b29f0 con 0x7f2fa00cdfb0 2023-11-26T07:22:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2f9f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fa00cbd50 0x7f2fa00cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2f97fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2fa015a710 0x7f2fa00ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36590/0 (socket says 172.21.15.31:36590) 2023-11-26T07:22:17.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2f97fff640 1 -- 172.21.15.31:0/4249220400 learned_addr learned my addr 172.21.15.31:0/4249220400 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2f9effd640 1 --2- 172.21.15.31:0/4249220400 >> v2:172.21.15.97:3300/0 conn(0x7f2fa00cdfb0 0x7f2fa00abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2f9f7fe640 1 -- 172.21.15.31:0/4249220400 >> v2:172.21.15.97:3300/0 conn(0x7f2fa00cdfb0 msgr2=0x7f2fa00abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2f9f7fe640 1 --2- 172.21.15.31:0/4249220400 >> v2:172.21.15.97:3300/0 conn(0x7f2fa00cdfb0 0x7f2fa00abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2f9f7fe640 1 -- 172.21.15.31:0/4249220400 >> v1:172.21.15.31:6789/0 conn(0x7f2fa015a710 legacy=0x7f2fa00ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.269+0000 7f2f9f7fe640 1 -- 172.21.15.31:0/4249220400 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00b2cd0 con 0x7f2fa00cbd50 2023-11-26T07:22:17.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2f9f7fe640 1 --2- 172.21.15.31:0/4249220400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 0x7f2fa00cc1e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2fa006c270 tx=0x7f2f9801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883b38052ecb17ba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2f9e7fc640 1 -- 172.21.15.31:0/4249220400 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f980a59c0 con 0x7f2fa00cbd50 2023-11-26T07:22:17.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2f9e7fc640 1 -- 172.21.15.31:0/4249220400 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f980a5b20 con 0x7f2fa00cbd50 2023-11-26T07:22:17.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2f9e7fc640 1 -- 172.21.15.31:0/4249220400 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f980ad440 con 0x7f2fa00cbd50 2023-11-26T07:22:17.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 -- 172.21.15.31:0/4249220400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 msgr2=0x7f2fa00cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 --2- 172.21.15.31:0/4249220400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 0x7f2fa00cc1e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2fa006c270 tx=0x7f2f9801c8e0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 -- 172.21.15.31:0/4249220400 shutdown_connections 2023-11-26T07:22:17.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 --2- 172.21.15.31:0/4249220400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 0x7f2fa00cc1e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 --2- 172.21.15.31:0/4249220400 >> v2:172.21.15.97:3300/0 conn(0x7f2fa00cdfb0 0x7f2fa00abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 -- 172.21.15.31:0/4249220400 >> 172.21.15.31:0/4249220400 conn(0x7f2fa00b0cb0 msgr2=0x7f2fa00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:17.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 -- 172.21.15.31:0/4249220400 shutdown_connections 2023-11-26T07:22:17.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 -- 172.21.15.31:0/4249220400 wait complete. 2023-11-26T07:22:17.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 Processor -- start 2023-11-26T07:22:17.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 -- start start 2023-11-26T07:22:17.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 0x7f2fa0152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fa00cdfb0 0x7f2fa0154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fa015a710 0x7f2fa0159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa01532b0 con 0x7f2fa00cbd50 2023-11-26T07:22:17.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa015be00 con 0x7f2fa00cdfb0 2023-11-26T07:22:17.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.273+0000 7f2fa545a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa015c0e0 con 0x7f2fa015a710 2023-11-26T07:22:17.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f9effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 0x7f2fa0152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f9effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 0x7f2fa0152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49396/0 (socket says 172.21.15.31:49396) 2023-11-26T07:22:17.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f9effd640 1 -- 172.21.15.31:0/756867958 learned_addr learned my addr 172.21.15.31:0/756867958 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:17.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f9f7fe640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fa015a710 0x7f2fa0159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f97fff640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fa00cdfb0 0x7f2fa0154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f9effd640 1 -- 172.21.15.31:0/756867958 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fa015a710 msgr2=0x7f2fa0159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f9effd640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fa015a710 0x7f2fa0159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f9effd640 1 -- 172.21.15.31:0/756867958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fa00cdfb0 msgr2=0x7f2fa0154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f9effd640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fa00cdfb0 0x7f2fa0154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f9effd640 1 -- 172.21.15.31:0/756867958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f98009a70 con 0x7f2fa00cbd50 2023-11-26T07:22:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f9effd640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 0x7f2fa0152d70 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000e930 tx=0x7f2f9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f977fe640 1 -- 172.21.15.31:0/756867958 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90009690 con 0x7f2fa00cbd50 2023-11-26T07:22:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f977fe640 1 -- 172.21.15.31:0/756867958 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9000ca00 con 0x7f2fa00cbd50 2023-11-26T07:22:17.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2fa545a640 1 -- 172.21.15.31:0/756867958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa015c3c0 con 0x7f2fa00cbd50 2023-11-26T07:22:17.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f977fe640 1 -- 172.21.15.31:0/756867958 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9000cd70 con 0x7f2fa00cbd50 2023-11-26T07:22:17.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2fa545a640 1 -- 172.21.15.31:0/756867958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa015c980 con 0x7f2fa00cbd50 2023-11-26T07:22:17.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2fa545a640 1 -- 172.21.15.31:0/756867958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa00671f0 con 0x7f2fa00cbd50 2023-11-26T07:22:17.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f977fe640 1 -- 172.21.15.31:0/756867958 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f2f90013470 con 0x7f2fa00cbd50 2023-11-26T07:22:17.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.277+0000 7f2f977fe640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2f80066590 0x7f2f80068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.281+0000 7f2f977fe640 1 -- 172.21.15.31:0/756867958 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f2f900842a0 con 0x7f2fa00cbd50 2023-11-26T07:22:17.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.281+0000 7f2f97fff640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2f80066590 0x7f2f80068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.281+0000 7f2f97fff640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2f80066590 0x7f2f80068a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2fa01645e0 tx=0x7f2f88009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.285+0000 7f2f977fe640 1 -- 172.21.15.31:0/756867958 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2f90051700 con 0x7f2fa00cbd50 2023-11-26T07:22:17.442 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.437+0000 7f2fa545a640 1 -- 172.21.15.31:0/756867958 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2fa006b460 con 0x7f2fa00cbd50 2023-11-26T07:22:17.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.441+0000 7f2f977fe640 1 -- 172.21.15.31:0/756867958 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2f90054d20 con 0x7f2fa00cbd50 2023-11-26T07:22:17.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 -- 172.21.15.31:0/756867958 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2f80066590 msgr2=0x7f2f80068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2f80066590 0x7f2f80068a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2fa01645e0 tx=0x7f2f88009040 comp rx=0 tx=0).stop 2023-11-26T07:22:17.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 -- 172.21.15.31:0/756867958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 msgr2=0x7f2fa0152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.448 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 0x7f2fa0152d70 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000e930 tx=0x7f2f9000edf0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 -- 172.21.15.31:0/756867958 shutdown_connections 2023-11-26T07:22:17.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2f80066590 0x7f2f80068a50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2fa015a710 0x7f2fa0159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2fa00cdfb0 0x7f2fa0154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 --2- 172.21.15.31:0/756867958 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fa00cbd50 0x7f2fa0152d70 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 -- 172.21.15.31:0/756867958 >> 172.21.15.31:0/756867958 conn(0x7f2fa00b0cb0 msgr2=0x7f2fa015dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:17.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 -- 172.21.15.31:0/756867958 shutdown_connections 2023-11-26T07:22:17.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.445+0000 7f2fa545a640 1 -- 172.21.15.31:0/756867958 wait complete. 2023-11-26T07:22:17.467 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:22:17.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:17.545 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:17.628 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:22:17.629 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:22:17.629 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:17.638 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:22:17.638 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:17.638 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:17.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:22:17.648 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:22:17.648 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:17.690 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:22:17.690 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:17.691 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:17.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:22:17.703 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:22:17.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:17.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e869a1640 1 Processor -- start 2023-11-26T07:22:17.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e869a1640 1 -- start start 2023-11-26T07:22:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e869a1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e80160d00 0x7f7e800d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e869a1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e800d3e40 0x7f7e800cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e869a1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e800ce1f0 con 0x7f7e801613a0 2023-11-26T07:22:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e869a1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e800cea70 con 0x7f7e800d3e40 2023-11-26T07:22:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e869a1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800d2380 con 0x7f7e80160d00 2023-11-26T07:22:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e7f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e800d3e40 0x7f7e800cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e7f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7e800d3e40 0x7f7e800cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49412/0 (socket says 172.21.15.31:49412) 2023-11-26T07:22:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e7f7fe640 1 -- 172.21.15.31:0/2254602297 learned_addr learned my addr 172.21.15.31:0/2254602297 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e7ffff640 1 --2- 172.21.15.31:0/2254602297 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e80160d00 0x7f7e800d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e7f7fe640 1 -- 172.21.15.31:0/2254602297 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e80160d00 msgr2=0x7f7e800d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:17.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e7f7fe640 1 --2- 172.21.15.31:0/2254602297 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e80160d00 0x7f7e800d1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e7f7fe640 1 -- 172.21.15.31:0/2254602297 >> v1:172.21.15.97:6789/0 conn(0x7f7e801613a0 legacy=0x7f7e8006b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.873+0000 7f7e7f7fe640 1 -- 172.21.15.31:0/2254602297 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e800ce420 con 0x7f7e800d3e40 2023-11-26T07:22:17.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e7f7fe640 1 --2- 172.21.15.31:0/2254602297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 0x7f7e800cef00 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009870 tx=0x7f7e7401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65a5cd713a43719 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e7effd640 1 -- 172.21.15.31:0/2254602297 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e740a8a30 con 0x7f7e800d3e40 2023-11-26T07:22:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e7effd640 1 -- 172.21.15.31:0/2254602297 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e740a8b90 con 0x7f7e800d3e40 2023-11-26T07:22:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e7effd640 1 -- 172.21.15.31:0/2254602297 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e740b0440 con 0x7f7e800d3e40 2023-11-26T07:22:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e869a1640 1 -- 172.21.15.31:0/2254602297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 msgr2=0x7f7e800cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e869a1640 1 --2- 172.21.15.31:0/2254602297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 0x7f7e800cef00 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009870 tx=0x7f7e7401ed30 comp rx=0 tx=0).stop 2023-11-26T07:22:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e869a1640 1 -- 172.21.15.31:0/2254602297 shutdown_connections 2023-11-26T07:22:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e869a1640 1 --2- 172.21.15.31:0/2254602297 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 0x7f7e800cef00 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e869a1640 1 --2- 172.21.15.31:0/2254602297 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e80160d00 0x7f7e800d1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e869a1640 1 -- 172.21.15.31:0/2254602297 >> 172.21.15.31:0/2254602297 conn(0x7f7e8008afc0 msgr2=0x7f7e800d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e869a1640 1 -- 172.21.15.31:0/2254602297 shutdown_connections 2023-11-26T07:22:17.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.877+0000 7f7e869a1640 1 -- 172.21.15.31:0/2254602297 wait complete. 2023-11-26T07:22:17.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e869a1640 1 Processor -- start 2023-11-26T07:22:17.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e869a1640 1 -- start start 2023-11-26T07:22:17.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e869a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 0x7f7e80151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e869a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e80160d00 0x7f7e800c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e869a1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e801613a0 0x7f7e800c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e869a1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800b6a40 con 0x7f7e800d3e40 2023-11-26T07:22:17.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e869a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800b5ae0 con 0x7f7e80160d00 2023-11-26T07:22:17.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e869a1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e800b5dc0 con 0x7f7e801613a0 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 0x7f7e80151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 0x7f7e80151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49424/0 (socket says 172.21.15.31:49424) 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7ffff640 1 -- 172.21.15.31:0/3677065748 learned_addr learned my addr 172.21.15.31:0/3677065748 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7f7fe640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e80160d00 0x7f7e800c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e84f3e640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e801613a0 0x7f7e800c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7ffff640 1 -- 172.21.15.31:0/3677065748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e801613a0 msgr2=0x7f7e800c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7ffff640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e801613a0 0x7f7e800c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7ffff640 1 -- 172.21.15.31:0/3677065748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e80160d00 msgr2=0x7f7e800c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7ffff640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e80160d00 0x7f7e800c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7ffff640 1 -- 172.21.15.31:0/3677065748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e740094d0 con 0x7f7e800d3e40 2023-11-26T07:22:17.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e84f3e640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e801613a0 0x7f7e800c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7ffff640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 0x7f7e80151a80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000d800 tx=0x7f7e7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7cff9640 1 -- 172.21.15.31:0/3677065748 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000b6c0 con 0x7f7e800d3e40 2023-11-26T07:22:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7cff9640 1 -- 172.21.15.31:0/3677065748 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7000b860 con 0x7f7e800d3e40 2023-11-26T07:22:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e7cff9640 1 -- 172.21.15.31:0/3677065748 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000bc10 con 0x7f7e800d3e40 2023-11-26T07:22:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e869a1640 1 -- 172.21.15.31:0/3677065748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e800b6380 con 0x7f7e800d3e40 2023-11-26T07:22:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.881+0000 7f7e869a1640 1 -- 172.21.15.31:0/3677065748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e80151fc0 con 0x7f7e800d3e40 2023-11-26T07:22:17.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.885+0000 7f7e869a1640 1 -- 172.21.15.31:0/3677065748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8004e850 con 0x7f7e800d3e40 2023-11-26T07:22:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.885+0000 7f7e7cff9640 1 -- 172.21.15.31:0/3677065748 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f7e7001e460 con 0x7f7e800d3e40 2023-11-26T07:22:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.885+0000 7f7e7cff9640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7e6c066590 0x7f7e6c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.885+0000 7f7e7cff9640 1 -- 172.21.15.31:0/3677065748 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f7e70084520 con 0x7f7e800d3e40 2023-11-26T07:22:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.885+0000 7f7e7f7fe640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7e6c066590 0x7f7e6c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:17.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.885+0000 7f7e7f7fe640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7e6c066590 0x7f7e6c068a50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7e740b2000 tx=0x7f7e74009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:17.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:17.889+0000 7f7e7cff9640 1 -- 172.21.15.31:0/3677065748 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7e700518c0 con 0x7f7e800d3e40 2023-11-26T07:22:18.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 -- 172.21.15.31:0/3677065748 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e8008b4a0 con 0x7f7e800d3e40 2023-11-26T07:22:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e7cff9640 1 -- 172.21.15.31:0/3677065748 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f7e70054ee0 con 0x7f7e800d3e40 2023-11-26T07:22:18.083 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:18.083 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/2851182363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":2851182363},{"type":"v1","addr":"172.21.15.31:6835","nonce":2851182363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":5822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/2056572548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6835","nonce":2056572548},{"type":"v1","addr":"172.21.15.97:6837","nonce":2056572548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6836/1012539369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1012539369},{"type":"v1","addr":"172.21.15.97:6836","nonce":1012539369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":7017,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/2563501152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":2563501152},{"type":"v1","addr":"172.21.15.31:6834","nonce":2563501152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[]} 2023-11-26T07:22:18.083 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 50 2023-11-26T07:22:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 -- 172.21.15.31:0/3677065748 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7e6c066590 msgr2=0x7f7e6c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7e6c066590 0x7f7e6c068a50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7e740b2000 tx=0x7f7e74009040 comp rx=0 tx=0).stop 2023-11-26T07:22:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 -- 172.21.15.31:0/3677065748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 msgr2=0x7f7e80151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 0x7f7e80151a80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000d800 tx=0x7f7e7000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 -- 172.21.15.31:0/3677065748 shutdown_connections 2023-11-26T07:22:18.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7e6c066590 0x7f7e6c068a50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7e801613a0 0x7f7e800c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e80160d00 0x7f7e800c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 --2- 172.21.15.31:0/3677065748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7e800d3e40 0x7f7e80151a80 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 -- 172.21.15.31:0/3677065748 >> 172.21.15.31:0/3677065748 conn(0x7f7e8008afc0 msgr2=0x7f7e800cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:18.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 -- 172.21.15.31:0/3677065748 shutdown_connections 2023-11-26T07:22:18.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.081+0000 7f7e869a1640 1 -- 172.21.15.31:0/3677065748 wait complete. 2023-11-26T07:22:18.105 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:22:18.105 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:22:18.105 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:22:18.105 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:18.105 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:22:18.105 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:22:18.106 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:22:18.106 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:18.106 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:22:18.106 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:22:18.106 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:22:18.106 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:18.107 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:22:18.107 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:22:18.107 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:22:18.107 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:18.119 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:22:18.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:22:18.120 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:22:18.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:22:18.121 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:22:18.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:22:18.123 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:22:18.123 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:22:18.168 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:22:18.169 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:18.171 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:22:18.171 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:18.172 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:22:18.172 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:18.174 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:22:18.175 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:18.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:22:18.257 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-11-26T07:22:18.260 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-11-26T07:22:18.263 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2023-11-26T07:22:18.263 INFO:tasks.ceph.mds.d.smithi097.stdout:starting mds.d at 2023-11-26T07:22:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f47191f5640 1 Processor -- start 2023-11-26T07:22:18.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f47191f5640 1 -- start start 2023-11-26T07:22:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f47191f5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4714160d00 0x7f47140d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f47191f5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f47140d3e40 0x7f47140cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f47191f5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47140ce1f0 con 0x7f47141613a0 2023-11-26T07:22:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f47191f5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47140cea70 con 0x7f4714160d00 2023-11-26T07:22:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f47191f5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47140d2380 con 0x7f47140d3e40 2023-11-26T07:22:18.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f4712d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4714160d00 0x7f47140d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f4712d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4714160d00 0x7f47140d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49466/0 (socket says 172.21.15.31:49466) 2023-11-26T07:22:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f4712d9d640 1 -- 172.21.15.31:0/2013369172 learned_addr learned my addr 172.21.15.31:0/2013369172 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f4712d9d640 1 -- 172.21.15.31:0/2013369172 >> v2:172.21.15.97:3300/0 conn(0x7f47140d3e40 msgr2=0x7f47140cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f4712d9d640 1 --2- 172.21.15.31:0/2013369172 >> v2:172.21.15.97:3300/0 conn(0x7f47140d3e40 0x7f47140cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f4712d9d640 1 -- 172.21.15.31:0/2013369172 >> v1:172.21.15.97:6789/0 conn(0x7f47141613a0 legacy=0x7f47141617c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:18.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.321+0000 7f4712d9d640 1 -- 172.21.15.31:0/2013369172 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47140ce420 con 0x7f4714160d00 2023-11-26T07:22:18.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f4712d9d640 1 --2- 172.21.15.31:0/2013369172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4714160d00 0x7f47140d1da0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f47140cfec0 tx=0x7f470401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc3d5b346f96f34 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f4711d9b640 1 -- 172.21.15.31:0/2013369172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040a89b0 con 0x7f4714160d00 2023-11-26T07:22:18.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f4711d9b640 1 -- 172.21.15.31:0/2013369172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47040a8b10 con 0x7f4714160d00 2023-11-26T07:22:18.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f4711d9b640 1 -- 172.21.15.31:0/2013369172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47040a8ea0 con 0x7f4714160d00 2023-11-26T07:22:18.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f47191f5640 1 -- 172.21.15.31:0/2013369172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4714160d00 msgr2=0x7f47140d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f47191f5640 1 --2- 172.21.15.31:0/2013369172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4714160d00 0x7f47140d1da0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f47140cfec0 tx=0x7f470401ecb0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f47191f5640 1 -- 172.21.15.31:0/2013369172 shutdown_connections 2023-11-26T07:22:18.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f47191f5640 1 --2- 172.21.15.31:0/2013369172 >> v2:172.21.15.97:3300/0 conn(0x7f47140d3e40 0x7f47140cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f47191f5640 1 --2- 172.21.15.31:0/2013369172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4714160d00 0x7f47140d1da0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f47191f5640 1 -- 172.21.15.31:0/2013369172 >> 172.21.15.31:0/2013369172 conn(0x7f471408afc0 msgr2=0x7f47140d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:18.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f47191f5640 1 -- 172.21.15.31:0/2013369172 shutdown_connections 2023-11-26T07:22:18.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.325+0000 7f47191f5640 1 -- 172.21.15.31:0/2013369172 wait complete. 2023-11-26T07:22:18.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f47191f5640 1 Processor -- start 2023-11-26T07:22:18.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f47191f5640 1 -- start start 2023-11-26T07:22:18.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f47191f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47140d3e40 0x7f471415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f47191f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4714160d00 0x7f47141e71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f47191f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47141613a0 0x7f47140b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f47191f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471414b7d0 con 0x7f47141613a0 2023-11-26T07:22:18.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f47191f5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471414abc0 con 0x7f4714160d00 2023-11-26T07:22:18.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f47191f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471414aea0 con 0x7f47140d3e40 2023-11-26T07:22:18.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47141613a0 0x7f47140b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47141613a0 0x7f47140b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49480/0 (socket says 172.21.15.31:49480) 2023-11-26T07:22:18.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471359e640 1 -- 172.21.15.31:0/276124871 learned_addr learned my addr 172.21.15.31:0/276124871 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:18.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471259c640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4714160d00 0x7f47141e71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f4712d9d640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47140d3e40 0x7f471415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471359e640 1 -- 172.21.15.31:0/276124871 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47140d3e40 msgr2=0x7f471415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471359e640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47140d3e40 0x7f471415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471359e640 1 -- 172.21.15.31:0/276124871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4714160d00 msgr2=0x7f47141e71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:18.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471359e640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4714160d00 0x7f47141e71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:18.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471359e640 1 -- 172.21.15.31:0/276124871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040094d0 con 0x7f47141613a0 2023-11-26T07:22:18.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471259c640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4714160d00 0x7f47141e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:18.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f471359e640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47141613a0 0x7f47140b3170 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f470c00c780 tx=0x7f470c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f46fb7fe640 1 -- 172.21.15.31:0/276124871 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c003d90 con 0x7f47141613a0 2023-11-26T07:22:18.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f46fb7fe640 1 -- 172.21.15.31:0/276124871 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c00ce30 con 0x7f47141613a0 2023-11-26T07:22:18.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f46fb7fe640 1 -- 172.21.15.31:0/276124871 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c005290 con 0x7f47141613a0 2023-11-26T07:22:18.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.329+0000 7f47191f5640 1 -- 172.21.15.31:0/276124871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471414b180 con 0x7f47141613a0 2023-11-26T07:22:18.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.333+0000 7f47191f5640 1 -- 172.21.15.31:0/276124871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471415db10 con 0x7f47141613a0 2023-11-26T07:22:18.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.333+0000 7f47191f5640 1 -- 172.21.15.31:0/276124871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471414b980 con 0x7f47141613a0 2023-11-26T07:22:18.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.333+0000 7f46fb7fe640 1 -- 172.21.15.31:0/276124871 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f470c0053f0 con 0x7f47141613a0 2023-11-26T07:22:18.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.333+0000 7f46fb7fe640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4700066590 0x7f4700068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:18.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.333+0000 7f46fb7fe640 1 -- 172.21.15.31:0/276124871 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f470c083290 con 0x7f47141613a0 2023-11-26T07:22:18.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.333+0000 7f4712d9d640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4700066590 0x7f4700068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:18.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.333+0000 7f4712d9d640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4700066590 0x7f4700068a50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47040b2000 tx=0x7f4704008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:18.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.337+0000 7f46fb7fe640 1 -- 172.21.15.31:0/276124871 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f471414b980 con 0x7f47141613a0 2023-11-26T07:22:18.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:18.505+0000 7f47191f5640 1 -- 172.21.15.31:0/276124871 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f471404e850 con 0x7f47141613a0 2023-11-26T07:22:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.105+0000 7f46fb7fe640 1 -- 172.21.15.31:0/276124871 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) v1 ==== 97+0+0 (secure 0 0 0) 0x7f470c050780 con 0x7f47141613a0 2023-11-26T07:22:19.110 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-11-26T07:22:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 -- 172.21.15.31:0/276124871 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4700066590 msgr2=0x7f4700068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4700066590 0x7f4700068a50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f47040b2000 tx=0x7f4704008040 comp rx=0 tx=0).stop 2023-11-26T07:22:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 -- 172.21.15.31:0/276124871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47141613a0 msgr2=0x7f47140b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47141613a0 0x7f47140b3170 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f470c00c780 tx=0x7f470c00cc40 comp rx=0 tx=0).stop 2023-11-26T07:22:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 -- 172.21.15.31:0/276124871 shutdown_connections 2023-11-26T07:22:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4700066590 0x7f4700068a50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47141613a0 0x7f47140b3170 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4714160d00 0x7f47141e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 --2- 172.21.15.31:0/276124871 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f47140d3e40 0x7f471415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 -- 172.21.15.31:0/276124871 >> 172.21.15.31:0/276124871 conn(0x7f471408afc0 msgr2=0x7f471415fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:19.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 -- 172.21.15.31:0/276124871 shutdown_connections 2023-11-26T07:22:19.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.109+0000 7f47191f5640 1 -- 172.21.15.31:0/276124871 wait complete. 2023-11-26T07:22:19.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:22:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fc22fe640 1 Processor -- start 2023-11-26T07:22:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fc22fe640 1 -- start start 2023-11-26T07:22:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fc22fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc15d7b0 0x7f9fbc0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fc22fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9fbc0cead0 0x7f9fbc0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fc22fe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fbc0cefd0 con 0x7f9fbc0d1eb0 2023-11-26T07:22:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fc22fe640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fbc0d2340 con 0x7f9fbc0cead0 2023-11-26T07:22:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fc22fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0cf2b0 con 0x7f9fbc15d7b0 2023-11-26T07:22:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fc089b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9fbc0d1eb0 0x7f9fbc0cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36618/0 (socket says 172.21.15.31:36618) 2023-11-26T07:22:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fc089b640 1 -- 172.21.15.31:0/36471535 learned_addr learned my addr 172.21.15.31:0/36471535 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fbbfff640 1 --2- 172.21.15.31:0/36471535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc15d7b0 0x7f9fbc0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fbb7fe640 1 --2- 172.21.15.31:0/36471535 >> v2:172.21.15.97:3300/0 conn(0x7f9fbc0cead0 0x7f9fbc0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fbaffd640 1 -- 172.21.15.31:0/36471535 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087090995 0 0) 0x7f9fbc0cefd0 con 0x7f9fbc0d1eb0 2023-11-26T07:22:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fbaffd640 1 -- 172.21.15.31:0/36471535 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa4003530 con 0x7f9fbc0d1eb0 2023-11-26T07:22:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fbbfff640 1 -- 172.21.15.31:0/36471535 >> v2:172.21.15.97:3300/0 conn(0x7f9fbc0cead0 msgr2=0x7f9fbc0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fbbfff640 1 --2- 172.21.15.31:0/36471535 >> v2:172.21.15.97:3300/0 conn(0x7f9fbc0cead0 0x7f9fbc0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fbbfff640 1 -- 172.21.15.31:0/36471535 >> v1:172.21.15.31:6789/0 conn(0x7f9fbc0d1eb0 legacy=0x7f9fbc0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.285+0000 7f9fbbfff640 1 -- 172.21.15.31:0/36471535 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc06b3f0 con 0x7f9fbc15d7b0 2023-11-26T07:22:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fbbfff640 1 --2- 172.21.15.31:0/36471535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc15d7b0 0x7f9fbc0d10d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0097c0 tx=0x7f9fac01c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88c055e1f38dd125 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fbaffd640 1 -- 172.21.15.31:0/36471535 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0ab600 con 0x7f9fbc15d7b0 2023-11-26T07:22:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fbaffd640 1 -- 172.21.15.31:0/36471535 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac0a7050 con 0x7f9fbc15d7b0 2023-11-26T07:22:19.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fbaffd640 1 -- 172.21.15.31:0/36471535 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0ab960 con 0x7f9fbc15d7b0 2023-11-26T07:22:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fc22fe640 1 -- 172.21.15.31:0/36471535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc15d7b0 msgr2=0x7f9fbc0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fc22fe640 1 --2- 172.21.15.31:0/36471535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc15d7b0 0x7f9fbc0d10d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0097c0 tx=0x7f9fac01c330 comp rx=0 tx=0).stop 2023-11-26T07:22:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fc22fe640 1 -- 172.21.15.31:0/36471535 shutdown_connections 2023-11-26T07:22:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fc22fe640 1 --2- 172.21.15.31:0/36471535 >> v2:172.21.15.97:3300/0 conn(0x7f9fbc0cead0 0x7f9fbc0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fc22fe640 1 --2- 172.21.15.31:0/36471535 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc15d7b0 0x7f9fbc0d10d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fc22fe640 1 -- 172.21.15.31:0/36471535 >> 172.21.15.31:0/36471535 conn(0x7f9fbc0d33b0 msgr2=0x7f9fbc0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:19.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fc22fe640 1 -- 172.21.15.31:0/36471535 shutdown_connections 2023-11-26T07:22:19.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.289+0000 7f9fc22fe640 1 -- 172.21.15.31:0/36471535 wait complete. 2023-11-26T07:22:19.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 Processor -- start 2023-11-26T07:22:19.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 -- start start 2023-11-26T07:22:19.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fbc0cead0 0x7f9fbc0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc0d1eb0 0x7f9fbc15b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fbc15d7b0 0x7f9fbc0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0cbb30 con 0x7f9fbc0cead0 2023-11-26T07:22:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0cbdd0 con 0x7f9fbc15d7b0 2023-11-26T07:22:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc0cc0b0 con 0x7f9fbc0d1eb0 2023-11-26T07:22:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fbbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fbc0cead0 0x7f9fbc0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fbbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fbc0cead0 0x7f9fbc0ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49496/0 (socket says 172.21.15.31:49496) 2023-11-26T07:22:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fbbfff640 1 -- 172.21.15.31:0/331087298 learned_addr learned my addr 172.21.15.31:0/331087298 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc089b640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fbc15d7b0 0x7f9fbc0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fbb7fe640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc0d1eb0 0x7f9fbc15b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fbbfff640 1 -- 172.21.15.31:0/331087298 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc0d1eb0 msgr2=0x7f9fbc15b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fbbfff640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc0d1eb0 0x7f9fbc15b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fbbfff640 1 -- 172.21.15.31:0/331087298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fbc15d7b0 msgr2=0x7f9fbc0cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fbbfff640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fbc15d7b0 0x7f9fbc0cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fbbfff640 1 -- 172.21.15.31:0/331087298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac009420 con 0x7f9fbc0cead0 2023-11-26T07:22:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc089b640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fbc15d7b0 0x7f9fbc0cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fbbfff640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fbc0cead0 0x7f9fbc0ccb90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9fac01a9e0 tx=0x7f9fac0190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fb8ff9640 1 -- 172.21.15.31:0/331087298 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac006e20 con 0x7f9fbc0cead0 2023-11-26T07:22:19.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fb8ff9640 1 -- 172.21.15.31:0/331087298 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac004020 con 0x7f9fbc0cead0 2023-11-26T07:22:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 -- 172.21.15.31:0/331087298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc0cc390 con 0x7f9fbc0cead0 2023-11-26T07:22:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fb8ff9640 1 -- 172.21.15.31:0/331087298 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac004390 con 0x7f9fbc0cead0 2023-11-26T07:22:19.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 -- 172.21.15.31:0/331087298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc1567a0 con 0x7f9fbc0cead0 2023-11-26T07:22:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.293+0000 7f9fc22fe640 1 -- 172.21.15.31:0/331087298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc04e850 con 0x7f9fbc0cead0 2023-11-26T07:22:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.297+0000 7f9fb8ff9640 1 -- 172.21.15.31:0/331087298 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f9fac0b2050 con 0x7f9fbc0cead0 2023-11-26T07:22:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.297+0000 7f9fb8ff9640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9fa8066590 0x7f9fa8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.297+0000 7f9fb8ff9640 1 -- 172.21.15.31:0/331087298 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9fac11de40 con 0x7f9fbc0cead0 2023-11-26T07:22:19.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.297+0000 7f9fbb7fe640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9fa8066590 0x7f9fa8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.297+0000 7f9fbb7fe640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9fa8066590 0x7f9fa8068a50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc08cf60 tx=0x7f9fb0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.301+0000 7f9fb8ff9640 1 -- 172.21.15.31:0/331087298 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9fac0eb3e0 con 0x7f9fbc0cead0 2023-11-26T07:22:19.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.517+0000 7f9fc22fe640 1 -- 172.21.15.31:0/331087298 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9fbc0b3040 con 0x7f9fbc0cead0 2023-11-26T07:22:19.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.517+0000 7f9fb8ff9640 1 -- 172.21.15.31:0/331087298 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9fac0b0020 con 0x7f9fbc0cead0 2023-11-26T07:22:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:19.521 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:22:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-11-26T07:22:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlRJmWGBAAuUAS68YFt9PaqDV8RFi+6w==", 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlIdH1JBAAmPlAQXq78ht4pJf6I7Cd5w==", 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlfuXoHxAAf7vKZr+LVZaN6HR3qiks5w==", 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlWmgkLBAAojAzX06O7kbwmj154qUGog==", 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:22:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDU8GJlc/bqEBAAueOQvSeK6MW+WOT746yQzA==", 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDX8GJlEaW6FxAA8q4rEKSJtIl/Hr2uCNSMUg==", 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-11-26T07:22:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDa8GJldV90HxAAQRgcLf8ABSU4Sfb3A1Xa4g==", 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDd8GJl2erPJhAAoC71eDwCOmskb+VCLYQyUw==", 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:22:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDj8GJllq08LxAAGuz9+Ou/ybWxdkN3PSMaEQ==", 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDm8GJlww4eNxAAjRaBq45nrNwJPlopyOhXNg==", 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDq8GJl2jxiAhAAhu2OULy84YRJYxTmShoVBg==", 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDt8GJld8HNCBAA3deA4HKTqRggbVc73Y8efg==", 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:22:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA==", 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlEBAQOhAAx5TsMN17L2fNm+j3yqUIOQ==", 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl/CN+NhAAeyry8gIOtxwLxCQb0t1eKw==", 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDR8GJlA9e7BBAATe3WYA79PgfLc5UHgKGkpw==", 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:22:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDP8GJlfT+4NhAACA0t23bQtWvdFLsB9X+X8w==", 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-11-26T07:22:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlnh/pExAA6qyPM7So2BLbxbGbquLZZA==", 2023-11-26T07:22:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:22:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:22:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlYhb/DxAAx1G3Ve23IdZ7cX9nPb2WhA==", 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:22:19.536 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:22:19.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 -- 172.21.15.31:0/331087298 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9fa8066590 msgr2=0x7f9fa8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9fa8066590 0x7f9fa8068a50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc08cf60 tx=0x7f9fb0009190 comp rx=0 tx=0).stop 2023-11-26T07:22:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 -- 172.21.15.31:0/331087298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fbc0cead0 msgr2=0x7f9fbc0ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fbc0cead0 0x7f9fbc0ccb90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9fac01a9e0 tx=0x7f9fac0190d0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 -- 172.21.15.31:0/331087298 shutdown_connections 2023-11-26T07:22:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9fa8066590 0x7f9fa8068a50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9fbc15d7b0 0x7f9fbc0cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9fbc0d1eb0 0x7f9fbc15b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 --2- 172.21.15.31:0/331087298 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9fbc0cead0 0x7f9fbc0ccb90 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 -- 172.21.15.31:0/331087298 >> 172.21.15.31:0/331087298 conn(0x7f9fbc0d33b0 msgr2=0x7f9fbc15ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 -- 172.21.15.31:0/331087298 shutdown_connections 2023-11-26T07:22:19.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.521+0000 7f9fc22fe640 1 -- 172.21.15.31:0/331087298 wait complete. 2023-11-26T07:22:19.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f3541ff5640 1 Processor -- start 2023-11-26T07:22:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f3541ff5640 1 -- start start 2023-11-26T07:22:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f3541ff5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f353c0ae3d0 0x7f353c06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f3541ff5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f353c0ae7f0 con 0x7f353c143ae0 2023-11-26T07:22:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f3541ff5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f353c06e980 con 0x7f353c06dfd0 2023-11-26T07:22:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f3541ff5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f353c070670 con 0x7f353c0ae3d0 2023-11-26T07:22:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f353bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f353c0ae3d0 0x7f353c06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f353b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f353c143ae0 0x7f353c0b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36634/0 (socket says 172.21.15.31:36634) 2023-11-26T07:22:19.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f353b7fe640 1 -- 172.21.15.31:0/1448465612 learned_addr learned my addr 172.21.15.31:0/1448465612 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:19.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f353bfff640 1 -- 172.21.15.31:0/1448465612 >> v1:172.21.15.97:6789/0 conn(0x7f353c06dfd0 legacy=0x7f353c0adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f353bfff640 1 -- 172.21.15.31:0/1448465612 >> v1:172.21.15.31:6789/0 conn(0x7f353c143ae0 legacy=0x7f353c0b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.685+0000 7f353bfff640 1 -- 172.21.15.31:0/1448465612 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353c070950 con 0x7f353c0ae3d0 2023-11-26T07:22:19.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f353bfff640 1 --2- 172.21.15.31:0/1448465612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c0ae3d0 0x7f353c06e440 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3534009d90 tx=0x7f353401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc29a8c2f56f24b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f353a7fc640 1 -- 172.21.15.31:0/1448465612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35340a59e0 con 0x7f353c0ae3d0 2023-11-26T07:22:19.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f353a7fc640 1 -- 172.21.15.31:0/1448465612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35340a5b40 con 0x7f353c0ae3d0 2023-11-26T07:22:19.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f353a7fc640 1 -- 172.21.15.31:0/1448465612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35340ad440 con 0x7f353c0ae3d0 2023-11-26T07:22:19.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f3541ff5640 1 -- 172.21.15.31:0/1448465612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c0ae3d0 msgr2=0x7f353c06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f3541ff5640 1 --2- 172.21.15.31:0/1448465612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c0ae3d0 0x7f353c06e440 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3534009d90 tx=0x7f353401c920 comp rx=0 tx=0).stop 2023-11-26T07:22:19.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f3541ff5640 1 -- 172.21.15.31:0/1448465612 shutdown_connections 2023-11-26T07:22:19.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f3541ff5640 1 --2- 172.21.15.31:0/1448465612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c0ae3d0 0x7f353c06e440 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f3541ff5640 1 -- 172.21.15.31:0/1448465612 >> 172.21.15.31:0/1448465612 conn(0x7f353c0b0c40 msgr2=0x7f353c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:19.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f3541ff5640 1 -- 172.21.15.31:0/1448465612 shutdown_connections 2023-11-26T07:22:19.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f3541ff5640 1 -- 172.21.15.31:0/1448465612 wait complete. 2023-11-26T07:22:19.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.689+0000 7f3541ff5640 1 Processor -- start 2023-11-26T07:22:19.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3541ff5640 1 -- start start 2023-11-26T07:22:19.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3541ff5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c06dfd0 0x7f353c15d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3541ff5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f353c0ae3d0 0x7f353c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3541ff5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c143ae0 0x7f353c1f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3541ff5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c159b60 con 0x7f353c06dfd0 2023-11-26T07:22:19.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3541ff5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c1320b0 con 0x7f353c143ae0 2023-11-26T07:22:19.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3541ff5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353c1314a0 con 0x7f353c0ae3d0 2023-11-26T07:22:19.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c06dfd0 0x7f353c15d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c06dfd0 0x7f353c15d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49504/0 (socket says 172.21.15.31:49504) 2023-11-26T07:22:19.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353b7fe640 1 -- 172.21.15.31:0/1906176857 learned_addr learned my addr 172.21.15.31:0/1906176857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:19.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353b7fe640 1 -- 172.21.15.31:0/1906176857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f353c0ae3d0 msgr2=0x7f353c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353bfff640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c143ae0 0x7f353c1f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353affd640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f353c0ae3d0 0x7f353c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353b7fe640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f353c0ae3d0 0x7f353c159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353b7fe640 1 -- 172.21.15.31:0/1906176857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c143ae0 msgr2=0x7f353c1f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353b7fe640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c143ae0 0x7f353c1f9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353b7fe640 1 -- 172.21.15.31:0/1906176857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3534009a70 con 0x7f353c06dfd0 2023-11-26T07:22:19.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353bfff640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c143ae0 0x7f353c1f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:19.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353affd640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f353c0ae3d0 0x7f353c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:19.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f353b7fe640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c06dfd0 0x7f353c15d250 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f352c00c400 tx=0x7f352c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3523fff640 1 -- 172.21.15.31:0/1906176857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c005cf0 con 0x7f353c06dfd0 2023-11-26T07:22:19.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3523fff640 1 -- 172.21.15.31:0/1906176857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352c005e50 con 0x7f353c06dfd0 2023-11-26T07:22:19.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3541ff5640 1 -- 172.21.15.31:0/1906176857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353c131780 con 0x7f353c06dfd0 2023-11-26T07:22:19.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3523fff640 1 -- 172.21.15.31:0/1906176857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c0072d0 con 0x7f353c06dfd0 2023-11-26T07:22:19.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3541ff5640 1 -- 172.21.15.31:0/1906176857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353c131d40 con 0x7f353c06dfd0 2023-11-26T07:22:19.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.693+0000 7f3541ff5640 1 -- 172.21.15.31:0/1906176857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353c04e850 con 0x7f353c06dfd0 2023-11-26T07:22:19.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.697+0000 7f3523fff640 1 -- 172.21.15.31:0/1906176857 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f352c007430 con 0x7f353c06dfd0 2023-11-26T07:22:19.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.697+0000 7f3523fff640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3528066590 0x7f3528068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:19.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.697+0000 7f3523fff640 1 -- 172.21.15.31:0/1906176857 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f352c083130 con 0x7f353c06dfd0 2023-11-26T07:22:19.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.697+0000 7f353affd640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3528066590 0x7f3528068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:19.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.697+0000 7f353affd640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3528066590 0x7f3528068a50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f353c1fadc0 tx=0x7f3530009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:19.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.701+0000 7f3523fff640 1 -- 172.21.15.31:0/1906176857 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f352c0508a0 con 0x7f353c06dfd0 2023-11-26T07:22:19.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.857+0000 7f3541ff5640 1 -- 172.21.15.31:0/1906176857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f353c06cc40 con 0x7f353c06dfd0 2023-11-26T07:22:19.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3523fff640 1 -- 172.21.15.31:0/1906176857 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+12774 (secure 0 0 0) 0x7f352c053ec0 con 0x7f353c06dfd0 2023-11-26T07:22:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:19.863 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":76,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:19.124006+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:19.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 -- 172.21.15.31:0/1906176857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3528066590 msgr2=0x7f3528068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3528066590 0x7f3528068a50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f353c1fadc0 tx=0x7f3530009040 comp rx=0 tx=0).stop 2023-11-26T07:22:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 -- 172.21.15.31:0/1906176857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c06dfd0 msgr2=0x7f353c15d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c06dfd0 0x7f353c15d250 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f352c00c400 tx=0x7f352c00c8c0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 -- 172.21.15.31:0/1906176857 shutdown_connections 2023-11-26T07:22:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3528066590 0x7f3528068a50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f353c143ae0 0x7f353c1f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f353c0ae3d0 0x7f353c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 --2- 172.21.15.31:0/1906176857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f353c06dfd0 0x7f353c15d250 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 -- 172.21.15.31:0/1906176857 >> 172.21.15.31:0/1906176857 conn(0x7f353c0b0c40 msgr2=0x7f353c0afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 -- 172.21.15.31:0/1906176857 shutdown_connections 2023-11-26T07:22:19.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:19.861+0000 7f3541ff5640 1 -- 172.21.15.31:0/1906176857 wait complete. 2023-11-26T07:22:19.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:19.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:20.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305edf5640 1 Processor -- start 2023-11-26T07:22:20.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305edf5640 1 -- start start 2023-11-26T07:22:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305edf5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30580ce5d0 0x7f305815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305edf5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3058159630 con 0x7f30580cb9b0 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305edf5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30580cd1d0 con 0x7f305806e3f0 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305edf5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305806e860 con 0x7f30580ce5d0 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305cb91640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f30580cb9b0 0x7f30580cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36642/0 (socket says 172.21.15.31:36642) 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305cb91640 1 -- 172.21.15.31:0/611626159 learned_addr learned my addr 172.21.15.31:0/611626159 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305d392640 1 --2- 172.21.15.31:0/611626159 >> v2:172.21.15.97:3300/0 conn(0x7f30580ce5d0 0x7f305815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f30577fe640 1 -- 172.21.15.31:0/611626159 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731593311 0 0) 0x7f3058159630 con 0x7f30580cb9b0 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f30577fe640 1 -- 172.21.15.31:0/611626159 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3048003530 con 0x7f30580cb9b0 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f30577fe640 1 -- 172.21.15.31:0/611626159 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482845193 0 0) 0x7f30580cd1d0 con 0x7f305806e3f0 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f30577fe640 1 -- 172.21.15.31:0/611626159 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3058159630 con 0x7f305806e3f0 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305d392640 1 -- 172.21.15.31:0/611626159 >> v1:172.21.15.97:6789/0 conn(0x7f305806e3f0 legacy=0x7f305815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305d392640 1 -- 172.21.15.31:0/611626159 >> v1:172.21.15.31:6789/0 conn(0x7f30580cb9b0 legacy=0x7f30580cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f30577fe640 1 -- 172.21.15.31:0/611626159 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2447492945 0 0) 0x7f3048003530 con 0x7f30580cb9b0 2023-11-26T07:22:20.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.029+0000 7f305d392640 1 -- 172.21.15.31:0/611626159 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30580cd4b0 con 0x7f30580ce5d0 2023-11-26T07:22:20.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f305d392640 1 --2- 172.21.15.31:0/611626159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30580ce5d0 0x7f305815bfd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3050009d90 tx=0x7f305001c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a3a51440c72fedd server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f30577fe640 1 -- 172.21.15.31:0/611626159 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c001680 con 0x7f30580ce5d0 2023-11-26T07:22:20.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f30577fe640 1 -- 172.21.15.31:0/611626159 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30500a64a0 con 0x7f30580ce5d0 2023-11-26T07:22:20.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f30577fe640 1 -- 172.21.15.31:0/611626159 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30500a6830 con 0x7f30580ce5d0 2023-11-26T07:22:20.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f305edf5640 1 -- 172.21.15.31:0/611626159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30580ce5d0 msgr2=0x7f305815bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f305edf5640 1 --2- 172.21.15.31:0/611626159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30580ce5d0 0x7f305815bfd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3050009d90 tx=0x7f305001c870 comp rx=0 tx=0).stop 2023-11-26T07:22:20.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f305edf5640 1 -- 172.21.15.31:0/611626159 shutdown_connections 2023-11-26T07:22:20.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f305edf5640 1 --2- 172.21.15.31:0/611626159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30580ce5d0 0x7f305815bfd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f305edf5640 1 -- 172.21.15.31:0/611626159 >> 172.21.15.31:0/611626159 conn(0x7f30580b0cb0 msgr2=0x7f30580b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:20.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f305edf5640 1 -- 172.21.15.31:0/611626159 shutdown_connections 2023-11-26T07:22:20.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.033+0000 7f305edf5640 1 -- 172.21.15.31:0/611626159 wait complete. 2023-11-26T07:22:20.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305edf5640 1 Processor -- start 2023-11-26T07:22:20.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305edf5640 1 -- start start 2023-11-26T07:22:20.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305edf5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305806e3f0 0x7f30580b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305edf5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30580cb9b0 0x7f3058150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305edf5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30580ce5d0 0x7f30581508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305edf5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30581583c0 con 0x7f30580ce5d0 2023-11-26T07:22:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305edf5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30581559a0 con 0x7f305806e3f0 2023-11-26T07:22:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305edf5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30580b0720 con 0x7f30580cb9b0 2023-11-26T07:22:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305d392640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30580ce5d0 0x7f30581508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305d392640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30580ce5d0 0x7f30581508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49512/0 (socket says 172.21.15.31:49512) 2023-11-26T07:22:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305d392640 1 -- 172.21.15.31:0/3007494359 learned_addr learned my addr 172.21.15.31:0/3007494359 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305cb91640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305806e3f0 0x7f30580b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f3057fff640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30580cb9b0 0x7f3058150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305d392640 1 -- 172.21.15.31:0/3007494359 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30580cb9b0 msgr2=0x7f3058150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305d392640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30580cb9b0 0x7f3058150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305d392640 1 -- 172.21.15.31:0/3007494359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305806e3f0 msgr2=0x7f30580b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305d392640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305806e3f0 0x7f30580b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305d392640 1 -- 172.21.15.31:0/3007494359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3050009a70 con 0x7f30580ce5d0 2023-11-26T07:22:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305d392640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30580ce5d0 0x7f30581508d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f305001a4a0 tx=0x7f3050019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f30557fa640 1 -- 172.21.15.31:0/3007494359 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30500b6070 con 0x7f30580ce5d0 2023-11-26T07:22:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f30557fa640 1 -- 172.21.15.31:0/3007494359 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3050009450 con 0x7f30580ce5d0 2023-11-26T07:22:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f30557fa640 1 -- 172.21.15.31:0/3007494359 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050003be0 con 0x7f30580ce5d0 2023-11-26T07:22:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305edf5640 1 -- 172.21.15.31:0/3007494359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3058155bf0 con 0x7f30580ce5d0 2023-11-26T07:22:20.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.037+0000 7f305edf5640 1 -- 172.21.15.31:0/3007494359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30581561b0 con 0x7f30580ce5d0 2023-11-26T07:22:20.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.041+0000 7f305edf5640 1 -- 172.21.15.31:0/3007494359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30580671f0 con 0x7f30580ce5d0 2023-11-26T07:22:20.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.041+0000 7f30557fa640 1 -- 172.21.15.31:0/3007494359 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f3050003d40 con 0x7f30580ce5d0 2023-11-26T07:22:20.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.041+0000 7f30557fa640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f303c0665c0 0x7f303c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.041+0000 7f30557fa640 1 -- 172.21.15.31:0/3007494359 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f305011e1a0 con 0x7f30580ce5d0 2023-11-26T07:22:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.041+0000 7f305cb91640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f303c0665c0 0x7f303c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.041+0000 7f305cb91640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f303c0665c0 0x7f303c068a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f304c000f70 tx=0x7f304c0016c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.045+0000 7f30557fa640 1 -- 172.21.15.31:0/3007494359 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f30500eb7f0 con 0x7f30580ce5d0 2023-11-26T07:22:20.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.205+0000 7f305edf5640 1 -- 172.21.15.31:0/3007494359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3058158500 con 0x7f30580ce5d0 2023-11-26T07:22:20.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.205+0000 7f30557fa640 1 -- 172.21.15.31:0/3007494359 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3058158500 con 0x7f30580ce5d0 2023-11-26T07:22:20.210 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:22:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 -- 172.21.15.31:0/3007494359 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f303c0665c0 msgr2=0x7f303c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f303c0665c0 0x7f303c068a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f304c000f70 tx=0x7f304c0016c0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 -- 172.21.15.31:0/3007494359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30580ce5d0 msgr2=0x7f30581508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30580ce5d0 0x7f30581508d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f305001a4a0 tx=0x7f3050019610 comp rx=0 tx=0).stop 2023-11-26T07:22:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 -- 172.21.15.31:0/3007494359 shutdown_connections 2023-11-26T07:22:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f303c0665c0 0x7f303c068a80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30580ce5d0 0x7f30581508d0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30580cb9b0 0x7f3058150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 --2- 172.21.15.31:0/3007494359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305806e3f0 0x7f30580b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 -- 172.21.15.31:0/3007494359 >> 172.21.15.31:0/3007494359 conn(0x7f30580b0cb0 msgr2=0x7f305815d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 -- 172.21.15.31:0/3007494359 shutdown_connections 2023-11-26T07:22:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.209+0000 7f305edf5640 1 -- 172.21.15.31:0/3007494359 wait complete. 2023-11-26T07:22:20.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:22:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc65b7aa640 1 Processor -- start 2023-11-26T07:22:20.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc65b7aa640 1 -- start start 2023-11-26T07:22:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc65b7aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc65415d7b0 0x7fc6540d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc65b7aa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6540cead0 0x7fc6540d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc65b7aa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc6540d1eb0 0x7fc6540cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc65b7aa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6540d2340 con 0x7fc6540cead0 2023-11-26T07:22:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc65b7aa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6540cefd0 con 0x7fc6540d1eb0 2023-11-26T07:22:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc65b7aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540cf2b0 con 0x7fc65415d7b0 2023-11-26T07:22:20.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc658d45640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6540cead0 0x7fc6540d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc658d45640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6540cead0 0x7fc6540d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49522/0 (socket says 172.21.15.31:49522) 2023-11-26T07:22:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc658d45640 1 -- 172.21.15.31:0/258536523 learned_addr learned my addr 172.21.15.31:0/258536523 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc659546640 1 --2- 172.21.15.31:0/258536523 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc65415d7b0 0x7fc6540d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc659d47640 1 --2- 172.21.15.31:0/258536523 >> v2:172.21.15.97:3300/0 conn(0x7fc6540d1eb0 0x7fc6540cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc658d45640 1 -- 172.21.15.31:0/258536523 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc65415d7b0 msgr2=0x7fc6540d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:20.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc658d45640 1 --2- 172.21.15.31:0/258536523 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc65415d7b0 0x7fc6540d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc658d45640 1 -- 172.21.15.31:0/258536523 >> v2:172.21.15.97:3300/0 conn(0x7fc6540d1eb0 msgr2=0x7fc6540cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc658d45640 1 --2- 172.21.15.31:0/258536523 >> v2:172.21.15.97:3300/0 conn(0x7fc6540d1eb0 0x7fc6540cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc658d45640 1 -- 172.21.15.31:0/258536523 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65406b3f0 con 0x7fc6540cead0 2023-11-26T07:22:20.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.373+0000 7fc659d47640 1 --2- 172.21.15.31:0/258536523 >> v2:172.21.15.97:3300/0 conn(0x7fc6540d1eb0 0x7fc6540cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:20.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc658d45640 1 --2- 172.21.15.31:0/258536523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 0x7fc6540d1970 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc65408c870 tx=0x7fc64401c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471a22f9ed7559cc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc64bfff640 1 -- 172.21.15.31:0/258536523 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440ab600 con 0x7fc6540cead0 2023-11-26T07:22:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc64bfff640 1 -- 172.21.15.31:0/258536523 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6440a7050 con 0x7fc6540cead0 2023-11-26T07:22:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc64bfff640 1 -- 172.21.15.31:0/258536523 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440ab980 con 0x7fc6540cead0 2023-11-26T07:22:20.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc65b7aa640 1 -- 172.21.15.31:0/258536523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 msgr2=0x7fc6540d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc65b7aa640 1 --2- 172.21.15.31:0/258536523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 0x7fc6540d1970 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc65408c870 tx=0x7fc64401c0f0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc65b7aa640 1 -- 172.21.15.31:0/258536523 shutdown_connections 2023-11-26T07:22:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc65b7aa640 1 --2- 172.21.15.31:0/258536523 >> v2:172.21.15.97:3300/0 conn(0x7fc6540d1eb0 0x7fc6540cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc65b7aa640 1 --2- 172.21.15.31:0/258536523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 0x7fc6540d1970 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc65b7aa640 1 --2- 172.21.15.31:0/258536523 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc65415d7b0 0x7fc6540d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc65b7aa640 1 -- 172.21.15.31:0/258536523 >> 172.21.15.31:0/258536523 conn(0x7fc6540d33b0 msgr2=0x7fc6540d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc65b7aa640 1 -- 172.21.15.31:0/258536523 shutdown_connections 2023-11-26T07:22:20.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.377+0000 7fc65b7aa640 1 -- 172.21.15.31:0/258536523 wait complete. 2023-11-26T07:22:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc65b7aa640 1 Processor -- start 2023-11-26T07:22:20.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc65b7aa640 1 -- start start 2023-11-26T07:22:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc65b7aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 0x7fc6540cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc65b7aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6540d1eb0 0x7fc65415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc65b7aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc65415d7b0 0x7fc6540ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc65b7aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540cc160 con 0x7fc6540cead0 2023-11-26T07:22:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc65b7aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540cc440 con 0x7fc65415d7b0 2023-11-26T07:22:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc65b7aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540cc720 con 0x7fc6540d1eb0 2023-11-26T07:22:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659546640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 0x7fc6540cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659546640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 0x7fc6540cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49524/0 (socket says 172.21.15.31:49524) 2023-11-26T07:22:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659546640 1 -- 172.21.15.31:0/10723031 learned_addr learned my addr 172.21.15.31:0/10723031 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:20.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659546640 1 -- 172.21.15.31:0/10723031 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6540d1eb0 msgr2=0x7fc65415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659546640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6540d1eb0 0x7fc65415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659546640 1 -- 172.21.15.31:0/10723031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc65415d7b0 msgr2=0x7fc6540ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659d47640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc65415d7b0 0x7fc6540ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659546640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc65415d7b0 0x7fc6540ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659546640 1 -- 172.21.15.31:0/10723031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440094d0 con 0x7fc6540cead0 2023-11-26T07:22:20.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659d47640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc65415d7b0 0x7fc6540ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:20.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc659546640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 0x7fc6540cb7f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc65000e870 tx=0x7fc65000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc649ffb640 1 -- 172.21.15.31:0/10723031 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650009510 con 0x7fc6540cead0 2023-11-26T07:22:20.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc649ffb640 1 -- 172.21.15.31:0/10723031 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65000eec0 con 0x7fc6540cead0 2023-11-26T07:22:20.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc649ffb640 1 -- 172.21.15.31:0/10723031 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000cb40 con 0x7fc6540cead0 2023-11-26T07:22:20.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc65b7aa640 1 -- 172.21.15.31:0/10723031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6540cca00 con 0x7fc6540cead0 2023-11-26T07:22:20.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.381+0000 7fc65b7aa640 1 -- 172.21.15.31:0/10723031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6540cbd30 con 0x7fc6540cead0 2023-11-26T07:22:20.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.385+0000 7fc65b7aa640 1 -- 172.21.15.31:0/10723031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65404e850 con 0x7fc6540cead0 2023-11-26T07:22:20.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.385+0000 7fc649ffb640 1 -- 172.21.15.31:0/10723031 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fc65000cca0 con 0x7fc6540cead0 2023-11-26T07:22:20.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.385+0000 7fc649ffb640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc638066590 0x7fc638068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.385+0000 7fc649ffb640 1 -- 172.21.15.31:0/10723031 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc650083e10 con 0x7fc6540cead0 2023-11-26T07:22:20.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.385+0000 7fc658d45640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc638066590 0x7fc638068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.385+0000 7fc658d45640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc638066590 0x7fc638068a50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ad000 tx=0x7fc644008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.389+0000 7fc649ffb640 1 -- 172.21.15.31:0/10723031 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc650087030 con 0x7fc6540cead0 2023-11-26T07:22:20.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.549+0000 7fc65b7aa640 1 -- 172.21.15.31:0/10723031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc6540b2490 con 0x7fc6540cead0 2023-11-26T07:22:20.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.549+0000 7fc649ffb640 1 -- 172.21.15.31:0/10723031 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc650051430 con 0x7fc6540cead0 2023-11-26T07:22:20.554 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:20.554 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:22:20.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:20.554 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-11-26T07:22:20.555 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-11-26T07:22:20.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:20.555 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:22:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 -- 172.21.15.31:0/10723031 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc638066590 msgr2=0x7fc638068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc638066590 0x7fc638068a50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ad000 tx=0x7fc644008040 comp rx=0 tx=0).stop 2023-11-26T07:22:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 -- 172.21.15.31:0/10723031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 msgr2=0x7fc6540cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 0x7fc6540cb7f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc65000e870 tx=0x7fc65000ec70 comp rx=0 tx=0).stop 2023-11-26T07:22:20.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 -- 172.21.15.31:0/10723031 shutdown_connections 2023-11-26T07:22:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc638066590 0x7fc638068a50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc65415d7b0 0x7fc6540ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6540d1eb0 0x7fc65415bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 --2- 172.21.15.31:0/10723031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6540cead0 0x7fc6540cb7f0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 -- 172.21.15.31:0/10723031 >> 172.21.15.31:0/10723031 conn(0x7fc6540d33b0 msgr2=0x7fc6540b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 -- 172.21.15.31:0/10723031 shutdown_connections 2023-11-26T07:22:20.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.553+0000 7fc65b7aa640 1 -- 172.21.15.31:0/10723031 wait complete. 2023-11-26T07:22:20.575 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:22:20.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:22:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.717+0000 7f5d44bf8640 1 Processor -- start 2023-11-26T07:22:20.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.717+0000 7f5d44bf8640 1 -- start start 2023-11-26T07:22:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d44bf8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5d400cb9b0 0x7f5d4006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d44bf8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d400cbd90 0x7f5d4015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d44bf8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d400ce5d0 0x7f5d4015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d44bf8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4015bba0 con 0x7f5d400cbd90 2023-11-26T07:22:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d44bf8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d400cdd50 con 0x7f5d400cb9b0 2023-11-26T07:22:20.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d44bf8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40159630 con 0x7f5d400ce5d0 2023-11-26T07:22:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d37fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d400cbd90 0x7f5d4015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d37fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d400cbd90 0x7f5d4015f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49528/0 (socket says 172.21.15.31:49528) 2023-11-26T07:22:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d37fff640 1 -- 172.21.15.31:0/431279873 learned_addr learned my addr 172.21.15.31:0/431279873 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d3e59c640 1 --2- 172.21.15.31:0/431279873 >> v2:172.21.15.97:3300/0 conn(0x7f5d400cb9b0 0x7f5d4006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d3ed9d640 1 --2- 172.21.15.31:0/431279873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d400ce5d0 0x7f5d4015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d37fff640 1 -- 172.21.15.31:0/431279873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d400ce5d0 msgr2=0x7f5d4015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d37fff640 1 --2- 172.21.15.31:0/431279873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d400ce5d0 0x7f5d4015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d37fff640 1 -- 172.21.15.31:0/431279873 >> v2:172.21.15.97:3300/0 conn(0x7f5d400cb9b0 msgr2=0x7f5d4006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:20.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d37fff640 1 --2- 172.21.15.31:0/431279873 >> v2:172.21.15.97:3300/0 conn(0x7f5d400cb9b0 0x7f5d4006e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d37fff640 1 -- 172.21.15.31:0/431279873 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4015f6d0 con 0x7f5d400cbd90 2023-11-26T07:22:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d3e59c640 1 --2- 172.21.15.31:0/431279873 >> v2:172.21.15.97:3300/0 conn(0x7f5d400cb9b0 0x7f5d4006e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:20.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.721+0000 7f5d3ed9d640 1 --2- 172.21.15.31:0/431279873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d400ce5d0 0x7f5d4015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d37fff640 1 --2- 172.21.15.31:0/431279873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cbd90 0x7f5d4015f190 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5d4006c360 tx=0x7f5d2801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2974e626cfd5bf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d3dd9b640 1 -- 172.21.15.31:0/431279873 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d280ab670 con 0x7f5d400cbd90 2023-11-26T07:22:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d3dd9b640 1 -- 172.21.15.31:0/431279873 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d280ab7d0 con 0x7f5d400cbd90 2023-11-26T07:22:20.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d3dd9b640 1 -- 172.21.15.31:0/431279873 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d280abb60 con 0x7f5d400cbd90 2023-11-26T07:22:20.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 -- 172.21.15.31:0/431279873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cbd90 msgr2=0x7f5d4015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 --2- 172.21.15.31:0/431279873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cbd90 0x7f5d4015f190 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5d4006c360 tx=0x7f5d2801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 -- 172.21.15.31:0/431279873 shutdown_connections 2023-11-26T07:22:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 --2- 172.21.15.31:0/431279873 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d400ce5d0 0x7f5d4015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 --2- 172.21.15.31:0/431279873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cbd90 0x7f5d4015f190 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 --2- 172.21.15.31:0/431279873 >> v2:172.21.15.97:3300/0 conn(0x7f5d400cb9b0 0x7f5d4006e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 -- 172.21.15.31:0/431279873 >> 172.21.15.31:0/431279873 conn(0x7f5d400b0cb0 msgr2=0x7f5d400b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 -- 172.21.15.31:0/431279873 shutdown_connections 2023-11-26T07:22:20.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 -- 172.21.15.31:0/431279873 wait complete. 2023-11-26T07:22:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 Processor -- start 2023-11-26T07:22:20.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.725+0000 7f5d44bf8640 1 -- start start 2023-11-26T07:22:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d44bf8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cb9b0 0x7f5d4009a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d44bf8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400cbd90 0x7f5d4015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d44bf8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d400ce5d0 0x7f5d40157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d44bf8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4009a120 con 0x7f5d400cb9b0 2023-11-26T07:22:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d44bf8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40099b60 con 0x7f5d400cbd90 2023-11-26T07:22:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d44bf8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40099e40 con 0x7f5d400ce5d0 2023-11-26T07:22:20.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d3e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cb9b0 0x7f5d4009a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d3e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cb9b0 0x7f5d4009a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49538/0 (socket says 172.21.15.31:49538) 2023-11-26T07:22:20.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d3e59c640 1 -- 172.21.15.31:0/1323647505 learned_addr learned my addr 172.21.15.31:0/1323647505 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d3e59c640 1 -- 172.21.15.31:0/1323647505 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d400ce5d0 msgr2=0x7f5d40157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d3e59c640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d400ce5d0 0x7f5d40157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d3e59c640 1 -- 172.21.15.31:0/1323647505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400cbd90 msgr2=0x7f5d4015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d3e59c640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400cbd90 0x7f5d4015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:20.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d3e59c640 1 -- 172.21.15.31:0/1323647505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d280094d0 con 0x7f5d400cb9b0 2023-11-26T07:22:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d3e59c640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cb9b0 0x7f5d4009a3c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c7c0 tx=0x7f5d3000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d36ffd640 1 -- 172.21.15.31:0/1323647505 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d30007a30 con 0x7f5d400cb9b0 2023-11-26T07:22:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d36ffd640 1 -- 172.21.15.31:0/1323647505 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d30007b90 con 0x7f5d400cb9b0 2023-11-26T07:22:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d36ffd640 1 -- 172.21.15.31:0/1323647505 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3000ce70 con 0x7f5d400cb9b0 2023-11-26T07:22:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d44bf8640 1 -- 172.21.15.31:0/1323647505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d400b2150 con 0x7f5d400cb9b0 2023-11-26T07:22:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d44bf8640 1 -- 172.21.15.31:0/1323647505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4009a900 con 0x7f5d400cb9b0 2023-11-26T07:22:20.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d44bf8640 1 -- 172.21.15.31:0/1323647505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d400671f0 con 0x7f5d400cb9b0 2023-11-26T07:22:20.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.729+0000 7f5d36ffd640 1 -- 172.21.15.31:0/1323647505 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f5d300113f0 con 0x7f5d400cb9b0 2023-11-26T07:22:20.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.733+0000 7f5d36ffd640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5d20066590 0x7f5d20068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:20.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.733+0000 7f5d36ffd640 1 -- 172.21.15.31:0/1323647505 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5d30084280 con 0x7f5d400cb9b0 2023-11-26T07:22:20.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.733+0000 7f5d37fff640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5d20066590 0x7f5d20068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:20.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.733+0000 7f5d37fff640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5d20066590 0x7f5d20068a50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d280af000 tx=0x7f5d28008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:20.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.737+0000 7f5d36ffd640 1 -- 172.21.15.31:0/1323647505 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5d30051820 con 0x7f5d400cb9b0 2023-11-26T07:22:20.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:20.893+0000 7f5d44bf8640 1 -- 172.21.15.31:0/1323647505 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5d4006b460 con 0x7f5d400cb9b0 2023-11-26T07:22:21.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d36ffd640 1 -- 172.21.15.31:0/1323647505 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5d30054e40 con 0x7f5d400cb9b0 2023-11-26T07:22:21.119 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-11-26T07:22:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 -- 172.21.15.31:0/1323647505 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5d20066590 msgr2=0x7f5d20068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5d20066590 0x7f5d20068a50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d280af000 tx=0x7f5d28008040 comp rx=0 tx=0).stop 2023-11-26T07:22:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 -- 172.21.15.31:0/1323647505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cb9b0 msgr2=0x7f5d4009a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cb9b0 0x7f5d4009a3c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000c7c0 tx=0x7f5d3000cc80 comp rx=0 tx=0).stop 2023-11-26T07:22:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 -- 172.21.15.31:0/1323647505 shutdown_connections 2023-11-26T07:22:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5d20066590 0x7f5d20068a50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5d400ce5d0 0x7f5d40157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d400cbd90 0x7f5d4015a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 --2- 172.21.15.31:0/1323647505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d400cb9b0 0x7f5d4009a3c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 -- 172.21.15.31:0/1323647505 >> 172.21.15.31:0/1323647505 conn(0x7f5d400b0cb0 msgr2=0x7f5d40159910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 -- 172.21.15.31:0/1323647505 shutdown_connections 2023-11-26T07:22:21.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.117+0000 7f5d44bf8640 1 -- 172.21.15.31:0/1323647505 wait complete. 2023-11-26T07:22:21.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:22:21.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.293+0000 7fb896cbb640 1 Processor -- start 2023-11-26T07:22:21.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.293+0000 7fb896cbb640 1 -- start start 2023-11-26T07:22:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb896cbb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb8900b4c40 0x7fb8900b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb896cbb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb8900d10e0 0x7fb8900d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb896cbb640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8900b55d0 con 0x7fb89015d2a0 2023-11-26T07:22:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb896cbb640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8900ce8c0 con 0x7fb8900b4c40 2023-11-26T07:22:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb896cbb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8900b6250 con 0x7fb8900d10e0 2023-11-26T07:22:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb895cb9640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb89015d2a0 0x7fb8900d26e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36656/0 (socket says 172.21.15.31:36656) 2023-11-26T07:22:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb895cb9640 1 -- 172.21.15.31:0/3341372618 learned_addr learned my addr 172.21.15.31:0/3341372618 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb8954b8640 1 --2- 172.21.15.31:0/3341372618 >> v2:172.21.15.31:3300/0 conn(0x7fb8900b4c40 0x7fb8900b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb8964ba640 1 --2- 172.21.15.31:0/3341372618 >> v2:172.21.15.97:3300/0 conn(0x7fb8900d10e0 0x7fb8900d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb8954b8640 1 -- 172.21.15.31:0/3341372618 >> v2:172.21.15.97:3300/0 conn(0x7fb8900d10e0 msgr2=0x7fb8900d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb8954b8640 1 --2- 172.21.15.31:0/3341372618 >> v2:172.21.15.97:3300/0 conn(0x7fb8900d10e0 0x7fb8900d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb8954b8640 1 -- 172.21.15.31:0/3341372618 >> v1:172.21.15.31:6789/0 conn(0x7fb89015d2a0 legacy=0x7fb8900d26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb8954b8640 1 -- 172.21.15.31:0/3341372618 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890161cf0 con 0x7fb8900b4c40 2023-11-26T07:22:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb8964ba640 1 --2- 172.21.15.31:0/3341372618 >> v2:172.21.15.97:3300/0 conn(0x7fb8900d10e0 0x7fb8900d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:21.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb894cb7640 1 -- 172.21.15.31:0/3341372618 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840250230 0 0) 0x7fb88c001120 con 0x7fb89015d2a0 2023-11-26T07:22:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.297+0000 7fb8954b8640 1 --2- 172.21.15.31:0/3341372618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 0x7fb8900b5090 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb8900b5c50 tx=0x7fb88001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe267707262d727 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb894cb7640 1 -- 172.21.15.31:0/3341372618 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8800a7070 con 0x7fb8900b4c40 2023-11-26T07:22:21.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb894cb7640 1 -- 172.21.15.31:0/3341372618 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8800ab430 con 0x7fb8900b4c40 2023-11-26T07:22:21.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb894cb7640 1 -- 172.21.15.31:0/3341372618 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8800ab7c0 con 0x7fb8900b4c40 2023-11-26T07:22:21.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 -- 172.21.15.31:0/3341372618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 msgr2=0x7fb8900b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 --2- 172.21.15.31:0/3341372618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 0x7fb8900b5090 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb8900b5c50 tx=0x7fb88001c430 comp rx=0 tx=0).stop 2023-11-26T07:22:21.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 -- 172.21.15.31:0/3341372618 shutdown_connections 2023-11-26T07:22:21.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 --2- 172.21.15.31:0/3341372618 >> v2:172.21.15.97:3300/0 conn(0x7fb8900d10e0 0x7fb8900d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 --2- 172.21.15.31:0/3341372618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 0x7fb8900b5090 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 -- 172.21.15.31:0/3341372618 >> 172.21.15.31:0/3341372618 conn(0x7fb8900668a0 msgr2=0x7fb8900cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:21.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 -- 172.21.15.31:0/3341372618 shutdown_connections 2023-11-26T07:22:21.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 -- 172.21.15.31:0/3341372618 wait complete. 2023-11-26T07:22:21.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 Processor -- start 2023-11-26T07:22:21.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 -- start start 2023-11-26T07:22:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 0x7fb890142410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900d10e0 0x7fb890151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb89015d2a0 0x7fb89015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb890155f20 con 0x7fb8900b4c40 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015ea40 con 0x7fb8900d10e0 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb896cbb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015ed20 con 0x7fb89015d2a0 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb895cb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 0x7fb890142410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb895cb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 0x7fb890142410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49558/0 (socket says 172.21.15.31:49558) 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.301+0000 7fb895cb9640 1 -- 172.21.15.31:0/185415978 learned_addr learned my addr 172.21.15.31:0/185415978 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb8954b8640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900d10e0 0x7fb890151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb8964ba640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb89015d2a0 0x7fb89015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb895cb9640 1 -- 172.21.15.31:0/185415978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb89015d2a0 msgr2=0x7fb89015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb895cb9640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb89015d2a0 0x7fb89015bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb895cb9640 1 -- 172.21.15.31:0/185415978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900d10e0 msgr2=0x7fb890151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb895cb9640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900d10e0 0x7fb890151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb895cb9640 1 -- 172.21.15.31:0/185415978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb880009420 con 0x7fb8900b4c40 2023-11-26T07:22:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb895cb9640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 0x7fb890142410 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00c780 tx=0x7fb88c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb8867fc640 1 -- 172.21.15.31:0/185415978 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c0072d0 con 0x7fb8900b4c40 2023-11-26T07:22:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb8867fc640 1 -- 172.21.15.31:0/185415978 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c007450 con 0x7fb8900b4c40 2023-11-26T07:22:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb8867fc640 1 -- 172.21.15.31:0/185415978 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c0047f0 con 0x7fb8900b4c40 2023-11-26T07:22:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb896cbb640 1 -- 172.21.15.31:0/185415978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb890155bb0 con 0x7fb8900b4c40 2023-11-26T07:22:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb896cbb640 1 -- 172.21.15.31:0/185415978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89015f000 con 0x7fb8900b4c40 2023-11-26T07:22:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb896cbb640 1 -- 172.21.15.31:0/185415978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb890074990 con 0x7fb8900b4c40 2023-11-26T07:22:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.305+0000 7fb8867fc640 1 -- 172.21.15.31:0/185415978 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fb88c00a640 con 0x7fb8900b4c40 2023-11-26T07:22:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.309+0000 7fb8867fc640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb874066590 0x7fb874068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.309+0000 7fb8867fc640 1 -- 172.21.15.31:0/185415978 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb88c083240 con 0x7fb8900b4c40 2023-11-26T07:22:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.309+0000 7fb8954b8640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb874066590 0x7fb874068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.309+0000 7fb8954b8640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb874066590 0x7fb874068a50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb8800af000 tx=0x7fb8800a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:21.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.309+0000 7fb8867fc640 1 -- 172.21.15.31:0/185415978 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb88c050900 con 0x7fb8900b4c40 2023-11-26T07:22:21.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:21.473+0000 7fb896cbb640 1 -- 172.21.15.31:0/185415978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb8900b55d0 con 0x7fb8900b4c40 2023-11-26T07:22:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.117+0000 7fb8867fc640 1 -- 172.21.15.31:0/185415978 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb8900b55d0 con 0x7fb8900b4c40 2023-11-26T07:22:22.122 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-11-26T07:22:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.121+0000 7fb896cbb640 1 -- 172.21.15.31:0/185415978 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb874066590 msgr2=0x7fb874068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.121+0000 7fb896cbb640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb874066590 0x7fb874068a50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb8800af000 tx=0x7fb8800a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.121+0000 7fb896cbb640 1 -- 172.21.15.31:0/185415978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 msgr2=0x7fb890142410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.121+0000 7fb896cbb640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 0x7fb890142410 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00c780 tx=0x7fb88c00cc40 comp rx=0 tx=0).stop 2023-11-26T07:22:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.121+0000 7fb896cbb640 1 -- 172.21.15.31:0/185415978 shutdown_connections 2023-11-26T07:22:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.121+0000 7fb896cbb640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb874066590 0x7fb874068a50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.121+0000 7fb896cbb640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb89015d2a0 0x7fb89015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.121+0000 7fb896cbb640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900d10e0 0x7fb890151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.121+0000 7fb896cbb640 1 --2- 172.21.15.31:0/185415978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900b4c40 0x7fb890142410 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.121+0000 7fb896cbb640 1 -- 172.21.15.31:0/185415978 >> 172.21.15.31:0/185415978 conn(0x7fb8900668a0 msgr2=0x7fb8900b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:22.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.125+0000 7fb896cbb640 1 -- 172.21.15.31:0/185415978 shutdown_connections 2023-11-26T07:22:22.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.125+0000 7fb896cbb640 1 -- 172.21.15.31:0/185415978 wait complete. 2023-11-26T07:22:22.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:22:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a1ce6b640 1 Processor -- start 2023-11-26T07:22:22.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a1ce6b640 1 -- start start 2023-11-26T07:22:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a1ce6b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4a180cdfb0 0x7f4a180abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a1ce6b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4a1815a710 0x7f4a180ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a1ce6b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a180cbd50 0x7f4a180cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a1ce6b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a180cca80 con 0x7f4a1815a710 2023-11-26T07:22:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a1ce6b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a180ce8d0 con 0x7f4a180cdfb0 2023-11-26T07:22:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a1ce6b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180b2710 con 0x7f4a180cbd50 2023-11-26T07:22:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a15d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4a1815a710 0x7f4a180ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a15d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4a1815a710 0x7f4a180ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49562/0 (socket says 172.21.15.31:49562) 2023-11-26T07:22:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a15d9b640 1 -- 172.21.15.31:0/11289755 learned_addr learned my addr 172.21.15.31:0/11289755 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a16d9d640 1 --2- 172.21.15.31:0/11289755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a180cbd50 0x7f4a180cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a1659c640 1 --2- 172.21.15.31:0/11289755 >> v2:172.21.15.97:3300/0 conn(0x7f4a180cdfb0 0x7f4a180abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a15d9b640 1 -- 172.21.15.31:0/11289755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a180cbd50 msgr2=0x7f4a180cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:22.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a15d9b640 1 --2- 172.21.15.31:0/11289755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a180cbd50 0x7f4a180cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a15d9b640 1 -- 172.21.15.31:0/11289755 >> v2:172.21.15.97:3300/0 conn(0x7f4a180cdfb0 msgr2=0x7f4a180abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:22.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a15d9b640 1 --2- 172.21.15.31:0/11289755 >> v2:172.21.15.97:3300/0 conn(0x7f4a180cdfb0 0x7f4a180abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a15d9b640 1 -- 172.21.15.31:0/11289755 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180b29f0 con 0x7f4a1815a710 2023-11-26T07:22:22.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.317+0000 7f4a1659c640 1 --2- 172.21.15.31:0/11289755 >> v2:172.21.15.97:3300/0 conn(0x7f4a180cdfb0 0x7f4a180abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:22.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a15d9b640 1 --2- 172.21.15.31:0/11289755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1815a710 0x7f4a180ce390 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4a1806c270 tx=0x7f4a0c01e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c2d208d1cbb8f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1559a640 1 -- 172.21.15.31:0/11289755 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c0a8670 con 0x7f4a1815a710 2023-11-26T07:22:22.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1559a640 1 -- 172.21.15.31:0/11289755 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c0a87d0 con 0x7f4a1815a710 2023-11-26T07:22:22.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1559a640 1 -- 172.21.15.31:0/11289755 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c0a8b60 con 0x7f4a1815a710 2023-11-26T07:22:22.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/11289755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1815a710 msgr2=0x7f4a180ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1ce6b640 1 --2- 172.21.15.31:0/11289755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1815a710 0x7f4a180ce390 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f4a1806c270 tx=0x7f4a0c01e6e0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/11289755 shutdown_connections 2023-11-26T07:22:22.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1ce6b640 1 --2- 172.21.15.31:0/11289755 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a180cbd50 0x7f4a180cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1ce6b640 1 --2- 172.21.15.31:0/11289755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a1815a710 0x7f4a180ce390 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1ce6b640 1 --2- 172.21.15.31:0/11289755 >> v2:172.21.15.97:3300/0 conn(0x7f4a180cdfb0 0x7f4a180abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/11289755 >> 172.21.15.31:0/11289755 conn(0x7f4a180b0cb0 msgr2=0x7f4a180b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:22.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/11289755 shutdown_connections 2023-11-26T07:22:22.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.321+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/11289755 wait complete. 2023-11-26T07:22:22.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 Processor -- start 2023-11-26T07:22:22.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 -- start start 2023-11-26T07:22:22.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a180cbd50 0x7f4a18159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a180cdfb0 0x7f4a181571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a1815a710 0x7f4a181576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a18151410 con 0x7f4a180cbd50 2023-11-26T07:22:22.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a18159640 con 0x7f4a1815a710 2023-11-26T07:22:22.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a181598a0 con 0x7f4a180cdfb0 2023-11-26T07:22:22.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a180cbd50 0x7f4a18159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a180cbd50 0x7f4a18159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49564/0 (socket says 172.21.15.31:49564) 2023-11-26T07:22:22.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1659c640 1 -- 172.21.15.31:0/3120438365 learned_addr learned my addr 172.21.15.31:0/3120438365 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:22.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a16d9d640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a1815a710 0x7f4a181576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a15d9b640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a180cdfb0 0x7f4a181571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1659c640 1 -- 172.21.15.31:0/3120438365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a180cdfb0 msgr2=0x7f4a181571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1659c640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a180cdfb0 0x7f4a181571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1659c640 1 -- 172.21.15.31:0/3120438365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a1815a710 msgr2=0x7f4a181576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:22.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1659c640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a1815a710 0x7f4a181576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:22.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1659c640 1 -- 172.21.15.31:0/3120438365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c0094d0 con 0x7f4a180cbd50 2023-11-26T07:22:22.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1659c640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a180cbd50 0x7f4a18159100 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800e810 tx=0x7f4a0800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a06ffd640 1 -- 172.21.15.31:0/3120438365 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800c8d0 con 0x7f4a180cbd50 2023-11-26T07:22:22.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a06ffd640 1 -- 172.21.15.31:0/3120438365 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0800ca30 con 0x7f4a180cbd50 2023-11-26T07:22:22.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a06ffd640 1 -- 172.21.15.31:0/3120438365 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800cde0 con 0x7f4a180cbd50 2023-11-26T07:22:22.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/3120438365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a18159b80 con 0x7f4a180cbd50 2023-11-26T07:22:22.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/3120438365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1815a140 con 0x7f4a180cbd50 2023-11-26T07:22:22.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.325+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/3120438365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a180671f0 con 0x7f4a180cbd50 2023-11-26T07:22:22.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.329+0000 7f4a06ffd640 1 -- 172.21.15.31:0/3120438365 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f4a08026480 con 0x7f4a180cbd50 2023-11-26T07:22:22.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.329+0000 7f4a06ffd640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f49f80665e0 0x7f49f8068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:22.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.329+0000 7f4a06ffd640 1 -- 172.21.15.31:0/3120438365 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4a08084180 con 0x7f4a180cbd50 2023-11-26T07:22:22.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.329+0000 7f4a15d9b640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f49f80665e0 0x7f49f8068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:22.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.329+0000 7f4a15d9b640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f49f80665e0 0x7f49f8068aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0b0000 tx=0x7f4a0c0a9000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:22.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.333+0000 7f4a06ffd640 1 -- 172.21.15.31:0/3120438365 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4a08051660 con 0x7f4a180cbd50 2023-11-26T07:22:22.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:22.521+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/3120438365 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4a1806b460 con 0x7f4a180cbd50 2023-11-26T07:22:23.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.157+0000 7f4a06ffd640 1 -- 172.21.15.31:0/3120438365 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4a08054c80 con 0x7f4a180cbd50 2023-11-26T07:22:23.159 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 12 and data pool 13 2023-11-26T07:22:23.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/3120438365 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f49f80665e0 msgr2=0x7f49f8068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f49f80665e0 0x7f49f8068aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0b0000 tx=0x7f4a0c0a9000 comp rx=0 tx=0).stop 2023-11-26T07:22:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/3120438365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a180cbd50 msgr2=0x7f4a18159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a180cbd50 0x7f4a18159100 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800e810 tx=0x7f4a0800ecd0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/3120438365 shutdown_connections 2023-11-26T07:22:23.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f49f80665e0 0x7f49f8068aa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a1815a710 0x7f4a181576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4a180cdfb0 0x7f4a181571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 --2- 172.21.15.31:0/3120438365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a180cbd50 0x7f4a18159100 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/3120438365 >> 172.21.15.31:0/3120438365 conn(0x7f4a180b0cb0 msgr2=0x7f4a180b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.165+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/3120438365 shutdown_connections 2023-11-26T07:22:23.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.169+0000 7f4a1ce6b640 1 -- 172.21.15.31:0/3120438365 wait complete. 2023-11-26T07:22:23.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:23.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.345+0000 7fef66f72640 1 Processor -- start 2023-11-26T07:22:23.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.345+0000 7fef66f72640 1 -- start start 2023-11-26T07:22:23.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef66f72640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 0x7fef601683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef66f72640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef600ad740 con 0x7fef6006e410 2023-11-26T07:22:23.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef66f72640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef600b3580 con 0x7fef60071cc0 2023-11-26T07:22:23.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef66f72640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef600b3e20 con 0x7fef600b30b0 2023-11-26T07:22:23.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef64d0e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fef6006e410 0x7fef600715b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:36658/0 (socket says 172.21.15.31:36658) 2023-11-26T07:22:23.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef64d0e640 1 -- 172.21.15.31:0/75350262 learned_addr learned my addr 172.21.15.31:0/75350262 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:23.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef6550f640 1 --2- 172.21.15.31:0/75350262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 0x7fef601683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef5f7fe640 1 -- 172.21.15.31:0/75350262 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060427171 0 0) 0x7fef600ad740 con 0x7fef6006e410 2023-11-26T07:22:23.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef5f7fe640 1 -- 172.21.15.31:0/75350262 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef50003530 con 0x7fef6006e410 2023-11-26T07:22:23.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef5f7fe640 1 -- 172.21.15.31:0/75350262 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506653834 0 0) 0x7fef600b3580 con 0x7fef60071cc0 2023-11-26T07:22:23.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef5f7fe640 1 -- 172.21.15.31:0/75350262 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef600ad740 con 0x7fef60071cc0 2023-11-26T07:22:23.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef5f7fe640 1 -- 172.21.15.31:0/75350262 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3915769490 0 0) 0x7fef50003530 con 0x7fef6006e410 2023-11-26T07:22:23.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef6550f640 1 -- 172.21.15.31:0/75350262 >> v1:172.21.15.97:6789/0 conn(0x7fef60071cc0 legacy=0x7fef600b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef6550f640 1 -- 172.21.15.31:0/75350262 >> v1:172.21.15.31:6789/0 conn(0x7fef6006e410 legacy=0x7fef600715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.349+0000 7fef6550f640 1 -- 172.21.15.31:0/75350262 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef600b3860 con 0x7fef600b30b0 2023-11-26T07:22:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef6550f640 1 --2- 172.21.15.31:0/75350262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 0x7fef601683c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fef58009dc0 tx=0x7fef5801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18176826f137a6e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef5f7fe640 1 -- 172.21.15.31:0/75350262 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54001680 con 0x7fef600b30b0 2023-11-26T07:22:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef5f7fe640 1 -- 172.21.15.31:0/75350262 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef580a55c0 con 0x7fef600b30b0 2023-11-26T07:22:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef5f7fe640 1 -- 172.21.15.31:0/75350262 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef580a5950 con 0x7fef600b30b0 2023-11-26T07:22:23.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef66f72640 1 -- 172.21.15.31:0/75350262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 msgr2=0x7fef601683c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef66f72640 1 --2- 172.21.15.31:0/75350262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 0x7fef601683c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fef58009dc0 tx=0x7fef5801c890 comp rx=0 tx=0).stop 2023-11-26T07:22:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef66f72640 1 -- 172.21.15.31:0/75350262 shutdown_connections 2023-11-26T07:22:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef66f72640 1 --2- 172.21.15.31:0/75350262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 0x7fef601683c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef66f72640 1 -- 172.21.15.31:0/75350262 >> 172.21.15.31:0/75350262 conn(0x7fef600b1e40 msgr2=0x7fef60070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef66f72640 1 -- 172.21.15.31:0/75350262 shutdown_connections 2023-11-26T07:22:23.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef66f72640 1 -- 172.21.15.31:0/75350262 wait complete. 2023-11-26T07:22:23.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.353+0000 7fef66f72640 1 Processor -- start 2023-11-26T07:22:23.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef66f72640 1 -- start start 2023-11-26T07:22:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef66f72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6006e410 0x7fef6014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef66f72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60071cc0 0x7fef60150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef66f72640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 0x7fef6014eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef66f72640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6014f7e0 con 0x7fef60071cc0 2023-11-26T07:22:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef66f72640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6014ca30 con 0x7fef6006e410 2023-11-26T07:22:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef66f72640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6014f500 con 0x7fef600b30b0 2023-11-26T07:22:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60071cc0 0x7fef60150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60071cc0 0x7fef60150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49574/0 (socket says 172.21.15.31:49574) 2023-11-26T07:22:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5ffff640 1 -- 172.21.15.31:0/2567407486 learned_addr learned my addr 172.21.15.31:0/2567407486 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef64d0e640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6006e410 0x7fef6014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef6550f640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 0x7fef6014eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5ffff640 1 -- 172.21.15.31:0/2567407486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 msgr2=0x7fef6014eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5ffff640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 0x7fef6014eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5ffff640 1 -- 172.21.15.31:0/2567407486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6006e410 msgr2=0x7fef6014cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5ffff640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6006e410 0x7fef6014cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5ffff640 1 -- 172.21.15.31:0/2567407486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58009a70 con 0x7fef60071cc0 2023-11-26T07:22:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5ffff640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60071cc0 0x7fef60150f10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00c490 tx=0x7fef4c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5d7fa640 1 -- 172.21.15.31:0/2567407486 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0052d0 con 0x7fef60071cc0 2023-11-26T07:22:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5d7fa640 1 -- 172.21.15.31:0/2567407486 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c005430 con 0x7fef60071cc0 2023-11-26T07:22:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef5d7fa640 1 -- 172.21.15.31:0/2567407486 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c012870 con 0x7fef60071cc0 2023-11-26T07:22:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef66f72640 1 -- 172.21.15.31:0/2567407486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6014c3e0 con 0x7fef60071cc0 2023-11-26T07:22:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef66f72640 1 -- 172.21.15.31:0/2567407486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6014d8f0 con 0x7fef60071cc0 2023-11-26T07:22:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.357+0000 7fef66f72640 1 -- 172.21.15.31:0/2567407486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6014cbe0 con 0x7fef60071cc0 2023-11-26T07:22:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.361+0000 7fef5d7fa640 1 -- 172.21.15.31:0/2567407486 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fef4c0129d0 con 0x7fef60071cc0 2023-11-26T07:22:23.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.361+0000 7fef5d7fa640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fef440665c0 0x7fef44068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.361+0000 7fef5d7fa640 1 -- 172.21.15.31:0/2567407486 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef4c083b90 con 0x7fef60071cc0 2023-11-26T07:22:23.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.361+0000 7fef64d0e640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fef440665c0 0x7fef44068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.361+0000 7fef64d0e640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fef440665c0 0x7fef44068a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef540011f0 tx=0x7fef54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.365+0000 7fef5d7fa640 1 -- 172.21.15.31:0/2567407486 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fef6014cbe0 con 0x7fef60071cc0 2023-11-26T07:22:23.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.525+0000 7fef66f72640 1 -- 172.21.15.31:0/2567407486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef6014cba0 con 0x7fef60071cc0 2023-11-26T07:22:23.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.525+0000 7fef5d7fa640 1 -- 172.21.15.31:0/2567407486 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16489 (secure 0 0 0) 0x7fef4c050ef0 con 0x7fef60071cc0 2023-11-26T07:22:23.529 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:23.529 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":79,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:22.526432+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:20.941587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:21.520554+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 -- 172.21.15.31:0/2567407486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fef440665c0 msgr2=0x7fef44068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fef440665c0 0x7fef44068a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fef540011f0 tx=0x7fef54009040 comp rx=0 tx=0).stop 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 -- 172.21.15.31:0/2567407486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60071cc0 msgr2=0x7fef60150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60071cc0 0x7fef60150f10 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00c490 tx=0x7fef4c00c950 comp rx=0 tx=0).stop 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 -- 172.21.15.31:0/2567407486 shutdown_connections 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fef440665c0 0x7fef44068a80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fef600b30b0 0x7fef6014eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fef60071cc0 0x7fef60150f10 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 --2- 172.21.15.31:0/2567407486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef6006e410 0x7fef6014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 -- 172.21.15.31:0/2567407486 >> 172.21.15.31:0/2567407486 conn(0x7fef600b1e40 msgr2=0x7fef6006d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 -- 172.21.15.31:0/2567407486 shutdown_connections 2023-11-26T07:22:23.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.529+0000 7fef66f72640 1 -- 172.21.15.31:0/2567407486 wait complete. 2023-11-26T07:22:23.550 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.689+0000 7f1679ee3640 1 Processor -- start 2023-11-26T07:22:23.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.689+0000 7f1679ee3640 1 -- start start 2023-11-26T07:22:23.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.689+0000 7f1679ee3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 0x7f16740abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.689+0000 7f1679ee3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1674145060 con 0x7f1674070df0 2023-11-26T07:22:23.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.689+0000 7f1679ee3640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1674071fd0 con 0x7f16740ac460 2023-11-26T07:22:23.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.689+0000 7f1679ee3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1674071250 con 0x7f1674144c80 2023-11-26T07:22:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f1673fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1674070df0 0x7f167406c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60458/0 (socket says 172.21.15.31:60458) 2023-11-26T07:22:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f1673fff640 1 -- 172.21.15.31:0/822122244 learned_addr learned my addr 172.21.15.31:0/822122244 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f16737fe640 1 --2- 172.21.15.31:0/822122244 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 0x7f16740abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f16737fe640 1 -- 172.21.15.31:0/822122244 >> v1:172.21.15.97:6789/0 conn(0x7f16740ac460 legacy=0x7f16740706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f16737fe640 1 -- 172.21.15.31:0/822122244 >> v1:172.21.15.31:6789/0 conn(0x7f1674070df0 legacy=0x7f167406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f16727fc640 1 -- 172.21.15.31:0/822122244 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003789837 0 0) 0x7f1674145060 con 0x7f1674070df0 2023-11-26T07:22:23.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f16737fe640 1 -- 172.21.15.31:0/822122244 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16740ac8c0 con 0x7f1674144c80 2023-11-26T07:22:23.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f16737fe640 1 --2- 172.21.15.31:0/822122244 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 0x7f16740abf20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1664009770 tx=0x7f166401c2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a05a962cb52b717 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f16727fc640 1 -- 172.21.15.31:0/822122244 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640ac640 con 0x7f1674144c80 2023-11-26T07:22:23.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f16727fc640 1 -- 172.21.15.31:0/822122244 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16640ac7a0 con 0x7f1674144c80 2023-11-26T07:22:23.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.693+0000 7f16727fc640 1 -- 172.21.15.31:0/822122244 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640acb30 con 0x7f1674144c80 2023-11-26T07:22:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 -- 172.21.15.31:0/822122244 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 msgr2=0x7f16740abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 --2- 172.21.15.31:0/822122244 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 0x7f16740abf20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1664009770 tx=0x7f166401c2f0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 -- 172.21.15.31:0/822122244 shutdown_connections 2023-11-26T07:22:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 --2- 172.21.15.31:0/822122244 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 0x7f16740abf20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 -- 172.21.15.31:0/822122244 >> 172.21.15.31:0/822122244 conn(0x7f16740b0c40 msgr2=0x7f16740b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 -- 172.21.15.31:0/822122244 shutdown_connections 2023-11-26T07:22:23.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 -- 172.21.15.31:0/822122244 wait complete. 2023-11-26T07:22:23.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 Processor -- start 2023-11-26T07:22:23.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 -- start start 2023-11-26T07:22:23.702 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1674070df0 0x7f1674159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16740ac460 0x7f167413c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 0x7f167406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167409c1d0 con 0x7f16740ac460 2023-11-26T07:22:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167409a260 con 0x7f1674070df0 2023-11-26T07:22:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1679ee3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167409a540 con 0x7f1674144c80 2023-11-26T07:22:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1672ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16740ac460 0x7f167413c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1672ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16740ac460 0x7f167413c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37820/0 (socket says 172.21.15.31:37820) 2023-11-26T07:22:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f16737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1674070df0 0x7f1674159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1672ffd640 1 -- 172.21.15.31:0/3853676068 learned_addr learned my addr 172.21.15.31:0/3853676068 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:23.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.697+0000 7f1673fff640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 0x7f167406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f1672ffd640 1 -- 172.21.15.31:0/3853676068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 msgr2=0x7f167406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f1672ffd640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 0x7f167406d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f1672ffd640 1 -- 172.21.15.31:0/3853676068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1674070df0 msgr2=0x7f1674159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f1672ffd640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1674070df0 0x7f1674159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f1672ffd640 1 -- 172.21.15.31:0/3853676068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1664009420 con 0x7f16740ac460 2023-11-26T07:22:23.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f1673fff640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 0x7f167406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:23.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f1672ffd640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16740ac460 0x7f167413c380 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1668009eb0 tx=0x7f166800c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f165bfff640 1 -- 172.21.15.31:0/3853676068 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166800ce10 con 0x7f16740ac460 2023-11-26T07:22:23.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f165bfff640 1 -- 172.21.15.31:0/3853676068 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668004020 con 0x7f16740ac460 2023-11-26T07:22:23.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f1679ee3640 1 -- 172.21.15.31:0/3853676068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167409a820 con 0x7f16740ac460 2023-11-26T07:22:23.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f165bfff640 1 -- 172.21.15.31:0/3853676068 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680043d0 con 0x7f16740ac460 2023-11-26T07:22:23.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f1679ee3640 1 -- 172.21.15.31:0/3853676068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167409ade0 con 0x7f16740ac460 2023-11-26T07:22:23.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f1679ee3640 1 -- 172.21.15.31:0/3853676068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16740b1ba0 con 0x7f16740ac460 2023-11-26T07:22:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f165bfff640 1 -- 172.21.15.31:0/3853676068 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f1668003600 con 0x7f16740ac460 2023-11-26T07:22:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f165bfff640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f16600665c0 0x7f1660068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f165bfff640 1 -- 172.21.15.31:0/3853676068 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1668086830 con 0x7f16740ac460 2023-11-26T07:22:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.701+0000 7f16737fe640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f16600665c0 0x7f1660068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:23.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.705+0000 7f16737fe640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f16600665c0 0x7f1660068a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16640ae000 tx=0x7f16640a6070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:23.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.705+0000 7f165bfff640 1 -- 172.21.15.31:0/3853676068 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1668053a80 con 0x7f16740ac460 2023-11-26T07:22:23.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.861+0000 7f1679ee3640 1 -- 172.21.15.31:0/3853676068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f167409b000 con 0x7f16740ac460 2023-11-26T07:22:23.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f165bfff640 1 -- 172.21.15.31:0/3853676068 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16489 (secure 0 0 0) 0x7f16680570a0 con 0x7f16740ac460 2023-11-26T07:22:23.867 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:23.867 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":79,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:22.526432+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:20.941587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:21.520554+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 -- 172.21.15.31:0/3853676068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f16600665c0 msgr2=0x7f1660068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f16600665c0 0x7f1660068a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16640ae000 tx=0x7f16640a6070 comp rx=0 tx=0).stop 2023-11-26T07:22:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 -- 172.21.15.31:0/3853676068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16740ac460 msgr2=0x7f167413c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:23.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16740ac460 0x7f167413c380 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1668009eb0 tx=0x7f166800c430 comp rx=0 tx=0).stop 2023-11-26T07:22:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 -- 172.21.15.31:0/3853676068 shutdown_connections 2023-11-26T07:22:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f16600665c0 0x7f1660068a80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1674144c80 0x7f167406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16740ac460 0x7f167413c380 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 --2- 172.21.15.31:0/3853676068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1674070df0 0x7f1674159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 -- 172.21.15.31:0/3853676068 >> 172.21.15.31:0/3853676068 conn(0x7f16740b0c40 msgr2=0x7f16740af790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:23.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 -- 172.21.15.31:0/3853676068 shutdown_connections 2023-11-26T07:22:23.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:23.865+0000 7f1679ee3640 1 -- 172.21.15.31:0/3853676068 wait complete. 2023-11-26T07:22:23.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:22:24.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0c232640 1 Processor -- start 2023-11-26T07:22:24.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0c232640 1 -- start start 2023-11-26T07:22:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0c232640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e040cded0 0x7f2e0413f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0c232640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e0413f7a0 0x7f2e040cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0c232640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0413cf40 con 0x7f2e04154180 2023-11-26T07:22:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0c232640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0413fc10 con 0x7f2e0413f7a0 2023-11-26T07:22:24.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0c232640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e040cf0a0 con 0x7f2e040cded0 2023-11-26T07:22:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0a7cf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e0413f7a0 0x7f2e040cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0a7cf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e0413f7a0 0x7f2e040cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37824/0 (socket says 172.21.15.31:37824) 2023-11-26T07:22:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0a7cf640 1 -- 172.21.15.31:0/1087118926 learned_addr learned my addr 172.21.15.31:0/1087118926 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e097cd640 1 --2- 172.21.15.31:0/1087118926 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e040cded0 0x7f2e0413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0a7cf640 1 -- 172.21.15.31:0/1087118926 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e040cded0 msgr2=0x7f2e0413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0a7cf640 1 --2- 172.21.15.31:0/1087118926 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e040cded0 0x7f2e0413f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0a7cf640 1 -- 172.21.15.31:0/1087118926 >> v1:172.21.15.97:6789/0 conn(0x7f2e04154180 legacy=0x7f2e0415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.033+0000 7f2e0a7cf640 1 -- 172.21.15.31:0/1087118926 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e041400e0 con 0x7f2e0413f7a0 2023-11-26T07:22:24.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e0a7cf640 1 --2- 172.21.15.31:0/1087118926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 0x7f2e040cbbb0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009e50 tx=0x7f2dfc01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0db0ea897d52865 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e08fcc640 1 -- 172.21.15.31:0/1087118926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0a56e0 con 0x7f2e0413f7a0 2023-11-26T07:22:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e08fcc640 1 -- 172.21.15.31:0/1087118926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc0a5840 con 0x7f2e0413f7a0 2023-11-26T07:22:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e08fcc640 1 -- 172.21.15.31:0/1087118926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0a5bd0 con 0x7f2e0413f7a0 2023-11-26T07:22:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e0c232640 1 -- 172.21.15.31:0/1087118926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 msgr2=0x7f2e040cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e0c232640 1 --2- 172.21.15.31:0/1087118926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 0x7f2e040cbbb0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc009e50 tx=0x7f2dfc01ca10 comp rx=0 tx=0).stop 2023-11-26T07:22:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e0c232640 1 -- 172.21.15.31:0/1087118926 shutdown_connections 2023-11-26T07:22:24.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e0c232640 1 --2- 172.21.15.31:0/1087118926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 0x7f2e040cbbb0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e0c232640 1 --2- 172.21.15.31:0/1087118926 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e040cded0 0x7f2e0413f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e0c232640 1 -- 172.21.15.31:0/1087118926 >> 172.21.15.31:0/1087118926 conn(0x7f2e04071d80 msgr2=0x7f2e0406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:24.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e0c232640 1 -- 172.21.15.31:0/1087118926 shutdown_connections 2023-11-26T07:22:24.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.037+0000 7f2e0c232640 1 -- 172.21.15.31:0/1087118926 wait complete. 2023-11-26T07:22:24.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 Processor -- start 2023-11-26T07:22:24.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 -- start start 2023-11-26T07:22:24.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e040cded0 0x7f2e04156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 0x7f2e041551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e04154180 0x7f2e041528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04157640 con 0x7f2e0413f7a0 2023-11-26T07:22:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04158e80 con 0x7f2e04154180 2023-11-26T07:22:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e04157360 con 0x7f2e040cded0 2023-11-26T07:22:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e097cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 0x7f2e041551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e097cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 0x7f2e041551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37838/0 (socket says 172.21.15.31:37838) 2023-11-26T07:22:24.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e097cd640 1 -- 172.21.15.31:0/118779961 learned_addr learned my addr 172.21.15.31:0/118779961 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e09fce640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e040cded0 0x7f2e04156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0a7cf640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e04154180 0x7f2e041528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e097cd640 1 -- 172.21.15.31:0/118779961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e040cded0 msgr2=0x7f2e04156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e097cd640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e040cded0 0x7f2e04156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e097cd640 1 -- 172.21.15.31:0/118779961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e04154180 msgr2=0x7f2e041528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e097cd640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e04154180 0x7f2e041528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e097cd640 1 -- 172.21.15.31:0/118779961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc009b30 con 0x7f2e0413f7a0 2023-11-26T07:22:24.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e097cd640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 0x7f2e041551c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2df400d800 tx=0x7f2df400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2dfa7fc640 1 -- 172.21.15.31:0/118779961 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df400b6c0 con 0x7f2e0413f7a0 2023-11-26T07:22:24.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2dfa7fc640 1 -- 172.21.15.31:0/118779961 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df400b820 con 0x7f2e0413f7a0 2023-11-26T07:22:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2dfa7fc640 1 -- 172.21.15.31:0/118779961 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df400bb90 con 0x7f2e0413f7a0 2023-11-26T07:22:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 -- 172.21.15.31:0/118779961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e04158830 con 0x7f2e0413f7a0 2023-11-26T07:22:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 -- 172.21.15.31:0/118779961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e04155700 con 0x7f2e0413f7a0 2023-11-26T07:22:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.041+0000 7f2e0c232640 1 -- 172.21.15.31:0/118779961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e04158690 con 0x7f2e0413f7a0 2023-11-26T07:22:24.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.045+0000 7f2dfa7fc640 1 -- 172.21.15.31:0/118779961 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f2df400bcf0 con 0x7f2e0413f7a0 2023-11-26T07:22:24.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.045+0000 7f2dfa7fc640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2de8066590 0x7f2de8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:24.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.045+0000 7f2dfa7fc640 1 -- 172.21.15.31:0/118779961 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2df4084410 con 0x7f2e0413f7a0 2023-11-26T07:22:24.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.045+0000 7f2e09fce640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2de8066590 0x7f2de8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:24.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.045+0000 7f2e09fce640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2de8066590 0x7f2de8068a50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e00007fd0 tx=0x7f2e00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:24.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.049+0000 7f2dfa7fc640 1 -- 172.21.15.31:0/118779961 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2e04158690 con 0x7f2e0413f7a0 2023-11-26T07:22:24.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:24.241+0000 7f2e0c232640 1 -- 172.21.15.31:0/118779961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2e04158690 con 0x7f2e0413f7a0 2023-11-26T07:22:25.118 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-11-26T07:22:25.117+0000 7fc4fb4d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:22:25.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.169+0000 7f2dfa7fc640 1 -- 172.21.15.31:0/118779961 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2e04158690 con 0x7f2e0413f7a0 2023-11-26T07:22:25.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 -- 172.21.15.31:0/118779961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2de8066590 msgr2=0x7f2de8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2de8066590 0x7f2de8068a50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e00007fd0 tx=0x7f2e00009040 comp rx=0 tx=0).stop 2023-11-26T07:22:25.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 -- 172.21.15.31:0/118779961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 msgr2=0x7f2e041551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 0x7f2e041551c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2df400d800 tx=0x7f2df400dcc0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 -- 172.21.15.31:0/118779961 shutdown_connections 2023-11-26T07:22:25.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2de8066590 0x7f2de8068a50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e04154180 0x7f2e041528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e0413f7a0 0x7f2e041551c0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 --2- 172.21.15.31:0/118779961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e040cded0 0x7f2e04156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 -- 172.21.15.31:0/118779961 >> 172.21.15.31:0/118779961 conn(0x7f2e04071d80 msgr2=0x7f2e0413d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:25.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 -- 172.21.15.31:0/118779961 shutdown_connections 2023-11-26T07:22:25.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.173+0000 7f2e0c232640 1 -- 172.21.15.31:0/118779961 wait complete. 2023-11-26T07:22:25.200 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:22:25.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:22:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dd1abb640 1 Processor -- start 2023-11-26T07:22:25.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dd1abb640 1 -- start start 2023-11-26T07:22:25.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dd1abb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3dcc06c960 0x7f3dcc0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dd1abb640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc1406c0 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dd1abb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dcc15dd20 con 0x7f3dcc13cf80 2023-11-26T07:22:25.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dd1abb640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dcc15e000 con 0x7f3dcc06c960 2023-11-26T07:22:25.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dd12ba640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3dcc06c960 0x7f3dcc0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dcbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3dcc13f8f0 0x7f3dcc06c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60460/0 (socket says 172.21.15.31:60460) 2023-11-26T07:22:25.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dcbfff640 1 -- 172.21.15.31:0/2101547447 learned_addr learned my addr 172.21.15.31:0/2101547447 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dd12ba640 1 -- 172.21.15.31:0/2101547447 >> v1:172.21.15.97:6789/0 conn(0x7f3dcc13cf80 legacy=0x7f3dcc13ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dd12ba640 1 -- 172.21.15.31:0/2101547447 >> v1:172.21.15.31:6789/0 conn(0x7f3dcc13f8f0 legacy=0x7f3dcc06c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.349+0000 7f3dd12ba640 1 -- 172.21.15.31:0/2101547447 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc15e2e0 con 0x7f3dcc06c960 2023-11-26T07:22:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dd12ba640 1 --2- 172.21.15.31:0/2101547447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc06c960 0x7f3dcc0ce760 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009d90 tx=0x7f3dc401ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fce9f81ad5a6654 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dcb7fe640 1 -- 172.21.15.31:0/2101547447 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40a7a30 con 0x7f3dcc06c960 2023-11-26T07:22:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dcb7fe640 1 -- 172.21.15.31:0/2101547447 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc40a7b90 con 0x7f3dcc06c960 2023-11-26T07:22:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dcb7fe640 1 -- 172.21.15.31:0/2101547447 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40af440 con 0x7f3dcc06c960 2023-11-26T07:22:25.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dd1abb640 1 -- 172.21.15.31:0/2101547447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc06c960 msgr2=0x7f3dcc0ce760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dd1abb640 1 --2- 172.21.15.31:0/2101547447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc06c960 0x7f3dcc0ce760 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009d90 tx=0x7f3dc401ea60 comp rx=0 tx=0).stop 2023-11-26T07:22:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dd1abb640 1 -- 172.21.15.31:0/2101547447 shutdown_connections 2023-11-26T07:22:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dd1abb640 1 --2- 172.21.15.31:0/2101547447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc06c960 0x7f3dcc0ce760 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dd1abb640 1 -- 172.21.15.31:0/2101547447 >> 172.21.15.31:0/2101547447 conn(0x7f3dcc06df30 msgr2=0x7f3dcc06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dd1abb640 1 -- 172.21.15.31:0/2101547447 shutdown_connections 2023-11-26T07:22:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.353+0000 7f3dd1abb640 1 -- 172.21.15.31:0/2101547447 wait complete. 2023-11-26T07:22:25.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd1abb640 1 Processor -- start 2023-11-26T07:22:25.360 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd1abb640 1 -- start start 2023-11-26T07:22:25.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd1abb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3dcc06c960 0x7f3dcc13f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd1abb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dcc13cf80 0x7f3dcc136970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd1abb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc13f8f0 0x7f3dcc1378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd1abb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc136eb0 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd1abb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc137e00 con 0x7f3dcc13cf80 2023-11-26T07:22:25.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd1abb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc138c40 con 0x7f3dcc06c960 2023-11-26T07:22:25.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd12ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc13f8f0 0x7f3dcc1378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd12ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc13f8f0 0x7f3dcc1378c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37868/0 (socket says 172.21.15.31:37868) 2023-11-26T07:22:25.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd12ba640 1 -- 172.21.15.31:0/65466904 learned_addr learned my addr 172.21.15.31:0/65466904 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dcbfff640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dcc13cf80 0x7f3dcc136970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd0ab9640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3dcc06c960 0x7f3dcc13f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd12ba640 1 -- 172.21.15.31:0/65466904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3dcc06c960 msgr2=0x7f3dcc13f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd12ba640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3dcc06c960 0x7f3dcc13f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd12ba640 1 -- 172.21.15.31:0/65466904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dcc13cf80 msgr2=0x7f3dcc136970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd12ba640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dcc13cf80 0x7f3dcc136970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd12ba640 1 -- 172.21.15.31:0/65466904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc4009a70 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dcbfff640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dcc13cf80 0x7f3dcc136970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd12ba640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc13f8f0 0x7f3dcc1378c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40b2040 tx=0x7f3dc401b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dc97fa640 1 -- 172.21.15.31:0/65466904 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40a7a30 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dc97fa640 1 -- 172.21.15.31:0/65466904 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc40afbc0 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd1abb640 1 -- 172.21.15.31:0/65466904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc138030 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dc97fa640 1 -- 172.21.15.31:0/65466904 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40ba470 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.357+0000 7f3dd1abb640 1 -- 172.21.15.31:0/65466904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc1385f0 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.361+0000 7f3dd1abb640 1 -- 172.21.15.31:0/65466904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc067a40 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.361+0000 7f3dc97fa640 1 -- 172.21.15.31:0/65466904 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f3dc40b6070 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.361+0000 7f3dc97fa640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3db0066590 0x7f3db0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:25.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.361+0000 7f3dc97fa640 1 -- 172.21.15.31:0/65466904 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dc411f600 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.361+0000 7f3dd0ab9640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3db0066590 0x7f3db0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:25.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.361+0000 7f3dd0ab9640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3db0066590 0x7f3db0068a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc154a70 tx=0x7f3dc0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:25.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.365+0000 7f3dc97fa640 1 -- 172.21.15.31:0/65466904 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3dc40ec960 con 0x7f3dcc13f8f0 2023-11-26T07:22:25.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:25.557+0000 7f3dd1abb640 1 -- 172.21.15.31:0/65466904 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3dcc067a40 con 0x7f3dcc13f8f0 2023-11-26T07:22:26.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.177+0000 7f3dc97fa640 1 -- 172.21.15.31:0/65466904 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3dc40eff80 con 0x7f3dcc13f8f0 2023-11-26T07:22:26.182 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 -- 172.21.15.31:0/65466904 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3db0066590 msgr2=0x7f3db0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3db0066590 0x7f3db0068a50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc154a70 tx=0x7f3dc0009040 comp rx=0 tx=0).stop 2023-11-26T07:22:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 -- 172.21.15.31:0/65466904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc13f8f0 msgr2=0x7f3dcc1378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc13f8f0 0x7f3dcc1378c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40b2040 tx=0x7f3dc401b800 comp rx=0 tx=0).stop 2023-11-26T07:22:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 -- 172.21.15.31:0/65466904 shutdown_connections 2023-11-26T07:22:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3db0066590 0x7f3db0068a50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dcc13f8f0 0x7f3dcc1378c0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3dcc13cf80 0x7f3dcc136970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 --2- 172.21.15.31:0/65466904 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3dcc06c960 0x7f3dcc13f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 -- 172.21.15.31:0/65466904 >> 172.21.15.31:0/65466904 conn(0x7f3dcc06df30 msgr2=0x7f3dcc06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 -- 172.21.15.31:0/65466904 shutdown_connections 2023-11-26T07:22:26.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.181+0000 7f3dd1abb640 1 -- 172.21.15.31:0/65466904 wait complete. 2023-11-26T07:22:26.205 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:22:26.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:26.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701e4ea640 1 Processor -- start 2023-11-26T07:22:26.351 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701e4ea640 1 -- start start 2023-11-26T07:22:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701e4ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70180cead0 0x7f70180d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701e4ea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f70180d1eb0 0x7f701806bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701e4ea640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70180cefd0 con 0x7f701815d7b0 2023-11-26T07:22:26.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701e4ea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70180d17e0 con 0x7f70180d1eb0 2023-11-26T07:22:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701e4ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70180d2340 con 0x7f70180cead0 2023-11-26T07:22:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f7017fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f701815d7b0 0x7f70180d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60476/0 (socket says 172.21.15.31:60476) 2023-11-26T07:22:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f7017fff640 1 -- 172.21.15.31:0/71021635 learned_addr learned my addr 172.21.15.31:0/71021635 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701ca87640 1 --2- 172.21.15.31:0/71021635 >> v2:172.21.15.97:3300/0 conn(0x7f70180d1eb0 0x7f701806bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f70177fe640 1 --2- 172.21.15.31:0/71021635 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70180cead0 0x7f70180d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f7016ffd640 1 -- 172.21.15.31:0/71021635 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823069676 0 0) 0x7f70180cefd0 con 0x7f701815d7b0 2023-11-26T07:22:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f7016ffd640 1 -- 172.21.15.31:0/71021635 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7000003530 con 0x7f701815d7b0 2023-11-26T07:22:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f7016ffd640 1 -- 172.21.15.31:0/71021635 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3882947750 0 0) 0x7f7000003530 con 0x7f701815d7b0 2023-11-26T07:22:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701ca87640 1 -- 172.21.15.31:0/71021635 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70180cead0 msgr2=0x7f70180d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:26.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701ca87640 1 --2- 172.21.15.31:0/71021635 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70180cead0 0x7f70180d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701ca87640 1 -- 172.21.15.31:0/71021635 >> v1:172.21.15.31:6789/0 conn(0x7f701815d7b0 legacy=0x7f70180d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f701ca87640 1 -- 172.21.15.31:0/71021635 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180cf2b0 con 0x7f70180d1eb0 2023-11-26T07:22:26.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.349+0000 7f70177fe640 1 --2- 172.21.15.31:0/71021635 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70180cead0 0x7f70180d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:26.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f701ca87640 1 --2- 172.21.15.31:0/71021635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70180d1eb0 0x7f701806bed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7010009d90 tx=0x7f701001c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aac1b3e27b0e685 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:26.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f7016ffd640 1 -- 172.21.15.31:0/71021635 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008001680 con 0x7f70180d1eb0 2023-11-26T07:22:26.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f7016ffd640 1 -- 172.21.15.31:0/71021635 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70100a5520 con 0x7f70180d1eb0 2023-11-26T07:22:26.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f7016ffd640 1 -- 172.21.15.31:0/71021635 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70100a58b0 con 0x7f70180d1eb0 2023-11-26T07:22:26.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f701e4ea640 1 -- 172.21.15.31:0/71021635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70180d1eb0 msgr2=0x7f701806bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f701e4ea640 1 --2- 172.21.15.31:0/71021635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70180d1eb0 0x7f701806bed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7010009d90 tx=0x7f701001c720 comp rx=0 tx=0).stop 2023-11-26T07:22:26.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f701e4ea640 1 -- 172.21.15.31:0/71021635 shutdown_connections 2023-11-26T07:22:26.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f701e4ea640 1 --2- 172.21.15.31:0/71021635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70180d1eb0 0x7f701806bed0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f701e4ea640 1 --2- 172.21.15.31:0/71021635 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70180cead0 0x7f70180d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f701e4ea640 1 -- 172.21.15.31:0/71021635 >> 172.21.15.31:0/71021635 conn(0x7f70180d33b0 msgr2=0x7f70180d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:26.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f701e4ea640 1 -- 172.21.15.31:0/71021635 shutdown_connections 2023-11-26T07:22:26.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.353+0000 7f701e4ea640 1 -- 172.21.15.31:0/71021635 wait complete. 2023-11-26T07:22:26.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701e4ea640 1 Processor -- start 2023-11-26T07:22:26.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701e4ea640 1 -- start start 2023-11-26T07:22:26.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701e4ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70180cead0 0x7f70180b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701e4ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70180d1eb0 0x7f70180cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701e4ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f701815d7b0 0x7f70180b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701e4ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70180c9d90 con 0x7f70180d1eb0 2023-11-26T07:22:26.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701e4ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70180cc6d0 con 0x7f70180cead0 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701e4ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70180cc9b0 con 0x7f701815d7b0 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f70177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70180d1eb0 0x7f70180cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f70177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70180d1eb0 0x7f70180cd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37874/0 (socket says 172.21.15.31:37874) 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f70177fe640 1 -- 172.21.15.31:0/4182810791 learned_addr learned my addr 172.21.15.31:0/4182810791 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701ca87640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f701815d7b0 0x7f70180b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f7017fff640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70180cead0 0x7f70180b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f70177fe640 1 -- 172.21.15.31:0/4182810791 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f701815d7b0 msgr2=0x7f70180b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f70177fe640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f701815d7b0 0x7f70180b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f70177fe640 1 -- 172.21.15.31:0/4182810791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70180cead0 msgr2=0x7f70180b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f70177fe640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70180cead0 0x7f70180b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f70177fe640 1 -- 172.21.15.31:0/4182810791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c009420 con 0x7f70180d1eb0 2023-11-26T07:22:26.362 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f70177fe640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70180d1eb0 0x7f70180cd1b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f700c00ebe0 tx=0x7f700c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:26.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f7014ff9640 1 -- 172.21.15.31:0/4182810791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c00eee0 con 0x7f70180d1eb0 2023-11-26T07:22:26.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f7014ff9640 1 -- 172.21.15.31:0/4182810791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c00cc30 con 0x7f70180d1eb0 2023-11-26T07:22:26.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f7014ff9640 1 -- 172.21.15.31:0/4182810791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c013470 con 0x7f70180d1eb0 2023-11-26T07:22:26.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701e4ea640 1 -- 172.21.15.31:0/4182810791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7010009a70 con 0x7f70180d1eb0 2023-11-26T07:22:26.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.357+0000 7f701e4ea640 1 -- 172.21.15.31:0/4182810791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70180ccd40 con 0x7f70180d1eb0 2023-11-26T07:22:26.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.361+0000 7f701e4ea640 1 -- 172.21.15.31:0/4182810791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701804e850 con 0x7f70180d1eb0 2023-11-26T07:22:26.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.361+0000 7f7014ff9640 1 -- 172.21.15.31:0/4182810791 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f700c0135d0 con 0x7f70180d1eb0 2023-11-26T07:22:26.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.361+0000 7f7014ff9640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 0x7f7004068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.361+0000 7f7014ff9640 1 -- 172.21.15.31:0/4182810791 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f700c084350 con 0x7f70180d1eb0 2023-11-26T07:22:26.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.361+0000 7f7017fff640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 0x7f7004068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.361+0000 7f7017fff640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 0x7f7004068a50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7008001b00 tx=0x7f7008009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:26.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.365+0000 7f7014ff9640 1 -- 172.21.15.31:0/4182810791 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f700c0516b0 con 0x7f70180d1eb0 2023-11-26T07:22:26.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 -- 172.21.15.31:0/4182810791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70180c9ed0 con 0x7f70180d1eb0 2023-11-26T07:22:26.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f7014ff9640 1 -- 172.21.15.31:0/4182810791 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f700c054cd0 con 0x7f70180d1eb0 2023-11-26T07:22:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:26.560 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1684629063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1684629063},{"type":"v1","addr":"172.21.15.31:6833","nonce":1684629063}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/1262063204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1262063204},{"type":"v1","addr":"172.21.15.97:6834","nonce":1262063204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/683007803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":683007803},{"type":"v1","addr":"172.21.15.97:6835","nonce":683007803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:22.526776+0000","modified":"2023-11-26T07:22:25.559499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7101":{"gid":7101,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3729395637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3729395637},{"type":"v1","addr":"172.21.15.31:6835","nonce":3729395637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:22:26.560 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 57 2023-11-26T07:22:26.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 -- 172.21.15.31:0/4182810791 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 msgr2=0x7f7004068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 0x7f7004068a50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7008001b00 tx=0x7f7008009040 comp rx=0 tx=0).stop 2023-11-26T07:22:26.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 -- 172.21.15.31:0/4182810791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70180d1eb0 msgr2=0x7f70180cd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70180d1eb0 0x7f70180cd1b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f700c00ebe0 tx=0x7f700c00c520 comp rx=0 tx=0).stop 2023-11-26T07:22:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 -- 172.21.15.31:0/4182810791 shutdown_connections 2023-11-26T07:22:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 0x7f7004068a50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f701815d7b0 0x7f70180b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70180d1eb0 0x7f70180cd1b0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 --2- 172.21.15.31:0/4182810791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70180cead0 0x7f70180b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 -- 172.21.15.31:0/4182810791 >> 172.21.15.31:0/4182810791 conn(0x7f70180d33b0 msgr2=0x7f70180d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 -- 172.21.15.31:0/4182810791 shutdown_connections 2023-11-26T07:22:26.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.557+0000 7f701e4ea640 1 -- 172.21.15.31:0/4182810791 wait complete. 2023-11-26T07:22:26.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:26.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7ade4ea640 1 Processor -- start 2023-11-26T07:22:26.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7ade4ea640 1 -- start start 2023-11-26T07:22:26.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7ade4ea640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7ad80d2080 0x7f7ad8168160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7ade4ea640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad80cefd0 con 0x7f7ad815d7b0 2023-11-26T07:22:26.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7ade4ea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad80cf2b0 con 0x7f7ad80cead0 2023-11-26T07:22:26.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7ade4ea640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad80d17e0 con 0x7f7ad80d2080 2023-11-26T07:22:26.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7ad7fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7ad815d7b0 0x7f7ad80d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60488/0 (socket says 172.21.15.31:60488) 2023-11-26T07:22:26.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7ad7fff640 1 -- 172.21.15.31:0/2433949538 learned_addr learned my addr 172.21.15.31:0/2433949538 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:26.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7adca87640 1 --2- 172.21.15.31:0/2433949538 >> v2:172.21.15.31:3300/0 conn(0x7f7ad80d2080 0x7f7ad8168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7ad6ffd640 1 -- 172.21.15.31:0/2433949538 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410305872 0 0) 0x7f7ad80cefd0 con 0x7f7ad815d7b0 2023-11-26T07:22:26.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7ad6ffd640 1 -- 172.21.15.31:0/2433949538 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ac0003530 con 0x7f7ad815d7b0 2023-11-26T07:22:26.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7adca87640 1 -- 172.21.15.31:0/2433949538 >> v1:172.21.15.97:6789/0 conn(0x7f7ad80cead0 legacy=0x7f7ad80d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7adca87640 1 -- 172.21.15.31:0/2433949538 >> v1:172.21.15.31:6789/0 conn(0x7f7ad815d7b0 legacy=0x7f7ad80d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.721+0000 7f7adca87640 1 -- 172.21.15.31:0/2433949538 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad806b3f0 con 0x7f7ad80d2080 2023-11-26T07:22:26.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7adca87640 1 --2- 172.21.15.31:0/2433949538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad80d2080 0x7f7ad8168160 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009d90 tx=0x7f7ad001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79457b6a1c3ef0ed server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:26.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7ad6ffd640 1 -- 172.21.15.31:0/2433949538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00a58c0 con 0x7f7ad80d2080 2023-11-26T07:22:26.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7ad6ffd640 1 -- 172.21.15.31:0/2433949538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad00a5a20 con 0x7f7ad80d2080 2023-11-26T07:22:26.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7ad6ffd640 1 -- 172.21.15.31:0/2433949538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00a5db0 con 0x7f7ad80d2080 2023-11-26T07:22:26.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2433949538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad80d2080 msgr2=0x7f7ad8168160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7ade4ea640 1 --2- 172.21.15.31:0/2433949538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad80d2080 0x7f7ad8168160 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009d90 tx=0x7f7ad001c800 comp rx=0 tx=0).stop 2023-11-26T07:22:26.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2433949538 shutdown_connections 2023-11-26T07:22:26.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7ade4ea640 1 --2- 172.21.15.31:0/2433949538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad80d2080 0x7f7ad8168160 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2433949538 >> 172.21.15.31:0/2433949538 conn(0x7f7ad80d33b0 msgr2=0x7f7ad80d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:26.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2433949538 shutdown_connections 2023-11-26T07:22:26.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.725+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2433949538 wait complete. 2023-11-26T07:22:26.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ade4ea640 1 Processor -- start 2023-11-26T07:22:26.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ade4ea640 1 -- start start 2023-11-26T07:22:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ade4ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad80cead0 0x7f7ad8153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ade4ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ad80d2080 0x7f7ad80c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ade4ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad815d7b0 0x7f7ad81529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ade4ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad81553a0 con 0x7f7ad815d7b0 2023-11-26T07:22:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ade4ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad80c9d20 con 0x7f7ad80cead0 2023-11-26T07:22:26.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ade4ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad8154ab0 con 0x7f7ad80d2080 2023-11-26T07:22:26.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7adca87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad815d7b0 0x7f7ad81529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7adca87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad815d7b0 0x7f7ad81529a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37894/0 (socket says 172.21.15.31:37894) 2023-11-26T07:22:26.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7adca87640 1 -- 172.21.15.31:0/2863262158 learned_addr learned my addr 172.21.15.31:0/2863262158 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:26.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ad7fff640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad80cead0 0x7f7ad8153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ad77fe640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ad80d2080 0x7f7ad80c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7adca87640 1 -- 172.21.15.31:0/2863262158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ad80d2080 msgr2=0x7f7ad80c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7adca87640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ad80d2080 0x7f7ad80c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7adca87640 1 -- 172.21.15.31:0/2863262158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad80cead0 msgr2=0x7f7ad8153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7adca87640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad80cead0 0x7f7ad8153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7adca87640 1 -- 172.21.15.31:0/2863262158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad0009a70 con 0x7f7ad815d7b0 2023-11-26T07:22:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ad77fe640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ad80d2080 0x7f7ad80c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7adca87640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad815d7b0 0x7f7ad81529a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00b0040 tx=0x7f7ad00195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ad4ff9640 1 -- 172.21.15.31:0/2863262158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00b9860 con 0x7f7ad815d7b0 2023-11-26T07:22:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ad4ff9640 1 -- 172.21.15.31:0/2863262158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad00b99c0 con 0x7f7ad815d7b0 2023-11-26T07:22:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2863262158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad8154d50 con 0x7f7ad815d7b0 2023-11-26T07:22:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ad4ff9640 1 -- 172.21.15.31:0/2863262158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00b9d70 con 0x7f7ad815d7b0 2023-11-26T07:22:26.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.729+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2863262158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad81538f0 con 0x7f7ad815d7b0 2023-11-26T07:22:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.733+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2863262158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad8155510 con 0x7f7ad815d7b0 2023-11-26T07:22:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.733+0000 7f7ad4ff9640 1 -- 172.21.15.31:0/2863262158 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f7ad00ad440 con 0x7f7ad815d7b0 2023-11-26T07:22:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.733+0000 7f7ad4ff9640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7ac4066590 0x7f7ac4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:26.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.733+0000 7f7ad4ff9640 1 -- 172.21.15.31:0/2863262158 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ad011f2d0 con 0x7f7ad815d7b0 2023-11-26T07:22:26.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.733+0000 7f7ad7fff640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7ac4066590 0x7f7ac4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:26.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.733+0000 7f7ad7fff640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7ac4066590 0x7f7ac4068a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80011f0 tx=0x7f7ac8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.737+0000 7f7ad4ff9640 1 -- 172.21.15.31:0/2863262158 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7ad00ec630 con 0x7f7ad815d7b0 2023-11-26T07:22:26.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.897+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2863262158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ad804e850 con 0x7f7ad815d7b0 2023-11-26T07:22:26.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.897+0000 7f7ad4ff9640 1 -- 172.21.15.31:0/2863262158 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16476 (secure 0 0 0) 0x7f7ad00efc50 con 0x7f7ad815d7b0 2023-11-26T07:22:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:26.902 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":80,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:24.134741+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:20.941587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:21.520554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:26.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2863262158 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7ac4066590 msgr2=0x7f7ac4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7ac4066590 0x7f7ac4068a50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80011f0 tx=0x7f7ac8009040 comp rx=0 tx=0).stop 2023-11-26T07:22:26.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2863262158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad815d7b0 msgr2=0x7f7ad81529a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:26.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad815d7b0 0x7f7ad81529a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00b0040 tx=0x7f7ad00195a0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2863262158 shutdown_connections 2023-11-26T07:22:26.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7ac4066590 0x7f7ac4068a50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ad815d7b0 0x7f7ad81529a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ad80d2080 0x7f7ad80c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 --2- 172.21.15.31:0/2863262158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ad80cead0 0x7f7ad8153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:26.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2863262158 >> 172.21.15.31:0/2863262158 conn(0x7f7ad80d33b0 msgr2=0x7f7ad80d1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:26.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2863262158 shutdown_connections 2023-11-26T07:22:26.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:26.901+0000 7f7ade4ea640 1 -- 172.21.15.31:0/2863262158 wait complete. 2023-11-26T07:22:26.923 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:22:26.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:22:27.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5688479640 1 Processor -- start 2023-11-26T07:22:27.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5688479640 1 -- start start 2023-11-26T07:22:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5688479640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f56800cead0 0x7f56800d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5688479640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56800d22c0 con 0x7f568015d7b0 2023-11-26T07:22:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5688479640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56800cefd0 con 0x7f56800d1eb0 2023-11-26T07:22:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5688479640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56800d17e0 con 0x7f56800cead0 2023-11-26T07:22:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5686215640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f568015d7b0 0x7f56800d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60492/0 (socket says 172.21.15.31:60492) 2023-11-26T07:22:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5686215640 1 -- 172.21.15.31:0/1700675597 learned_addr learned my addr 172.21.15.31:0/1700675597 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5685a14640 1 --2- 172.21.15.31:0/1700675597 >> v2:172.21.15.97:3300/0 conn(0x7f56800cead0 0x7f56800d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5685213640 1 -- 172.21.15.31:0/1700675597 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344838695 0 0) 0x7f56800d22c0 con 0x7f568015d7b0 2023-11-26T07:22:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5685213640 1 -- 172.21.15.31:0/1700675597 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f566c003530 con 0x7f568015d7b0 2023-11-26T07:22:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5685213640 1 -- 172.21.15.31:0/1700675597 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188579633 0 0) 0x7f56800cefd0 con 0x7f56800d1eb0 2023-11-26T07:22:27.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5685213640 1 -- 172.21.15.31:0/1700675597 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56800d22c0 con 0x7f56800d1eb0 2023-11-26T07:22:27.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5685a14640 1 -- 172.21.15.31:0/1700675597 >> v1:172.21.15.97:6789/0 conn(0x7f56800d1eb0 legacy=0x7f568006bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5685a14640 1 -- 172.21.15.31:0/1700675597 >> v1:172.21.15.31:6789/0 conn(0x7f568015d7b0 legacy=0x7f56800d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.065+0000 7f5685a14640 1 -- 172.21.15.31:0/1700675597 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56800cf2b0 con 0x7f56800cead0 2023-11-26T07:22:27.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.069+0000 7f5685a14640 1 --2- 172.21.15.31:0/1700675597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56800cead0 0x7f56800d1970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f56700097c0 tx=0x7f567001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f25e112c34a69b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.075 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5685213640 1 -- 172.21.15.31:0/1700675597 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56700ab600 con 0x7f56800cead0 2023-11-26T07:22:27.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5685213640 1 -- 172.21.15.31:0/1700675597 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56700a7050 con 0x7f56800cead0 2023-11-26T07:22:27.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5685213640 1 -- 172.21.15.31:0/1700675597 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56700ab980 con 0x7f56800cead0 2023-11-26T07:22:27.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5688479640 1 -- 172.21.15.31:0/1700675597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56800cead0 msgr2=0x7f56800d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5688479640 1 --2- 172.21.15.31:0/1700675597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56800cead0 0x7f56800d1970 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f56700097c0 tx=0x7f567001c330 comp rx=0 tx=0).stop 2023-11-26T07:22:27.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5688479640 1 -- 172.21.15.31:0/1700675597 shutdown_connections 2023-11-26T07:22:27.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5688479640 1 --2- 172.21.15.31:0/1700675597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56800cead0 0x7f56800d1970 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5688479640 1 -- 172.21.15.31:0/1700675597 >> 172.21.15.31:0/1700675597 conn(0x7f56800d33b0 msgr2=0x7f56800d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:27.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5688479640 1 -- 172.21.15.31:0/1700675597 shutdown_connections 2023-11-26T07:22:27.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5688479640 1 -- 172.21.15.31:0/1700675597 wait complete. 2023-11-26T07:22:27.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5688479640 1 Processor -- start 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.073+0000 7f5688479640 1 -- start start 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5688479640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56800cead0 0x7f56800ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5688479640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56800d1eb0 0x7f56800c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5688479640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568015d7b0 0x7f56800cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5688479640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56800cab00 con 0x7f56800d1eb0 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5688479640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56800b7cb0 con 0x7f568015d7b0 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5688479640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56800b7f90 con 0x7f56800cead0 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5685a14640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56800d1eb0 0x7f56800c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5685a14640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56800d1eb0 0x7f56800c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37904/0 (socket says 172.21.15.31:37904) 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5685a14640 1 -- 172.21.15.31:0/2429186754 learned_addr learned my addr 172.21.15.31:0/2429186754 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:27.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5686215640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56800cead0 0x7f56800ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5686a16640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568015d7b0 0x7f56800cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5685a14640 1 -- 172.21.15.31:0/2429186754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56800cead0 msgr2=0x7f56800ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5685a14640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56800cead0 0x7f56800ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5685a14640 1 -- 172.21.15.31:0/2429186754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568015d7b0 msgr2=0x7f56800cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5685a14640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568015d7b0 0x7f56800cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5685a14640 1 -- 172.21.15.31:0/2429186754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5670009420 con 0x7f56800d1eb0 2023-11-26T07:22:27.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5685a14640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56800d1eb0 0x7f56800c9760 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f56700097c0 tx=0x7f56700190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5676ffd640 1 -- 172.21.15.31:0/2429186754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56700abe50 con 0x7f56800d1eb0 2023-11-26T07:22:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5676ffd640 1 -- 172.21.15.31:0/2429186754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5670004020 con 0x7f56800d1eb0 2023-11-26T07:22:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5688479640 1 -- 172.21.15.31:0/2429186754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56800b8270 con 0x7f56800d1eb0 2023-11-26T07:22:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5676ffd640 1 -- 172.21.15.31:0/2429186754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56700043d0 con 0x7f56800d1eb0 2023-11-26T07:22:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5688479640 1 -- 172.21.15.31:0/2429186754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56800b7990 con 0x7f56800d1eb0 2023-11-26T07:22:27.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.077+0000 7f5688479640 1 -- 172.21.15.31:0/2429186754 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568004e850 con 0x7f56800d1eb0 2023-11-26T07:22:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.081+0000 7f5676ffd640 1 -- 172.21.15.31:0/2429186754 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f56700b2040 con 0x7f56800d1eb0 2023-11-26T07:22:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.081+0000 7f5676ffd640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5664066590 0x7f5664068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.081+0000 7f5676ffd640 1 -- 172.21.15.31:0/2429186754 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f567011dea0 con 0x7f56800d1eb0 2023-11-26T07:22:27.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.081+0000 7f5686215640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5664066590 0x7f5664068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.081+0000 7f5686215640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5664066590 0x7f5664068a50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f567c000f70 tx=0x7f567c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.085+0000 7f5676ffd640 1 -- 172.21.15.31:0/2429186754 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f56700eb190 con 0x7f56800d1eb0 2023-11-26T07:22:27.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.241+0000 7f5688479640 1 -- 172.21.15.31:0/2429186754 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56800b7e20 con 0x7f5664066590 2023-11-26T07:22:27.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.253+0000 7f5676ffd640 1 -- 172.21.15.31:0/2429186754 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283525 (secure 0 0 0) 0x7f56800b7e20 con 0x7f5664066590 2023-11-26T07:22:27.257 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:22:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 -- 172.21.15.31:0/2429186754 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5664066590 msgr2=0x7f5664068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5664066590 0x7f5664068a50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f567c000f70 tx=0x7f567c009040 comp rx=0 tx=0).stop 2023-11-26T07:22:27.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 -- 172.21.15.31:0/2429186754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56800d1eb0 msgr2=0x7f56800c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56800d1eb0 0x7f56800c9760 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f56700097c0 tx=0x7f56700190d0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 -- 172.21.15.31:0/2429186754 shutdown_connections 2023-11-26T07:22:27.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5664066590 0x7f5664068a50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f568015d7b0 0x7f56800cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56800d1eb0 0x7f56800c9760 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 --2- 172.21.15.31:0/2429186754 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f56800cead0 0x7f56800ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 -- 172.21.15.31:0/2429186754 >> 172.21.15.31:0/2429186754 conn(0x7f56800d33b0 msgr2=0x7f56800b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:27.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 -- 172.21.15.31:0/2429186754 shutdown_connections 2023-11-26T07:22:27.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.257+0000 7f5688479640 1 -- 172.21.15.31:0/2429186754 wait complete. 2023-11-26T07:22:27.278 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-26T07:22:26.200917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258048,"kb_used_data":8548,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716672,"statfs":{"total":773094113280,"available":772829872128,"internally_reserved":0,"allocated":8753152,"data_stored":4393893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.080419"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131043+0000","last_change":"2023-11-26T07:22:22.131043+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131043+0000","last_undegraded":"2023-11-26T07:22:22.131043+0000","last_fullsized":"2023-11-26T07:22:22.131043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129528+0000","last_change":"2023-11-26T07:22:22.129528+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129528+0000","last_undegraded":"2023-11-26T07:22:22.129528+0000","last_fullsized":"2023-11-26T07:22:22.129528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.133354+0000","last_change":"2023-11-26T07:22:22.133354+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.133354+0000","last_undegraded":"2023-11-26T07:22:22.133354+0000","last_fullsized":"2023-11-26T07:22:22.133354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135222+0000","last_change":"2023-11-26T07:22:22.135222+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135222+0000","last_undegraded":"2023-11-26T07:22:22.135222+0000","last_fullsized":"2023-11-26T07:22:22.135222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.143110+0000","last_change":"2023-11-26T07:22:22.143110+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.143110+0000","last_undegraded":"2023-11-26T07:22:22.143110+0000","last_fullsized":"2023-11-26T07:22:22.143110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130504+0000","last_change":"2023-11-26T07:22:22.130504+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130504+0000","last_undegraded":"2023-11-26T07:22:22.130504+0000","last_fullsized":"2023-11-26T07:22:22.130504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130438+0000","last_change":"2023-11-26T07:22:22.130438+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130438+0000","last_undegraded":"2023-11-26T07:22:22.130438+0000","last_fullsized":"2023-11-26T07:22:22.130438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130281+0000","last_change":"2023-11-26T07:22:22.130281+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130281+0000","last_undegraded":"2023-11-26T07:22:22.130281+0000","last_fullsized":"2023-11-26T07:22:22.130281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131416+0000","last_change":"2023-11-26T07:22:22.131416+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131416+0000","last_undegraded":"2023-11-26T07:22:22.131416+0000","last_fullsized":"2023-11-26T07:22:22.131416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131033+0000","last_change":"2023-11-26T07:22:22.131033+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131033+0000","last_undegraded":"2023-11-26T07:22:22.131033+0000","last_fullsized":"2023-11-26T07:22:22.131033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135119+0000","last_change":"2023-11-26T07:22:22.135119+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135119+0000","last_undegraded":"2023-11-26T07:22:22.135119+0000","last_fullsized":"2023-11-26T07:22:22.135119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127615+0000","last_change":"2023-11-26T07:22:22.127615+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127615+0000","last_undegraded":"2023-11-26T07:22:22.127615+0000","last_fullsized":"2023-11-26T07:22:22.127615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135357+0000","last_change":"2023-11-26T07:22:22.135357+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135357+0000","last_undegraded":"2023-11-26T07:22:22.135357+0000","last_fullsized":"2023-11-26T07:22:22.135357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132827+0000","last_change":"2023-11-26T07:22:22.132827+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132827+0000","last_undegraded":"2023-11-26T07:22:22.132827+0000","last_fullsized":"2023-11-26T07:22:22.132827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129520+0000","last_change":"2023-11-26T07:22:22.129520+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129520+0000","last_undegraded":"2023-11-26T07:22:22.129520+0000","last_fullsized":"2023-11-26T07:22:22.129520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.142126+0000","last_change":"2023-11-26T07:22:22.142126+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.142126+0000","last_undegraded":"2023-11-26T07:22:22.142126+0000","last_fullsized":"2023-11-26T07:22:22.142126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139694+0000","last_change":"2023-11-26T07:22:22.139694+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139694+0000","last_undegraded":"2023-11-26T07:22:22.139694+0000","last_fullsized":"2023-11-26T07:22:22.139694+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134761+0000","last_change":"2023-11-26T07:22:22.134761+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134761+0000","last_undegraded":"2023-11-26T07:22:22.134761+0000","last_fullsized":"2023-11-26T07:22:22.134761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.138624+0000","last_change":"2023-11-26T07:22:22.138624+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.138624+0000","last_undegraded":"2023-11-26T07:22:22.138624+0000","last_fullsized":"2023-11-26T07:22:22.138624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134691+0000","last_change":"2023-11-26T07:22:22.134691+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134691+0000","last_undegraded":"2023-11-26T07:22:22.134691+0000","last_fullsized":"2023-11-26T07:22:22.134691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131538+0000","last_change":"2023-11-26T07:22:22.131538+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131538+0000","last_undegraded":"2023-11-26T07:22:22.131538+0000","last_fullsized":"2023-11-26T07:22:22.131538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129384+0000","last_change":"2023-11-26T07:22:22.129384+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129384+0000","last_undegraded":"2023-11-26T07:22:22.129384+0000","last_fullsized":"2023-11-26T07:22:22.129384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127661+0000","last_change":"2023-11-26T07:22:22.127661+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127661+0000","last_undegraded":"2023-11-26T07:22:22.127661+0000","last_fullsized":"2023-11-26T07:22:22.127661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131711+0000","last_change":"2023-11-26T07:22:22.131711+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131711+0000","last_undegraded":"2023-11-26T07:22:22.131711+0000","last_fullsized":"2023-11-26T07:22:22.131711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134720+0000","last_change":"2023-11-26T07:22:22.134720+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134720+0000","last_undegraded":"2023-11-26T07:22:22.134720+0000","last_fullsized":"2023-11-26T07:22:22.134720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134643+0000","last_change":"2023-11-26T07:22:22.134643+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134643+0000","last_undegraded":"2023-11-26T07:22:22.134643+0000","last_fullsized":"2023-11-26T07:22:22.134643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.143843+0000","last_change":"2023-11-26T07:22:22.143843+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.143843+0000","last_undegraded":"2023-11-26T07:22:22.143843+0000","last_fullsized":"2023-11-26T07:22:22.143843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130684+0000","last_change":"2023-11-26T07:22:22.130684+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130684+0000","last_undegraded":"2023-11-26T07:22:22.130684+0000","last_fullsized":"2023-11-26T07:22:22.130684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128838+0000","last_change":"2023-11-26T07:22:22.128838+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128838+0000","last_undegraded":"2023-11-26T07:22:22.128838+0000","last_fullsized":"2023-11-26T07:22:22.128838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128896+0000","last_change":"2023-11-26T07:22:22.128896+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128896+0000","last_undegraded":"2023-11-26T07:22:22.128896+0000","last_fullsized":"2023-11-26T07:22:22.128896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139213+0000","last_change":"2023-11-26T07:22:22.139213+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139213+0000","last_undegraded":"2023-11-26T07:22:22.139213+0000","last_fullsized":"2023-11-26T07:22:22.139213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127366+0000","last_change":"2023-11-26T07:22:22.127366+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127366+0000","last_undegraded":"2023-11-26T07:22:22.127366+0000","last_fullsized":"2023-11-26T07:22:22.127366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.144178+0000","last_change":"2023-11-26T07:22:22.144178+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.144178+0000","last_undegraded":"2023-11-26T07:22:22.144178+0000","last_fullsized":"2023-11-26T07:22:22.144178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151203+0000","last_change":"2023-11-26T07:22:22.149321+0000","last_active":"2023-11-26T07:22:22.151203+0000","last_peered":"2023-11-26T07:22:22.151203+0000","last_clean":"2023-11-26T07:22:22.151203+0000","last_became_active":"2023-11-26T07:22:22.148856+0000","last_became_peered":"2023-11-26T07:22:22.148856+0000","last_unstale":"2023-11-26T07:22:22.151203+0000","last_undegraded":"2023-11-26T07:22:22.151203+0000","last_fullsized":"2023-11-26T07:22:22.151203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:21.234148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129444+0000","last_change":"2023-11-26T07:22:22.129444+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129444+0000","last_undegraded":"2023-11-26T07:22:22.129444+0000","last_fullsized":"2023-11-26T07:22:22.129444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153379+0000","last_change":"2023-11-26T07:22:22.150877+0000","last_active":"2023-11-26T07:22:22.153379+0000","last_peered":"2023-11-26T07:22:22.153379+0000","last_clean":"2023-11-26T07:22:22.153379+0000","last_became_active":"2023-11-26T07:22:22.150413+0000","last_became_peered":"2023-11-26T07:22:22.150413+0000","last_unstale":"2023-11-26T07:22:22.153379+0000","last_undegraded":"2023-11-26T07:22:22.153379+0000","last_fullsized":"2023-11-26T07:22:22.153379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:21.234148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129600+0000","last_change":"2023-11-26T07:22:22.129600+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129600+0000","last_undegraded":"2023-11-26T07:22:22.129600+0000","last_fullsized":"2023-11-26T07:22:22.129600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152466+0000","last_change":"2023-11-26T07:22:22.148977+0000","last_active":"2023-11-26T07:22:22.152466+0000","last_peered":"2023-11-26T07:22:22.152466+0000","last_clean":"2023-11-26T07:22:22.152466+0000","last_became_active":"2023-11-26T07:22:22.148703+0000","last_became_peered":"2023-11-26T07:22:22.148703+0000","last_unstale":"2023-11-26T07:22:22.152466+0000","last_undegraded":"2023-11-26T07:22:22.152466+0000","last_fullsized":"2023-11-26T07:22:22.152466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:35.311859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.133925+0000","last_change":"2023-11-26T07:22:22.133925+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.133925+0000","last_undegraded":"2023-11-26T07:22:22.133925+0000","last_fullsized":"2023-11-26T07:22:22.133925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153820+0000","last_change":"2023-11-26T07:22:22.151630+0000","last_active":"2023-11-26T07:22:22.153820+0000","last_peered":"2023-11-26T07:22:22.153820+0000","last_clean":"2023-11-26T07:22:22.153820+0000","last_became_active":"2023-11-26T07:22:22.151230+0000","last_became_peered":"2023-11-26T07:22:22.151230+0000","last_unstale":"2023-11-26T07:22:22.153820+0000","last_undegraded":"2023-11-26T07:22:22.153820+0000","last_fullsized":"2023-11-26T07:22:22.153820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:38.450764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128444+0000","last_change":"2023-11-26T07:22:22.128444+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128444+0000","last_undegraded":"2023-11-26T07:22:22.128444+0000","last_fullsized":"2023-11-26T07:22:22.128444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.158977+0000","last_change":"2023-11-26T07:22:22.153463+0000","last_active":"2023-11-26T07:22:22.158977+0000","last_peered":"2023-11-26T07:22:22.158977+0000","last_clean":"2023-11-26T07:22:22.158977+0000","last_became_active":"2023-11-26T07:22:22.153132+0000","last_became_peered":"2023-11-26T07:22:22.153132+0000","last_unstale":"2023-11-26T07:22:22.158977+0000","last_undegraded":"2023-11-26T07:22:22.158977+0000","last_fullsized":"2023-11-26T07:22:22.158977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:15.107224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.142692+0000","last_change":"2023-11-26T07:22:22.142692+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.142692+0000","last_undegraded":"2023-11-26T07:22:22.142692+0000","last_fullsized":"2023-11-26T07:22:22.142692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152289+0000","last_change":"2023-11-26T07:22:22.148229+0000","last_active":"2023-11-26T07:22:22.152289+0000","last_peered":"2023-11-26T07:22:22.152289+0000","last_clean":"2023-11-26T07:22:22.152289+0000","last_became_active":"2023-11-26T07:22:22.147927+0000","last_became_peered":"2023-11-26T07:22:22.147927+0000","last_unstale":"2023-11-26T07:22:22.152289+0000","last_undegraded":"2023-11-26T07:22:22.152289+0000","last_fullsized":"2023-11-26T07:22:22.152289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:21:24.263764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132776+0000","last_change":"2023-11-26T07:22:22.132776+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132776+0000","last_undegraded":"2023-11-26T07:22:22.132776+0000","last_fullsized":"2023-11-26T07:22:22.132776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146154+0000","last_change":"2023-11-26T07:22:22.144638+0000","last_active":"2023-11-26T07:22:22.146154+0000","last_peered":"2023-11-26T07:22:22.146154+0000","last_clean":"2023-11-26T07:22:22.146154+0000","last_became_active":"2023-11-26T07:22:22.144230+0000","last_became_peered":"2023-11-26T07:22:22.144230+0000","last_unstale":"2023-11-26T07:22:22.146154+0000","last_undegraded":"2023-11-26T07:22:22.146154+0000","last_fullsized":"2023-11-26T07:22:22.146154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:33.883653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128946+0000","last_change":"2023-11-26T07:22:22.128946+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128946+0000","last_undegraded":"2023-11-26T07:22:22.128946+0000","last_fullsized":"2023-11-26T07:22:22.128946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.155657+0000","last_change":"2023-11-26T07:22:22.153839+0000","last_active":"2023-11-26T07:22:22.155657+0000","last_peered":"2023-11-26T07:22:22.155657+0000","last_clean":"2023-11-26T07:22:22.155657+0000","last_became_active":"2023-11-26T07:22:22.153535+0000","last_became_peered":"2023-11-26T07:22:22.153535+0000","last_unstale":"2023-11-26T07:22:22.155657+0000","last_undegraded":"2023-11-26T07:22:22.155657+0000","last_fullsized":"2023-11-26T07:22:22.155657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:56.712024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130940+0000","last_change":"2023-11-26T07:22:22.130940+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130940+0000","last_undegraded":"2023-11-26T07:22:22.130940+0000","last_fullsized":"2023-11-26T07:22:22.130940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146666+0000","last_change":"2023-11-26T07:22:22.140913+0000","last_active":"2023-11-26T07:22:22.146666+0000","last_peered":"2023-11-26T07:22:22.146666+0000","last_clean":"2023-11-26T07:22:22.146666+0000","last_became_active":"2023-11-26T07:22:22.140643+0000","last_became_peered":"2023-11-26T07:22:22.140643+0000","last_unstale":"2023-11-26T07:22:22.146666+0000","last_undegraded":"2023-11-26T07:22:22.146666+0000","last_fullsized":"2023-11-26T07:22:22.146666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:15.107224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130873+0000","last_change":"2023-11-26T07:22:22.130873+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130873+0000","last_undegraded":"2023-11-26T07:22:22.130873+0000","last_fullsized":"2023-11-26T07:22:22.130873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153050+0000","last_change":"2023-11-26T07:22:22.151535+0000","last_active":"2023-11-26T07:22:22.153050+0000","last_peered":"2023-11-26T07:22:22.153050+0000","last_clean":"2023-11-26T07:22:22.153050+0000","last_became_active":"2023-11-26T07:22:22.151097+0000","last_became_peered":"2023-11-26T07:22:22.151097+0000","last_unstale":"2023-11-26T07:22:22.153050+0000","last_undegraded":"2023-11-26T07:22:22.153050+0000","last_fullsized":"2023-11-26T07:22:22.153050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:49.532057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.141167+0000","last_change":"2023-11-26T07:22:22.141167+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.141167+0000","last_undegraded":"2023-11-26T07:22:22.141167+0000","last_fullsized":"2023-11-26T07:22:22.141167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152282+0000","last_change":"2023-11-26T07:22:22.149767+0000","last_active":"2023-11-26T07:22:22.152282+0000","last_peered":"2023-11-26T07:22:22.152282+0000","last_clean":"2023-11-26T07:22:22.152282+0000","last_became_active":"2023-11-26T07:22:22.149435+0000","last_became_peered":"2023-11-26T07:22:22.149435+0000","last_unstale":"2023-11-26T07:22:22.152282+0000","last_undegraded":"2023-11-26T07:22:22.152282+0000","last_fullsized":"2023-11-26T07:22:22.152282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:21:24.263764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139846+0000","last_change":"2023-11-26T07:22:22.139846+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139846+0000","last_undegraded":"2023-11-26T07:22:22.139846+0000","last_fullsized":"2023-11-26T07:22:22.139846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.143475+0000","last_change":"2023-11-26T07:22:22.138119+0000","last_active":"2023-11-26T07:22:22.143475+0000","last_peered":"2023-11-26T07:22:22.143475+0000","last_clean":"2023-11-26T07:22:22.143475+0000","last_became_active":"2023-11-26T07:22:22.137770+0000","last_became_peered":"2023-11-26T07:22:22.137770+0000","last_unstale":"2023-11-26T07:22:22.143475+0000","last_undegraded":"2023-11-26T07:22:22.143475+0000","last_fullsized":"2023-11-26T07:22:22.143475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:25.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139244+0000","last_change":"2023-11-26T07:22:22.139244+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139244+0000","last_undegraded":"2023-11-26T07:22:22.139244+0000","last_fullsized":"2023-11-26T07:22:22.139244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151664+0000","last_change":"2023-11-26T07:22:22.145272+0000","last_active":"2023-11-26T07:22:22.151664+0000","last_peered":"2023-11-26T07:22:22.151664+0000","last_clean":"2023-11-26T07:22:22.151664+0000","last_became_active":"2023-11-26T07:22:22.145011+0000","last_became_peered":"2023-11-26T07:22:22.145011+0000","last_unstale":"2023-11-26T07:22:22.151664+0000","last_undegraded":"2023-11-26T07:22:22.151664+0000","last_fullsized":"2023-11-26T07:22:22.151664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:17.509070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134871+0000","last_change":"2023-11-26T07:22:22.134871+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134871+0000","last_undegraded":"2023-11-26T07:22:22.134871+0000","last_fullsized":"2023-11-26T07:22:22.134871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153970+0000","last_change":"2023-11-26T07:22:22.149360+0000","last_active":"2023-11-26T07:22:22.153970+0000","last_peered":"2023-11-26T07:22:22.153970+0000","last_clean":"2023-11-26T07:22:22.153970+0000","last_became_active":"2023-11-26T07:22:22.149068+0000","last_became_peered":"2023-11-26T07:22:22.149068+0000","last_unstale":"2023-11-26T07:22:22.153970+0000","last_undegraded":"2023-11-26T07:22:22.153970+0000","last_fullsized":"2023-11-26T07:22:22.153970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:29.708891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128109+0000","last_change":"2023-11-26T07:22:22.128109+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128109+0000","last_undegraded":"2023-11-26T07:22:22.128109+0000","last_fullsized":"2023-11-26T07:22:22.128109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153325+0000","last_change":"2023-11-26T07:22:22.151868+0000","last_active":"2023-11-26T07:22:22.153325+0000","last_peered":"2023-11-26T07:22:22.153325+0000","last_clean":"2023-11-26T07:22:22.153325+0000","last_became_active":"2023-11-26T07:22:22.151535+0000","last_became_peered":"2023-11-26T07:22:22.151535+0000","last_unstale":"2023-11-26T07:22:22.153325+0000","last_undegraded":"2023-11-26T07:22:22.153325+0000","last_fullsized":"2023-11-26T07:22:22.153325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:33.883653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135370+0000","last_change":"2023-11-26T07:22:22.135370+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135370+0000","last_undegraded":"2023-11-26T07:22:22.135370+0000","last_fullsized":"2023-11-26T07:22:22.135370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152260+0000","last_change":"2023-11-26T07:22:22.145842+0000","last_active":"2023-11-26T07:22:22.152260+0000","last_peered":"2023-11-26T07:22:22.152260+0000","last_clean":"2023-11-26T07:22:22.152260+0000","last_became_active":"2023-11-26T07:22:22.145573+0000","last_became_peered":"2023-11-26T07:22:22.145573+0000","last_unstale":"2023-11-26T07:22:22.152260+0000","last_undegraded":"2023-11-26T07:22:22.152260+0000","last_fullsized":"2023-11-26T07:22:22.152260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:49.532057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134114+0000","last_change":"2023-11-26T07:22:22.134114+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134114+0000","last_undegraded":"2023-11-26T07:22:22.134114+0000","last_fullsized":"2023-11-26T07:22:22.134114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.145214+0000","last_change":"2023-11-26T07:22:22.140377+0000","last_active":"2023-11-26T07:22:22.145214+0000","last_peered":"2023-11-26T07:22:22.145214+0000","last_clean":"2023-11-26T07:22:22.145214+0000","last_became_active":"2023-11-26T07:22:22.138993+0000","last_became_peered":"2023-11-26T07:22:22.138993+0000","last_unstale":"2023-11-26T07:22:22.145214+0000","last_undegraded":"2023-11-26T07:22:22.145214+0000","last_fullsized":"2023-11-26T07:22:22.145214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:57.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128923+0000","last_change":"2023-11-26T07:22:22.128923+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128923+0000","last_undegraded":"2023-11-26T07:22:22.128923+0000","last_fullsized":"2023-11-26T07:22:22.128923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.154062+0000","last_change":"2023-11-26T07:22:22.152564+0000","last_active":"2023-11-26T07:22:22.154062+0000","last_peered":"2023-11-26T07:22:22.154062+0000","last_clean":"2023-11-26T07:22:22.154062+0000","last_became_active":"2023-11-26T07:22:22.152161+0000","last_became_peered":"2023-11-26T07:22:22.152161+0000","last_unstale":"2023-11-26T07:22:22.154062+0000","last_undegraded":"2023-11-26T07:22:22.154062+0000","last_fullsized":"2023-11-26T07:22:22.154062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:08.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.137359+0000","last_change":"2023-11-26T07:22:22.137359+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.137359+0000","last_undegraded":"2023-11-26T07:22:22.137359+0000","last_fullsized":"2023-11-26T07:22:22.137359+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151089+0000","last_change":"2023-11-26T07:22:22.145038+0000","last_active":"2023-11-26T07:22:22.151089+0000","last_peered":"2023-11-26T07:22:22.151089+0000","last_clean":"2023-11-26T07:22:22.151089+0000","last_became_active":"2023-11-26T07:22:22.144754+0000","last_became_peered":"2023-11-26T07:22:22.144754+0000","last_unstale":"2023-11-26T07:22:22.151089+0000","last_undegraded":"2023-11-26T07:22:22.151089+0000","last_fullsized":"2023-11-26T07:22:22.151089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:53:45.151325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132275+0000","last_change":"2023-11-26T07:22:22.132275+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132275+0000","last_undegraded":"2023-11-26T07:22:22.132275+0000","last_fullsized":"2023-11-26T07:22:22.132275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146987+0000","last_change":"2023-11-26T07:22:22.140578+0000","last_active":"2023-11-26T07:22:22.146987+0000","last_peered":"2023-11-26T07:22:22.146987+0000","last_clean":"2023-11-26T07:22:22.146987+0000","last_became_active":"2023-11-26T07:22:22.140222+0000","last_became_peered":"2023-11-26T07:22:22.140222+0000","last_unstale":"2023-11-26T07:22:22.146987+0000","last_undegraded":"2023-11-26T07:22:22.146987+0000","last_fullsized":"2023-11-26T07:22:22.146987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:59.850726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129550+0000","last_change":"2023-11-26T07:22:22.129550+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129550+0000","last_undegraded":"2023-11-26T07:22:22.129550+0000","last_fullsized":"2023-11-26T07:22:22.129550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.149688+0000","last_change":"2023-11-26T07:22:22.147493+0000","last_active":"2023-11-26T07:22:22.149688+0000","last_peered":"2023-11-26T07:22:22.149688+0000","last_clean":"2023-11-26T07:22:22.149688+0000","last_became_active":"2023-11-26T07:22:22.147187+0000","last_became_peered":"2023-11-26T07:22:22.147187+0000","last_unstale":"2023-11-26T07:22:22.149688+0000","last_undegraded":"2023-11-26T07:22:22.149688+0000","last_fullsized":"2023-11-26T07:22:22.149688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.357020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131250+0000","last_change":"2023-11-26T07:22:22.131250+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131250+0000","last_undegraded":"2023-11-26T07:22:22.131250+0000","last_fullsized":"2023-11-26T07:22:22.131250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153430+0000","last_change":"2023-11-26T07:22:22.148994+0000","last_active":"2023-11-26T07:22:22.153430+0000","last_peered":"2023-11-26T07:22:22.153430+0000","last_clean":"2023-11-26T07:22:22.153430+0000","last_became_active":"2023-11-26T07:22:22.147943+0000","last_became_peered":"2023-11-26T07:22:22.147943+0000","last_unstale":"2023-11-26T07:22:22.153430+0000","last_undegraded":"2023-11-26T07:22:22.153430+0000","last_fullsized":"2023-11-26T07:22:22.153430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:53:21.611291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129655+0000","last_change":"2023-11-26T07:22:22.129655+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129655+0000","last_undegraded":"2023-11-26T07:22:22.129655+0000","last_fullsized":"2023-11-26T07:22:22.129655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146830+0000","last_change":"2023-11-26T07:22:22.139465+0000","last_active":"2023-11-26T07:22:22.146830+0000","last_peered":"2023-11-26T07:22:22.146830+0000","last_clean":"2023-11-26T07:22:22.146830+0000","last_became_active":"2023-11-26T07:22:22.139180+0000","last_became_peered":"2023-11-26T07:22:22.139180+0000","last_unstale":"2023-11-26T07:22:22.146830+0000","last_undegraded":"2023-11-26T07:22:22.146830+0000","last_fullsized":"2023-11-26T07:22:22.146830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:06.932370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.143648+0000","last_change":"2023-11-26T07:22:22.143648+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.143648+0000","last_undegraded":"2023-11-26T07:22:22.143648+0000","last_fullsized":"2023-11-26T07:22:22.143648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151859+0000","last_change":"2023-11-26T07:22:22.146130+0000","last_active":"2023-11-26T07:22:22.151859+0000","last_peered":"2023-11-26T07:22:22.151859+0000","last_clean":"2023-11-26T07:22:22.151859+0000","last_became_active":"2023-11-26T07:22:22.144230+0000","last_became_peered":"2023-11-26T07:22:22.144230+0000","last_unstale":"2023-11-26T07:22:22.151859+0000","last_undegraded":"2023-11-26T07:22:22.151859+0000","last_fullsized":"2023-11-26T07:22:22.151859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:44.104269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132436+0000","last_change":"2023-11-26T07:22:22.132436+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132436+0000","last_undegraded":"2023-11-26T07:22:22.132436+0000","last_fullsized":"2023-11-26T07:22:22.132436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151642+0000","last_change":"2023-11-26T07:22:22.145175+0000","last_active":"2023-11-26T07:22:22.151642+0000","last_peered":"2023-11-26T07:22:22.151642+0000","last_clean":"2023-11-26T07:22:22.151642+0000","last_became_active":"2023-11-26T07:22:22.144917+0000","last_became_peered":"2023-11-26T07:22:22.144917+0000","last_unstale":"2023-11-26T07:22:22.151642+0000","last_undegraded":"2023-11-26T07:22:22.151642+0000","last_fullsized":"2023-11-26T07:22:22.151642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:15.998685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129092+0000","last_change":"2023-11-26T07:22:22.129092+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129092+0000","last_undegraded":"2023-11-26T07:22:22.129092+0000","last_fullsized":"2023-11-26T07:22:22.129092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.147011+0000","last_change":"2023-11-26T07:22:22.143708+0000","last_active":"2023-11-26T07:22:22.147011+0000","last_peered":"2023-11-26T07:22:22.147011+0000","last_clean":"2023-11-26T07:22:22.147011+0000","last_became_active":"2023-11-26T07:22:22.143395+0000","last_became_peered":"2023-11-26T07:22:22.143395+0000","last_unstale":"2023-11-26T07:22:22.147011+0000","last_undegraded":"2023-11-26T07:22:22.147011+0000","last_fullsized":"2023-11-26T07:22:22.147011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:58:46.089931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.136808+0000","last_change":"2023-11-26T07:22:22.136808+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.136808+0000","last_undegraded":"2023-11-26T07:22:22.136808+0000","last_fullsized":"2023-11-26T07:22:22.136808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.144998+0000","last_change":"2023-11-26T07:22:22.139927+0000","last_active":"2023-11-26T07:22:22.144998+0000","last_peered":"2023-11-26T07:22:22.144998+0000","last_clean":"2023-11-26T07:22:22.144998+0000","last_became_active":"2023-11-26T07:22:22.139628+0000","last_became_peered":"2023-11-26T07:22:22.139628+0000","last_unstale":"2023-11-26T07:22:22.144998+0000","last_undegraded":"2023-11-26T07:22:22.144998+0000","last_fullsized":"2023-11-26T07:22:22.144998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:44:29.019321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129561+0000","last_change":"2023-11-26T07:22:22.129561+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129561+0000","last_undegraded":"2023-11-26T07:22:22.129561+0000","last_fullsized":"2023-11-26T07:22:22.129561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152296+0000","last_change":"2023-11-26T07:22:22.146618+0000","last_active":"2023-11-26T07:22:22.152296+0000","last_peered":"2023-11-26T07:22:22.152296+0000","last_clean":"2023-11-26T07:22:22.152296+0000","last_became_active":"2023-11-26T07:22:22.146214+0000","last_became_peered":"2023-11-26T07:22:22.146214+0000","last_unstale":"2023-11-26T07:22:22.152296+0000","last_undegraded":"2023-11-26T07:22:22.152296+0000","last_fullsized":"2023-11-26T07:22:22.152296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:25.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.137697+0000","last_change":"2023-11-26T07:22:22.137697+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.137697+0000","last_undegraded":"2023-11-26T07:22:22.137697+0000","last_fullsized":"2023-11-26T07:22:22.137697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152272+0000","last_change":"2023-11-26T07:22:22.148139+0000","last_active":"2023-11-26T07:22:22.152272+0000","last_peered":"2023-11-26T07:22:22.152272+0000","last_clean":"2023-11-26T07:22:22.152272+0000","last_became_active":"2023-11-26T07:22:22.147844+0000","last_became_peered":"2023-11-26T07:22:22.147844+0000","last_unstale":"2023-11-26T07:22:22.152272+0000","last_undegraded":"2023-11-26T07:22:22.152272+0000","last_fullsized":"2023-11-26T07:22:22.152272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:57.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132256+0000","last_change":"2023-11-26T07:22:22.132256+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132256+0000","last_undegraded":"2023-11-26T07:22:22.132256+0000","last_fullsized":"2023-11-26T07:22:22.132256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146674+0000","last_change":"2023-11-26T07:22:22.140709+0000","last_active":"2023-11-26T07:22:22.146674+0000","last_peered":"2023-11-26T07:22:22.146674+0000","last_clean":"2023-11-26T07:22:22.146674+0000","last_became_active":"2023-11-26T07:22:22.140379+0000","last_became_peered":"2023-11-26T07:22:22.140379+0000","last_unstale":"2023-11-26T07:22:22.146674+0000","last_undegraded":"2023-11-26T07:22:22.146674+0000","last_fullsized":"2023-11-26T07:22:22.146674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:41.677508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128291+0000","last_change":"2023-11-26T07:22:22.128291+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128291+0000","last_undegraded":"2023-11-26T07:22:22.128291+0000","last_fullsized":"2023-11-26T07:22:22.128291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152324+0000","last_change":"2023-11-26T07:22:22.148901+0000","last_active":"2023-11-26T07:22:22.152324+0000","last_peered":"2023-11-26T07:22:22.152324+0000","last_clean":"2023-11-26T07:22:22.152324+0000","last_became_active":"2023-11-26T07:22:22.148566+0000","last_became_peered":"2023-11-26T07:22:22.148566+0000","last_unstale":"2023-11-26T07:22:22.152324+0000","last_undegraded":"2023-11-26T07:22:22.152324+0000","last_fullsized":"2023-11-26T07:22:22.152324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:58:46.089931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127322+0000","last_change":"2023-11-26T07:22:22.127322+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127322+0000","last_undegraded":"2023-11-26T07:22:22.127322+0000","last_fullsized":"2023-11-26T07:22:22.127322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153518+0000","last_change":"2023-11-26T07:22:22.152219+0000","last_active":"2023-11-26T07:22:22.153518+0000","last_peered":"2023-11-26T07:22:22.153518+0000","last_clean":"2023-11-26T07:22:22.153518+0000","last_became_active":"2023-11-26T07:22:22.151927+0000","last_became_peered":"2023-11-26T07:22:22.151927+0000","last_unstale":"2023-11-26T07:22:22.153518+0000","last_undegraded":"2023-11-26T07:22:22.153518+0000","last_fullsized":"2023-11-26T07:22:22.153518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:17.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"69'128","reported_seq":251,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-26T07:22:21.128526+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:22:21.128526+0000","last_peered":"2023-11-26T07:22:21.128526+0000","last_clean":"2023-11-26T07:22:21.128526+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:22:21.128526+0000","last_undegraded":"2023-11-26T07:22:21.128526+0000","last_fullsized":"2023-11-26T07:22:21.128526+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":992140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.56599999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":0.995},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84799999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45200000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.48499999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.92500000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71399999999999997},"min":{"1min":0.56899999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84499999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.58999999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92500000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.50600000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":1.016,"15min":1.016},"last":0.79500000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.85099999999999998,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":0.81899999999999995},"min":{"1min":0.68100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93500000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.71099999999999997},"min":{"1min":0.47199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73299999999999998,"15min":0.77500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95899999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.84899999999999998},"min":{"1min":0.58699999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91500000000000004,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82099999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.499,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93400000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.82999999999999996},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.012,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":0.86699999999999999,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.222000000000001,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":3.4020000000000001,"5min":1.246,"15min":0.91500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":54.737000000000002,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.80600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.40400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.1880000000000002,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.6689999999999996,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.96599999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.439,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.8479999999999999,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.72499999999999998,"15min":0.68899999999999995},"min":{"1min":0.57099999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0170000000000003,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.73199999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.81299999999999994,"15min":0.751},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.4870000000000001,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.601,"5min":0.94399999999999995,"15min":0.874},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":18.77,"5min":18.77,"15min":18.77},"last":0.78100000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.53100000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.749},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.121,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.88100000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67400000000000004,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.77900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.93300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.626,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.64400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.87,"5min":1.073,"15min":1.073},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.96499999999999997,"5min":1.036,"15min":1.036},"last":0.77700000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.012},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.49099999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79000000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.79000000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.71299999999999997},"min":{"1min":0.495,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.028,"5min":1.097,"15min":1.097},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.88700000000000001,"5min":1.071,"15min":1.071},"last":0.80700000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65200000000000002,"15min":0.68600000000000005},"min":{"1min":0.42699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.137,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.77000000000000002},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.0029999999999999,"5min":1.206,"15min":1.206},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.62,"15min":0.67000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93100000000000005,"5min":1.143,"15min":1.143},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.73199999999999998},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.1020000000000001,"5min":1.228,"15min":1.228},"last":0.84599999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0580000000000001,"5min":1.264,"15min":1.264},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.077,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.87}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84899999999999998,"5min":1.016,"15min":1.016},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93100000000000005,"5min":1.26,"15min":1.26},"last":0.72099999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90200000000000002,"5min":1.077,"15min":1.077},"last":0.69699999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.302,"15min":1.302},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.753},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":1.181,"15min":1.181},"last":0.77000000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.747},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97999999999999998,"5min":1.702,"15min":1.702},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.78700000000000003},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0409999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.72999999999999998},"min":{"1min":0.42199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97099999999999997,"5min":1.129,"15min":1.129},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.78800000000000003},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.012,"15min":1.012},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88300000000000001,"5min":1.016,"15min":1.016},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.81200000000000006},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.0980000000000001,"5min":1.242,"15min":1.242},"last":0.67600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.68999999999999995,"5min":1.294,"15min":1.294},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.40500000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":1.456,"15min":1.456},"last":0.434},{"interface":"front","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.52900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.212,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2649999999999999,"5min":1.395,"15min":1.395},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1080000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.504}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.64700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.333,"15min":1.333},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.228,"5min":1.347,"15min":1.347},"last":0.59699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.397,"15min":1.397},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1859999999999999,"5min":1.363,"15min":1.363},"last":0.66200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56000000000000005,"15min":0.64800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65800000000000003,"5min":3.931,"15min":3.931},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.71099999999999997},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60199999999999998,"15min":0.69199999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95499999999999996,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.69299999999999995},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.7650000000000001,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.434}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.751,"15min":0.90300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98899999999999999,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.59899999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.85199999999999998,"15min":0.92900000000000005},"min":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1240000000000001,"5min":4.431,"15min":4.431},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.274,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.63500000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.61199999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89400000000000002,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8069999999999999,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.754}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.98399999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.222,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.93899999999999995,"15min":0.995},"min":{"1min":0.621,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.4740000000000002,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.052,"5min":1.609,"15min":1.609},"last":0.61899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.57499999999999996,"5min":1.034,"15min":1.034},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69999999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.33000000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66900000000000004,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.499,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48599999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95599999999999996,"5min":1.143,"15min":1.143},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.216,"15min":1.216},"last":0.75800000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.58999999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97399999999999998,"5min":1.262,"15min":1.262},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.998,"5min":1.54,"15min":1.54},"last":0.67500000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.61599999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.054,"5min":1.145,"15min":1.145},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.91600000000000004,"5min":1.333,"15min":1.333},"last":0.69899999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95699999999999996,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0189999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1536000,"data_stored":991121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.45300000000000001},"min":{"1min":0.29799999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.68300000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.48499999999999999,"15min":0.47799999999999998},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.78600000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79700000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.438}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.38400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83599999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.81499999999999995},"min":{"1min":0.59499999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.67800000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":1.0980000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.63400000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.76700000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.64800000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.61899999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.80400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1180000000000001,"5min":1.123,"15min":1.123},"last":1.079},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.83099999999999996,"15min":0.81899999999999995},"min":{"1min":0.69399999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.0349999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:22:27.283 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:22:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.465+0000 7fd5e7806640 1 Processor -- start 2023-11-26T07:22:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.465+0000 7fd5e7806640 1 -- start start 2023-11-26T07:22:27.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e7806640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5e0146c60 0x7fd5e0070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e7806640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e015dba0 con 0x7fd5e0147040 2023-11-26T07:22:27.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e7806640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e006c860 con 0x7fd5e006c220 2023-11-26T07:22:27.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e7806640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e006c670 con 0x7fd5e0146c60 2023-11-26T07:22:27.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e55a2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5e0146c60 0x7fd5e0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e4da1640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd5e0147040 0x7fd5e006bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60506/0 (socket says 172.21.15.31:60506) 2023-11-26T07:22:27.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e55a2640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5e0146c60 0x7fd5e0070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37910/0 (socket says 172.21.15.31:37910) 2023-11-26T07:22:27.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e4da1640 1 -- 172.21.15.31:0/1951701471 learned_addr learned my addr 172.21.15.31:0/1951701471 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:27.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e55a2640 1 -- 172.21.15.31:0/1951701471 >> v1:172.21.15.97:6789/0 conn(0x7fd5e006c220 legacy=0x7fd5e015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e55a2640 1 -- 172.21.15.31:0/1951701471 >> v1:172.21.15.31:6789/0 conn(0x7fd5e0147040 legacy=0x7fd5e006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e55a2640 1 -- 172.21.15.31:0/1951701471 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e006cb40 con 0x7fd5e0146c60 2023-11-26T07:22:27.473 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5e55a2640 1 --2- 172.21.15.31:0/1951701471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0146c60 0x7fd5e0070020 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0097c0 tx=0x7fd5dc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d13df637310d984 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.469+0000 7fd5d7fff640 1 -- 172.21.15.31:0/1951701471 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0ab600 con 0x7fd5e0146c60 2023-11-26T07:22:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5d7fff640 1 -- 172.21.15.31:0/1951701471 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc0a7050 con 0x7fd5e0146c60 2023-11-26T07:22:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5d7fff640 1 -- 172.21.15.31:0/1951701471 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0ab980 con 0x7fd5e0146c60 2023-11-26T07:22:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 -- 172.21.15.31:0/1951701471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0146c60 msgr2=0x7fd5e0070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 --2- 172.21.15.31:0/1951701471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0146c60 0x7fd5e0070020 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0097c0 tx=0x7fd5dc01c3f0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 -- 172.21.15.31:0/1951701471 shutdown_connections 2023-11-26T07:22:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 --2- 172.21.15.31:0/1951701471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0146c60 0x7fd5e0070020 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 -- 172.21.15.31:0/1951701471 >> 172.21.15.31:0/1951701471 conn(0x7fd5e00b2ea0 msgr2=0x7fd5e00b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 -- 172.21.15.31:0/1951701471 shutdown_connections 2023-11-26T07:22:27.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 -- 172.21.15.31:0/1951701471 wait complete. 2023-11-26T07:22:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 Processor -- start 2023-11-26T07:22:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 -- start start 2023-11-26T07:22:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5e006c220 0x7fd5e0138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5e0146c60 0x7fd5e0158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0147040 0x7fd5e01591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0152d10 con 0x7fd5e0147040 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e012af50 con 0x7fd5e0146c60 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e7806640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0151f50 con 0x7fd5e006c220 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e5da3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0147040 0x7fd5e01591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e5da3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0147040 0x7fd5e01591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37926/0 (socket says 172.21.15.31:37926) 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e5da3640 1 -- 172.21.15.31:0/2721768198 learned_addr learned my addr 172.21.15.31:0/2721768198 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e55a2640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5e006c220 0x7fd5e0138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.473+0000 7fd5e4da1640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5e0146c60 0x7fd5e0158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5e5da3640 1 -- 172.21.15.31:0/2721768198 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5e006c220 msgr2=0x7fd5e0138100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5e5da3640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5e006c220 0x7fd5e0138100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5e5da3640 1 -- 172.21.15.31:0/2721768198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5e0146c60 msgr2=0x7fd5e0158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5e5da3640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5e0146c60 0x7fd5e0158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5e5da3640 1 -- 172.21.15.31:0/2721768198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc009420 con 0x7fd5e0147040 2023-11-26T07:22:27.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5e5da3640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0147040 0x7fd5e01591e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c770 tx=0x7fd5d800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5d5ffb640 1 -- 172.21.15.31:0/2721768198 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8003b80 con 0x7fd5e0147040 2023-11-26T07:22:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5d5ffb640 1 -- 172.21.15.31:0/2721768198 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d8003ce0 con 0x7fd5e0147040 2023-11-26T07:22:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5d5ffb640 1 -- 172.21.15.31:0/2721768198 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d800aa30 con 0x7fd5e0147040 2023-11-26T07:22:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5e7806640 1 -- 172.21.15.31:0/2721768198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e0152230 con 0x7fd5e0147040 2023-11-26T07:22:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5e7806640 1 -- 172.21.15.31:0/2721768198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e01527f0 con 0x7fd5e0147040 2023-11-26T07:22:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5e7806640 1 -- 172.21.15.31:0/2721768198 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e0067a40 con 0x7fd5e0147040 2023-11-26T07:22:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.477+0000 7fd5d5ffb640 1 -- 172.21.15.31:0/2721768198 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fd5d800ab90 con 0x7fd5e0147040 2023-11-26T07:22:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.481+0000 7fd5d5ffb640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd5c4066590 0x7fd5c4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.481+0000 7fd5d5ffb640 1 -- 172.21.15.31:0/2721768198 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5d8083d50 con 0x7fd5e0147040 2023-11-26T07:22:27.483 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.481+0000 7fd5e55a2640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd5c4066590 0x7fd5c4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.481+0000 7fd5e55a2640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd5c4066590 0x7fd5c4068a50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0ad000 tx=0x7fd5dc0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.485+0000 7fd5d5ffb640 1 -- 172.21.15.31:0/2721768198 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5d80510d0 con 0x7fd5e0147040 2023-11-26T07:22:27.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.645+0000 7fd5e7806640 1 -- 172.21.15.31:0/2721768198 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5e00b3a90 con 0x7fd5c4066590 2023-11-26T07:22:27.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.653+0000 7fd5d5ffb640 1 -- 172.21.15.31:0/2721768198 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283525 (secure 0 0 0) 0x7fd5e00b3a90 con 0x7fd5c4066590 2023-11-26T07:22:27.659 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:22:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 -- 172.21.15.31:0/2721768198 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd5c4066590 msgr2=0x7fd5c4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd5c4066590 0x7fd5c4068a50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0ad000 tx=0x7fd5dc0a6000 comp rx=0 tx=0).stop 2023-11-26T07:22:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 -- 172.21.15.31:0/2721768198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0147040 msgr2=0x7fd5e01591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0147040 0x7fd5e01591e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c770 tx=0x7fd5d800cc30 comp rx=0 tx=0).stop 2023-11-26T07:22:27.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 -- 172.21.15.31:0/2721768198 shutdown_connections 2023-11-26T07:22:27.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd5c4066590 0x7fd5c4068a50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5e0147040 0x7fd5e01591e0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5e0146c60 0x7fd5e0158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 --2- 172.21.15.31:0/2721768198 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5e006c220 0x7fd5e0138100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 -- 172.21.15.31:0/2721768198 >> 172.21.15.31:0/2721768198 conn(0x7fd5e00b2ea0 msgr2=0x7fd5e015f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:27.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 -- 172.21.15.31:0/2721768198 shutdown_connections 2023-11-26T07:22:27.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.657+0000 7fd5e7806640 1 -- 172.21.15.31:0/2721768198 wait complete. 2023-11-26T07:22:27.679 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-26T07:22:26.200917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258048,"kb_used_data":8548,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716672,"statfs":{"total":773094113280,"available":772829872128,"internally_reserved":0,"allocated":8753152,"data_stored":4393893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.080419"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131043+0000","last_change":"2023-11-26T07:22:22.131043+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131043+0000","last_undegraded":"2023-11-26T07:22:22.131043+0000","last_fullsized":"2023-11-26T07:22:22.131043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129528+0000","last_change":"2023-11-26T07:22:22.129528+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129528+0000","last_undegraded":"2023-11-26T07:22:22.129528+0000","last_fullsized":"2023-11-26T07:22:22.129528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.133354+0000","last_change":"2023-11-26T07:22:22.133354+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.133354+0000","last_undegraded":"2023-11-26T07:22:22.133354+0000","last_fullsized":"2023-11-26T07:22:22.133354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135222+0000","last_change":"2023-11-26T07:22:22.135222+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135222+0000","last_undegraded":"2023-11-26T07:22:22.135222+0000","last_fullsized":"2023-11-26T07:22:22.135222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.143110+0000","last_change":"2023-11-26T07:22:22.143110+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.143110+0000","last_undegraded":"2023-11-26T07:22:22.143110+0000","last_fullsized":"2023-11-26T07:22:22.143110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130504+0000","last_change":"2023-11-26T07:22:22.130504+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130504+0000","last_undegraded":"2023-11-26T07:22:22.130504+0000","last_fullsized":"2023-11-26T07:22:22.130504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130438+0000","last_change":"2023-11-26T07:22:22.130438+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130438+0000","last_undegraded":"2023-11-26T07:22:22.130438+0000","last_fullsized":"2023-11-26T07:22:22.130438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130281+0000","last_change":"2023-11-26T07:22:22.130281+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130281+0000","last_undegraded":"2023-11-26T07:22:22.130281+0000","last_fullsized":"2023-11-26T07:22:22.130281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131416+0000","last_change":"2023-11-26T07:22:22.131416+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131416+0000","last_undegraded":"2023-11-26T07:22:22.131416+0000","last_fullsized":"2023-11-26T07:22:22.131416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131033+0000","last_change":"2023-11-26T07:22:22.131033+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131033+0000","last_undegraded":"2023-11-26T07:22:22.131033+0000","last_fullsized":"2023-11-26T07:22:22.131033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135119+0000","last_change":"2023-11-26T07:22:22.135119+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135119+0000","last_undegraded":"2023-11-26T07:22:22.135119+0000","last_fullsized":"2023-11-26T07:22:22.135119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127615+0000","last_change":"2023-11-26T07:22:22.127615+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127615+0000","last_undegraded":"2023-11-26T07:22:22.127615+0000","last_fullsized":"2023-11-26T07:22:22.127615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135357+0000","last_change":"2023-11-26T07:22:22.135357+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135357+0000","last_undegraded":"2023-11-26T07:22:22.135357+0000","last_fullsized":"2023-11-26T07:22:22.135357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132827+0000","last_change":"2023-11-26T07:22:22.132827+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132827+0000","last_undegraded":"2023-11-26T07:22:22.132827+0000","last_fullsized":"2023-11-26T07:22:22.132827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129520+0000","last_change":"2023-11-26T07:22:22.129520+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129520+0000","last_undegraded":"2023-11-26T07:22:22.129520+0000","last_fullsized":"2023-11-26T07:22:22.129520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.142126+0000","last_change":"2023-11-26T07:22:22.142126+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.142126+0000","last_undegraded":"2023-11-26T07:22:22.142126+0000","last_fullsized":"2023-11-26T07:22:22.142126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139694+0000","last_change":"2023-11-26T07:22:22.139694+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139694+0000","last_undegraded":"2023-11-26T07:22:22.139694+0000","last_fullsized":"2023-11-26T07:22:22.139694+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134761+0000","last_change":"2023-11-26T07:22:22.134761+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134761+0000","last_undegraded":"2023-11-26T07:22:22.134761+0000","last_fullsized":"2023-11-26T07:22:22.134761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.138624+0000","last_change":"2023-11-26T07:22:22.138624+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.138624+0000","last_undegraded":"2023-11-26T07:22:22.138624+0000","last_fullsized":"2023-11-26T07:22:22.138624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134691+0000","last_change":"2023-11-26T07:22:22.134691+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134691+0000","last_undegraded":"2023-11-26T07:22:22.134691+0000","last_fullsized":"2023-11-26T07:22:22.134691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131538+0000","last_change":"2023-11-26T07:22:22.131538+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131538+0000","last_undegraded":"2023-11-26T07:22:22.131538+0000","last_fullsized":"2023-11-26T07:22:22.131538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129384+0000","last_change":"2023-11-26T07:22:22.129384+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129384+0000","last_undegraded":"2023-11-26T07:22:22.129384+0000","last_fullsized":"2023-11-26T07:22:22.129384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127661+0000","last_change":"2023-11-26T07:22:22.127661+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127661+0000","last_undegraded":"2023-11-26T07:22:22.127661+0000","last_fullsized":"2023-11-26T07:22:22.127661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131711+0000","last_change":"2023-11-26T07:22:22.131711+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131711+0000","last_undegraded":"2023-11-26T07:22:22.131711+0000","last_fullsized":"2023-11-26T07:22:22.131711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134720+0000","last_change":"2023-11-26T07:22:22.134720+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134720+0000","last_undegraded":"2023-11-26T07:22:22.134720+0000","last_fullsized":"2023-11-26T07:22:22.134720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134643+0000","last_change":"2023-11-26T07:22:22.134643+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134643+0000","last_undegraded":"2023-11-26T07:22:22.134643+0000","last_fullsized":"2023-11-26T07:22:22.134643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.143843+0000","last_change":"2023-11-26T07:22:22.143843+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.143843+0000","last_undegraded":"2023-11-26T07:22:22.143843+0000","last_fullsized":"2023-11-26T07:22:22.143843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130684+0000","last_change":"2023-11-26T07:22:22.130684+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130684+0000","last_undegraded":"2023-11-26T07:22:22.130684+0000","last_fullsized":"2023-11-26T07:22:22.130684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128838+0000","last_change":"2023-11-26T07:22:22.128838+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128838+0000","last_undegraded":"2023-11-26T07:22:22.128838+0000","last_fullsized":"2023-11-26T07:22:22.128838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128896+0000","last_change":"2023-11-26T07:22:22.128896+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128896+0000","last_undegraded":"2023-11-26T07:22:22.128896+0000","last_fullsized":"2023-11-26T07:22:22.128896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139213+0000","last_change":"2023-11-26T07:22:22.139213+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139213+0000","last_undegraded":"2023-11-26T07:22:22.139213+0000","last_fullsized":"2023-11-26T07:22:22.139213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127366+0000","last_change":"2023-11-26T07:22:22.127366+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127366+0000","last_undegraded":"2023-11-26T07:22:22.127366+0000","last_fullsized":"2023-11-26T07:22:22.127366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.144178+0000","last_change":"2023-11-26T07:22:22.144178+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.144178+0000","last_undegraded":"2023-11-26T07:22:22.144178+0000","last_fullsized":"2023-11-26T07:22:22.144178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151203+0000","last_change":"2023-11-26T07:22:22.149321+0000","last_active":"2023-11-26T07:22:22.151203+0000","last_peered":"2023-11-26T07:22:22.151203+0000","last_clean":"2023-11-26T07:22:22.151203+0000","last_became_active":"2023-11-26T07:22:22.148856+0000","last_became_peered":"2023-11-26T07:22:22.148856+0000","last_unstale":"2023-11-26T07:22:22.151203+0000","last_undegraded":"2023-11-26T07:22:22.151203+0000","last_fullsized":"2023-11-26T07:22:22.151203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:21.234148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129444+0000","last_change":"2023-11-26T07:22:22.129444+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129444+0000","last_undegraded":"2023-11-26T07:22:22.129444+0000","last_fullsized":"2023-11-26T07:22:22.129444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153379+0000","last_change":"2023-11-26T07:22:22.150877+0000","last_active":"2023-11-26T07:22:22.153379+0000","last_peered":"2023-11-26T07:22:22.153379+0000","last_clean":"2023-11-26T07:22:22.153379+0000","last_became_active":"2023-11-26T07:22:22.150413+0000","last_became_peered":"2023-11-26T07:22:22.150413+0000","last_unstale":"2023-11-26T07:22:22.153379+0000","last_undegraded":"2023-11-26T07:22:22.153379+0000","last_fullsized":"2023-11-26T07:22:22.153379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:21.234148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129600+0000","last_change":"2023-11-26T07:22:22.129600+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129600+0000","last_undegraded":"2023-11-26T07:22:22.129600+0000","last_fullsized":"2023-11-26T07:22:22.129600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152466+0000","last_change":"2023-11-26T07:22:22.148977+0000","last_active":"2023-11-26T07:22:22.152466+0000","last_peered":"2023-11-26T07:22:22.152466+0000","last_clean":"2023-11-26T07:22:22.152466+0000","last_became_active":"2023-11-26T07:22:22.148703+0000","last_became_peered":"2023-11-26T07:22:22.148703+0000","last_unstale":"2023-11-26T07:22:22.152466+0000","last_undegraded":"2023-11-26T07:22:22.152466+0000","last_fullsized":"2023-11-26T07:22:22.152466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:35.311859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.133925+0000","last_change":"2023-11-26T07:22:22.133925+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.133925+0000","last_undegraded":"2023-11-26T07:22:22.133925+0000","last_fullsized":"2023-11-26T07:22:22.133925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153820+0000","last_change":"2023-11-26T07:22:22.151630+0000","last_active":"2023-11-26T07:22:22.153820+0000","last_peered":"2023-11-26T07:22:22.153820+0000","last_clean":"2023-11-26T07:22:22.153820+0000","last_became_active":"2023-11-26T07:22:22.151230+0000","last_became_peered":"2023-11-26T07:22:22.151230+0000","last_unstale":"2023-11-26T07:22:22.153820+0000","last_undegraded":"2023-11-26T07:22:22.153820+0000","last_fullsized":"2023-11-26T07:22:22.153820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:38.450764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128444+0000","last_change":"2023-11-26T07:22:22.128444+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128444+0000","last_undegraded":"2023-11-26T07:22:22.128444+0000","last_fullsized":"2023-11-26T07:22:22.128444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.158977+0000","last_change":"2023-11-26T07:22:22.153463+0000","last_active":"2023-11-26T07:22:22.158977+0000","last_peered":"2023-11-26T07:22:22.158977+0000","last_clean":"2023-11-26T07:22:22.158977+0000","last_became_active":"2023-11-26T07:22:22.153132+0000","last_became_peered":"2023-11-26T07:22:22.153132+0000","last_unstale":"2023-11-26T07:22:22.158977+0000","last_undegraded":"2023-11-26T07:22:22.158977+0000","last_fullsized":"2023-11-26T07:22:22.158977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:15.107224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.142692+0000","last_change":"2023-11-26T07:22:22.142692+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.142692+0000","last_undegraded":"2023-11-26T07:22:22.142692+0000","last_fullsized":"2023-11-26T07:22:22.142692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152289+0000","last_change":"2023-11-26T07:22:22.148229+0000","last_active":"2023-11-26T07:22:22.152289+0000","last_peered":"2023-11-26T07:22:22.152289+0000","last_clean":"2023-11-26T07:22:22.152289+0000","last_became_active":"2023-11-26T07:22:22.147927+0000","last_became_peered":"2023-11-26T07:22:22.147927+0000","last_unstale":"2023-11-26T07:22:22.152289+0000","last_undegraded":"2023-11-26T07:22:22.152289+0000","last_fullsized":"2023-11-26T07:22:22.152289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:21:24.263764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132776+0000","last_change":"2023-11-26T07:22:22.132776+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132776+0000","last_undegraded":"2023-11-26T07:22:22.132776+0000","last_fullsized":"2023-11-26T07:22:22.132776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146154+0000","last_change":"2023-11-26T07:22:22.144638+0000","last_active":"2023-11-26T07:22:22.146154+0000","last_peered":"2023-11-26T07:22:22.146154+0000","last_clean":"2023-11-26T07:22:22.146154+0000","last_became_active":"2023-11-26T07:22:22.144230+0000","last_became_peered":"2023-11-26T07:22:22.144230+0000","last_unstale":"2023-11-26T07:22:22.146154+0000","last_undegraded":"2023-11-26T07:22:22.146154+0000","last_fullsized":"2023-11-26T07:22:22.146154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:33.883653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128946+0000","last_change":"2023-11-26T07:22:22.128946+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128946+0000","last_undegraded":"2023-11-26T07:22:22.128946+0000","last_fullsized":"2023-11-26T07:22:22.128946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.155657+0000","last_change":"2023-11-26T07:22:22.153839+0000","last_active":"2023-11-26T07:22:22.155657+0000","last_peered":"2023-11-26T07:22:22.155657+0000","last_clean":"2023-11-26T07:22:22.155657+0000","last_became_active":"2023-11-26T07:22:22.153535+0000","last_became_peered":"2023-11-26T07:22:22.153535+0000","last_unstale":"2023-11-26T07:22:22.155657+0000","last_undegraded":"2023-11-26T07:22:22.155657+0000","last_fullsized":"2023-11-26T07:22:22.155657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:56.712024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130940+0000","last_change":"2023-11-26T07:22:22.130940+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130940+0000","last_undegraded":"2023-11-26T07:22:22.130940+0000","last_fullsized":"2023-11-26T07:22:22.130940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146666+0000","last_change":"2023-11-26T07:22:22.140913+0000","last_active":"2023-11-26T07:22:22.146666+0000","last_peered":"2023-11-26T07:22:22.146666+0000","last_clean":"2023-11-26T07:22:22.146666+0000","last_became_active":"2023-11-26T07:22:22.140643+0000","last_became_peered":"2023-11-26T07:22:22.140643+0000","last_unstale":"2023-11-26T07:22:22.146666+0000","last_undegraded":"2023-11-26T07:22:22.146666+0000","last_fullsized":"2023-11-26T07:22:22.146666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:15.107224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130873+0000","last_change":"2023-11-26T07:22:22.130873+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130873+0000","last_undegraded":"2023-11-26T07:22:22.130873+0000","last_fullsized":"2023-11-26T07:22:22.130873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153050+0000","last_change":"2023-11-26T07:22:22.151535+0000","last_active":"2023-11-26T07:22:22.153050+0000","last_peered":"2023-11-26T07:22:22.153050+0000","last_clean":"2023-11-26T07:22:22.153050+0000","last_became_active":"2023-11-26T07:22:22.151097+0000","last_became_peered":"2023-11-26T07:22:22.151097+0000","last_unstale":"2023-11-26T07:22:22.153050+0000","last_undegraded":"2023-11-26T07:22:22.153050+0000","last_fullsized":"2023-11-26T07:22:22.153050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:49.532057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.141167+0000","last_change":"2023-11-26T07:22:22.141167+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.141167+0000","last_undegraded":"2023-11-26T07:22:22.141167+0000","last_fullsized":"2023-11-26T07:22:22.141167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152282+0000","last_change":"2023-11-26T07:22:22.149767+0000","last_active":"2023-11-26T07:22:22.152282+0000","last_peered":"2023-11-26T07:22:22.152282+0000","last_clean":"2023-11-26T07:22:22.152282+0000","last_became_active":"2023-11-26T07:22:22.149435+0000","last_became_peered":"2023-11-26T07:22:22.149435+0000","last_unstale":"2023-11-26T07:22:22.152282+0000","last_undegraded":"2023-11-26T07:22:22.152282+0000","last_fullsized":"2023-11-26T07:22:22.152282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:21:24.263764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139846+0000","last_change":"2023-11-26T07:22:22.139846+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139846+0000","last_undegraded":"2023-11-26T07:22:22.139846+0000","last_fullsized":"2023-11-26T07:22:22.139846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.143475+0000","last_change":"2023-11-26T07:22:22.138119+0000","last_active":"2023-11-26T07:22:22.143475+0000","last_peered":"2023-11-26T07:22:22.143475+0000","last_clean":"2023-11-26T07:22:22.143475+0000","last_became_active":"2023-11-26T07:22:22.137770+0000","last_became_peered":"2023-11-26T07:22:22.137770+0000","last_unstale":"2023-11-26T07:22:22.143475+0000","last_undegraded":"2023-11-26T07:22:22.143475+0000","last_fullsized":"2023-11-26T07:22:22.143475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:25.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139244+0000","last_change":"2023-11-26T07:22:22.139244+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139244+0000","last_undegraded":"2023-11-26T07:22:22.139244+0000","last_fullsized":"2023-11-26T07:22:22.139244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151664+0000","last_change":"2023-11-26T07:22:22.145272+0000","last_active":"2023-11-26T07:22:22.151664+0000","last_peered":"2023-11-26T07:22:22.151664+0000","last_clean":"2023-11-26T07:22:22.151664+0000","last_became_active":"2023-11-26T07:22:22.145011+0000","last_became_peered":"2023-11-26T07:22:22.145011+0000","last_unstale":"2023-11-26T07:22:22.151664+0000","last_undegraded":"2023-11-26T07:22:22.151664+0000","last_fullsized":"2023-11-26T07:22:22.151664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:17.509070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134871+0000","last_change":"2023-11-26T07:22:22.134871+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134871+0000","last_undegraded":"2023-11-26T07:22:22.134871+0000","last_fullsized":"2023-11-26T07:22:22.134871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153970+0000","last_change":"2023-11-26T07:22:22.149360+0000","last_active":"2023-11-26T07:22:22.153970+0000","last_peered":"2023-11-26T07:22:22.153970+0000","last_clean":"2023-11-26T07:22:22.153970+0000","last_became_active":"2023-11-26T07:22:22.149068+0000","last_became_peered":"2023-11-26T07:22:22.149068+0000","last_unstale":"2023-11-26T07:22:22.153970+0000","last_undegraded":"2023-11-26T07:22:22.153970+0000","last_fullsized":"2023-11-26T07:22:22.153970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:29.708891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128109+0000","last_change":"2023-11-26T07:22:22.128109+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128109+0000","last_undegraded":"2023-11-26T07:22:22.128109+0000","last_fullsized":"2023-11-26T07:22:22.128109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153325+0000","last_change":"2023-11-26T07:22:22.151868+0000","last_active":"2023-11-26T07:22:22.153325+0000","last_peered":"2023-11-26T07:22:22.153325+0000","last_clean":"2023-11-26T07:22:22.153325+0000","last_became_active":"2023-11-26T07:22:22.151535+0000","last_became_peered":"2023-11-26T07:22:22.151535+0000","last_unstale":"2023-11-26T07:22:22.153325+0000","last_undegraded":"2023-11-26T07:22:22.153325+0000","last_fullsized":"2023-11-26T07:22:22.153325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:33.883653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135370+0000","last_change":"2023-11-26T07:22:22.135370+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135370+0000","last_undegraded":"2023-11-26T07:22:22.135370+0000","last_fullsized":"2023-11-26T07:22:22.135370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152260+0000","last_change":"2023-11-26T07:22:22.145842+0000","last_active":"2023-11-26T07:22:22.152260+0000","last_peered":"2023-11-26T07:22:22.152260+0000","last_clean":"2023-11-26T07:22:22.152260+0000","last_became_active":"2023-11-26T07:22:22.145573+0000","last_became_peered":"2023-11-26T07:22:22.145573+0000","last_unstale":"2023-11-26T07:22:22.152260+0000","last_undegraded":"2023-11-26T07:22:22.152260+0000","last_fullsized":"2023-11-26T07:22:22.152260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:49.532057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134114+0000","last_change":"2023-11-26T07:22:22.134114+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134114+0000","last_undegraded":"2023-11-26T07:22:22.134114+0000","last_fullsized":"2023-11-26T07:22:22.134114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.145214+0000","last_change":"2023-11-26T07:22:22.140377+0000","last_active":"2023-11-26T07:22:22.145214+0000","last_peered":"2023-11-26T07:22:22.145214+0000","last_clean":"2023-11-26T07:22:22.145214+0000","last_became_active":"2023-11-26T07:22:22.138993+0000","last_became_peered":"2023-11-26T07:22:22.138993+0000","last_unstale":"2023-11-26T07:22:22.145214+0000","last_undegraded":"2023-11-26T07:22:22.145214+0000","last_fullsized":"2023-11-26T07:22:22.145214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:57.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128923+0000","last_change":"2023-11-26T07:22:22.128923+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128923+0000","last_undegraded":"2023-11-26T07:22:22.128923+0000","last_fullsized":"2023-11-26T07:22:22.128923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.154062+0000","last_change":"2023-11-26T07:22:22.152564+0000","last_active":"2023-11-26T07:22:22.154062+0000","last_peered":"2023-11-26T07:22:22.154062+0000","last_clean":"2023-11-26T07:22:22.154062+0000","last_became_active":"2023-11-26T07:22:22.152161+0000","last_became_peered":"2023-11-26T07:22:22.152161+0000","last_unstale":"2023-11-26T07:22:22.154062+0000","last_undegraded":"2023-11-26T07:22:22.154062+0000","last_fullsized":"2023-11-26T07:22:22.154062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:08.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.137359+0000","last_change":"2023-11-26T07:22:22.137359+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.137359+0000","last_undegraded":"2023-11-26T07:22:22.137359+0000","last_fullsized":"2023-11-26T07:22:22.137359+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151089+0000","last_change":"2023-11-26T07:22:22.145038+0000","last_active":"2023-11-26T07:22:22.151089+0000","last_peered":"2023-11-26T07:22:22.151089+0000","last_clean":"2023-11-26T07:22:22.151089+0000","last_became_active":"2023-11-26T07:22:22.144754+0000","last_became_peered":"2023-11-26T07:22:22.144754+0000","last_unstale":"2023-11-26T07:22:22.151089+0000","last_undegraded":"2023-11-26T07:22:22.151089+0000","last_fullsized":"2023-11-26T07:22:22.151089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:53:45.151325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132275+0000","last_change":"2023-11-26T07:22:22.132275+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132275+0000","last_undegraded":"2023-11-26T07:22:22.132275+0000","last_fullsized":"2023-11-26T07:22:22.132275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146987+0000","last_change":"2023-11-26T07:22:22.140578+0000","last_active":"2023-11-26T07:22:22.146987+0000","last_peered":"2023-11-26T07:22:22.146987+0000","last_clean":"2023-11-26T07:22:22.146987+0000","last_became_active":"2023-11-26T07:22:22.140222+0000","last_became_peered":"2023-11-26T07:22:22.140222+0000","last_unstale":"2023-11-26T07:22:22.146987+0000","last_undegraded":"2023-11-26T07:22:22.146987+0000","last_fullsized":"2023-11-26T07:22:22.146987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:59.850726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129550+0000","last_change":"2023-11-26T07:22:22.129550+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129550+0000","last_undegraded":"2023-11-26T07:22:22.129550+0000","last_fullsized":"2023-11-26T07:22:22.129550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.149688+0000","last_change":"2023-11-26T07:22:22.147493+0000","last_active":"2023-11-26T07:22:22.149688+0000","last_peered":"2023-11-26T07:22:22.149688+0000","last_clean":"2023-11-26T07:22:22.149688+0000","last_became_active":"2023-11-26T07:22:22.147187+0000","last_became_peered":"2023-11-26T07:22:22.147187+0000","last_unstale":"2023-11-26T07:22:22.149688+0000","last_undegraded":"2023-11-26T07:22:22.149688+0000","last_fullsized":"2023-11-26T07:22:22.149688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.357020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131250+0000","last_change":"2023-11-26T07:22:22.131250+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131250+0000","last_undegraded":"2023-11-26T07:22:22.131250+0000","last_fullsized":"2023-11-26T07:22:22.131250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153430+0000","last_change":"2023-11-26T07:22:22.148994+0000","last_active":"2023-11-26T07:22:22.153430+0000","last_peered":"2023-11-26T07:22:22.153430+0000","last_clean":"2023-11-26T07:22:22.153430+0000","last_became_active":"2023-11-26T07:22:22.147943+0000","last_became_peered":"2023-11-26T07:22:22.147943+0000","last_unstale":"2023-11-26T07:22:22.153430+0000","last_undegraded":"2023-11-26T07:22:22.153430+0000","last_fullsized":"2023-11-26T07:22:22.153430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:53:21.611291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129655+0000","last_change":"2023-11-26T07:22:22.129655+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129655+0000","last_undegraded":"2023-11-26T07:22:22.129655+0000","last_fullsized":"2023-11-26T07:22:22.129655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146830+0000","last_change":"2023-11-26T07:22:22.139465+0000","last_active":"2023-11-26T07:22:22.146830+0000","last_peered":"2023-11-26T07:22:22.146830+0000","last_clean":"2023-11-26T07:22:22.146830+0000","last_became_active":"2023-11-26T07:22:22.139180+0000","last_became_peered":"2023-11-26T07:22:22.139180+0000","last_unstale":"2023-11-26T07:22:22.146830+0000","last_undegraded":"2023-11-26T07:22:22.146830+0000","last_fullsized":"2023-11-26T07:22:22.146830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:06.932370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.143648+0000","last_change":"2023-11-26T07:22:22.143648+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.143648+0000","last_undegraded":"2023-11-26T07:22:22.143648+0000","last_fullsized":"2023-11-26T07:22:22.143648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151859+0000","last_change":"2023-11-26T07:22:22.146130+0000","last_active":"2023-11-26T07:22:22.151859+0000","last_peered":"2023-11-26T07:22:22.151859+0000","last_clean":"2023-11-26T07:22:22.151859+0000","last_became_active":"2023-11-26T07:22:22.144230+0000","last_became_peered":"2023-11-26T07:22:22.144230+0000","last_unstale":"2023-11-26T07:22:22.151859+0000","last_undegraded":"2023-11-26T07:22:22.151859+0000","last_fullsized":"2023-11-26T07:22:22.151859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:44.104269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132436+0000","last_change":"2023-11-26T07:22:22.132436+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132436+0000","last_undegraded":"2023-11-26T07:22:22.132436+0000","last_fullsized":"2023-11-26T07:22:22.132436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151642+0000","last_change":"2023-11-26T07:22:22.145175+0000","last_active":"2023-11-26T07:22:22.151642+0000","last_peered":"2023-11-26T07:22:22.151642+0000","last_clean":"2023-11-26T07:22:22.151642+0000","last_became_active":"2023-11-26T07:22:22.144917+0000","last_became_peered":"2023-11-26T07:22:22.144917+0000","last_unstale":"2023-11-26T07:22:22.151642+0000","last_undegraded":"2023-11-26T07:22:22.151642+0000","last_fullsized":"2023-11-26T07:22:22.151642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:15.998685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129092+0000","last_change":"2023-11-26T07:22:22.129092+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129092+0000","last_undegraded":"2023-11-26T07:22:22.129092+0000","last_fullsized":"2023-11-26T07:22:22.129092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.147011+0000","last_change":"2023-11-26T07:22:22.143708+0000","last_active":"2023-11-26T07:22:22.147011+0000","last_peered":"2023-11-26T07:22:22.147011+0000","last_clean":"2023-11-26T07:22:22.147011+0000","last_became_active":"2023-11-26T07:22:22.143395+0000","last_became_peered":"2023-11-26T07:22:22.143395+0000","last_unstale":"2023-11-26T07:22:22.147011+0000","last_undegraded":"2023-11-26T07:22:22.147011+0000","last_fullsized":"2023-11-26T07:22:22.147011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:58:46.089931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.136808+0000","last_change":"2023-11-26T07:22:22.136808+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.136808+0000","last_undegraded":"2023-11-26T07:22:22.136808+0000","last_fullsized":"2023-11-26T07:22:22.136808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.144998+0000","last_change":"2023-11-26T07:22:22.139927+0000","last_active":"2023-11-26T07:22:22.144998+0000","last_peered":"2023-11-26T07:22:22.144998+0000","last_clean":"2023-11-26T07:22:22.144998+0000","last_became_active":"2023-11-26T07:22:22.139628+0000","last_became_peered":"2023-11-26T07:22:22.139628+0000","last_unstale":"2023-11-26T07:22:22.144998+0000","last_undegraded":"2023-11-26T07:22:22.144998+0000","last_fullsized":"2023-11-26T07:22:22.144998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:44:29.019321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129561+0000","last_change":"2023-11-26T07:22:22.129561+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129561+0000","last_undegraded":"2023-11-26T07:22:22.129561+0000","last_fullsized":"2023-11-26T07:22:22.129561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152296+0000","last_change":"2023-11-26T07:22:22.146618+0000","last_active":"2023-11-26T07:22:22.152296+0000","last_peered":"2023-11-26T07:22:22.152296+0000","last_clean":"2023-11-26T07:22:22.152296+0000","last_became_active":"2023-11-26T07:22:22.146214+0000","last_became_peered":"2023-11-26T07:22:22.146214+0000","last_unstale":"2023-11-26T07:22:22.152296+0000","last_undegraded":"2023-11-26T07:22:22.152296+0000","last_fullsized":"2023-11-26T07:22:22.152296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:25.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.137697+0000","last_change":"2023-11-26T07:22:22.137697+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.137697+0000","last_undegraded":"2023-11-26T07:22:22.137697+0000","last_fullsized":"2023-11-26T07:22:22.137697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152272+0000","last_change":"2023-11-26T07:22:22.148139+0000","last_active":"2023-11-26T07:22:22.152272+0000","last_peered":"2023-11-26T07:22:22.152272+0000","last_clean":"2023-11-26T07:22:22.152272+0000","last_became_active":"2023-11-26T07:22:22.147844+0000","last_became_peered":"2023-11-26T07:22:22.147844+0000","last_unstale":"2023-11-26T07:22:22.152272+0000","last_undegraded":"2023-11-26T07:22:22.152272+0000","last_fullsized":"2023-11-26T07:22:22.152272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:57.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132256+0000","last_change":"2023-11-26T07:22:22.132256+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132256+0000","last_undegraded":"2023-11-26T07:22:22.132256+0000","last_fullsized":"2023-11-26T07:22:22.132256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146674+0000","last_change":"2023-11-26T07:22:22.140709+0000","last_active":"2023-11-26T07:22:22.146674+0000","last_peered":"2023-11-26T07:22:22.146674+0000","last_clean":"2023-11-26T07:22:22.146674+0000","last_became_active":"2023-11-26T07:22:22.140379+0000","last_became_peered":"2023-11-26T07:22:22.140379+0000","last_unstale":"2023-11-26T07:22:22.146674+0000","last_undegraded":"2023-11-26T07:22:22.146674+0000","last_fullsized":"2023-11-26T07:22:22.146674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:41.677508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128291+0000","last_change":"2023-11-26T07:22:22.128291+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128291+0000","last_undegraded":"2023-11-26T07:22:22.128291+0000","last_fullsized":"2023-11-26T07:22:22.128291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152324+0000","last_change":"2023-11-26T07:22:22.148901+0000","last_active":"2023-11-26T07:22:22.152324+0000","last_peered":"2023-11-26T07:22:22.152324+0000","last_clean":"2023-11-26T07:22:22.152324+0000","last_became_active":"2023-11-26T07:22:22.148566+0000","last_became_peered":"2023-11-26T07:22:22.148566+0000","last_unstale":"2023-11-26T07:22:22.152324+0000","last_undegraded":"2023-11-26T07:22:22.152324+0000","last_fullsized":"2023-11-26T07:22:22.152324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:58:46.089931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127322+0000","last_change":"2023-11-26T07:22:22.127322+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127322+0000","last_undegraded":"2023-11-26T07:22:22.127322+0000","last_fullsized":"2023-11-26T07:22:22.127322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153518+0000","last_change":"2023-11-26T07:22:22.152219+0000","last_active":"2023-11-26T07:22:22.153518+0000","last_peered":"2023-11-26T07:22:22.153518+0000","last_clean":"2023-11-26T07:22:22.153518+0000","last_became_active":"2023-11-26T07:22:22.151927+0000","last_became_peered":"2023-11-26T07:22:22.151927+0000","last_unstale":"2023-11-26T07:22:22.153518+0000","last_undegraded":"2023-11-26T07:22:22.153518+0000","last_fullsized":"2023-11-26T07:22:22.153518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:17.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"69'128","reported_seq":251,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-26T07:22:21.128526+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:22:21.128526+0000","last_peered":"2023-11-26T07:22:21.128526+0000","last_clean":"2023-11-26T07:22:21.128526+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:22:21.128526+0000","last_undegraded":"2023-11-26T07:22:21.128526+0000","last_fullsized":"2023-11-26T07:22:21.128526+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":992140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.56599999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":0.995},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84799999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45200000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.48499999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.92500000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71399999999999997},"min":{"1min":0.56899999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84499999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.58999999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92500000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.50600000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":1.016,"15min":1.016},"last":0.79500000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.85099999999999998,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":0.81899999999999995},"min":{"1min":0.68100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93500000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.71099999999999997},"min":{"1min":0.47199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73299999999999998,"15min":0.77500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95899999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.84899999999999998},"min":{"1min":0.58699999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91500000000000004,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82099999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.499,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93400000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.82999999999999996},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.012,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":0.86699999999999999,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.222000000000001,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":3.4020000000000001,"5min":1.246,"15min":0.91500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":54.737000000000002,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.80600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.40400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.1880000000000002,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.6689999999999996,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.96599999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.439,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.8479999999999999,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.72499999999999998,"15min":0.68899999999999995},"min":{"1min":0.57099999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0170000000000003,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.73199999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.81299999999999994,"15min":0.751},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.4870000000000001,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.601,"5min":0.94399999999999995,"15min":0.874},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":18.77,"5min":18.77,"15min":18.77},"last":0.78100000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.53100000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.749},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.121,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.88100000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67400000000000004,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.77900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.93300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.626,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.64400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.87,"5min":1.073,"15min":1.073},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.96499999999999997,"5min":1.036,"15min":1.036},"last":0.77700000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.012},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.49099999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79000000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.79000000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.71299999999999997},"min":{"1min":0.495,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.028,"5min":1.097,"15min":1.097},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.88700000000000001,"5min":1.071,"15min":1.071},"last":0.80700000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65200000000000002,"15min":0.68600000000000005},"min":{"1min":0.42699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.137,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.77000000000000002},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.0029999999999999,"5min":1.206,"15min":1.206},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.62,"15min":0.67000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93100000000000005,"5min":1.143,"15min":1.143},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.73199999999999998},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.1020000000000001,"5min":1.228,"15min":1.228},"last":0.84599999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0580000000000001,"5min":1.264,"15min":1.264},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.077,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.87}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84899999999999998,"5min":1.016,"15min":1.016},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93100000000000005,"5min":1.26,"15min":1.26},"last":0.72099999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90200000000000002,"5min":1.077,"15min":1.077},"last":0.69699999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.302,"15min":1.302},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.753},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":1.181,"15min":1.181},"last":0.77000000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.747},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97999999999999998,"5min":1.702,"15min":1.702},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.78700000000000003},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0409999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.72999999999999998},"min":{"1min":0.42199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97099999999999997,"5min":1.129,"15min":1.129},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.78800000000000003},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.012,"15min":1.012},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88300000000000001,"5min":1.016,"15min":1.016},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.81200000000000006},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.0980000000000001,"5min":1.242,"15min":1.242},"last":0.67600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.68999999999999995,"5min":1.294,"15min":1.294},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.40500000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":1.456,"15min":1.456},"last":0.434},{"interface":"front","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.52900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.212,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2649999999999999,"5min":1.395,"15min":1.395},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1080000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.504}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.64700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.333,"15min":1.333},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.228,"5min":1.347,"15min":1.347},"last":0.59699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.397,"15min":1.397},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1859999999999999,"5min":1.363,"15min":1.363},"last":0.66200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56000000000000005,"15min":0.64800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65800000000000003,"5min":3.931,"15min":3.931},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.71099999999999997},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60199999999999998,"15min":0.69199999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95499999999999996,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.69299999999999995},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.7650000000000001,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.434}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.751,"15min":0.90300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98899999999999999,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.59899999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.85199999999999998,"15min":0.92900000000000005},"min":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1240000000000001,"5min":4.431,"15min":4.431},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.274,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.63500000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.61199999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89400000000000002,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8069999999999999,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.754}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.98399999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.222,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.93899999999999995,"15min":0.995},"min":{"1min":0.621,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.4740000000000002,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.052,"5min":1.609,"15min":1.609},"last":0.61899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.57499999999999996,"5min":1.034,"15min":1.034},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69999999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.33000000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66900000000000004,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.499,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48599999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95599999999999996,"5min":1.143,"15min":1.143},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.216,"15min":1.216},"last":0.75800000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.58999999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97399999999999998,"5min":1.262,"15min":1.262},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.998,"5min":1.54,"15min":1.54},"last":0.67500000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.61599999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.054,"5min":1.145,"15min":1.145},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.91600000000000004,"5min":1.333,"15min":1.333},"last":0.69899999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95699999999999996,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0189999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1536000,"data_stored":991121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.45300000000000001},"min":{"1min":0.29799999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.68300000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.48499999999999999,"15min":0.47799999999999998},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.78600000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79700000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.438}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.38400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83599999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.81499999999999995},"min":{"1min":0.59499999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.67800000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":1.0980000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.63400000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.76700000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.64800000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.61899999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.80400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1180000000000001,"5min":1.123,"15min":1.123},"last":1.079},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.83099999999999996,"15min":0.81899999999999995},"min":{"1min":0.69399999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.0349999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:22:27.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:22:27.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc37df640 1 Processor -- start 2023-11-26T07:22:27.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc37df640 1 -- start start 2023-11-26T07:22:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc37df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc15d7b0 0x7facbc0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc37df640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facbc0cead0 0x7facbc0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc37df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facbc0cefd0 con 0x7facbc0d1eb0 2023-11-26T07:22:27.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc37df640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facbc0d2340 con 0x7facbc0cead0 2023-11-26T07:22:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc37df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facbc0cf2b0 con 0x7facbc15d7b0 2023-11-26T07:22:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc157b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc15d7b0 0x7facbc0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc0d7a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7facbc0cead0 0x7facbc0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc157b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc15d7b0 0x7facbc0d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:55690/0 (socket says 172.21.15.31:55690) 2023-11-26T07:22:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc157b640 1 -- 172.21.15.31:0/3673474783 learned_addr learned my addr 172.21.15.31:0/3673474783 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc157b640 1 -- 172.21.15.31:0/3673474783 >> v2:172.21.15.97:3300/0 conn(0x7facbc0cead0 msgr2=0x7facbc0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:27.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc157b640 1 --2- 172.21.15.31:0/3673474783 >> v2:172.21.15.97:3300/0 conn(0x7facbc0cead0 0x7facbc0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc157b640 1 -- 172.21.15.31:0/3673474783 >> v1:172.21.15.97:6789/0 conn(0x7facbc0d1eb0 legacy=0x7facbc0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.865+0000 7facc157b640 1 -- 172.21.15.31:0/3673474783 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc06b3f0 con 0x7facbc15d7b0 2023-11-26T07:22:27.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facc157b640 1 --2- 172.21.15.31:0/3673474783 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc15d7b0 0x7facbc0d10d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7facb8009870 tx=0x7facb801bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e254618e1a539757 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facb3fff640 1 -- 172.21.15.31:0/3673474783 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80a4760 con 0x7facbc15d7b0 2023-11-26T07:22:27.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facb3fff640 1 -- 172.21.15.31:0/3673474783 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb80a48c0 con 0x7facbc15d7b0 2023-11-26T07:22:27.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facb3fff640 1 -- 172.21.15.31:0/3673474783 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80a4c50 con 0x7facbc15d7b0 2023-11-26T07:22:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facc37df640 1 -- 172.21.15.31:0/3673474783 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc15d7b0 msgr2=0x7facbc0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facc37df640 1 --2- 172.21.15.31:0/3673474783 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc15d7b0 0x7facbc0d10d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7facb8009870 tx=0x7facb801bfc0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facc37df640 1 -- 172.21.15.31:0/3673474783 shutdown_connections 2023-11-26T07:22:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facc37df640 1 --2- 172.21.15.31:0/3673474783 >> v2:172.21.15.97:3300/0 conn(0x7facbc0cead0 0x7facbc0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facc37df640 1 --2- 172.21.15.31:0/3673474783 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc15d7b0 0x7facbc0d10d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facc37df640 1 -- 172.21.15.31:0/3673474783 >> 172.21.15.31:0/3673474783 conn(0x7facbc0d33b0 msgr2=0x7facbc0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facc37df640 1 -- 172.21.15.31:0/3673474783 shutdown_connections 2023-11-26T07:22:27.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.869+0000 7facc37df640 1 -- 172.21.15.31:0/3673474783 wait complete. 2023-11-26T07:22:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc37df640 1 Processor -- start 2023-11-26T07:22:27.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc37df640 1 -- start start 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc37df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0cead0 0x7facbc156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc37df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc0d1eb0 0x7facbc0cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc37df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facbc15d7b0 0x7facbc0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc37df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc0b76c0 con 0x7facbc0cead0 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc37df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc0cb600 con 0x7facbc15d7b0 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc37df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facbc0cb8e0 con 0x7facbc0d1eb0 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc157b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0cead0 0x7facbc156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc157b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0cead0 0x7facbc156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37936/0 (socket says 172.21.15.31:37936) 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc157b640 1 -- 172.21.15.31:0/4068106304 learned_addr learned my addr 172.21.15.31:0/4068106304 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc0d7a640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc0d1eb0 0x7facbc0cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc1d7c640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facbc15d7b0 0x7facbc0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc157b640 1 -- 172.21.15.31:0/4068106304 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc0d1eb0 msgr2=0x7facbc0cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc157b640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc0d1eb0 0x7facbc0cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc157b640 1 -- 172.21.15.31:0/4068106304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facbc15d7b0 msgr2=0x7facbc0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc157b640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facbc15d7b0 0x7facbc0b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc157b640 1 -- 172.21.15.31:0/4068106304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb80094d0 con 0x7facbc0cead0 2023-11-26T07:22:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc157b640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0cead0 0x7facbc156400 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7facb80af040 tx=0x7facb80190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facb1ffb640 1 -- 172.21.15.31:0/4068106304 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80ac850 con 0x7facbc0cead0 2023-11-26T07:22:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facb1ffb640 1 -- 172.21.15.31:0/4068106304 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb80ac9d0 con 0x7facbc0cead0 2023-11-26T07:22:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facb1ffb640 1 -- 172.21.15.31:0/4068106304 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80acd80 con 0x7facbc0cead0 2023-11-26T07:22:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.873+0000 7facc37df640 1 -- 172.21.15.31:0/4068106304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facbc0b72e0 con 0x7facbc0cead0 2023-11-26T07:22:27.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.877+0000 7facc37df640 1 -- 172.21.15.31:0/4068106304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facbc156940 con 0x7facbc0cead0 2023-11-26T07:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.877+0000 7facc37df640 1 -- 172.21.15.31:0/4068106304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facbc04e850 con 0x7facbc0cead0 2023-11-26T07:22:27.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.877+0000 7facb1ffb640 1 -- 172.21.15.31:0/4068106304 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7facb80b3040 con 0x7facbc0cead0 2023-11-26T07:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.877+0000 7facb1ffb640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faca0066590 0x7faca0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.877+0000 7facb1ffb640 1 -- 172.21.15.31:0/4068106304 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facb811da20 con 0x7facbc0cead0 2023-11-26T07:22:27.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.877+0000 7facc0d7a640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faca0066590 0x7faca0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:27.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.877+0000 7facc0d7a640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faca0066590 0x7faca0068a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7facbc0b3960 tx=0x7facac009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:27.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:27.881+0000 7facb1ffb640 1 -- 172.21.15.31:0/4068106304 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7facb80eaf40 con 0x7facbc0cead0 2023-11-26T07:22:28.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.037+0000 7facc37df640 1 -- 172.21.15.31:0/4068106304 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7facbc0b1330 con 0x7faca0066590 2023-11-26T07:22:28.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.045+0000 7facb1ffb640 1 -- 172.21.15.31:0/4068106304 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283525 (secure 0 0 0) 0x7facbc0b1330 con 0x7faca0066590 2023-11-26T07:22:28.051 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:28.053 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:22:28.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 -- 172.21.15.31:0/4068106304 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faca0066590 msgr2=0x7faca0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faca0066590 0x7faca0068a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7facbc0b3960 tx=0x7facac009190 comp rx=0 tx=0).stop 2023-11-26T07:22:28.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 -- 172.21.15.31:0/4068106304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0cead0 msgr2=0x7facbc156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0cead0 0x7facbc156400 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7facb80af040 tx=0x7facb80190b0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 -- 172.21.15.31:0/4068106304 shutdown_connections 2023-11-26T07:22:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faca0066590 0x7faca0068a50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7facbc15d7b0 0x7facbc0b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7facbc0d1eb0 0x7facbc0cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 --2- 172.21.15.31:0/4068106304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0cead0 0x7facbc156400 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 -- 172.21.15.31:0/4068106304 >> 172.21.15.31:0/4068106304 conn(0x7facbc0d33b0 msgr2=0x7facbc0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 -- 172.21.15.31:0/4068106304 shutdown_connections 2023-11-26T07:22:28.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.049+0000 7facc37df640 1 -- 172.21.15.31:0/4068106304 wait complete. 2023-11-26T07:22:28.072 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-26T07:22:26.200917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258048,"kb_used_data":8548,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716672,"statfs":{"total":773094113280,"available":772829872128,"internally_reserved":0,"allocated":8753152,"data_stored":4393893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.080419"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131043+0000","last_change":"2023-11-26T07:22:22.131043+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131043+0000","last_undegraded":"2023-11-26T07:22:22.131043+0000","last_fullsized":"2023-11-26T07:22:22.131043+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129528+0000","last_change":"2023-11-26T07:22:22.129528+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129528+0000","last_undegraded":"2023-11-26T07:22:22.129528+0000","last_fullsized":"2023-11-26T07:22:22.129528+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.133354+0000","last_change":"2023-11-26T07:22:22.133354+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.133354+0000","last_undegraded":"2023-11-26T07:22:22.133354+0000","last_fullsized":"2023-11-26T07:22:22.133354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135222+0000","last_change":"2023-11-26T07:22:22.135222+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135222+0000","last_undegraded":"2023-11-26T07:22:22.135222+0000","last_fullsized":"2023-11-26T07:22:22.135222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.143110+0000","last_change":"2023-11-26T07:22:22.143110+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.143110+0000","last_undegraded":"2023-11-26T07:22:22.143110+0000","last_fullsized":"2023-11-26T07:22:22.143110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130504+0000","last_change":"2023-11-26T07:22:22.130504+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130504+0000","last_undegraded":"2023-11-26T07:22:22.130504+0000","last_fullsized":"2023-11-26T07:22:22.130504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130438+0000","last_change":"2023-11-26T07:22:22.130438+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130438+0000","last_undegraded":"2023-11-26T07:22:22.130438+0000","last_fullsized":"2023-11-26T07:22:22.130438+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130281+0000","last_change":"2023-11-26T07:22:22.130281+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130281+0000","last_undegraded":"2023-11-26T07:22:22.130281+0000","last_fullsized":"2023-11-26T07:22:22.130281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131416+0000","last_change":"2023-11-26T07:22:22.131416+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131416+0000","last_undegraded":"2023-11-26T07:22:22.131416+0000","last_fullsized":"2023-11-26T07:22:22.131416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131033+0000","last_change":"2023-11-26T07:22:22.131033+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131033+0000","last_undegraded":"2023-11-26T07:22:22.131033+0000","last_fullsized":"2023-11-26T07:22:22.131033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135119+0000","last_change":"2023-11-26T07:22:22.135119+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135119+0000","last_undegraded":"2023-11-26T07:22:22.135119+0000","last_fullsized":"2023-11-26T07:22:22.135119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127615+0000","last_change":"2023-11-26T07:22:22.127615+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127615+0000","last_undegraded":"2023-11-26T07:22:22.127615+0000","last_fullsized":"2023-11-26T07:22:22.127615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135357+0000","last_change":"2023-11-26T07:22:22.135357+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135357+0000","last_undegraded":"2023-11-26T07:22:22.135357+0000","last_fullsized":"2023-11-26T07:22:22.135357+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132827+0000","last_change":"2023-11-26T07:22:22.132827+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132827+0000","last_undegraded":"2023-11-26T07:22:22.132827+0000","last_fullsized":"2023-11-26T07:22:22.132827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129520+0000","last_change":"2023-11-26T07:22:22.129520+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129520+0000","last_undegraded":"2023-11-26T07:22:22.129520+0000","last_fullsized":"2023-11-26T07:22:22.129520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.142126+0000","last_change":"2023-11-26T07:22:22.142126+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.142126+0000","last_undegraded":"2023-11-26T07:22:22.142126+0000","last_fullsized":"2023-11-26T07:22:22.142126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139694+0000","last_change":"2023-11-26T07:22:22.139694+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139694+0000","last_undegraded":"2023-11-26T07:22:22.139694+0000","last_fullsized":"2023-11-26T07:22:22.139694+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134761+0000","last_change":"2023-11-26T07:22:22.134761+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134761+0000","last_undegraded":"2023-11-26T07:22:22.134761+0000","last_fullsized":"2023-11-26T07:22:22.134761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.138624+0000","last_change":"2023-11-26T07:22:22.138624+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.138624+0000","last_undegraded":"2023-11-26T07:22:22.138624+0000","last_fullsized":"2023-11-26T07:22:22.138624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134691+0000","last_change":"2023-11-26T07:22:22.134691+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134691+0000","last_undegraded":"2023-11-26T07:22:22.134691+0000","last_fullsized":"2023-11-26T07:22:22.134691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131538+0000","last_change":"2023-11-26T07:22:22.131538+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131538+0000","last_undegraded":"2023-11-26T07:22:22.131538+0000","last_fullsized":"2023-11-26T07:22:22.131538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129384+0000","last_change":"2023-11-26T07:22:22.129384+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129384+0000","last_undegraded":"2023-11-26T07:22:22.129384+0000","last_fullsized":"2023-11-26T07:22:22.129384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127661+0000","last_change":"2023-11-26T07:22:22.127661+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127661+0000","last_undegraded":"2023-11-26T07:22:22.127661+0000","last_fullsized":"2023-11-26T07:22:22.127661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131711+0000","last_change":"2023-11-26T07:22:22.131711+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131711+0000","last_undegraded":"2023-11-26T07:22:22.131711+0000","last_fullsized":"2023-11-26T07:22:22.131711+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134720+0000","last_change":"2023-11-26T07:22:22.134720+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134720+0000","last_undegraded":"2023-11-26T07:22:22.134720+0000","last_fullsized":"2023-11-26T07:22:22.134720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134643+0000","last_change":"2023-11-26T07:22:22.134643+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134643+0000","last_undegraded":"2023-11-26T07:22:22.134643+0000","last_fullsized":"2023-11-26T07:22:22.134643+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.143843+0000","last_change":"2023-11-26T07:22:22.143843+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.143843+0000","last_undegraded":"2023-11-26T07:22:22.143843+0000","last_fullsized":"2023-11-26T07:22:22.143843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130684+0000","last_change":"2023-11-26T07:22:22.130684+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130684+0000","last_undegraded":"2023-11-26T07:22:22.130684+0000","last_fullsized":"2023-11-26T07:22:22.130684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128838+0000","last_change":"2023-11-26T07:22:22.128838+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128838+0000","last_undegraded":"2023-11-26T07:22:22.128838+0000","last_fullsized":"2023-11-26T07:22:22.128838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128896+0000","last_change":"2023-11-26T07:22:22.128896+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128896+0000","last_undegraded":"2023-11-26T07:22:22.128896+0000","last_fullsized":"2023-11-26T07:22:22.128896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139213+0000","last_change":"2023-11-26T07:22:22.139213+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139213+0000","last_undegraded":"2023-11-26T07:22:22.139213+0000","last_fullsized":"2023-11-26T07:22:22.139213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127366+0000","last_change":"2023-11-26T07:22:22.127366+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127366+0000","last_undegraded":"2023-11-26T07:22:22.127366+0000","last_fullsized":"2023-11-26T07:22:22.127366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.144178+0000","last_change":"2023-11-26T07:22:22.144178+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.144178+0000","last_undegraded":"2023-11-26T07:22:22.144178+0000","last_fullsized":"2023-11-26T07:22:22.144178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151203+0000","last_change":"2023-11-26T07:22:22.149321+0000","last_active":"2023-11-26T07:22:22.151203+0000","last_peered":"2023-11-26T07:22:22.151203+0000","last_clean":"2023-11-26T07:22:22.151203+0000","last_became_active":"2023-11-26T07:22:22.148856+0000","last_became_peered":"2023-11-26T07:22:22.148856+0000","last_unstale":"2023-11-26T07:22:22.151203+0000","last_undegraded":"2023-11-26T07:22:22.151203+0000","last_fullsized":"2023-11-26T07:22:22.151203+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:21.234148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129444+0000","last_change":"2023-11-26T07:22:22.129444+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129444+0000","last_undegraded":"2023-11-26T07:22:22.129444+0000","last_fullsized":"2023-11-26T07:22:22.129444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153379+0000","last_change":"2023-11-26T07:22:22.150877+0000","last_active":"2023-11-26T07:22:22.153379+0000","last_peered":"2023-11-26T07:22:22.153379+0000","last_clean":"2023-11-26T07:22:22.153379+0000","last_became_active":"2023-11-26T07:22:22.150413+0000","last_became_peered":"2023-11-26T07:22:22.150413+0000","last_unstale":"2023-11-26T07:22:22.153379+0000","last_undegraded":"2023-11-26T07:22:22.153379+0000","last_fullsized":"2023-11-26T07:22:22.153379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:21.234148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129600+0000","last_change":"2023-11-26T07:22:22.129600+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129600+0000","last_undegraded":"2023-11-26T07:22:22.129600+0000","last_fullsized":"2023-11-26T07:22:22.129600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152466+0000","last_change":"2023-11-26T07:22:22.148977+0000","last_active":"2023-11-26T07:22:22.152466+0000","last_peered":"2023-11-26T07:22:22.152466+0000","last_clean":"2023-11-26T07:22:22.152466+0000","last_became_active":"2023-11-26T07:22:22.148703+0000","last_became_peered":"2023-11-26T07:22:22.148703+0000","last_unstale":"2023-11-26T07:22:22.152466+0000","last_undegraded":"2023-11-26T07:22:22.152466+0000","last_fullsized":"2023-11-26T07:22:22.152466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:35.311859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.133925+0000","last_change":"2023-11-26T07:22:22.133925+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.133925+0000","last_undegraded":"2023-11-26T07:22:22.133925+0000","last_fullsized":"2023-11-26T07:22:22.133925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153820+0000","last_change":"2023-11-26T07:22:22.151630+0000","last_active":"2023-11-26T07:22:22.153820+0000","last_peered":"2023-11-26T07:22:22.153820+0000","last_clean":"2023-11-26T07:22:22.153820+0000","last_became_active":"2023-11-26T07:22:22.151230+0000","last_became_peered":"2023-11-26T07:22:22.151230+0000","last_unstale":"2023-11-26T07:22:22.153820+0000","last_undegraded":"2023-11-26T07:22:22.153820+0000","last_fullsized":"2023-11-26T07:22:22.153820+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:38.450764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128444+0000","last_change":"2023-11-26T07:22:22.128444+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128444+0000","last_undegraded":"2023-11-26T07:22:22.128444+0000","last_fullsized":"2023-11-26T07:22:22.128444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.158977+0000","last_change":"2023-11-26T07:22:22.153463+0000","last_active":"2023-11-26T07:22:22.158977+0000","last_peered":"2023-11-26T07:22:22.158977+0000","last_clean":"2023-11-26T07:22:22.158977+0000","last_became_active":"2023-11-26T07:22:22.153132+0000","last_became_peered":"2023-11-26T07:22:22.153132+0000","last_unstale":"2023-11-26T07:22:22.158977+0000","last_undegraded":"2023-11-26T07:22:22.158977+0000","last_fullsized":"2023-11-26T07:22:22.158977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:15.107224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.142692+0000","last_change":"2023-11-26T07:22:22.142692+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.142692+0000","last_undegraded":"2023-11-26T07:22:22.142692+0000","last_fullsized":"2023-11-26T07:22:22.142692+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152289+0000","last_change":"2023-11-26T07:22:22.148229+0000","last_active":"2023-11-26T07:22:22.152289+0000","last_peered":"2023-11-26T07:22:22.152289+0000","last_clean":"2023-11-26T07:22:22.152289+0000","last_became_active":"2023-11-26T07:22:22.147927+0000","last_became_peered":"2023-11-26T07:22:22.147927+0000","last_unstale":"2023-11-26T07:22:22.152289+0000","last_undegraded":"2023-11-26T07:22:22.152289+0000","last_fullsized":"2023-11-26T07:22:22.152289+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:21:24.263764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132776+0000","last_change":"2023-11-26T07:22:22.132776+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132776+0000","last_undegraded":"2023-11-26T07:22:22.132776+0000","last_fullsized":"2023-11-26T07:22:22.132776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146154+0000","last_change":"2023-11-26T07:22:22.144638+0000","last_active":"2023-11-26T07:22:22.146154+0000","last_peered":"2023-11-26T07:22:22.146154+0000","last_clean":"2023-11-26T07:22:22.146154+0000","last_became_active":"2023-11-26T07:22:22.144230+0000","last_became_peered":"2023-11-26T07:22:22.144230+0000","last_unstale":"2023-11-26T07:22:22.146154+0000","last_undegraded":"2023-11-26T07:22:22.146154+0000","last_fullsized":"2023-11-26T07:22:22.146154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:33.883653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128946+0000","last_change":"2023-11-26T07:22:22.128946+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128946+0000","last_undegraded":"2023-11-26T07:22:22.128946+0000","last_fullsized":"2023-11-26T07:22:22.128946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.155657+0000","last_change":"2023-11-26T07:22:22.153839+0000","last_active":"2023-11-26T07:22:22.155657+0000","last_peered":"2023-11-26T07:22:22.155657+0000","last_clean":"2023-11-26T07:22:22.155657+0000","last_became_active":"2023-11-26T07:22:22.153535+0000","last_became_peered":"2023-11-26T07:22:22.153535+0000","last_unstale":"2023-11-26T07:22:22.155657+0000","last_undegraded":"2023-11-26T07:22:22.155657+0000","last_fullsized":"2023-11-26T07:22:22.155657+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:56.712024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130940+0000","last_change":"2023-11-26T07:22:22.130940+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130940+0000","last_undegraded":"2023-11-26T07:22:22.130940+0000","last_fullsized":"2023-11-26T07:22:22.130940+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146666+0000","last_change":"2023-11-26T07:22:22.140913+0000","last_active":"2023-11-26T07:22:22.146666+0000","last_peered":"2023-11-26T07:22:22.146666+0000","last_clean":"2023-11-26T07:22:22.146666+0000","last_became_active":"2023-11-26T07:22:22.140643+0000","last_became_peered":"2023-11-26T07:22:22.140643+0000","last_unstale":"2023-11-26T07:22:22.146666+0000","last_undegraded":"2023-11-26T07:22:22.146666+0000","last_fullsized":"2023-11-26T07:22:22.146666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:15.107224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.130873+0000","last_change":"2023-11-26T07:22:22.130873+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.130873+0000","last_undegraded":"2023-11-26T07:22:22.130873+0000","last_fullsized":"2023-11-26T07:22:22.130873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153050+0000","last_change":"2023-11-26T07:22:22.151535+0000","last_active":"2023-11-26T07:22:22.153050+0000","last_peered":"2023-11-26T07:22:22.153050+0000","last_clean":"2023-11-26T07:22:22.153050+0000","last_became_active":"2023-11-26T07:22:22.151097+0000","last_became_peered":"2023-11-26T07:22:22.151097+0000","last_unstale":"2023-11-26T07:22:22.153050+0000","last_undegraded":"2023-11-26T07:22:22.153050+0000","last_fullsized":"2023-11-26T07:22:22.153050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:49.532057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.141167+0000","last_change":"2023-11-26T07:22:22.141167+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.141167+0000","last_undegraded":"2023-11-26T07:22:22.141167+0000","last_fullsized":"2023-11-26T07:22:22.141167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152282+0000","last_change":"2023-11-26T07:22:22.149767+0000","last_active":"2023-11-26T07:22:22.152282+0000","last_peered":"2023-11-26T07:22:22.152282+0000","last_clean":"2023-11-26T07:22:22.152282+0000","last_became_active":"2023-11-26T07:22:22.149435+0000","last_became_peered":"2023-11-26T07:22:22.149435+0000","last_unstale":"2023-11-26T07:22:22.152282+0000","last_undegraded":"2023-11-26T07:22:22.152282+0000","last_fullsized":"2023-11-26T07:22:22.152282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:21:24.263764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139846+0000","last_change":"2023-11-26T07:22:22.139846+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139846+0000","last_undegraded":"2023-11-26T07:22:22.139846+0000","last_fullsized":"2023-11-26T07:22:22.139846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.143475+0000","last_change":"2023-11-26T07:22:22.138119+0000","last_active":"2023-11-26T07:22:22.143475+0000","last_peered":"2023-11-26T07:22:22.143475+0000","last_clean":"2023-11-26T07:22:22.143475+0000","last_became_active":"2023-11-26T07:22:22.137770+0000","last_became_peered":"2023-11-26T07:22:22.137770+0000","last_unstale":"2023-11-26T07:22:22.143475+0000","last_undegraded":"2023-11-26T07:22:22.143475+0000","last_fullsized":"2023-11-26T07:22:22.143475+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:25.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.139244+0000","last_change":"2023-11-26T07:22:22.139244+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.139244+0000","last_undegraded":"2023-11-26T07:22:22.139244+0000","last_fullsized":"2023-11-26T07:22:22.139244+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151664+0000","last_change":"2023-11-26T07:22:22.145272+0000","last_active":"2023-11-26T07:22:22.151664+0000","last_peered":"2023-11-26T07:22:22.151664+0000","last_clean":"2023-11-26T07:22:22.151664+0000","last_became_active":"2023-11-26T07:22:22.145011+0000","last_became_peered":"2023-11-26T07:22:22.145011+0000","last_unstale":"2023-11-26T07:22:22.151664+0000","last_undegraded":"2023-11-26T07:22:22.151664+0000","last_fullsized":"2023-11-26T07:22:22.151664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:17.509070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134871+0000","last_change":"2023-11-26T07:22:22.134871+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134871+0000","last_undegraded":"2023-11-26T07:22:22.134871+0000","last_fullsized":"2023-11-26T07:22:22.134871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153970+0000","last_change":"2023-11-26T07:22:22.149360+0000","last_active":"2023-11-26T07:22:22.153970+0000","last_peered":"2023-11-26T07:22:22.153970+0000","last_clean":"2023-11-26T07:22:22.153970+0000","last_became_active":"2023-11-26T07:22:22.149068+0000","last_became_peered":"2023-11-26T07:22:22.149068+0000","last_unstale":"2023-11-26T07:22:22.153970+0000","last_undegraded":"2023-11-26T07:22:22.153970+0000","last_fullsized":"2023-11-26T07:22:22.153970+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:29.708891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128109+0000","last_change":"2023-11-26T07:22:22.128109+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128109+0000","last_undegraded":"2023-11-26T07:22:22.128109+0000","last_fullsized":"2023-11-26T07:22:22.128109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153325+0000","last_change":"2023-11-26T07:22:22.151868+0000","last_active":"2023-11-26T07:22:22.153325+0000","last_peered":"2023-11-26T07:22:22.153325+0000","last_clean":"2023-11-26T07:22:22.153325+0000","last_became_active":"2023-11-26T07:22:22.151535+0000","last_became_peered":"2023-11-26T07:22:22.151535+0000","last_unstale":"2023-11-26T07:22:22.153325+0000","last_undegraded":"2023-11-26T07:22:22.153325+0000","last_fullsized":"2023-11-26T07:22:22.153325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:33.883653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.135370+0000","last_change":"2023-11-26T07:22:22.135370+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.135370+0000","last_undegraded":"2023-11-26T07:22:22.135370+0000","last_fullsized":"2023-11-26T07:22:22.135370+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152260+0000","last_change":"2023-11-26T07:22:22.145842+0000","last_active":"2023-11-26T07:22:22.152260+0000","last_peered":"2023-11-26T07:22:22.152260+0000","last_clean":"2023-11-26T07:22:22.152260+0000","last_became_active":"2023-11-26T07:22:22.145573+0000","last_became_peered":"2023-11-26T07:22:22.145573+0000","last_unstale":"2023-11-26T07:22:22.152260+0000","last_undegraded":"2023-11-26T07:22:22.152260+0000","last_fullsized":"2023-11-26T07:22:22.152260+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:49.532057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.134114+0000","last_change":"2023-11-26T07:22:22.134114+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.134114+0000","last_undegraded":"2023-11-26T07:22:22.134114+0000","last_fullsized":"2023-11-26T07:22:22.134114+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.145214+0000","last_change":"2023-11-26T07:22:22.140377+0000","last_active":"2023-11-26T07:22:22.145214+0000","last_peered":"2023-11-26T07:22:22.145214+0000","last_clean":"2023-11-26T07:22:22.145214+0000","last_became_active":"2023-11-26T07:22:22.138993+0000","last_became_peered":"2023-11-26T07:22:22.138993+0000","last_unstale":"2023-11-26T07:22:22.145214+0000","last_undegraded":"2023-11-26T07:22:22.145214+0000","last_fullsized":"2023-11-26T07:22:22.145214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:57.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128923+0000","last_change":"2023-11-26T07:22:22.128923+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128923+0000","last_undegraded":"2023-11-26T07:22:22.128923+0000","last_fullsized":"2023-11-26T07:22:22.128923+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.154062+0000","last_change":"2023-11-26T07:22:22.152564+0000","last_active":"2023-11-26T07:22:22.154062+0000","last_peered":"2023-11-26T07:22:22.154062+0000","last_clean":"2023-11-26T07:22:22.154062+0000","last_became_active":"2023-11-26T07:22:22.152161+0000","last_became_peered":"2023-11-26T07:22:22.152161+0000","last_unstale":"2023-11-26T07:22:22.154062+0000","last_undegraded":"2023-11-26T07:22:22.154062+0000","last_fullsized":"2023-11-26T07:22:22.154062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:08.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.137359+0000","last_change":"2023-11-26T07:22:22.137359+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.137359+0000","last_undegraded":"2023-11-26T07:22:22.137359+0000","last_fullsized":"2023-11-26T07:22:22.137359+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151089+0000","last_change":"2023-11-26T07:22:22.145038+0000","last_active":"2023-11-26T07:22:22.151089+0000","last_peered":"2023-11-26T07:22:22.151089+0000","last_clean":"2023-11-26T07:22:22.151089+0000","last_became_active":"2023-11-26T07:22:22.144754+0000","last_became_peered":"2023-11-26T07:22:22.144754+0000","last_unstale":"2023-11-26T07:22:22.151089+0000","last_undegraded":"2023-11-26T07:22:22.151089+0000","last_fullsized":"2023-11-26T07:22:22.151089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:53:45.151325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132275+0000","last_change":"2023-11-26T07:22:22.132275+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132275+0000","last_undegraded":"2023-11-26T07:22:22.132275+0000","last_fullsized":"2023-11-26T07:22:22.132275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146987+0000","last_change":"2023-11-26T07:22:22.140578+0000","last_active":"2023-11-26T07:22:22.146987+0000","last_peered":"2023-11-26T07:22:22.146987+0000","last_clean":"2023-11-26T07:22:22.146987+0000","last_became_active":"2023-11-26T07:22:22.140222+0000","last_became_peered":"2023-11-26T07:22:22.140222+0000","last_unstale":"2023-11-26T07:22:22.146987+0000","last_undegraded":"2023-11-26T07:22:22.146987+0000","last_fullsized":"2023-11-26T07:22:22.146987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:12:59.850726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129550+0000","last_change":"2023-11-26T07:22:22.129550+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129550+0000","last_undegraded":"2023-11-26T07:22:22.129550+0000","last_fullsized":"2023-11-26T07:22:22.129550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.149688+0000","last_change":"2023-11-26T07:22:22.147493+0000","last_active":"2023-11-26T07:22:22.149688+0000","last_peered":"2023-11-26T07:22:22.149688+0000","last_clean":"2023-11-26T07:22:22.149688+0000","last_became_active":"2023-11-26T07:22:22.147187+0000","last_became_peered":"2023-11-26T07:22:22.147187+0000","last_unstale":"2023-11-26T07:22:22.149688+0000","last_undegraded":"2023-11-26T07:22:22.149688+0000","last_fullsized":"2023-11-26T07:22:22.149688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:59:16.357020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.131250+0000","last_change":"2023-11-26T07:22:22.131250+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.131250+0000","last_undegraded":"2023-11-26T07:22:22.131250+0000","last_fullsized":"2023-11-26T07:22:22.131250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153430+0000","last_change":"2023-11-26T07:22:22.148994+0000","last_active":"2023-11-26T07:22:22.153430+0000","last_peered":"2023-11-26T07:22:22.153430+0000","last_clean":"2023-11-26T07:22:22.153430+0000","last_became_active":"2023-11-26T07:22:22.147943+0000","last_became_peered":"2023-11-26T07:22:22.147943+0000","last_unstale":"2023-11-26T07:22:22.153430+0000","last_undegraded":"2023-11-26T07:22:22.153430+0000","last_fullsized":"2023-11-26T07:22:22.153430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:53:21.611291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129655+0000","last_change":"2023-11-26T07:22:22.129655+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129655+0000","last_undegraded":"2023-11-26T07:22:22.129655+0000","last_fullsized":"2023-11-26T07:22:22.129655+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146830+0000","last_change":"2023-11-26T07:22:22.139465+0000","last_active":"2023-11-26T07:22:22.146830+0000","last_peered":"2023-11-26T07:22:22.146830+0000","last_clean":"2023-11-26T07:22:22.146830+0000","last_became_active":"2023-11-26T07:22:22.139180+0000","last_became_peered":"2023-11-26T07:22:22.139180+0000","last_unstale":"2023-11-26T07:22:22.146830+0000","last_undegraded":"2023-11-26T07:22:22.146830+0000","last_fullsized":"2023-11-26T07:22:22.146830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:06.932370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.143648+0000","last_change":"2023-11-26T07:22:22.143648+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.143648+0000","last_undegraded":"2023-11-26T07:22:22.143648+0000","last_fullsized":"2023-11-26T07:22:22.143648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151859+0000","last_change":"2023-11-26T07:22:22.146130+0000","last_active":"2023-11-26T07:22:22.151859+0000","last_peered":"2023-11-26T07:22:22.151859+0000","last_clean":"2023-11-26T07:22:22.151859+0000","last_became_active":"2023-11-26T07:22:22.144230+0000","last_became_peered":"2023-11-26T07:22:22.144230+0000","last_unstale":"2023-11-26T07:22:22.151859+0000","last_undegraded":"2023-11-26T07:22:22.151859+0000","last_fullsized":"2023-11-26T07:22:22.151859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:44.104269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132436+0000","last_change":"2023-11-26T07:22:22.132436+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132436+0000","last_undegraded":"2023-11-26T07:22:22.132436+0000","last_fullsized":"2023-11-26T07:22:22.132436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.151642+0000","last_change":"2023-11-26T07:22:22.145175+0000","last_active":"2023-11-26T07:22:22.151642+0000","last_peered":"2023-11-26T07:22:22.151642+0000","last_clean":"2023-11-26T07:22:22.151642+0000","last_became_active":"2023-11-26T07:22:22.144917+0000","last_became_peered":"2023-11-26T07:22:22.144917+0000","last_unstale":"2023-11-26T07:22:22.151642+0000","last_undegraded":"2023-11-26T07:22:22.151642+0000","last_fullsized":"2023-11-26T07:22:22.151642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:15.998685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129092+0000","last_change":"2023-11-26T07:22:22.129092+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129092+0000","last_undegraded":"2023-11-26T07:22:22.129092+0000","last_fullsized":"2023-11-26T07:22:22.129092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.147011+0000","last_change":"2023-11-26T07:22:22.143708+0000","last_active":"2023-11-26T07:22:22.147011+0000","last_peered":"2023-11-26T07:22:22.147011+0000","last_clean":"2023-11-26T07:22:22.147011+0000","last_became_active":"2023-11-26T07:22:22.143395+0000","last_became_peered":"2023-11-26T07:22:22.143395+0000","last_unstale":"2023-11-26T07:22:22.147011+0000","last_undegraded":"2023-11-26T07:22:22.147011+0000","last_fullsized":"2023-11-26T07:22:22.147011+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:58:46.089931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.136808+0000","last_change":"2023-11-26T07:22:22.136808+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.136808+0000","last_undegraded":"2023-11-26T07:22:22.136808+0000","last_fullsized":"2023-11-26T07:22:22.136808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.144998+0000","last_change":"2023-11-26T07:22:22.139927+0000","last_active":"2023-11-26T07:22:22.144998+0000","last_peered":"2023-11-26T07:22:22.144998+0000","last_clean":"2023-11-26T07:22:22.144998+0000","last_became_active":"2023-11-26T07:22:22.139628+0000","last_became_peered":"2023-11-26T07:22:22.139628+0000","last_unstale":"2023-11-26T07:22:22.144998+0000","last_undegraded":"2023-11-26T07:22:22.144998+0000","last_fullsized":"2023-11-26T07:22:22.144998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:44:29.019321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.129561+0000","last_change":"2023-11-26T07:22:22.129561+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.129561+0000","last_undegraded":"2023-11-26T07:22:22.129561+0000","last_fullsized":"2023-11-26T07:22:22.129561+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152296+0000","last_change":"2023-11-26T07:22:22.146618+0000","last_active":"2023-11-26T07:22:22.152296+0000","last_peered":"2023-11-26T07:22:22.152296+0000","last_clean":"2023-11-26T07:22:22.152296+0000","last_became_active":"2023-11-26T07:22:22.146214+0000","last_became_peered":"2023-11-26T07:22:22.146214+0000","last_unstale":"2023-11-26T07:22:22.152296+0000","last_undegraded":"2023-11-26T07:22:22.152296+0000","last_fullsized":"2023-11-26T07:22:22.152296+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:25.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.137697+0000","last_change":"2023-11-26T07:22:22.137697+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.137697+0000","last_undegraded":"2023-11-26T07:22:22.137697+0000","last_fullsized":"2023-11-26T07:22:22.137697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152272+0000","last_change":"2023-11-26T07:22:22.148139+0000","last_active":"2023-11-26T07:22:22.152272+0000","last_peered":"2023-11-26T07:22:22.152272+0000","last_clean":"2023-11-26T07:22:22.152272+0000","last_became_active":"2023-11-26T07:22:22.147844+0000","last_became_peered":"2023-11-26T07:22:22.147844+0000","last_unstale":"2023-11-26T07:22:22.152272+0000","last_undegraded":"2023-11-26T07:22:22.152272+0000","last_fullsized":"2023-11-26T07:22:22.152272+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:57.802160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.132256+0000","last_change":"2023-11-26T07:22:22.132256+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.132256+0000","last_undegraded":"2023-11-26T07:22:22.132256+0000","last_fullsized":"2023-11-26T07:22:22.132256+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.146674+0000","last_change":"2023-11-26T07:22:22.140709+0000","last_active":"2023-11-26T07:22:22.146674+0000","last_peered":"2023-11-26T07:22:22.146674+0000","last_clean":"2023-11-26T07:22:22.146674+0000","last_became_active":"2023-11-26T07:22:22.140379+0000","last_became_peered":"2023-11-26T07:22:22.140379+0000","last_unstale":"2023-11-26T07:22:22.146674+0000","last_undegraded":"2023-11-26T07:22:22.146674+0000","last_fullsized":"2023-11-26T07:22:22.146674+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:41.677508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.128291+0000","last_change":"2023-11-26T07:22:22.128291+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.128291+0000","last_undegraded":"2023-11-26T07:22:22.128291+0000","last_fullsized":"2023-11-26T07:22:22.128291+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.152324+0000","last_change":"2023-11-26T07:22:22.148901+0000","last_active":"2023-11-26T07:22:22.152324+0000","last_peered":"2023-11-26T07:22:22.152324+0000","last_clean":"2023-11-26T07:22:22.152324+0000","last_became_active":"2023-11-26T07:22:22.148566+0000","last_became_peered":"2023-11-26T07:22:22.148566+0000","last_unstale":"2023-11-26T07:22:22.152324+0000","last_undegraded":"2023-11-26T07:22:22.152324+0000","last_fullsized":"2023-11-26T07:22:22.152324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:58:46.089931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":78,"state":"creating+peering","last_fresh":"2023-11-26T07:22:22.127322+0000","last_change":"2023-11-26T07:22:22.127322+0000","last_active":"2023-11-26T07:22:22.119714+0000","last_peered":"2023-11-26T07:22:22.119714+0000","last_clean":"2023-11-26T07:22:22.119714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-26T07:22:22.127322+0000","last_undegraded":"2023-11-26T07:22:22.127322+0000","last_fullsized":"2023-11-26T07:22:22.127322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-11-26T07:22:22.153518+0000","last_change":"2023-11-26T07:22:22.152219+0000","last_active":"2023-11-26T07:22:22.153518+0000","last_peered":"2023-11-26T07:22:22.153518+0000","last_clean":"2023-11-26T07:22:22.153518+0000","last_became_active":"2023-11-26T07:22:22.151927+0000","last_became_peered":"2023-11-26T07:22:22.151927+0000","last_unstale":"2023-11-26T07:22:22.153518+0000","last_undegraded":"2023-11-26T07:22:22.153518+0000","last_fullsized":"2023-11-26T07:22:22.153518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:17.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"69'128","reported_seq":251,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-11-26T07:22:21.128526+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:22:21.128526+0000","last_peered":"2023-11-26T07:22:21.128526+0000","last_clean":"2023-11-26T07:22:21.128526+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:22:21.128526+0000","last_undegraded":"2023-11-26T07:22:21.128526+0000","last_fullsized":"2023-11-26T07:22:21.128526+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1540096,"data_stored":992140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.56599999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":0.995},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84799999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45200000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.48499999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.92500000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71399999999999997},"min":{"1min":0.56899999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84499999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.58999999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92500000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.50600000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":1.016,"15min":1.016},"last":0.79500000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.85099999999999998,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":0.81899999999999995},"min":{"1min":0.68100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93500000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68899999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.71099999999999997},"min":{"1min":0.47199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.55900000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73299999999999998,"15min":0.77500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95899999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.84899999999999998},"min":{"1min":0.58699999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91500000000000004,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82099999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.499,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93400000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.82999999999999996},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.012,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":0.86699999999999999,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.222000000000001,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":3.4020000000000001,"5min":1.246,"15min":0.91500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":54.737000000000002,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.80600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.40400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.1880000000000002,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.6689999999999996,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.96599999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.439,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.8479999999999999,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.72499999999999998,"15min":0.68899999999999995},"min":{"1min":0.57099999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0170000000000003,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.73199999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.81299999999999994,"15min":0.751},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.4870000000000001,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":1.601,"5min":0.94399999999999995,"15min":0.874},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":18.77,"5min":18.77,"15min":18.77},"last":0.78100000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.53100000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.749},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.121,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.88100000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67400000000000004,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.77900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.93300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.626,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.90900000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.64400000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.87,"5min":1.073,"15min":1.073},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.96499999999999997,"5min":1.036,"15min":1.036},"last":0.77700000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68899999999999995,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93000000000000005,"5min":1.012,"15min":1.012},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.49099999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79000000000000004,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.79000000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.71299999999999997},"min":{"1min":0.495,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.028,"5min":1.097,"15min":1.097},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.46700000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.88700000000000001,"5min":1.071,"15min":1.071},"last":0.80700000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65200000000000002,"15min":0.68600000000000005},"min":{"1min":0.42699999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.137,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.77000000000000002},"min":{"1min":0.40999999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.0029999999999999,"5min":1.206,"15min":1.206},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.62,"15min":0.67000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93100000000000005,"5min":1.143,"15min":1.143},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68600000000000005,"15min":0.73199999999999998},"min":{"1min":0.46000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":1.1020000000000001,"5min":1.228,"15min":1.228},"last":0.84599999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.55900000000000005,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0580000000000001,"5min":1.264,"15min":1.264},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.75900000000000001,"15min":0.80900000000000005},"min":{"1min":0.47999999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.077,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.87}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84899999999999998,"5min":1.016,"15min":1.016},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93100000000000005,"5min":1.26,"15min":1.26},"last":0.72099999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90200000000000002,"5min":1.077,"15min":1.077},"last":0.69699999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.302,"15min":1.302},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.753},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":1.181,"15min":1.181},"last":0.77000000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.747},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97999999999999998,"5min":1.702,"15min":1.702},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.78700000000000003},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0409999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.63700000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.72999999999999998},"min":{"1min":0.42199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97099999999999997,"5min":1.129,"15min":1.129},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.78800000000000003},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.012,"15min":1.012},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88300000000000001,"5min":1.016,"15min":1.016},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.81200000000000006},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.0980000000000001,"5min":1.242,"15min":1.242},"last":0.67600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.68999999999999995,"5min":1.294,"15min":1.294},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.40500000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":1.456,"15min":1.456},"last":0.434},{"interface":"front","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.52900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.212,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2649999999999999,"5min":1.395,"15min":1.395},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1080000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.504}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.64700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.333,"15min":1.333},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.228,"5min":1.347,"15min":1.347},"last":0.59699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.397,"15min":1.397},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1859999999999999,"5min":1.363,"15min":1.363},"last":0.66200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56000000000000005,"15min":0.64800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65800000000000003,"5min":3.931,"15min":3.931},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.71099999999999997},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60199999999999998,"15min":0.69199999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95499999999999996,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.69299999999999995},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.7650000000000001,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.434}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.751,"15min":0.90300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98899999999999999,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.59899999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.85199999999999998,"15min":0.92900000000000005},"min":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1240000000000001,"5min":4.431,"15min":4.431},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.274,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.63500000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.61199999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89400000000000002,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8069999999999999,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.754}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.98399999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.222,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.64900000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.93899999999999995,"15min":0.995},"min":{"1min":0.621,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.4740000000000002,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.052,"5min":1.609,"15min":1.609},"last":0.61899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":401772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.32300000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.57499999999999996,"5min":1.034,"15min":1.034},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.39500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.439,"15min":0.45400000000000001},"min":{"1min":0.30199999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.38200000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.69999999999999996,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.33000000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.66900000000000004,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.499,"15min":0.503},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48599999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.67900000000000005},"min":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95599999999999996,"5min":1.143,"15min":1.143},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0740000000000001,"5min":1.216,"15min":1.216},"last":0.75800000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.58999999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97399999999999998,"5min":1.262,"15min":1.262},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.57399999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.998,"5min":1.54,"15min":1.54},"last":0.67500000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.61599999999999999,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.054,"5min":1.145,"15min":1.145},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77400000000000002,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.91600000000000004,"5min":1.333,"15min":1.333},"last":0.69899999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.58399999999999996,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95699999999999996,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0189999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.61699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574911,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1536000,"data_stored":991121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.45300000000000001},"min":{"1min":0.29799999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.68300000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.48499999999999999,"15min":0.47799999999999998},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.78600000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46899999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.34799999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.51700000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79700000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.438}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55800000000000005,"15min":0.54600000000000004},"min":{"1min":0.38400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83599999999999997,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.82099999999999995,"15min":0.81499999999999995},"min":{"1min":0.59499999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.67800000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":1.0980000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76200000000000001,"15min":0.76000000000000001},"min":{"1min":0.63400000000000001,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.77400000000000002,"15min":0.77000000000000002},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.76700000000000002}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.64800000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.61899999999999999,"5min":0.503,"15min":0.503},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.80400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82799999999999996,"15min":0.82699999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1180000000000001,"5min":1.123,"15min":1.123},"last":1.079},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.83099999999999996,"15min":0.81899999999999995},"min":{"1min":0.69399999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.0349999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:22:28.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3f is not active+clean 2023-11-26T07:22:28.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.131043+0000', 'last_change': '2023-11-26T07:22:22.131043+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.131043+0000', 'last_undegraded': '2023-11-26T07:22:22.131043+0000', 'last_fullsized': '2023-11-26T07:22:22.131043+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2023-11-26T07:22:28.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.129528+0000', 'last_change': '2023-11-26T07:22:22.129528+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.129528+0000', 'last_undegraded': '2023-11-26T07:22:22.129528+0000', 'last_fullsized': '2023-11-26T07:22:22.129528+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:22:28.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3d is not active+clean 2023-11-26T07:22:28.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.133354+0000', 'last_change': '2023-11-26T07:22:22.133354+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.133354+0000', 'last_undegraded': '2023-11-26T07:22:22.133354+0000', 'last_fullsized': '2023-11-26T07:22:22.133354+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:22:28.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2023-11-26T07:22:28.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.135222+0000', 'last_change': '2023-11-26T07:22:22.135222+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.135222+0000', 'last_undegraded': '2023-11-26T07:22:22.135222+0000', 'last_fullsized': '2023-11-26T07:22:22.135222+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:22:28.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3b is not active+clean 2023-11-26T07:22:28.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.143110+0000', 'last_change': '2023-11-26T07:22:22.143110+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.143110+0000', 'last_undegraded': '2023-11-26T07:22:22.143110+0000', 'last_fullsized': '2023-11-26T07:22:22.143110+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3a is not active+clean 2023-11-26T07:22:28.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.130504+0000', 'last_change': '2023-11-26T07:22:22.130504+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.130504+0000', 'last_undegraded': '2023-11-26T07:22:22.130504+0000', 'last_fullsized': '2023-11-26T07:22:22.130504+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:22:28.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2023-11-26T07:22:28.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.130438+0000', 'last_change': '2023-11-26T07:22:22.130438+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.130438+0000', 'last_undegraded': '2023-11-26T07:22:22.130438+0000', 'last_fullsized': '2023-11-26T07:22:22.130438+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:22:28.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.38 is not active+clean 2023-11-26T07:22:28.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.130281+0000', 'last_change': '2023-11-26T07:22:22.130281+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.130281+0000', 'last_undegraded': '2023-11-26T07:22:22.130281+0000', 'last_fullsized': '2023-11-26T07:22:22.130281+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:22:28.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.37 is not active+clean 2023-11-26T07:22:28.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.131416+0000', 'last_change': '2023-11-26T07:22:22.131416+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.131416+0000', 'last_undegraded': '2023-11-26T07:22:22.131416+0000', 'last_fullsized': '2023-11-26T07:22:22.131416+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:22:28.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.36 is not active+clean 2023-11-26T07:22:28.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.131033+0000', 'last_change': '2023-11-26T07:22:22.131033+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.131033+0000', 'last_undegraded': '2023-11-26T07:22:22.131033+0000', 'last_fullsized': '2023-11-26T07:22:22.131033+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2023-11-26T07:22:28.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.135119+0000', 'last_change': '2023-11-26T07:22:22.135119+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.135119+0000', 'last_undegraded': '2023-11-26T07:22:22.135119+0000', 'last_fullsized': '2023-11-26T07:22:22.135119+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:22:28.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.34 is not active+clean 2023-11-26T07:22:28.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.127615+0000', 'last_change': '2023-11-26T07:22:22.127615+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.127615+0000', 'last_undegraded': '2023-11-26T07:22:22.127615+0000', 'last_fullsized': '2023-11-26T07:22:22.127615+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.33 is not active+clean 2023-11-26T07:22:28.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.135357+0000', 'last_change': '2023-11-26T07:22:22.135357+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.135357+0000', 'last_undegraded': '2023-11-26T07:22:22.135357+0000', 'last_fullsized': '2023-11-26T07:22:22.135357+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:22:28.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2023-11-26T07:22:28.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.132827+0000', 'last_change': '2023-11-26T07:22:22.132827+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.132827+0000', 'last_undegraded': '2023-11-26T07:22:22.132827+0000', 'last_fullsized': '2023-11-26T07:22:22.132827+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:22:28.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2023-11-26T07:22:28.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.129520+0000', 'last_change': '2023-11-26T07:22:22.129520+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.129520+0000', 'last_undegraded': '2023-11-26T07:22:22.129520+0000', 'last_fullsized': '2023-11-26T07:22:22.129520+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:22:28.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.30 is not active+clean 2023-11-26T07:22:28.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.142126+0000', 'last_change': '2023-11-26T07:22:22.142126+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.142126+0000', 'last_undegraded': '2023-11-26T07:22:22.142126+0000', 'last_fullsized': '2023-11-26T07:22:22.142126+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2f is not active+clean 2023-11-26T07:22:28.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.139694+0000', 'last_change': '2023-11-26T07:22:22.139694+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.139694+0000', 'last_undegraded': '2023-11-26T07:22:22.139694+0000', 'last_fullsized': '2023-11-26T07:22:22.139694+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2023-11-26T07:22:28.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.134761+0000', 'last_change': '2023-11-26T07:22:22.134761+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.134761+0000', 'last_undegraded': '2023-11-26T07:22:22.134761+0000', 'last_fullsized': '2023-11-26T07:22:22.134761+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:22:28.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2023-11-26T07:22:28.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.138624+0000', 'last_change': '2023-11-26T07:22:22.138624+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.138624+0000', 'last_undegraded': '2023-11-26T07:22:22.138624+0000', 'last_fullsized': '2023-11-26T07:22:22.138624+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:22:28.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2023-11-26T07:22:28.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.134691+0000', 'last_change': '2023-11-26T07:22:22.134691+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.134691+0000', 'last_undegraded': '2023-11-26T07:22:22.134691+0000', 'last_fullsized': '2023-11-26T07:22:22.134691+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:22:28.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2023-11-26T07:22:28.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.131538+0000', 'last_change': '2023-11-26T07:22:22.131538+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.131538+0000', 'last_undegraded': '2023-11-26T07:22:22.131538+0000', 'last_fullsized': '2023-11-26T07:22:22.131538+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:22:28.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2023-11-26T07:22:28.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.129384+0000', 'last_change': '2023-11-26T07:22:22.129384+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.129384+0000', 'last_undegraded': '2023-11-26T07:22:22.129384+0000', 'last_fullsized': '2023-11-26T07:22:22.129384+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:22:28.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.29 is not active+clean 2023-11-26T07:22:28.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.127661+0000', 'last_change': '2023-11-26T07:22:22.127661+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.127661+0000', 'last_undegraded': '2023-11-26T07:22:22.127661+0000', 'last_fullsized': '2023-11-26T07:22:22.127661+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2023-11-26T07:22:28.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.131711+0000', 'last_change': '2023-11-26T07:22:22.131711+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.131711+0000', 'last_undegraded': '2023-11-26T07:22:22.131711+0000', 'last_fullsized': '2023-11-26T07:22:22.131711+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:22:28.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2023-11-26T07:22:28.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.134720+0000', 'last_change': '2023-11-26T07:22:22.134720+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.134720+0000', 'last_undegraded': '2023-11-26T07:22:22.134720+0000', 'last_fullsized': '2023-11-26T07:22:22.134720+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:22:28.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.26 is not active+clean 2023-11-26T07:22:28.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.134643+0000', 'last_change': '2023-11-26T07:22:22.134643+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.134643+0000', 'last_undegraded': '2023-11-26T07:22:22.134643+0000', 'last_fullsized': '2023-11-26T07:22:22.134643+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:22:28.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.25 is not active+clean 2023-11-26T07:22:28.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.143843+0000', 'last_change': '2023-11-26T07:22:22.143843+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.143843+0000', 'last_undegraded': '2023-11-26T07:22:22.143843+0000', 'last_fullsized': '2023-11-26T07:22:22.143843+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.24 is not active+clean 2023-11-26T07:22:28.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.130684+0000', 'last_change': '2023-11-26T07:22:22.130684+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.130684+0000', 'last_undegraded': '2023-11-26T07:22:22.130684+0000', 'last_fullsized': '2023-11-26T07:22:22.130684+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2023-11-26T07:22:28.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.128838+0000', 'last_change': '2023-11-26T07:22:22.128838+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.128838+0000', 'last_undegraded': '2023-11-26T07:22:22.128838+0000', 'last_fullsized': '2023-11-26T07:22:22.128838+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:22:28.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2023-11-26T07:22:28.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.128896+0000', 'last_change': '2023-11-26T07:22:22.128896+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.128896+0000', 'last_undegraded': '2023-11-26T07:22:22.128896+0000', 'last_fullsized': '2023-11-26T07:22:22.128896+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:22:28.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.21 is not active+clean 2023-11-26T07:22:28.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.139213+0000', 'last_change': '2023-11-26T07:22:22.139213+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.139213+0000', 'last_undegraded': '2023-11-26T07:22:22.139213+0000', 'last_fullsized': '2023-11-26T07:22:22.139213+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.20 is not active+clean 2023-11-26T07:22:28.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.127366+0000', 'last_change': '2023-11-26T07:22:22.127366+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.127366+0000', 'last_undegraded': '2023-11-26T07:22:22.127366+0000', 'last_fullsized': '2023-11-26T07:22:22.127366+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1e is not active+clean 2023-11-26T07:22:28.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.144178+0000', 'last_change': '2023-11-26T07:22:22.144178+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.144178+0000', 'last_undegraded': '2023-11-26T07:22:22.144178+0000', 'last_fullsized': '2023-11-26T07:22:22.144178+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1f is not active+clean 2023-11-26T07:22:28.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.129444+0000', 'last_change': '2023-11-26T07:22:22.129444+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.129444+0000', 'last_undegraded': '2023-11-26T07:22:22.129444+0000', 'last_fullsized': '2023-11-26T07:22:22.129444+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1c is not active+clean 2023-11-26T07:22:28.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.129600+0000', 'last_change': '2023-11-26T07:22:22.129600+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.129600+0000', 'last_undegraded': '2023-11-26T07:22:22.129600+0000', 'last_fullsized': '2023-11-26T07:22:22.129600+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1d is not active+clean 2023-11-26T07:22:28.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.133925+0000', 'last_change': '2023-11-26T07:22:22.133925+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.133925+0000', 'last_undegraded': '2023-11-26T07:22:22.133925+0000', 'last_fullsized': '2023-11-26T07:22:22.133925+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:22:28.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2023-11-26T07:22:28.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.128444+0000', 'last_change': '2023-11-26T07:22:22.128444+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.128444+0000', 'last_undegraded': '2023-11-26T07:22:22.128444+0000', 'last_fullsized': '2023-11-26T07:22:22.128444+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:22:28.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1b is not active+clean 2023-11-26T07:22:28.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.142692+0000', 'last_change': '2023-11-26T07:22:22.142692+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.142692+0000', 'last_undegraded': '2023-11-26T07:22:22.142692+0000', 'last_fullsized': '2023-11-26T07:22:22.142692+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.18 is not active+clean 2023-11-26T07:22:28.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.132776+0000', 'last_change': '2023-11-26T07:22:22.132776+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.132776+0000', 'last_undegraded': '2023-11-26T07:22:22.132776+0000', 'last_fullsized': '2023-11-26T07:22:22.132776+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:22:28.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2023-11-26T07:22:28.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.128946+0000', 'last_change': '2023-11-26T07:22:22.128946+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.128946+0000', 'last_undegraded': '2023-11-26T07:22:22.128946+0000', 'last_fullsized': '2023-11-26T07:22:22.128946+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:22:28.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2023-11-26T07:22:28.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.130940+0000', 'last_change': '2023-11-26T07:22:22.130940+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.130940+0000', 'last_undegraded': '2023-11-26T07:22:22.130940+0000', 'last_fullsized': '2023-11-26T07:22:22.130940+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-26T07:22:28.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.17 is not active+clean 2023-11-26T07:22:28.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.130873+0000', 'last_change': '2023-11-26T07:22:22.130873+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.130873+0000', 'last_undegraded': '2023-11-26T07:22:22.130873+0000', 'last_fullsized': '2023-11-26T07:22:22.130873+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:22:28.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2023-11-26T07:22:28.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.141167+0000', 'last_change': '2023-11-26T07:22:22.141167+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.141167+0000', 'last_undegraded': '2023-11-26T07:22:22.141167+0000', 'last_fullsized': '2023-11-26T07:22:22.141167+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:22:28.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.15 is not active+clean 2023-11-26T07:22:28.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.139846+0000', 'last_change': '2023-11-26T07:22:22.139846+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.139846+0000', 'last_undegraded': '2023-11-26T07:22:22.139846+0000', 'last_fullsized': '2023-11-26T07:22:22.139846+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2023-11-26T07:22:28.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.139244+0000', 'last_change': '2023-11-26T07:22:22.139244+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.139244+0000', 'last_undegraded': '2023-11-26T07:22:22.139244+0000', 'last_fullsized': '2023-11-26T07:22:22.139244+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:22:28.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.13 is not active+clean 2023-11-26T07:22:28.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.134871+0000', 'last_change': '2023-11-26T07:22:22.134871+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.134871+0000', 'last_undegraded': '2023-11-26T07:22:22.134871+0000', 'last_fullsized': '2023-11-26T07:22:22.134871+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:22:28.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2023-11-26T07:22:28.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.128109+0000', 'last_change': '2023-11-26T07:22:22.128109+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.128109+0000', 'last_undegraded': '2023-11-26T07:22:22.128109+0000', 'last_fullsized': '2023-11-26T07:22:22.128109+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:22:28.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2023-11-26T07:22:28.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.135370+0000', 'last_change': '2023-11-26T07:22:22.135370+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.135370+0000', 'last_undegraded': '2023-11-26T07:22:22.135370+0000', 'last_fullsized': '2023-11-26T07:22:22.135370+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:22:28.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.e is not active+clean 2023-11-26T07:22:28.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.134114+0000', 'last_change': '2023-11-26T07:22:22.134114+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.134114+0000', 'last_undegraded': '2023-11-26T07:22:22.134114+0000', 'last_fullsized': '2023-11-26T07:22:22.134114+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:22:28.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.f is not active+clean 2023-11-26T07:22:28.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.128923+0000', 'last_change': '2023-11-26T07:22:22.128923+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.128923+0000', 'last_undegraded': '2023-11-26T07:22:22.128923+0000', 'last_fullsized': '2023-11-26T07:22:22.128923+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.d is not active+clean 2023-11-26T07:22:28.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.137359+0000', 'last_change': '2023-11-26T07:22:22.137359+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.137359+0000', 'last_undegraded': '2023-11-26T07:22:22.137359+0000', 'last_fullsized': '2023-11-26T07:22:22.137359+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2023-11-26T07:22:28.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.132275+0000', 'last_change': '2023-11-26T07:22:22.132275+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.132275+0000', 'last_undegraded': '2023-11-26T07:22:22.132275+0000', 'last_fullsized': '2023-11-26T07:22:22.132275+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:22:28.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.b is not active+clean 2023-11-26T07:22:28.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.129550+0000', 'last_change': '2023-11-26T07:22:22.129550+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.129550+0000', 'last_undegraded': '2023-11-26T07:22:22.129550+0000', 'last_fullsized': '2023-11-26T07:22:22.129550+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:22:28.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2023-11-26T07:22:28.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.131250+0000', 'last_change': '2023-11-26T07:22:22.131250+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.131250+0000', 'last_undegraded': '2023-11-26T07:22:22.131250+0000', 'last_fullsized': '2023-11-26T07:22:22.131250+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-26T07:22:28.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.9 is not active+clean 2023-11-26T07:22:28.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.129655+0000', 'last_change': '2023-11-26T07:22:22.129655+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.129655+0000', 'last_undegraded': '2023-11-26T07:22:22.129655+0000', 'last_fullsized': '2023-11-26T07:22:22.129655+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:22:28.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.6 is not active+clean 2023-11-26T07:22:28.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.143648+0000', 'last_change': '2023-11-26T07:22:22.143648+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.143648+0000', 'last_undegraded': '2023-11-26T07:22:22.143648+0000', 'last_fullsized': '2023-11-26T07:22:22.143648+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.7 is not active+clean 2023-11-26T07:22:28.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.132436+0000', 'last_change': '2023-11-26T07:22:22.132436+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.132436+0000', 'last_undegraded': '2023-11-26T07:22:22.132436+0000', 'last_fullsized': '2023-11-26T07:22:22.132436+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:22:28.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.4 is not active+clean 2023-11-26T07:22:28.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.129092+0000', 'last_change': '2023-11-26T07:22:22.129092+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.129092+0000', 'last_undegraded': '2023-11-26T07:22:22.129092+0000', 'last_fullsized': '2023-11-26T07:22:22.129092+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.5 is not active+clean 2023-11-26T07:22:28.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.136808+0000', 'last_change': '2023-11-26T07:22:22.136808+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.136808+0000', 'last_undegraded': '2023-11-26T07:22:22.136808+0000', 'last_fullsized': '2023-11-26T07:22:22.136808+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-26T07:22:28.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2 is not active+clean 2023-11-26T07:22:28.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.129561+0000', 'last_change': '2023-11-26T07:22:22.129561+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.129561+0000', 'last_undegraded': '2023-11-26T07:22:22.129561+0000', 'last_fullsized': '2023-11-26T07:22:22.129561+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-26T07:22:28.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2023-11-26T07:22:28.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.137697+0000', 'last_change': '2023-11-26T07:22:22.137697+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.137697+0000', 'last_undegraded': '2023-11-26T07:22:22.137697+0000', 'last_fullsized': '2023-11-26T07:22:22.137697+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-26T07:22:28.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.0 is not active+clean 2023-11-26T07:22:28.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.132256+0000', 'last_change': '2023-11-26T07:22:22.132256+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.132256+0000', 'last_undegraded': '2023-11-26T07:22:22.132256+0000', 'last_fullsized': '2023-11-26T07:22:22.132256+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-26T07:22:28.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1 is not active+clean 2023-11-26T07:22:28.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.128291+0000', 'last_change': '2023-11-26T07:22:22.128291+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.128291+0000', 'last_undegraded': '2023-11-26T07:22:22.128291+0000', 'last_fullsized': '2023-11-26T07:22:22.128291+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-26T07:22:28.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2023-11-26T07:22:28.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 78, 'state': 'creating+peering', 'last_fresh': '2023-11-26T07:22:22.127322+0000', 'last_change': '2023-11-26T07:22:22.127322+0000', 'last_active': '2023-11-26T07:22:22.119714+0000', 'last_peered': '2023-11-26T07:22:22.119714+0000', 'last_clean': '2023-11-26T07:22:22.119714+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-26T07:22:22.127322+0000', 'last_undegraded': '2023-11-26T07:22:22.127322+0000', 'last_fullsized': '2023-11-26T07:22:22.127322+0000', 'mapping_epoch': 78, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'last_clean_scrub_stamp': '2023-11-26T07:22:22.119714+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-26T07:22:28.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-26T07:22:28.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.253+0000 7fa665a6b640 1 Processor -- start 2023-11-26T07:22:28.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.253+0000 7fa665a6b640 1 -- start start 2023-11-26T07:22:28.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.253+0000 7fa665a6b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa66015d8b0 0x7fa6600b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.253+0000 7fa665a6b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa6600cecd0 0x7fa66006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.253+0000 7fa665a6b640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6600b0a70 con 0x7fa6600b5120 2023-11-26T07:22:28.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.253+0000 7fa665a6b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6600cddb0 con 0x7fa6600cecd0 2023-11-26T07:22:28.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.253+0000 7fa665a6b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6600b4100 con 0x7fa66015d8b0 2023-11-26T07:22:28.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65effd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa6600b5120 0x7fa6600ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60508/0 (socket says 172.21.15.31:60508) 2023-11-26T07:22:28.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65effd640 1 -- 172.21.15.31:0/2402488567 learned_addr learned my addr 172.21.15.31:0/2402488567 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65f7fe640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa66015d8b0 0x7fa6600b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65ffff640 1 --2- 172.21.15.31:0/2402488567 >> v2:172.21.15.97:3300/0 conn(0x7fa6600cecd0 0x7fa66006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253214075 0 0) 0x7fa6600b0a70 con 0x7fa6600b5120 2023-11-26T07:22:28.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa648003530 con 0x7fa6600b5120 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1770975154 0 0) 0x7fa648003530 con 0x7fa6600b5120 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa66015d8b0 msgr2=0x7fa6600b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa66015d8b0 0x7fa6600b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 >> v2:172.21.15.97:3300/0 conn(0x7fa6600cecd0 msgr2=0x7fa66006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 --2- 172.21.15.31:0/2402488567 >> v2:172.21.15.97:3300/0 conn(0x7fa6600cecd0 0x7fa66006a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6600b43e0 con 0x7fa6600b5120 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fa654001680 con 0x7fa6600b5120 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 >> v1:172.21.15.31:6789/0 conn(0x7fa6600b5120 legacy=0x7fa6600ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa648003980 0x7fa648003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa66015d8b0 0x7fa6480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa648004d10 0x7fa648005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6600cddb0 con 0x7fa648004d10 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6600b4100 con 0x7fa648003980 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64800b990 con 0x7fa66015d8b0 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65ffff640 1 --2- 172.21.15.31:0/2402488567 >> v2:172.21.15.97:3300/0 conn(0x7fa6600cecd0 0x7fa66006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65ffff640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa648004d10 0x7fa648005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65effd640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa66015d8b0 0x7fa6480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65f7fe640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa648003980 0x7fa648003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65ffff640 1 -- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa66015d8b0 msgr2=0x7fa6480047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65ffff640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa66015d8b0 0x7fa6480047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65ffff640 1 -- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa648003980 msgr2=0x7fa648003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:28.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65ffff640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa648003980 0x7fa648003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.257+0000 7fa65ffff640 1 -- 172.21.15.31:0/2402488567 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648003530 con 0x7fa648004d10 2023-11-26T07:22:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa65ffff640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa648004d10 0x7fa648005190 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa65800e350 tx=0x7fa65801ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1af879c6f25b050 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6580a5e50 con 0x7fa648004d10 2023-11-26T07:22:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65800f040 con 0x7fa648004d10 2023-11-26T07:22:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa65e7fc640 1 -- 172.21.15.31:0/2402488567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658003ba0 con 0x7fa648004d10 2023-11-26T07:22:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa665a6b640 1 -- 172.21.15.31:0/2402488567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa648004d10 msgr2=0x7fa648005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa665a6b640 1 --2- 172.21.15.31:0/2402488567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa648004d10 0x7fa648005190 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa65800e350 tx=0x7fa65801ce40 comp rx=0 tx=0).stop 2023-11-26T07:22:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa65ffff640 1 -- 172.21.15.31:0/2402488567 reap_dead start 2023-11-26T07:22:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa665a6b640 1 -- 172.21.15.31:0/2402488567 shutdown_connections 2023-11-26T07:22:28.264 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa665a6b640 1 -- 172.21.15.31:0/2402488567 >> 172.21.15.31:0/2402488567 conn(0x7fa6600b5570 msgr2=0x7fa6600b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:28.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa665a6b640 1 -- 172.21.15.31:0/2402488567 shutdown_connections 2023-11-26T07:22:28.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa665a6b640 1 -- 172.21.15.31:0/2402488567 wait complete. 2023-11-26T07:22:28.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa665a6b640 1 Processor -- start 2023-11-26T07:22:28.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.261+0000 7fa665a6b640 1 -- start start 2023-11-26T07:22:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa665a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa66014ae80 0x7fa660160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa665a6b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa66015dc90 0x7fa6600aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa665a6b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6600af4e0 0x7fa6600ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa665a6b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6600af960 con 0x7fa66015dc90 2023-11-26T07:22:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa665a6b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66015e4e0 con 0x7fa66014ae80 2023-11-26T07:22:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa665a6b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66015e7c0 con 0x7fa6600af4e0 2023-11-26T07:22:28.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa66015dc90 0x7fa6600aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa66015dc90 0x7fa6600aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37950/0 (socket says 172.21.15.31:37950) 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65effd640 1 -- 172.21.15.31:0/2538036730 learned_addr learned my addr 172.21.15.31:0/2538036730 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65f7fe640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa66014ae80 0x7fa660160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65ffff640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6600af4e0 0x7fa6600ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65effd640 1 -- 172.21.15.31:0/2538036730 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6600af4e0 msgr2=0x7fa6600ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65effd640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6600af4e0 0x7fa6600ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65effd640 1 -- 172.21.15.31:0/2538036730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa66014ae80 msgr2=0x7fa660160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65effd640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa66014ae80 0x7fa660160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65effd640 1 -- 172.21.15.31:0/2538036730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66015eaa0 con 0x7fa66015dc90 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65f7fe640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa66014ae80 0x7fa660160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65ffff640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6600af4e0 0x7fa6600ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:28.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa65effd640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa66015dc90 0x7fa6600aefa0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa654007340 tx=0x7fa654004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa647fff640 1 -- 172.21.15.31:0/2538036730 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa654001ba0 con 0x7fa66015dc90 2023-11-26T07:22:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa647fff640 1 -- 172.21.15.31:0/2538036730 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65400f9c0 con 0x7fa66015dc90 2023-11-26T07:22:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa665a6b640 1 -- 172.21.15.31:0/2538036730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66015ed80 con 0x7fa66015dc90 2023-11-26T07:22:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa647fff640 1 -- 172.21.15.31:0/2538036730 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65400fd50 con 0x7fa66015dc90 2023-11-26T07:22:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa665a6b640 1 -- 172.21.15.31:0/2538036730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66015f580 con 0x7fa66015dc90 2023-11-26T07:22:28.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.265+0000 7fa665a6b640 1 -- 172.21.15.31:0/2538036730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa660160c80 con 0x7fa66015dc90 2023-11-26T07:22:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.269+0000 7fa647fff640 1 -- 172.21.15.31:0/2538036730 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fa65401d460 con 0x7fa66015dc90 2023-11-26T07:22:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.269+0000 7fa647fff640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa64c0665c0 0x7fa64c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.269+0000 7fa647fff640 1 -- 172.21.15.31:0/2538036730 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa654084230 con 0x7fa66015dc90 2023-11-26T07:22:28.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.269+0000 7fa65f7fe640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa64c0665c0 0x7fa64c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.269+0000 7fa65f7fe640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa64c0665c0 0x7fa64c068a80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa650009ad0 tx=0x7fa650008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.273+0000 7fa647fff640 1 -- 172.21.15.31:0/2538036730 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa654051500 con 0x7fa66015dc90 2023-11-26T07:22:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 -- 172.21.15.31:0/2538036730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa66004e850 con 0x7fa66015dc90 2023-11-26T07:22:28.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa647fff640 1 -- 172.21.15.31:0/2538036730 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1452 (secure 0 0 0) 0x7fa654054b20 con 0x7fa66015dc90 2023-11-26T07:22:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:28.487 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":321,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":80,"num_osds":8,"num_up_osds":8,"osd_up_since":1700983039,"num_in_osds":8,"osd_in_since":1700983029,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":264241152,"bytes_avail":772829872128,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":57,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7101}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-11-26T07:21:36.368671+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 -- 172.21.15.31:0/2538036730 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa64c0665c0 msgr2=0x7fa64c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa64c0665c0 0x7fa64c068a80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa650009ad0 tx=0x7fa650008040 comp rx=0 tx=0).stop 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 -- 172.21.15.31:0/2538036730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa66015dc90 msgr2=0x7fa6600aefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa66015dc90 0x7fa6600aefa0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa654007340 tx=0x7fa654004f90 comp rx=0 tx=0).stop 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 -- 172.21.15.31:0/2538036730 shutdown_connections 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa64c0665c0 0x7fa64c068a80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa6600af4e0 0x7fa6600ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa66015dc90 0x7fa6600aefa0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 --2- 172.21.15.31:0/2538036730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa66014ae80 0x7fa660160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 -- 172.21.15.31:0/2538036730 >> 172.21.15.31:0/2538036730 conn(0x7fa6600b5570 msgr2=0x7fa6600cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:28.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 -- 172.21.15.31:0/2538036730 shutdown_connections 2023-11-26T07:22:28.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.485+0000 7fa665a6b640 1 -- 172.21.15.31:0/2538036730 wait complete. 2023-11-26T07:22:28.508 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-26T07:22:28.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:22:28.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601eb08640 1 Processor -- start 2023-11-26T07:22:28.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601eb08640 1 -- start start 2023-11-26T07:22:28.652 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601eb08640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f60180ae3d0 0x7f601806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601eb08640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60180ae7f0 con 0x7f601806dfd0 2023-11-26T07:22:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601eb08640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601806e980 con 0x7f6018143ae0 2023-11-26T07:22:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601eb08640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6018070670 con 0x7f60180ae3d0 2023-11-26T07:22:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601d0a5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f60180ae3d0 0x7f601806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f6017fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f601806dfd0 0x7f60180adcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60510/0 (socket says 172.21.15.31:60510) 2023-11-26T07:22:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f6017fff640 1 -- 172.21.15.31:0/1065715299 learned_addr learned my addr 172.21.15.31:0/1065715299 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:28.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601d0a5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f60180ae3d0 0x7f601806e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37956/0 (socket says 172.21.15.31:37956) 2023-11-26T07:22:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601d0a5640 1 -- 172.21.15.31:0/1065715299 >> v1:172.21.15.97:6789/0 conn(0x7f6018143ae0 legacy=0x7f60180b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601d0a5640 1 -- 172.21.15.31:0/1065715299 >> v1:172.21.15.31:6789/0 conn(0x7f601806dfd0 legacy=0x7f60180adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f601d0a5640 1 -- 172.21.15.31:0/1065715299 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6018070950 con 0x7f60180ae3d0 2023-11-26T07:22:28.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.649+0000 7f60177fe640 1 -- 172.21.15.31:0/1065715299 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000182873 0 0) 0x7f60180ae7f0 con 0x7f601806dfd0 2023-11-26T07:22:28.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f601d0a5640 1 --2- 172.21.15.31:0/1065715299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60180ae3d0 0x7f601806e440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6010009d90 tx=0x7f601001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4404a5f18a43d8ad server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f60177fe640 1 -- 172.21.15.31:0/1065715299 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60100a59a0 con 0x7f60180ae3d0 2023-11-26T07:22:28.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f60177fe640 1 -- 172.21.15.31:0/1065715299 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60100a5b00 con 0x7f60180ae3d0 2023-11-26T07:22:28.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f60177fe640 1 -- 172.21.15.31:0/1065715299 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60100a5e90 con 0x7f60180ae3d0 2023-11-26T07:22:28.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f601eb08640 1 -- 172.21.15.31:0/1065715299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60180ae3d0 msgr2=0x7f601806e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f601eb08640 1 --2- 172.21.15.31:0/1065715299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60180ae3d0 0x7f601806e440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6010009d90 tx=0x7f601001c8e0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f601eb08640 1 -- 172.21.15.31:0/1065715299 shutdown_connections 2023-11-26T07:22:28.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f601eb08640 1 --2- 172.21.15.31:0/1065715299 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60180ae3d0 0x7f601806e440 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f601eb08640 1 -- 172.21.15.31:0/1065715299 >> 172.21.15.31:0/1065715299 conn(0x7f60180b0c40 msgr2=0x7f60180b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:28.657 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f601eb08640 1 -- 172.21.15.31:0/1065715299 shutdown_connections 2023-11-26T07:22:28.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.653+0000 7f601eb08640 1 -- 172.21.15.31:0/1065715299 wait complete. 2023-11-26T07:22:28.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601eb08640 1 Processor -- start 2023-11-26T07:22:28.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601eb08640 1 -- start start 2023-11-26T07:22:28.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601eb08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601806dfd0 0x7f601809cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601eb08640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60180ae3d0 0x7f601815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601eb08640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6018143ae0 0x7f6018159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601eb08640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601809bfb0 con 0x7f601806dfd0 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601eb08640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6018140d50 con 0x7f60180ae3d0 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601eb08640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601815cd50 con 0x7f6018143ae0 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601c8a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601806dfd0 0x7f601809cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601c8a4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601806dfd0 0x7f601809cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37962/0 (socket says 172.21.15.31:37962) 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601c8a4640 1 -- 172.21.15.31:0/2867833930 learned_addr learned my addr 172.21.15.31:0/2867833930 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601d0a5640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6018143ae0 0x7f6018159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f6017fff640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60180ae3d0 0x7f601815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601c8a4640 1 -- 172.21.15.31:0/2867833930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6018143ae0 msgr2=0x7f6018159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601c8a4640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6018143ae0 0x7f6018159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601c8a4640 1 -- 172.21.15.31:0/2867833930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60180ae3d0 msgr2=0x7f601815bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601c8a4640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60180ae3d0 0x7f601815bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601c8a4640 1 -- 172.21.15.31:0/2867833930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6010009a70 con 0x7f601806dfd0 2023-11-26T07:22:28.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601c8a4640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601806dfd0 0x7f601809cb60 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f600c00b410 tx=0x7f600c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f60157fa640 1 -- 172.21.15.31:0/2867833930 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c004020 con 0x7f601806dfd0 2023-11-26T07:22:28.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f60157fa640 1 -- 172.21.15.31:0/2867833930 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c004180 con 0x7f601806dfd0 2023-11-26T07:22:28.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601eb08640 1 -- 172.21.15.31:0/2867833930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f601815c140 con 0x7f601806dfd0 2023-11-26T07:22:28.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f60157fa640 1 -- 172.21.15.31:0/2867833930 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c011630 con 0x7f601806dfd0 2023-11-26T07:22:28.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.657+0000 7f601eb08640 1 -- 172.21.15.31:0/2867833930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601815c700 con 0x7f601806dfd0 2023-11-26T07:22:28.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.661+0000 7f601eb08640 1 -- 172.21.15.31:0/2867833930 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601809d110 con 0x7f601806dfd0 2023-11-26T07:22:28.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.661+0000 7f60157fa640 1 -- 172.21.15.31:0/2867833930 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f600c011790 con 0x7f601806dfd0 2023-11-26T07:22:28.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.661+0000 7f60157fa640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ffc066590 0x7f5ffc068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:28.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.661+0000 7f60157fa640 1 -- 172.21.15.31:0/2867833930 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f600c083d00 con 0x7f601806dfd0 2023-11-26T07:22:28.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.661+0000 7f6017fff640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ffc066590 0x7f5ffc068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:28.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.661+0000 7f6017fff640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ffc066590 0x7f5ffc068a50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60181655b0 tx=0x7f6004007d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:28.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.665+0000 7f60157fa640 1 -- 172.21.15.31:0/2867833930 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f600c051080 con 0x7f601806dfd0 2023-11-26T07:22:28.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.821+0000 7f601eb08640 1 -- 172.21.15.31:0/2867833930 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6018143ec0 con 0x7f5ffc066590 2023-11-26T07:22:28.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.829+0000 7f60157fa640 1 -- 172.21.15.31:0/2867833930 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288657 (secure 0 0 0) 0x7f6004004c80 con 0x7f5ffc066590 2023-11-26T07:22:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:28.835 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:22:28.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.829+0000 7f601eb08640 1 -- 172.21.15.31:0/2867833930 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ffc066590 msgr2=0x7f5ffc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.829+0000 7f601eb08640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ffc066590 0x7f5ffc068a50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60181655b0 tx=0x7f6004007d60 comp rx=0 tx=0).stop 2023-11-26T07:22:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.829+0000 7f601eb08640 1 -- 172.21.15.31:0/2867833930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601806dfd0 msgr2=0x7f601809cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.829+0000 7f601eb08640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601806dfd0 0x7f601809cb60 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f600c00b410 tx=0x7f600c00b8d0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.833+0000 7f601eb08640 1 -- 172.21.15.31:0/2867833930 shutdown_connections 2023-11-26T07:22:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.833+0000 7f601eb08640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ffc066590 0x7f5ffc068a50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.833+0000 7f601eb08640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6018143ae0 0x7f6018159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.833+0000 7f601eb08640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f60180ae3d0 0x7f601815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.833+0000 7f601eb08640 1 --2- 172.21.15.31:0/2867833930 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601806dfd0 0x7f601809cb60 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.833+0000 7f601eb08640 1 -- 172.21.15.31:0/2867833930 >> 172.21.15.31:0/2867833930 conn(0x7f60180b0c40 msgr2=0x7f60180afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.833+0000 7f601eb08640 1 -- 172.21.15.31:0/2867833930 shutdown_connections 2023-11-26T07:22:28.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:28.833+0000 7f601eb08640 1 -- 172.21.15.31:0/2867833930 wait complete. 2023-11-26T07:22:28.853 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-11-26T07:22:28.201259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258380,"kb_used_data":8900,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716340,"statfs":{"total":773094113280,"available":772829532160,"internally_reserved":0,"allocated":9113600,"data_stored":4553513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.077659"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144176+0000","last_change":"2023-11-26T07:22:23.150661+0000","last_active":"2023-11-26T07:22:24.144176+0000","last_peered":"2023-11-26T07:22:24.144176+0000","last_clean":"2023-11-26T07:22:24.144176+0000","last_became_active":"2023-11-26T07:22:23.150364+0000","last_became_peered":"2023-11-26T07:22:23.150364+0000","last_unstale":"2023-11-26T07:22:24.144176+0000","last_undegraded":"2023-11-26T07:22:24.144176+0000","last_fullsized":"2023-11-26T07:22:24.144176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:30.716028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.463202+0000","last_change":"2023-11-26T07:22:23.161637+0000","last_active":"2023-11-26T07:22:24.463202+0000","last_peered":"2023-11-26T07:22:24.463202+0000","last_clean":"2023-11-26T07:22:24.463202+0000","last_became_active":"2023-11-26T07:22:23.161224+0000","last_became_peered":"2023-11-26T07:22:23.161224+0000","last_unstale":"2023-11-26T07:22:24.463202+0000","last_undegraded":"2023-11-26T07:22:24.463202+0000","last_fullsized":"2023-11-26T07:22:24.463202+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:53:46.158461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461946+0000","last_change":"2023-11-26T07:22:23.162459+0000","last_active":"2023-11-26T07:22:24.461946+0000","last_peered":"2023-11-26T07:22:24.461946+0000","last_clean":"2023-11-26T07:22:24.461946+0000","last_became_active":"2023-11-26T07:22:23.162090+0000","last_became_peered":"2023-11-26T07:22:23.162090+0000","last_unstale":"2023-11-26T07:22:24.461946+0000","last_undegraded":"2023-11-26T07:22:24.461946+0000","last_fullsized":"2023-11-26T07:22:24.461946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:07.939507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140214+0000","last_change":"2023-11-26T07:22:23.171323+0000","last_active":"2023-11-26T07:22:24.140214+0000","last_peered":"2023-11-26T07:22:24.140214+0000","last_clean":"2023-11-26T07:22:24.140214+0000","last_became_active":"2023-11-26T07:22:23.171024+0000","last_became_peered":"2023-11-26T07:22:23.171024+0000","last_unstale":"2023-11-26T07:22:24.140214+0000","last_undegraded":"2023-11-26T07:22:24.140214+0000","last_fullsized":"2023-11-26T07:22:24.140214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:08.063434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142616+0000","last_change":"2023-11-26T07:22:23.166535+0000","last_active":"2023-11-26T07:22:24.142616+0000","last_peered":"2023-11-26T07:22:24.142616+0000","last_clean":"2023-11-26T07:22:24.142616+0000","last_became_active":"2023-11-26T07:22:23.165856+0000","last_became_peered":"2023-11-26T07:22:23.165856+0000","last_unstale":"2023-11-26T07:22:24.142616+0000","last_undegraded":"2023-11-26T07:22:24.142616+0000","last_fullsized":"2023-11-26T07:22:24.142616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:06.468579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461591+0000","last_change":"2023-11-26T07:22:23.160483+0000","last_active":"2023-11-26T07:22:24.461591+0000","last_peered":"2023-11-26T07:22:24.461591+0000","last_clean":"2023-11-26T07:22:24.461591+0000","last_became_active":"2023-11-26T07:22:23.160186+0000","last_became_peered":"2023-11-26T07:22:23.160186+0000","last_unstale":"2023-11-26T07:22:24.461591+0000","last_undegraded":"2023-11-26T07:22:24.461591+0000","last_fullsized":"2023-11-26T07:22:24.461591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:11:11.552689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.139868+0000","last_change":"2023-11-26T07:22:23.161985+0000","last_active":"2023-11-26T07:22:24.139868+0000","last_peered":"2023-11-26T07:22:24.139868+0000","last_clean":"2023-11-26T07:22:24.139868+0000","last_became_active":"2023-11-26T07:22:23.161700+0000","last_became_peered":"2023-11-26T07:22:23.161700+0000","last_unstale":"2023-11-26T07:22:24.139868+0000","last_undegraded":"2023-11-26T07:22:24.139868+0000","last_fullsized":"2023-11-26T07:22:24.139868+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:18.516207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461739+0000","last_change":"2023-11-26T07:22:23.156569+0000","last_active":"2023-11-26T07:22:24.461739+0000","last_peered":"2023-11-26T07:22:24.461739+0000","last_clean":"2023-11-26T07:22:24.461739+0000","last_became_active":"2023-11-26T07:22:23.155834+0000","last_became_peered":"2023-11-26T07:22:23.155834+0000","last_unstale":"2023-11-26T07:22:24.461739+0000","last_undegraded":"2023-11-26T07:22:24.461739+0000","last_fullsized":"2023-11-26T07:22:24.461739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:23:30.693319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462480+0000","last_change":"2023-11-26T07:22:23.161805+0000","last_active":"2023-11-26T07:22:24.462480+0000","last_peered":"2023-11-26T07:22:24.462480+0000","last_clean":"2023-11-26T07:22:24.462480+0000","last_became_active":"2023-11-26T07:22:23.161520+0000","last_became_peered":"2023-11-26T07:22:23.161520+0000","last_unstale":"2023-11-26T07:22:24.462480+0000","last_undegraded":"2023-11-26T07:22:24.462480+0000","last_fullsized":"2023-11-26T07:22:24.462480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:06.468579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144155+0000","last_change":"2023-11-26T07:22:23.148510+0000","last_active":"2023-11-26T07:22:24.144155+0000","last_peered":"2023-11-26T07:22:24.144155+0000","last_clean":"2023-11-26T07:22:24.144155+0000","last_became_active":"2023-11-26T07:22:23.148096+0000","last_became_peered":"2023-11-26T07:22:23.148096+0000","last_unstale":"2023-11-26T07:22:24.144155+0000","last_undegraded":"2023-11-26T07:22:24.144155+0000","last_fullsized":"2023-11-26T07:22:24.144155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:13:00.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.463061+0000","last_change":"2023-11-26T07:22:23.167271+0000","last_active":"2023-11-26T07:22:24.463061+0000","last_peered":"2023-11-26T07:22:24.463061+0000","last_clean":"2023-11-26T07:22:24.463061+0000","last_became_active":"2023-11-26T07:22:23.166179+0000","last_became_peered":"2023-11-26T07:22:23.166179+0000","last_unstale":"2023-11-26T07:22:24.463061+0000","last_undegraded":"2023-11-26T07:22:24.463061+0000","last_fullsized":"2023-11-26T07:22:24.463061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:34.890789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144148+0000","last_change":"2023-11-26T07:22:23.144450+0000","last_active":"2023-11-26T07:22:24.144148+0000","last_peered":"2023-11-26T07:22:24.144148+0000","last_clean":"2023-11-26T07:22:24.144148+0000","last_became_active":"2023-11-26T07:22:23.144127+0000","last_became_peered":"2023-11-26T07:22:23.144127+0000","last_unstale":"2023-11-26T07:22:24.144148+0000","last_undegraded":"2023-11-26T07:22:24.144148+0000","last_fullsized":"2023-11-26T07:22:24.144148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:07.939507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461514+0000","last_change":"2023-11-26T07:22:23.161346+0000","last_active":"2023-11-26T07:22:24.461514+0000","last_peered":"2023-11-26T07:22:24.461514+0000","last_clean":"2023-11-26T07:22:24.461514+0000","last_became_active":"2023-11-26T07:22:23.161059+0000","last_became_peered":"2023-11-26T07:22:23.161059+0000","last_unstale":"2023-11-26T07:22:24.461514+0000","last_undegraded":"2023-11-26T07:22:24.461514+0000","last_fullsized":"2023-11-26T07:22:24.461514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:31.249944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140099+0000","last_change":"2023-11-26T07:22:23.170673+0000","last_active":"2023-11-26T07:22:24.140099+0000","last_peered":"2023-11-26T07:22:24.140099+0000","last_clean":"2023-11-26T07:22:24.140099+0000","last_became_active":"2023-11-26T07:22:23.170227+0000","last_became_peered":"2023-11-26T07:22:23.170227+0000","last_unstale":"2023-11-26T07:22:24.140099+0000","last_undegraded":"2023-11-26T07:22:24.140099+0000","last_fullsized":"2023-11-26T07:22:24.140099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:30:14.851987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461806+0000","last_change":"2023-11-26T07:22:23.155339+0000","last_active":"2023-11-26T07:22:24.461806+0000","last_peered":"2023-11-26T07:22:24.461806+0000","last_clean":"2023-11-26T07:22:24.461806+0000","last_became_active":"2023-11-26T07:22:23.155010+0000","last_became_peered":"2023-11-26T07:22:23.155010+0000","last_unstale":"2023-11-26T07:22:24.461806+0000","last_undegraded":"2023-11-26T07:22:24.461806+0000","last_fullsized":"2023-11-26T07:22:24.461806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:51.056578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142356+0000","last_change":"2023-11-26T07:22:23.152403+0000","last_active":"2023-11-26T07:22:24.142356+0000","last_peered":"2023-11-26T07:22:24.142356+0000","last_clean":"2023-11-26T07:22:24.142356+0000","last_became_active":"2023-11-26T07:22:23.152082+0000","last_became_peered":"2023-11-26T07:22:23.152082+0000","last_unstale":"2023-11-26T07:22:24.142356+0000","last_undegraded":"2023-11-26T07:22:24.142356+0000","last_fullsized":"2023-11-26T07:22:24.142356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:15.445163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142974+0000","last_change":"2023-11-26T07:22:23.160642+0000","last_active":"2023-11-26T07:22:24.142974+0000","last_peered":"2023-11-26T07:22:24.142974+0000","last_clean":"2023-11-26T07:22:24.142974+0000","last_became_active":"2023-11-26T07:22:23.160335+0000","last_became_peered":"2023-11-26T07:22:23.160335+0000","last_unstale":"2023-11-26T07:22:24.142974+0000","last_undegraded":"2023-11-26T07:22:24.142974+0000","last_fullsized":"2023-11-26T07:22:24.142974+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:26:35.758938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140360+0000","last_change":"2023-11-26T07:22:23.161871+0000","last_active":"2023-11-26T07:22:24.140360+0000","last_peered":"2023-11-26T07:22:24.140360+0000","last_clean":"2023-11-26T07:22:24.140360+0000","last_became_active":"2023-11-26T07:22:23.161576+0000","last_became_peered":"2023-11-26T07:22:23.161576+0000","last_unstale":"2023-11-26T07:22:24.140360+0000","last_undegraded":"2023-11-26T07:22:24.140360+0000","last_fullsized":"2023-11-26T07:22:24.140360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:24.741346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462367+0000","last_change":"2023-11-26T07:22:23.158365+0000","last_active":"2023-11-26T07:22:24.462367+0000","last_peered":"2023-11-26T07:22:24.462367+0000","last_clean":"2023-11-26T07:22:24.462367+0000","last_became_active":"2023-11-26T07:22:23.158040+0000","last_became_peered":"2023-11-26T07:22:23.158040+0000","last_unstale":"2023-11-26T07:22:24.462367+0000","last_undegraded":"2023-11-26T07:22:24.462367+0000","last_fullsized":"2023-11-26T07:22:24.462367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:35:19.658117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140287+0000","last_change":"2023-11-26T07:22:23.167696+0000","last_active":"2023-11-26T07:22:24.140287+0000","last_peered":"2023-11-26T07:22:24.140287+0000","last_clean":"2023-11-26T07:22:24.140287+0000","last_became_active":"2023-11-26T07:22:23.167415+0000","last_became_peered":"2023-11-26T07:22:23.167415+0000","last_unstale":"2023-11-26T07:22:24.140287+0000","last_undegraded":"2023-11-26T07:22:24.140287+0000","last_fullsized":"2023-11-26T07:22:24.140287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:25:14.378387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140048+0000","last_change":"2023-11-26T07:22:23.166610+0000","last_active":"2023-11-26T07:22:24.140048+0000","last_peered":"2023-11-26T07:22:24.140048+0000","last_clean":"2023-11-26T07:22:24.140048+0000","last_became_active":"2023-11-26T07:22:23.166345+0000","last_became_peered":"2023-11-26T07:22:23.166345+0000","last_unstale":"2023-11-26T07:22:24.140048+0000","last_undegraded":"2023-11-26T07:22:24.140048+0000","last_fullsized":"2023-11-26T07:22:24.140048+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:45.111405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462949+0000","last_change":"2023-11-26T07:22:23.168094+0000","last_active":"2023-11-26T07:22:24.462949+0000","last_peered":"2023-11-26T07:22:24.462949+0000","last_clean":"2023-11-26T07:22:24.462949+0000","last_became_active":"2023-11-26T07:22:23.167796+0000","last_became_peered":"2023-11-26T07:22:23.167796+0000","last_unstale":"2023-11-26T07:22:24.462949+0000","last_undegraded":"2023-11-26T07:22:24.462949+0000","last_fullsized":"2023-11-26T07:22:24.462949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:09.121233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143794+0000","last_change":"2023-11-26T07:22:23.161411+0000","last_active":"2023-11-26T07:22:24.143794+0000","last_peered":"2023-11-26T07:22:24.143794+0000","last_clean":"2023-11-26T07:22:24.143794+0000","last_became_active":"2023-11-26T07:22:23.161151+0000","last_became_peered":"2023-11-26T07:22:23.161151+0000","last_unstale":"2023-11-26T07:22:24.143794+0000","last_undegraded":"2023-11-26T07:22:24.143794+0000","last_fullsized":"2023-11-26T07:22:24.143794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:26:35.758938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.139748+0000","last_change":"2023-11-26T07:22:23.162896+0000","last_active":"2023-11-26T07:22:24.139748+0000","last_peered":"2023-11-26T07:22:24.139748+0000","last_clean":"2023-11-26T07:22:24.139748+0000","last_became_active":"2023-11-26T07:22:23.162559+0000","last_became_peered":"2023-11-26T07:22:23.162559+0000","last_unstale":"2023-11-26T07:22:24.139748+0000","last_undegraded":"2023-11-26T07:22:24.139748+0000","last_fullsized":"2023-11-26T07:22:24.139748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:57.719160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.463146+0000","last_change":"2023-11-26T07:22:23.165648+0000","last_active":"2023-11-26T07:22:24.463146+0000","last_peered":"2023-11-26T07:22:24.463146+0000","last_clean":"2023-11-26T07:22:24.463146+0000","last_became_active":"2023-11-26T07:22:23.162343+0000","last_became_peered":"2023-11-26T07:22:23.162343+0000","last_unstale":"2023-11-26T07:22:24.463146+0000","last_undegraded":"2023-11-26T07:22:24.463146+0000","last_fullsized":"2023-11-26T07:22:24.463146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:36.318995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462040+0000","last_change":"2023-11-26T07:22:23.165553+0000","last_active":"2023-11-26T07:22:24.462040+0000","last_peered":"2023-11-26T07:22:24.462040+0000","last_clean":"2023-11-26T07:22:24.462040+0000","last_became_active":"2023-11-26T07:22:23.165220+0000","last_became_peered":"2023-11-26T07:22:23.165220+0000","last_unstale":"2023-11-26T07:22:24.462040+0000","last_undegraded":"2023-11-26T07:22:24.462040+0000","last_fullsized":"2023-11-26T07:22:24.462040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:30.716028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142778+0000","last_change":"2023-11-26T07:22:23.153109+0000","last_active":"2023-11-26T07:22:24.142778+0000","last_peered":"2023-11-26T07:22:24.142778+0000","last_clean":"2023-11-26T07:22:24.142778+0000","last_became_active":"2023-11-26T07:22:23.152820+0000","last_became_peered":"2023-11-26T07:22:23.152820+0000","last_unstale":"2023-11-26T07:22:24.142778+0000","last_undegraded":"2023-11-26T07:22:24.142778+0000","last_fullsized":"2023-11-26T07:22:24.142778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:55.059095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143853+0000","last_change":"2023-11-26T07:22:23.147508+0000","last_active":"2023-11-26T07:22:24.143853+0000","last_peered":"2023-11-26T07:22:24.143853+0000","last_clean":"2023-11-26T07:22:24.143853+0000","last_became_active":"2023-11-26T07:22:23.147164+0000","last_became_peered":"2023-11-26T07:22:23.147164+0000","last_unstale":"2023-11-26T07:22:24.143853+0000","last_undegraded":"2023-11-26T07:22:24.143853+0000","last_fullsized":"2023-11-26T07:22:24.143853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:31.249944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462721+0000","last_change":"2023-11-26T07:22:23.167488+0000","last_active":"2023-11-26T07:22:24.462721+0000","last_peered":"2023-11-26T07:22:24.462721+0000","last_clean":"2023-11-26T07:22:24.462721+0000","last_became_active":"2023-11-26T07:22:23.166958+0000","last_became_peered":"2023-11-26T07:22:23.166958+0000","last_unstale":"2023-11-26T07:22:24.462721+0000","last_undegraded":"2023-11-26T07:22:24.462721+0000","last_fullsized":"2023-11-26T07:22:24.462721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:16.114360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.463110+0000","last_change":"2023-11-26T07:22:23.165307+0000","last_active":"2023-11-26T07:22:24.463110+0000","last_peered":"2023-11-26T07:22:24.463110+0000","last_clean":"2023-11-26T07:22:24.463110+0000","last_became_active":"2023-11-26T07:22:23.162369+0000","last_became_peered":"2023-11-26T07:22:23.162369+0000","last_unstale":"2023-11-26T07:22:24.463110+0000","last_undegraded":"2023-11-26T07:22:24.463110+0000","last_fullsized":"2023-11-26T07:22:24.463110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:26.560742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142354+0000","last_change":"2023-11-26T07:22:23.160679+0000","last_active":"2023-11-26T07:22:24.142354+0000","last_peered":"2023-11-26T07:22:24.142354+0000","last_clean":"2023-11-26T07:22:24.142354+0000","last_became_active":"2023-11-26T07:22:23.158286+0000","last_became_peered":"2023-11-26T07:22:23.158286+0000","last_unstale":"2023-11-26T07:22:24.142354+0000","last_undegraded":"2023-11-26T07:22:24.142354+0000","last_fullsized":"2023-11-26T07:22:24.142354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:16:14.174321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143683+0000","last_change":"2023-11-26T07:22:23.154277+0000","last_active":"2023-11-26T07:22:24.143683+0000","last_peered":"2023-11-26T07:22:24.143683+0000","last_clean":"2023-11-26T07:22:24.143683+0000","last_became_active":"2023-11-26T07:22:23.153959+0000","last_became_peered":"2023-11-26T07:22:23.153959+0000","last_unstale":"2023-11-26T07:22:24.143683+0000","last_undegraded":"2023-11-26T07:22:24.143683+0000","last_fullsized":"2023-11-26T07:22:24.143683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:55.059095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142990+0000","last_change":"2023-11-26T07:22:23.152460+0000","last_active":"2023-11-26T07:22:24.142990+0000","last_peered":"2023-11-26T07:22:24.142990+0000","last_clean":"2023-11-26T07:22:24.142990+0000","last_became_active":"2023-11-26T07:22:23.152146+0000","last_became_peered":"2023-11-26T07:22:23.152146+0000","last_unstale":"2023-11-26T07:22:24.142990+0000","last_undegraded":"2023-11-26T07:22:24.142990+0000","last_fullsized":"2023-11-26T07:22:24.142990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:27.564861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.185563+0000","last_change":"2023-11-26T07:22:22.149321+0000","last_active":"2023-11-26T07:22:23.185563+0000","last_peered":"2023-11-26T07:22:23.185563+0000","last_clean":"2023-11-26T07:22:23.185563+0000","last_became_active":"2023-11-26T07:22:22.148856+0000","last_became_peered":"2023-11-26T07:22:22.148856+0000","last_unstale":"2023-11-26T07:22:23.185563+0000","last_undegraded":"2023-11-26T07:22:23.185563+0000","last_fullsized":"2023-11-26T07:22:23.185563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:14.438027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144240+0000","last_change":"2023-11-26T07:22:23.156120+0000","last_active":"2023-11-26T07:22:24.144240+0000","last_peered":"2023-11-26T07:22:24.144240+0000","last_clean":"2023-11-26T07:22:24.144240+0000","last_became_active":"2023-11-26T07:22:23.155837+0000","last_became_peered":"2023-11-26T07:22:23.155837+0000","last_unstale":"2023-11-26T07:22:24.144240+0000","last_undegraded":"2023-11-26T07:22:24.144240+0000","last_fullsized":"2023-11-26T07:22:24.144240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:55:01.934153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.183528+0000","last_change":"2023-11-26T07:22:22.150877+0000","last_active":"2023-11-26T07:22:23.183528+0000","last_peered":"2023-11-26T07:22:23.183528+0000","last_clean":"2023-11-26T07:22:23.183528+0000","last_became_active":"2023-11-26T07:22:22.150413+0000","last_became_peered":"2023-11-26T07:22:22.150413+0000","last_unstale":"2023-11-26T07:22:23.183528+0000","last_undegraded":"2023-11-26T07:22:23.183528+0000","last_fullsized":"2023-11-26T07:22:23.183528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:23:29.686183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143928+0000","last_change":"2023-11-26T07:22:23.164704+0000","last_active":"2023-11-26T07:22:24.143928+0000","last_peered":"2023-11-26T07:22:24.143928+0000","last_clean":"2023-11-26T07:22:24.143928+0000","last_became_active":"2023-11-26T07:22:23.164149+0000","last_became_peered":"2023-11-26T07:22:23.164149+0000","last_unstale":"2023-11-26T07:22:24.143928+0000","last_undegraded":"2023-11-26T07:22:24.143928+0000","last_fullsized":"2023-11-26T07:22:24.143928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:27.564861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.189503+0000","last_change":"2023-11-26T07:22:22.148977+0000","last_active":"2023-11-26T07:22:23.189503+0000","last_peered":"2023-11-26T07:22:23.189503+0000","last_clean":"2023-11-26T07:22:23.189503+0000","last_became_active":"2023-11-26T07:22:22.148703+0000","last_became_peered":"2023-11-26T07:22:22.148703+0000","last_unstale":"2023-11-26T07:22:23.189503+0000","last_undegraded":"2023-11-26T07:22:23.189503+0000","last_fullsized":"2023-11-26T07:22:23.189503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:16:13.167185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461894+0000","last_change":"2023-11-26T07:22:23.162066+0000","last_active":"2023-11-26T07:22:24.461894+0000","last_peered":"2023-11-26T07:22:24.461894+0000","last_clean":"2023-11-26T07:22:24.461894+0000","last_became_active":"2023-11-26T07:22:23.161712+0000","last_became_peered":"2023-11-26T07:22:23.161712+0000","last_unstale":"2023-11-26T07:22:24.461894+0000","last_undegraded":"2023-11-26T07:22:24.461894+0000","last_fullsized":"2023-11-26T07:22:24.461894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:27.564861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.187659+0000","last_change":"2023-11-26T07:22:22.151630+0000","last_active":"2023-11-26T07:22:23.187659+0000","last_peered":"2023-11-26T07:22:23.187659+0000","last_clean":"2023-11-26T07:22:23.187659+0000","last_became_active":"2023-11-26T07:22:22.151230+0000","last_became_peered":"2023-11-26T07:22:22.151230+0000","last_unstale":"2023-11-26T07:22:23.187659+0000","last_undegraded":"2023-11-26T07:22:23.187659+0000","last_fullsized":"2023-11-26T07:22:23.187659+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:23:29.686183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.463243+0000","last_change":"2023-11-26T07:22:23.163643+0000","last_active":"2023-11-26T07:22:24.463243+0000","last_peered":"2023-11-26T07:22:24.463243+0000","last_clean":"2023-11-26T07:22:24.463243+0000","last_became_active":"2023-11-26T07:22:23.161981+0000","last_became_peered":"2023-11-26T07:22:23.161981+0000","last_unstale":"2023-11-26T07:22:24.463243+0000","last_undegraded":"2023-11-26T07:22:24.463243+0000","last_fullsized":"2023-11-26T07:22:24.463243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:39.457900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.189320+0000","last_change":"2023-11-26T07:22:22.153463+0000","last_active":"2023-11-26T07:22:23.189320+0000","last_peered":"2023-11-26T07:22:23.189320+0000","last_clean":"2023-11-26T07:22:23.189320+0000","last_became_active":"2023-11-26T07:22:22.153132+0000","last_became_peered":"2023-11-26T07:22:22.153132+0000","last_unstale":"2023-11-26T07:22:23.189320+0000","last_undegraded":"2023-11-26T07:22:23.189320+0000","last_fullsized":"2023-11-26T07:22:23.189320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:30:47.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142663+0000","last_change":"2023-11-26T07:22:23.159636+0000","last_active":"2023-11-26T07:22:24.142663+0000","last_peered":"2023-11-26T07:22:24.142663+0000","last_clean":"2023-11-26T07:22:24.142663+0000","last_became_active":"2023-11-26T07:22:23.159101+0000","last_became_peered":"2023-11-26T07:22:23.159101+0000","last_unstale":"2023-11-26T07:22:24.142663+0000","last_undegraded":"2023-11-26T07:22:24.142663+0000","last_fullsized":"2023-11-26T07:22:24.142663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:53.313195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.183154+0000","last_change":"2023-11-26T07:22:22.148229+0000","last_active":"2023-11-26T07:22:23.183154+0000","last_peered":"2023-11-26T07:22:23.183154+0000","last_clean":"2023-11-26T07:22:23.183154+0000","last_became_active":"2023-11-26T07:22:22.147927+0000","last_became_peered":"2023-11-26T07:22:22.147927+0000","last_unstale":"2023-11-26T07:22:23.183154+0000","last_undegraded":"2023-11-26T07:22:23.183154+0000","last_fullsized":"2023-11-26T07:22:23.183154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:58:34.847257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461408+0000","last_change":"2023-11-26T07:22:23.167749+0000","last_active":"2023-11-26T07:22:24.461408+0000","last_peered":"2023-11-26T07:22:24.461408+0000","last_clean":"2023-11-26T07:22:24.461408+0000","last_became_active":"2023-11-26T07:22:23.167411+0000","last_became_peered":"2023-11-26T07:22:23.167411+0000","last_unstale":"2023-11-26T07:22:24.461408+0000","last_undegraded":"2023-11-26T07:22:24.461408+0000","last_fullsized":"2023-11-26T07:22:24.461408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:35:19.658117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.145891+0000","last_change":"2023-11-26T07:22:22.144638+0000","last_active":"2023-11-26T07:22:23.145891+0000","last_peered":"2023-11-26T07:22:23.145891+0000","last_clean":"2023-11-26T07:22:23.145891+0000","last_became_active":"2023-11-26T07:22:22.144230+0000","last_became_peered":"2023-11-26T07:22:22.144230+0000","last_unstale":"2023-11-26T07:22:23.145891+0000","last_undegraded":"2023-11-26T07:22:23.145891+0000","last_fullsized":"2023-11-26T07:22:23.145891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:17.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461813+0000","last_change":"2023-11-26T07:22:23.156605+0000","last_active":"2023-11-26T07:22:24.461813+0000","last_peered":"2023-11-26T07:22:24.461813+0000","last_clean":"2023-11-26T07:22:24.461813+0000","last_became_active":"2023-11-26T07:22:23.156291+0000","last_became_peered":"2023-11-26T07:22:23.156291+0000","last_unstale":"2023-11-26T07:22:24.461813+0000","last_undegraded":"2023-11-26T07:22:24.461813+0000","last_fullsized":"2023-11-26T07:22:24.461813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:33:28.090781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.149654+0000","last_change":"2023-11-26T07:22:22.153839+0000","last_active":"2023-11-26T07:22:23.149654+0000","last_peered":"2023-11-26T07:22:23.149654+0000","last_clean":"2023-11-26T07:22:23.149654+0000","last_became_active":"2023-11-26T07:22:22.153535+0000","last_became_peered":"2023-11-26T07:22:22.153535+0000","last_unstale":"2023-11-26T07:22:23.149654+0000","last_undegraded":"2023-11-26T07:22:23.149654+0000","last_fullsized":"2023-11-26T07:22:23.149654+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:23.734209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462173+0000","last_change":"2023-11-26T07:22:23.165238+0000","last_active":"2023-11-26T07:22:24.462173+0000","last_peered":"2023-11-26T07:22:24.462173+0000","last_clean":"2023-11-26T07:22:24.462173+0000","last_became_active":"2023-11-26T07:22:23.164949+0000","last_became_peered":"2023-11-26T07:22:23.164949+0000","last_unstale":"2023-11-26T07:22:24.462173+0000","last_undegraded":"2023-11-26T07:22:24.462173+0000","last_fullsized":"2023-11-26T07:22:24.462173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:29:04.949568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.141594+0000","last_change":"2023-11-26T07:22:22.140913+0000","last_active":"2023-11-26T07:22:23.141594+0000","last_peered":"2023-11-26T07:22:23.141594+0000","last_clean":"2023-11-26T07:22:23.141594+0000","last_became_active":"2023-11-26T07:22:22.140643+0000","last_became_peered":"2023-11-26T07:22:22.140643+0000","last_unstale":"2023-11-26T07:22:23.141594+0000","last_undegraded":"2023-11-26T07:22:23.141594+0000","last_fullsized":"2023-11-26T07:22:23.141594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:05.461443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462479+0000","last_change":"2023-11-26T07:22:23.162405+0000","last_active":"2023-11-26T07:22:24.462479+0000","last_peered":"2023-11-26T07:22:24.462479+0000","last_clean":"2023-11-26T07:22:24.462479+0000","last_became_active":"2023-11-26T07:22:23.161150+0000","last_became_peered":"2023-11-26T07:22:23.161150+0000","last_unstale":"2023-11-26T07:22:24.462479+0000","last_undegraded":"2023-11-26T07:22:24.462479+0000","last_fullsized":"2023-11-26T07:22:24.462479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:53.313195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.140267+0000","last_change":"2023-11-26T07:22:22.151535+0000","last_active":"2023-11-26T07:22:23.140267+0000","last_peered":"2023-11-26T07:22:23.140267+0000","last_clean":"2023-11-26T07:22:23.140267+0000","last_became_active":"2023-11-26T07:22:22.151097+0000","last_became_peered":"2023-11-26T07:22:22.151097+0000","last_unstale":"2023-11-26T07:22:23.140267+0000","last_undegraded":"2023-11-26T07:22:23.140267+0000","last_fullsized":"2023-11-26T07:22:23.140267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:17.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461610+0000","last_change":"2023-11-26T07:22:23.158807+0000","last_active":"2023-11-26T07:22:24.461610+0000","last_peered":"2023-11-26T07:22:24.461610+0000","last_clean":"2023-11-26T07:22:24.461610+0000","last_became_active":"2023-11-26T07:22:23.158510+0000","last_became_peered":"2023-11-26T07:22:23.158510+0000","last_unstale":"2023-11-26T07:22:24.461610+0000","last_undegraded":"2023-11-26T07:22:24.461610+0000","last_fullsized":"2023-11-26T07:22:24.461610+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:53.313195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.188560+0000","last_change":"2023-11-26T07:22:22.149767+0000","last_active":"2023-11-26T07:22:23.188560+0000","last_peered":"2023-11-26T07:22:23.188560+0000","last_clean":"2023-11-26T07:22:23.188560+0000","last_became_active":"2023-11-26T07:22:22.149435+0000","last_became_peered":"2023-11-26T07:22:22.149435+0000","last_unstale":"2023-11-26T07:22:23.188560+0000","last_undegraded":"2023-11-26T07:22:23.188560+0000","last_fullsized":"2023-11-26T07:22:23.188560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:58:34.847257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142780+0000","last_change":"2023-11-26T07:22:23.160003+0000","last_active":"2023-11-26T07:22:24.142780+0000","last_peered":"2023-11-26T07:22:24.142780+0000","last_clean":"2023-11-26T07:22:24.142780+0000","last_became_active":"2023-11-26T07:22:23.159479+0000","last_became_peered":"2023-11-26T07:22:23.159479+0000","last_unstale":"2023-11-26T07:22:24.142780+0000","last_undegraded":"2023-11-26T07:22:24.142780+0000","last_fullsized":"2023-11-26T07:22:24.142780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:35:19.658117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.191377+0000","last_change":"2023-11-26T07:22:22.138119+0000","last_active":"2023-11-26T07:22:23.191377+0000","last_peered":"2023-11-26T07:22:23.191377+0000","last_clean":"2023-11-26T07:22:23.191377+0000","last_became_active":"2023-11-26T07:22:22.137770+0000","last_became_peered":"2023-11-26T07:22:22.137770+0000","last_unstale":"2023-11-26T07:22:23.191377+0000","last_undegraded":"2023-11-26T07:22:23.191377+0000","last_fullsized":"2023-11-26T07:22:23.191377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:30:47.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461469+0000","last_change":"2023-11-26T07:22:23.158226+0000","last_active":"2023-11-26T07:22:24.461469+0000","last_peered":"2023-11-26T07:22:24.461469+0000","last_clean":"2023-11-26T07:22:24.461469+0000","last_became_active":"2023-11-26T07:22:23.157937+0000","last_became_peered":"2023-11-26T07:22:23.157937+0000","last_unstale":"2023-11-26T07:22:24.461469+0000","last_undegraded":"2023-11-26T07:22:24.461469+0000","last_fullsized":"2023-11-26T07:22:24.461469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:06.468579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.189651+0000","last_change":"2023-11-26T07:22:22.145272+0000","last_active":"2023-11-26T07:22:23.189651+0000","last_peered":"2023-11-26T07:22:23.189651+0000","last_clean":"2023-11-26T07:22:23.189651+0000","last_became_active":"2023-11-26T07:22:22.145011+0000","last_became_peered":"2023-11-26T07:22:22.145011+0000","last_unstale":"2023-11-26T07:22:23.189651+0000","last_undegraded":"2023-11-26T07:22:23.189651+0000","last_fullsized":"2023-11-26T07:22:23.189651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:58:46.089931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461932+0000","last_change":"2023-11-26T07:22:23.168174+0000","last_active":"2023-11-26T07:22:24.461932+0000","last_peered":"2023-11-26T07:22:24.461932+0000","last_clean":"2023-11-26T07:22:24.461932+0000","last_became_active":"2023-11-26T07:22:23.167943+0000","last_became_peered":"2023-11-26T07:22:23.167943+0000","last_unstale":"2023-11-26T07:22:24.461932+0000","last_undegraded":"2023-11-26T07:22:24.461932+0000","last_fullsized":"2023-11-26T07:22:24.461932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:03:00.736312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.140195+0000","last_change":"2023-11-26T07:22:22.149360+0000","last_active":"2023-11-26T07:22:23.140195+0000","last_peered":"2023-11-26T07:22:23.140195+0000","last_clean":"2023-11-26T07:22:23.140195+0000","last_became_active":"2023-11-26T07:22:22.149068+0000","last_became_peered":"2023-11-26T07:22:22.149068+0000","last_unstale":"2023-11-26T07:22:23.140195+0000","last_undegraded":"2023-11-26T07:22:23.140195+0000","last_fullsized":"2023-11-26T07:22:23.140195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:47:50.273842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462810+0000","last_change":"2023-11-26T07:22:23.163332+0000","last_active":"2023-11-26T07:22:24.462810+0000","last_peered":"2023-11-26T07:22:24.462810+0000","last_clean":"2023-11-26T07:22:24.462810+0000","last_became_active":"2023-11-26T07:22:23.163063+0000","last_became_peered":"2023-11-26T07:22:23.163063+0000","last_unstale":"2023-11-26T07:22:24.462810+0000","last_undegraded":"2023-11-26T07:22:24.462810+0000","last_fullsized":"2023-11-26T07:22:24.462810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:42.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.147364+0000","last_change":"2023-11-26T07:22:22.151868+0000","last_active":"2023-11-26T07:22:23.147364+0000","last_peered":"2023-11-26T07:22:23.147364+0000","last_clean":"2023-11-26T07:22:23.147364+0000","last_became_active":"2023-11-26T07:22:22.151535+0000","last_became_peered":"2023-11-26T07:22:22.151535+0000","last_unstale":"2023-11-26T07:22:23.147364+0000","last_undegraded":"2023-11-26T07:22:23.147364+0000","last_fullsized":"2023-11-26T07:22:23.147364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:09:43.742463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462578+0000","last_change":"2023-11-26T07:22:23.156407+0000","last_active":"2023-11-26T07:22:24.462578+0000","last_peered":"2023-11-26T07:22:24.462578+0000","last_clean":"2023-11-26T07:22:24.462578+0000","last_became_active":"2023-11-26T07:22:23.156110+0000","last_became_peered":"2023-11-26T07:22:23.156110+0000","last_unstale":"2023-11-26T07:22:24.462578+0000","last_undegraded":"2023-11-26T07:22:24.462578+0000","last_fullsized":"2023-11-26T07:22:24.462578+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:22.241284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.189422+0000","last_change":"2023-11-26T07:22:22.145842+0000","last_active":"2023-11-26T07:22:23.189422+0000","last_peered":"2023-11-26T07:22:23.189422+0000","last_clean":"2023-11-26T07:22:23.189422+0000","last_became_active":"2023-11-26T07:22:22.145573+0000","last_became_peered":"2023-11-26T07:22:22.145573+0000","last_unstale":"2023-11-26T07:22:23.189422+0000","last_undegraded":"2023-11-26T07:22:23.189422+0000","last_fullsized":"2023-11-26T07:22:23.189422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:30:47.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462132+0000","last_change":"2023-11-26T07:22:23.162197+0000","last_active":"2023-11-26T07:22:24.462132+0000","last_peered":"2023-11-26T07:22:24.462132+0000","last_clean":"2023-11-26T07:22:24.462132+0000","last_became_active":"2023-11-26T07:22:23.160525+0000","last_became_peered":"2023-11-26T07:22:23.160525+0000","last_unstale":"2023-11-26T07:22:24.462132+0000","last_undegraded":"2023-11-26T07:22:24.462132+0000","last_fullsized":"2023-11-26T07:22:24.462132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:13:00.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.185964+0000","last_change":"2023-11-26T07:22:22.140377+0000","last_active":"2023-11-26T07:22:23.185964+0000","last_peered":"2023-11-26T07:22:23.185964+0000","last_clean":"2023-11-26T07:22:23.185964+0000","last_became_active":"2023-11-26T07:22:22.138993+0000","last_became_peered":"2023-11-26T07:22:22.138993+0000","last_unstale":"2023-11-26T07:22:23.185964+0000","last_undegraded":"2023-11-26T07:22:23.185964+0000","last_fullsized":"2023-11-26T07:22:23.185964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:52.306059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144297+0000","last_change":"2023-11-26T07:22:23.158016+0000","last_active":"2023-11-26T07:22:24.144297+0000","last_peered":"2023-11-26T07:22:24.144297+0000","last_clean":"2023-11-26T07:22:24.144297+0000","last_became_active":"2023-11-26T07:22:23.157356+0000","last_became_peered":"2023-11-26T07:22:23.157356+0000","last_unstale":"2023-11-26T07:22:24.144297+0000","last_undegraded":"2023-11-26T07:22:24.144297+0000","last_fullsized":"2023-11-26T07:22:24.144297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:47:51.280978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.189131+0000","last_change":"2023-11-26T07:22:22.152564+0000","last_active":"2023-11-26T07:22:23.189131+0000","last_peered":"2023-11-26T07:22:23.189131+0000","last_clean":"2023-11-26T07:22:23.189131+0000","last_became_active":"2023-11-26T07:22:22.152161+0000","last_became_peered":"2023-11-26T07:22:22.152161+0000","last_unstale":"2023-11-26T07:22:23.189131+0000","last_undegraded":"2023-11-26T07:22:23.189131+0000","last_fullsized":"2023-11-26T07:22:23.189131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:14.438027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143085+0000","last_change":"2023-11-26T07:22:23.166236+0000","last_active":"2023-11-26T07:22:24.143085+0000","last_peered":"2023-11-26T07:22:24.143085+0000","last_clean":"2023-11-26T07:22:24.143085+0000","last_became_active":"2023-11-26T07:22:23.165942+0000","last_became_peered":"2023-11-26T07:22:23.165942+0000","last_unstale":"2023-11-26T07:22:24.143085+0000","last_undegraded":"2023-11-26T07:22:24.143085+0000","last_fullsized":"2023-11-26T07:22:24.143085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:03:00.736312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.148063+0000","last_change":"2023-11-26T07:22:22.145038+0000","last_active":"2023-11-26T07:22:23.148063+0000","last_peered":"2023-11-26T07:22:23.148063+0000","last_clean":"2023-11-26T07:22:23.148063+0000","last_became_active":"2023-11-26T07:22:22.144754+0000","last_became_peered":"2023-11-26T07:22:22.144754+0000","last_unstale":"2023-11-26T07:22:23.148063+0000","last_undegraded":"2023-11-26T07:22:23.148063+0000","last_fullsized":"2023-11-26T07:22:23.148063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:33:39.670858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140079+0000","last_change":"2023-11-26T07:22:23.166017+0000","last_active":"2023-11-26T07:22:24.140079+0000","last_peered":"2023-11-26T07:22:24.140079+0000","last_clean":"2023-11-26T07:22:24.140079+0000","last_became_active":"2023-11-26T07:22:23.165709+0000","last_became_peered":"2023-11-26T07:22:23.165709+0000","last_unstale":"2023-11-26T07:22:24.140079+0000","last_undegraded":"2023-11-26T07:22:24.140079+0000","last_fullsized":"2023-11-26T07:22:24.140079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:54:34.684641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.191492+0000","last_change":"2023-11-26T07:22:22.140578+0000","last_active":"2023-11-26T07:22:23.191492+0000","last_peered":"2023-11-26T07:22:23.191492+0000","last_clean":"2023-11-26T07:22:23.191492+0000","last_became_active":"2023-11-26T07:22:22.140222+0000","last_became_peered":"2023-11-26T07:22:22.140222+0000","last_unstale":"2023-11-26T07:22:23.191492+0000","last_undegraded":"2023-11-26T07:22:23.191492+0000","last_fullsized":"2023-11-26T07:22:23.191492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:55:00.927017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461820+0000","last_change":"2023-11-26T07:22:23.155279+0000","last_active":"2023-11-26T07:22:24.461820+0000","last_peered":"2023-11-26T07:22:24.461820+0000","last_clean":"2023-11-26T07:22:24.461820+0000","last_became_active":"2023-11-26T07:22:23.154787+0000","last_became_peered":"2023-11-26T07:22:23.154787+0000","last_unstale":"2023-11-26T07:22:24.461820+0000","last_undegraded":"2023-11-26T07:22:24.461820+0000","last_fullsized":"2023-11-26T07:22:24.461820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:16:14.174321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.151818+0000","last_change":"2023-11-26T07:22:22.147493+0000","last_active":"2023-11-26T07:22:23.151818+0000","last_peered":"2023-11-26T07:22:23.151818+0000","last_clean":"2023-11-26T07:22:23.151818+0000","last_became_active":"2023-11-26T07:22:22.147187+0000","last_became_peered":"2023-11-26T07:22:22.147187+0000","last_unstale":"2023-11-26T07:22:23.151818+0000","last_undegraded":"2023-11-26T07:22:23.151818+0000","last_fullsized":"2023-11-26T07:22:23.151818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:07:34.009756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.139893+0000","last_change":"2023-11-26T07:22:23.153837+0000","last_active":"2023-11-26T07:22:24.139893+0000","last_peered":"2023-11-26T07:22:24.139893+0000","last_clean":"2023-11-26T07:22:24.139893+0000","last_became_active":"2023-11-26T07:22:23.153537+0000","last_became_peered":"2023-11-26T07:22:23.153537+0000","last_unstale":"2023-11-26T07:22:24.139893+0000","last_undegraded":"2023-11-26T07:22:24.139893+0000","last_fullsized":"2023-11-26T07:22:24.139893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:17.005822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.153966+0000","last_change":"2023-11-26T07:22:22.148994+0000","last_active":"2023-11-26T07:22:23.153966+0000","last_peered":"2023-11-26T07:22:23.153966+0000","last_clean":"2023-11-26T07:22:23.153966+0000","last_became_active":"2023-11-26T07:22:22.147943+0000","last_became_peered":"2023-11-26T07:22:22.147943+0000","last_unstale":"2023-11-26T07:22:23.153966+0000","last_undegraded":"2023-11-26T07:22:23.153966+0000","last_fullsized":"2023-11-26T07:22:23.153966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:47:08.727890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461752+0000","last_change":"2023-11-26T07:22:23.160348+0000","last_active":"2023-11-26T07:22:24.461752+0000","last_peered":"2023-11-26T07:22:24.461752+0000","last_clean":"2023-11-26T07:22:24.461752+0000","last_became_active":"2023-11-26T07:22:23.159979+0000","last_became_peered":"2023-11-26T07:22:23.159979+0000","last_unstale":"2023-11-26T07:22:24.461752+0000","last_undegraded":"2023-11-26T07:22:24.461752+0000","last_fullsized":"2023-11-26T07:22:24.461752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:33:40.677994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.140309+0000","last_change":"2023-11-26T07:22:22.139465+0000","last_active":"2023-11-26T07:22:23.140309+0000","last_peered":"2023-11-26T07:22:23.140309+0000","last_clean":"2023-11-26T07:22:23.140309+0000","last_became_active":"2023-11-26T07:22:22.139180+0000","last_became_peered":"2023-11-26T07:22:22.139180+0000","last_unstale":"2023-11-26T07:22:23.140309+0000","last_undegraded":"2023-11-26T07:22:23.140309+0000","last_fullsized":"2023-11-26T07:22:23.140309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:19:47.928556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143222+0000","last_change":"2023-11-26T07:22:23.157400+0000","last_active":"2023-11-26T07:22:24.143222+0000","last_peered":"2023-11-26T07:22:24.143222+0000","last_clean":"2023-11-26T07:22:24.143222+0000","last_became_active":"2023-11-26T07:22:23.156768+0000","last_became_peered":"2023-11-26T07:22:23.156768+0000","last_unstale":"2023-11-26T07:22:24.143222+0000","last_undegraded":"2023-11-26T07:22:24.143222+0000","last_fullsized":"2023-11-26T07:22:24.143222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:07.939507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.188783+0000","last_change":"2023-11-26T07:22:22.146130+0000","last_active":"2023-11-26T07:22:23.188783+0000","last_peered":"2023-11-26T07:22:23.188783+0000","last_clean":"2023-11-26T07:22:23.188783+0000","last_became_active":"2023-11-26T07:22:22.144230+0000","last_became_peered":"2023-11-26T07:22:22.144230+0000","last_unstale":"2023-11-26T07:22:23.188783+0000","last_undegraded":"2023-11-26T07:22:23.188783+0000","last_fullsized":"2023-11-26T07:22:23.188783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:21:24.263764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462231+0000","last_change":"2023-11-26T07:22:23.161644+0000","last_active":"2023-11-26T07:22:24.462231+0000","last_peered":"2023-11-26T07:22:24.462231+0000","last_clean":"2023-11-26T07:22:24.462231+0000","last_became_active":"2023-11-26T07:22:23.161336+0000","last_became_peered":"2023-11-26T07:22:23.161336+0000","last_unstale":"2023-11-26T07:22:24.462231+0000","last_undegraded":"2023-11-26T07:22:24.462231+0000","last_fullsized":"2023-11-26T07:22:24.462231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:55:01.934153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.152180+0000","last_change":"2023-11-26T07:22:22.145175+0000","last_active":"2023-11-26T07:22:23.152180+0000","last_peered":"2023-11-26T07:22:23.152180+0000","last_clean":"2023-11-26T07:22:23.152180+0000","last_became_active":"2023-11-26T07:22:22.144917+0000","last_became_peered":"2023-11-26T07:22:22.144917+0000","last_unstale":"2023-11-26T07:22:23.152180+0000","last_undegraded":"2023-11-26T07:22:23.152180+0000","last_fullsized":"2023-11-26T07:22:23.152180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:57.303446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144077+0000","last_change":"2023-11-26T07:22:23.158894+0000","last_active":"2023-11-26T07:22:24.144077+0000","last_peered":"2023-11-26T07:22:24.144077+0000","last_clean":"2023-11-26T07:22:24.144077+0000","last_became_active":"2023-11-26T07:22:23.158534+0000","last_became_peered":"2023-11-26T07:22:23.158534+0000","last_unstale":"2023-11-26T07:22:24.144077+0000","last_undegraded":"2023-11-26T07:22:24.144077+0000","last_fullsized":"2023-11-26T07:22:24.144077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:03:00.736312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.140267+0000","last_change":"2023-11-26T07:22:22.143708+0000","last_active":"2023-11-26T07:22:23.140267+0000","last_peered":"2023-11-26T07:22:23.140267+0000","last_clean":"2023-11-26T07:22:23.140267+0000","last_became_active":"2023-11-26T07:22:22.143395+0000","last_became_peered":"2023-11-26T07:22:22.143395+0000","last_unstale":"2023-11-26T07:22:23.140267+0000","last_undegraded":"2023-11-26T07:22:23.140267+0000","last_fullsized":"2023-11-26T07:22:23.140267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:09:43.742463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142964+0000","last_change":"2023-11-26T07:22:23.159788+0000","last_active":"2023-11-26T07:22:24.142964+0000","last_peered":"2023-11-26T07:22:24.142964+0000","last_clean":"2023-11-26T07:22:24.142964+0000","last_became_active":"2023-11-26T07:22:23.159061+0000","last_became_peered":"2023-11-26T07:22:23.159061+0000","last_unstale":"2023-11-26T07:22:24.142964+0000","last_undegraded":"2023-11-26T07:22:24.142964+0000","last_fullsized":"2023-11-26T07:22:24.142964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:31.249944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.140932+0000","last_change":"2023-11-26T07:22:22.139927+0000","last_active":"2023-11-26T07:22:23.140932+0000","last_peered":"2023-11-26T07:22:23.140932+0000","last_clean":"2023-11-26T07:22:23.140932+0000","last_became_active":"2023-11-26T07:22:22.139628+0000","last_became_peered":"2023-11-26T07:22:22.139628+0000","last_unstale":"2023-11-26T07:22:23.140932+0000","last_undegraded":"2023-11-26T07:22:23.140932+0000","last_fullsized":"2023-11-26T07:22:23.140932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:23:29.686183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461832+0000","last_change":"2023-11-26T07:22:23.152817+0000","last_active":"2023-11-26T07:22:24.461832+0000","last_peered":"2023-11-26T07:22:24.461832+0000","last_clean":"2023-11-26T07:22:24.461832+0000","last_became_active":"2023-11-26T07:22:23.151586+0000","last_became_peered":"2023-11-26T07:22:23.151586+0000","last_unstale":"2023-11-26T07:22:24.461832+0000","last_undegraded":"2023-11-26T07:22:24.461832+0000","last_fullsized":"2023-11-26T07:22:24.461832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:15.445163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.145784+0000","last_change":"2023-11-26T07:22:22.146618+0000","last_active":"2023-11-26T07:22:23.145784+0000","last_peered":"2023-11-26T07:22:23.145784+0000","last_clean":"2023-11-26T07:22:23.145784+0000","last_became_active":"2023-11-26T07:22:22.146214+0000","last_became_peered":"2023-11-26T07:22:22.146214+0000","last_unstale":"2023-11-26T07:22:23.145784+0000","last_undegraded":"2023-11-26T07:22:23.145784+0000","last_fullsized":"2023-11-26T07:22:23.145784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:58:34.847257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461683+0000","last_change":"2023-11-26T07:22:23.158035+0000","last_active":"2023-11-26T07:22:24.461683+0000","last_peered":"2023-11-26T07:22:24.461683+0000","last_clean":"2023-11-26T07:22:24.461683+0000","last_became_active":"2023-11-26T07:22:23.157737+0000","last_became_peered":"2023-11-26T07:22:23.157737+0000","last_unstale":"2023-11-26T07:22:24.461683+0000","last_undegraded":"2023-11-26T07:22:24.461683+0000","last_fullsized":"2023-11-26T07:22:24.461683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:55.059095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.141972+0000","last_change":"2023-11-26T07:22:22.148139+0000","last_active":"2023-11-26T07:22:23.141972+0000","last_peered":"2023-11-26T07:22:23.141972+0000","last_clean":"2023-11-26T07:22:23.141972+0000","last_became_active":"2023-11-26T07:22:22.147844+0000","last_became_peered":"2023-11-26T07:22:22.147844+0000","last_unstale":"2023-11-26T07:22:23.141972+0000","last_undegraded":"2023-11-26T07:22:23.141972+0000","last_fullsized":"2023-11-26T07:22:23.141972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:17.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461497+0000","last_change":"2023-11-26T07:22:23.169021+0000","last_active":"2023-11-26T07:22:24.461497+0000","last_peered":"2023-11-26T07:22:24.461497+0000","last_clean":"2023-11-26T07:22:24.461497+0000","last_became_active":"2023-11-26T07:22:23.168529+0000","last_became_peered":"2023-11-26T07:22:23.168529+0000","last_unstale":"2023-11-26T07:22:24.461497+0000","last_undegraded":"2023-11-26T07:22:24.461497+0000","last_fullsized":"2023-11-26T07:22:24.461497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:26:35.758938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.139641+0000","last_change":"2023-11-26T07:22:22.140709+0000","last_active":"2023-11-26T07:22:23.139641+0000","last_peered":"2023-11-26T07:22:23.139641+0000","last_clean":"2023-11-26T07:22:23.139641+0000","last_became_active":"2023-11-26T07:22:22.140379+0000","last_became_peered":"2023-11-26T07:22:22.140379+0000","last_unstale":"2023-11-26T07:22:23.139641+0000","last_undegraded":"2023-11-26T07:22:23.139641+0000","last_fullsized":"2023-11-26T07:22:23.139641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:22.927191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143724+0000","last_change":"2023-11-26T07:22:23.163722+0000","last_active":"2023-11-26T07:22:24.143724+0000","last_peered":"2023-11-26T07:22:24.143724+0000","last_clean":"2023-11-26T07:22:24.143724+0000","last_became_active":"2023-11-26T07:22:23.163046+0000","last_became_peered":"2023-11-26T07:22:23.163046+0000","last_unstale":"2023-11-26T07:22:24.143724+0000","last_undegraded":"2023-11-26T07:22:24.143724+0000","last_fullsized":"2023-11-26T07:22:24.143724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:35:19.658117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.142323+0000","last_change":"2023-11-26T07:22:22.148901+0000","last_active":"2023-11-26T07:22:23.142323+0000","last_peered":"2023-11-26T07:22:23.142323+0000","last_clean":"2023-11-26T07:22:23.142323+0000","last_became_active":"2023-11-26T07:22:22.148566+0000","last_became_peered":"2023-11-26T07:22:22.148566+0000","last_unstale":"2023-11-26T07:22:23.142323+0000","last_undegraded":"2023-11-26T07:22:23.142323+0000","last_fullsized":"2023-11-26T07:22:23.142323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:09:43.742463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462873+0000","last_change":"2023-11-26T07:22:23.163774+0000","last_active":"2023-11-26T07:22:24.462873+0000","last_peered":"2023-11-26T07:22:24.462873+0000","last_clean":"2023-11-26T07:22:24.462873+0000","last_became_active":"2023-11-26T07:22:23.161329+0000","last_became_peered":"2023-11-26T07:22:23.161329+0000","last_unstale":"2023-11-26T07:22:24.462873+0000","last_undegraded":"2023-11-26T07:22:24.462873+0000","last_fullsized":"2023-11-26T07:22:24.462873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:50.539193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.188629+0000","last_change":"2023-11-26T07:22:22.152219+0000","last_active":"2023-11-26T07:22:23.188629+0000","last_peered":"2023-11-26T07:22:23.188629+0000","last_clean":"2023-11-26T07:22:23.188629+0000","last_became_active":"2023-11-26T07:22:22.151927+0000","last_became_peered":"2023-11-26T07:22:22.151927+0000","last_unstale":"2023-11-26T07:22:23.188629+0000","last_undegraded":"2023-11-26T07:22:23.188629+0000","last_fullsized":"2023-11-26T07:22:23.188629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:33:39.670858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"69'128","reported_seq":253,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.150517+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:22:23.150517+0000","last_peered":"2023-11-26T07:22:23.150517+0000","last_clean":"2023-11-26T07:22:23.150517+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:22:23.150517+0000","last_undegraded":"2023-11-26T07:22:23.150517+0000","last_fullsized":"2023-11-26T07:22:23.150517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1581056,"data_stored":1011560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.56599999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":0.995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84799999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68000000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.48499999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.92500000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71399999999999997},"min":{"1min":0.56899999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84499999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.58999999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92500000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.50600000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":1.016,"15min":1.016},"last":0.64700000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.85099999999999998,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.625},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":0.81899999999999995},"min":{"1min":0.68100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93500000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.80400000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.71099999999999997},"min":{"1min":0.47199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.49199999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73299999999999998,"15min":0.77500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95899999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.84899999999999998},"min":{"1min":0.58699999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91500000000000004,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.499,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93400000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.82999999999999996},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.012,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":421816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":0.86699999999999999,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.222000000000001,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":3.4020000000000001,"5min":1.246,"15min":0.91500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":54.737000000000002,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.67300000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.40400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.1880000000000002,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.6689999999999996,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.56699999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.439,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.8479999999999999,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.72499999999999998,"15min":0.68899999999999995},"min":{"1min":0.57099999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0170000000000003,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.71899999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.81299999999999994,"15min":0.751},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.4870000000000001,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.601,"5min":0.94399999999999995,"15min":0.874},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":18.77,"5min":18.77,"15min":18.77},"last":0.77300000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.53100000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.121,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67400000000000004,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.82999999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.77900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.74199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":422756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.55800000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.97799999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.778},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.143}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.68600000000000005},"min":{"1min":0.58099999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":1.081},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67800000000000005,"15min":0.69199999999999995},"min":{"1min":0.45200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.823}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":1.119}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64700000000000002,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.105,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.105},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.77100000000000002},"min":{"1min":0.65500000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.3360000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.625,"15min":0.67200000000000004},"min":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.9059999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.268}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.755,"15min":0.76000000000000001},"min":{"1min":0.39900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":2.0830000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.754,"15min":0.80700000000000005},"min":{"1min":0.48499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.861}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":983040,"data_stored":421122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84899999999999998,"5min":1.016,"15min":1.016},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93100000000000005,"5min":1.26,"15min":1.26},"last":0.74399999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90200000000000002,"5min":1.077,"15min":1.077},"last":0.67800000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.302,"15min":1.302},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.753},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":1.181,"15min":1.181},"last":0.52500000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.747},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97999999999999998,"5min":1.702,"15min":1.702},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.78700000000000003},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0409999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78200000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.72999999999999998},"min":{"1min":0.42199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97099999999999997,"5min":1.129,"15min":1.129},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.78800000000000003},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.012,"15min":1.012},"last":0.71999999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88300000000000001,"5min":1.016,"15min":1.016},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.81200000000000006},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.0980000000000001,"5min":1.242,"15min":1.242},"last":0.76600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":422734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.68999999999999995,"5min":1.294,"15min":1.294},"last":0.44}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":1.456,"15min":1.456},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.48999999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.52900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.212,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2649999999999999,"5min":1.395,"15min":1.395},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1080000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.623}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.64700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.333,"15min":1.333},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.228,"5min":1.347,"15min":1.347},"last":0.753}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.397,"15min":1.397},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1859999999999999,"5min":1.363,"15min":1.363},"last":0.64400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":987136,"data_stored":421704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56000000000000005,"15min":0.64800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65800000000000003,"5min":3.931,"15min":3.931},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.71099999999999997},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.44}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60199999999999998,"15min":0.69199999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95499999999999996,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.69299999999999995},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.7650000000000001,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.49299999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.751,"15min":0.90300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98899999999999999,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.85199999999999998,"15min":0.92900000000000005},"min":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1240000000000001,"5min":4.431,"15min":4.431},"last":0.621},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.274,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.73399999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.61199999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89400000000000002,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8069999999999999,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.98399999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.222,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.753}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.93899999999999995,"15min":0.995},"min":{"1min":0.621,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.4740000000000002,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.052,"5min":1.609,"15min":1.609},"last":0.63500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":421316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.56499999999999995,"5min":1.034,"15min":1.034},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.41399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68999999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.43099999999999999,"15min":0.45100000000000001},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65200000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64400000000000002,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54900000000000004,"15min":0.57899999999999996},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68000000000000005,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.76600000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.52100000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0860000000000001,"5min":1.143,"15min":1.143},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0449999999999999,"5min":1.216,"15min":1.216},"last":0.872}]},{"osd":5,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.171,"5min":1.262,"15min":1.262},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.746,"15min":0.748},"min":{"1min":0.57099999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.343,"5min":1.54,"15min":1.54},"last":0.81599999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76700000000000002,"15min":0.73899999999999999},"min":{"1min":0.59699999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.77900000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.254,"5min":1.333,"15min":1.333},"last":0.90400000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.751,"15min":0.75900000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2090000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.751,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.85299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":1010505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.43099999999999999,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.57899999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.47999999999999998,"15min":0.47599999999999998},"min":{"1min":0.35599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.63800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56399999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.622,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48799999999999999,"15min":0.47799999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73799999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.73799999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.56299999999999994,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0289999999999999,"5min":1.365,"15min":1.365},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.80900000000000005},"min":{"1min":0.51300000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0980000000000001,"5min":1.181,"15min":1.181},"last":0.93300000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90300000000000002,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.76900000000000002},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.147,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.78500000000000003,"15min":0.80300000000000005},"min":{"1min":0.56100000000000005,"5min":0.503,"15min":0.503},"max":{"1min":0.91700000000000004,"5min":1.323,"15min":1.323},"last":0.71299999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.089,"5min":1.123,"15min":1.123},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.82799999999999996,"15min":0.81799999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.111,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.88400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:22:30.674 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:22:30.679 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:22:31.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:22:32.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.009+0000 7fa67f92a640 1 Processor -- start 2023-11-26T07:22:32.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.009+0000 7fa67f92a640 1 -- start start 2023-11-26T07:22:32.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67f92a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa6780b39c0 0x7fa67806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67f92a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa67806cb70 0x7fa678070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67f92a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67806cf90 con 0x7fa67815f540 2023-11-26T07:22:32.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67f92a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6780b2650 con 0x7fa6780b39c0 2023-11-26T07:22:32.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67f92a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6780b2920 con 0x7fa67806cb70 2023-11-26T07:22:32.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67d6c6640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa67815f540 0x7fa6780b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60512/0 (socket says 172.21.15.31:60512) 2023-11-26T07:22:32.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67d6c6640 1 -- 172.21.15.31:0/1031303451 learned_addr learned my addr 172.21.15.31:0/1031303451 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:32.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67cec5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa6780b39c0 0x7fa67806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67dec7640 1 --2- 172.21.15.31:0/1031303451 >> v2:172.21.15.97:3300/0 conn(0x7fa67806cb70 0x7fa678070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa66ffff640 1 -- 172.21.15.31:0/1031303451 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998494622 0 0) 0x7fa67806cf90 con 0x7fa67815f540 2023-11-26T07:22:32.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa66ffff640 1 -- 172.21.15.31:0/1031303451 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa664003530 con 0x7fa67815f540 2023-11-26T07:22:32.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67cec5640 1 -- 172.21.15.31:0/1031303451 >> v2:172.21.15.97:3300/0 conn(0x7fa67806cb70 msgr2=0x7fa678070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:32.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67cec5640 1 --2- 172.21.15.31:0/1031303451 >> v2:172.21.15.97:3300/0 conn(0x7fa67806cb70 0x7fa678070880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67cec5640 1 -- 172.21.15.31:0/1031303451 >> v1:172.21.15.31:6789/0 conn(0x7fa67815f540 legacy=0x7fa6780b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67cec5640 1 -- 172.21.15.31:0/1031303451 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780b2c00 con 0x7fa6780b39c0 2023-11-26T07:22:32.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.013+0000 7fa67dec7640 1 --2- 172.21.15.31:0/1031303451 >> v2:172.21.15.97:3300/0 conn(0x7fa67806cb70 0x7fa678070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:32.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67cec5640 1 --2- 172.21.15.31:0/1031303451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6780b39c0 0x7fa67806c630 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa67809d240 tx=0x7fa66801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ff7f0061d8a606 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa66ffff640 1 -- 172.21.15.31:0/1031303451 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680ac600 con 0x7fa6780b39c0 2023-11-26T07:22:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa66ffff640 1 -- 172.21.15.31:0/1031303451 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6680a8050 con 0x7fa6780b39c0 2023-11-26T07:22:32.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa66ffff640 1 -- 172.21.15.31:0/1031303451 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680ac980 con 0x7fa6780b39c0 2023-11-26T07:22:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67f92a640 1 -- 172.21.15.31:0/1031303451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6780b39c0 msgr2=0x7fa67806c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67f92a640 1 --2- 172.21.15.31:0/1031303451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6780b39c0 0x7fa67806c630 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa67809d240 tx=0x7fa66801c3a0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67f92a640 1 -- 172.21.15.31:0/1031303451 shutdown_connections 2023-11-26T07:22:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67f92a640 1 --2- 172.21.15.31:0/1031303451 >> v2:172.21.15.97:3300/0 conn(0x7fa67806cb70 0x7fa678070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67f92a640 1 --2- 172.21.15.31:0/1031303451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6780b39c0 0x7fa67806c630 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67f92a640 1 -- 172.21.15.31:0/1031303451 >> 172.21.15.31:0/1031303451 conn(0x7fa6780b0c40 msgr2=0x7fa6780b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67f92a640 1 -- 172.21.15.31:0/1031303451 shutdown_connections 2023-11-26T07:22:32.020 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67f92a640 1 -- 172.21.15.31:0/1031303451 wait complete. 2023-11-26T07:22:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67f92a640 1 Processor -- start 2023-11-26T07:22:32.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.017+0000 7fa67f92a640 1 -- start start 2023-11-26T07:22:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67f92a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa67806cb70 0x7fa678154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67f92a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6780b39c0 0x7fa6780ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67f92a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa67815f540 0x7fa678157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67f92a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa678154840 con 0x7fa67815f540 2023-11-26T07:22:32.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67f92a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780ad380 con 0x7fa6780b39c0 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67f92a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780ad660 con 0x7fa67806cb70 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67dec7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa67815f540 0x7fa678157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67dec7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa67815f540 0x7fa678157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37978/0 (socket says 172.21.15.31:37978) 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67dec7640 1 -- 172.21.15.31:0/3426692532 learned_addr learned my addr 172.21.15.31:0/3426692532 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67cec5640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6780b39c0 0x7fa6780ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67d6c6640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa67806cb70 0x7fa678154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67dec7640 1 -- 172.21.15.31:0/3426692532 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa67806cb70 msgr2=0x7fa678154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67dec7640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa67806cb70 0x7fa678154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67dec7640 1 -- 172.21.15.31:0/3426692532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6780b39c0 msgr2=0x7fa6780ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67dec7640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6780b39c0 0x7fa6780ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67dec7640 1 -- 172.21.15.31:0/3426692532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa668009420 con 0x7fa67815f540 2023-11-26T07:22:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67dec7640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa67815f540 0x7fa678157690 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa670009fa0 tx=0x7fa67000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:32.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa66dffb640 1 -- 172.21.15.31:0/3426692532 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670003ba0 con 0x7fa67815f540 2023-11-26T07:22:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa66dffb640 1 -- 172.21.15.31:0/3426692532 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67000f040 con 0x7fa67815f540 2023-11-26T07:22:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa66dffb640 1 -- 172.21.15.31:0/3426692532 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670004030 con 0x7fa67815f540 2023-11-26T07:22:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67f92a640 1 -- 172.21.15.31:0/3426692532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6780ad940 con 0x7fa67815f540 2023-11-26T07:22:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67f92a640 1 -- 172.21.15.31:0/3426692532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67815a5c0 con 0x7fa67815f540 2023-11-26T07:22:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.021+0000 7fa67f92a640 1 -- 172.21.15.31:0/3426692532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67804e850 con 0x7fa67815f540 2023-11-26T07:22:32.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.025+0000 7fa66dffb640 1 -- 172.21.15.31:0/3426692532 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fa670004190 con 0x7fa67815f540 2023-11-26T07:22:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.025+0000 7fa66dffb640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa65c0665c0 0x7fa65c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.025+0000 7fa66dffb640 1 -- 172.21.15.31:0/3426692532 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa670084520 con 0x7fa67815f540 2023-11-26T07:22:32.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.025+0000 7fa67d6c6640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa65c0665c0 0x7fa65c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.025+0000 7fa67d6c6640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa65c0665c0 0x7fa65c068a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6740011f0 tx=0x7fa674007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:32.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.029+0000 7fa66dffb640 1 -- 172.21.15.31:0/3426692532 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa670051700 con 0x7fa67815f540 2023-11-26T07:22:32.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.181+0000 7fa67f92a640 1 -- 172.21.15.31:0/3426692532 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa67815a200 con 0x7fa65c0665c0 2023-11-26T07:22:32.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.189+0000 7fa66dffb640 1 -- 172.21.15.31:0/3426692532 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288657 (secure 0 0 0) 0x7fa67815a200 con 0x7fa65c0665c0 2023-11-26T07:22:32.196 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:32.198 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:22:32.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 -- 172.21.15.31:0/3426692532 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa65c0665c0 msgr2=0x7fa65c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa65c0665c0 0x7fa65c068a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa6740011f0 tx=0x7fa674007d40 comp rx=0 tx=0).stop 2023-11-26T07:22:32.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 -- 172.21.15.31:0/3426692532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa67815f540 msgr2=0x7fa678157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa67815f540 0x7fa678157690 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa670009fa0 tx=0x7fa67000d560 comp rx=0 tx=0).stop 2023-11-26T07:22:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 -- 172.21.15.31:0/3426692532 shutdown_connections 2023-11-26T07:22:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa65c0665c0 0x7fa65c068a80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa67815f540 0x7fa678157690 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6780b39c0 0x7fa6780ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 --2- 172.21.15.31:0/3426692532 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa67806cb70 0x7fa678154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 -- 172.21.15.31:0/3426692532 >> 172.21.15.31:0/3426692532 conn(0x7fa6780b0c40 msgr2=0x7fa678159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 -- 172.21.15.31:0/3426692532 shutdown_connections 2023-11-26T07:22:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.193+0000 7fa67f92a640 1 -- 172.21.15.31:0/3426692532 wait complete. 2023-11-26T07:22:32.217 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-26T07:22:30.201714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258380,"kb_used_data":8900,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716340,"statfs":{"total":773094113280,"available":772829532160,"internally_reserved":0,"allocated":9113600,"data_stored":4553513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001484"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144176+0000","last_change":"2023-11-26T07:22:23.150661+0000","last_active":"2023-11-26T07:22:24.144176+0000","last_peered":"2023-11-26T07:22:24.144176+0000","last_clean":"2023-11-26T07:22:24.144176+0000","last_became_active":"2023-11-26T07:22:23.150364+0000","last_became_peered":"2023-11-26T07:22:23.150364+0000","last_unstale":"2023-11-26T07:22:24.144176+0000","last_undegraded":"2023-11-26T07:22:24.144176+0000","last_fullsized":"2023-11-26T07:22:24.144176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:30.716028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.463202+0000","last_change":"2023-11-26T07:22:23.161637+0000","last_active":"2023-11-26T07:22:24.463202+0000","last_peered":"2023-11-26T07:22:24.463202+0000","last_clean":"2023-11-26T07:22:24.463202+0000","last_became_active":"2023-11-26T07:22:23.161224+0000","last_became_peered":"2023-11-26T07:22:23.161224+0000","last_unstale":"2023-11-26T07:22:24.463202+0000","last_undegraded":"2023-11-26T07:22:24.463202+0000","last_fullsized":"2023-11-26T07:22:24.463202+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:53:46.158461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461946+0000","last_change":"2023-11-26T07:22:23.162459+0000","last_active":"2023-11-26T07:22:24.461946+0000","last_peered":"2023-11-26T07:22:24.461946+0000","last_clean":"2023-11-26T07:22:24.461946+0000","last_became_active":"2023-11-26T07:22:23.162090+0000","last_became_peered":"2023-11-26T07:22:23.162090+0000","last_unstale":"2023-11-26T07:22:24.461946+0000","last_undegraded":"2023-11-26T07:22:24.461946+0000","last_fullsized":"2023-11-26T07:22:24.461946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:07.939507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140214+0000","last_change":"2023-11-26T07:22:23.171323+0000","last_active":"2023-11-26T07:22:24.140214+0000","last_peered":"2023-11-26T07:22:24.140214+0000","last_clean":"2023-11-26T07:22:24.140214+0000","last_became_active":"2023-11-26T07:22:23.171024+0000","last_became_peered":"2023-11-26T07:22:23.171024+0000","last_unstale":"2023-11-26T07:22:24.140214+0000","last_undegraded":"2023-11-26T07:22:24.140214+0000","last_fullsized":"2023-11-26T07:22:24.140214+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:36:08.063434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142616+0000","last_change":"2023-11-26T07:22:23.166535+0000","last_active":"2023-11-26T07:22:24.142616+0000","last_peered":"2023-11-26T07:22:24.142616+0000","last_clean":"2023-11-26T07:22:24.142616+0000","last_became_active":"2023-11-26T07:22:23.165856+0000","last_became_peered":"2023-11-26T07:22:23.165856+0000","last_unstale":"2023-11-26T07:22:24.142616+0000","last_undegraded":"2023-11-26T07:22:24.142616+0000","last_fullsized":"2023-11-26T07:22:24.142616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:06.468579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461591+0000","last_change":"2023-11-26T07:22:23.160483+0000","last_active":"2023-11-26T07:22:24.461591+0000","last_peered":"2023-11-26T07:22:24.461591+0000","last_clean":"2023-11-26T07:22:24.461591+0000","last_became_active":"2023-11-26T07:22:23.160186+0000","last_became_peered":"2023-11-26T07:22:23.160186+0000","last_unstale":"2023-11-26T07:22:24.461591+0000","last_undegraded":"2023-11-26T07:22:24.461591+0000","last_fullsized":"2023-11-26T07:22:24.461591+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:11:11.552689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.139868+0000","last_change":"2023-11-26T07:22:23.161985+0000","last_active":"2023-11-26T07:22:24.139868+0000","last_peered":"2023-11-26T07:22:24.139868+0000","last_clean":"2023-11-26T07:22:24.139868+0000","last_became_active":"2023-11-26T07:22:23.161700+0000","last_became_peered":"2023-11-26T07:22:23.161700+0000","last_unstale":"2023-11-26T07:22:24.139868+0000","last_undegraded":"2023-11-26T07:22:24.139868+0000","last_fullsized":"2023-11-26T07:22:24.139868+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:35:18.516207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461739+0000","last_change":"2023-11-26T07:22:23.156569+0000","last_active":"2023-11-26T07:22:24.461739+0000","last_peered":"2023-11-26T07:22:24.461739+0000","last_clean":"2023-11-26T07:22:24.461739+0000","last_became_active":"2023-11-26T07:22:23.155834+0000","last_became_peered":"2023-11-26T07:22:23.155834+0000","last_unstale":"2023-11-26T07:22:24.461739+0000","last_undegraded":"2023-11-26T07:22:24.461739+0000","last_fullsized":"2023-11-26T07:22:24.461739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:23:30.693319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462480+0000","last_change":"2023-11-26T07:22:23.161805+0000","last_active":"2023-11-26T07:22:24.462480+0000","last_peered":"2023-11-26T07:22:24.462480+0000","last_clean":"2023-11-26T07:22:24.462480+0000","last_became_active":"2023-11-26T07:22:23.161520+0000","last_became_peered":"2023-11-26T07:22:23.161520+0000","last_unstale":"2023-11-26T07:22:24.462480+0000","last_undegraded":"2023-11-26T07:22:24.462480+0000","last_fullsized":"2023-11-26T07:22:24.462480+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:06.468579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144155+0000","last_change":"2023-11-26T07:22:23.148510+0000","last_active":"2023-11-26T07:22:24.144155+0000","last_peered":"2023-11-26T07:22:24.144155+0000","last_clean":"2023-11-26T07:22:24.144155+0000","last_became_active":"2023-11-26T07:22:23.148096+0000","last_became_peered":"2023-11-26T07:22:23.148096+0000","last_unstale":"2023-11-26T07:22:24.144155+0000","last_undegraded":"2023-11-26T07:22:24.144155+0000","last_fullsized":"2023-11-26T07:22:24.144155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:13:00.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.463061+0000","last_change":"2023-11-26T07:22:23.167271+0000","last_active":"2023-11-26T07:22:24.463061+0000","last_peered":"2023-11-26T07:22:24.463061+0000","last_clean":"2023-11-26T07:22:24.463061+0000","last_became_active":"2023-11-26T07:22:23.166179+0000","last_became_peered":"2023-11-26T07:22:23.166179+0000","last_unstale":"2023-11-26T07:22:24.463061+0000","last_undegraded":"2023-11-26T07:22:24.463061+0000","last_fullsized":"2023-11-26T07:22:24.463061+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:36:34.890789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144148+0000","last_change":"2023-11-26T07:22:23.144450+0000","last_active":"2023-11-26T07:22:24.144148+0000","last_peered":"2023-11-26T07:22:24.144148+0000","last_clean":"2023-11-26T07:22:24.144148+0000","last_became_active":"2023-11-26T07:22:23.144127+0000","last_became_peered":"2023-11-26T07:22:23.144127+0000","last_unstale":"2023-11-26T07:22:24.144148+0000","last_undegraded":"2023-11-26T07:22:24.144148+0000","last_fullsized":"2023-11-26T07:22:24.144148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:07.939507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461514+0000","last_change":"2023-11-26T07:22:23.161346+0000","last_active":"2023-11-26T07:22:24.461514+0000","last_peered":"2023-11-26T07:22:24.461514+0000","last_clean":"2023-11-26T07:22:24.461514+0000","last_became_active":"2023-11-26T07:22:23.161059+0000","last_became_peered":"2023-11-26T07:22:23.161059+0000","last_unstale":"2023-11-26T07:22:24.461514+0000","last_undegraded":"2023-11-26T07:22:24.461514+0000","last_fullsized":"2023-11-26T07:22:24.461514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:31.249944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140099+0000","last_change":"2023-11-26T07:22:23.170673+0000","last_active":"2023-11-26T07:22:24.140099+0000","last_peered":"2023-11-26T07:22:24.140099+0000","last_clean":"2023-11-26T07:22:24.140099+0000","last_became_active":"2023-11-26T07:22:23.170227+0000","last_became_peered":"2023-11-26T07:22:23.170227+0000","last_unstale":"2023-11-26T07:22:24.140099+0000","last_undegraded":"2023-11-26T07:22:24.140099+0000","last_fullsized":"2023-11-26T07:22:24.140099+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:30:14.851987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461806+0000","last_change":"2023-11-26T07:22:23.155339+0000","last_active":"2023-11-26T07:22:24.461806+0000","last_peered":"2023-11-26T07:22:24.461806+0000","last_clean":"2023-11-26T07:22:24.461806+0000","last_became_active":"2023-11-26T07:22:23.155010+0000","last_became_peered":"2023-11-26T07:22:23.155010+0000","last_unstale":"2023-11-26T07:22:24.461806+0000","last_undegraded":"2023-11-26T07:22:24.461806+0000","last_fullsized":"2023-11-26T07:22:24.461806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:51.056578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142356+0000","last_change":"2023-11-26T07:22:23.152403+0000","last_active":"2023-11-26T07:22:24.142356+0000","last_peered":"2023-11-26T07:22:24.142356+0000","last_clean":"2023-11-26T07:22:24.142356+0000","last_became_active":"2023-11-26T07:22:23.152082+0000","last_became_peered":"2023-11-26T07:22:23.152082+0000","last_unstale":"2023-11-26T07:22:24.142356+0000","last_undegraded":"2023-11-26T07:22:24.142356+0000","last_fullsized":"2023-11-26T07:22:24.142356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:15.445163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142974+0000","last_change":"2023-11-26T07:22:23.160642+0000","last_active":"2023-11-26T07:22:24.142974+0000","last_peered":"2023-11-26T07:22:24.142974+0000","last_clean":"2023-11-26T07:22:24.142974+0000","last_became_active":"2023-11-26T07:22:23.160335+0000","last_became_peered":"2023-11-26T07:22:23.160335+0000","last_unstale":"2023-11-26T07:22:24.142974+0000","last_undegraded":"2023-11-26T07:22:24.142974+0000","last_fullsized":"2023-11-26T07:22:24.142974+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:26:35.758938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140360+0000","last_change":"2023-11-26T07:22:23.161871+0000","last_active":"2023-11-26T07:22:24.140360+0000","last_peered":"2023-11-26T07:22:24.140360+0000","last_clean":"2023-11-26T07:22:24.140360+0000","last_became_active":"2023-11-26T07:22:23.161576+0000","last_became_peered":"2023-11-26T07:22:23.161576+0000","last_unstale":"2023-11-26T07:22:24.140360+0000","last_undegraded":"2023-11-26T07:22:24.140360+0000","last_fullsized":"2023-11-26T07:22:24.140360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:24.741346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462367+0000","last_change":"2023-11-26T07:22:23.158365+0000","last_active":"2023-11-26T07:22:24.462367+0000","last_peered":"2023-11-26T07:22:24.462367+0000","last_clean":"2023-11-26T07:22:24.462367+0000","last_became_active":"2023-11-26T07:22:23.158040+0000","last_became_peered":"2023-11-26T07:22:23.158040+0000","last_unstale":"2023-11-26T07:22:24.462367+0000","last_undegraded":"2023-11-26T07:22:24.462367+0000","last_fullsized":"2023-11-26T07:22:24.462367+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:35:19.658117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140287+0000","last_change":"2023-11-26T07:22:23.167696+0000","last_active":"2023-11-26T07:22:24.140287+0000","last_peered":"2023-11-26T07:22:24.140287+0000","last_clean":"2023-11-26T07:22:24.140287+0000","last_became_active":"2023-11-26T07:22:23.167415+0000","last_became_peered":"2023-11-26T07:22:23.167415+0000","last_unstale":"2023-11-26T07:22:24.140287+0000","last_undegraded":"2023-11-26T07:22:24.140287+0000","last_fullsized":"2023-11-26T07:22:24.140287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:25:14.378387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140048+0000","last_change":"2023-11-26T07:22:23.166610+0000","last_active":"2023-11-26T07:22:24.140048+0000","last_peered":"2023-11-26T07:22:24.140048+0000","last_clean":"2023-11-26T07:22:24.140048+0000","last_became_active":"2023-11-26T07:22:23.166345+0000","last_became_peered":"2023-11-26T07:22:23.166345+0000","last_unstale":"2023-11-26T07:22:24.140048+0000","last_undegraded":"2023-11-26T07:22:24.140048+0000","last_fullsized":"2023-11-26T07:22:24.140048+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:45.111405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462949+0000","last_change":"2023-11-26T07:22:23.168094+0000","last_active":"2023-11-26T07:22:24.462949+0000","last_peered":"2023-11-26T07:22:24.462949+0000","last_clean":"2023-11-26T07:22:24.462949+0000","last_became_active":"2023-11-26T07:22:23.167796+0000","last_became_peered":"2023-11-26T07:22:23.167796+0000","last_unstale":"2023-11-26T07:22:24.462949+0000","last_undegraded":"2023-11-26T07:22:24.462949+0000","last_fullsized":"2023-11-26T07:22:24.462949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:47:09.121233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143794+0000","last_change":"2023-11-26T07:22:23.161411+0000","last_active":"2023-11-26T07:22:24.143794+0000","last_peered":"2023-11-26T07:22:24.143794+0000","last_clean":"2023-11-26T07:22:24.143794+0000","last_became_active":"2023-11-26T07:22:23.161151+0000","last_became_peered":"2023-11-26T07:22:23.161151+0000","last_unstale":"2023-11-26T07:22:24.143794+0000","last_undegraded":"2023-11-26T07:22:24.143794+0000","last_fullsized":"2023-11-26T07:22:24.143794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:26:35.758938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.139748+0000","last_change":"2023-11-26T07:22:23.162896+0000","last_active":"2023-11-26T07:22:24.139748+0000","last_peered":"2023-11-26T07:22:24.139748+0000","last_clean":"2023-11-26T07:22:24.139748+0000","last_became_active":"2023-11-26T07:22:23.162559+0000","last_became_peered":"2023-11-26T07:22:23.162559+0000","last_unstale":"2023-11-26T07:22:24.139748+0000","last_undegraded":"2023-11-26T07:22:24.139748+0000","last_fullsized":"2023-11-26T07:22:24.139748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:30:57.719160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.463146+0000","last_change":"2023-11-26T07:22:23.165648+0000","last_active":"2023-11-26T07:22:24.463146+0000","last_peered":"2023-11-26T07:22:24.463146+0000","last_clean":"2023-11-26T07:22:24.463146+0000","last_became_active":"2023-11-26T07:22:23.162343+0000","last_became_peered":"2023-11-26T07:22:23.162343+0000","last_unstale":"2023-11-26T07:22:24.463146+0000","last_undegraded":"2023-11-26T07:22:24.463146+0000","last_fullsized":"2023-11-26T07:22:24.463146+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:44:36.318995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462040+0000","last_change":"2023-11-26T07:22:23.165553+0000","last_active":"2023-11-26T07:22:24.462040+0000","last_peered":"2023-11-26T07:22:24.462040+0000","last_clean":"2023-11-26T07:22:24.462040+0000","last_became_active":"2023-11-26T07:22:23.165220+0000","last_became_peered":"2023-11-26T07:22:23.165220+0000","last_unstale":"2023-11-26T07:22:24.462040+0000","last_undegraded":"2023-11-26T07:22:24.462040+0000","last_fullsized":"2023-11-26T07:22:24.462040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:56:30.716028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142778+0000","last_change":"2023-11-26T07:22:23.153109+0000","last_active":"2023-11-26T07:22:24.142778+0000","last_peered":"2023-11-26T07:22:24.142778+0000","last_clean":"2023-11-26T07:22:24.142778+0000","last_became_active":"2023-11-26T07:22:23.152820+0000","last_became_peered":"2023-11-26T07:22:23.152820+0000","last_unstale":"2023-11-26T07:22:24.142778+0000","last_undegraded":"2023-11-26T07:22:24.142778+0000","last_fullsized":"2023-11-26T07:22:24.142778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:55.059095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143853+0000","last_change":"2023-11-26T07:22:23.147508+0000","last_active":"2023-11-26T07:22:24.143853+0000","last_peered":"2023-11-26T07:22:24.143853+0000","last_clean":"2023-11-26T07:22:24.143853+0000","last_became_active":"2023-11-26T07:22:23.147164+0000","last_became_peered":"2023-11-26T07:22:23.147164+0000","last_unstale":"2023-11-26T07:22:24.143853+0000","last_undegraded":"2023-11-26T07:22:24.143853+0000","last_fullsized":"2023-11-26T07:22:24.143853+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:31.249944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462721+0000","last_change":"2023-11-26T07:22:23.167488+0000","last_active":"2023-11-26T07:22:24.462721+0000","last_peered":"2023-11-26T07:22:24.462721+0000","last_clean":"2023-11-26T07:22:24.462721+0000","last_became_active":"2023-11-26T07:22:23.166958+0000","last_became_peered":"2023-11-26T07:22:23.166958+0000","last_unstale":"2023-11-26T07:22:24.462721+0000","last_undegraded":"2023-11-26T07:22:24.462721+0000","last_fullsized":"2023-11-26T07:22:24.462721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:38:16.114360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.463110+0000","last_change":"2023-11-26T07:22:23.165307+0000","last_active":"2023-11-26T07:22:24.463110+0000","last_peered":"2023-11-26T07:22:24.463110+0000","last_clean":"2023-11-26T07:22:24.463110+0000","last_became_active":"2023-11-26T07:22:23.162369+0000","last_became_peered":"2023-11-26T07:22:23.162369+0000","last_unstale":"2023-11-26T07:22:24.463110+0000","last_undegraded":"2023-11-26T07:22:24.463110+0000","last_fullsized":"2023-11-26T07:22:24.463110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:00:26.560742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142354+0000","last_change":"2023-11-26T07:22:23.160679+0000","last_active":"2023-11-26T07:22:24.142354+0000","last_peered":"2023-11-26T07:22:24.142354+0000","last_clean":"2023-11-26T07:22:24.142354+0000","last_became_active":"2023-11-26T07:22:23.158286+0000","last_became_peered":"2023-11-26T07:22:23.158286+0000","last_unstale":"2023-11-26T07:22:24.142354+0000","last_undegraded":"2023-11-26T07:22:24.142354+0000","last_fullsized":"2023-11-26T07:22:24.142354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:16:14.174321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143683+0000","last_change":"2023-11-26T07:22:23.154277+0000","last_active":"2023-11-26T07:22:24.143683+0000","last_peered":"2023-11-26T07:22:24.143683+0000","last_clean":"2023-11-26T07:22:24.143683+0000","last_became_active":"2023-11-26T07:22:23.153959+0000","last_became_peered":"2023-11-26T07:22:23.153959+0000","last_unstale":"2023-11-26T07:22:24.143683+0000","last_undegraded":"2023-11-26T07:22:24.143683+0000","last_fullsized":"2023-11-26T07:22:24.143683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:55.059095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142990+0000","last_change":"2023-11-26T07:22:23.152460+0000","last_active":"2023-11-26T07:22:24.142990+0000","last_peered":"2023-11-26T07:22:24.142990+0000","last_clean":"2023-11-26T07:22:24.142990+0000","last_became_active":"2023-11-26T07:22:23.152146+0000","last_became_peered":"2023-11-26T07:22:23.152146+0000","last_unstale":"2023-11-26T07:22:24.142990+0000","last_undegraded":"2023-11-26T07:22:24.142990+0000","last_fullsized":"2023-11-26T07:22:24.142990+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:27.564861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.185563+0000","last_change":"2023-11-26T07:22:22.149321+0000","last_active":"2023-11-26T07:22:23.185563+0000","last_peered":"2023-11-26T07:22:23.185563+0000","last_clean":"2023-11-26T07:22:23.185563+0000","last_became_active":"2023-11-26T07:22:22.148856+0000","last_became_peered":"2023-11-26T07:22:22.148856+0000","last_unstale":"2023-11-26T07:22:23.185563+0000","last_undegraded":"2023-11-26T07:22:23.185563+0000","last_fullsized":"2023-11-26T07:22:23.185563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:14.438027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144240+0000","last_change":"2023-11-26T07:22:23.156120+0000","last_active":"2023-11-26T07:22:24.144240+0000","last_peered":"2023-11-26T07:22:24.144240+0000","last_clean":"2023-11-26T07:22:24.144240+0000","last_became_active":"2023-11-26T07:22:23.155837+0000","last_became_peered":"2023-11-26T07:22:23.155837+0000","last_unstale":"2023-11-26T07:22:24.144240+0000","last_undegraded":"2023-11-26T07:22:24.144240+0000","last_fullsized":"2023-11-26T07:22:24.144240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:55:01.934153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.183528+0000","last_change":"2023-11-26T07:22:22.150877+0000","last_active":"2023-11-26T07:22:23.183528+0000","last_peered":"2023-11-26T07:22:23.183528+0000","last_clean":"2023-11-26T07:22:23.183528+0000","last_became_active":"2023-11-26T07:22:22.150413+0000","last_became_peered":"2023-11-26T07:22:22.150413+0000","last_unstale":"2023-11-26T07:22:23.183528+0000","last_undegraded":"2023-11-26T07:22:23.183528+0000","last_fullsized":"2023-11-26T07:22:23.183528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:23:29.686183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143928+0000","last_change":"2023-11-26T07:22:23.164704+0000","last_active":"2023-11-26T07:22:24.143928+0000","last_peered":"2023-11-26T07:22:24.143928+0000","last_clean":"2023-11-26T07:22:24.143928+0000","last_became_active":"2023-11-26T07:22:23.164149+0000","last_became_peered":"2023-11-26T07:22:23.164149+0000","last_unstale":"2023-11-26T07:22:24.143928+0000","last_undegraded":"2023-11-26T07:22:24.143928+0000","last_fullsized":"2023-11-26T07:22:24.143928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:27.564861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.189503+0000","last_change":"2023-11-26T07:22:22.148977+0000","last_active":"2023-11-26T07:22:23.189503+0000","last_peered":"2023-11-26T07:22:23.189503+0000","last_clean":"2023-11-26T07:22:23.189503+0000","last_became_active":"2023-11-26T07:22:22.148703+0000","last_became_peered":"2023-11-26T07:22:22.148703+0000","last_unstale":"2023-11-26T07:22:23.189503+0000","last_undegraded":"2023-11-26T07:22:23.189503+0000","last_fullsized":"2023-11-26T07:22:23.189503+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:16:13.167185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461894+0000","last_change":"2023-11-26T07:22:23.162066+0000","last_active":"2023-11-26T07:22:24.461894+0000","last_peered":"2023-11-26T07:22:24.461894+0000","last_clean":"2023-11-26T07:22:24.461894+0000","last_became_active":"2023-11-26T07:22:23.161712+0000","last_became_peered":"2023-11-26T07:22:23.161712+0000","last_unstale":"2023-11-26T07:22:24.461894+0000","last_undegraded":"2023-11-26T07:22:24.461894+0000","last_fullsized":"2023-11-26T07:22:24.461894+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:57:27.564861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.187659+0000","last_change":"2023-11-26T07:22:22.151630+0000","last_active":"2023-11-26T07:22:23.187659+0000","last_peered":"2023-11-26T07:22:23.187659+0000","last_clean":"2023-11-26T07:22:23.187659+0000","last_became_active":"2023-11-26T07:22:22.151230+0000","last_became_peered":"2023-11-26T07:22:22.151230+0000","last_unstale":"2023-11-26T07:22:23.187659+0000","last_undegraded":"2023-11-26T07:22:23.187659+0000","last_fullsized":"2023-11-26T07:22:23.187659+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:23:29.686183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.463243+0000","last_change":"2023-11-26T07:22:23.163643+0000","last_active":"2023-11-26T07:22:24.463243+0000","last_peered":"2023-11-26T07:22:24.463243+0000","last_clean":"2023-11-26T07:22:24.463243+0000","last_became_active":"2023-11-26T07:22:23.161981+0000","last_became_peered":"2023-11-26T07:22:23.161981+0000","last_unstale":"2023-11-26T07:22:24.463243+0000","last_undegraded":"2023-11-26T07:22:24.463243+0000","last_fullsized":"2023-11-26T07:22:24.463243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:45:39.457900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.189320+0000","last_change":"2023-11-26T07:22:22.153463+0000","last_active":"2023-11-26T07:22:23.189320+0000","last_peered":"2023-11-26T07:22:23.189320+0000","last_clean":"2023-11-26T07:22:23.189320+0000","last_became_active":"2023-11-26T07:22:22.153132+0000","last_became_peered":"2023-11-26T07:22:22.153132+0000","last_unstale":"2023-11-26T07:22:23.189320+0000","last_undegraded":"2023-11-26T07:22:23.189320+0000","last_fullsized":"2023-11-26T07:22:23.189320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:30:47.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142663+0000","last_change":"2023-11-26T07:22:23.159636+0000","last_active":"2023-11-26T07:22:24.142663+0000","last_peered":"2023-11-26T07:22:24.142663+0000","last_clean":"2023-11-26T07:22:24.142663+0000","last_became_active":"2023-11-26T07:22:23.159101+0000","last_became_peered":"2023-11-26T07:22:23.159101+0000","last_unstale":"2023-11-26T07:22:24.142663+0000","last_undegraded":"2023-11-26T07:22:24.142663+0000","last_fullsized":"2023-11-26T07:22:24.142663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:53.313195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.183154+0000","last_change":"2023-11-26T07:22:22.148229+0000","last_active":"2023-11-26T07:22:23.183154+0000","last_peered":"2023-11-26T07:22:23.183154+0000","last_clean":"2023-11-26T07:22:23.183154+0000","last_became_active":"2023-11-26T07:22:22.147927+0000","last_became_peered":"2023-11-26T07:22:22.147927+0000","last_unstale":"2023-11-26T07:22:23.183154+0000","last_undegraded":"2023-11-26T07:22:23.183154+0000","last_fullsized":"2023-11-26T07:22:23.183154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:58:34.847257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461408+0000","last_change":"2023-11-26T07:22:23.167749+0000","last_active":"2023-11-26T07:22:24.461408+0000","last_peered":"2023-11-26T07:22:24.461408+0000","last_clean":"2023-11-26T07:22:24.461408+0000","last_became_active":"2023-11-26T07:22:23.167411+0000","last_became_peered":"2023-11-26T07:22:23.167411+0000","last_unstale":"2023-11-26T07:22:24.461408+0000","last_undegraded":"2023-11-26T07:22:24.461408+0000","last_fullsized":"2023-11-26T07:22:24.461408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:35:19.658117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.145891+0000","last_change":"2023-11-26T07:22:22.144638+0000","last_active":"2023-11-26T07:22:23.145891+0000","last_peered":"2023-11-26T07:22:23.145891+0000","last_clean":"2023-11-26T07:22:23.145891+0000","last_became_active":"2023-11-26T07:22:22.144230+0000","last_became_peered":"2023-11-26T07:22:22.144230+0000","last_unstale":"2023-11-26T07:22:23.145891+0000","last_undegraded":"2023-11-26T07:22:23.145891+0000","last_fullsized":"2023-11-26T07:22:23.145891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:17.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461813+0000","last_change":"2023-11-26T07:22:23.156605+0000","last_active":"2023-11-26T07:22:24.461813+0000","last_peered":"2023-11-26T07:22:24.461813+0000","last_clean":"2023-11-26T07:22:24.461813+0000","last_became_active":"2023-11-26T07:22:23.156291+0000","last_became_peered":"2023-11-26T07:22:23.156291+0000","last_unstale":"2023-11-26T07:22:24.461813+0000","last_undegraded":"2023-11-26T07:22:24.461813+0000","last_fullsized":"2023-11-26T07:22:24.461813+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:33:28.090781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.149654+0000","last_change":"2023-11-26T07:22:22.153839+0000","last_active":"2023-11-26T07:22:23.149654+0000","last_peered":"2023-11-26T07:22:23.149654+0000","last_clean":"2023-11-26T07:22:23.149654+0000","last_became_active":"2023-11-26T07:22:22.153535+0000","last_became_peered":"2023-11-26T07:22:22.153535+0000","last_unstale":"2023-11-26T07:22:23.149654+0000","last_undegraded":"2023-11-26T07:22:23.149654+0000","last_fullsized":"2023-11-26T07:22:23.149654+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:23.734209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462173+0000","last_change":"2023-11-26T07:22:23.165238+0000","last_active":"2023-11-26T07:22:24.462173+0000","last_peered":"2023-11-26T07:22:24.462173+0000","last_clean":"2023-11-26T07:22:24.462173+0000","last_became_active":"2023-11-26T07:22:23.164949+0000","last_became_peered":"2023-11-26T07:22:23.164949+0000","last_unstale":"2023-11-26T07:22:24.462173+0000","last_undegraded":"2023-11-26T07:22:24.462173+0000","last_fullsized":"2023-11-26T07:22:24.462173+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:29:04.949568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.141594+0000","last_change":"2023-11-26T07:22:22.140913+0000","last_active":"2023-11-26T07:22:23.141594+0000","last_peered":"2023-11-26T07:22:23.141594+0000","last_clean":"2023-11-26T07:22:23.141594+0000","last_became_active":"2023-11-26T07:22:22.140643+0000","last_became_peered":"2023-11-26T07:22:22.140643+0000","last_unstale":"2023-11-26T07:22:23.141594+0000","last_undegraded":"2023-11-26T07:22:23.141594+0000","last_fullsized":"2023-11-26T07:22:23.141594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:05.461443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462479+0000","last_change":"2023-11-26T07:22:23.162405+0000","last_active":"2023-11-26T07:22:24.462479+0000","last_peered":"2023-11-26T07:22:24.462479+0000","last_clean":"2023-11-26T07:22:24.462479+0000","last_became_active":"2023-11-26T07:22:23.161150+0000","last_became_peered":"2023-11-26T07:22:23.161150+0000","last_unstale":"2023-11-26T07:22:24.462479+0000","last_undegraded":"2023-11-26T07:22:24.462479+0000","last_fullsized":"2023-11-26T07:22:24.462479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:53.313195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.140267+0000","last_change":"2023-11-26T07:22:22.151535+0000","last_active":"2023-11-26T07:22:23.140267+0000","last_peered":"2023-11-26T07:22:23.140267+0000","last_clean":"2023-11-26T07:22:23.140267+0000","last_became_active":"2023-11-26T07:22:22.151097+0000","last_became_peered":"2023-11-26T07:22:22.151097+0000","last_unstale":"2023-11-26T07:22:23.140267+0000","last_undegraded":"2023-11-26T07:22:23.140267+0000","last_fullsized":"2023-11-26T07:22:23.140267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:17.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461610+0000","last_change":"2023-11-26T07:22:23.158807+0000","last_active":"2023-11-26T07:22:24.461610+0000","last_peered":"2023-11-26T07:22:24.461610+0000","last_clean":"2023-11-26T07:22:24.461610+0000","last_became_active":"2023-11-26T07:22:23.158510+0000","last_became_peered":"2023-11-26T07:22:23.158510+0000","last_unstale":"2023-11-26T07:22:24.461610+0000","last_undegraded":"2023-11-26T07:22:24.461610+0000","last_fullsized":"2023-11-26T07:22:24.461610+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:53.313195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.188560+0000","last_change":"2023-11-26T07:22:22.149767+0000","last_active":"2023-11-26T07:22:23.188560+0000","last_peered":"2023-11-26T07:22:23.188560+0000","last_clean":"2023-11-26T07:22:23.188560+0000","last_became_active":"2023-11-26T07:22:22.149435+0000","last_became_peered":"2023-11-26T07:22:22.149435+0000","last_unstale":"2023-11-26T07:22:23.188560+0000","last_undegraded":"2023-11-26T07:22:23.188560+0000","last_fullsized":"2023-11-26T07:22:23.188560+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:58:34.847257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142780+0000","last_change":"2023-11-26T07:22:23.160003+0000","last_active":"2023-11-26T07:22:24.142780+0000","last_peered":"2023-11-26T07:22:24.142780+0000","last_clean":"2023-11-26T07:22:24.142780+0000","last_became_active":"2023-11-26T07:22:23.159479+0000","last_became_peered":"2023-11-26T07:22:23.159479+0000","last_unstale":"2023-11-26T07:22:24.142780+0000","last_undegraded":"2023-11-26T07:22:24.142780+0000","last_fullsized":"2023-11-26T07:22:24.142780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:35:19.658117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.191377+0000","last_change":"2023-11-26T07:22:22.138119+0000","last_active":"2023-11-26T07:22:23.191377+0000","last_peered":"2023-11-26T07:22:23.191377+0000","last_clean":"2023-11-26T07:22:23.191377+0000","last_became_active":"2023-11-26T07:22:22.137770+0000","last_became_peered":"2023-11-26T07:22:22.137770+0000","last_unstale":"2023-11-26T07:22:23.191377+0000","last_undegraded":"2023-11-26T07:22:23.191377+0000","last_fullsized":"2023-11-26T07:22:23.191377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:30:47.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461469+0000","last_change":"2023-11-26T07:22:23.158226+0000","last_active":"2023-11-26T07:22:24.461469+0000","last_peered":"2023-11-26T07:22:24.461469+0000","last_clean":"2023-11-26T07:22:24.461469+0000","last_became_active":"2023-11-26T07:22:23.157937+0000","last_became_peered":"2023-11-26T07:22:23.157937+0000","last_unstale":"2023-11-26T07:22:24.461469+0000","last_undegraded":"2023-11-26T07:22:24.461469+0000","last_fullsized":"2023-11-26T07:22:24.461469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:43:06.468579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.189651+0000","last_change":"2023-11-26T07:22:22.145272+0000","last_active":"2023-11-26T07:22:23.189651+0000","last_peered":"2023-11-26T07:22:23.189651+0000","last_clean":"2023-11-26T07:22:23.189651+0000","last_became_active":"2023-11-26T07:22:22.145011+0000","last_became_peered":"2023-11-26T07:22:22.145011+0000","last_unstale":"2023-11-26T07:22:23.189651+0000","last_undegraded":"2023-11-26T07:22:23.189651+0000","last_fullsized":"2023-11-26T07:22:23.189651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:58:46.089931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461932+0000","last_change":"2023-11-26T07:22:23.168174+0000","last_active":"2023-11-26T07:22:24.461932+0000","last_peered":"2023-11-26T07:22:24.461932+0000","last_clean":"2023-11-26T07:22:24.461932+0000","last_became_active":"2023-11-26T07:22:23.167943+0000","last_became_peered":"2023-11-26T07:22:23.167943+0000","last_unstale":"2023-11-26T07:22:24.461932+0000","last_undegraded":"2023-11-26T07:22:24.461932+0000","last_fullsized":"2023-11-26T07:22:24.461932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:03:00.736312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.140195+0000","last_change":"2023-11-26T07:22:22.149360+0000","last_active":"2023-11-26T07:22:23.140195+0000","last_peered":"2023-11-26T07:22:23.140195+0000","last_clean":"2023-11-26T07:22:23.140195+0000","last_became_active":"2023-11-26T07:22:22.149068+0000","last_became_peered":"2023-11-26T07:22:22.149068+0000","last_unstale":"2023-11-26T07:22:23.140195+0000","last_undegraded":"2023-11-26T07:22:23.140195+0000","last_fullsized":"2023-11-26T07:22:23.140195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:47:50.273842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462810+0000","last_change":"2023-11-26T07:22:23.163332+0000","last_active":"2023-11-26T07:22:24.462810+0000","last_peered":"2023-11-26T07:22:24.462810+0000","last_clean":"2023-11-26T07:22:24.462810+0000","last_became_active":"2023-11-26T07:22:23.163063+0000","last_became_peered":"2023-11-26T07:22:23.163063+0000","last_unstale":"2023-11-26T07:22:24.462810+0000","last_undegraded":"2023-11-26T07:22:24.462810+0000","last_fullsized":"2023-11-26T07:22:24.462810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:13:42.684644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.147364+0000","last_change":"2023-11-26T07:22:22.151868+0000","last_active":"2023-11-26T07:22:23.147364+0000","last_peered":"2023-11-26T07:22:23.147364+0000","last_clean":"2023-11-26T07:22:23.147364+0000","last_became_active":"2023-11-26T07:22:22.151535+0000","last_became_peered":"2023-11-26T07:22:22.151535+0000","last_unstale":"2023-11-26T07:22:23.147364+0000","last_undegraded":"2023-11-26T07:22:23.147364+0000","last_fullsized":"2023-11-26T07:22:23.147364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:09:43.742463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462578+0000","last_change":"2023-11-26T07:22:23.156407+0000","last_active":"2023-11-26T07:22:24.462578+0000","last_peered":"2023-11-26T07:22:24.462578+0000","last_clean":"2023-11-26T07:22:24.462578+0000","last_became_active":"2023-11-26T07:22:23.156110+0000","last_became_peered":"2023-11-26T07:22:23.156110+0000","last_unstale":"2023-11-26T07:22:24.462578+0000","last_undegraded":"2023-11-26T07:22:24.462578+0000","last_fullsized":"2023-11-26T07:22:24.462578+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:32:22.241284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.189422+0000","last_change":"2023-11-26T07:22:22.145842+0000","last_active":"2023-11-26T07:22:23.189422+0000","last_peered":"2023-11-26T07:22:23.189422+0000","last_clean":"2023-11-26T07:22:23.189422+0000","last_became_active":"2023-11-26T07:22:22.145573+0000","last_became_peered":"2023-11-26T07:22:22.145573+0000","last_unstale":"2023-11-26T07:22:23.189422+0000","last_undegraded":"2023-11-26T07:22:23.189422+0000","last_fullsized":"2023-11-26T07:22:23.189422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:30:47.412185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462132+0000","last_change":"2023-11-26T07:22:23.162197+0000","last_active":"2023-11-26T07:22:24.462132+0000","last_peered":"2023-11-26T07:22:24.462132+0000","last_clean":"2023-11-26T07:22:24.462132+0000","last_became_active":"2023-11-26T07:22:23.160525+0000","last_became_peered":"2023-11-26T07:22:23.160525+0000","last_unstale":"2023-11-26T07:22:24.462132+0000","last_undegraded":"2023-11-26T07:22:24.462132+0000","last_fullsized":"2023-11-26T07:22:24.462132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:13:00.857863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.185964+0000","last_change":"2023-11-26T07:22:22.140377+0000","last_active":"2023-11-26T07:22:23.185964+0000","last_peered":"2023-11-26T07:22:23.185964+0000","last_clean":"2023-11-26T07:22:23.185964+0000","last_became_active":"2023-11-26T07:22:22.138993+0000","last_became_peered":"2023-11-26T07:22:22.138993+0000","last_unstale":"2023-11-26T07:22:23.185964+0000","last_undegraded":"2023-11-26T07:22:23.185964+0000","last_fullsized":"2023-11-26T07:22:23.185964+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:52:52.306059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144297+0000","last_change":"2023-11-26T07:22:23.158016+0000","last_active":"2023-11-26T07:22:24.144297+0000","last_peered":"2023-11-26T07:22:24.144297+0000","last_clean":"2023-11-26T07:22:24.144297+0000","last_became_active":"2023-11-26T07:22:23.157356+0000","last_became_peered":"2023-11-26T07:22:23.157356+0000","last_unstale":"2023-11-26T07:22:24.144297+0000","last_undegraded":"2023-11-26T07:22:24.144297+0000","last_fullsized":"2023-11-26T07:22:24.144297+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:47:51.280978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.189131+0000","last_change":"2023-11-26T07:22:22.152564+0000","last_active":"2023-11-26T07:22:23.189131+0000","last_peered":"2023-11-26T07:22:23.189131+0000","last_clean":"2023-11-26T07:22:23.189131+0000","last_became_active":"2023-11-26T07:22:22.152161+0000","last_became_peered":"2023-11-26T07:22:22.152161+0000","last_unstale":"2023-11-26T07:22:23.189131+0000","last_undegraded":"2023-11-26T07:22:23.189131+0000","last_fullsized":"2023-11-26T07:22:23.189131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:14.438027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143085+0000","last_change":"2023-11-26T07:22:23.166236+0000","last_active":"2023-11-26T07:22:24.143085+0000","last_peered":"2023-11-26T07:22:24.143085+0000","last_clean":"2023-11-26T07:22:24.143085+0000","last_became_active":"2023-11-26T07:22:23.165942+0000","last_became_peered":"2023-11-26T07:22:23.165942+0000","last_unstale":"2023-11-26T07:22:24.143085+0000","last_undegraded":"2023-11-26T07:22:24.143085+0000","last_fullsized":"2023-11-26T07:22:24.143085+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:03:00.736312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.148063+0000","last_change":"2023-11-26T07:22:22.145038+0000","last_active":"2023-11-26T07:22:23.148063+0000","last_peered":"2023-11-26T07:22:23.148063+0000","last_clean":"2023-11-26T07:22:23.148063+0000","last_became_active":"2023-11-26T07:22:22.144754+0000","last_became_peered":"2023-11-26T07:22:22.144754+0000","last_unstale":"2023-11-26T07:22:23.148063+0000","last_undegraded":"2023-11-26T07:22:23.148063+0000","last_fullsized":"2023-11-26T07:22:23.148063+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:33:39.670858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.140079+0000","last_change":"2023-11-26T07:22:23.166017+0000","last_active":"2023-11-26T07:22:24.140079+0000","last_peered":"2023-11-26T07:22:24.140079+0000","last_clean":"2023-11-26T07:22:24.140079+0000","last_became_active":"2023-11-26T07:22:23.165709+0000","last_became_peered":"2023-11-26T07:22:23.165709+0000","last_unstale":"2023-11-26T07:22:24.140079+0000","last_undegraded":"2023-11-26T07:22:24.140079+0000","last_fullsized":"2023-11-26T07:22:24.140079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:54:34.684641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.191492+0000","last_change":"2023-11-26T07:22:22.140578+0000","last_active":"2023-11-26T07:22:23.191492+0000","last_peered":"2023-11-26T07:22:23.191492+0000","last_clean":"2023-11-26T07:22:23.191492+0000","last_became_active":"2023-11-26T07:22:22.140222+0000","last_became_peered":"2023-11-26T07:22:22.140222+0000","last_unstale":"2023-11-26T07:22:23.191492+0000","last_undegraded":"2023-11-26T07:22:23.191492+0000","last_fullsized":"2023-11-26T07:22:23.191492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:55:00.927017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461820+0000","last_change":"2023-11-26T07:22:23.155279+0000","last_active":"2023-11-26T07:22:24.461820+0000","last_peered":"2023-11-26T07:22:24.461820+0000","last_clean":"2023-11-26T07:22:24.461820+0000","last_became_active":"2023-11-26T07:22:23.154787+0000","last_became_peered":"2023-11-26T07:22:23.154787+0000","last_unstale":"2023-11-26T07:22:24.461820+0000","last_undegraded":"2023-11-26T07:22:24.461820+0000","last_fullsized":"2023-11-26T07:22:24.461820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:16:14.174321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.151818+0000","last_change":"2023-11-26T07:22:22.147493+0000","last_active":"2023-11-26T07:22:23.151818+0000","last_peered":"2023-11-26T07:22:23.151818+0000","last_clean":"2023-11-26T07:22:23.151818+0000","last_became_active":"2023-11-26T07:22:22.147187+0000","last_became_peered":"2023-11-26T07:22:22.147187+0000","last_unstale":"2023-11-26T07:22:23.151818+0000","last_undegraded":"2023-11-26T07:22:23.151818+0000","last_fullsized":"2023-11-26T07:22:23.151818+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:07:34.009756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.139893+0000","last_change":"2023-11-26T07:22:23.153837+0000","last_active":"2023-11-26T07:22:24.139893+0000","last_peered":"2023-11-26T07:22:24.139893+0000","last_clean":"2023-11-26T07:22:24.139893+0000","last_became_active":"2023-11-26T07:22:23.153537+0000","last_became_peered":"2023-11-26T07:22:23.153537+0000","last_unstale":"2023-11-26T07:22:24.139893+0000","last_undegraded":"2023-11-26T07:22:24.139893+0000","last_fullsized":"2023-11-26T07:22:24.139893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:16:17.005822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.153966+0000","last_change":"2023-11-26T07:22:22.148994+0000","last_active":"2023-11-26T07:22:23.153966+0000","last_peered":"2023-11-26T07:22:23.153966+0000","last_clean":"2023-11-26T07:22:23.153966+0000","last_became_active":"2023-11-26T07:22:22.147943+0000","last_became_peered":"2023-11-26T07:22:22.147943+0000","last_unstale":"2023-11-26T07:22:23.153966+0000","last_undegraded":"2023-11-26T07:22:23.153966+0000","last_fullsized":"2023-11-26T07:22:23.153966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:47:08.727890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461752+0000","last_change":"2023-11-26T07:22:23.160348+0000","last_active":"2023-11-26T07:22:24.461752+0000","last_peered":"2023-11-26T07:22:24.461752+0000","last_clean":"2023-11-26T07:22:24.461752+0000","last_became_active":"2023-11-26T07:22:23.159979+0000","last_became_peered":"2023-11-26T07:22:23.159979+0000","last_unstale":"2023-11-26T07:22:24.461752+0000","last_undegraded":"2023-11-26T07:22:24.461752+0000","last_fullsized":"2023-11-26T07:22:24.461752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:33:40.677994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.140309+0000","last_change":"2023-11-26T07:22:22.139465+0000","last_active":"2023-11-26T07:22:23.140309+0000","last_peered":"2023-11-26T07:22:23.140309+0000","last_clean":"2023-11-26T07:22:23.140309+0000","last_became_active":"2023-11-26T07:22:22.139180+0000","last_became_peered":"2023-11-26T07:22:22.139180+0000","last_unstale":"2023-11-26T07:22:23.140309+0000","last_undegraded":"2023-11-26T07:22:23.140309+0000","last_fullsized":"2023-11-26T07:22:23.140309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:19:47.928556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143222+0000","last_change":"2023-11-26T07:22:23.157400+0000","last_active":"2023-11-26T07:22:24.143222+0000","last_peered":"2023-11-26T07:22:24.143222+0000","last_clean":"2023-11-26T07:22:24.143222+0000","last_became_active":"2023-11-26T07:22:23.156768+0000","last_became_peered":"2023-11-26T07:22:23.156768+0000","last_unstale":"2023-11-26T07:22:24.143222+0000","last_undegraded":"2023-11-26T07:22:24.143222+0000","last_fullsized":"2023-11-26T07:22:24.143222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:07.939507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.188783+0000","last_change":"2023-11-26T07:22:22.146130+0000","last_active":"2023-11-26T07:22:23.188783+0000","last_peered":"2023-11-26T07:22:23.188783+0000","last_clean":"2023-11-26T07:22:23.188783+0000","last_became_active":"2023-11-26T07:22:22.144230+0000","last_became_peered":"2023-11-26T07:22:22.144230+0000","last_unstale":"2023-11-26T07:22:23.188783+0000","last_undegraded":"2023-11-26T07:22:23.188783+0000","last_fullsized":"2023-11-26T07:22:23.188783+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:21:24.263764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462231+0000","last_change":"2023-11-26T07:22:23.161644+0000","last_active":"2023-11-26T07:22:24.462231+0000","last_peered":"2023-11-26T07:22:24.462231+0000","last_clean":"2023-11-26T07:22:24.462231+0000","last_became_active":"2023-11-26T07:22:23.161336+0000","last_became_peered":"2023-11-26T07:22:23.161336+0000","last_unstale":"2023-11-26T07:22:24.462231+0000","last_undegraded":"2023-11-26T07:22:24.462231+0000","last_fullsized":"2023-11-26T07:22:24.462231+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:55:01.934153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.152180+0000","last_change":"2023-11-26T07:22:22.145175+0000","last_active":"2023-11-26T07:22:23.152180+0000","last_peered":"2023-11-26T07:22:23.152180+0000","last_clean":"2023-11-26T07:22:23.152180+0000","last_became_active":"2023-11-26T07:22:22.144917+0000","last_became_peered":"2023-11-26T07:22:22.144917+0000","last_unstale":"2023-11-26T07:22:23.152180+0000","last_undegraded":"2023-11-26T07:22:23.152180+0000","last_fullsized":"2023-11-26T07:22:23.152180+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:28:57.303446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.144077+0000","last_change":"2023-11-26T07:22:23.158894+0000","last_active":"2023-11-26T07:22:24.144077+0000","last_peered":"2023-11-26T07:22:24.144077+0000","last_clean":"2023-11-26T07:22:24.144077+0000","last_became_active":"2023-11-26T07:22:23.158534+0000","last_became_peered":"2023-11-26T07:22:23.158534+0000","last_unstale":"2023-11-26T07:22:24.144077+0000","last_undegraded":"2023-11-26T07:22:24.144077+0000","last_fullsized":"2023-11-26T07:22:24.144077+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:03:00.736312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.140267+0000","last_change":"2023-11-26T07:22:22.143708+0000","last_active":"2023-11-26T07:22:23.140267+0000","last_peered":"2023-11-26T07:22:23.140267+0000","last_clean":"2023-11-26T07:22:23.140267+0000","last_became_active":"2023-11-26T07:22:22.143395+0000","last_became_peered":"2023-11-26T07:22:22.143395+0000","last_unstale":"2023-11-26T07:22:23.140267+0000","last_undegraded":"2023-11-26T07:22:23.140267+0000","last_fullsized":"2023-11-26T07:22:23.140267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:09:43.742463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.142964+0000","last_change":"2023-11-26T07:22:23.159788+0000","last_active":"2023-11-26T07:22:24.142964+0000","last_peered":"2023-11-26T07:22:24.142964+0000","last_clean":"2023-11-26T07:22:24.142964+0000","last_became_active":"2023-11-26T07:22:23.159061+0000","last_became_peered":"2023-11-26T07:22:23.159061+0000","last_unstale":"2023-11-26T07:22:24.142964+0000","last_undegraded":"2023-11-26T07:22:24.142964+0000","last_fullsized":"2023-11-26T07:22:24.142964+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:31.249944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.140932+0000","last_change":"2023-11-26T07:22:22.139927+0000","last_active":"2023-11-26T07:22:23.140932+0000","last_peered":"2023-11-26T07:22:23.140932+0000","last_clean":"2023-11-26T07:22:23.140932+0000","last_became_active":"2023-11-26T07:22:22.139628+0000","last_became_peered":"2023-11-26T07:22:22.139628+0000","last_unstale":"2023-11-26T07:22:23.140932+0000","last_undegraded":"2023-11-26T07:22:23.140932+0000","last_fullsized":"2023-11-26T07:22:23.140932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:23:29.686183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461832+0000","last_change":"2023-11-26T07:22:23.152817+0000","last_active":"2023-11-26T07:22:24.461832+0000","last_peered":"2023-11-26T07:22:24.461832+0000","last_clean":"2023-11-26T07:22:24.461832+0000","last_became_active":"2023-11-26T07:22:23.151586+0000","last_became_peered":"2023-11-26T07:22:23.151586+0000","last_unstale":"2023-11-26T07:22:24.461832+0000","last_undegraded":"2023-11-26T07:22:24.461832+0000","last_fullsized":"2023-11-26T07:22:24.461832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:34:15.445163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.145784+0000","last_change":"2023-11-26T07:22:22.146618+0000","last_active":"2023-11-26T07:22:23.145784+0000","last_peered":"2023-11-26T07:22:23.145784+0000","last_clean":"2023-11-26T07:22:23.145784+0000","last_became_active":"2023-11-26T07:22:22.146214+0000","last_became_peered":"2023-11-26T07:22:22.146214+0000","last_unstale":"2023-11-26T07:22:23.145784+0000","last_undegraded":"2023-11-26T07:22:23.145784+0000","last_fullsized":"2023-11-26T07:22:23.145784+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:58:34.847257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461683+0000","last_change":"2023-11-26T07:22:23.158035+0000","last_active":"2023-11-26T07:22:24.461683+0000","last_peered":"2023-11-26T07:22:24.461683+0000","last_clean":"2023-11-26T07:22:24.461683+0000","last_became_active":"2023-11-26T07:22:23.157737+0000","last_became_peered":"2023-11-26T07:22:23.157737+0000","last_unstale":"2023-11-26T07:22:24.461683+0000","last_undegraded":"2023-11-26T07:22:24.461683+0000","last_fullsized":"2023-11-26T07:22:24.461683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:54:55.059095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.141972+0000","last_change":"2023-11-26T07:22:22.148139+0000","last_active":"2023-11-26T07:22:23.141972+0000","last_peered":"2023-11-26T07:22:23.141972+0000","last_clean":"2023-11-26T07:22:23.141972+0000","last_became_active":"2023-11-26T07:22:22.147844+0000","last_became_peered":"2023-11-26T07:22:22.147844+0000","last_unstale":"2023-11-26T07:22:23.141972+0000","last_undegraded":"2023-11-26T07:22:23.141972+0000","last_fullsized":"2023-11-26T07:22:23.141972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:17.813279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.461497+0000","last_change":"2023-11-26T07:22:23.169021+0000","last_active":"2023-11-26T07:22:24.461497+0000","last_peered":"2023-11-26T07:22:24.461497+0000","last_clean":"2023-11-26T07:22:24.461497+0000","last_became_active":"2023-11-26T07:22:23.168529+0000","last_became_peered":"2023-11-26T07:22:23.168529+0000","last_unstale":"2023-11-26T07:22:24.461497+0000","last_undegraded":"2023-11-26T07:22:24.461497+0000","last_fullsized":"2023-11-26T07:22:24.461497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:26:35.758938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.139641+0000","last_change":"2023-11-26T07:22:22.140709+0000","last_active":"2023-11-26T07:22:23.139641+0000","last_peered":"2023-11-26T07:22:23.139641+0000","last_clean":"2023-11-26T07:22:23.139641+0000","last_became_active":"2023-11-26T07:22:22.140379+0000","last_became_peered":"2023-11-26T07:22:22.140379+0000","last_unstale":"2023-11-26T07:22:23.139641+0000","last_undegraded":"2023-11-26T07:22:23.139641+0000","last_fullsized":"2023-11-26T07:22:23.139641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:09:22.927191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.143724+0000","last_change":"2023-11-26T07:22:23.163722+0000","last_active":"2023-11-26T07:22:24.143724+0000","last_peered":"2023-11-26T07:22:24.143724+0000","last_clean":"2023-11-26T07:22:24.143724+0000","last_became_active":"2023-11-26T07:22:23.163046+0000","last_became_peered":"2023-11-26T07:22:23.163046+0000","last_unstale":"2023-11-26T07:22:24.143724+0000","last_undegraded":"2023-11-26T07:22:24.143724+0000","last_fullsized":"2023-11-26T07:22:24.143724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:35:19.658117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.142323+0000","last_change":"2023-11-26T07:22:22.148901+0000","last_active":"2023-11-26T07:22:23.142323+0000","last_peered":"2023-11-26T07:22:23.142323+0000","last_clean":"2023-11-26T07:22:23.142323+0000","last_became_active":"2023-11-26T07:22:22.148566+0000","last_became_peered":"2023-11-26T07:22:22.148566+0000","last_unstale":"2023-11-26T07:22:23.142323+0000","last_undegraded":"2023-11-26T07:22:23.142323+0000","last_fullsized":"2023-11-26T07:22:23.142323+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:09:43.742463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-26T07:22:24.462873+0000","last_change":"2023-11-26T07:22:23.163774+0000","last_active":"2023-11-26T07:22:24.462873+0000","last_peered":"2023-11-26T07:22:24.462873+0000","last_clean":"2023-11-26T07:22:24.462873+0000","last_became_active":"2023-11-26T07:22:23.161329+0000","last_became_peered":"2023-11-26T07:22:23.161329+0000","last_unstale":"2023-11-26T07:22:24.462873+0000","last_undegraded":"2023-11-26T07:22:24.462873+0000","last_fullsized":"2023-11-26T07:22:24.462873+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:22.119714+0000","last_clean_scrub_stamp":"2023-11-26T07:22:22.119714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:40:50.539193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.188629+0000","last_change":"2023-11-26T07:22:22.152219+0000","last_active":"2023-11-26T07:22:23.188629+0000","last_peered":"2023-11-26T07:22:23.188629+0000","last_clean":"2023-11-26T07:22:23.188629+0000","last_became_active":"2023-11-26T07:22:22.151927+0000","last_became_peered":"2023-11-26T07:22:22.151927+0000","last_unstale":"2023-11-26T07:22:23.188629+0000","last_undegraded":"2023-11-26T07:22:23.188629+0000","last_fullsized":"2023-11-26T07:22:23.188629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:22:21.112577+0000","last_clean_scrub_stamp":"2023-11-26T07:22:21.112577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:33:39.670858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"69'128","reported_seq":253,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-11-26T07:22:23.150517+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:22:23.150517+0000","last_peered":"2023-11-26T07:22:23.150517+0000","last_clean":"2023-11-26T07:22:23.150517+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:22:23.150517+0000","last_undegraded":"2023-11-26T07:22:23.150517+0000","last_fullsized":"2023-11-26T07:22:23.150517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1581056,"data_stored":1011560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68000000000000005,"15min":0.68300000000000005},"min":{"1min":0.56599999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88400000000000001,"5min":0.995,"15min":0.995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84799999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68000000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.48499999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.92500000000000004,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69599999999999995,"15min":0.71399999999999997},"min":{"1min":0.56899999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.84499999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.70399999999999996,"15min":0.70699999999999996},"min":{"1min":0.58999999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92500000000000004,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.50600000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":1.016,"15min":1.016},"last":0.64700000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.73999999999999999},"min":{"1min":0.56200000000000006,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.85099999999999998,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.625},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":0.81899999999999995},"min":{"1min":0.68100000000000005,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93500000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.80400000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.71099999999999997},"min":{"1min":0.47199999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.91800000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.373,"15min":0.373},"max":{"1min":0.90800000000000003,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.49199999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73299999999999998,"15min":0.77500000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95899999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78700000000000003,"15min":0.84899999999999998},"min":{"1min":0.58699999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.91500000000000004,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.76300000000000001},"min":{"1min":0.499,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93400000000000005,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.82999999999999996},"min":{"1min":0.53100000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.012,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":995328,"data_stored":421816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":0.86699999999999999,"15min":0.78100000000000003},"min":{"1min":0.54800000000000004,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":17.222000000000001,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":3.4020000000000001,"5min":1.246,"15min":0.91500000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":54.737000000000002,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.67300000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73399999999999999,"15min":0.75},"min":{"1min":0.40400000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.1880000000000002,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":6.6689999999999996,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.56699999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.79000000000000004,"15min":0.76400000000000001},"min":{"1min":0.439,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":6.8479999999999999,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.72499999999999998,"15min":0.68899999999999995},"min":{"1min":0.57099999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.0170000000000003,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.71899999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":0.81299999999999994,"15min":0.751},"min":{"1min":0.499,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.4870000000000001,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.601,"5min":0.94399999999999995,"15min":0.874},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":18.77,"5min":18.77,"15min":18.77},"last":0.77300000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.53100000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.71899999999999997},"min":{"1min":0.46800000000000003,"5min":0.44,"15min":0.44},"max":{"1min":1.121,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.67400000000000004,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.82999999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.75900000000000001,"15min":0.78900000000000003},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.497},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.77900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.74199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":422756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.55800000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.97799999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.778},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.143}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.68600000000000005},"min":{"1min":0.58099999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":1.081},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67800000000000005,"15min":0.69199999999999995},"min":{"1min":0.45200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.823}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":1.119}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64700000000000002,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.105,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.105},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.77100000000000002},"min":{"1min":0.65500000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.3360000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.625,"15min":0.67200000000000004},"min":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.9059999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.268}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.755,"15min":0.76000000000000001},"min":{"1min":0.39900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":2.0830000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.754,"15min":0.80700000000000005},"min":{"1min":0.48499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.861}]}]},{"osd":4,"up_from":13,"seq":55834574915,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":983040,"data_stored":421122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.39300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.84899999999999998,"5min":1.016,"15min":1.016},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65200000000000002,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93100000000000005,"5min":1.26,"15min":1.26},"last":0.74399999999999999}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95099999999999996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69999999999999996,"15min":0.71199999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90200000000000002,"5min":1.077,"15min":1.077},"last":0.67800000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92000000000000004,"5min":1.302,"15min":1.302},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.753},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91100000000000003,"5min":1.181,"15min":1.181},"last":0.52500000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.747},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97999999999999998,"5min":1.702,"15min":1.702},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.78700000000000003},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0409999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.60699999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.65000000000000002,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.999,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78200000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70499999999999996,"15min":0.72999999999999998},"min":{"1min":0.42199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97099999999999997,"5min":1.129,"15min":1.129},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.754,"15min":0.78800000000000003},"min":{"1min":0.56299999999999994,"5min":0.501,"15min":0.501},"max":{"1min":0.94799999999999995,"5min":1.012,"15min":1.012},"last":0.71999999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88300000000000001,"5min":1.016,"15min":1.016},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":0.81200000000000006},"min":{"1min":0.56499999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.0980000000000001,"5min":1.242,"15min":1.242},"last":0.76600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":422734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48299999999999998,"15min":0.47699999999999998},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72999999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.48999999999999999},"min":{"1min":0.375,"5min":0.371,"15min":0.371},"max":{"1min":0.68999999999999995,"5min":1.294,"15min":1.294},"last":0.44}]},{"osd":1,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.45500000000000002,"15min":0.44600000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.66900000000000004,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.35799999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.38,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81799999999999995,"5min":1.456,"15min":1.456},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.48999999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73699999999999999,"15min":0.72899999999999998},"min":{"1min":0.52900000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.212,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.77100000000000002,"15min":0.77400000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2509999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77000000000000002,"15min":0.78000000000000003},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.2649999999999999,"5min":1.395,"15min":1.395},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1080000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.623}]},{"osd":6,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.64700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.333,"15min":1.333},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80200000000000005,"15min":0.81000000000000005},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.228,"5min":1.347,"15min":1.347},"last":0.753}]},{"osd":7,"last update":"Sun Nov 26 07:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79600000000000004,"15min":0.80100000000000005},"min":{"1min":0.57499999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.167,"5min":1.397,"15min":1.397},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.56399999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1859999999999999,"5min":1.363,"15min":1.363},"last":0.64400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":987136,"data_stored":421704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56000000000000005,"15min":0.64800000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.65800000000000003,"5min":3.931,"15min":3.931},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.63100000000000001,"15min":0.71099999999999997},"min":{"1min":0.39800000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.44}]},{"osd":1,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60199999999999998,"15min":0.69199999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.95499999999999996,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.621,"15min":0.69299999999999995},"min":{"1min":0.36499999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":2.7650000000000001,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.49299999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.34200000000000003,"5min":0.318,"15min":0.318},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.751,"15min":0.90300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.98899999999999999,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.85199999999999998,"15min":0.92900000000000005},"min":{"1min":0.57699999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.1240000000000001,"5min":4.431,"15min":4.431},"last":0.621},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85699999999999998,"15min":0.91100000000000003},"min":{"1min":0.57199999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.274,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.73399999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.61199999999999999,"5min":0.5,"15min":0.5},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.89400000000000002,"15min":0.96799999999999997},"min":{"1min":0.61799999999999999,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8069999999999999,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":0.98399999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.222,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.82599999999999996,"15min":0.82799999999999996},"min":{"1min":0.59999999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.753}]},{"osd":7,"last update":"Sun Nov 26 07:21:25 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.93899999999999995,"15min":0.995},"min":{"1min":0.621,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.4740000000000002,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.052,"5min":1.609,"15min":1.609},"last":0.63500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":421316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.56499999999999995,"5min":1.034,"15min":1.034},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.41399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68999999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.43099999999999999,"15min":0.45100000000000001},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65200000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64400000000000002,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54900000000000004,"15min":0.57899999999999996},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68000000000000005,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.76600000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.52100000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0860000000000001,"5min":1.143,"15min":1.143},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0449999999999999,"5min":1.216,"15min":1.216},"last":0.872}]},{"osd":5,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.171,"5min":1.262,"15min":1.262},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.746,"15min":0.748},"min":{"1min":0.57099999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.343,"5min":1.54,"15min":1.54},"last":0.81599999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76700000000000002,"15min":0.73899999999999999},"min":{"1min":0.59699999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.77900000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.254,"5min":1.333,"15min":1.333},"last":0.90400000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.751,"15min":0.75900000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2090000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.751,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.85299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":1010505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.43099999999999999,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.57899999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.47999999999999998,"15min":0.47599999999999998},"min":{"1min":0.35599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.63800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.56399999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.622,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48799999999999999,"15min":0.47799999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73799999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.73799999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.56299999999999994,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0289999999999999,"5min":1.365,"15min":1.365},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.80900000000000005},"min":{"1min":0.51300000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0980000000000001,"5min":1.181,"15min":1.181},"last":0.93300000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90300000000000002,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.76900000000000002},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.147,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.78500000000000003,"15min":0.80300000000000005},"min":{"1min":0.56100000000000005,"5min":0.503,"15min":0.503},"max":{"1min":0.91700000000000004,"5min":1.323,"15min":1.323},"last":0.71299999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.089,"5min":1.123,"15min":1.123},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.82799999999999996,"15min":0.81799999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.111,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.88400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:22:32.222 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:22:32.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:22:32.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4567f01640 1 Processor -- start 2023-11-26T07:22:32.407 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4567f01640 1 -- start start 2023-11-26T07:22:32.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4567f01640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4560159f00 0x7f45600ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4567f01640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4560070890 0x7f4560070d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4567f01640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4560072950 con 0x7f45600ae680 2023-11-26T07:22:32.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4567f01640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456006e2b0 con 0x7f4560159f00 2023-11-26T07:22:32.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4567f01640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456006c6f0 con 0x7f4560070890 2023-11-26T07:22:32.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f456549c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f45600ae680 0x7f45600cd3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60522/0 (socket says 172.21.15.31:60522) 2023-11-26T07:22:32.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f456549c640 1 -- 172.21.15.31:0/3010668744 learned_addr learned my addr 172.21.15.31:0/3010668744 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:32.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f456649e640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4560070890 0x7f4560070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4565c9d640 1 --2- 172.21.15.31:0/3010668744 >> v2:172.21.15.97:3300/0 conn(0x7f4560159f00 0x7f45600ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809887658 0 0) 0x7f4560072950 con 0x7f45600ae680 2023-11-26T07:22:32.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f454c003530 con 0x7f45600ae680 2023-11-26T07:22:32.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3751469498 0 0) 0x7f454c003530 con 0x7f45600ae680 2023-11-26T07:22:32.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4560070890 msgr2=0x7f4560070d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:32.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4560070890 0x7f4560070d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 >> v2:172.21.15.97:3300/0 conn(0x7f4560159f00 msgr2=0x7f45600ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 --2- 172.21.15.31:0/3010668744 >> v2:172.21.15.97:3300/0 conn(0x7f4560159f00 0x7f45600ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456006e550 con 0x7f45600ae680 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f4550001680 con 0x7f45600ae680 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f456649e640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4560070890 0x7f4560070d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 >> v1:172.21.15.31:6789/0 conn(0x7f45600ae680 legacy=0x7f45600cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c0039b0 0x7f454c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f454c0047d0 0x7f454c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4560070890 0x7f454c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456006e2b0 con 0x7f454c0047d0 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456006c6f0 con 0x7f454c0039b0 2023-11-26T07:22:32.412 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.405+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c00b990 con 0x7f4560070890 2023-11-26T07:22:32.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f4565c9d640 1 --2- 172.21.15.31:0/3010668744 >> v2:172.21.15.97:3300/0 conn(0x7f4560159f00 0x7f45600ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:32.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f456549c640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f454c0047d0 0x7f454c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f456549c640 1 -- 172.21.15.31:0/3010668744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4560070890 msgr2=0x7f454c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:32.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f456549c640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4560070890 0x7f454c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f456549c640 1 -- 172.21.15.31:0/3010668744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c0039b0 msgr2=0x7f454c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:32.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f456549c640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f454c0039b0 0x7f454c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f456549c640 1 -- 172.21.15.31:0/3010668744 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558009a70 con 0x7f454c0047d0 2023-11-26T07:22:32.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f456549c640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f454c0047d0 0x7f454c004c50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f455000c270 tx=0x7f455001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6103b1e2f33f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:32.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500058e0 con 0x7f454c0047d0 2023-11-26T07:22:32.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4550005a40 con 0x7f454c0047d0 2023-11-26T07:22:32.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.409+0000 7f4564c9b640 1 -- 172.21.15.31:0/3010668744 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550005df0 con 0x7f454c0047d0 2023-11-26T07:22:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 -- 172.21.15.31:0/3010668744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f454c0047d0 msgr2=0x7f454c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 --2- 172.21.15.31:0/3010668744 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f454c0047d0 0x7f454c004c50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f455000c270 tx=0x7f455001f390 comp rx=0 tx=0).stop 2023-11-26T07:22:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f456649e640 1 -- 172.21.15.31:0/3010668744 reap_dead start 2023-11-26T07:22:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 -- 172.21.15.31:0/3010668744 shutdown_connections 2023-11-26T07:22:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 -- 172.21.15.31:0/3010668744 >> 172.21.15.31:0/3010668744 conn(0x7f45600b0cb0 msgr2=0x7f45600b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 -- 172.21.15.31:0/3010668744 shutdown_connections 2023-11-26T07:22:32.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 -- 172.21.15.31:0/3010668744 wait complete. 2023-11-26T07:22:32.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 Processor -- start 2023-11-26T07:22:32.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 -- start start 2023-11-26T07:22:32.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f454c0047d0 0x7f456015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f456014ef60 0x7f4560150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4560099c50 0x7f456009c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456014f460 con 0x7f456014ef60 2023-11-26T07:22:32.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456014f740 con 0x7f4560099c50 2023-11-26T07:22:32.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4567f01640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456009bb80 con 0x7f454c0047d0 2023-11-26T07:22:32.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f456549c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f456014ef60 0x7f4560150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f456549c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f456014ef60 0x7f4560150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38004/0 (socket says 172.21.15.31:38004) 2023-11-26T07:22:32.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f456549c640 1 -- 172.21.15.31:0/2526241671 learned_addr learned my addr 172.21.15.31:0/2526241671 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:32.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f4565c9d640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f454c0047d0 0x7f456015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f456649e640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4560099c50 0x7f456009c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f456549c640 1 -- 172.21.15.31:0/2526241671 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f454c0047d0 msgr2=0x7f456015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f456549c640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f454c0047d0 0x7f456015a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f456549c640 1 -- 172.21.15.31:0/2526241671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4560099c50 msgr2=0x7f456009c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f456549c640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4560099c50 0x7f456009c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.413+0000 7f456549c640 1 -- 172.21.15.31:0/2526241671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456009be60 con 0x7f456014ef60 2023-11-26T07:22:32.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f456549c640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f456014ef60 0x7f4560150690 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f455000c270 tx=0x7f454c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:32.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f45567fc640 1 -- 172.21.15.31:0/2526241671 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500b7070 con 0x7f456014ef60 2023-11-26T07:22:32.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f45567fc640 1 -- 172.21.15.31:0/2526241671 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4550005880 con 0x7f456014ef60 2023-11-26T07:22:32.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f45567fc640 1 -- 172.21.15.31:0/2526241671 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550001660 con 0x7f456014ef60 2023-11-26T07:22:32.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f4567f01640 1 -- 172.21.15.31:0/2526241671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456009b830 con 0x7f456014ef60 2023-11-26T07:22:32.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f4567f01640 1 -- 172.21.15.31:0/2526241671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4560150da0 con 0x7f456014ef60 2023-11-26T07:22:32.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f4567f01640 1 -- 172.21.15.31:0/2526241671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45600671f0 con 0x7f456014ef60 2023-11-26T07:22:32.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f45567fc640 1 -- 172.21.15.31:0/2526241671 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f4550007030 con 0x7f456014ef60 2023-11-26T07:22:32.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f45567fc640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45440665c0 0x7f4544068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f45567fc640 1 -- 172.21.15.31:0/2526241671 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4550120900 con 0x7f456014ef60 2023-11-26T07:22:32.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.417+0000 7f4565c9d640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45440665c0 0x7f4544068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.421+0000 7f4565c9d640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45440665c0 0x7f4544068a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f455c009c00 tx=0x7f455c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:32.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.421+0000 7f45567fc640 1 -- 172.21.15.31:0/2526241671 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f45500edd00 con 0x7f456014ef60 2023-11-26T07:22:32.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.633+0000 7f4567f01640 1 -- 172.21.15.31:0/2526241671 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f456009bcc0 con 0x7f456014ef60 2023-11-26T07:22:32.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f45567fc640 1 -- 172.21.15.31:0/2526241671 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7f456009bcc0 con 0x7f456014ef60 2023-11-26T07:22:32.639 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-11-26T07:22:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA== 2023-11-26T07:22:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2023-11-26T07:22:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2023-11-26T07:22:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:22:32.640 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-11-26T07:22:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 -- 172.21.15.31:0/2526241671 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45440665c0 msgr2=0x7f4544068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45440665c0 0x7f4544068a80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f455c009c00 tx=0x7f455c008040 comp rx=0 tx=0).stop 2023-11-26T07:22:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 -- 172.21.15.31:0/2526241671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f456014ef60 msgr2=0x7f4560150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f456014ef60 0x7f4560150690 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f455000c270 tx=0x7f454c005800 comp rx=0 tx=0).stop 2023-11-26T07:22:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 -- 172.21.15.31:0/2526241671 shutdown_connections 2023-11-26T07:22:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45440665c0 0x7f4544068a80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4560099c50 0x7f456009c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f456014ef60 0x7f4560150690 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 --2- 172.21.15.31:0/2526241671 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f454c0047d0 0x7f456015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 -- 172.21.15.31:0/2526241671 >> 172.21.15.31:0/2526241671 conn(0x7f45600b0cb0 msgr2=0x7f45600b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 -- 172.21.15.31:0/2526241671 shutdown_connections 2023-11-26T07:22:32.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.637+0000 7f4567f01640 1 -- 172.21.15.31:0/2526241671 wait complete. 2023-11-26T07:22:32.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:32.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.801+0000 7f6168291640 1 Processor -- start 2023-11-26T07:22:32.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.801+0000 7f6168291640 1 -- start start 2023-11-26T07:22:32.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.801+0000 7f6168291640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6160144c80 0x7f61600abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.801+0000 7f6168291640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6160070df0 0x7f616006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.801+0000 7f6168291640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6160145060 con 0x7f61600ac460 2023-11-26T07:22:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.801+0000 7f6168291640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6160071fd0 con 0x7f6160070df0 2023-11-26T07:22:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.801+0000 7f6168291640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61600ac8c0 con 0x7f6160144c80 2023-11-26T07:22:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.801+0000 7f616682e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6160070df0 0x7f616006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.801+0000 7f616582c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f61600ac460 0x7f61600706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60526/0 (socket says 172.21.15.31:60526) 2023-11-26T07:22:32.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.801+0000 7f616582c640 1 -- 172.21.15.31:0/484006518 learned_addr learned my addr 172.21.15.31:0/484006518 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f616602d640 1 --2- 172.21.15.31:0/484006518 >> v2:172.21.15.97:3300/0 conn(0x7f6160144c80 0x7f61600abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f616502b640 1 -- 172.21.15.31:0/484006518 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408768760 0 0) 0x7f6160145060 con 0x7f61600ac460 2023-11-26T07:22:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f616682e640 1 -- 172.21.15.31:0/484006518 >> v2:172.21.15.97:3300/0 conn(0x7f6160144c80 msgr2=0x7f61600abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f616682e640 1 --2- 172.21.15.31:0/484006518 >> v2:172.21.15.97:3300/0 conn(0x7f6160144c80 0x7f61600abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f616682e640 1 -- 172.21.15.31:0/484006518 >> v1:172.21.15.31:6789/0 conn(0x7f61600ac460 legacy=0x7f61600706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f616682e640 1 -- 172.21.15.31:0/484006518 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6160071450 con 0x7f6160070df0 2023-11-26T07:22:32.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f616682e640 1 --2- 172.21.15.31:0/484006518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 0x7f616006c910 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f616006eb80 tx=0x7f615801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5ca535f953cc06 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:32.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f616502b640 1 -- 172.21.15.31:0/484006518 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61580a58c0 con 0x7f6160070df0 2023-11-26T07:22:32.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f616502b640 1 -- 172.21.15.31:0/484006518 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61580a5a20 con 0x7f6160070df0 2023-11-26T07:22:32.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f616502b640 1 -- 172.21.15.31:0/484006518 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61580a5db0 con 0x7f6160070df0 2023-11-26T07:22:32.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f6168291640 1 -- 172.21.15.31:0/484006518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 msgr2=0x7f616006c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.805+0000 7f6168291640 1 --2- 172.21.15.31:0/484006518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 0x7f616006c910 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f616006eb80 tx=0x7f615801c910 comp rx=0 tx=0).stop 2023-11-26T07:22:32.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 -- 172.21.15.31:0/484006518 shutdown_connections 2023-11-26T07:22:32.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 --2- 172.21.15.31:0/484006518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 0x7f616006c910 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 --2- 172.21.15.31:0/484006518 >> v2:172.21.15.97:3300/0 conn(0x7f6160144c80 0x7f61600abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 -- 172.21.15.31:0/484006518 >> 172.21.15.31:0/484006518 conn(0x7f61600b0c40 msgr2=0x7f61600b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:32.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 -- 172.21.15.31:0/484006518 shutdown_connections 2023-11-26T07:22:32.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 -- 172.21.15.31:0/484006518 wait complete. 2023-11-26T07:22:32.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 Processor -- start 2023-11-26T07:22:32.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 -- start start 2023-11-26T07:22:32.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 0x7f616015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.813 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61600ac460 0x7f6160141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6160144c80 0x7f6160143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616009c190 con 0x7f6160070df0 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616009c3a0 con 0x7f6160144c80 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616009c680 con 0x7f61600ac460 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616602d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 0x7f616015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616602d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 0x7f616015a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38022/0 (socket says 172.21.15.31:38022) 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616602d640 1 -- 172.21.15.31:0/477344447 learned_addr learned my addr 172.21.15.31:0/477344447 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616682e640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6160144c80 0x7f6160143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616582c640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61600ac460 0x7f6160141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616602d640 1 -- 172.21.15.31:0/477344447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61600ac460 msgr2=0x7f6160141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616602d640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61600ac460 0x7f6160141080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616602d640 1 -- 172.21.15.31:0/477344447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6160144c80 msgr2=0x7f6160143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:32.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616602d640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6160144c80 0x7f6160143730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:32.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616602d640 1 -- 172.21.15.31:0/477344447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6158009a70 con 0x7f6160070df0 2023-11-26T07:22:32.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616682e640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6160144c80 0x7f6160143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:32.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616582c640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61600ac460 0x7f6160141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:32.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f616602d640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 0x7f616015a570 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f615c00d5f0 tx=0x7f615c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:32.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6156ffd640 1 -- 172.21.15.31:0/477344447 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c0125f0 con 0x7f6160070df0 2023-11-26T07:22:32.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6156ffd640 1 -- 172.21.15.31:0/477344447 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c012750 con 0x7f6160070df0 2023-11-26T07:22:32.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6156ffd640 1 -- 172.21.15.31:0/477344447 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c012ae0 con 0x7f6160070df0 2023-11-26T07:22:32.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.809+0000 7f6168291640 1 -- 172.21.15.31:0/477344447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6160099bb0 con 0x7f6160070df0 2023-11-26T07:22:32.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.813+0000 7f6168291640 1 -- 172.21.15.31:0/477344447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61601415c0 con 0x7f6160070df0 2023-11-26T07:22:32.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.813+0000 7f6168291640 1 -- 172.21.15.31:0/477344447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61600b1ba0 con 0x7f6160070df0 2023-11-26T07:22:32.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.813+0000 7f6156ffd640 1 -- 172.21.15.31:0/477344447 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f615c012c40 con 0x7f6160070df0 2023-11-26T07:22:32.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.813+0000 7f6156ffd640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6144066590 0x7f6144068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:32.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.813+0000 7f6156ffd640 1 -- 172.21.15.31:0/477344447 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f615c0845a0 con 0x7f6160070df0 2023-11-26T07:22:32.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.813+0000 7f616582c640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6144066590 0x7f6144068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:32.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.813+0000 7f616582c640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6144066590 0x7f6144068a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f616013d190 tx=0x7f6150009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:32.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:32.817+0000 7f6156ffd640 1 -- 172.21.15.31:0/477344447 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f615c051870 con 0x7f6160070df0 2023-11-26T07:22:33.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.005+0000 7f6168291640 1 -- 172.21.15.31:0/477344447 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f616009a200 con 0x7f6160070df0 2023-11-26T07:22:33.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.005+0000 7f6156ffd640 1 -- 172.21.15.31:0/477344447 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f615c007380 con 0x7f6160070df0 2023-11-26T07:22:33.010 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:33.010 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1684629063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1684629063},{"type":"v1","addr":"172.21.15.31:6833","nonce":1684629063}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/1262063204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1262063204},{"type":"v1","addr":"172.21.15.97:6834","nonce":1262063204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6778,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/683007803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":683007803},{"type":"v1","addr":"172.21.15.97:6835","nonce":683007803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:22.526776+0000","modified":"2023-11-26T07:22:25.559499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7101":{"gid":7101,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3729395637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3729395637},{"type":"v1","addr":"172.21.15.31:6835","nonce":3729395637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:22:33.010 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 57 2023-11-26T07:22:33.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 -- 172.21.15.31:0/477344447 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6144066590 msgr2=0x7f6144068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6144066590 0x7f6144068a50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f616013d190 tx=0x7f6150009040 comp rx=0 tx=0).stop 2023-11-26T07:22:33.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 -- 172.21.15.31:0/477344447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 msgr2=0x7f616015a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 0x7f616015a570 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f615c00d5f0 tx=0x7f615c00dab0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 -- 172.21.15.31:0/477344447 shutdown_connections 2023-11-26T07:22:33.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6144066590 0x7f6144068a50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6160144c80 0x7f6160143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f61600ac460 0x7f6160141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 --2- 172.21.15.31:0/477344447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6160070df0 0x7f616015a570 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 -- 172.21.15.31:0/477344447 >> 172.21.15.31:0/477344447 conn(0x7f61600b0c40 msgr2=0x7f616015f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 -- 172.21.15.31:0/477344447 shutdown_connections 2023-11-26T07:22:33.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.009+0000 7f6168291640 1 -- 172.21.15.31:0/477344447 wait complete. 2023-11-26T07:22:33.433 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:22:22.526776+0000', 'modified': '2023-11-26T07:22:25.559499+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7101}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7101': {'gid': 7101, 'name': 'c', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/3729395637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6834', 'nonce': 3729395637}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 3729395637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:22:33.433 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:22:33.433 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.577+0000 7f5503380640 1 Processor -- start 2023-11-26T07:22:33.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.577+0000 7f5503380640 1 -- start start 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.577+0000 7f5503380640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54fc13f970 0x7f54fc06d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.577+0000 7f5503380640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc13cf40 con 0x7f54fc0cded0 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.577+0000 7f5503380640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc1400e0 con 0x7f54fc154180 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.577+0000 7f5503380640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc0cf0a0 con 0x7f54fc13f970 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f550191d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54fc13f970 0x7f54fc06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f550091b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f54fc0cded0 0x7f54fc13f260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60532/0 (socket says 172.21.15.31:60532) 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f550191d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f54fc13f970 0x7f54fc06d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38030/0 (socket says 172.21.15.31:38030) 2023-11-26T07:22:33.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f550091b640 1 -- 172.21.15.31:0/670030407 learned_addr learned my addr 172.21.15.31:0/670030407 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f550191d640 1 -- 172.21.15.31:0/670030407 >> v1:172.21.15.97:6789/0 conn(0x7f54fc154180 legacy=0x7f54fc15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f550191d640 1 -- 172.21.15.31:0/670030407 >> v1:172.21.15.31:6789/0 conn(0x7f54fc0cded0 legacy=0x7f54fc13f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f550191d640 1 -- 172.21.15.31:0/670030407 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc154560 con 0x7f54fc13f970 2023-11-26T07:22:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f550191d640 1 --2- 172.21.15.31:0/670030407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc13f970 0x7f54fc06d220 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f54fc1546f0 tx=0x7f54f401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280173544e71889 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f54f3fff640 1 -- 172.21.15.31:0/670030407 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f40a58b0 con 0x7f54fc13f970 2023-11-26T07:22:33.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f54f3fff640 1 -- 172.21.15.31:0/670030407 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f40a5a10 con 0x7f54fc13f970 2023-11-26T07:22:33.586 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.581+0000 7f54f3fff640 1 -- 172.21.15.31:0/670030407 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f40a5da0 con 0x7f54fc13f970 2023-11-26T07:22:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 -- 172.21.15.31:0/670030407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc13f970 msgr2=0x7f54fc06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 --2- 172.21.15.31:0/670030407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc13f970 0x7f54fc06d220 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f54fc1546f0 tx=0x7f54f401c870 comp rx=0 tx=0).stop 2023-11-26T07:22:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 -- 172.21.15.31:0/670030407 shutdown_connections 2023-11-26T07:22:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 --2- 172.21.15.31:0/670030407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc13f970 0x7f54fc06d220 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 -- 172.21.15.31:0/670030407 >> 172.21.15.31:0/670030407 conn(0x7f54fc071d80 msgr2=0x7f54fc06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 -- 172.21.15.31:0/670030407 shutdown_connections 2023-11-26T07:22:33.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 -- 172.21.15.31:0/670030407 wait complete. 2023-11-26T07:22:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 Processor -- start 2023-11-26T07:22:33.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 -- start start 2023-11-26T07:22:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc0cded0 0x7f54fc1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc13f970 0x7f54fc15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc154180 0x7f54fc13c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc156cf0 con 0x7f54fc0cded0 2023-11-26T07:22:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc155e20 con 0x7f54fc154180 2023-11-26T07:22:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f5503380640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc157110 con 0x7f54fc13f970 2023-11-26T07:22:33.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550111c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc0cded0 0x7f54fc1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550111c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc0cded0 0x7f54fc1558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38034/0 (socket says 172.21.15.31:38034) 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550111c640 1 -- 172.21.15.31:0/1586636207 learned_addr learned my addr 172.21.15.31:0/1586636207 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550091b640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc13f970 0x7f54fc15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550191d640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc154180 0x7f54fc13c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550111c640 1 -- 172.21.15.31:0/1586636207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc13f970 msgr2=0x7f54fc15ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550111c640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc13f970 0x7f54fc15ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550111c640 1 -- 172.21.15.31:0/1586636207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc154180 msgr2=0x7f54fc13c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550111c640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc154180 0x7f54fc13c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550111c640 1 -- 172.21.15.31:0/1586636207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f4009a70 con 0x7f54fc0cded0 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.585+0000 7f550191d640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc154180 0x7f54fc13c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:33.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f550111c640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc0cded0 0x7f54fc1558e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f54f800c3c0 tx=0x7f54f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f54f1ffb640 1 -- 172.21.15.31:0/1586636207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80052d0 con 0x7f54fc0cded0 2023-11-26T07:22:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f54f1ffb640 1 -- 172.21.15.31:0/1586636207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f8005430 con 0x7f54fc0cded0 2023-11-26T07:22:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f5503380640 1 -- 172.21.15.31:0/1586636207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc15cf60 con 0x7f54fc0cded0 2023-11-26T07:22:33.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f54f1ffb640 1 -- 172.21.15.31:0/1586636207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f8003840 con 0x7f54fc0cded0 2023-11-26T07:22:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f5503380640 1 -- 172.21.15.31:0/1586636207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc156120 con 0x7f54fc0cded0 2023-11-26T07:22:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f54f1ffb640 1 -- 172.21.15.31:0/1586636207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f54f80039a0 con 0x7f54fc0cded0 2023-11-26T07:22:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f54f1ffb640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f54e0066590 0x7f54e0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f54f1ffb640 1 -- 172.21.15.31:0/1586636207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54f80834a0 con 0x7f54fc0cded0 2023-11-26T07:22:33.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f550091b640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f54e0066590 0x7f54e0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.589+0000 7f55066f7000 1 Processor -- start 2023-11-26T07:22:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 -- start start 2023-11-26T07:22:33.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550091b640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f54e0066590 0x7f54e0068a50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0ae7a0 tx=0x7f54ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb6fc90 0x55780cb70070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb705b0 0x55780cb25de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 0x55780cb26700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780c990130 con 0x55780cb705b0 2023-11-26T07:22:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780c97d9a0 con 0x55780cb26320 2023-11-26T07:22:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55780c97aed0 con 0x55780cb6fc90 2023-11-26T07:22:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550091b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb705b0 0x55780cb25de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550091b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb705b0 0x55780cb25de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38038/0 (socket says 172.21.15.31:38038) 2023-11-26T07:22:33.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550091b640 1 -- 172.21.15.31:0/1366656419 learned_addr learned my addr 172.21.15.31:0/1366656419 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550091b640 1 -- 172.21.15.31:0/1366656419 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb6fc90 msgr2=0x55780cb70070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550091b640 1 --2- 172.21.15.31:0/1366656419 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb6fc90 0x55780cb70070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550091b640 1 -- 172.21.15.31:0/1366656419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 msgr2=0x55780cb26700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550191d640 1 --2- 172.21.15.31:0/1366656419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 0x55780cb26700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550091b640 1 --2- 172.21.15.31:0/1366656419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 0x55780cb26700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550091b640 1 -- 172.21.15.31:0/1366656419 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55780c66e890 con 0x55780cb705b0 2023-11-26T07:22:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550191d640 1 --2- 172.21.15.31:0/1366656419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 0x55780cb26700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:33.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f550091b640 1 --2- 172.21.15.31:0/1366656419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb705b0 0x55780cb25de0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00f850 tx=0x7f54ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38d432eb038fc4e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f54e6ffd640 1 -- 172.21.15.31:0/1366656419 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00fe90 con 0x55780cb705b0 2023-11-26T07:22:33.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f54e6ffd640 1 -- 172.21.15.31:0/1366656419 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec014be0 con 0x55780cb705b0 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f54e6ffd640 1 -- 172.21.15.31:0/1366656419 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec020410 con 0x55780cb705b0 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 -- 172.21.15.31:0/1366656419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb705b0 msgr2=0x55780cb25de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 --2- 172.21.15.31:0/1366656419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb705b0 0x55780cb25de0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00f850 tx=0x7f54ec012000 comp rx=0 tx=0).stop 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 -- 172.21.15.31:0/1366656419 shutdown_connections 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 --2- 172.21.15.31:0/1366656419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 0x55780cb26700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 --2- 172.21.15.31:0/1366656419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb705b0 0x55780cb25de0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 --2- 172.21.15.31:0/1366656419 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb6fc90 0x55780cb70070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 -- 172.21.15.31:0/1366656419 >> 172.21.15.31:0/1366656419 conn(0x55780cb222d0 msgr2=0x55780cb246c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 -- 172.21.15.31:0/1366656419 shutdown_connections 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 -- 172.21.15.31:0/1366656419 wait complete. 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 -- start start 2023-11-26T07:22:33.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.593+0000 7f55066f7000 1 Processor -- start 2023-11-26T07:22:33.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 0x55780cb2ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb2b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb705b0 0x55780cb3e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780c9767c0 con 0x55780cb6fc90 2023-11-26T07:22:33.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780c92ebf0 con 0x55780cb26320 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55780c934a90 con 0x55780cb705b0 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 10 client.0 _ll_register_callbacks cb 0x55780cb113c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550091b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb2b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550091b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb2b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38054/0 (socket says 172.21.15.31:38054) 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550091b640 1 -- 172.21.15.31:0/1667958158 learned_addr learned my addr 172.21.15.31:0/1667958158 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550111c640 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 0x55780cb2ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550191d640 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb705b0 0x55780cb3e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550091b640 1 -- 172.21.15.31:0/1667958158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb705b0 msgr2=0x55780cb3e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550091b640 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb705b0 0x55780cb3e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550091b640 1 -- 172.21.15.31:0/1667958158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 msgr2=0x55780cb2ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550091b640 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 0x55780cb2ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550091b640 1 -- 172.21.15.31:0/1667958158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc13cf40 con 0x55780cb6fc90 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550091b640 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb2b250 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000f70 tx=0x7f54ec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa96247b241438b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f54dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f54dbfff640 1 -- 172.21.15.31:0/1667958158 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec020c30 con 0x55780cb6fc90 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 1 -- 172.21.15.31:0/1667958158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55780c938470 con 0x55780cb6fc90 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f54dbfff640 1 -- 172.21.15.31:0/1667958158 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54ec01c040 con 0x55780cb6fc90 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f54dbfff640 1 -- 172.21.15.31:0/1667958158 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54ec020d90 con 0x55780cb6fc90 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 10 client.7275 fetch_fsmap learned FSMap version 57 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 1 -- 172.21.15.31:0/1667958158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55780c937190 con 0x55780cb6fc90 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f54dbfff640 1 -- 172.21.15.31:0/1667958158 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55780c938470 con 0x55780cb6fc90 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f54dbfff640 1 -- 172.21.15.31:0/1667958158 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f54ec012070 con 0x55780cb6fc90 2023-11-26T07:22:33.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 10 client.7275 fetch_fsmap finished waiting for FSMap version 57 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 10 client.7275 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 20 client.7275 populate_metadata read hostname 'smithi031' 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.31:6834/3729395637,v1:172.21.15.31:6835/3729395637] conn(0x55780cb3f320 0x55780cb81f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 4 client.7275 mds_command: new command op to 7101 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f55066f7000 1 -- 172.21.15.31:0/1667958158 --> [v2:172.21.15.31:6834/3729395637,v1:172.21.15.31:6835/3729395637] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55780c93bb50 con 0x55780cb3f320 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550111c640 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.31:6834/3729395637,v1:172.21.15.31:6835/3729395637] conn(0x55780cb3f320 0x55780cb81f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f550111c640 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.31:6834/3729395637,v1:172.21.15.31:6835/3729395637] conn(0x55780cb3f320 0x55780cb81f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e209c74457cf1fa6 server_cookie=51097867455b1e24 in_seq=0 out_seq=0 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.597+0000 7f54dbfff640 10 client.7275 ms_handle_connect on v2:172.21.15.31:6834/3729395637 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f54dbfff640 1 -- 172.21.15.31:0/1667958158 <== mds.0 v2:172.21.15.31:6834/3729395637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55780c93bb50 con 0x55780cb3f320 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f54dbfff640 10 client.7275 handle_command_reply: tid=0 2023-11-26T07:22:33.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 client.7275 shutdown 2023-11-26T07:22:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 -- 172.21.15.31:0/1667958158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 msgr2=0x55780cb2b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb2b250 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f54ec000f70 tx=0x7f54ec005800 comp rx=0 tx=0).stop 2023-11-26T07:22:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 -- 172.21.15.31:0/1667958158 shutdown_connections 2023-11-26T07:22:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb705b0 0x55780cb3e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.31:6834/3729395637,v1:172.21.15.31:6835/3729395637] conn(0x55780cb3f320 0x55780cb81f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb2b250 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 --2- 172.21.15.31:0/1667958158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb26320 0x55780cb2ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 -- 172.21.15.31:0/1667958158 >> 172.21.15.31:0/1667958158 conn(0x55780cb222d0 msgr2=0x55780cb246c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f54dbfff640 0 client.7275 ms_handle_reset on v2:172.21.15.31:6834/3729395637 2023-11-26T07:22:33.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 -- 172.21.15.31:0/1667958158 shutdown_connections 2023-11-26T07:22:33.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 1 -- 172.21.15.31:0/1667958158 wait complete. 2023-11-26T07:22:33.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.601+0000 7f55066f7000 20 client.7275 trim_cache size 0 max 16384 2023-11-26T07:22:33.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 Processor -- start 2023-11-26T07:22:33.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 -- start start 2023-11-26T07:22:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb26320 0x55780cb6ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb6db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 0x55780cb6e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780c62f940 con 0x55780cb6fc90 2023-11-26T07:22:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780cb2bfb0 con 0x55780cb705b0 2023-11-26T07:22:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55780c970ac0 con 0x55780cb26320 2023-11-26T07:22:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550091b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb6db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550091b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb6db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38064/0 (socket says 172.21.15.31:38064) 2023-11-26T07:22:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550091b640 1 -- 172.21.15.31:0/3399415590 learned_addr learned my addr 172.21.15.31:0/3399415590 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:33.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550111c640 1 --2- 172.21.15.31:0/3399415590 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb26320 0x55780cb6ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550191d640 1 --2- 172.21.15.31:0/3399415590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 0x55780cb6e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550091b640 1 -- 172.21.15.31:0/3399415590 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb26320 msgr2=0x55780cb6ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550091b640 1 --2- 172.21.15.31:0/3399415590 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb26320 0x55780cb6ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550091b640 1 -- 172.21.15.31:0/3399415590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 msgr2=0x55780cb6e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550091b640 1 --2- 172.21.15.31:0/3399415590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 0x55780cb6e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550091b640 1 -- 172.21.15.31:0/3399415590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55780c91d490 con 0x55780cb6fc90 2023-11-26T07:22:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550191d640 1 --2- 172.21.15.31:0/3399415590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 0x55780cb6e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:33.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f550091b640 1 --2- 172.21.15.31:0/3399415590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb6db30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f54ec001350 tx=0x7f54ec029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4a3cfcc0acef5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f54e77fe640 1 -- 172.21.15.31:0/3399415590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec02a6d0 con 0x55780cb6fc90 2023-11-26T07:22:33.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f54e77fe640 1 -- 172.21.15.31:0/3399415590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec00fe90 con 0x55780cb6fc90 2023-11-26T07:22:33.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f54e77fe640 1 -- 172.21.15.31:0/3399415590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec020c70 con 0x55780cb6fc90 2023-11-26T07:22:33.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 -- 172.21.15.31:0/3399415590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 msgr2=0x55780cb6db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 --2- 172.21.15.31:0/3399415590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb6db30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f54ec001350 tx=0x7f54ec029000 comp rx=0 tx=0).stop 2023-11-26T07:22:33.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 -- 172.21.15.31:0/3399415590 shutdown_connections 2023-11-26T07:22:33.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 --2- 172.21.15.31:0/3399415590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 0x55780cb6e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 --2- 172.21.15.31:0/3399415590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb6fc90 0x55780cb6db30 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 --2- 172.21.15.31:0/3399415590 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb26320 0x55780cb6ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 -- 172.21.15.31:0/3399415590 >> 172.21.15.31:0/3399415590 conn(0x55780cb222d0 msgr2=0x55780cb7d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 -- 172.21.15.31:0/3399415590 shutdown_connections 2023-11-26T07:22:33.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.617+0000 7f55066f7000 1 -- 172.21.15.31:0/3399415590 wait complete. 2023-11-26T07:22:33.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 -- start start 2023-11-26T07:22:33.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 Processor -- start 2023-11-26T07:22:33.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb26320 0x55780cb27a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb6fc90 0x55780cb82fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 0x55780cb834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780c93f910 con 0x55780cb26320 2023-11-26T07:22:33.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55780c9fdfa0 con 0x55780cb705b0 2023-11-26T07:22:33.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55780c97a030 con 0x55780cb6fc90 2023-11-26T07:22:33.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 10 client.0 _ll_register_callbacks cb 0x55780c9c4c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:22:33.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550111c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb26320 0x55780cb27a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550111c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb26320 0x55780cb27a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38066/0 (socket says 172.21.15.31:38066) 2023-11-26T07:22:33.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550111c640 1 -- 172.21.15.31:0/2435993109 learned_addr learned my addr 172.21.15.31:0/2435993109 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:33.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550091b640 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb6fc90 0x55780cb82fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550191d640 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 0x55780cb834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550111c640 1 -- 172.21.15.31:0/2435993109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb6fc90 msgr2=0x55780cb82fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550111c640 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb6fc90 0x55780cb82fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550111c640 1 -- 172.21.15.31:0/2435993109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 msgr2=0x55780cb834e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550111c640 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 0x55780cb834e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550111c640 1 -- 172.21.15.31:0/2435993109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55780c937190 con 0x55780cb26320 2023-11-26T07:22:33.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550091b640 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb6fc90 0x55780cb82fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:33.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550191d640 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 0x55780cb834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:33.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f550111c640 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb26320 0x55780cb27a90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f54f800cd80 tx=0x7f54f805c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9762aa1f3fcfa70 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f54dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:22:33.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f54dbfff640 1 -- 172.21.15.31:0/2435993109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80529e0 con 0x55780cb26320 2023-11-26T07:22:33.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f54dbfff640 1 -- 172.21.15.31:0/2435993109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54f8053a70 con 0x55780cb26320 2023-11-26T07:22:33.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 -- 172.21.15.31:0/2435993109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55780c975110 con 0x55780cb26320 2023-11-26T07:22:33.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f54dbfff640 1 -- 172.21.15.31:0/2435993109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54f8053c30 con 0x55780cb26320 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 10 client.7281 fetch_fsmap learned FSMap version 57 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 -- 172.21.15.31:0/2435993109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55780c66ecc0 con 0x55780cb26320 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f54dbfff640 1 -- 172.21.15.31:0/2435993109 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55780c975110 con 0x55780cb26320 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f54dbfff640 1 -- 172.21.15.31:0/2435993109 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f54f8072280 con 0x55780cb26320 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 10 client.7281 fetch_fsmap finished waiting for FSMap version 57 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 10 client.7281 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 20 client.7281 populate_metadata read hostname 'smithi031' 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.31:6834/3729395637,v1:172.21.15.31:6835/3729395637] conn(0x55780cbd0010 0x55780cbd03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 4 client.7281 mds_command: new command op to 7101 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.621+0000 7f55066f7000 1 -- 172.21.15.31:0/2435993109 --> [v2:172.21.15.31:6834/3729395637,v1:172.21.15.31:6835/3729395637] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55780c9a49a0 con 0x55780cbd0010 2023-11-26T07:22:33.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f550091b640 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.31:6834/3729395637,v1:172.21.15.31:6835/3729395637] conn(0x55780cbd0010 0x55780cbd03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f550091b640 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.31:6834/3729395637,v1:172.21.15.31:6835/3729395637] conn(0x55780cbd0010 0x55780cbd03f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386cd6c7815ee958 server_cookie=e94091d1f74c5e19 in_seq=0 out_seq=0 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f54dbfff640 10 client.7281 ms_handle_connect on v2:172.21.15.31:6834/3729395637 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f54dbfff640 1 -- 172.21.15.31:0/2435993109 <== mds.0 v2:172.21.15.31:6834/3729395637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55780c9a49a0 con 0x55780cbd0010 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f54dbfff640 10 client.7281 handle_command_reply: tid=0 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 client.7281 shutdown 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 -- 172.21.15.31:0/2435993109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb26320 msgr2=0x55780cb27a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb26320 0x55780cb27a90 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f54f800cd80 tx=0x7f54f805c7d0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 -- 172.21.15.31:0/2435993109 shutdown_connections 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55780cb705b0 0x55780cb834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55780cb6fc90 0x55780cb82fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.31:6834/3729395637,v1:172.21.15.31:6835/3729395637] conn(0x55780cbd0010 0x55780cbd03f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 --2- 172.21.15.31:0/2435993109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55780cb26320 0x55780cb27a90 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 -- 172.21.15.31:0/2435993109 >> 172.21.15.31:0/2435993109 conn(0x55780cb222d0 msgr2=0x55780cb246c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f54dbfff640 0 client.7281 ms_handle_reset on v2:172.21.15.31:6834/3729395637 2023-11-26T07:22:33.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 -- 172.21.15.31:0/2435993109 shutdown_connections 2023-11-26T07:22:33.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 1 -- 172.21.15.31:0/2435993109 wait complete. 2023-11-26T07:22:33.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f55066f7000 20 client.7281 trim_cache size 0 max 16384 2023-11-26T07:22:33.633 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:22:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "4c2b4df1-71b1-4841-8807-af144d2610e7", 2023-11-26T07:22:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-11-26T07:22:33.633 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 7101, 2023-11-26T07:22:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-11-26T07:22:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-11-26T07:22:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-11-26T07:22:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 10.465903484, 2023-11-26T07:22:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 57, 2023-11-26T07:22:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 80, 2023-11-26T07:22:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 80, 2023-11-26T07:22:33.634 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 15.362114677999999 2023-11-26T07:22:33.634 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:22:33.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 -- 172.21.15.31:0/1586636207 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f54e0066590 msgr2=0x7f54e0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f54e0066590 0x7f54e0068a50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0ae7a0 tx=0x7f54ec009040 comp rx=0 tx=0).stop 2023-11-26T07:22:33.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 -- 172.21.15.31:0/1586636207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc0cded0 msgr2=0x7f54fc1558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc0cded0 0x7f54fc1558e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f54f800c3c0 tx=0x7f54f800c880 comp rx=0 tx=0).stop 2023-11-26T07:22:33.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 -- 172.21.15.31:0/1586636207 shutdown_connections 2023-11-26T07:22:33.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f54e0066590 0x7f54e0068a50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54fc154180 0x7f54fc13c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f54fc13f970 0x7f54fc15ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 --2- 172.21.15.31:0/1586636207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54fc0cded0 0x7f54fc1558e0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 -- 172.21.15.31:0/1586636207 >> 172.21.15.31:0/1586636207 conn(0x7f54fc071d80 msgr2=0x7f54fc0b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 -- 172.21.15.31:0/1586636207 shutdown_connections 2023-11-26T07:22:33.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.625+0000 7f5503380640 1 -- 172.21.15.31:0/1586636207 wait complete. 2023-11-26T07:22:33.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:22:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65bf01640 1 Processor -- start 2023-11-26T07:22:33.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65bf01640 1 -- start start 2023-11-26T07:22:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65bf01640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65415d8b0 0x7ff6540b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65bf01640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff6540b5120 0x7ff6540ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65bf01640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6540b0a70 con 0x7ff6540ceb00 2023-11-26T07:22:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65bf01640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6540cddb0 con 0x7ff6540b5120 2023-11-26T07:22:33.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65bf01640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6540b4100 con 0x7ff65415d8b0 2023-11-26T07:22:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff659c9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65415d8b0 0x7ff6540b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65949c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff6540b5120 0x7ff6540ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65a49e640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff6540ceb00 0x7ff6540b3050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:60598/0 (socket says 172.21.15.31:60598) 2023-11-26T07:22:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65a49e640 1 -- 172.21.15.31:0/3179482069 learned_addr learned my addr 172.21.15.31:0/3179482069 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff659c9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65415d8b0 0x7ff6540b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:49392/0 (socket says 172.21.15.31:49392) 2023-11-26T07:22:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65949c640 1 -- 172.21.15.31:0/3179482069 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65415d8b0 msgr2=0x7ff6540b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65949c640 1 --2- 172.21.15.31:0/3179482069 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65415d8b0 0x7ff6540b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65949c640 1 -- 172.21.15.31:0/3179482069 >> v1:172.21.15.97:6789/0 conn(0x7ff6540ceb00 legacy=0x7ff6540b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff65949c640 1 -- 172.21.15.31:0/3179482069 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6540b43e0 con 0x7ff6540b5120 2023-11-26T07:22:33.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.789+0000 7ff658c9b640 1 -- 172.21.15.31:0/3179482069 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245011084 0 0) 0x7ff6540b0a70 con 0x7ff6540ceb00 2023-11-26T07:22:33.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff65949c640 1 --2- 172.21.15.31:0/3179482069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540b5120 0x7ff6540ce5c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff6440097c0 tx=0x7ff64401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=684570f9ace6151 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff658c9b640 1 -- 172.21.15.31:0/3179482069 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6440a8970 con 0x7ff6540b5120 2023-11-26T07:22:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff658c9b640 1 -- 172.21.15.31:0/3179482069 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6440a8ad0 con 0x7ff6540b5120 2023-11-26T07:22:33.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff658c9b640 1 -- 172.21.15.31:0/3179482069 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6440a8e60 con 0x7ff6540b5120 2023-11-26T07:22:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff65bf01640 1 -- 172.21.15.31:0/3179482069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540b5120 msgr2=0x7ff6540ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff65bf01640 1 --2- 172.21.15.31:0/3179482069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540b5120 0x7ff6540ce5c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff6440097c0 tx=0x7ff64401ec70 comp rx=0 tx=0).stop 2023-11-26T07:22:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff65bf01640 1 -- 172.21.15.31:0/3179482069 shutdown_connections 2023-11-26T07:22:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff65bf01640 1 --2- 172.21.15.31:0/3179482069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6540b5120 0x7ff6540ce5c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff65bf01640 1 --2- 172.21.15.31:0/3179482069 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff65415d8b0 0x7ff6540b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff65bf01640 1 -- 172.21.15.31:0/3179482069 >> 172.21.15.31:0/3179482069 conn(0x7ff6540b5570 msgr2=0x7ff6540b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff65bf01640 1 -- 172.21.15.31:0/3179482069 shutdown_connections 2023-11-26T07:22:33.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.793+0000 7ff65bf01640 1 -- 172.21.15.31:0/3179482069 wait complete. 2023-11-26T07:22:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65bf01640 1 Processor -- start 2023-11-26T07:22:33.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65bf01640 1 -- start start 2023-11-26T07:22:33.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65bf01640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6540b5120 0x7ff6540cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65bf01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6540ceb00 0x7ff6540c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65bf01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65415d8b0 0x7ff654146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65bf01640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6540c9910 con 0x7ff6540ceb00 2023-11-26T07:22:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65bf01640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6540ca730 con 0x7ff65415d8b0 2023-11-26T07:22:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65bf01640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6540c9b20 con 0x7ff6540b5120 2023-11-26T07:22:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65949c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6540ceb00 0x7ff6540c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65949c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6540ceb00 0x7ff6540c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42668/0 (socket says 172.21.15.31:42668) 2023-11-26T07:22:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65949c640 1 -- 172.21.15.31:0/1870111683 learned_addr learned my addr 172.21.15.31:0/1870111683 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:33.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff659c9d640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6540b5120 0x7ff6540cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65a49e640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65415d8b0 0x7ff654146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65949c640 1 -- 172.21.15.31:0/1870111683 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6540b5120 msgr2=0x7ff6540cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65949c640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6540b5120 0x7ff6540cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65949c640 1 -- 172.21.15.31:0/1870111683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65415d8b0 msgr2=0x7ff654146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65949c640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65415d8b0 0x7ff654146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65949c640 1 -- 172.21.15.31:0/1870111683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff644009420 con 0x7ff6540ceb00 2023-11-26T07:22:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65949c640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6540ceb00 0x7ff6540c93d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff6440b3040 tx=0x7ff64401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff64a7fc640 1 -- 172.21.15.31:0/1870111683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6440b0e10 con 0x7ff6540ceb00 2023-11-26T07:22:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff64a7fc640 1 -- 172.21.15.31:0/1870111683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6440b9410 con 0x7ff6540ceb00 2023-11-26T07:22:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65bf01640 1 -- 172.21.15.31:0/1870111683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6540c9e00 con 0x7ff6540ceb00 2023-11-26T07:22:33.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff64a7fc640 1 -- 172.21.15.31:0/1870111683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6440b97c0 con 0x7ff6540ceb00 2023-11-26T07:22:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.797+0000 7ff65bf01640 1 -- 172.21.15.31:0/1870111683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6540ca3c0 con 0x7ff6540ceb00 2023-11-26T07:22:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.801+0000 7ff65bf01640 1 -- 172.21.15.31:0/1870111683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6540cb100 con 0x7ff6540ceb00 2023-11-26T07:22:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.801+0000 7ff64a7fc640 1 -- 172.21.15.31:0/1870111683 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ff6440b7040 con 0x7ff6540ceb00 2023-11-26T07:22:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.801+0000 7ff64a7fc640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff638066590 0x7ff638068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:33.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.801+0000 7ff64a7fc640 1 -- 172.21.15.31:0/1870111683 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff644121a90 con 0x7ff6540ceb00 2023-11-26T07:22:33.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.801+0000 7ff659c9d640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff638066590 0x7ff638068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:33.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.801+0000 7ff659c9d640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff638066590 0x7ff638068a50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff650009ac0 tx=0x7ff650009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:33.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.805+0000 7ff64a7fc640 1 -- 172.21.15.31:0/1870111683 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6440eeeb0 con 0x7ff6540ceb00 2023-11-26T07:22:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.997+0000 7ff65bf01640 1 -- 172.21.15.31:0/1870111683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff6540b0650 con 0x7ff6540ceb00 2023-11-26T07:22:34.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:33.997+0000 7ff64a7fc640 1 -- 172.21.15.31:0/1870111683 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff6440b5070 con 0x7ff6540ceb00 2023-11-26T07:22:34.001 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:22:34.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 -- 172.21.15.31:0/1870111683 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff638066590 msgr2=0x7ff638068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff638066590 0x7ff638068a50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff650009ac0 tx=0x7ff650009190 comp rx=0 tx=0).stop 2023-11-26T07:22:34.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 -- 172.21.15.31:0/1870111683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6540ceb00 msgr2=0x7ff6540c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6540ceb00 0x7ff6540c93d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff6440b3040 tx=0x7ff64401ba10 comp rx=0 tx=0).stop 2023-11-26T07:22:34.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 -- 172.21.15.31:0/1870111683 shutdown_connections 2023-11-26T07:22:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff638066590 0x7ff638068a50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff65415d8b0 0x7ff654146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6540ceb00 0x7ff6540c93d0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 --2- 172.21.15.31:0/1870111683 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6540b5120 0x7ff6540cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 -- 172.21.15.31:0/1870111683 >> 172.21.15.31:0/1870111683 conn(0x7ff6540b5570 msgr2=0x7ff6540d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 -- 172.21.15.31:0/1870111683 shutdown_connections 2023-11-26T07:22:34.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.001+0000 7ff65bf01640 1 -- 172.21.15.31:0/1870111683 wait complete. 2023-11-26T07:22:34.022 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:22:34.022 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:22:34.022 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-11-26T07:22:34.022 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:22:34.023 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:34.023 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:22:34.023 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:22:34.023 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:22:34.023 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:22:34.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:34.023 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-11-26T07:22:34.033 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:22:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:22:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:22:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42086sec preferred_lft 42086sec 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 02:42:1b:e4:a9:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:22:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:22:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:22:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:34.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:34.037 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:22:34.097 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-11-26T07:22:34.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:34.100 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-11-26T07:22:34.155 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-11-26T07:22:34.155 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:22:34.155 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-26T07:22:34.156 INFO:teuthology.orchestra.run.smithi031.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:22:34.156 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-26T07:22:34.156 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:22:34.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:34.156 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:22:34.156 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:22:34.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-26T07:22:34.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-26T07:22:34.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-26T07:22:34.157 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:22:34.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:34.255 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:22:34.265 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:22:34.265 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:22:34.266 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:22:34.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:34.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-11-26T07:22:34.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:34.327 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:34.378 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:22:34.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:34.379 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:34.429 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:22:34.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:34.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:22:34.521 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:22:34.521 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:22:34.534 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:22:34.534 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:22:34.534 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:22:34.534 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:22:34.534 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:22:34.538 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:22:34.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:34.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:34.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:34.560 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:22:34.619 INFO:teuthology.orchestra.run.smithi031.stdout:client6269 2023-11-26T07:22:34.620 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:22:34.620 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:34.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:22:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f3817640 1 Processor -- start 2023-11-26T07:22:34.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f3817640 1 -- start start 2023-11-26T07:22:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f3817640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec15d7b0 0x7f01ec0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f3817640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f01ec0d2080 0x7f01ec06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f3817640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0cefd0 con 0x7f01ec0cead0 2023-11-26T07:22:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f3817640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec0cf2b0 con 0x7f01ec0d2080 2023-11-26T07:22:34.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f3817640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec06b3f0 con 0x7f01ec15d7b0 2023-11-26T07:22:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f0db2640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f01ec0cead0 0x7f01ec0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:32954/0 (socket says 172.21.15.31:32954) 2023-11-26T07:22:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f0db2640 1 -- 172.21.15.31:0/3097505807 learned_addr learned my addr 172.21.15.31:0/3097505807 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f15b3640 1 --2- 172.21.15.31:0/3097505807 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec15d7b0 0x7f01ec0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f1db4640 1 --2- 172.21.15.31:0/3097505807 >> v2:172.21.15.97:3300/0 conn(0x7f01ec0d2080 0x7f01ec06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01e3fff640 1 -- 172.21.15.31:0/3097505807 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176155415 0 0) 0x7f01ec0cefd0 con 0x7f01ec0cead0 2023-11-26T07:22:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01e3fff640 1 -- 172.21.15.31:0/3097505807 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d8003530 con 0x7f01ec0cead0 2023-11-26T07:22:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f15b3640 1 -- 172.21.15.31:0/3097505807 >> v2:172.21.15.97:3300/0 conn(0x7f01ec0d2080 msgr2=0x7f01ec06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f15b3640 1 --2- 172.21.15.31:0/3097505807 >> v2:172.21.15.97:3300/0 conn(0x7f01ec0d2080 0x7f01ec06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f15b3640 1 -- 172.21.15.31:0/3097505807 >> v1:172.21.15.31:6789/0 conn(0x7f01ec0cead0 legacy=0x7f01ec0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.853+0000 7f01f15b3640 1 -- 172.21.15.31:0/3097505807 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec06b6d0 con 0x7f01ec15d7b0 2023-11-26T07:22:34.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01f15b3640 1 --2- 172.21.15.31:0/3097505807 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec15d7b0 0x7f01ec0d10d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01e80097c0 tx=0x7f01e801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76bcbec668307aeb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01e3fff640 1 -- 172.21.15.31:0/3097505807 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8004230 con 0x7f01ec15d7b0 2023-11-26T07:22:34.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01e3fff640 1 -- 172.21.15.31:0/3097505807 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e80a7050 con 0x7f01ec15d7b0 2023-11-26T07:22:34.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01e3fff640 1 -- 172.21.15.31:0/3097505807 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8003620 con 0x7f01ec15d7b0 2023-11-26T07:22:34.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01f3817640 1 -- 172.21.15.31:0/3097505807 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec15d7b0 msgr2=0x7f01ec0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01f3817640 1 --2- 172.21.15.31:0/3097505807 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec15d7b0 0x7f01ec0d10d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f01e80097c0 tx=0x7f01e801c330 comp rx=0 tx=0).stop 2023-11-26T07:22:34.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01f3817640 1 -- 172.21.15.31:0/3097505807 shutdown_connections 2023-11-26T07:22:34.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01f3817640 1 --2- 172.21.15.31:0/3097505807 >> v2:172.21.15.97:3300/0 conn(0x7f01ec0d2080 0x7f01ec06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01f3817640 1 --2- 172.21.15.31:0/3097505807 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec15d7b0 0x7f01ec0d10d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01f3817640 1 -- 172.21.15.31:0/3097505807 >> 172.21.15.31:0/3097505807 conn(0x7f01ec0d33b0 msgr2=0x7f01ec0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:34.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01f3817640 1 -- 172.21.15.31:0/3097505807 shutdown_connections 2023-11-26T07:22:34.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.857+0000 7f01f3817640 1 -- 172.21.15.31:0/3097505807 wait complete. 2023-11-26T07:22:34.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f3817640 1 Processor -- start 2023-11-26T07:22:34.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f3817640 1 -- start start 2023-11-26T07:22:34.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f3817640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec0cead0 0x7f01ec0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f3817640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ec0d2080 0x7f01ec153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f3817640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ec15d7b0 0x7f01ec0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f3817640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0c7d90 con 0x7f01ec0d2080 2023-11-26T07:22:34.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f3817640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0cae60 con 0x7f01ec15d7b0 2023-11-26T07:22:34.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f3817640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0cb140 con 0x7f01ec0cead0 2023-11-26T07:22:34.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f0db2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ec0d2080 0x7f01ec153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f0db2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ec0d2080 0x7f01ec153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42672/0 (socket says 172.21.15.31:42672) 2023-11-26T07:22:34.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f0db2640 1 -- 172.21.15.31:0/449975686 learned_addr learned my addr 172.21.15.31:0/449975686 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:34.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f15b3640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec0cead0 0x7f01ec0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f1db4640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ec15d7b0 0x7f01ec0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f0db2640 1 -- 172.21.15.31:0/449975686 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec0cead0 msgr2=0x7f01ec0c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f0db2640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec0cead0 0x7f01ec0c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f0db2640 1 -- 172.21.15.31:0/449975686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ec15d7b0 msgr2=0x7f01ec0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:34.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f0db2640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ec15d7b0 0x7f01ec0ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:34.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f0db2640 1 -- 172.21.15.31:0/449975686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e8009420 con 0x7f01ec0d2080 2023-11-26T07:22:34.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f1db4640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ec15d7b0 0x7f01ec0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:34.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f0db2640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ec0d2080 0x7f01ec153580 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00c420 tx=0x7f01dc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01e1ffb640 1 -- 172.21.15.31:0/449975686 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc005cf0 con 0x7f01ec0d2080 2023-11-26T07:22:34.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01e1ffb640 1 -- 172.21.15.31:0/449975686 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc005e50 con 0x7f01ec0d2080 2023-11-26T07:22:34.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01e1ffb640 1 -- 172.21.15.31:0/449975686 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0072d0 con 0x7f01ec0d2080 2023-11-26T07:22:34.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.861+0000 7f01f3817640 1 -- 172.21.15.31:0/449975686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec0cb420 con 0x7f01ec0d2080 2023-11-26T07:22:34.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.865+0000 7f01f3817640 1 -- 172.21.15.31:0/449975686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec0cbf70 con 0x7f01ec0d2080 2023-11-26T07:22:34.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.865+0000 7f01f3817640 1 -- 172.21.15.31:0/449975686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec04e850 con 0x7f01ec0d2080 2023-11-26T07:22:34.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.865+0000 7f01e1ffb640 1 -- 172.21.15.31:0/449975686 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f01dc007430 con 0x7f01ec0d2080 2023-11-26T07:22:34.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.865+0000 7f01e1ffb640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f01d00665c0 0x7f01d0068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:34.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.865+0000 7f01e1ffb640 1 -- 172.21.15.31:0/449975686 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01dc0834b0 con 0x7f01ec0d2080 2023-11-26T07:22:34.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.865+0000 7f01f15b3640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f01d00665c0 0x7f01d0068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:34.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.865+0000 7f01f15b3640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f01d00665c0 0x7f01d0068a80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01e80ad000 tx=0x7f01e80a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:34.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:34.869+0000 7f01e1ffb640 1 -- 172.21.15.31:0/449975686 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f01dc0508c0 con 0x7f01ec0d2080 2023-11-26T07:22:35.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.061+0000 7f01f3817640 1 -- 172.21.15.31:0/449975686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f01ec153ac0 con 0x7f01ec0d2080 2023-11-26T07:22:35.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.061+0000 7f01e1ffb640 1 -- 172.21.15.31:0/449975686 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f01dc017060 con 0x7f01ec0d2080 2023-11-26T07:22:35.064 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:22:35.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 -- 172.21.15.31:0/449975686 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f01d00665c0 msgr2=0x7f01d0068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f01d00665c0 0x7f01d0068a80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01e80ad000 tx=0x7f01e80a6000 comp rx=0 tx=0).stop 2023-11-26T07:22:35.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 -- 172.21.15.31:0/449975686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ec0d2080 msgr2=0x7f01ec153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ec0d2080 0x7f01ec153580 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00c420 tx=0x7f01dc00c8e0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 -- 172.21.15.31:0/449975686 shutdown_connections 2023-11-26T07:22:35.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f01d00665c0 0x7f01d0068a80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f01ec15d7b0 0x7f01ec0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01ec0d2080 0x7f01ec153580 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 --2- 172.21.15.31:0/449975686 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01ec0cead0 0x7f01ec0c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 -- 172.21.15.31:0/449975686 >> 172.21.15.31:0/449975686 conn(0x7f01ec0d33b0 msgr2=0x7f01ec0cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 -- 172.21.15.31:0/449975686 shutdown_connections 2023-11-26T07:22:35.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.065+0000 7f01f3817640 1 -- 172.21.15.31:0/449975686 wait complete. 2023-11-26T07:22:35.086 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:22:35.086 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:22:35.086 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-26T07:22:35.086 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:22:35.086 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:35.086 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:22:35.086 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:22:35.087 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:22:35.087 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:22:35.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:35.087 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-26T07:22:35.098 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:22:35.098 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:22:35.098 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:22:35.098 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:35.098 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-26T07:22:35.098 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:35.098 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:35.098 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:35.098 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42108sec preferred_lft 42108sec 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:35.099 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:61:bc:5b:3d brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:22:35.100 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:35.101 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:22:35.101 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:22:35.101 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:22:35.101 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:22:35.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:35.102 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:22:35.170 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-26T07:22:35.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:35.172 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-26T07:22:35.224 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-26T07:22:35.224 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:22:35.224 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:22:35.224 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:22:35.225 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:22:35.225 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:22:35.225 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:35.225 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:35.225 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:22:35.225 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:22:35.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:35.226 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:22:35.226 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:22:35.226 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:22:35.226 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:22:35.226 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:22:35.226 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:22:35.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:35.322 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:22:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:22:35.331 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:22:35.332 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:22:35.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:35.333 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-26T07:22:35.398 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:35.398 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:35.450 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:22:35.450 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:22:35.450 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:35.500 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:22:35.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:35.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:22:35.593 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:22:35.593 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:22:35.608 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:22:35.608 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:22:35.608 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:22:35.608 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 1@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:22:35.608 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:22:35.612 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:22:35.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:35.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:35.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:35.633 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:22:35.688 INFO:teuthology.orchestra.run.smithi097.stdout:client7293 2023-11-26T07:22:35.688 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:35.688 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:35.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:22:35.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f7026b2a640 1 Processor -- start 2023-11-26T07:22:35.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f7026b2a640 1 -- start start 2023-11-26T07:22:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f7026b2a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f70200cd7b0 0x7f702015b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f7026b2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70200b4ee0 0x7f70200b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f7026b2a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702015bc10 con 0x7f702015b2f0 2023-11-26T07:22:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f7026b2a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f702006d5a0 con 0x7f70200cd7b0 2023-11-26T07:22:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f7026b2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70200b5860 con 0x7f70200b4ee0 2023-11-26T07:22:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f701ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f70200cd7b0 0x7f702015b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f701ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f70200cd7b0 0x7f702015b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42688/0 (socket says 172.21.15.31:42688) 2023-11-26T07:22:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f701ffff640 1 -- 172.21.15.31:0/3045644011 learned_addr learned my addr 172.21.15.31:0/3045644011 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f70250c7640 1 --2- 172.21.15.31:0/3045644011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70200b4ee0 0x7f70200b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f701ffff640 1 -- 172.21.15.31:0/3045644011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70200b4ee0 msgr2=0x7f70200b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f701ffff640 1 --2- 172.21.15.31:0/3045644011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70200b4ee0 0x7f70200b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f701ffff640 1 -- 172.21.15.31:0/3045644011 >> v1:172.21.15.31:6789/0 conn(0x7f702015b2f0 legacy=0x7f70200ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f701ffff640 1 -- 172.21.15.31:0/3045644011 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702006ea40 con 0x7f70200cd7b0 2023-11-26T07:22:35.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.889+0000 7f701f7fe640 1 -- 172.21.15.31:0/3045644011 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304421005 0 0) 0x7f702015bc10 con 0x7f702015b2f0 2023-11-26T07:22:35.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f701ffff640 1 --2- 172.21.15.31:0/3045644011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200cd7b0 0x7f702015b6d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f700c009870 tx=0x7f700c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49362c77e26b9462 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f701f7fe640 1 -- 172.21.15.31:0/3045644011 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c0aa670 con 0x7f70200cd7b0 2023-11-26T07:22:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f701f7fe640 1 -- 172.21.15.31:0/3045644011 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c0aa7d0 con 0x7f70200cd7b0 2023-11-26T07:22:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f701f7fe640 1 -- 172.21.15.31:0/3045644011 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c0aab60 con 0x7f70200cd7b0 2023-11-26T07:22:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f7026b2a640 1 -- 172.21.15.31:0/3045644011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200cd7b0 msgr2=0x7f702015b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f7026b2a640 1 --2- 172.21.15.31:0/3045644011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200cd7b0 0x7f702015b6d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f700c009870 tx=0x7f700c01c510 comp rx=0 tx=0).stop 2023-11-26T07:22:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f7026b2a640 1 -- 172.21.15.31:0/3045644011 shutdown_connections 2023-11-26T07:22:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f7026b2a640 1 --2- 172.21.15.31:0/3045644011 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70200b4ee0 0x7f70200b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f7026b2a640 1 --2- 172.21.15.31:0/3045644011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200cd7b0 0x7f702015b6d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f7026b2a640 1 -- 172.21.15.31:0/3045644011 >> 172.21.15.31:0/3045644011 conn(0x7f7020071d80 msgr2=0x7f702006a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f7026b2a640 1 -- 172.21.15.31:0/3045644011 shutdown_connections 2023-11-26T07:22:35.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f7026b2a640 1 -- 172.21.15.31:0/3045644011 wait complete. 2023-11-26T07:22:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.893+0000 7f7026b2a640 1 Processor -- start 2023-11-26T07:22:35.898 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f7026b2a640 1 -- start start 2023-11-26T07:22:35.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f7026b2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200b4ee0 0x7f7020137300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f7026b2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70200cd7b0 0x7f702006ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f7026b2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f702015b2f0 0x7f702006b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f7026b2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7020158560 con 0x7f70200b4ee0 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f7026b2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7020158280 con 0x7f70200cd7b0 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f7026b2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70201517c0 con 0x7f702015b2f0 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70248c6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200b4ee0 0x7f7020137300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70248c6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200b4ee0 0x7f7020137300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42692/0 (socket says 172.21.15.31:42692) 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70248c6640 1 -- 172.21.15.31:0/1528616826 learned_addr learned my addr 172.21.15.31:0/1528616826 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70250c7640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f702015b2f0 0x7f702006b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f701ffff640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70200cd7b0 0x7f702006ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70248c6640 1 -- 172.21.15.31:0/1528616826 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f702015b2f0 msgr2=0x7f702006b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70248c6640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f702015b2f0 0x7f702006b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70248c6640 1 -- 172.21.15.31:0/1528616826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70200cd7b0 msgr2=0x7f702006ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70248c6640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70200cd7b0 0x7f702006ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70248c6640 1 -- 172.21.15.31:0/1528616826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0094d0 con 0x7f70200b4ee0 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70250c7640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f702015b2f0 0x7f702006b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f70248c6640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200b4ee0 0x7f7020137300 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f701400c3f0 tx=0x7f701400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f701d7fa640 1 -- 172.21.15.31:0/1528616826 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70140052d0 con 0x7f70200b4ee0 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f701d7fa640 1 -- 172.21.15.31:0/1528616826 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7014005450 con 0x7f70200b4ee0 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f701d7fa640 1 -- 172.21.15.31:0/1528616826 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014003860 con 0x7f70200b4ee0 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f7026b2a640 1 -- 172.21.15.31:0/1528616826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7020151aa0 con 0x7f70200b4ee0 2023-11-26T07:22:35.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f7026b2a640 1 -- 172.21.15.31:0/1528616826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7020137840 con 0x7f70200b4ee0 2023-11-26T07:22:35.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.897+0000 7f7026b2a640 1 -- 172.21.15.31:0/1528616826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7020151620 con 0x7f70200b4ee0 2023-11-26T07:22:35.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.901+0000 7f701d7fa640 1 -- 172.21.15.31:0/1528616826 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f70140039c0 con 0x7f70200b4ee0 2023-11-26T07:22:35.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.901+0000 7f701d7fa640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 0x7f7004068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:35.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.901+0000 7f701d7fa640 1 -- 172.21.15.31:0/1528616826 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70140836b0 con 0x7f70200b4ee0 2023-11-26T07:22:35.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.901+0000 7f701ffff640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 0x7f7004068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:35.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.901+0000 7f701ffff640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 0x7f7004068a50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f700c0ae000 tx=0x7f700c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:35.907 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:35.905+0000 7f701d7fa640 1 -- 172.21.15.31:0/1528616826 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7020151620 con 0x7f70200b4ee0 2023-11-26T07:22:36.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.097+0000 7f7026b2a640 1 -- 172.21.15.31:0/1528616826 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7020151620 con 0x7f70200b4ee0 2023-11-26T07:22:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f701d7fa640 1 -- 172.21.15.31:0/1528616826 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v58) v1 ==== 136+0+0 (secure 0 0 0) 0x7f7020151620 con 0x7f70200b4ee0 2023-11-26T07:22:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 -- 172.21.15.31:0/1528616826 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 msgr2=0x7f7004068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 0x7f7004068a50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f700c0ae000 tx=0x7f700c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 -- 172.21.15.31:0/1528616826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200b4ee0 msgr2=0x7f7020137300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200b4ee0 0x7f7020137300 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f701400c3f0 tx=0x7f701400c8b0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 -- 172.21.15.31:0/1528616826 shutdown_connections 2023-11-26T07:22:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7004066590 0x7f7004068a50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f702015b2f0 0x7f702006b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70200cd7b0 0x7f702006ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 --2- 172.21.15.31:0/1528616826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70200b4ee0 0x7f7020137300 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 -- 172.21.15.31:0/1528616826 >> 172.21.15.31:0/1528616826 conn(0x7f7020071d80 msgr2=0x7f7020158760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:36.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 -- 172.21.15.31:0/1528616826 shutdown_connections 2023-11-26T07:22:36.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.149+0000 7f7026b2a640 1 -- 172.21.15.31:0/1528616826 wait complete. 2023-11-26T07:22:36.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:22:36.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d22a6640 1 Processor -- start 2023-11-26T07:22:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d22a6640 1 -- start start 2023-11-26T07:22:36.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d22a6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f45cc1609c0 0x7f45cc0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d22a6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45cc0b8c80 0x7f45cc0cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d22a6640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45cc0d2d10 0x7f45cc0ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d22a6640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45cc160da0 con 0x7f45cc0d2d10 2023-11-26T07:22:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d22a6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45cc0d1c70 con 0x7f45cc1609c0 2023-11-26T07:22:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d22a6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc0d1f50 con 0x7f45cc0b8c80 2023-11-26T07:22:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d1aa5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45cc0d2d10 0x7f45cc0ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d1aa5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f45cc0d2d10 0x7f45cc0ce4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42704/0 (socket says 172.21.15.31:42704) 2023-11-26T07:22:36.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d1aa5640 1 -- 172.21.15.31:0/1509329711 learned_addr learned my addr 172.21.15.31:0/1509329711 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:36.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d0aa3640 1 --2- 172.21.15.31:0/1509329711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45cc0b8c80 0x7f45cc0cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d12a4640 1 --2- 172.21.15.31:0/1509329711 >> v2:172.21.15.97:3300/0 conn(0x7f45cc1609c0 0x7f45cc0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d1aa5640 1 -- 172.21.15.31:0/1509329711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45cc0b8c80 msgr2=0x7f45cc0cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:36.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d1aa5640 1 --2- 172.21.15.31:0/1509329711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45cc0b8c80 0x7f45cc0cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d1aa5640 1 -- 172.21.15.31:0/1509329711 >> v2:172.21.15.97:3300/0 conn(0x7f45cc1609c0 msgr2=0x7f45cc0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:36.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d1aa5640 1 --2- 172.21.15.31:0/1509329711 >> v2:172.21.15.97:3300/0 conn(0x7f45cc1609c0 0x7f45cc0d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d1aa5640 1 -- 172.21.15.31:0/1509329711 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc0d2230 con 0x7f45cc0d2d10 2023-11-26T07:22:36.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.317+0000 7f45d12a4640 1 --2- 172.21.15.31:0/1509329711 >> v2:172.21.15.97:3300/0 conn(0x7f45cc1609c0 0x7f45cc0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45d1aa5640 1 --2- 172.21.15.31:0/1509329711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc0d2d10 0x7f45cc0ce4e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f45cc08c120 tx=0x7f45c401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad11761bf68457d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:36.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45c3fff640 1 -- 172.21.15.31:0/1509329711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c40a5820 con 0x7f45cc0d2d10 2023-11-26T07:22:36.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45c3fff640 1 -- 172.21.15.31:0/1509329711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c40a5980 con 0x7f45cc0d2d10 2023-11-26T07:22:36.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45c3fff640 1 -- 172.21.15.31:0/1509329711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c40a5d10 con 0x7f45cc0d2d10 2023-11-26T07:22:36.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45d22a6640 1 -- 172.21.15.31:0/1509329711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc0d2d10 msgr2=0x7f45cc0ce4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45d22a6640 1 --2- 172.21.15.31:0/1509329711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc0d2d10 0x7f45cc0ce4e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f45cc08c120 tx=0x7f45c401c9a0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45d22a6640 1 -- 172.21.15.31:0/1509329711 shutdown_connections 2023-11-26T07:22:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45d22a6640 1 --2- 172.21.15.31:0/1509329711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc0d2d10 0x7f45cc0ce4e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45d22a6640 1 --2- 172.21.15.31:0/1509329711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45cc0b8c80 0x7f45cc0cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45d22a6640 1 --2- 172.21.15.31:0/1509329711 >> v2:172.21.15.97:3300/0 conn(0x7f45cc1609c0 0x7f45cc0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45d22a6640 1 -- 172.21.15.31:0/1509329711 >> 172.21.15.31:0/1509329711 conn(0x7f45cc0668a0 msgr2=0x7f45cc0cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45d22a6640 1 -- 172.21.15.31:0/1509329711 shutdown_connections 2023-11-26T07:22:36.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.321+0000 7f45d22a6640 1 -- 172.21.15.31:0/1509329711 wait complete. 2023-11-26T07:22:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d22a6640 1 Processor -- start 2023-11-26T07:22:36.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d22a6640 1 -- start start 2023-11-26T07:22:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d22a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45cc0d2d10 0x7f45cc150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d22a6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc1609c0 0x7f45cc151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d22a6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45cc1525f0 0x7f45cc152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d22a6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc152fb0 con 0x7f45cc1609c0 2023-11-26T07:22:36.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d22a6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc152300 con 0x7f45cc0d2d10 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d22a6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc150d20 con 0x7f45cc1525f0 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d0aa3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc1609c0 0x7f45cc151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d0aa3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc1609c0 0x7f45cc151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42714/0 (socket says 172.21.15.31:42714) 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d0aa3640 1 -- 172.21.15.31:0/2196098741 learned_addr learned my addr 172.21.15.31:0/2196098741 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d12a4640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45cc0d2d10 0x7f45cc150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d1aa5640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45cc1525f0 0x7f45cc152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d0aa3640 1 -- 172.21.15.31:0/2196098741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45cc1525f0 msgr2=0x7f45cc152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d0aa3640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45cc1525f0 0x7f45cc152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d0aa3640 1 -- 172.21.15.31:0/2196098741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45cc0d2d10 msgr2=0x7f45cc150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d0aa3640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45cc0d2d10 0x7f45cc150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d0aa3640 1 -- 172.21.15.31:0/2196098741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c4009b30 con 0x7f45cc1609c0 2023-11-26T07:22:36.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d0aa3640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc1609c0 0x7f45cc151dc0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00d830 tx=0x7f45bc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:36.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45c1ffb640 1 -- 172.21.15.31:0/2196098741 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0096a0 con 0x7f45cc1609c0 2023-11-26T07:22:36.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45c1ffb640 1 -- 172.21.15.31:0/2196098741 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc009800 con 0x7f45cc1609c0 2023-11-26T07:22:36.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d22a6640 1 -- 172.21.15.31:0/2196098741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc151000 con 0x7f45cc1609c0 2023-11-26T07:22:36.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45c1ffb640 1 -- 172.21.15.31:0/2196098741 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc009b70 con 0x7f45cc1609c0 2023-11-26T07:22:36.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.325+0000 7f45d22a6640 1 -- 172.21.15.31:0/2196098741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc1515c0 con 0x7f45cc1609c0 2023-11-26T07:22:36.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.329+0000 7f45d22a6640 1 -- 172.21.15.31:0/2196098741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45cc073d80 con 0x7f45cc1609c0 2023-11-26T07:22:36.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.329+0000 7f45c1ffb640 1 -- 172.21.15.31:0/2196098741 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f45bc014410 con 0x7f45cc1609c0 2023-11-26T07:22:36.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.329+0000 7f45c1ffb640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45b0066590 0x7f45b0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.329+0000 7f45c1ffb640 1 -- 172.21.15.31:0/2196098741 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45bc084940 con 0x7f45cc1609c0 2023-11-26T07:22:36.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.329+0000 7f45d12a4640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45b0066590 0x7f45b0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.329+0000 7f45d12a4640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45b0066590 0x7f45b0068a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0b5640 tx=0x7f45c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:36.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.333+0000 7f45c1ffb640 1 -- 172.21.15.31:0/2196098741 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f45bc051c10 con 0x7f45cc1609c0 2023-11-26T07:22:36.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.513+0000 7f45d22a6640 1 -- 172.21.15.31:0/2196098741 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f45cc15f040 con 0x7f45b0066590 2023-11-26T07:22:36.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.513+0000 7f45c1ffb640 1 -- 172.21.15.31:0/2196098741 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f45cc15f040 con 0x7f45b0066590 2023-11-26T07:22:36.516 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:22:36.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:22:36.517 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-11-26T07:22:36.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:22:36.517 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:22:36.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 -- 172.21.15.31:0/2196098741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45b0066590 msgr2=0x7f45b0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45b0066590 0x7f45b0068a50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0b5640 tx=0x7f45c8008040 comp rx=0 tx=0).stop 2023-11-26T07:22:36.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 -- 172.21.15.31:0/2196098741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc1609c0 msgr2=0x7f45cc151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc1609c0 0x7f45cc151dc0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00d830 tx=0x7f45bc00dcf0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 -- 172.21.15.31:0/2196098741 shutdown_connections 2023-11-26T07:22:36.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f45b0066590 0x7f45b0068a50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f45cc1525f0 0x7f45cc152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45cc1609c0 0x7f45cc151dc0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 --2- 172.21.15.31:0/2196098741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45cc0d2d10 0x7f45cc150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 -- 172.21.15.31:0/2196098741 >> 172.21.15.31:0/2196098741 conn(0x7f45cc0668a0 msgr2=0x7f45cc0b72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:36.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 -- 172.21.15.31:0/2196098741 shutdown_connections 2023-11-26T07:22:36.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.517+0000 7f45d22a6640 1 -- 172.21.15.31:0/2196098741 wait complete. 2023-11-26T07:22:36.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:22:36.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2eb1d640 1 Processor -- start 2023-11-26T07:22:36.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2eb1d640 1 -- start start 2023-11-26T07:22:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2eb1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d280ced30 0x7f6d280cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2eb1d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d280cf1e0 0x7f6d280d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2eb1d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d280cdfb0 con 0x7f6d2815d8f0 2023-11-26T07:22:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2eb1d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d280d4420 con 0x7f6d280cf1e0 2023-11-26T07:22:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2eb1d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d280d4140 con 0x7f6d280ced30 2023-11-26T07:22:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2d0ba640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d280cf1e0 0x7f6d280d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2d0ba640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d280cf1e0 0x7f6d280d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42730/0 (socket says 172.21.15.31:42730) 2023-11-26T07:22:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2d0ba640 1 -- 172.21.15.31:0/2057102481 learned_addr learned my addr 172.21.15.31:0/2057102481 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d27fff640 1 --2- 172.21.15.31:0/2057102481 >> v2:172.21.15.97:3300/0 conn(0x7f6d280ced30 0x7f6d280cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2d0ba640 1 -- 172.21.15.31:0/2057102481 >> v2:172.21.15.97:3300/0 conn(0x7f6d280ced30 msgr2=0x7f6d280cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2d0ba640 1 --2- 172.21.15.31:0/2057102481 >> v2:172.21.15.97:3300/0 conn(0x7f6d280ced30 0x7f6d280cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2d0ba640 1 -- 172.21.15.31:0/2057102481 >> v1:172.21.15.97:6789/0 conn(0x7f6d2815d8f0 legacy=0x7f6d2815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d2d0ba640 1 -- 172.21.15.31:0/2057102481 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d280ce250 con 0x7f6d280cf1e0 2023-11-26T07:22:36.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.681+0000 7f6d27fff640 1 --2- 172.21.15.31:0/2057102481 >> v2:172.21.15.97:3300/0 conn(0x7f6d280ced30 0x7f6d280cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:36.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d2d0ba640 1 --2- 172.21.15.31:0/2057102481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280cf1e0 0x7f6d280d0340 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6d28162280 tx=0x7f6d2001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5da6b2bd63a0d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:36.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d277fe640 1 -- 172.21.15.31:0/2057102481 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d200a5820 con 0x7f6d280cf1e0 2023-11-26T07:22:36.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d277fe640 1 -- 172.21.15.31:0/2057102481 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d200a5980 con 0x7f6d280cf1e0 2023-11-26T07:22:36.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d277fe640 1 -- 172.21.15.31:0/2057102481 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d200a5d10 con 0x7f6d280cf1e0 2023-11-26T07:22:36.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/2057102481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280cf1e0 msgr2=0x7f6d280d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d2eb1d640 1 --2- 172.21.15.31:0/2057102481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280cf1e0 0x7f6d280d0340 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6d28162280 tx=0x7f6d2001c960 comp rx=0 tx=0).stop 2023-11-26T07:22:36.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/2057102481 shutdown_connections 2023-11-26T07:22:36.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d2eb1d640 1 --2- 172.21.15.31:0/2057102481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280cf1e0 0x7f6d280d0340 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d2eb1d640 1 --2- 172.21.15.31:0/2057102481 >> v2:172.21.15.97:3300/0 conn(0x7f6d280ced30 0x7f6d280cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/2057102481 >> 172.21.15.31:0/2057102481 conn(0x7f6d2809c750 msgr2=0x7f6d280d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/2057102481 shutdown_connections 2023-11-26T07:22:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.685+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/2057102481 wait complete. 2023-11-26T07:22:36.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2eb1d640 1 Processor -- start 2023-11-26T07:22:36.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2eb1d640 1 -- start start 2023-11-26T07:22:36.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2eb1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280ced30 0x7f6d280ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2eb1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d280cf1e0 0x7f6d281554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2eb1d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6d2815d8f0 0x7f6d281538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2eb1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d281549c0 con 0x7f6d280ced30 2023-11-26T07:22:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2eb1d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28153de0 con 0x7f6d280cf1e0 2023-11-26T07:22:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2eb1d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d281574e0 con 0x7f6d2815d8f0 2023-11-26T07:22:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2c8b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280ced30 0x7f6d280ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2c8b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280ced30 0x7f6d280ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42732/0 (socket says 172.21.15.31:42732) 2023-11-26T07:22:36.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2c8b9640 1 -- 172.21.15.31:0/1459184859 learned_addr learned my addr 172.21.15.31:0/1459184859 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2d0ba640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6d2815d8f0 0x7f6d281538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d27fff640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d280cf1e0 0x7f6d281554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2c8b9640 1 -- 172.21.15.31:0/1459184859 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6d2815d8f0 msgr2=0x7f6d281538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2c8b9640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6d2815d8f0 0x7f6d281538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2c8b9640 1 -- 172.21.15.31:0/1459184859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d280cf1e0 msgr2=0x7f6d281554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2c8b9640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d280cf1e0 0x7f6d281554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2c8b9640 1 -- 172.21.15.31:0/1459184859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d20009b30 con 0x7f6d280ced30 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d27fff640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d280cf1e0 0x7f6d281554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2c8b9640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280ced30 0x7f6d280ccf90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c450 tx=0x7f6d1c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d257fa640 1 -- 172.21.15.31:0/1459184859 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c012830 con 0x7f6d280ced30 2023-11-26T07:22:36.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d257fa640 1 -- 172.21.15.31:0/1459184859 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c0129b0 con 0x7f6d280ced30 2023-11-26T07:22:36.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d257fa640 1 -- 172.21.15.31:0/1459184859 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c012d60 con 0x7f6d280ced30 2023-11-26T07:22:36.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/1459184859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d28154c10 con 0x7f6d280ced30 2023-11-26T07:22:36.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.689+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/1459184859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d281551d0 con 0x7f6d280ced30 2023-11-26T07:22:36.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.693+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/1459184859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d281576a0 con 0x7f6d280ced30 2023-11-26T07:22:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.693+0000 7f6d257fa640 1 -- 172.21.15.31:0/1459184859 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f6d1c01c460 con 0x7f6d280ced30 2023-11-26T07:22:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.693+0000 7f6d257fa640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d0c066590 0x7f6d0c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.693+0000 7f6d257fa640 1 -- 172.21.15.31:0/1459184859 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d1c083d80 con 0x7f6d280ced30 2023-11-26T07:22:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.693+0000 7f6d27fff640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d0c066590 0x7f6d0c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:36.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.693+0000 7f6d27fff640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d0c066590 0x7f6d0c068a50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6d2806d040 tx=0x7f6d14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:36.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.697+0000 7f6d257fa640 1 -- 172.21.15.31:0/1459184859 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6d1c0510e0 con 0x7f6d280ced30 2023-11-26T07:22:36.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/1459184859 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6d2804e850 con 0x7f6d280ced30 2023-11-26T07:22:36.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d257fa640 1 -- 172.21.15.31:0/1459184859 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6d1c054700 con 0x7f6d280ced30 2023-11-26T07:22:36.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/1459184859 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d0c066590 msgr2=0x7f6d0c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d0c066590 0x7f6d0c068a50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6d2806d040 tx=0x7f6d14008040 comp rx=0 tx=0).stop 2023-11-26T07:22:36.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/1459184859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280ced30 msgr2=0x7f6d280ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280ced30 0x7f6d280ccf90 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c450 tx=0x7f6d1c00c910 comp rx=0 tx=0).stop 2023-11-26T07:22:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/1459184859 shutdown_connections 2023-11-26T07:22:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d0c066590 0x7f6d0c068a50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6d2815d8f0 0x7f6d281538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d280cf1e0 0x7f6d281554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 --2- 172.21.15.31:0/1459184859 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d280ced30 0x7f6d280ccf90 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/1459184859 >> 172.21.15.31:0/1459184859 conn(0x7f6d2809c750 msgr2=0x7f6d28167dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/1459184859 shutdown_connections 2023-11-26T07:22:36.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:36.857+0000 7f6d2eb1d640 1 -- 172.21.15.31:0/1459184859 wait complete. 2023-11-26T07:22:36.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2023-11-26T07:22:37.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.021+0000 7f34e6cc9640 1 Processor -- start 2023-11-26T07:22:37.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.021+0000 7f34e6cc9640 1 -- start start 2023-11-26T07:22:37.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e6cc9640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34e00cb9b0 0x7f34e006e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e6cc9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f34e00cbd90 0x7f34e015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e6cc9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34e00ce5d0 0x7f34e015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e6cc9640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e015bba0 con 0x7f34e00cb9b0 2023-11-26T07:22:37.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e6cc9640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e00cdd50 con 0x7f34e00cbd90 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e6cc9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e0159630 con 0x7f34e00ce5d0 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e4a65640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34e00cb9b0 0x7f34e006e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e4a65640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f34e00cb9b0 0x7f34e006e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42738/0 (socket says 172.21.15.31:42738) 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e4a65640 1 -- 172.21.15.31:0/679830348 learned_addr learned my addr 172.21.15.31:0/679830348 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34dffff640 1 --2- 172.21.15.31:0/679830348 >> v2:172.21.15.97:3300/0 conn(0x7f34e00cbd90 0x7f34e015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e5266640 1 --2- 172.21.15.31:0/679830348 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34e00ce5d0 0x7f34e015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e4a65640 1 -- 172.21.15.31:0/679830348 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34e00ce5d0 msgr2=0x7f34e015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:37.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e4a65640 1 --2- 172.21.15.31:0/679830348 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34e00ce5d0 0x7f34e015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e4a65640 1 -- 172.21.15.31:0/679830348 >> v2:172.21.15.97:3300/0 conn(0x7f34e00cbd90 msgr2=0x7f34e015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:37.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e4a65640 1 --2- 172.21.15.31:0/679830348 >> v2:172.21.15.97:3300/0 conn(0x7f34e00cbd90 0x7f34e015f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.025+0000 7f34e4a65640 1 -- 172.21.15.31:0/679830348 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e015f6d0 con 0x7f34e00cb9b0 2023-11-26T07:22:37.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e4a65640 1 --2- 172.21.15.31:0/679830348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e00cb9b0 0x7f34e006e3f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f34e00b5200 tx=0x7f34d401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c667469e330a5e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34df7fe640 1 -- 172.21.15.31:0/679830348 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d40ab670 con 0x7f34e00cb9b0 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34df7fe640 1 -- 172.21.15.31:0/679830348 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d40ab7d0 con 0x7f34e00cb9b0 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34df7fe640 1 -- 172.21.15.31:0/679830348 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d40abb60 con 0x7f34e00cb9b0 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 -- 172.21.15.31:0/679830348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e00cb9b0 msgr2=0x7f34e006e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 --2- 172.21.15.31:0/679830348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e00cb9b0 0x7f34e006e3f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f34e00b5200 tx=0x7f34d401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 -- 172.21.15.31:0/679830348 shutdown_connections 2023-11-26T07:22:37.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 --2- 172.21.15.31:0/679830348 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34e00ce5d0 0x7f34e015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 --2- 172.21.15.31:0/679830348 >> v2:172.21.15.97:3300/0 conn(0x7f34e00cbd90 0x7f34e015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 --2- 172.21.15.31:0/679830348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e00cb9b0 0x7f34e006e3f0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 -- 172.21.15.31:0/679830348 >> 172.21.15.31:0/679830348 conn(0x7f34e00b0cb0 msgr2=0x7f34e00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:37.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 -- 172.21.15.31:0/679830348 shutdown_connections 2023-11-26T07:22:37.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 -- 172.21.15.31:0/679830348 wait complete. 2023-11-26T07:22:37.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 Processor -- start 2023-11-26T07:22:37.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.029+0000 7f34e6cc9640 1 -- start start 2023-11-26T07:22:37.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e6cc9640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34e00cb9b0 0x7f34e0158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e6cc9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34e00cbd90 0x7f34e009cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e6cc9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e00ce5d0 0x7f34e0071690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e6cc9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e009d100 con 0x7f34e00ce5d0 2023-11-26T07:22:37.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e6cc9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e009a0d0 con 0x7f34e00cbd90 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e6cc9640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e0099df0 con 0x7f34e00cb9b0 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e5266640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e00ce5d0 0x7f34e0071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e5266640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e00ce5d0 0x7f34e0071690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42752/0 (socket says 172.21.15.31:42752) 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e5266640 1 -- 172.21.15.31:0/4196553055 learned_addr learned my addr 172.21.15.31:0/4196553055 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e4a65640 1 --2- 172.21.15.31:0/4196553055 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34e00cb9b0 0x7f34e0158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34dffff640 1 --2- 172.21.15.31:0/4196553055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34e00cbd90 0x7f34e009cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e5266640 1 -- 172.21.15.31:0/4196553055 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34e00cb9b0 msgr2=0x7f34e0158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e5266640 1 --2- 172.21.15.31:0/4196553055 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f34e00cb9b0 0x7f34e0158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e5266640 1 -- 172.21.15.31:0/4196553055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34e00cbd90 msgr2=0x7f34e009cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e5266640 1 --2- 172.21.15.31:0/4196553055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34e00cbd90 0x7f34e009cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e5266640 1 -- 172.21.15.31:0/4196553055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d40094d0 con 0x7f34e00ce5d0 2023-11-26T07:22:37.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e5266640 1 --2- 172.21.15.31:0/4196553055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34e00ce5d0 0x7f34e0071690 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f34d8007e60 tx=0x7f34d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34dd7fa640 1 -- 172.21.15.31:0/4196553055 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8014420 con 0x7f34e00ce5d0 2023-11-26T07:22:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34dd7fa640 1 -- 172.21.15.31:0/4196553055 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d80145a0 con 0x7f34e00ce5d0 2023-11-26T07:22:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34dd7fa640 1 -- 172.21.15.31:0/4196553055 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8014950 con 0x7f34e00ce5d0 2023-11-26T07:22:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e6cc9640 1 -- 172.21.15.31:0/4196553055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e009a510 con 0x7f34e00ce5d0 2023-11-26T07:22:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e6cc9640 1 -- 172.21.15.31:0/4196553055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e009aad0 con 0x7f34e00ce5d0 2023-11-26T07:22:37.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34e6cc9640 1 -- 172.21.15.31:0/4196553055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e00671f0 con 0x7f34e00ce5d0 2023-11-26T07:22:37.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.033+0000 7f34dd7fa640 1 -- 172.21.15.31:0/4196553055 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f34d8014ab0 con 0x7f34e00ce5d0 2023-11-26T07:22:37.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.037+0000 7f34dd7fa640 1 --2- 172.21.15.31:0/4196553055 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f34c40665c0 0x7f34c4068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.037+0000 7f34dd7fa640 1 -- 172.21.15.31:0/4196553055 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34d8084710 con 0x7f34e00ce5d0 2023-11-26T07:22:37.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.037+0000 7f34e4a65640 1 --2- 172.21.15.31:0/4196553055 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f34c40665c0 0x7f34c4068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.037+0000 7f34e4a65640 1 --2- 172.21.15.31:0/4196553055 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f34c40665c0 0x7f34c4068a80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34d40af000 tx=0x7f34d40b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:37.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.041+0000 7f34dd7fa640 1 -- 172.21.15.31:0/4196553055 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f34d8051b20 con 0x7f34e00ce5d0 2023-11-26T07:22:37.225 INFO:teuthology.orchestra.run.smithi031.stderr:Invalid command: invalid chars #@ in abcd@# 2023-11-26T07:22:37.225 INFO:teuthology.orchestra.run.smithi031.stderr:fs volume create [] : Create a CephFS volume 2023-11-26T07:22:37.225 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2023-11-26T07:22:37.231 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-26T07:22:37.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2023-11-26T07:22:37.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.373+0000 7f6d53432640 1 Processor -- start 2023-11-26T07:22:37.377 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.373+0000 7f6d53432640 1 -- start start 2023-11-26T07:22:37.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d53432640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6d4c0b07a0 0x7f6d4c1619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d53432640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d4c0b15f0 0x7f6d4c0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d53432640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d4c161290 con 0x7f6d4c161f20 2023-11-26T07:22:37.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d53432640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4c0b0b80 con 0x7f6d4c0b15f0 2023-11-26T07:22:37.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d53432640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d4c15ebc0 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d519cf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d4c0b15f0 0x7f6d4c0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d519cf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d4c0b15f0 0x7f6d4c0cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42764/0 (socket says 172.21.15.31:42764) 2023-11-26T07:22:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d519cf640 1 -- 172.21.15.31:0/1002756567 learned_addr learned my addr 172.21.15.31:0/1002756567 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d511ce640 1 --2- 172.21.15.31:0/1002756567 >> v2:172.21.15.97:3300/0 conn(0x7f6d4c0b07a0 0x7f6d4c1619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d519cf640 1 -- 172.21.15.31:0/1002756567 >> v2:172.21.15.97:3300/0 conn(0x7f6d4c0b07a0 msgr2=0x7f6d4c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:37.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d519cf640 1 --2- 172.21.15.31:0/1002756567 >> v2:172.21.15.97:3300/0 conn(0x7f6d4c0b07a0 0x7f6d4c1619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d519cf640 1 -- 172.21.15.31:0/1002756567 >> v1:172.21.15.97:6789/0 conn(0x7f6d4c161f20 legacy=0x7f6d4c0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.377+0000 7f6d519cf640 1 -- 172.21.15.31:0/1002756567 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c161550 con 0x7f6d4c0b15f0 2023-11-26T07:22:37.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d519cf640 1 --2- 172.21.15.31:0/1002756567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b15f0 0x7f6d4c0cc9a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c06c6a0 tx=0x7f6d4401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f2a56feccccda0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:37.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d43fff640 1 -- 172.21.15.31:0/1002756567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d440a5780 con 0x7f6d4c0b15f0 2023-11-26T07:22:37.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d43fff640 1 -- 172.21.15.31:0/1002756567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d440a58e0 con 0x7f6d4c0b15f0 2023-11-26T07:22:37.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d43fff640 1 -- 172.21.15.31:0/1002756567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d440a5c70 con 0x7f6d4c0b15f0 2023-11-26T07:22:37.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d53432640 1 -- 172.21.15.31:0/1002756567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b15f0 msgr2=0x7f6d4c0cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d53432640 1 --2- 172.21.15.31:0/1002756567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b15f0 0x7f6d4c0cc9a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c06c6a0 tx=0x7f6d4401c9d0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d53432640 1 -- 172.21.15.31:0/1002756567 shutdown_connections 2023-11-26T07:22:37.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d53432640 1 --2- 172.21.15.31:0/1002756567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b15f0 0x7f6d4c0cc9a0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d53432640 1 --2- 172.21.15.31:0/1002756567 >> v2:172.21.15.97:3300/0 conn(0x7f6d4c0b07a0 0x7f6d4c1619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d53432640 1 -- 172.21.15.31:0/1002756567 >> 172.21.15.31:0/1002756567 conn(0x7f6d4c0b1d50 msgr2=0x7f6d4c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:37.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d53432640 1 -- 172.21.15.31:0/1002756567 shutdown_connections 2023-11-26T07:22:37.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d53432640 1 -- 172.21.15.31:0/1002756567 wait complete. 2023-11-26T07:22:37.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d53432640 1 Processor -- start 2023-11-26T07:22:37.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.381+0000 7f6d53432640 1 -- start start 2023-11-26T07:22:37.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d53432640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b07a0 0x7f6d4c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d53432640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6d4c0b15f0 0x7f6d4c1555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d53432640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d4c161f20 0x7f6d4c143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d53432640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c155af0 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d53432640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c155d20 con 0x7f6d4c161f20 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d53432640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4c156000 con 0x7f6d4c0b15f0 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d511ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b07a0 0x7f6d4c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d511ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b07a0 0x7f6d4c0af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42780/0 (socket says 172.21.15.31:42780) 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d511ce640 1 -- 172.21.15.31:0/767915436 learned_addr learned my addr 172.21.15.31:0/767915436 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d519cf640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d4c161f20 0x7f6d4c143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d509cd640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6d4c0b15f0 0x7f6d4c1555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d511ce640 1 -- 172.21.15.31:0/767915436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6d4c0b15f0 msgr2=0x7f6d4c1555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d511ce640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6d4c0b15f0 0x7f6d4c1555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d511ce640 1 -- 172.21.15.31:0/767915436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d4c161f20 msgr2=0x7f6d4c143e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d511ce640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d4c161f20 0x7f6d4c143e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d511ce640 1 -- 172.21.15.31:0/767915436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d44009b30 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d511ce640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b07a0 0x7f6d4c0af790 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800e930 tx=0x7f6d4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d41ffb640 1 -- 172.21.15.31:0/767915436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d48009690 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d41ffb640 1 -- 172.21.15.31:0/767915436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4800ca00 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d41ffb640 1 -- 172.21.15.31:0/767915436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4800cd70 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d53432640 1 -- 172.21.15.31:0/767915436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4c1562e0 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d53432640 1 -- 172.21.15.31:0/767915436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4c144340 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d53432640 1 -- 172.21.15.31:0/767915436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d4c04e850 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.385+0000 7f6d41ffb640 1 -- 172.21.15.31:0/767915436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f6d48013470 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.389+0000 7f6d41ffb640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d30066590 0x7f6d30068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.389+0000 7f6d41ffb640 1 -- 172.21.15.31:0/767915436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d48084260 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.389+0000 7f6d509cd640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d30066590 0x7f6d30068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:37.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.389+0000 7f6d509cd640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d30066590 0x7f6d30068a50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c15e670 tx=0x7f6d3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:37.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.393+0000 7f6d41ffb640 1 -- 172.21.15.31:0/767915436 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6d48051680 con 0x7f6d4c0b07a0 2023-11-26T07:22:37.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:37.573+0000 7f6d53432640 1 -- 172.21.15.31:0/767915436 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d4c0ccee0 con 0x7f6d30066590 2023-11-26T07:22:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.177+0000 7f6d41ffb640 1 -- 172.21.15.31:0/767915436 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f6d3c0012d0 con 0x7f6d30066590 2023-11-26T07:22:40.179 INFO:teuthology.orchestra.run.smithi031.stderr:Volume created successfully (no MDS daemons created) 2023-11-26T07:22:40.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 -- 172.21.15.31:0/767915436 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d30066590 msgr2=0x7f6d30068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d30066590 0x7f6d30068a50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c15e670 tx=0x7f6d3c009040 comp rx=0 tx=0).stop 2023-11-26T07:22:40.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 -- 172.21.15.31:0/767915436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b07a0 msgr2=0x7f6d4c0af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b07a0 0x7f6d4c0af790 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6d4800e930 tx=0x7f6d4800edf0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 -- 172.21.15.31:0/767915436 shutdown_connections 2023-11-26T07:22:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6d30066590 0x7f6d30068a50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d4c161f20 0x7f6d4c143e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6d4c0b15f0 0x7f6d4c1555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 --2- 172.21.15.31:0/767915436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d4c0b07a0 0x7f6d4c0af790 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 -- 172.21.15.31:0/767915436 >> 172.21.15.31:0/767915436 conn(0x7f6d4c0b1d50 msgr2=0x7f6d4c0b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:40.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 -- 172.21.15.31:0/767915436 shutdown_connections 2023-11-26T07:22:40.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.181+0000 7f6d53432640 1 -- 172.21.15.31:0/767915436 wait complete. 2023-11-26T07:22:40.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2023-11-26T07:22:40.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.369+0000 7f922a817640 1 Processor -- start 2023-11-26T07:22:40.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.369+0000 7f922a817640 1 -- start start 2023-11-26T07:22:40.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.369+0000 7f922a817640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f92240b5830 0x7f922415e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.369+0000 7f922a817640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f92240acd70 0x7f92240ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.369+0000 7f922a817640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92240af630 con 0x7f92240ad710 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.369+0000 7f922a817640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92240acb10 con 0x7f92240b5830 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.369+0000 7f922a817640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92240ae5b0 con 0x7f92240acd70 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9223fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f92240b5830 0x7f922415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9228db4640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f92240ad710 0x7f9224161200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:32976/0 (socket says 172.21.15.31:32976) 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9228db4640 1 -- 172.21.15.31:0/1733878602 learned_addr learned my addr 172.21.15.31:0/1733878602 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f92237fe640 1 --2- 172.21.15.31:0/1733878602 >> v2:172.21.15.97:3300/0 conn(0x7f92240acd70 0x7f92240ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9223fff640 1 -- 172.21.15.31:0/1733878602 >> v2:172.21.15.97:3300/0 conn(0x7f92240acd70 msgr2=0x7f92240ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9223fff640 1 --2- 172.21.15.31:0/1733878602 >> v2:172.21.15.97:3300/0 conn(0x7f92240acd70 0x7f92240ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9223fff640 1 -- 172.21.15.31:0/1733878602 >> v1:172.21.15.31:6789/0 conn(0x7f92240ad710 legacy=0x7f9224161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9223fff640 1 -- 172.21.15.31:0/1733878602 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92240ae870 con 0x7f92240b5830 2023-11-26T07:22:40.376 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9222ffd640 1 -- 172.21.15.31:0/1733878602 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934239612 0 0) 0x7f921c0014a0 con 0x7f92240ad710 2023-11-26T07:22:40.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9223fff640 1 --2- 172.21.15.31:0/1733878602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92240b5830 0x7f922415e690 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9224070570 tx=0x7f921401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed3ee09dc15620 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:40.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9222ffd640 1 -- 172.21.15.31:0/1733878602 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92140ad680 con 0x7f92240b5830 2023-11-26T07:22:40.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9222ffd640 1 -- 172.21.15.31:0/1733878602 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92140ad7e0 con 0x7f92240b5830 2023-11-26T07:22:40.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.373+0000 7f9222ffd640 1 -- 172.21.15.31:0/1733878602 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92140adb70 con 0x7f92240b5830 2023-11-26T07:22:40.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 -- 172.21.15.31:0/1733878602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92240b5830 msgr2=0x7f922415e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 --2- 172.21.15.31:0/1733878602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92240b5830 0x7f922415e690 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9224070570 tx=0x7f921401c460 comp rx=0 tx=0).stop 2023-11-26T07:22:40.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 -- 172.21.15.31:0/1733878602 shutdown_connections 2023-11-26T07:22:40.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 --2- 172.21.15.31:0/1733878602 >> v2:172.21.15.97:3300/0 conn(0x7f92240acd70 0x7f92240ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 --2- 172.21.15.31:0/1733878602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92240b5830 0x7f922415e690 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.379 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 -- 172.21.15.31:0/1733878602 >> 172.21.15.31:0/1733878602 conn(0x7f9224071d80 msgr2=0x7f922406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:40.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 -- 172.21.15.31:0/1733878602 shutdown_connections 2023-11-26T07:22:40.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 -- 172.21.15.31:0/1733878602 wait complete. 2023-11-26T07:22:40.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 Processor -- start 2023-11-26T07:22:40.380 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 -- start start 2023-11-26T07:22:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92240acd70 0x7f92241472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92240ad710 0x7f9224154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f92240b5830 0x7f9224152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9224157330 con 0x7f92240ad710 2023-11-26T07:22:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92241477f0 con 0x7f92240acd70 2023-11-26T07:22:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f922a817640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9224156d10 con 0x7f92240b5830 2023-11-26T07:22:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f92237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92240ad710 0x7f9224154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f92237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92240ad710 0x7f9224154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42798/0 (socket says 172.21.15.31:42798) 2023-11-26T07:22:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.377+0000 7f92237fe640 1 -- 172.21.15.31:0/2432689396 learned_addr learned my addr 172.21.15.31:0/2432689396 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f9228db4640 1 --2- 172.21.15.31:0/2432689396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f92240b5830 0x7f9224152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f9223fff640 1 --2- 172.21.15.31:0/2432689396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92240acd70 0x7f92241472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f92237fe640 1 -- 172.21.15.31:0/2432689396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f92240b5830 msgr2=0x7f9224152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f92237fe640 1 --2- 172.21.15.31:0/2432689396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f92240b5830 0x7f9224152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f92237fe640 1 -- 172.21.15.31:0/2432689396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92240acd70 msgr2=0x7f92241472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f92237fe640 1 --2- 172.21.15.31:0/2432689396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92240acd70 0x7f92241472b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f92237fe640 1 -- 172.21.15.31:0/2432689396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9214009420 con 0x7f92240ad710 2023-11-26T07:22:40.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f9223fff640 1 --2- 172.21.15.31:0/2432689396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f92240acd70 0x7f92241472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f9228db4640 1 --2- 172.21.15.31:0/2432689396 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f92240b5830 0x7f9224152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f92237fe640 1 --2- 172.21.15.31:0/2432689396 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92240ad710 0x7f9224154220 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f921800d5f0 tx=0x7f921800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f9220ff9640 1 -- 172.21.15.31:0/2432689396 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92180125f0 con 0x7f92240ad710 2023-11-26T07:22:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f9220ff9640 1 -- 172.21.15.31:0/2432689396 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9218012750 con 0x7f92240ad710 2023-11-26T07:22:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f9220ff9640 1 -- 172.21.15.31:0/2432689396 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218012ae0 con 0x7f92240ad710 2023-11-26T07:22:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f922a817640 1 -- 172.21.15.31:0/2432689396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9224157510 con 0x7f92240ad710 2023-11-26T07:22:40.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f922a817640 1 -- 172.21.15.31:0/2432689396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9224157ad0 con 0x7f92240ad710 2023-11-26T07:22:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f922a817640 1 -- 172.21.15.31:0/2432689396 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922404e850 con 0x7f92240ad710 2023-11-26T07:22:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.381+0000 7f9220ff9640 1 -- 172.21.15.31:0/2432689396 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f9218012c40 con 0x7f92240ad710 2023-11-26T07:22:40.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.385+0000 7f9220ff9640 1 --2- 172.21.15.31:0/2432689396 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9210066590 0x7f9210068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.385+0000 7f9220ff9640 1 -- 172.21.15.31:0/2432689396 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f92180852a0 con 0x7f92240ad710 2023-11-26T07:22:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.385+0000 7f9223fff640 1 --2- 172.21.15.31:0/2432689396 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9210066590 0x7f9210068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.385+0000 7f9223fff640 1 --2- 172.21.15.31:0/2432689396 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9210066590 0x7f9210068a50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92140af000 tx=0x7f9214008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:40.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.389+0000 7f9220ff9640 1 -- 172.21.15.31:0/2432689396 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9218052340 con 0x7f92240ad710 2023-11-26T07:22:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:Invalid command: invalid chars #@ in abcd@# 2023-11-26T07:22:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2023-11-26T07:22:40.569 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2023-11-26T07:22:40.575 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-26T07:22:40.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2023-11-26T07:22:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.717+0000 7f9eccddf640 1 Processor -- start 2023-11-26T07:22:40.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.717+0000 7f9eccddf640 1 -- start start 2023-11-26T07:22:40.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9eccddf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9ec80cc460 0x7f9ec80abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9eccddf640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec80b2710 con 0x7f9ec815a710 2023-11-26T07:22:40.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9eccddf640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec80b29f0 con 0x7f9ec80cdfb0 2023-11-26T07:22:40.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9eccddf640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec80cca80 con 0x7f9ec80cc460 2023-11-26T07:22:40.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ebffff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9ec815a710 0x7f9ec80cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:32986/0 (socket says 172.21.15.31:32986) 2023-11-26T07:22:40.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ebffff640 1 -- 172.21.15.31:0/485798541 learned_addr learned my addr 172.21.15.31:0/485798541 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:40.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec6d9d640 1 --2- 172.21.15.31:0/485798541 >> v2:172.21.15.97:3300/0 conn(0x7f9ec80cc460 0x7f9ec80abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682488125 0 0) 0x7f9ec80b2710 con 0x7f9ec815a710 2023-11-26T07:22:40.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb4003530 con 0x7f9ec815a710 2023-11-26T07:22:40.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 354484373 0 0) 0x7f9eb4003530 con 0x7f9ec815a710 2023-11-26T07:22:40.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 >> v2:172.21.15.97:3300/0 conn(0x7f9ec80cc460 msgr2=0x7f9ec80abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:40.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 --2- 172.21.15.31:0/485798541 >> v2:172.21.15.97:3300/0 conn(0x7f9ec80cc460 0x7f9ec80abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 >> v1:172.21.15.97:6789/0 conn(0x7f9ec80cdfb0 legacy=0x7f9ec80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80b2cd0 con 0x7f9ec815a710 2023-11-26T07:22:40.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f9eb0001680 con 0x7f9ec815a710 2023-11-26T07:22:40.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 >> v1:172.21.15.31:6789/0 conn(0x7f9ec815a710 legacy=0x7f9ec80cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eb4003980 0x7f9eb4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9eb4004380 0x7f9eb4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9eb4005050 0x7f9eb400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80cca80 con 0x7f9eb4005050 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb400bfb0 con 0x7f9eb4003980 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb400c170 con 0x7f9eb4004380 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec6d9d640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9eb4005050 0x7f9eb400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec659c640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eb4003980 0x7f9eb4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ebffff640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9eb4004380 0x7f9eb4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec6d9d640 1 -- 172.21.15.31:0/485798541 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9eb4004380 msgr2=0x7f9eb4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:40.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec6d9d640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9eb4004380 0x7f9eb4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec6d9d640 1 -- 172.21.15.31:0/485798541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eb4003980 msgr2=0x7f9eb4003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:40.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec6d9d640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9eb4003980 0x7f9eb4003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.721+0000 7f9ec6d9d640 1 -- 172.21.15.31:0/485798541 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4003530 con 0x7f9eb4005050 2023-11-26T07:22:40.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9ec6d9d640 1 -- 172.21.15.31:0/485798541 reap_dead start 2023-11-26T07:22:40.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9ec6d9d640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9eb4005050 0x7f9eb400b990 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009fc0 tx=0x7f9ec001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a3e319b2996133 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:40.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0003ba0 con 0x7f9eb4005050 2023-11-26T07:22:40.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec000f040 con 0x7f9eb4005050 2023-11-26T07:22:40.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9ec5d9b640 1 -- 172.21.15.31:0/485798541 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec00a59a0 con 0x7f9eb4005050 2023-11-26T07:22:40.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9eccddf640 1 -- 172.21.15.31:0/485798541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9eb4005050 msgr2=0x7f9eb400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9eccddf640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9eb4005050 0x7f9eb400b990 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009fc0 tx=0x7f9ec001cca0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9eccddf640 1 -- 172.21.15.31:0/485798541 shutdown_connections 2023-11-26T07:22:40.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9eccddf640 1 --2- 172.21.15.31:0/485798541 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9eb4005050 0x7f9eb400b990 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9eccddf640 1 -- 172.21.15.31:0/485798541 >> 172.21.15.31:0/485798541 conn(0x7f9ec80b0cb0 msgr2=0x7f9ec80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:40.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9eccddf640 1 -- 172.21.15.31:0/485798541 shutdown_connections 2023-11-26T07:22:40.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.725+0000 7f9eccddf640 1 -- 172.21.15.31:0/485798541 wait complete. 2023-11-26T07:22:40.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9eccddf640 1 Processor -- start 2023-11-26T07:22:40.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9eccddf640 1 -- start start 2023-11-26T07:22:40.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9eccddf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9eb4005050 0x7f9ec8071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9eccddf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec8070f60 0x7f9ec81ee280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9eccddf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ec81ee7c0 0x7f9ec80cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9eccddf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec815c690 con 0x7f9eb4005050 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9eccddf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80728f0 con 0x7f9ec8070f60 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9eccddf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec8071d70 con 0x7f9ec81ee7c0 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ec659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9eb4005050 0x7f9ec8071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ec659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9eb4005050 0x7f9ec8071520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42820/0 (socket says 172.21.15.31:42820) 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ec659c640 1 -- 172.21.15.31:0/3935322384 learned_addr learned my addr 172.21.15.31:0/3935322384 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ebffff640 1 --2- 172.21.15.31:0/3935322384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec8070f60 0x7f9ec81ee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ec6d9d640 1 --2- 172.21.15.31:0/3935322384 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ec81ee7c0 0x7f9ec80cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ec659c640 1 -- 172.21.15.31:0/3935322384 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ec81ee7c0 msgr2=0x7f9ec80cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ec659c640 1 --2- 172.21.15.31:0/3935322384 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ec81ee7c0 0x7f9ec80cef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ec659c640 1 -- 172.21.15.31:0/3935322384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec8070f60 msgr2=0x7f9ec81ee280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:40.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ec659c640 1 --2- 172.21.15.31:0/3935322384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec8070f60 0x7f9ec81ee280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:40.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ec659c640 1 -- 172.21.15.31:0/3935322384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80cc8f0 con 0x7f9eb4005050 2023-11-26T07:22:40.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ec659c640 1 --2- 172.21.15.31:0/3935322384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9eb4005050 0x7f9ec8071520 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800f840 tx=0x7f9eb800fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:40.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ebeffd640 1 -- 172.21.15.31:0/3935322384 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb80014a0 con 0x7f9eb4005050 2023-11-26T07:22:40.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ebeffd640 1 -- 172.21.15.31:0/3935322384 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb8005320 con 0x7f9eb4005050 2023-11-26T07:22:40.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9eccddf640 1 -- 172.21.15.31:0/3935322384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec8072050 con 0x7f9eb4005050 2023-11-26T07:22:40.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9ebeffd640 1 -- 172.21.15.31:0/3935322384 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8006a20 con 0x7f9eb4005050 2023-11-26T07:22:40.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.729+0000 7f9eccddf640 1 -- 172.21.15.31:0/3935322384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec8072610 con 0x7f9eb4005050 2023-11-26T07:22:40.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.733+0000 7f9eccddf640 1 -- 172.21.15.31:0/3935322384 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec80671f0 con 0x7f9eb4005050 2023-11-26T07:22:40.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.733+0000 7f9ebeffd640 1 -- 172.21.15.31:0/3935322384 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f9eb8006b80 con 0x7f9eb4005050 2023-11-26T07:22:40.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.733+0000 7f9ebeffd640 1 --2- 172.21.15.31:0/3935322384 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9ea8066590 0x7f9ea8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:40.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.733+0000 7f9ebeffd640 1 -- 172.21.15.31:0/3935322384 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f9eb8083aa0 con 0x7f9eb4005050 2023-11-26T07:22:40.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.733+0000 7f9ebffff640 1 --2- 172.21.15.31:0/3935322384 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9ea8066590 0x7f9ea8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:40.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.733+0000 7f9ebffff640 1 --2- 172.21.15.31:0/3935322384 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9ea8066590 0x7f9ea8068a50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8155a10 tx=0x7f9eb0001990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:40.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:40.737+0000 7f9ebeffd640 1 -- 172.21.15.31:0/3935322384 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9eb8050ae0 con 0x7f9eb4005050 2023-11-26T07:22:40.923 INFO:teuthology.orchestra.run.smithi031.stderr:Invalid command: invalid chars @# in abcd@# 2023-11-26T07:22:40.923 INFO:teuthology.orchestra.run.smithi031.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2023-11-26T07:22:40.923 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2023-11-26T07:22:40.929 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-26T07:22:40.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2023-11-26T07:22:41.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.073+0000 7ff53dd65640 1 Processor -- start 2023-11-26T07:22:41.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.073+0000 7ff53dd65640 1 -- start start 2023-11-26T07:22:41.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.073+0000 7ff53dd65640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cc460 0x7ff5380abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.073+0000 7ff53dd65640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5380b2710 con 0x7ff53815a710 2023-11-26T07:22:41.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.073+0000 7ff53dd65640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5380b29f0 con 0x7ff5380cdfb0 2023-11-26T07:22:41.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.073+0000 7ff53dd65640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380cca80 con 0x7ff5380cc460 2023-11-26T07:22:41.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.073+0000 7ff536ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff53815a710 0x7ff5380cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:32994/0 (socket says 172.21.15.31:32994) 2023-11-26T07:22:41.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.073+0000 7ff536ffd640 1 -- 172.21.15.31:0/4039351395 learned_addr learned my addr 172.21.15.31:0/4039351395 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:41.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.073+0000 7ff537fff640 1 --2- 172.21.15.31:0/4039351395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cc460 0x7ff5380abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff5367fc640 1 -- 172.21.15.31:0/4039351395 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364249251 0 0) 0x7ff5380b2710 con 0x7ff53815a710 2023-11-26T07:22:41.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff5367fc640 1 -- 172.21.15.31:0/4039351395 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff520003530 con 0x7ff53815a710 2023-11-26T07:22:41.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff5367fc640 1 -- 172.21.15.31:0/4039351395 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447599558 0 0) 0x7ff5380b29f0 con 0x7ff5380cdfb0 2023-11-26T07:22:41.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff5367fc640 1 -- 172.21.15.31:0/4039351395 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5380b2710 con 0x7ff5380cdfb0 2023-11-26T07:22:41.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff537fff640 1 -- 172.21.15.31:0/4039351395 >> v1:172.21.15.97:6789/0 conn(0x7ff5380cdfb0 legacy=0x7ff5380ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff537fff640 1 -- 172.21.15.31:0/4039351395 >> v1:172.21.15.31:6789/0 conn(0x7ff53815a710 legacy=0x7ff5380cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff537fff640 1 -- 172.21.15.31:0/4039351395 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5380b2cd0 con 0x7ff5380cc460 2023-11-26T07:22:41.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff537fff640 1 --2- 172.21.15.31:0/4039351395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cc460 0x7ff5380abf90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff530009dc0 tx=0x7ff53001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db93f377ae5e3a69 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff5367fc640 1 -- 172.21.15.31:0/4039351395 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5300a5a20 con 0x7ff5380cc460 2023-11-26T07:22:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff5367fc640 1 -- 172.21.15.31:0/4039351395 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5300a5b80 con 0x7ff5380cc460 2023-11-26T07:22:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.077+0000 7ff5367fc640 1 -- 172.21.15.31:0/4039351395 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5300ad440 con 0x7ff5380cc460 2023-11-26T07:22:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 -- 172.21.15.31:0/4039351395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cc460 msgr2=0x7ff5380abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 --2- 172.21.15.31:0/4039351395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cc460 0x7ff5380abf90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff530009dc0 tx=0x7ff53001c940 comp rx=0 tx=0).stop 2023-11-26T07:22:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 -- 172.21.15.31:0/4039351395 shutdown_connections 2023-11-26T07:22:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 --2- 172.21.15.31:0/4039351395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cc460 0x7ff5380abf90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 -- 172.21.15.31:0/4039351395 >> 172.21.15.31:0/4039351395 conn(0x7ff5380b0cb0 msgr2=0x7ff5380b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 -- 172.21.15.31:0/4039351395 shutdown_connections 2023-11-26T07:22:41.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 -- 172.21.15.31:0/4039351395 wait complete. 2023-11-26T07:22:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 Processor -- start 2023-11-26T07:22:41.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 -- start start 2023-11-26T07:22:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5380cc460 0x7ff538151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cdfb0 0x7ff538153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff53815a710 0x7ff5381540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53809c800 con 0x7ff53815a710 2023-11-26T07:22:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53809bb80 con 0x7ff5380cc460 2023-11-26T07:22:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff53dd65640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53809be60 con 0x7ff5380cdfb0 2023-11-26T07:22:41.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff537fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff53815a710 0x7ff5381540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff537fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff53815a710 0x7ff5381540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42830/0 (socket says 172.21.15.31:42830) 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff537fff640 1 -- 172.21.15.31:0/3794005073 learned_addr learned my addr 172.21.15.31:0/3794005073 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff5377fe640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5380cc460 0x7ff538151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff536ffd640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cdfb0 0x7ff538153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff537fff640 1 -- 172.21.15.31:0/3794005073 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cdfb0 msgr2=0x7ff538153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff537fff640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cdfb0 0x7ff538153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff537fff640 1 -- 172.21.15.31:0/3794005073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5380cc460 msgr2=0x7ff538151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff537fff640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5380cc460 0x7ff538151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff537fff640 1 -- 172.21.15.31:0/3794005073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff530009a70 con 0x7ff53815a710 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.081+0000 7ff536ffd640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cdfb0 0x7ff538153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:41.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff537fff640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff53815a710 0x7ff5381540b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff530019550 tx=0x7ff5300196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff51ffff640 1 -- 172.21.15.31:0/3794005073 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530007000 con 0x7ff53815a710 2023-11-26T07:22:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff51ffff640 1 -- 172.21.15.31:0/3794005073 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff530009450 con 0x7ff53815a710 2023-11-26T07:22:41.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff53dd65640 1 -- 172.21.15.31:0/3794005073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53809c140 con 0x7ff53815a710 2023-11-26T07:22:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff51ffff640 1 -- 172.21.15.31:0/3794005073 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5300a5a20 con 0x7ff53815a710 2023-11-26T07:22:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff53dd65640 1 -- 172.21.15.31:0/3794005073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff538152010 con 0x7ff53815a710 2023-11-26T07:22:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff53dd65640 1 -- 172.21.15.31:0/3794005073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5380671f0 con 0x7ff53815a710 2023-11-26T07:22:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff51ffff640 1 -- 172.21.15.31:0/3794005073 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ff5300b4050 con 0x7ff53815a710 2023-11-26T07:22:41.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff51ffff640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff524066590 0x7ff524068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff51ffff640 1 -- 172.21.15.31:0/3794005073 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6543+0+0 (secure 0 0 0) 0x7ff53011f650 con 0x7ff53815a710 2023-11-26T07:22:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.085+0000 7ff5377fe640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff524066590 0x7ff524068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:41.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.089+0000 7ff5377fe640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff524066590 0x7ff524068a50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff5280011f0 tx=0x7ff528009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:41.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.089+0000 7ff51ffff640 1 -- 172.21.15.31:0/3794005073 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff5300ec5e0 con 0x7ff53815a710 2023-11-26T07:22:41.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:41.273+0000 7ff53dd65640 1 -- 172.21.15.31:0/3794005073 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7ff53809bfd0 con 0x7ff524066590 2023-11-26T07:22:41.280 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-11-26T07:22:41.277+0000 7fe86499d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:22:41.330 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2023-11-26T07:22:44.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.297+0000 7ff51ffff640 1 -- 172.21.15.31:0/3794005073 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+78 (secure 0 0 0) 0x7ff53809bfd0 con 0x7ff524066590 2023-11-26T07:22:44.301 INFO:teuthology.orchestra.run.smithi031.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2023-11-26T07:22:44.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 -- 172.21.15.31:0/3794005073 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff524066590 msgr2=0x7ff524068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff524066590 0x7ff524068a50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff5280011f0 tx=0x7ff528009040 comp rx=0 tx=0).stop 2023-11-26T07:22:44.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 -- 172.21.15.31:0/3794005073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff53815a710 msgr2=0x7ff5381540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff53815a710 0x7ff5381540b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff530019550 tx=0x7ff5300196e0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 -- 172.21.15.31:0/3794005073 shutdown_connections 2023-11-26T07:22:44.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff524066590 0x7ff524068a50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff53815a710 0x7ff5381540b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff5380cdfb0 0x7ff538153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 --2- 172.21.15.31:0/3794005073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5380cc460 0x7ff538151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 -- 172.21.15.31:0/3794005073 >> 172.21.15.31:0/3794005073 conn(0x7ff5380b0cb0 msgr2=0x7ff5380ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 -- 172.21.15.31:0/3794005073 shutdown_connections 2023-11-26T07:22:44.306 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.301+0000 7ff53dd65640 1 -- 172.21.15.31:0/3794005073 wait complete. 2023-11-26T07:22:44.331 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:22:44.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:44.381 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:44.468 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:22:44.468 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:22:44.468 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:44.478 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:22:44.478 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:44.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.721+0000 7f8d10032640 1 Processor -- start 2023-11-26T07:22:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.721+0000 7f8d10032640 1 -- start start 2023-11-26T07:22:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d10032640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f8d080b39c0 0x7f8d0806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:44.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d10032640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0806cf90 con 0x7f8d0815f540 2023-11-26T07:22:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d10032640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d080b2650 con 0x7f8d0806cb70 2023-11-26T07:22:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d10032640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d080b2920 con 0x7f8d080b39c0 2023-11-26T07:22:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0ddce640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8d0815f540 0x7f8d080b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59510/0 (socket says 172.21.15.31:59510) 2023-11-26T07:22:44.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0ddce640 1 -- 172.21.15.31:0/2971883948 learned_addr learned my addr 172.21.15.31:0/2971883948 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:44.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0d5cd640 1 --2- 172.21.15.31:0/2971883948 >> v2:172.21.15.97:3300/0 conn(0x7f8d080b39c0 0x7f8d0806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:44.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935241466 0 0) 0x7f8d0806cf90 con 0x7f8d0815f540 2023-11-26T07:22:44.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cf4003530 con 0x7f8d0815f540 2023-11-26T07:22:44.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1807262899 0 0) 0x7f8cf4003530 con 0x7f8d0815f540 2023-11-26T07:22:44.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 >> v2:172.21.15.97:3300/0 conn(0x7f8d080b39c0 msgr2=0x7f8d0806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:44.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 --2- 172.21.15.31:0/2971883948 >> v2:172.21.15.97:3300/0 conn(0x7f8d080b39c0 0x7f8d0806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 >> v1:172.21.15.97:6789/0 conn(0x7f8d0806cb70 legacy=0x7f8d08070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d080b2c00 con 0x7f8d0815f540 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f8d04001680 con 0x7f8d0815f540 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 >> v1:172.21.15.31:6789/0 conn(0x7f8d0815f540 legacy=0x7f8d080b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cf4003980 0x7f8cf4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cf40047d0 0x7f8cf4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8cf4005190 0x7f8cf400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d080b2920 con 0x7f8cf40047d0 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf400c130 con 0x7f8cf4003980 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf400c2f0 con 0x7f8cf4005190 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0d5cd640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cf40047d0 0x7f8cf4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0ddce640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cf4003980 0x7f8cf4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:44.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0e5cf640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8cf4005190 0x7f8cf400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:44.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0d5cd640 1 -- 172.21.15.31:0/2971883948 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8cf4005190 msgr2=0x7f8cf400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:44.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0d5cd640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8cf4005190 0x7f8cf400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0d5cd640 1 -- 172.21.15.31:0/2971883948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cf4003980 msgr2=0x7f8cf4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:44.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0d5cd640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cf4003980 0x7f8cf4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0d5cd640 1 -- 172.21.15.31:0/2971883948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf4003530 con 0x7f8cf40047d0 2023-11-26T07:22:44.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.725+0000 7f8d0e5cf640 1 -- 172.21.15.31:0/2971883948 reap_dead start 2023-11-26T07:22:44.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d0d5cd640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cf40047d0 0x7f8cf4004c50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009ae0 tx=0x7f8cf801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fd813942693135 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:44.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80a6510 con 0x7f8cf40047d0 2023-11-26T07:22:44.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf80a6670 con 0x7f8cf40047d0 2023-11-26T07:22:44.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d0cdcc640 1 -- 172.21.15.31:0/2971883948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80a6a20 con 0x7f8cf40047d0 2023-11-26T07:22:44.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d10032640 1 -- 172.21.15.31:0/2971883948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cf40047d0 msgr2=0x7f8cf4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d10032640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cf40047d0 0x7f8cf4004c50 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8009ae0 tx=0x7f8cf801ca20 comp rx=0 tx=0).stop 2023-11-26T07:22:44.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d10032640 1 -- 172.21.15.31:0/2971883948 shutdown_connections 2023-11-26T07:22:44.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d10032640 1 --2- 172.21.15.31:0/2971883948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8cf40047d0 0x7f8cf4004c50 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d10032640 1 -- 172.21.15.31:0/2971883948 >> 172.21.15.31:0/2971883948 conn(0x7f8d080b0c40 msgr2=0x7f8d080b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:44.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d10032640 1 -- 172.21.15.31:0/2971883948 shutdown_connections 2023-11-26T07:22:44.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.729+0000 7f8d10032640 1 -- 172.21.15.31:0/2971883948 wait complete. 2023-11-26T07:22:44.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 Processor -- start 2023-11-26T07:22:44.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 -- start start 2023-11-26T07:22:44.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8cf40047d0 0x7f8d0809ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:44.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d081ecd00 0x7f8d08160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:44.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d081552b0 0x7f8d08155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:44.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d08155c70 con 0x7f8d081ecd00 2023-11-26T07:22:44.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0809b4a0 con 0x7f8d081552b0 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0809b780 con 0x7f8cf40047d0 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0d5cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d081ecd00 0x7f8d08160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0d5cd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d081ecd00 0x7f8d08160bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41648/0 (socket says 172.21.15.31:41648) 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0d5cd640 1 -- 172.21.15.31:0/3266656440 learned_addr learned my addr 172.21.15.31:0/3266656440 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0ddce640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8cf40047d0 0x7f8d0809ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0e5cf640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d081552b0 0x7f8d08155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0d5cd640 1 -- 172.21.15.31:0/3266656440 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8cf40047d0 msgr2=0x7f8d0809ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0d5cd640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8cf40047d0 0x7f8d0809ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0d5cd640 1 -- 172.21.15.31:0/3266656440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d081552b0 msgr2=0x7f8d08155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0d5cd640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d081552b0 0x7f8d08155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0d5cd640 1 -- 172.21.15.31:0/3266656440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0809ba60 con 0x7f8d081ecd00 2023-11-26T07:22:44.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d0d5cd640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d081ecd00 0x7f8d08160bd0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80b0040 tx=0x7f8cf80197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8cfe7fc640 1 -- 172.21.15.31:0/3266656440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf800f080 con 0x7f8d081ecd00 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8cfe7fc640 1 -- 172.21.15.31:0/3266656440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf80a67e0 con 0x7f8d081ecd00 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8cfe7fc640 1 -- 172.21.15.31:0/3266656440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80a6b50 con 0x7f8d081ecd00 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 -- 172.21.15.31:0/3266656440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0809bd40 con 0x7f8d081ecd00 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 -- 172.21.15.31:0/3266656440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0809b190 con 0x7f8d081ecd00 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.733+0000 7f8d10032640 1 -- 172.21.15.31:0/3266656440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0804e850 con 0x7f8d081ecd00 2023-11-26T07:22:44.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.737+0000 7f8cfe7fc640 1 -- 172.21.15.31:0/3266656440 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f8cf80a6cb0 con 0x7f8d081ecd00 2023-11-26T07:22:44.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.737+0000 7f8cfe7fc640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8cec066590 0x7f8cec068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:44.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.737+0000 7f8cfe7fc640 1 -- 172.21.15.31:0/3266656440 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8cf80aa070 con 0x7f8d081ecd00 2023-11-26T07:22:44.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.737+0000 7f8d0ddce640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8cec066590 0x7f8cec068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:44.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.737+0000 7f8d0ddce640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8cec066590 0x7f8cec068a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8d04000c10 tx=0x7f8d04001b30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:44.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.741+0000 7f8cfe7fc640 1 -- 172.21.15.31:0/3266656440 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8cf80ef730 con 0x7f8d081ecd00 2023-11-26T07:22:44.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.897+0000 7f8d10032640 1 -- 172.21.15.31:0/3266656440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d08161f70 con 0x7f8d081ecd00 2023-11-26T07:22:44.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8cfe7fc640 1 -- 172.21.15.31:0/3266656440 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16603 (secure 0 0 0) 0x7f8cf80ef2c0 con 0x7f8d081ecd00 2023-11-26T07:22:44.903 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:44.903 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:44.288769+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:20.941587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:21.520554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/683007803":"2023-11-27T07:22:41.280769+0000","172.21.15.97:6833/683007803":"2023-11-27T07:22:41.280769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:44.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 -- 172.21.15.31:0/3266656440 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8cec066590 msgr2=0x7f8cec068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8cec066590 0x7f8cec068a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8d04000c10 tx=0x7f8d04001b30 comp rx=0 tx=0).stop 2023-11-26T07:22:44.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 -- 172.21.15.31:0/3266656440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d081ecd00 msgr2=0x7f8d08160bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:44.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d081ecd00 0x7f8d08160bd0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8cf80b0040 tx=0x7f8cf80197c0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 -- 172.21.15.31:0/3266656440 shutdown_connections 2023-11-26T07:22:44.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f8cec066590 0x7f8cec068a50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d081552b0 0x7f8d08155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d081ecd00 0x7f8d08160bd0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 --2- 172.21.15.31:0/3266656440 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f8cf40047d0 0x7f8d0809ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:44.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 -- 172.21.15.31:0/3266656440 >> 172.21.15.31:0/3266656440 conn(0x7f8d080b0c40 msgr2=0x7f8d080aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:44.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 -- 172.21.15.31:0/3266656440 shutdown_connections 2023-11-26T07:22:44.906 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:44.901+0000 7f8d10032640 1 -- 172.21.15.31:0/3266656440 wait complete. 2023-11-26T07:22:44.924 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:22:20.941587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:22:21.520554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:44.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:45.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd370780640 1 Processor -- start 2023-11-26T07:22:45.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd370780640 1 -- start start 2023-11-26T07:22:45.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd370780640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd36806e120 0x7fd36815df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd370780640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3680cd870 con 0x7fd36815e4d0 2023-11-26T07:22:45.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd370780640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36815ecc0 con 0x7fd368154940 2023-11-26T07:22:45.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd370780640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd368154d20 con 0x7fd36806e120 2023-11-26T07:22:45.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36ed1d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd36815e4d0 0x7fd3680ad080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59516/0 (socket says 172.21.15.31:59516) 2023-11-26T07:22:45.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36ed1d640 1 -- 172.21.15.31:0/214694772 learned_addr learned my addr 172.21.15.31:0/214694772 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:45.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36dd1b640 1 --2- 172.21.15.31:0/214694772 >> v2:172.21.15.97:3300/0 conn(0x7fd36806e120 0x7fd36815df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712471719 0 0) 0x7fd3680cd870 con 0x7fd36815e4d0 2023-11-26T07:22:45.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd354003530 con 0x7fd36815e4d0 2023-11-26T07:22:45.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 515439486 0 0) 0x7fd354003530 con 0x7fd36815e4d0 2023-11-26T07:22:45.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 >> v2:172.21.15.97:3300/0 conn(0x7fd36806e120 msgr2=0x7fd36815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:45.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 --2- 172.21.15.31:0/214694772 >> v2:172.21.15.97:3300/0 conn(0x7fd36806e120 0x7fd36815df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 >> v1:172.21.15.97:6789/0 conn(0x7fd368154940 legacy=0x7fd36806da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3680ccf40 con 0x7fd36815e4d0 2023-11-26T07:22:45.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fd360001a00 con 0x7fd36815e4d0 2023-11-26T07:22:45.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36dd1b640 1 --2- 172.21.15.31:0/214694772 >> v2:172.21.15.97:3300/0 conn(0x7fd36806e120 0x7fd36815df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 >> v1:172.21.15.31:6789/0 conn(0x7fd36815e4d0 legacy=0x7fd3680ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd354003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd3540047d0 0x7fd354004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd354005190 0x7fd35400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd368154d20 con 0x7fd354003980 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35400c030 con 0x7fd354005190 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35400c1f0 con 0x7fd3540047d0 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36e51c640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd354003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36dd1b640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd3540047d0 0x7fd354004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.081+0000 7fd36ed1d640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd354005190 0x7fd35400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36e51c640 1 -- 172.21.15.31:0/214694772 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd3540047d0 msgr2=0x7fd354004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:45.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36e51c640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd3540047d0 0x7fd354004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36e51c640 1 -- 172.21.15.31:0/214694772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd354005190 msgr2=0x7fd35400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:45.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36e51c640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd354005190 0x7fd35400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36e51c640 1 -- 172.21.15.31:0/214694772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd354003530 con 0x7fd354003980 2023-11-26T07:22:45.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36ed1d640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd354005190 0x7fd35400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:45.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36ed1d640 1 -- 172.21.15.31:0/214694772 reap_dead start 2023-11-26T07:22:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36e51c640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd354003da0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd364001780 tx=0x7fd36401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9129b3ca4d433ba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3640052d0 con 0x7fd354003980 2023-11-26T07:22:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3640a7050 con 0x7fd354003980 2023-11-26T07:22:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd36d51a640 1 -- 172.21.15.31:0/214694772 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd364003600 con 0x7fd354003980 2023-11-26T07:22:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd370780640 1 -- 172.21.15.31:0/214694772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 msgr2=0x7fd354003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd370780640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd354003da0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd364001780 tx=0x7fd36401c3b0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd370780640 1 -- 172.21.15.31:0/214694772 shutdown_connections 2023-11-26T07:22:45.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd370780640 1 --2- 172.21.15.31:0/214694772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd354003da0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.085+0000 7fd370780640 1 -- 172.21.15.31:0/214694772 >> 172.21.15.31:0/214694772 conn(0x7fd3680b0cb0 msgr2=0x7fd3680b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:45.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd370780640 1 -- 172.21.15.31:0/214694772 shutdown_connections 2023-11-26T07:22:45.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd370780640 1 -- 172.21.15.31:0/214694772 wait complete. 2023-11-26T07:22:45.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd370780640 1 Processor -- start 2023-11-26T07:22:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd370780640 1 -- start start 2023-11-26T07:22:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd370780640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd36814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd370780640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd36814c8c0 0x7fd368152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd370780640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3681532c0 0x7fd36809ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd370780640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36814f760 con 0x7fd354003980 2023-11-26T07:22:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd370780640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd368153740 con 0x7fd3681532c0 2023-11-26T07:22:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd370780640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3681500c0 con 0x7fd36814c8c0 2023-11-26T07:22:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36e51c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd36814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36e51c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd36814fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41670/0 (socket says 172.21.15.31:41670) 2023-11-26T07:22:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36e51c640 1 -- 172.21.15.31:0/1845630247 learned_addr learned my addr 172.21.15.31:0/1845630247 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36dd1b640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd36814c8c0 0x7fd368152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36ed1d640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3681532c0 0x7fd36809ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36e51c640 1 -- 172.21.15.31:0/1845630247 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd36814c8c0 msgr2=0x7fd368152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36e51c640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd36814c8c0 0x7fd368152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36e51c640 1 -- 172.21.15.31:0/1845630247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3681532c0 msgr2=0x7fd36809ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36e51c640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3681532c0 0x7fd36809ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36e51c640 1 -- 172.21.15.31:0/1845630247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36814e840 con 0x7fd354003980 2023-11-26T07:22:45.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.089+0000 7fd36e51c640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd36814fb80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd36401c1e0 tx=0x7fd354005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.093+0000 7fd35effd640 1 -- 172.21.15.31:0/1845630247 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd364007500 con 0x7fd354003980 2023-11-26T07:22:45.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.093+0000 7fd35effd640 1 -- 172.21.15.31:0/1845630247 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd364007660 con 0x7fd354003980 2023-11-26T07:22:45.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.093+0000 7fd35effd640 1 -- 172.21.15.31:0/1845630247 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3640069e0 con 0x7fd354003980 2023-11-26T07:22:45.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.093+0000 7fd370780640 1 -- 172.21.15.31:0/1845630247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36814eae0 con 0x7fd354003980 2023-11-26T07:22:45.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.093+0000 7fd370780640 1 -- 172.21.15.31:0/1845630247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36814f0a0 con 0x7fd354003980 2023-11-26T07:22:45.097 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.093+0000 7fd370780640 1 -- 172.21.15.31:0/1845630247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3680671f0 con 0x7fd354003980 2023-11-26T07:22:45.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.093+0000 7fd35effd640 1 -- 172.21.15.31:0/1845630247 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fd364006b40 con 0x7fd354003980 2023-11-26T07:22:45.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.093+0000 7fd35effd640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd34c066590 0x7fd34c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.093+0000 7fd35effd640 1 -- 172.21.15.31:0/1845630247 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd36411e4f0 con 0x7fd354003980 2023-11-26T07:22:45.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.093+0000 7fd36dd1b640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd34c066590 0x7fd34c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.097+0000 7fd36dd1b640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd34c066590 0x7fd34c068a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd36814e370 tx=0x7fd358008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.097+0000 7fd35effd640 1 -- 172.21.15.31:0/1845630247 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd3640eb6e0 con 0x7fd354003980 2023-11-26T07:22:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 -- 172.21.15.31:0/1845630247 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd36815de00 con 0x7fd354003980 2023-11-26T07:22:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd35effd640 1 -- 172.21.15.31:0/1845630247 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3640eed00 con 0x7fd354003980 2023-11-26T07:22:45.260 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:22:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 -- 172.21.15.31:0/1845630247 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd34c066590 msgr2=0x7fd34c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd34c066590 0x7fd34c068a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd36814e370 tx=0x7fd358008040 comp rx=0 tx=0).stop 2023-11-26T07:22:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 -- 172.21.15.31:0/1845630247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 msgr2=0x7fd36814fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd36814fb80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd36401c1e0 tx=0x7fd354005800 comp rx=0 tx=0).stop 2023-11-26T07:22:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 -- 172.21.15.31:0/1845630247 shutdown_connections 2023-11-26T07:22:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd34c066590 0x7fd34c068a50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3681532c0 0x7fd36809ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd36814c8c0 0x7fd368152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 --2- 172.21.15.31:0/1845630247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd354003980 0x7fd36814fb80 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 -- 172.21.15.31:0/1845630247 >> 172.21.15.31:0/1845630247 conn(0x7fd3680b0cb0 msgr2=0x7fd3680adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:45.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 -- 172.21.15.31:0/1845630247 shutdown_connections 2023-11-26T07:22:45.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.257+0000 7fd370780640 1 -- 172.21.15.31:0/1845630247 wait complete. 2023-11-26T07:22:45.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f80511df640 1 Processor -- start 2023-11-26T07:22:45.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f80511df640 1 -- start start 2023-11-26T07:22:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f80511df640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f804c15d8f0 0x7f804c15e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f80511df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f804c0d0510 0x7f804c0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f80511df640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804c15ecc0 con 0x7f804c0ced30 2023-11-26T07:22:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f80511df640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c0d4420 con 0x7f804c15d8f0 2023-11-26T07:22:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f80511df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c0cdfb0 con 0x7f804c0d0510 2023-11-26T07:22:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f804ad9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f804c15d8f0 0x7f804c15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f8043fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f804c0ced30 0x7f804c0cfe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59522/0 (socket says 172.21.15.31:59522) 2023-11-26T07:22:45.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f804ad9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f804c15d8f0 0x7f804c15e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41676/0 (socket says 172.21.15.31:41676) 2023-11-26T07:22:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f8043fff640 1 -- 172.21.15.31:0/2345798659 learned_addr learned my addr 172.21.15.31:0/2345798659 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f804b59e640 1 --2- 172.21.15.31:0/2345798659 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f804c0d0510 0x7f804c0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f804ad9d640 1 -- 172.21.15.31:0/2345798659 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f804c0d0510 msgr2=0x7f804c0d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f804ad9d640 1 --2- 172.21.15.31:0/2345798659 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f804c0d0510 0x7f804c0d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f804ad9d640 1 -- 172.21.15.31:0/2345798659 >> v1:172.21.15.31:6789/0 conn(0x7f804c0ced30 legacy=0x7f804c0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f804ad9d640 1 -- 172.21.15.31:0/2345798659 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c0d4140 con 0x7f804c15d8f0 2023-11-26T07:22:45.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.425+0000 7f804a59c640 1 -- 172.21.15.31:0/2345798659 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118408075 0 0) 0x7f804c15ecc0 con 0x7f804c0ced30 2023-11-26T07:22:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f804ad9d640 1 --2- 172.21.15.31:0/2345798659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 0x7f804c15e780 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f803c0097f0 tx=0x7f803c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f9cd352c3a2ad6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f804a59c640 1 -- 172.21.15.31:0/2345798659 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c0ab650 con 0x7f804c15d8f0 2023-11-26T07:22:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f804a59c640 1 -- 172.21.15.31:0/2345798659 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c0ab7b0 con 0x7f804c15d8f0 2023-11-26T07:22:45.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f804a59c640 1 -- 172.21.15.31:0/2345798659 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c0abb40 con 0x7f804c15d8f0 2023-11-26T07:22:45.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f80511df640 1 -- 172.21.15.31:0/2345798659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 msgr2=0x7f804c15e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f80511df640 1 --2- 172.21.15.31:0/2345798659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 0x7f804c15e780 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f803c0097f0 tx=0x7f803c01c450 comp rx=0 tx=0).stop 2023-11-26T07:22:45.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f80511df640 1 -- 172.21.15.31:0/2345798659 shutdown_connections 2023-11-26T07:22:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f80511df640 1 --2- 172.21.15.31:0/2345798659 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f804c0d0510 0x7f804c0d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f80511df640 1 --2- 172.21.15.31:0/2345798659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 0x7f804c15e780 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f80511df640 1 -- 172.21.15.31:0/2345798659 >> 172.21.15.31:0/2345798659 conn(0x7f804c09c750 msgr2=0x7f804c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f80511df640 1 -- 172.21.15.31:0/2345798659 shutdown_connections 2023-11-26T07:22:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.429+0000 7f80511df640 1 -- 172.21.15.31:0/2345798659 wait complete. 2023-11-26T07:22:45.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80511df640 1 Processor -- start 2023-11-26T07:22:45.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80511df640 1 -- start start 2023-11-26T07:22:45.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80511df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f804c0ced30 0x7f804c154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80511df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f804c0d0510 0x7f804c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80511df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 0x7f804c0ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80511df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c155c10 con 0x7f804c15d8f0 2023-11-26T07:22:45.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80511df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c155ef0 con 0x7f804c0ced30 2023-11-26T07:22:45.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80511df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c1561d0 con 0x7f804c0d0510 2023-11-26T07:22:45.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f804b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 0x7f804c0ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f804b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 0x7f804c0ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41686/0 (socket says 172.21.15.31:41686) 2023-11-26T07:22:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f804b59e640 1 -- 172.21.15.31:0/4208847563 learned_addr learned my addr 172.21.15.31:0/4208847563 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f804ad9d640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f804c0ced30 0x7f804c154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f8043fff640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f804c0d0510 0x7f804c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f804b59e640 1 -- 172.21.15.31:0/4208847563 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f804c0d0510 msgr2=0x7f804c154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f804b59e640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f804c0d0510 0x7f804c154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f804b59e640 1 -- 172.21.15.31:0/4208847563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f804c0ced30 msgr2=0x7f804c154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f804b59e640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f804c0ced30 0x7f804c154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.439 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f804b59e640 1 -- 172.21.15.31:0/4208847563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c009420 con 0x7f804c15d8f0 2023-11-26T07:22:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f804b59e640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 0x7f804c0ca500 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8044007e30 tx=0x7f804400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80437fe640 1 -- 172.21.15.31:0/4208847563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8044014620 con 0x7f804c15d8f0 2023-11-26T07:22:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80437fe640 1 -- 172.21.15.31:0/4208847563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804400ee90 con 0x7f804c15d8f0 2023-11-26T07:22:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80511df640 1 -- 172.21.15.31:0/4208847563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c1564b0 con 0x7f804c15d8f0 2023-11-26T07:22:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80437fe640 1 -- 172.21.15.31:0/4208847563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80440149c0 con 0x7f804c15d8f0 2023-11-26T07:22:45.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.433+0000 7f80511df640 1 -- 172.21.15.31:0/4208847563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c155020 con 0x7f804c15d8f0 2023-11-26T07:22:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.437+0000 7f80511df640 1 -- 172.21.15.31:0/4208847563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804c04e850 con 0x7f804c15d8f0 2023-11-26T07:22:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.437+0000 7f80437fe640 1 -- 172.21.15.31:0/4208847563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f8044014b20 con 0x7f804c15d8f0 2023-11-26T07:22:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.437+0000 7f80437fe640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f802c066590 0x7f802c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.437+0000 7f80437fe640 1 -- 172.21.15.31:0/4208847563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f80440848d0 con 0x7f804c15d8f0 2023-11-26T07:22:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.437+0000 7f804ad9d640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f802c066590 0x7f802c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.437+0000 7f804ad9d640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f802c066590 0x7f802c068a50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f803c0af000 tx=0x7f803c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.441+0000 7f80437fe640 1 -- 172.21.15.31:0/4208847563 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8044051c80 con 0x7f804c15d8f0 2023-11-26T07:22:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 -- 172.21.15.31:0/4208847563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f804c0cf9d0 con 0x7f804c15d8f0 2023-11-26T07:22:45.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80437fe640 1 -- 172.21.15.31:0/4208847563 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f80440552a0 con 0x7f804c15d8f0 2023-11-26T07:22:45.603 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:22:45.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 -- 172.21.15.31:0/4208847563 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f802c066590 msgr2=0x7f802c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f802c066590 0x7f802c068a50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f803c0af000 tx=0x7f803c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 -- 172.21.15.31:0/4208847563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 msgr2=0x7f804c0ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 0x7f804c0ca500 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8044007e30 tx=0x7f804400e4b0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 -- 172.21.15.31:0/4208847563 shutdown_connections 2023-11-26T07:22:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f802c066590 0x7f802c068a50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f804c15d8f0 0x7f804c0ca500 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f804c0d0510 0x7f804c154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 --2- 172.21.15.31:0/4208847563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f804c0ced30 0x7f804c154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 -- 172.21.15.31:0/4208847563 >> 172.21.15.31:0/4208847563 conn(0x7f804c09c750 msgr2=0x7f804c0b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 -- 172.21.15.31:0/4208847563 shutdown_connections 2023-11-26T07:22:45.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.601+0000 7f80511df640 1 -- 172.21.15.31:0/4208847563 wait complete. 2023-11-26T07:22:45.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae942aa640 1 Processor -- start 2023-11-26T07:22:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae942aa640 1 -- start start 2023-11-26T07:22:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae942aa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fae8c0b5830 0x7fae8c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae942aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0acd70 0x7fae8c0ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae942aa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae8c0ad710 0x7fae8c161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae942aa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8c0af630 con 0x7fae8c0ad710 2023-11-26T07:22:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae942aa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8c0acb10 con 0x7fae8c0b5830 2023-11-26T07:22:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae942aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c0adba0 con 0x7fae8c0acd70 2023-11-26T07:22:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae92847640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae8c0ad710 0x7fae8c161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae92847640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae8c0ad710 0x7fae8c161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41688/0 (socket says 172.21.15.31:41688) 2023-11-26T07:22:45.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae92847640 1 -- 172.21.15.31:0/2054846676 learned_addr learned my addr 172.21.15.31:0/2054846676 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae92046640 1 --2- 172.21.15.31:0/2054846676 >> v2:172.21.15.97:3300/0 conn(0x7fae8c0b5830 0x7fae8c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae91845640 1 --2- 172.21.15.31:0/2054846676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0acd70 0x7fae8c0ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae92847640 1 -- 172.21.15.31:0/2054846676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0acd70 msgr2=0x7fae8c0ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae92847640 1 --2- 172.21.15.31:0/2054846676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0acd70 0x7fae8c0ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae92847640 1 -- 172.21.15.31:0/2054846676 >> v2:172.21.15.97:3300/0 conn(0x7fae8c0b5830 msgr2=0x7fae8c15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae92847640 1 --2- 172.21.15.31:0/2054846676 >> v2:172.21.15.97:3300/0 conn(0x7fae8c0b5830 0x7fae8c15e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae92847640 1 -- 172.21.15.31:0/2054846676 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c0ae5b0 con 0x7fae8c0ad710 2023-11-26T07:22:45.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.769+0000 7fae91845640 1 --2- 172.21.15.31:0/2054846676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0acd70 0x7fae8c0ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:45.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae92847640 1 --2- 172.21.15.31:0/2054846676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 0x7fae8c161200 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fae8c070a60 tx=0x7fae8401c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ac1f384bbd04e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae91044640 1 -- 172.21.15.31:0/2054846676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840a5590 con 0x7fae8c0ad710 2023-11-26T07:22:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae91044640 1 -- 172.21.15.31:0/2054846676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae840a56f0 con 0x7fae8c0ad710 2023-11-26T07:22:45.776 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae91044640 1 -- 172.21.15.31:0/2054846676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840a5a80 con 0x7fae8c0ad710 2023-11-26T07:22:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae942aa640 1 -- 172.21.15.31:0/2054846676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 msgr2=0x7fae8c161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae942aa640 1 --2- 172.21.15.31:0/2054846676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 0x7fae8c161200 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fae8c070a60 tx=0x7fae8401c710 comp rx=0 tx=0).stop 2023-11-26T07:22:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae942aa640 1 -- 172.21.15.31:0/2054846676 shutdown_connections 2023-11-26T07:22:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae942aa640 1 --2- 172.21.15.31:0/2054846676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 0x7fae8c161200 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae942aa640 1 --2- 172.21.15.31:0/2054846676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0acd70 0x7fae8c0ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.777 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae942aa640 1 --2- 172.21.15.31:0/2054846676 >> v2:172.21.15.97:3300/0 conn(0x7fae8c0b5830 0x7fae8c15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae942aa640 1 -- 172.21.15.31:0/2054846676 >> 172.21.15.31:0/2054846676 conn(0x7fae8c071d80 msgr2=0x7fae8c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae942aa640 1 -- 172.21.15.31:0/2054846676 shutdown_connections 2023-11-26T07:22:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.773+0000 7fae942aa640 1 -- 172.21.15.31:0/2054846676 wait complete. 2023-11-26T07:22:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae942aa640 1 Processor -- start 2023-11-26T07:22:45.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae942aa640 1 -- start start 2023-11-26T07:22:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae942aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae8c0acd70 0x7fae8c153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae942aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 0x7fae8c151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae942aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0b5830 0x7fae8c1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae942aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c153070 con 0x7fae8c0ad710 2023-11-26T07:22:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae942aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c15e260 con 0x7fae8c0acd70 2023-11-26T07:22:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae942aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c1529f0 con 0x7fae8c0b5830 2023-11-26T07:22:45.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae91845640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 0x7fae8c151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae91845640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 0x7fae8c151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41696/0 (socket says 172.21.15.31:41696) 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae91845640 1 -- 172.21.15.31:0/4192155840 learned_addr learned my addr 172.21.15.31:0/4192155840 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae92046640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae8c0acd70 0x7fae8c153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae92847640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0b5830 0x7fae8c1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae91845640 1 -- 172.21.15.31:0/4192155840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0b5830 msgr2=0x7fae8c1518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae91845640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0b5830 0x7fae8c1518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae91845640 1 -- 172.21.15.31:0/4192155840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae8c0acd70 msgr2=0x7fae8c153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae91845640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae8c0acd70 0x7fae8c153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae91845640 1 -- 172.21.15.31:0/4192155840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae84009b30 con 0x7fae8c0ad710 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae91845640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 0x7fae8c151390 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00c7f0 tx=0x7fae7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae82ffd640 1 -- 172.21.15.31:0/4192155840 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c007820 con 0x7fae8c0ad710 2023-11-26T07:22:45.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae82ffd640 1 -- 172.21.15.31:0/4192155840 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7c007980 con 0x7fae8c0ad710 2023-11-26T07:22:45.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae942aa640 1 -- 172.21.15.31:0/4192155840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c15e470 con 0x7fae8c0ad710 2023-11-26T07:22:45.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae82ffd640 1 -- 172.21.15.31:0/4192155840 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7c007d30 con 0x7fae8c0ad710 2023-11-26T07:22:45.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.777+0000 7fae942aa640 1 -- 172.21.15.31:0/4192155840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c15e750 con 0x7fae8c0ad710 2023-11-26T07:22:45.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.781+0000 7fae942aa640 1 -- 172.21.15.31:0/4192155840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8c04e850 con 0x7fae8c0ad710 2023-11-26T07:22:45.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.781+0000 7fae82ffd640 1 -- 172.21.15.31:0/4192155840 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fae7c0113f0 con 0x7fae8c0ad710 2023-11-26T07:22:45.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.781+0000 7fae82ffd640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fae70066590 0x7fae70068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:45.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.781+0000 7fae82ffd640 1 -- 172.21.15.31:0/4192155840 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fae7c0843d0 con 0x7fae8c0ad710 2023-11-26T07:22:45.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.781+0000 7fae92046640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fae70066590 0x7fae70068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:45.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.781+0000 7fae92046640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fae70066590 0x7fae70068a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae8c159a10 tx=0x7fae88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:45.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.785+0000 7fae82ffd640 1 -- 172.21.15.31:0/4192155840 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fae7c0515e0 con 0x7fae8c0ad710 2023-11-26T07:22:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.941+0000 7fae942aa640 1 -- 172.21.15.31:0/4192155840 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae8c066450 con 0x7fae8c0ad710 2023-11-26T07:22:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.941+0000 7fae82ffd640 1 -- 172.21.15.31:0/4192155840 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae7c054c00 con 0x7fae8c0ad710 2023-11-26T07:22:45.944 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:22:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.941+0000 7fae942aa640 1 -- 172.21.15.31:0/4192155840 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fae70066590 msgr2=0x7fae70068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.941+0000 7fae942aa640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fae70066590 0x7fae70068a50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae8c159a10 tx=0x7fae88008040 comp rx=0 tx=0).stop 2023-11-26T07:22:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.945+0000 7fae942aa640 1 -- 172.21.15.31:0/4192155840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 msgr2=0x7fae8c151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.945+0000 7fae942aa640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 0x7fae8c151390 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00c7f0 tx=0x7fae7c00ccb0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.945+0000 7fae942aa640 1 -- 172.21.15.31:0/4192155840 shutdown_connections 2023-11-26T07:22:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.945+0000 7fae942aa640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fae70066590 0x7fae70068a50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.945+0000 7fae942aa640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fae8c0b5830 0x7fae8c1518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.945+0000 7fae942aa640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae8c0ad710 0x7fae8c151390 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.945+0000 7fae942aa640 1 --2- 172.21.15.31:0/4192155840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae8c0acd70 0x7fae8c153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.945+0000 7fae942aa640 1 -- 172.21.15.31:0/4192155840 >> 172.21.15.31:0/4192155840 conn(0x7fae8c071d80 msgr2=0x7fae8c06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.945+0000 7fae942aa640 1 -- 172.21.15.31:0/4192155840 shutdown_connections 2023-11-26T07:22:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:45.945+0000 7fae942aa640 1 -- 172.21.15.31:0/4192155840 wait complete. 2023-11-26T07:22:45.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.105+0000 7f5301216640 1 Processor -- start 2023-11-26T07:22:46.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.105+0000 7f5301216640 1 -- start start 2023-11-26T07:22:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f5301216640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f5301216640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f52fc0cbd50 0x7f52fc0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f5301216640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc0b2710 con 0x7f52fc0cdfb0 2023-11-26T07:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f5301216640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc0cca80 con 0x7f52fc0cbd50 2023-11-26T07:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f5301216640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc0b29f0 con 0x7f52fc15a710 2023-11-26T07:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52f3fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52fb59e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f52fc0cbd50 0x7f52fc0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52f3fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc0abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:35614/0 (socket says 172.21.15.31:35614) 2023-11-26T07:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52f3fff640 1 -- 172.21.15.31:0/2592203744 learned_addr learned my addr 172.21.15.31:0/2592203744 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52fb59e640 1 --2- 172.21.15.31:0/2592203744 >> v2:172.21.15.97:3300/0 conn(0x7f52fc0cbd50 0x7f52fc0cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:37334/0 (socket says 172.21.15.31:37334) 2023-11-26T07:22:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52f3fff640 1 -- 172.21.15.31:0/2592203744 >> v2:172.21.15.97:3300/0 conn(0x7f52fc0cbd50 msgr2=0x7f52fc0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:46.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52f3fff640 1 --2- 172.21.15.31:0/2592203744 >> v2:172.21.15.97:3300/0 conn(0x7f52fc0cbd50 0x7f52fc0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52f3fff640 1 -- 172.21.15.31:0/2592203744 >> v1:172.21.15.97:6789/0 conn(0x7f52fc0cdfb0 legacy=0x7f52fc0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52f3fff640 1 -- 172.21.15.31:0/2592203744 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc0b2cd0 con 0x7f52fc15a710 2023-11-26T07:22:46.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52fb59e640 1 --2- 172.21.15.31:0/2592203744 >> v2:172.21.15.97:3300/0 conn(0x7f52fc0cbd50 0x7f52fc0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:46.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.109+0000 7f52f3fff640 1 --2- 172.21.15.31:0/2592203744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc0abf90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52e400bb10 tx=0x7f52e401ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6ca849e586e440 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f52fa59c640 1 -- 172.21.15.31:0/2592203744 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e40a7cd0 con 0x7f52fc15a710 2023-11-26T07:22:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f52fa59c640 1 -- 172.21.15.31:0/2592203744 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e40a7e30 con 0x7f52fc15a710 2023-11-26T07:22:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f52fa59c640 1 -- 172.21.15.31:0/2592203744 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e40af660 con 0x7f52fc15a710 2023-11-26T07:22:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 -- 172.21.15.31:0/2592203744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 msgr2=0x7f52fc0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 --2- 172.21.15.31:0/2592203744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc0abf90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f52e400bb10 tx=0x7f52e401ebe0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 -- 172.21.15.31:0/2592203744 shutdown_connections 2023-11-26T07:22:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 --2- 172.21.15.31:0/2592203744 >> v2:172.21.15.97:3300/0 conn(0x7f52fc0cbd50 0x7f52fc0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 --2- 172.21.15.31:0/2592203744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc0abf90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 -- 172.21.15.31:0/2592203744 >> 172.21.15.31:0/2592203744 conn(0x7f52fc0b0cb0 msgr2=0x7f52fc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 -- 172.21.15.31:0/2592203744 shutdown_connections 2023-11-26T07:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 -- 172.21.15.31:0/2592203744 wait complete. 2023-11-26T07:22:46.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 Processor -- start 2023-11-26T07:22:46.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 -- start start 2023-11-26T07:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52fc0cbd50 0x7f52fc1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52fc0cdfb0 0x7f52fc158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc15cbc0 con 0x7f52fc0cbd50 2023-11-26T07:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc15cea0 con 0x7f52fc0cdfb0 2023-11-26T07:22:46.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.113+0000 7f5301216640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc15d180 con 0x7f52fc15a710 2023-11-26T07:22:46.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52fc0cbd50 0x7f52fc1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52fc0cbd50 0x7f52fc1530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41704/0 (socket says 172.21.15.31:41704) 2023-11-26T07:22:46.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fad9d640 1 -- 172.21.15.31:0/1340541349 learned_addr learned my addr 172.21.15.31:0/1340541349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fad9d640 1 -- 172.21.15.31:0/1340541349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 msgr2=0x7f52fc159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fb59e640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52f3fff640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52fc0cdfb0 0x7f52fc158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fad9d640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fad9d640 1 -- 172.21.15.31:0/1340541349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52fc0cdfb0 msgr2=0x7f52fc158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fad9d640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52fc0cdfb0 0x7f52fc158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fad9d640 1 -- 172.21.15.31:0/1340541349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4009a70 con 0x7f52fc0cbd50 2023-11-26T07:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fb59e640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52f3fff640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52fc0cdfb0 0x7f52fc158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52fad9d640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52fc0cbd50 0x7f52fc1530a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00c500 tx=0x7f52ec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52f37fe640 1 -- 172.21.15.31:0/1340541349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec005220 con 0x7f52fc0cbd50 2023-11-26T07:22:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52f37fe640 1 -- 172.21.15.31:0/1340541349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec005380 con 0x7f52fc0cbd50 2023-11-26T07:22:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f5301216640 1 -- 172.21.15.31:0/1340541349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e400b770 con 0x7f52fc0cbd50 2023-11-26T07:22:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52f37fe640 1 -- 172.21.15.31:0/1340541349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec007270 con 0x7f52fc0cbd50 2023-11-26T07:22:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f5301216640 1 -- 172.21.15.31:0/1340541349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc156520 con 0x7f52fc0cbd50 2023-11-26T07:22:46.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f5301216640 1 -- 172.21.15.31:0/1340541349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc0671f0 con 0x7f52fc0cbd50 2023-11-26T07:22:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.117+0000 7f52f37fe640 1 -- 172.21.15.31:0/1340541349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f52ec0073d0 con 0x7f52fc0cbd50 2023-11-26T07:22:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.121+0000 7f52f37fe640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f52dc066590 0x7f52dc068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.121+0000 7f52f37fe640 1 -- 172.21.15.31:0/1340541349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f52ec0839e0 con 0x7f52fc0cbd50 2023-11-26T07:22:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.121+0000 7f52f3fff640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f52dc066590 0x7f52dc068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.123 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.121+0000 7f52f3fff640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f52dc066590 0x7f52dc068a50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52e40b1000 tx=0x7f52e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.121+0000 7f52f37fe640 1 -- 172.21.15.31:0/1340541349 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f52ec050c60 con 0x7f52fc0cbd50 2023-11-26T07:22:46.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.297+0000 7f5301216640 1 -- 172.21.15.31:0/1340541349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f52fc06b460 con 0x7f52fc0cbd50 2023-11-26T07:22:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.297+0000 7f52f37fe640 1 -- 172.21.15.31:0/1340541349 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v88) v1 ==== 80+0+120 (secure 0 0 0) 0x7f52ec054280 con 0x7f52fc0cbd50 2023-11-26T07:22:46.301 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6835/683007803 2023-11-27T07:22:41.280769+0000 2023-11-26T07:22:46.301 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6833/683007803 2023-11-27T07:22:41.280769+0000 2023-11-26T07:22:46.301 INFO:teuthology.orchestra.run.smithi031.stderr:listed 2 entries 2023-11-26T07:22:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 -- 172.21.15.31:0/1340541349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f52dc066590 msgr2=0x7f52dc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f52dc066590 0x7f52dc068a50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52e40b1000 tx=0x7f52e4008040 comp rx=0 tx=0).stop 2023-11-26T07:22:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 -- 172.21.15.31:0/1340541349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52fc0cbd50 msgr2=0x7f52fc1530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52fc0cbd50 0x7f52fc1530a0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00c500 tx=0x7f52ec00c9c0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 -- 172.21.15.31:0/1340541349 shutdown_connections 2023-11-26T07:22:46.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f52dc066590 0x7f52dc068a50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52fc15a710 0x7f52fc159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52fc0cdfb0 0x7f52fc158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 --2- 172.21.15.31:0/1340541349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52fc0cbd50 0x7f52fc1530a0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 -- 172.21.15.31:0/1340541349 >> 172.21.15.31:0/1340541349 conn(0x7f52fc0b0cb0 msgr2=0x7f52fc0ac510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:46.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 -- 172.21.15.31:0/1340541349 shutdown_connections 2023-11-26T07:22:46.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.301+0000 7f5301216640 1 -- 172.21.15.31:0/1340541349 wait complete. 2023-11-26T07:22:46.323 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:22:46.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:46.323 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:22:46.351 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:22:46.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:46.351 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:22:46.407 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:46.407 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:46.417 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:46.417 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:46.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:46.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa88ebb640 1 Processor -- start 2023-11-26T07:22:46.616 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa88ebb640 1 -- start start 2023-11-26T07:22:46.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa88ebb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa8415d7b0 0x7faa840d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.617 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa88ebb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7faa840cead0 0x7faa840d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa88ebb640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa840cefd0 con 0x7faa840d1eb0 2023-11-26T07:22:46.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa88ebb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa840d2340 con 0x7faa840cead0 2023-11-26T07:22:46.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa88ebb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa840cf2b0 con 0x7faa8415d7b0 2023-11-26T07:22:46.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa82d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7faa840d1eb0 0x7faa840cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59538/0 (socket says 172.21.15.31:59538) 2023-11-26T07:22:46.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa82d9d640 1 -- 172.21.15.31:0/3331166570 learned_addr learned my addr 172.21.15.31:0/3331166570 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:46.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa8259c640 1 --2- 172.21.15.31:0/3331166570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa8415d7b0 0x7faa840d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa7bfff640 1 --2- 172.21.15.31:0/3331166570 >> v2:172.21.15.97:3300/0 conn(0x7faa840cead0 0x7faa840d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa81d9b640 1 -- 172.21.15.31:0/3331166570 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486621679 0 0) 0x7faa840cefd0 con 0x7faa840d1eb0 2023-11-26T07:22:46.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.613+0000 7faa81d9b640 1 -- 172.21.15.31:0/3331166570 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa70003530 con 0x7faa840d1eb0 2023-11-26T07:22:46.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa7bfff640 1 -- 172.21.15.31:0/3331166570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa8415d7b0 msgr2=0x7faa840d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:46.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa7bfff640 1 --2- 172.21.15.31:0/3331166570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa8415d7b0 0x7faa840d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa7bfff640 1 -- 172.21.15.31:0/3331166570 >> v1:172.21.15.31:6789/0 conn(0x7faa840d1eb0 legacy=0x7faa840cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa7bfff640 1 -- 172.21.15.31:0/3331166570 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8406b3f0 con 0x7faa840cead0 2023-11-26T07:22:46.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa8259c640 1 --2- 172.21.15.31:0/3331166570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa8415d7b0 0x7faa840d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:46.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa7bfff640 1 --2- 172.21.15.31:0/3331166570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa840cead0 0x7faa840d1970 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0097c0 tx=0x7faa6c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc8cfda826915ace server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa81d9b640 1 -- 172.21.15.31:0/3331166570 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0ab600 con 0x7faa840cead0 2023-11-26T07:22:46.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa81d9b640 1 -- 172.21.15.31:0/3331166570 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c0a7050 con 0x7faa840cead0 2023-11-26T07:22:46.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa81d9b640 1 -- 172.21.15.31:0/3331166570 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0ab960 con 0x7faa840cead0 2023-11-26T07:22:46.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa88ebb640 1 -- 172.21.15.31:0/3331166570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa840cead0 msgr2=0x7faa840d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.617+0000 7faa88ebb640 1 --2- 172.21.15.31:0/3331166570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa840cead0 0x7faa840d1970 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0097c0 tx=0x7faa6c01c330 comp rx=0 tx=0).stop 2023-11-26T07:22:46.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 -- 172.21.15.31:0/3331166570 shutdown_connections 2023-11-26T07:22:46.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 --2- 172.21.15.31:0/3331166570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa840cead0 0x7faa840d1970 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 --2- 172.21.15.31:0/3331166570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa8415d7b0 0x7faa840d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 -- 172.21.15.31:0/3331166570 >> 172.21.15.31:0/3331166570 conn(0x7faa840d33b0 msgr2=0x7faa840d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:46.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 -- 172.21.15.31:0/3331166570 shutdown_connections 2023-11-26T07:22:46.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 -- 172.21.15.31:0/3331166570 wait complete. 2023-11-26T07:22:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 Processor -- start 2023-11-26T07:22:46.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 -- start start 2023-11-26T07:22:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa840cead0 0x7faa841f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa840d1eb0 0x7faa84158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa8415d7b0 0x7faa8414b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa841fa180 con 0x7faa840d1eb0 2023-11-26T07:22:46.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa841517c0 con 0x7faa8415d7b0 2023-11-26T07:22:46.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa88ebb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8414efe0 con 0x7faa840cead0 2023-11-26T07:22:46.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa840d1eb0 0x7faa84158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa7bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa840d1eb0 0x7faa84158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41708/0 (socket says 172.21.15.31:41708) 2023-11-26T07:22:46.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa7bfff640 1 -- 172.21.15.31:0/211559709 learned_addr learned my addr 172.21.15.31:0/211559709 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:46.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa8259c640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa840cead0 0x7faa841f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa82d9d640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa8415d7b0 0x7faa8414b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa7bfff640 1 -- 172.21.15.31:0/211559709 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa840cead0 msgr2=0x7faa841f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa7bfff640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa840cead0 0x7faa841f9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa7bfff640 1 -- 172.21.15.31:0/211559709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa8415d7b0 msgr2=0x7faa8414b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa7bfff640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa8415d7b0 0x7faa8414b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa7bfff640 1 -- 172.21.15.31:0/211559709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa74009420 con 0x7faa840d1eb0 2023-11-26T07:22:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.621+0000 7faa7bfff640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa840d1eb0 0x7faa84158a10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0097c0 tx=0x7faa6c0190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.625+0000 7faa7affd640 1 -- 172.21.15.31:0/211559709 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c006e20 con 0x7faa840d1eb0 2023-11-26T07:22:46.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.625+0000 7faa7affd640 1 -- 172.21.15.31:0/211559709 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c004030 con 0x7faa840d1eb0 2023-11-26T07:22:46.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.625+0000 7faa7affd640 1 -- 172.21.15.31:0/211559709 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0043a0 con 0x7faa840d1eb0 2023-11-26T07:22:46.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.625+0000 7faa88ebb640 1 -- 172.21.15.31:0/211559709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6c009420 con 0x7faa840d1eb0 2023-11-26T07:22:46.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.625+0000 7faa88ebb640 1 -- 172.21.15.31:0/211559709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa841f9bc0 con 0x7faa840d1eb0 2023-11-26T07:22:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.625+0000 7faa88ebb640 1 -- 172.21.15.31:0/211559709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa84151520 con 0x7faa840d1eb0 2023-11-26T07:22:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.625+0000 7faa7affd640 1 -- 172.21.15.31:0/211559709 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7faa6c0b2040 con 0x7faa840d1eb0 2023-11-26T07:22:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.625+0000 7faa7affd640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faa640665e0 0x7faa64068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.625+0000 7faa7affd640 1 -- 172.21.15.31:0/211559709 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faa6c11e1e0 con 0x7faa840d1eb0 2023-11-26T07:22:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.625+0000 7faa8259c640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faa640665e0 0x7faa64068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.629+0000 7faa8259c640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faa640665e0 0x7faa64068aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faa840c6b00 tx=0x7faa74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.629+0000 7faa7affd640 1 -- 172.21.15.31:0/211559709 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faa84151520 con 0x7faa840d1eb0 2023-11-26T07:22:46.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.789+0000 7faa88ebb640 1 -- 172.21.15.31:0/211559709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa84151900 con 0x7faa840d1eb0 2023-11-26T07:22:46.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa7affd640 1 -- 172.21.15.31:0/211559709 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16603 (secure 0 0 0) 0x7faa84151900 con 0x7faa840d1eb0 2023-11-26T07:22:46.794 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:46.794 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:44.288769+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:20.941587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:21.520554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/683007803":"2023-11-27T07:22:41.280769+0000","172.21.15.97:6833/683007803":"2023-11-27T07:22:41.280769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:46.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 -- 172.21.15.31:0/211559709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faa640665e0 msgr2=0x7faa64068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faa640665e0 0x7faa64068aa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7faa840c6b00 tx=0x7faa74008040 comp rx=0 tx=0).stop 2023-11-26T07:22:46.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 -- 172.21.15.31:0/211559709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa840d1eb0 msgr2=0x7faa84158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa840d1eb0 0x7faa84158a10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0097c0 tx=0x7faa6c0190d0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 -- 172.21.15.31:0/211559709 shutdown_connections 2023-11-26T07:22:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7faa640665e0 0x7faa64068aa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa8415d7b0 0x7faa8414b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa840d1eb0 0x7faa84158a10 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 --2- 172.21.15.31:0/211559709 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7faa840cead0 0x7faa841f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 -- 172.21.15.31:0/211559709 >> 172.21.15.31:0/211559709 conn(0x7faa840d33b0 msgr2=0x7faa84159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 -- 172.21.15.31:0/211559709 shutdown_connections 2023-11-26T07:22:46.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.793+0000 7faa88ebb640 1 -- 172.21.15.31:0/211559709 wait complete. 2023-11-26T07:22:46.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:22:20.941587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:22:21.520554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:46.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:46.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.957+0000 7f4d4d365640 1 Processor -- start 2023-11-26T07:22:46.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.957+0000 7f4d4d365640 1 -- start start 2023-11-26T07:22:46.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d4d365640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d480b2870 0x7f4d480b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d4d365640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f4d4806afc0 0x7f4d480b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d4d365640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d48161290 con 0x7f4d480b07a0 2023-11-26T07:22:46.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d4d365640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d480b0b80 con 0x7f4d480b2870 2023-11-26T07:22:46.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d4d365640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4815ebc0 con 0x7f4d4806afc0 2023-11-26T07:22:46.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d3ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d480b2870 0x7f4d480b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d46ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4d480b07a0 0x7f4d481619e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59542/0 (socket says 172.21.15.31:59542) 2023-11-26T07:22:46.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d46ffd640 1 -- 172.21.15.31:0/3459895305 learned_addr learned my addr 172.21.15.31:0/3459895305 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:46.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d3ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d480b2870 0x7f4d480b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41718/0 (socket says 172.21.15.31:41718) 2023-11-26T07:22:46.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d477fe640 1 --2- 172.21.15.31:0/3459895305 >> v2:172.21.15.97:3300/0 conn(0x7f4d4806afc0 0x7f4d480b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d3ffff640 1 -- 172.21.15.31:0/3459895305 >> v2:172.21.15.97:3300/0 conn(0x7f4d4806afc0 msgr2=0x7f4d480b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:46.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d3ffff640 1 --2- 172.21.15.31:0/3459895305 >> v2:172.21.15.97:3300/0 conn(0x7f4d4806afc0 0x7f4d480b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d3ffff640 1 -- 172.21.15.31:0/3459895305 >> v1:172.21.15.31:6789/0 conn(0x7f4d480b07a0 legacy=0x7f4d481619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d3ffff640 1 -- 172.21.15.31:0/3459895305 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d48161550 con 0x7f4d480b2870 2023-11-26T07:22:46.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.961+0000 7f4d467fc640 1 -- 172.21.15.31:0/3459895305 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234067190 0 0) 0x7f4d48161290 con 0x7f4d480b07a0 2023-11-26T07:22:46.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d3ffff640 1 --2- 172.21.15.31:0/3459895305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b2870 0x7f4d480b0ee0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4d48071bf0 tx=0x7f4d3001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438285e4ce3fed65 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d467fc640 1 -- 172.21.15.31:0/3459895305 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d300ab600 con 0x7f4d480b2870 2023-11-26T07:22:46.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d467fc640 1 -- 172.21.15.31:0/3459895305 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d300a7050 con 0x7f4d480b2870 2023-11-26T07:22:46.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d467fc640 1 -- 172.21.15.31:0/3459895305 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d300ab980 con 0x7f4d480b2870 2023-11-26T07:22:46.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d4d365640 1 -- 172.21.15.31:0/3459895305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b2870 msgr2=0x7f4d480b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d4d365640 1 --2- 172.21.15.31:0/3459895305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b2870 0x7f4d480b0ee0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4d48071bf0 tx=0x7f4d3001c440 comp rx=0 tx=0).stop 2023-11-26T07:22:46.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d4d365640 1 -- 172.21.15.31:0/3459895305 shutdown_connections 2023-11-26T07:22:46.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d4d365640 1 --2- 172.21.15.31:0/3459895305 >> v2:172.21.15.97:3300/0 conn(0x7f4d4806afc0 0x7f4d480b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d4d365640 1 --2- 172.21.15.31:0/3459895305 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b2870 0x7f4d480b0ee0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d4d365640 1 -- 172.21.15.31:0/3459895305 >> 172.21.15.31:0/3459895305 conn(0x7f4d480b1d50 msgr2=0x7f4d480b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:46.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d4d365640 1 -- 172.21.15.31:0/3459895305 shutdown_connections 2023-11-26T07:22:46.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d4d365640 1 -- 172.21.15.31:0/3459895305 wait complete. 2023-11-26T07:22:46.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d4d365640 1 Processor -- start 2023-11-26T07:22:46.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.965+0000 7f4d4d365640 1 -- start start 2023-11-26T07:22:46.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d4d365640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d4806afc0 0x7f4d48159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d4d365640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b07a0 0x7f4d480ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d4d365640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d480b2870 0x7f4d480ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d4d365640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480aeb10 con 0x7f4d480b07a0 2023-11-26T07:22:46.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d4d365640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480adce0 con 0x7f4d4806afc0 2023-11-26T07:22:46.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d4d365640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480adf90 con 0x7f4d480b2870 2023-11-26T07:22:46.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b07a0 0x7f4d480ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b07a0 0x7f4d480ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41726/0 (socket says 172.21.15.31:41726) 2023-11-26T07:22:46.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3ffff640 1 -- 172.21.15.31:0/3448692540 learned_addr learned my addr 172.21.15.31:0/3448692540 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:46.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d46ffd640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d4806afc0 0x7f4d48159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d477fe640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d480b2870 0x7f4d480ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3ffff640 1 -- 172.21.15.31:0/3448692540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d480b2870 msgr2=0x7f4d480ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3ffff640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d480b2870 0x7f4d480ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3ffff640 1 -- 172.21.15.31:0/3448692540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d4806afc0 msgr2=0x7f4d48159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:46.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3ffff640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d4806afc0 0x7f4d48159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:46.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3ffff640 1 -- 172.21.15.31:0/3448692540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d30009420 con 0x7f4d480b07a0 2023-11-26T07:22:46.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3ffff640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b07a0 0x7f4d480ad260 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4d30019050 tx=0x7f4d300191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3f7fe640 1 -- 172.21.15.31:0/3448692540 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d300b5470 con 0x7f4d480b07a0 2023-11-26T07:22:46.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3f7fe640 1 -- 172.21.15.31:0/3448692540 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d300b55f0 con 0x7f4d480b07a0 2023-11-26T07:22:46.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d4d365640 1 -- 172.21.15.31:0/3448692540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d480ae270 con 0x7f4d480b07a0 2023-11-26T07:22:46.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3f7fe640 1 -- 172.21.15.31:0/3448692540 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d300b59a0 con 0x7f4d480b07a0 2023-11-26T07:22:46.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d4d365640 1 -- 172.21.15.31:0/3448692540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d480ae830 con 0x7f4d480b07a0 2023-11-26T07:22:46.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d4d365640 1 -- 172.21.15.31:0/3448692540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4804e850 con 0x7f4d480b07a0 2023-11-26T07:22:46.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.969+0000 7f4d3f7fe640 1 -- 172.21.15.31:0/3448692540 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f4d300b2040 con 0x7f4d480b07a0 2023-11-26T07:22:46.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.973+0000 7f4d3f7fe640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4d28066590 0x7f4d28068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:46.975 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.973+0000 7f4d3f7fe640 1 -- 172.21.15.31:0/3448692540 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4d3011e570 con 0x7f4d480b07a0 2023-11-26T07:22:46.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.973+0000 7f4d46ffd640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4d28066590 0x7f4d28068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:46.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.973+0000 7f4d46ffd640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4d28066590 0x7f4d28068a50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4d38000f70 tx=0x7f4d38007b80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:46.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:46.977+0000 7f4d3f7fe640 1 -- 172.21.15.31:0/3448692540 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4d300eb760 con 0x7f4d480b07a0 2023-11-26T07:22:47.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 -- 172.21.15.31:0/3448692540 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d4806c6d0 con 0x7f4d480b07a0 2023-11-26T07:22:47.138 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d3f7fe640 1 -- 172.21.15.31:0/3448692540 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d300b00a0 con 0x7f4d480b07a0 2023-11-26T07:22:47.139 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:22:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 -- 172.21.15.31:0/3448692540 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4d28066590 msgr2=0x7f4d28068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4d28066590 0x7f4d28068a50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4d38000f70 tx=0x7f4d38007b80 comp rx=0 tx=0).stop 2023-11-26T07:22:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 -- 172.21.15.31:0/3448692540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b07a0 msgr2=0x7f4d480ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b07a0 0x7f4d480ad260 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4d30019050 tx=0x7f4d300191e0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 -- 172.21.15.31:0/3448692540 shutdown_connections 2023-11-26T07:22:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4d28066590 0x7f4d28068a50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4d480b2870 0x7f4d480ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d480b07a0 0x7f4d480ad260 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 --2- 172.21.15.31:0/3448692540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d4806afc0 0x7f4d48159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 -- 172.21.15.31:0/3448692540 >> 172.21.15.31:0/3448692540 conn(0x7f4d480b1d50 msgr2=0x7f4d480b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 -- 172.21.15.31:0/3448692540 shutdown_connections 2023-11-26T07:22:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.137+0000 7f4d4d365640 1 -- 172.21.15.31:0/3448692540 wait complete. 2023-11-26T07:22:47.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:47.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f305af57640 1 Processor -- start 2023-11-26T07:22:47.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f305af57640 1 -- start start 2023-11-26T07:22:47.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f305af57640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30540d20d0 0x7f3054168720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f305af57640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30540cec90 con 0x7f30540d07c0 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f305af57640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30540d16e0 con 0x7f305415d8a0 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f305af57640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30540ce360 con 0x7f30540d20d0 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f3053fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f30540d07c0 0x7f30540d19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59558/0 (socket says 172.21.15.31:59558) 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f3053fff640 1 -- 172.21.15.31:0/3341321103 learned_addr learned my addr 172.21.15.31:0/3341321103 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f30594f4640 1 --2- 172.21.15.31:0/3341321103 >> v2:172.21.15.31:3300/0 conn(0x7f30540d20d0 0x7f3054168720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f30537fe640 1 -- 172.21.15.31:0/3341321103 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478218560 0 0) 0x7f30540cec90 con 0x7f30540d07c0 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f30537fe640 1 -- 172.21.15.31:0/3341321103 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3044003530 con 0x7f30540d07c0 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f30594f4640 1 -- 172.21.15.31:0/3341321103 >> v1:172.21.15.97:6789/0 conn(0x7f305415d8a0 legacy=0x7f30540d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f30537fe640 1 -- 172.21.15.31:0/3341321103 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17911151 0 0) 0x7f30540d16e0 con 0x7f305415d8a0 2023-11-26T07:22:47.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f30594f4640 1 -- 172.21.15.31:0/3341321103 >> v1:172.21.15.31:6789/0 conn(0x7f30540d07c0 legacy=0x7f30540d19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.309+0000 7f30594f4640 1 -- 172.21.15.31:0/3341321103 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30540ce640 con 0x7f30540d20d0 2023-11-26T07:22:47.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f30594f4640 1 --2- 172.21.15.31:0/3341321103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d20d0 0x7f3054168720 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f304c009d90 tx=0x7f304c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820051923dd79b18 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f30537fe640 1 -- 172.21.15.31:0/3341321103 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c0a59a0 con 0x7f30540d20d0 2023-11-26T07:22:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f30537fe640 1 -- 172.21.15.31:0/3341321103 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304c0a5b00 con 0x7f30540d20d0 2023-11-26T07:22:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f30537fe640 1 -- 172.21.15.31:0/3341321103 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c0a5e90 con 0x7f30540d20d0 2023-11-26T07:22:47.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f305af57640 1 -- 172.21.15.31:0/3341321103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d20d0 msgr2=0x7f3054168720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f305af57640 1 --2- 172.21.15.31:0/3341321103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d20d0 0x7f3054168720 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f304c009d90 tx=0x7f304c01c920 comp rx=0 tx=0).stop 2023-11-26T07:22:47.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f305af57640 1 -- 172.21.15.31:0/3341321103 shutdown_connections 2023-11-26T07:22:47.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f305af57640 1 --2- 172.21.15.31:0/3341321103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d20d0 0x7f3054168720 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f305af57640 1 -- 172.21.15.31:0/3341321103 >> 172.21.15.31:0/3341321103 conn(0x7f305409c6f0 msgr2=0x7f30540d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:47.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f305af57640 1 -- 172.21.15.31:0/3341321103 shutdown_connections 2023-11-26T07:22:47.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.313+0000 7f305af57640 1 -- 172.21.15.31:0/3341321103 wait complete. 2023-11-26T07:22:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f305af57640 1 Processor -- start 2023-11-26T07:22:47.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f305af57640 1 -- start start 2023-11-26T07:22:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f305af57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d07c0 0x7f30540cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f305af57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30540d20d0 0x7f30541573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f305af57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305415d8a0 0x7f30541578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f305af57640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30540cc470 con 0x7f30540d07c0 2023-11-26T07:22:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f305af57640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30540cd490 con 0x7f305415d8a0 2023-11-26T07:22:47.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f305af57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3054159f60 con 0x7f30540d20d0 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f3058cf3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d07c0 0x7f30540cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f3058cf3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d07c0 0x7f30540cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41744/0 (socket says 172.21.15.31:41744) 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f3058cf3640 1 -- 172.21.15.31:0/1337713667 learned_addr learned my addr 172.21.15.31:0/1337713667 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f30594f4640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305415d8a0 0x7f30541578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f3053fff640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30540d20d0 0x7f30541573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f3058cf3640 1 -- 172.21.15.31:0/1337713667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30540d20d0 msgr2=0x7f30541573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f3058cf3640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30540d20d0 0x7f30541573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f3058cf3640 1 -- 172.21.15.31:0/1337713667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305415d8a0 msgr2=0x7f30541578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f3058cf3640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305415d8a0 0x7f30541578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f3058cf3640 1 -- 172.21.15.31:0/1337713667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c009a70 con 0x7f30540d07c0 2023-11-26T07:22:47.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f3058cf3640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d07c0 0x7f30540cbd70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f304800c4d0 tx=0x7f304800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:47.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f30517fa640 1 -- 172.21.15.31:0/1337713667 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048005480 con 0x7f30540d07c0 2023-11-26T07:22:47.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f30517fa640 1 -- 172.21.15.31:0/1337713667 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048003600 con 0x7f30540d07c0 2023-11-26T07:22:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f30517fa640 1 -- 172.21.15.31:0/1337713667 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30480039b0 con 0x7f30540d07c0 2023-11-26T07:22:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f305af57640 1 -- 172.21.15.31:0/1337713667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305415a200 con 0x7f30540d07c0 2023-11-26T07:22:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.317+0000 7f305af57640 1 -- 172.21.15.31:0/1337713667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305415a7c0 con 0x7f30540d07c0 2023-11-26T07:22:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.321+0000 7f305af57640 1 -- 172.21.15.31:0/1337713667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3054067a40 con 0x7f30540d07c0 2023-11-26T07:22:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.321+0000 7f30517fa640 1 -- 172.21.15.31:0/1337713667 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f3048003b10 con 0x7f30540d07c0 2023-11-26T07:22:47.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.321+0000 7f30517fa640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3038066590 0x7f3038068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.321+0000 7f30517fa640 1 -- 172.21.15.31:0/1337713667 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f30480835a0 con 0x7f30540d07c0 2023-11-26T07:22:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.321+0000 7f3053fff640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3038066590 0x7f3038068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.321+0000 7f3053fff640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3038066590 0x7f3038068a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30540b1410 tx=0x7f3040009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:47.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.325+0000 7f30517fa640 1 -- 172.21.15.31:0/1337713667 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3048050a10 con 0x7f30540d07c0 2023-11-26T07:22:47.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 -- 172.21.15.31:0/1337713667 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f305404e850 con 0x7f30540d07c0 2023-11-26T07:22:47.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f30517fa640 1 -- 172.21.15.31:0/1337713667 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3048017020 con 0x7f30540d07c0 2023-11-26T07:22:47.487 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:22:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 -- 172.21.15.31:0/1337713667 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3038066590 msgr2=0x7f3038068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3038066590 0x7f3038068a50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30540b1410 tx=0x7f3040009040 comp rx=0 tx=0).stop 2023-11-26T07:22:47.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 -- 172.21.15.31:0/1337713667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d07c0 msgr2=0x7f30540cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d07c0 0x7f30540cbd70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f304800c4d0 tx=0x7f304800c990 comp rx=0 tx=0).stop 2023-11-26T07:22:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 -- 172.21.15.31:0/1337713667 shutdown_connections 2023-11-26T07:22:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3038066590 0x7f3038068a50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f305415d8a0 0x7f30541578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30540d20d0 0x7f30541573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 --2- 172.21.15.31:0/1337713667 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30540d07c0 0x7f30540cbd70 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 -- 172.21.15.31:0/1337713667 >> 172.21.15.31:0/1337713667 conn(0x7f305409c6f0 msgr2=0x7f3054158690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 -- 172.21.15.31:0/1337713667 shutdown_connections 2023-11-26T07:22:47.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.485+0000 7f305af57640 1 -- 172.21.15.31:0/1337713667 wait complete. 2023-11-26T07:22:47.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.649+0000 7f9db12fe640 1 Processor -- start 2023-11-26T07:22:47.653 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.649+0000 7f9db12fe640 1 -- start start 2023-11-26T07:22:47.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.653+0000 7f9db12fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9dac0ced30 0x7f9dac0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.653+0000 7f9db12fe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dac0cdfb0 con 0x7f9dac15d8f0 2023-11-26T07:22:47.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.653+0000 7f9db12fe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dac0d4140 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.653+0000 7f9db12fe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dac0d4420 con 0x7f9dac0ced30 2023-11-26T07:22:47.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.653+0000 7f9daaffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9dac15d8f0 0x7f9dac15e780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59564/0 (socket says 172.21.15.31:59564) 2023-11-26T07:22:47.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.653+0000 7f9daaffd640 1 -- 172.21.15.31:0/1187464203 learned_addr learned my addr 172.21.15.31:0/1187464203 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:47.655 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.653+0000 7f9da27fc640 1 --2- 172.21.15.31:0/1187464203 >> v2:172.21.15.31:3300/0 conn(0x7f9dac0ced30 0x7f9dac0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.653+0000 7f9da27fc640 1 -- 172.21.15.31:0/1187464203 >> v1:172.21.15.97:6789/0 conn(0x7f9dac0cf1e0 legacy=0x7f9dac0d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:47.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.653+0000 7f9da27fc640 1 -- 172.21.15.31:0/1187464203 >> v1:172.21.15.31:6789/0 conn(0x7f9dac15d8f0 legacy=0x7f9dac15e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.656 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.653+0000 7f9da27fc640 1 -- 172.21.15.31:0/1187464203 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac0ce250 con 0x7f9dac0ced30 2023-11-26T07:22:47.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9da27fc640 1 --2- 172.21.15.31:0/1187464203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0ced30 0x7f9dac0cfe00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9d940097c0 tx=0x7f9d9401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97692feb17ef81c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:47.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9daa7fc640 1 -- 172.21.15.31:0/1187464203 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d940ab610 con 0x7f9dac0ced30 2023-11-26T07:22:47.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9daa7fc640 1 -- 172.21.15.31:0/1187464203 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d940ab770 con 0x7f9dac0ced30 2023-11-26T07:22:47.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9daa7fc640 1 -- 172.21.15.31:0/1187464203 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d940abb00 con 0x7f9dac0ced30 2023-11-26T07:22:47.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9db12fe640 1 -- 172.21.15.31:0/1187464203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0ced30 msgr2=0x7f9dac0cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9db12fe640 1 --2- 172.21.15.31:0/1187464203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0ced30 0x7f9dac0cfe00 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9d940097c0 tx=0x7f9d9401c410 comp rx=0 tx=0).stop 2023-11-26T07:22:47.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9db12fe640 1 -- 172.21.15.31:0/1187464203 shutdown_connections 2023-11-26T07:22:47.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9db12fe640 1 --2- 172.21.15.31:0/1187464203 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0ced30 0x7f9dac0cfe00 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9db12fe640 1 -- 172.21.15.31:0/1187464203 >> 172.21.15.31:0/1187464203 conn(0x7f9dac09c750 msgr2=0x7f9dac0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:47.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9db12fe640 1 -- 172.21.15.31:0/1187464203 shutdown_connections 2023-11-26T07:22:47.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9db12fe640 1 -- 172.21.15.31:0/1187464203 wait complete. 2023-11-26T07:22:47.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9db12fe640 1 Processor -- start 2023-11-26T07:22:47.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.657+0000 7f9db12fe640 1 -- start start 2023-11-26T07:22:47.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9db12fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dac0ced30 0x7f9dac0cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9db12fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0cf1e0 0x7f9dac0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9db12fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dac15d8f0 0x7f9dac0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9db12fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac1591a0 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9db12fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac0caa70 con 0x7f9dac0ced30 2023-11-26T07:22:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9db12fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac0cad50 con 0x7f9dac15d8f0 2023-11-26T07:22:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0cf1e0 0x7f9dac0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da27fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0cf1e0 0x7f9dac0c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41764/0 (socket says 172.21.15.31:41764) 2023-11-26T07:22:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da27fc640 1 -- 172.21.15.31:0/363464637 learned_addr learned my addr 172.21.15.31:0/363464637 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9daaffd640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dac0ced30 0x7f9dac0cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9dab7fe640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dac15d8f0 0x7f9dac0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da27fc640 1 -- 172.21.15.31:0/363464637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dac15d8f0 msgr2=0x7f9dac0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da27fc640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dac15d8f0 0x7f9dac0ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da27fc640 1 -- 172.21.15.31:0/363464637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dac0ced30 msgr2=0x7f9dac0cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da27fc640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dac0ced30 0x7f9dac0cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da27fc640 1 -- 172.21.15.31:0/363464637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94009420 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9dab7fe640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dac15d8f0 0x7f9dac0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da27fc640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0cf1e0 0x7f9dac0c9030 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9d940ad040 tx=0x7f9d9401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:47.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da3fff640 1 -- 172.21.15.31:0/363464637 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d940b5810 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da3fff640 1 -- 172.21.15.31:0/363464637 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d940b5990 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da3fff640 1 -- 172.21.15.31:0/363464637 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d940b5d40 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9db12fe640 1 -- 172.21.15.31:0/363464637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dac0cb030 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9db12fe640 1 -- 172.21.15.31:0/363464637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dac0cb5f0 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9db12fe640 1 -- 172.21.15.31:0/363464637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dac0cb7e0 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.661+0000 7f9da3fff640 1 -- 172.21.15.31:0/363464637 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f9d940bf410 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.665+0000 7f9da3fff640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9d8c066590 0x7f9d8c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.665+0000 7f9da3fff640 1 -- 172.21.15.31:0/363464637 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9d9411eb20 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.665+0000 7f9daaffd640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9d8c066590 0x7f9d8c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.665+0000 7f9daaffd640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9d8c066590 0x7f9d8c068a50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c005890 tx=0x7f9d9c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:47.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.669+0000 7f9da3fff640 1 -- 172.21.15.31:0/363464637 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9d940ebdd0 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.825+0000 7f9db12fe640 1 -- 172.21.15.31:0/363464637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9dac0cae90 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.825+0000 7f9da3fff640 1 -- 172.21.15.31:0/363464637 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9dac0cae90 con 0x7f9dac0cf1e0 2023-11-26T07:22:47.829 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:22:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 -- 172.21.15.31:0/363464637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9d8c066590 msgr2=0x7f9d8c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9d8c066590 0x7f9d8c068a50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c005890 tx=0x7f9d9c009040 comp rx=0 tx=0).stop 2023-11-26T07:22:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 -- 172.21.15.31:0/363464637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0cf1e0 msgr2=0x7f9dac0c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0cf1e0 0x7f9dac0c9030 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9d940ad040 tx=0x7f9d9401ac50 comp rx=0 tx=0).stop 2023-11-26T07:22:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 -- 172.21.15.31:0/363464637 shutdown_connections 2023-11-26T07:22:47.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9d8c066590 0x7f9d8c068a50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dac15d8f0 0x7f9dac0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9dac0cf1e0 0x7f9dac0c9030 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 --2- 172.21.15.31:0/363464637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dac0ced30 0x7f9dac0cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 -- 172.21.15.31:0/363464637 >> 172.21.15.31:0/363464637 conn(0x7f9dac09c750 msgr2=0x7f9dac0b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:47.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 -- 172.21.15.31:0/363464637 shutdown_connections 2023-11-26T07:22:47.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.829+0000 7f9db12fe640 1 -- 172.21.15.31:0/363464637 wait complete. 2023-11-26T07:22:47.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:22:47.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7baf6e4640 1 Processor -- start 2023-11-26T07:22:47.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7baf6e4640 1 -- start start 2023-11-26T07:22:47.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7baf6e4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba815a710 0x7f7ba80abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7baf6e4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ba80cbd50 0x7f7ba80cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:47.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7baf6e4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba80b2710 con 0x7f7ba80cdfb0 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7baf6e4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba80cca80 con 0x7f7ba80cbd50 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7baf6e4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba80b29f0 con 0x7f7ba815a710 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7bad480640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7ba80cdfb0 0x7f7ba80ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59572/0 (socket says 172.21.15.31:59572) 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7bad480640 1 -- 172.21.15.31:0/818437833 learned_addr learned my addr 172.21.15.31:0/818437833 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7bacc7f640 1 --2- 172.21.15.31:0/818437833 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba815a710 0x7f7ba80abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7badc81640 1 --2- 172.21.15.31:0/818437833 >> v2:172.21.15.97:3300/0 conn(0x7f7ba80cbd50 0x7f7ba80cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7b9ffff640 1 -- 172.21.15.31:0/818437833 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196249863 0 0) 0x7f7ba80b2710 con 0x7f7ba80cdfb0 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7b9ffff640 1 -- 172.21.15.31:0/818437833 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b94003530 con 0x7f7ba80cdfb0 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7badc81640 1 -- 172.21.15.31:0/818437833 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba815a710 msgr2=0x7f7ba80abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7badc81640 1 --2- 172.21.15.31:0/818437833 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba815a710 0x7f7ba80abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:47.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7badc81640 1 -- 172.21.15.31:0/818437833 >> v1:172.21.15.31:6789/0 conn(0x7f7ba80cdfb0 legacy=0x7f7ba80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7badc81640 1 -- 172.21.15.31:0/818437833 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba80b2cd0 con 0x7f7ba80cbd50 2023-11-26T07:22:47.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.993+0000 7f7bacc7f640 1 --2- 172.21.15.31:0/818437833 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba815a710 0x7f7ba80abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7badc81640 1 --2- 172.21.15.31:0/818437833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba80cbd50 0x7f7ba80cc1e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009d90 tx=0x7f7ba001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a68261c8d9bc9571 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:47.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7b9ffff640 1 -- 172.21.15.31:0/818437833 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00a5a00 con 0x7f7ba80cbd50 2023-11-26T07:22:48.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7b9ffff640 1 -- 172.21.15.31:0/818437833 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba00a5b60 con 0x7f7ba80cbd50 2023-11-26T07:22:48.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7b9ffff640 1 -- 172.21.15.31:0/818437833 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00ad440 con 0x7f7ba80cbd50 2023-11-26T07:22:48.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7baf6e4640 1 -- 172.21.15.31:0/818437833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba80cbd50 msgr2=0x7f7ba80cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7baf6e4640 1 --2- 172.21.15.31:0/818437833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba80cbd50 0x7f7ba80cc1e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009d90 tx=0x7f7ba001c920 comp rx=0 tx=0).stop 2023-11-26T07:22:48.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7baf6e4640 1 -- 172.21.15.31:0/818437833 shutdown_connections 2023-11-26T07:22:48.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7baf6e4640 1 --2- 172.21.15.31:0/818437833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba80cbd50 0x7f7ba80cc1e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7baf6e4640 1 --2- 172.21.15.31:0/818437833 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba815a710 0x7f7ba80abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7baf6e4640 1 -- 172.21.15.31:0/818437833 >> 172.21.15.31:0/818437833 conn(0x7f7ba80b0cb0 msgr2=0x7f7ba80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7baf6e4640 1 -- 172.21.15.31:0/818437833 shutdown_connections 2023-11-26T07:22:48.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:47.997+0000 7f7baf6e4640 1 -- 172.21.15.31:0/818437833 wait complete. 2023-11-26T07:22:48.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 Processor -- start 2023-11-26T07:22:48.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 -- start start 2023-11-26T07:22:48.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba80cbd50 0x7f7ba815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba80cdfb0 0x7f7ba81635a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba815a710 0x7f7ba8163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba81444c0 con 0x7f7ba815a710 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba8144710 con 0x7f7ba80cdfb0 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba81449f0 con 0x7f7ba80cbd50 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7badc81640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba815a710 0x7f7ba8163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7badc81640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba815a710 0x7f7ba8163ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41768/0 (socket says 172.21.15.31:41768) 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7badc81640 1 -- 172.21.15.31:0/1599799035 learned_addr learned my addr 172.21.15.31:0/1599799035 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7bad480640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba80cbd50 0x7f7ba815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7bacc7f640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba80cdfb0 0x7f7ba81635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7badc81640 1 -- 172.21.15.31:0/1599799035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba80cbd50 msgr2=0x7f7ba815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7badc81640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba80cbd50 0x7f7ba815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7badc81640 1 -- 172.21.15.31:0/1599799035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba80cdfb0 msgr2=0x7f7ba81635a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7badc81640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba80cdfb0 0x7f7ba81635a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7badc81640 1 -- 172.21.15.31:0/1599799035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b98009420 con 0x7f7ba815a710 2023-11-26T07:22:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7bacc7f640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba80cdfb0 0x7f7ba81635a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7bad480640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba80cbd50 0x7f7ba815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7badc81640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba815a710 0x7f7ba8163ae0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7ba001a550 tx=0x7f7ba00196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7b9dffb640 1 -- 172.21.15.31:0/1599799035 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00aed30 con 0x7f7ba815a710 2023-11-26T07:22:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7b9dffb640 1 -- 172.21.15.31:0/1599799035 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba00aee90 con 0x7f7ba815a710 2023-11-26T07:22:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7b9dffb640 1 -- 172.21.15.31:0/1599799035 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00b7610 con 0x7f7ba815a710 2023-11-26T07:22:48.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 -- 172.21.15.31:0/1599799035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba0009a70 con 0x7f7ba815a710 2023-11-26T07:22:48.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 -- 172.21.15.31:0/1599799035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba8144fb0 con 0x7f7ba815a710 2023-11-26T07:22:48.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.001+0000 7f7baf6e4640 1 -- 172.21.15.31:0/1599799035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba80671f0 con 0x7f7ba815a710 2023-11-26T07:22:48.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.005+0000 7f7b9dffb640 1 -- 172.21.15.31:0/1599799035 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f7ba00b4050 con 0x7f7ba815a710 2023-11-26T07:22:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.005+0000 7f7b9dffb640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7b8c066590 0x7f7b8c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.005+0000 7f7b9dffb640 1 -- 172.21.15.31:0/1599799035 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7ba011f350 con 0x7f7ba815a710 2023-11-26T07:22:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.005+0000 7f7bad480640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7b8c066590 0x7f7b8c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.005+0000 7f7bad480640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7b8c066590 0x7f7b8c068a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4005890 tx=0x7f7ba4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.009+0000 7f7b9dffb640 1 -- 172.21.15.31:0/1599799035 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7ba00ec820 con 0x7f7ba815a710 2023-11-26T07:22:48.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.181+0000 7f7baf6e4640 1 -- 172.21.15.31:0/1599799035 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ba806b460 con 0x7f7ba815a710 2023-11-26T07:22:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.181+0000 7f7b9dffb640 1 -- 172.21.15.31:0/1599799035 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v88) v1 ==== 80+0+120 (secure 0 0 0) 0x7f7ba00b2090 con 0x7f7ba815a710 2023-11-26T07:22:48.185 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6835/683007803 2023-11-27T07:22:41.280769+0000 2023-11-26T07:22:48.185 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6833/683007803 2023-11-27T07:22:41.280769+0000 2023-11-26T07:22:48.185 INFO:teuthology.orchestra.run.smithi031.stderr:listed 2 entries 2023-11-26T07:22:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 -- 172.21.15.31:0/1599799035 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7b8c066590 msgr2=0x7f7b8c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7b8c066590 0x7f7b8c068a50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4005890 tx=0x7f7ba4009040 comp rx=0 tx=0).stop 2023-11-26T07:22:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 -- 172.21.15.31:0/1599799035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba815a710 msgr2=0x7f7ba8163ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba815a710 0x7f7ba8163ae0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f7ba001a550 tx=0x7f7ba00196c0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 -- 172.21.15.31:0/1599799035 shutdown_connections 2023-11-26T07:22:48.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7b8c066590 0x7f7b8c068a50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ba815a710 0x7f7ba8163ae0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ba80cdfb0 0x7f7ba81635a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 --2- 172.21.15.31:0/1599799035 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ba80cbd50 0x7f7ba815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 -- 172.21.15.31:0/1599799035 >> 172.21.15.31:0/1599799035 conn(0x7f7ba80b0cb0 msgr2=0x7f7ba80b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 -- 172.21.15.31:0/1599799035 shutdown_connections 2023-11-26T07:22:48.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.185+0000 7f7baf6e4640 1 -- 172.21.15.31:0/1599799035 wait complete. 2023-11-26T07:22:48.206 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:22:48.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:48.206 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:22:48.236 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:22:48.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:22:48.236 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:22:48.291 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:22:48.291 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:48.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:22:48.300 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:48.300 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:48.342 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:22:48.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:48.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.481+0000 7f0ed9157640 1 Processor -- start 2023-11-26T07:22:48.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.481+0000 7f0ed9157640 1 -- start start 2023-11-26T07:22:48.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.481+0000 7f0ed9157640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f0ed4146c60 0x7f0ed4070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.481+0000 7f0ed9157640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ed4147040 0x7f0ed4070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed9157640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed415dba0 con 0x7f0ed406bb10 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed9157640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed406c670 con 0x7f0ed4147040 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed9157640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed406d100 con 0x7f0ed4146c60 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed259c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ed4147040 0x7f0ed4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed259c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0ed4147040 0x7f0ed4070560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41784/0 (socket says 172.21.15.31:41784) 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed259c640 1 -- 172.21.15.31:0/1201136717 learned_addr learned my addr 172.21.15.31:0/1201136717 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed2d9d640 1 --2- 172.21.15.31:0/1201136717 >> v2:172.21.15.97:3300/0 conn(0x7f0ed4146c60 0x7f0ed4070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed259c640 1 -- 172.21.15.31:0/1201136717 >> v2:172.21.15.97:3300/0 conn(0x7f0ed4146c60 msgr2=0x7f0ed4070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed259c640 1 --2- 172.21.15.31:0/1201136717 >> v2:172.21.15.97:3300/0 conn(0x7f0ed4146c60 0x7f0ed4070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed259c640 1 -- 172.21.15.31:0/1201136717 >> v1:172.21.15.97:6789/0 conn(0x7f0ed406bb10 legacy=0x7f0ed406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:48.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed259c640 1 -- 172.21.15.31:0/1201136717 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed406c860 con 0x7f0ed4147040 2023-11-26T07:22:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed2d9d640 1 --2- 172.21.15.31:0/1201136717 >> v2:172.21.15.97:3300/0 conn(0x7f0ed4146c60 0x7f0ed4070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:48.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed259c640 1 --2- 172.21.15.31:0/1201136717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4147040 0x7f0ed4070560 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0ed406f970 tx=0x7f0ec801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b5187d11e1b51c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed1d9b640 1 -- 172.21.15.31:0/1201136717 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80ab600 con 0x7f0ed4147040 2023-11-26T07:22:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed1d9b640 1 -- 172.21.15.31:0/1201136717 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec80ab760 con 0x7f0ed4147040 2023-11-26T07:22:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.485+0000 7f0ed1d9b640 1 -- 172.21.15.31:0/1201136717 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80abaf0 con 0x7f0ed4147040 2023-11-26T07:22:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 -- 172.21.15.31:0/1201136717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4147040 msgr2=0x7f0ed4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 --2- 172.21.15.31:0/1201136717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4147040 0x7f0ed4070560 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0ed406f970 tx=0x7f0ec801c400 comp rx=0 tx=0).stop 2023-11-26T07:22:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 -- 172.21.15.31:0/1201136717 shutdown_connections 2023-11-26T07:22:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 --2- 172.21.15.31:0/1201136717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4147040 0x7f0ed4070560 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 --2- 172.21.15.31:0/1201136717 >> v2:172.21.15.97:3300/0 conn(0x7f0ed4146c60 0x7f0ed4070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 -- 172.21.15.31:0/1201136717 >> 172.21.15.31:0/1201136717 conn(0x7f0ed40b2ea0 msgr2=0x7f0ed40b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 -- 172.21.15.31:0/1201136717 shutdown_connections 2023-11-26T07:22:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 -- 172.21.15.31:0/1201136717 wait complete. 2023-11-26T07:22:48.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 Processor -- start 2023-11-26T07:22:48.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 -- start start 2023-11-26T07:22:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ed406bb10 0x7f0ed41392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4146c60 0x7f0ed415ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ed4147040 0x7f0ed406e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed415b650 con 0x7f0ed4146c60 2023-11-26T07:22:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed415b8c0 con 0x7f0ed4147040 2023-11-26T07:22:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed9157640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed415bba0 con 0x7f0ed406bb10 2023-11-26T07:22:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4146c60 0x7f0ed415ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4146c60 0x7f0ed415ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41794/0 (socket says 172.21.15.31:41794) 2023-11-26T07:22:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.489+0000 7f0ed259c640 1 -- 172.21.15.31:0/3515795277 learned_addr learned my addr 172.21.15.31:0/3515795277 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed2d9d640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ed406bb10 0x7f0ed41392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed359e640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ed4147040 0x7f0ed406e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed259c640 1 -- 172.21.15.31:0/3515795277 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ed406bb10 msgr2=0x7f0ed41392b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed259c640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ed406bb10 0x7f0ed41392b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed259c640 1 -- 172.21.15.31:0/3515795277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ed4147040 msgr2=0x7f0ed406e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed259c640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ed4147040 0x7f0ed406e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed259c640 1 -- 172.21.15.31:0/3515795277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec80094d0 con 0x7f0ed4146c60 2023-11-26T07:22:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed259c640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4146c60 0x7f0ed415ad50 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0ec801a030 tx=0x7f0ec801ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ebb7fe640 1 -- 172.21.15.31:0/3515795277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80b4800 con 0x7f0ed4146c60 2023-11-26T07:22:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ebb7fe640 1 -- 172.21.15.31:0/3515795277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec80b4960 con 0x7f0ed4146c60 2023-11-26T07:22:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ebb7fe640 1 -- 172.21.15.31:0/3515795277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80b4cf0 con 0x7f0ed4146c60 2023-11-26T07:22:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed9157640 1 -- 172.21.15.31:0/3515795277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed415be80 con 0x7f0ed4146c60 2023-11-26T07:22:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed9157640 1 -- 172.21.15.31:0/3515795277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed41397f0 con 0x7f0ed4146c60 2023-11-26T07:22:48.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ed9157640 1 -- 172.21.15.31:0/3515795277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed404e850 con 0x7f0ed4146c60 2023-11-26T07:22:48.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.493+0000 7f0ebb7fe640 1 -- 172.21.15.31:0/3515795277 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f0ec80c7440 con 0x7f0ed4146c60 2023-11-26T07:22:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.497+0000 7f0ebb7fe640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0ec0066590 0x7f0ec0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.497+0000 7f0ebb7fe640 1 -- 172.21.15.31:0/3515795277 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0ec811ecb0 con 0x7f0ed4146c60 2023-11-26T07:22:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.497+0000 7f0ed2d9d640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0ec0066590 0x7f0ec0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.497+0000 7f0ed2d9d640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0ec0066590 0x7f0ec0068a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ed41441c0 tx=0x7f0ec4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.501+0000 7f0ebb7fe640 1 -- 172.21.15.31:0/3515795277 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0ec80ebf20 con 0x7f0ed4146c60 2023-11-26T07:22:48.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.693+0000 7f0ed9157640 1 -- 172.21.15.31:0/3515795277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ed415c330 con 0x7f0ed4146c60 2023-11-26T07:22:48.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.693+0000 7f0ebb7fe640 1 -- 172.21.15.31:0/3515795277 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f0ec80ef540 con 0x7f0ed4146c60 2023-11-26T07:22:48.697 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:48.697 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1684629063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1684629063},{"type":"v1","addr":"172.21.15.31:6833","nonce":1684629063}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/1262063204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1262063204},{"type":"v1","addr":"172.21.15.97:6834","nonce":1262063204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2863533780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2863533780},{"type":"v1","addr":"172.21.15.97:6835","nonce":2863533780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:22.526776+0000","modified":"2023-11-26T07:22:25.559499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7101":{"gid":7101,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3729395637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3729395637},{"type":"v1","addr":"172.21.15.31:6835","nonce":3729395637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:22:48.697 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 63 2023-11-26T07:22:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 -- 172.21.15.31:0/3515795277 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0ec0066590 msgr2=0x7f0ec0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0ec0066590 0x7f0ec0068a50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ed41441c0 tx=0x7f0ec4008040 comp rx=0 tx=0).stop 2023-11-26T07:22:48.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 -- 172.21.15.31:0/3515795277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4146c60 msgr2=0x7f0ed415ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4146c60 0x7f0ed415ad50 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0ec801a030 tx=0x7f0ec801ac40 comp rx=0 tx=0).stop 2023-11-26T07:22:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 -- 172.21.15.31:0/3515795277 shutdown_connections 2023-11-26T07:22:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0ec0066590 0x7f0ec0068a50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ed4147040 0x7f0ed406e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4146c60 0x7f0ed415ad50 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 --2- 172.21.15.31:0/3515795277 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0ed406bb10 0x7f0ed41392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 -- 172.21.15.31:0/3515795277 >> 172.21.15.31:0/3515795277 conn(0x7f0ed40b2ea0 msgr2=0x7f0ed4070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 -- 172.21.15.31:0/3515795277 shutdown_connections 2023-11-26T07:22:48.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.697+0000 7f0ed9157640 1 -- 172.21.15.31:0/3515795277 wait complete. 2023-11-26T07:22:48.719 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:48.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.857+0000 7f167fba1640 1 Processor -- start 2023-11-26T07:22:48.862 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.857+0000 7f167fba1640 1 -- start start 2023-11-26T07:22:48.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167fba1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f167815f540 0x7f16780b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167fba1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f167806cd40 0x7f1678070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167fba1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780b2920 con 0x7f16780b1f40 2023-11-26T07:22:48.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167fba1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16780b2c00 con 0x7f167806cd40 2023-11-26T07:22:48.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167fba1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780b2ee0 con 0x7f167815f540 2023-11-26T07:22:48.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167e13e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f167806cd40 0x7f1678070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167e13e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f167806cd40 0x7f1678070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41796/0 (socket says 172.21.15.31:41796) 2023-11-26T07:22:48.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167e13e640 1 -- 172.21.15.31:0/1117129857 learned_addr learned my addr 172.21.15.31:0/1117129857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:48.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167d93d640 1 --2- 172.21.15.31:0/1117129857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f167815f540 0x7f16780b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167e13e640 1 -- 172.21.15.31:0/1117129857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f167815f540 msgr2=0x7f16780b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:48.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167e13e640 1 --2- 172.21.15.31:0/1117129857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f167815f540 0x7f16780b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167e13e640 1 -- 172.21.15.31:0/1117129857 >> v1:172.21.15.97:6789/0 conn(0x7f16780b1f40 legacy=0x7f167806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.861+0000 7f167e13e640 1 -- 172.21.15.31:0/1117129857 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780b31c0 con 0x7f167806cd40 2023-11-26T07:22:48.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167e13e640 1 --2- 172.21.15.31:0/1117129857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f167806cd40 0x7f1678070880 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1670009e50 tx=0x7f167001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202e4274f961777b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167c93b640 1 -- 172.21.15.31:0/1117129857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16700a57c0 con 0x7f167806cd40 2023-11-26T07:22:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167c93b640 1 -- 172.21.15.31:0/1117129857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16700a5920 con 0x7f167806cd40 2023-11-26T07:22:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167c93b640 1 -- 172.21.15.31:0/1117129857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16700a5cb0 con 0x7f167806cd40 2023-11-26T07:22:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167fba1640 1 -- 172.21.15.31:0/1117129857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f167806cd40 msgr2=0x7f1678070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167fba1640 1 --2- 172.21.15.31:0/1117129857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f167806cd40 0x7f1678070880 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1670009e50 tx=0x7f167001c9e0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167fba1640 1 -- 172.21.15.31:0/1117129857 shutdown_connections 2023-11-26T07:22:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167fba1640 1 --2- 172.21.15.31:0/1117129857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f167806cd40 0x7f1678070880 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167fba1640 1 --2- 172.21.15.31:0/1117129857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f167815f540 0x7f16780b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167fba1640 1 -- 172.21.15.31:0/1117129857 >> 172.21.15.31:0/1117129857 conn(0x7f16780b0c40 msgr2=0x7f16780b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:48.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167fba1640 1 -- 172.21.15.31:0/1117129857 shutdown_connections 2023-11-26T07:22:48.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167fba1640 1 -- 172.21.15.31:0/1117129857 wait complete. 2023-11-26T07:22:48.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167fba1640 1 Processor -- start 2023-11-26T07:22:48.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.865+0000 7f167fba1640 1 -- start start 2023-11-26T07:22:48.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167fba1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f167806cd40 0x7f16781595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167fba1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16780b1f40 0x7f167814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167fba1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167815f540 0x7f1678157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167fba1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678156600 con 0x7f16780b1f40 2023-11-26T07:22:48.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167fba1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678156a20 con 0x7f167815f540 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167fba1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780ada90 con 0x7f167806cd40 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167d13c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16780b1f40 0x7f167814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167d13c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16780b1f40 0x7f167814fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41800/0 (socket says 172.21.15.31:41800) 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167d13c640 1 -- 172.21.15.31:0/45023621 learned_addr learned my addr 172.21.15.31:0/45023621 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167d93d640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f167806cd40 0x7f16781595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167e13e640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167815f540 0x7f1678157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167d13c640 1 -- 172.21.15.31:0/45023621 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f167806cd40 msgr2=0x7f16781595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167d13c640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f167806cd40 0x7f16781595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167d13c640 1 -- 172.21.15.31:0/45023621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167815f540 msgr2=0x7f1678157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167d13c640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167815f540 0x7f1678157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:48.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167d13c640 1 -- 172.21.15.31:0/45023621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1670009b30 con 0x7f16780b1f40 2023-11-26T07:22:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167d13c640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16780b1f40 0x7f167814fbb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f166800c450 tx=0x7f166800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f166e7fc640 1 -- 172.21.15.31:0/45023621 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16680052d0 con 0x7f16780b1f40 2023-11-26T07:22:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f166e7fc640 1 -- 172.21.15.31:0/45023621 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668005450 con 0x7f16780b1f40 2023-11-26T07:22:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f166e7fc640 1 -- 172.21.15.31:0/45023621 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668012870 con 0x7f16780b1f40 2023-11-26T07:22:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167fba1640 1 -- 172.21.15.31:0/45023621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1678155980 con 0x7f16780b1f40 2023-11-26T07:22:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167fba1640 1 -- 172.21.15.31:0/45023621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1678155f40 con 0x7f16780b1f40 2023-11-26T07:22:48.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.869+0000 7f167fba1640 1 -- 172.21.15.31:0/45023621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1678156740 con 0x7f16780b1f40 2023-11-26T07:22:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.873+0000 7f166e7fc640 1 -- 172.21.15.31:0/45023621 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f16680129d0 con 0x7f16780b1f40 2023-11-26T07:22:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.873+0000 7f166e7fc640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f165c066590 0x7f165c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.873+0000 7f166e7fc640 1 -- 172.21.15.31:0/45023621 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1668083a30 con 0x7f16780b1f40 2023-11-26T07:22:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.873+0000 7f167d93d640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f165c066590 0x7f165c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:48.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.873+0000 7f167d93d640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f165c066590 0x7f165c068a50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f16740079c0 tx=0x7f1674008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:48.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:48.877+0000 7f166e7fc640 1 -- 172.21.15.31:0/45023621 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1678156740 con 0x7f16780b1f40 2023-11-26T07:22:49.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.033+0000 7f167fba1640 1 -- 172.21.15.31:0/45023621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1678156740 con 0x7f16780b1f40 2023-11-26T07:22:49.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f166e7fc640 1 -- 172.21.15.31:0/45023621 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16603 (secure 0 0 0) 0x7f1678156740 con 0x7f16780b1f40 2023-11-26T07:22:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:49.039 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:44.288769+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:20.941587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:21.520554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/683007803":"2023-11-27T07:22:41.280769+0000","172.21.15.97:6833/683007803":"2023-11-27T07:22:41.280769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 -- 172.21.15.31:0/45023621 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f165c066590 msgr2=0x7f165c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f165c066590 0x7f165c068a50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f16740079c0 tx=0x7f1674008040 comp rx=0 tx=0).stop 2023-11-26T07:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 -- 172.21.15.31:0/45023621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16780b1f40 msgr2=0x7f167814fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16780b1f40 0x7f167814fbb0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f166800c450 tx=0x7f166800c910 comp rx=0 tx=0).stop 2023-11-26T07:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 -- 172.21.15.31:0/45023621 shutdown_connections 2023-11-26T07:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f165c066590 0x7f165c068a50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f167815f540 0x7f1678157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16780b1f40 0x7f167814fbb0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 --2- 172.21.15.31:0/45023621 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f167806cd40 0x7f16781595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 -- 172.21.15.31:0/45023621 >> 172.21.15.31:0/45023621 conn(0x7f16780b0c40 msgr2=0x7f1678164840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:49.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 -- 172.21.15.31:0/45023621 shutdown_connections 2023-11-26T07:22:49.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.037+0000 7f167fba1640 1 -- 172.21.15.31:0/45023621 wait complete. 2023-11-26T07:22:49.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:22:20.941587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:22:21.520554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:49.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:49.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e87e7640 1 Processor -- start 2023-11-26T07:22:49.206 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e87e7640 1 -- start start 2023-11-26T07:22:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e87e7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd3e015a710 0x7fd3e00abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e87e7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3e00cbd50 0x7fd3e00cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e87e7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e00b2710 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e87e7640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e00cca80 con 0x7fd3e00cbd50 2023-11-26T07:22:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e87e7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e00b29f0 con 0x7fd3e015a710 2023-11-26T07:22:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e6d84640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3e00cbd50 0x7fd3e00cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e6d84640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd3e00cbd50 0x7fd3e00cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41804/0 (socket says 172.21.15.31:41804) 2023-11-26T07:22:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e6d84640 1 -- 172.21.15.31:0/900177824 learned_addr learned my addr 172.21.15.31:0/900177824 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e5d82640 1 --2- 172.21.15.31:0/900177824 >> v2:172.21.15.97:3300/0 conn(0x7fd3e015a710 0x7fd3e00abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e6d84640 1 -- 172.21.15.31:0/900177824 >> v2:172.21.15.97:3300/0 conn(0x7fd3e015a710 msgr2=0x7fd3e00abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e6d84640 1 --2- 172.21.15.31:0/900177824 >> v2:172.21.15.97:3300/0 conn(0x7fd3e015a710 0x7fd3e00abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e6d84640 1 -- 172.21.15.31:0/900177824 >> v1:172.21.15.97:6789/0 conn(0x7fd3e00cdfb0 legacy=0x7fd3e00ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.205+0000 7fd3e6d84640 1 -- 172.21.15.31:0/900177824 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e00b2cd0 con 0x7fd3e00cbd50 2023-11-26T07:22:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e6d84640 1 --2- 172.21.15.31:0/900177824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cbd50 0x7fd3e00cc1e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00d0040 tx=0x7fd3d801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec18539931d33989 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e5581640 1 -- 172.21.15.31:0/900177824 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80a5880 con 0x7fd3e00cbd50 2023-11-26T07:22:49.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e5581640 1 -- 172.21.15.31:0/900177824 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d80a59e0 con 0x7fd3e00cbd50 2023-11-26T07:22:49.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e5581640 1 -- 172.21.15.31:0/900177824 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80a5d70 con 0x7fd3e00cbd50 2023-11-26T07:22:49.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e87e7640 1 -- 172.21.15.31:0/900177824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cbd50 msgr2=0x7fd3e00cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e87e7640 1 --2- 172.21.15.31:0/900177824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cbd50 0x7fd3e00cc1e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00d0040 tx=0x7fd3d801c9a0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e87e7640 1 -- 172.21.15.31:0/900177824 shutdown_connections 2023-11-26T07:22:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e87e7640 1 --2- 172.21.15.31:0/900177824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cbd50 0x7fd3e00cc1e0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e87e7640 1 --2- 172.21.15.31:0/900177824 >> v2:172.21.15.97:3300/0 conn(0x7fd3e015a710 0x7fd3e00abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e87e7640 1 -- 172.21.15.31:0/900177824 >> 172.21.15.31:0/900177824 conn(0x7fd3e00b0cb0 msgr2=0x7fd3e00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e87e7640 1 -- 172.21.15.31:0/900177824 shutdown_connections 2023-11-26T07:22:49.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e87e7640 1 -- 172.21.15.31:0/900177824 wait complete. 2023-11-26T07:22:49.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.209+0000 7fd3e87e7640 1 Processor -- start 2023-11-26T07:22:49.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e87e7640 1 -- start start 2023-11-26T07:22:49.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e87e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd3e00cbd50 0x7fd3e01f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e87e7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cdfb0 0x7fd3e01f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e87e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3e015a710 0x7fd3e015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e87e7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e01f93b0 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e87e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e015d680 con 0x7fd3e015a710 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e87e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e015d960 con 0x7fd3e00cbd50 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e5d82640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cdfb0 0x7fd3e01f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e5d82640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cdfb0 0x7fd3e01f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41818/0 (socket says 172.21.15.31:41818) 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e5d82640 1 -- 172.21.15.31:0/4164852643 learned_addr learned my addr 172.21.15.31:0/4164852643 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e6583640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd3e00cbd50 0x7fd3e01f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e6d84640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3e015a710 0x7fd3e015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e5d82640 1 -- 172.21.15.31:0/4164852643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd3e00cbd50 msgr2=0x7fd3e01f9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e5d82640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd3e00cbd50 0x7fd3e01f9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e5d82640 1 -- 172.21.15.31:0/4164852643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3e015a710 msgr2=0x7fd3e015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e5d82640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3e015a710 0x7fd3e015cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e5d82640 1 -- 172.21.15.31:0/4164852643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d8009b30 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e5d82640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cdfb0 0x7fd3e01f66c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000d7d0 tx=0x7fd3d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3d6ffd640 1 -- 172.21.15.31:0/4164852643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d000b6c0 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3d6ffd640 1 -- 172.21.15.31:0/4164852643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d000b820 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e87e7640 1 -- 172.21.15.31:0/4164852643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e01f9670 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3d6ffd640 1 -- 172.21.15.31:0/4164852643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d000bbb0 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e87e7640 1 -- 172.21.15.31:0/4164852643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e0153a80 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.213+0000 7fd3e87e7640 1 -- 172.21.15.31:0/4164852643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e00671f0 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.217+0000 7fd3d6ffd640 1 -- 172.21.15.31:0/4164852643 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fd3d000bd10 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.217+0000 7fd3d6ffd640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd3c40665c0 0x7fd3c4068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.217+0000 7fd3d6ffd640 1 -- 172.21.15.31:0/4164852643 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd3d0084f40 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.217+0000 7fd3e6583640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd3c40665c0 0x7fd3c4068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.217+0000 7fd3e6583640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd3c40665c0 0x7fd3c4068a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc007fd0 tx=0x7fd3dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.221+0000 7fd3d6ffd640 1 -- 172.21.15.31:0/4164852643 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd3d00522e0 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.377+0000 7fd3e87e7640 1 -- 172.21.15.31:0/4164852643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3e006b460 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.381 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.377+0000 7fd3d6ffd640 1 -- 172.21.15.31:0/4164852643 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3d0055900 con 0x7fd3e00cdfb0 2023-11-26T07:22:49.381 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:22:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 -- 172.21.15.31:0/4164852643 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd3c40665c0 msgr2=0x7fd3c4068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd3c40665c0 0x7fd3c4068a80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc007fd0 tx=0x7fd3dc009040 comp rx=0 tx=0).stop 2023-11-26T07:22:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 -- 172.21.15.31:0/4164852643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cdfb0 msgr2=0x7fd3e01f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cdfb0 0x7fd3e01f66c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000d7d0 tx=0x7fd3d000dc90 comp rx=0 tx=0).stop 2023-11-26T07:22:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 -- 172.21.15.31:0/4164852643 shutdown_connections 2023-11-26T07:22:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd3c40665c0 0x7fd3c4068a80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3e015a710 0x7fd3e015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3e00cdfb0 0x7fd3e01f66c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 --2- 172.21.15.31:0/4164852643 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd3e00cbd50 0x7fd3e01f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 -- 172.21.15.31:0/4164852643 >> 172.21.15.31:0/4164852643 conn(0x7fd3e00b0cb0 msgr2=0x7fd3e00b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:49.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 -- 172.21.15.31:0/4164852643 shutdown_connections 2023-11-26T07:22:49.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.381+0000 7fd3e87e7640 1 -- 172.21.15.31:0/4164852643 wait complete. 2023-11-26T07:22:49.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:22:49.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.545+0000 7f52bf074640 1 Processor -- start 2023-11-26T07:22:49.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.545+0000 7f52bf074640 1 -- start start 2023-11-26T07:22:49.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.549+0000 7f52bf074640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52b806ef60 0x7f52b8168010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.549+0000 7f52bf074640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b806f510 con 0x7f52b8146b90 2023-11-26T07:22:49.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.549+0000 7f52bf074640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b80707f0 con 0x7f52b80b4650 2023-11-26T07:22:49.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.549+0000 7f52bf074640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b8146f70 con 0x7f52b806ef60 2023-11-26T07:22:49.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.549+0000 7f52be072640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f52b8146b90 0x7f52b80b3f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59586/0 (socket says 172.21.15.31:59586) 2023-11-26T07:22:49.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.549+0000 7f52be072640 1 -- 172.21.15.31:0/134938874 learned_addr learned my addr 172.21.15.31:0/134938874 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:49.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.549+0000 7f52be873640 1 --2- 172.21.15.31:0/134938874 >> v2:172.21.15.31:3300/0 conn(0x7f52b806ef60 0x7f52b8168010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.549+0000 7f52be873640 1 -- 172.21.15.31:0/134938874 >> v1:172.21.15.97:6789/0 conn(0x7f52b80b4650 legacy=0x7f52b806e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.549+0000 7f52be873640 1 -- 172.21.15.31:0/134938874 >> v1:172.21.15.31:6789/0 conn(0x7f52b8146b90 legacy=0x7f52b80b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.549+0000 7f52be873640 1 -- 172.21.15.31:0/134938874 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b806fec0 con 0x7f52b806ef60 2023-11-26T07:22:49.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52be873640 1 --2- 172.21.15.31:0/134938874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b806ef60 0x7f52b8168010 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009d90 tx=0x7f52b001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1284149570b6c85c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bd070640 1 -- 172.21.15.31:0/134938874 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00a5950 con 0x7f52b806ef60 2023-11-26T07:22:49.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bd070640 1 -- 172.21.15.31:0/134938874 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b00a5ab0 con 0x7f52b806ef60 2023-11-26T07:22:49.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bd070640 1 -- 172.21.15.31:0/134938874 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00a5e40 con 0x7f52b806ef60 2023-11-26T07:22:49.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 -- 172.21.15.31:0/134938874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b806ef60 msgr2=0x7f52b8168010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 --2- 172.21.15.31:0/134938874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b806ef60 0x7f52b8168010 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009d90 tx=0x7f52b001c890 comp rx=0 tx=0).stop 2023-11-26T07:22:49.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 -- 172.21.15.31:0/134938874 shutdown_connections 2023-11-26T07:22:49.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 --2- 172.21.15.31:0/134938874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b806ef60 0x7f52b8168010 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 -- 172.21.15.31:0/134938874 >> 172.21.15.31:0/134938874 conn(0x7f52b80d0fb0 msgr2=0x7f52b80d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:49.555 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 -- 172.21.15.31:0/134938874 shutdown_connections 2023-11-26T07:22:49.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 -- 172.21.15.31:0/134938874 wait complete. 2023-11-26T07:22:49.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 Processor -- start 2023-11-26T07:22:49.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 -- start start 2023-11-26T07:22:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52b806ef60 0x7f52b813d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52b80b4650 0x7f52b8163850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b8146b90 0x7f52b80cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b81633c0 con 0x7f52b8146b90 2023-11-26T07:22:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b815c840 con 0x7f52b80b4650 2023-11-26T07:22:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.553+0000 7f52bf074640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b8162a90 con 0x7f52b806ef60 2023-11-26T07:22:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be873640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b8146b90 0x7f52b80cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be873640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b8146b90 0x7f52b80cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41836/0 (socket says 172.21.15.31:41836) 2023-11-26T07:22:49.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be873640 1 -- 172.21.15.31:0/1165086775 learned_addr learned my addr 172.21.15.31:0/1165086775 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be072640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52b806ef60 0x7f52b813d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52bd871640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52b80b4650 0x7f52b8163850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be873640 1 -- 172.21.15.31:0/1165086775 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52b806ef60 msgr2=0x7f52b813d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be873640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52b806ef60 0x7f52b813d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be873640 1 -- 172.21.15.31:0/1165086775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52b80b4650 msgr2=0x7f52b8163850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be873640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52b80b4650 0x7f52b8163850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be873640 1 -- 172.21.15.31:0/1165086775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b0009a70 con 0x7f52b8146b90 2023-11-26T07:22:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be072640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52b806ef60 0x7f52b813d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:49.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52bd871640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52b80b4650 0x7f52b8163850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52be873640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b8146b90 0x7f52b80cfc30 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f52b001a4c0 tx=0x7f52b0019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52aeffd640 1 -- 172.21.15.31:0/1165086775 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00ade40 con 0x7f52b8146b90 2023-11-26T07:22:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52aeffd640 1 -- 172.21.15.31:0/1165086775 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b00b9450 con 0x7f52b8146b90 2023-11-26T07:22:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52aeffd640 1 -- 172.21.15.31:0/1165086775 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00b97e0 con 0x7f52b8146b90 2023-11-26T07:22:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52bf074640 1 -- 172.21.15.31:0/1165086775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b8162d70 con 0x7f52b8146b90 2023-11-26T07:22:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52bf074640 1 -- 172.21.15.31:0/1165086775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b815c460 con 0x7f52b8146b90 2023-11-26T07:22:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52bf074640 1 -- 172.21.15.31:0/1165086775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b8163d90 con 0x7f52b8146b90 2023-11-26T07:22:49.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.557+0000 7f52aeffd640 1 -- 172.21.15.31:0/1165086775 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f52b00b9940 con 0x7f52b8146b90 2023-11-26T07:22:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.561+0000 7f52aeffd640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f529c0665c0 0x7f529c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.561+0000 7f52aeffd640 1 -- 172.21.15.31:0/1165086775 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f52b011f3f0 con 0x7f52b8146b90 2023-11-26T07:22:49.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.561+0000 7f52be072640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f529c0665c0 0x7f529c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:49.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.561+0000 7f52be072640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f529c0665c0 0x7f529c068a80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005890 tx=0x7f52b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:49.566 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.561+0000 7f52aeffd640 1 -- 172.21.15.31:0/1165086775 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f52b00ec840 con 0x7f52b8146b90 2023-11-26T07:22:49.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.721+0000 7f52bf074640 1 -- 172.21.15.31:0/1165086775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52b8070c40 con 0x7f52b8146b90 2023-11-26T07:22:49.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.721+0000 7f52aeffd640 1 -- 172.21.15.31:0/1165086775 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52b8070c40 con 0x7f52b8146b90 2023-11-26T07:22:49.925 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:22:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 -- 172.21.15.31:0/1165086775 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f529c0665c0 msgr2=0x7f529c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f529c0665c0 0x7f529c068a80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005890 tx=0x7f52b4009040 comp rx=0 tx=0).stop 2023-11-26T07:22:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 -- 172.21.15.31:0/1165086775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b8146b90 msgr2=0x7f52b80cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:49.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b8146b90 0x7f52b80cfc30 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f52b001a4c0 tx=0x7f52b0019630 comp rx=0 tx=0).stop 2023-11-26T07:22:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 -- 172.21.15.31:0/1165086775 shutdown_connections 2023-11-26T07:22:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f529c0665c0 0x7f529c068a80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52b8146b90 0x7f52b80cfc30 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f52b80b4650 0x7f52b8163850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 --2- 172.21.15.31:0/1165086775 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f52b806ef60 0x7f52b813d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 -- 172.21.15.31:0/1165086775 >> 172.21.15.31:0/1165086775 conn(0x7f52b80d0fb0 msgr2=0x7f52b813ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 -- 172.21.15.31:0/1165086775 shutdown_connections 2023-11-26T07:22:49.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:49.725+0000 7f52bf074640 1 -- 172.21.15.31:0/1165086775 wait complete. 2023-11-26T07:22:49.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:22:50.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c496fe640 1 Processor -- start 2023-11-26T07:22:50.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c496fe640 1 -- start start 2023-11-26T07:22:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c496fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c4415a710 0x7f7c440abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c496fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c440cbd50 0x7f7c440cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c496fe640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c440b2710 con 0x7f7c440cdfb0 2023-11-26T07:22:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c496fe640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c440cca80 con 0x7f7c4415a710 2023-11-26T07:22:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c496fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440b29f0 con 0x7f7c440cbd50 2023-11-26T07:22:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c3bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c4415a710 0x7f7c440abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c3bfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7c4415a710 0x7f7c440abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41844/0 (socket says 172.21.15.31:41844) 2023-11-26T07:22:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c3bfff640 1 -- 172.21.15.31:0/4011000917 learned_addr learned my addr 172.21.15.31:0/4011000917 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:50.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c437fe640 1 --2- 172.21.15.31:0/4011000917 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c440cbd50 0x7f7c440cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c3bfff640 1 -- 172.21.15.31:0/4011000917 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c440cbd50 msgr2=0x7f7c440cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c3bfff640 1 --2- 172.21.15.31:0/4011000917 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c440cbd50 0x7f7c440cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c3bfff640 1 -- 172.21.15.31:0/4011000917 >> v1:172.21.15.97:6789/0 conn(0x7f7c440cdfb0 legacy=0x7f7c440ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.077+0000 7f7c3bfff640 1 -- 172.21.15.31:0/4011000917 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440b2cd0 con 0x7f7c4415a710 2023-11-26T07:22:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c3bfff640 1 --2- 172.21.15.31:0/4011000917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4415a710 0x7f7c440abf90 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009870 tx=0x7f7c2c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f3b86c37d437bb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c427fc640 1 -- 172.21.15.31:0/4011000917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c0a95f0 con 0x7f7c4415a710 2023-11-26T07:22:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c427fc640 1 -- 172.21.15.31:0/4011000917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c0a9750 con 0x7f7c4415a710 2023-11-26T07:22:50.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c427fc640 1 -- 172.21.15.31:0/4011000917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c0a9ae0 con 0x7f7c4415a710 2023-11-26T07:22:50.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c496fe640 1 -- 172.21.15.31:0/4011000917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4415a710 msgr2=0x7f7c440abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c496fe640 1 --2- 172.21.15.31:0/4011000917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4415a710 0x7f7c440abf90 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c009870 tx=0x7f7c2c01c510 comp rx=0 tx=0).stop 2023-11-26T07:22:50.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c496fe640 1 -- 172.21.15.31:0/4011000917 shutdown_connections 2023-11-26T07:22:50.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c496fe640 1 --2- 172.21.15.31:0/4011000917 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c440cbd50 0x7f7c440cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c496fe640 1 --2- 172.21.15.31:0/4011000917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c4415a710 0x7f7c440abf90 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c496fe640 1 -- 172.21.15.31:0/4011000917 >> 172.21.15.31:0/4011000917 conn(0x7f7c440b0cb0 msgr2=0x7f7c440b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c496fe640 1 -- 172.21.15.31:0/4011000917 shutdown_connections 2023-11-26T07:22:50.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c496fe640 1 -- 172.21.15.31:0/4011000917 wait complete. 2023-11-26T07:22:50.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c496fe640 1 Processor -- start 2023-11-26T07:22:50.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.085+0000 7f7c496fe640 1 -- start start 2023-11-26T07:22:50.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c496fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c440cbd50 0x7f7c44153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c496fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c440cdfb0 0x7f7c44099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c496fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c4415a710 0x7f7c441575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c496fe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44156070 con 0x7f7c440cdfb0 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c496fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44157af0 con 0x7f7c440cbd50 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c496fe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44154330 con 0x7f7c4415a710 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c440cdfb0 0x7f7c44099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c440cdfb0 0x7f7c44099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41848/0 (socket says 172.21.15.31:41848) 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3bfff640 1 -- 172.21.15.31:0/4037868180 learned_addr learned my addr 172.21.15.31:0/4037868180 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c42ffd640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c440cbd50 0x7f7c44153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c437fe640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c4415a710 0x7f7c441575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3bfff640 1 -- 172.21.15.31:0/4037868180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c4415a710 msgr2=0x7f7c441575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3bfff640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c4415a710 0x7f7c441575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3bfff640 1 -- 172.21.15.31:0/4037868180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c440cbd50 msgr2=0x7f7c44153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3bfff640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c440cbd50 0x7f7c44153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3bfff640 1 -- 172.21.15.31:0/4037868180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c0094d0 con 0x7f7c440cdfb0 2023-11-26T07:22:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3bfff640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c440cdfb0 0x7f7c44099a30 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c01abc0 tx=0x7f7c2c0192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3b7fe640 1 -- 172.21.15.31:0/4037868180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c0b9620 con 0x7f7c440cdfb0 2023-11-26T07:22:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3b7fe640 1 -- 172.21.15.31:0/4037868180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c0b9780 con 0x7f7c440cdfb0 2023-11-26T07:22:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c496fe640 1 -- 172.21.15.31:0/4037868180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c44154610 con 0x7f7c440cdfb0 2023-11-26T07:22:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c3b7fe640 1 -- 172.21.15.31:0/4037868180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c0b9b10 con 0x7f7c440cdfb0 2023-11-26T07:22:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c496fe640 1 -- 172.21.15.31:0/4037868180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c44155d00 con 0x7f7c440cdfb0 2023-11-26T07:22:50.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.089+0000 7f7c496fe640 1 -- 172.21.15.31:0/4037868180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c440671f0 con 0x7f7c440cdfb0 2023-11-26T07:22:50.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.093+0000 7f7c3b7fe640 1 -- 172.21.15.31:0/4037868180 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f7c2c0b13f0 con 0x7f7c440cdfb0 2023-11-26T07:22:50.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.093+0000 7f7c3b7fe640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7c24066590 0x7f7c24068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.093+0000 7f7c3b7fe640 1 -- 172.21.15.31:0/4037868180 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7c2c11e280 con 0x7f7c440cdfb0 2023-11-26T07:22:50.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.093+0000 7f7c42ffd640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7c24066590 0x7f7c24068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.093+0000 7f7c42ffd640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7c24066590 0x7f7c24068a50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c34000f70 tx=0x7f7c34009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.097+0000 7f7c3b7fe640 1 -- 172.21.15.31:0/4037868180 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7c2c0eb580 con 0x7f7c440cdfb0 2023-11-26T07:22:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.253+0000 7f7c496fe640 1 -- 172.21.15.31:0/4037868180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c4406b460 con 0x7f7c440cdfb0 2023-11-26T07:22:50.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.253+0000 7f7c3b7fe640 1 -- 172.21.15.31:0/4037868180 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7c2c0eeba0 con 0x7f7c440cdfb0 2023-11-26T07:22:50.257 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 -- 172.21.15.31:0/4037868180 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7c24066590 msgr2=0x7f7c24068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7c24066590 0x7f7c24068a50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c34000f70 tx=0x7f7c34009040 comp rx=0 tx=0).stop 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 -- 172.21.15.31:0/4037868180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c440cdfb0 msgr2=0x7f7c44099a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c440cdfb0 0x7f7c44099a30 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c01abc0 tx=0x7f7c2c0192b0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 -- 172.21.15.31:0/4037868180 shutdown_connections 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7c24066590 0x7f7c24068a50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7c4415a710 0x7f7c441575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c440cdfb0 0x7f7c44099a30 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 --2- 172.21.15.31:0/4037868180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c440cbd50 0x7f7c44153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 -- 172.21.15.31:0/4037868180 >> 172.21.15.31:0/4037868180 conn(0x7f7c440b0cb0 msgr2=0x7f7c440af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 -- 172.21.15.31:0/4037868180 shutdown_connections 2023-11-26T07:22:50.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.257+0000 7f7c496fe640 1 -- 172.21.15.31:0/4037868180 wait complete. 2023-11-26T07:22:50.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:50.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.417+0000 7fb4d27f8640 1 Processor -- start 2023-11-26T07:22:50.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.417+0000 7fb4d27f8640 1 -- start start 2023-11-26T07:22:50.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4d27f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc06d920 0x7fb4cc070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4d27f8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc145060 con 0x7fb4cc0ac630 2023-11-26T07:22:50.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4d27f8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc071fd0 con 0x7fb4cc144c80 2023-11-26T07:22:50.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4d27f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc071450 con 0x7fb4cc06d920 2023-11-26T07:22:50.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4cb7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb4cc0ac630 0x7fb4cc0706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59600/0 (socket says 172.21.15.31:59600) 2023-11-26T07:22:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4cb7fe640 1 -- 172.21.15.31:0/535399075 learned_addr learned my addr 172.21.15.31:0/535399075 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4d0d95640 1 --2- 172.21.15.31:0/535399075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc06d920 0x7fb4cc070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4caffd640 1 -- 172.21.15.31:0/535399075 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364622740 0 0) 0x7fb4cc145060 con 0x7fb4cc0ac630 2023-11-26T07:22:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4caffd640 1 -- 172.21.15.31:0/535399075 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b4003530 con 0x7fb4cc0ac630 2023-11-26T07:22:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4caffd640 1 -- 172.21.15.31:0/535399075 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376887952 0 0) 0x7fb4cc071fd0 con 0x7fb4cc144c80 2023-11-26T07:22:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4d0d95640 1 -- 172.21.15.31:0/535399075 >> v1:172.21.15.97:6789/0 conn(0x7fb4cc144c80 legacy=0x7fb4cc0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4d0d95640 1 -- 172.21.15.31:0/535399075 >> v1:172.21.15.31:6789/0 conn(0x7fb4cc0ac630 legacy=0x7fb4cc0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.421+0000 7fb4d0d95640 1 -- 172.21.15.31:0/535399075 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc071730 con 0x7fb4cc06d920 2023-11-26T07:22:50.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4d0d95640 1 --2- 172.21.15.31:0/535399075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc06d920 0x7fb4cc070df0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009dc0 tx=0x7fb4c401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=112c88cfc0e24f65 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4caffd640 1 -- 172.21.15.31:0/535399075 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40a5a00 con 0x7fb4cc06d920 2023-11-26T07:22:50.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4caffd640 1 -- 172.21.15.31:0/535399075 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c40a5b60 con 0x7fb4cc06d920 2023-11-26T07:22:50.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4caffd640 1 -- 172.21.15.31:0/535399075 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40ad440 con 0x7fb4cc06d920 2023-11-26T07:22:50.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4d27f8640 1 -- 172.21.15.31:0/535399075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc06d920 msgr2=0x7fb4cc070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4d27f8640 1 --2- 172.21.15.31:0/535399075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc06d920 0x7fb4cc070df0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009dc0 tx=0x7fb4c401c940 comp rx=0 tx=0).stop 2023-11-26T07:22:50.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4d27f8640 1 -- 172.21.15.31:0/535399075 shutdown_connections 2023-11-26T07:22:50.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4d27f8640 1 --2- 172.21.15.31:0/535399075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc06d920 0x7fb4cc070df0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4d27f8640 1 -- 172.21.15.31:0/535399075 >> 172.21.15.31:0/535399075 conn(0x7fb4cc0b0c40 msgr2=0x7fb4cc0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4d27f8640 1 -- 172.21.15.31:0/535399075 shutdown_connections 2023-11-26T07:22:50.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4d27f8640 1 -- 172.21.15.31:0/535399075 wait complete. 2023-11-26T07:22:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4d27f8640 1 Processor -- start 2023-11-26T07:22:50.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.425+0000 7fb4d27f8640 1 -- start start 2023-11-26T07:22:50.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4d27f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc06d920 0x7fb4cc15e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4d27f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0ac630 0x7fb4cc13a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4d27f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc144c80 0x7fb4cc15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4d27f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc13a640 con 0x7fb4cc06d920 2023-11-26T07:22:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4d27f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc13bf60 con 0x7fb4cc0ac630 2023-11-26T07:22:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4d27f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc13c240 con 0x7fb4cc144c80 2023-11-26T07:22:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc06d920 0x7fb4cc15e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc06d920 0x7fb4cc15e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41862/0 (socket says 172.21.15.31:41862) 2023-11-26T07:22:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4cbfff640 1 -- 172.21.15.31:0/1248899889 learned_addr learned my addr 172.21.15.31:0/1248899889 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4d0d95640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc144c80 0x7fb4cc15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4cb7fe640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0ac630 0x7fb4cc13a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4cbfff640 1 -- 172.21.15.31:0/1248899889 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc144c80 msgr2=0x7fb4cc15a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4cbfff640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc144c80 0x7fb4cc15a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4cbfff640 1 -- 172.21.15.31:0/1248899889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0ac630 msgr2=0x7fb4cc13a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4cbfff640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0ac630 0x7fb4cc13a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4cbfff640 1 -- 172.21.15.31:0/1248899889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c4009a70 con 0x7fb4cc06d920 2023-11-26T07:22:50.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4cbfff640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc06d920 0x7fb4cc15e3e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc001ec0 tx=0x7fb4bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4c8ff9640 1 -- 172.21.15.31:0/1248899889 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0054e0 con 0x7fb4cc06d920 2023-11-26T07:22:50.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4c8ff9640 1 -- 172.21.15.31:0/1248899889 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc003600 con 0x7fb4cc06d920 2023-11-26T07:22:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4c8ff9640 1 -- 172.21.15.31:0/1248899889 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0039b0 con 0x7fb4cc06d920 2023-11-26T07:22:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4d27f8640 1 -- 172.21.15.31:0/1248899889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc13c520 con 0x7fb4cc06d920 2023-11-26T07:22:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4d27f8640 1 -- 172.21.15.31:0/1248899889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc13a870 con 0x7fb4cc06d920 2023-11-26T07:22:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4d27f8640 1 -- 172.21.15.31:0/1248899889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc0b1ba0 con 0x7fb4cc06d920 2023-11-26T07:22:50.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.429+0000 7fb4c8ff9640 1 -- 172.21.15.31:0/1248899889 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fb4bc003b10 con 0x7fb4cc06d920 2023-11-26T07:22:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.433+0000 7fb4c8ff9640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb4b8066590 0x7fb4b8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.433+0000 7fb4c8ff9640 1 -- 172.21.15.31:0/1248899889 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb4bc0835a0 con 0x7fb4cc06d920 2023-11-26T07:22:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.433+0000 7fb4cb7fe640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb4b8066590 0x7fb4b8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.433+0000 7fb4cb7fe640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb4b8066590 0x7fb4b8068a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0007fd0 tx=0x7fb4c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.437+0000 7fb4c8ff9640 1 -- 172.21.15.31:0/1248899889 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4bc050950 con 0x7fb4cc06d920 2023-11-26T07:22:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 -- 172.21.15.31:0/1248899889 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4cc04e850 con 0x7fb4cc06d920 2023-11-26T07:22:50.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4c8ff9640 1 -- 172.21.15.31:0/1248899889 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fb4bc017090 con 0x7fb4cc06d920 2023-11-26T07:22:50.628 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:50.628 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1684629063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1684629063},{"type":"v1","addr":"172.21.15.31:6833","nonce":1684629063}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/1262063204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1262063204},{"type":"v1","addr":"172.21.15.97:6834","nonce":1262063204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2863533780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2863533780},{"type":"v1","addr":"172.21.15.97:6835","nonce":2863533780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:22.526776+0000","modified":"2023-11-26T07:22:25.559499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7101":{"gid":7101,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3729395637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3729395637},{"type":"v1","addr":"172.21.15.31:6835","nonce":3729395637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:22:50.628 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 63 2023-11-26T07:22:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 -- 172.21.15.31:0/1248899889 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb4b8066590 msgr2=0x7fb4b8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb4b8066590 0x7fb4b8068a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0007fd0 tx=0x7fb4c0009040 comp rx=0 tx=0).stop 2023-11-26T07:22:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 -- 172.21.15.31:0/1248899889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc06d920 msgr2=0x7fb4cc15e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc06d920 0x7fb4cc15e3e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc001ec0 tx=0x7fb4bc00c8e0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 -- 172.21.15.31:0/1248899889 shutdown_connections 2023-11-26T07:22:50.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb4b8066590 0x7fb4b8068a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb4cc144c80 0x7fb4cc15a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4cc0ac630 0x7fb4cc13a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 --2- 172.21.15.31:0/1248899889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc06d920 0x7fb4cc15e3e0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.625+0000 7fb4d27f8640 1 -- 172.21.15.31:0/1248899889 >> 172.21.15.31:0/1248899889 conn(0x7fb4cc0b0c40 msgr2=0x7fb4cc0add30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.629+0000 7fb4d27f8640 1 -- 172.21.15.31:0/1248899889 shutdown_connections 2023-11-26T07:22:50.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.629+0000 7fb4d27f8640 1 -- 172.21.15.31:0/1248899889 wait complete. 2023-11-26T07:22:50.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:50.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.789+0000 7f4016580640 1 Processor -- start 2023-11-26T07:22:50.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.789+0000 7f4016580640 1 -- start start 2023-11-26T07:22:50.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.789+0000 7f4016580640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f401006dfd0 0x7f401006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.789+0000 7f4016580640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401006e980 con 0x7f40100adcc0 2023-11-26T07:22:50.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.789+0000 7f4016580640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40100ae7f0 con 0x7f4010143ae0 2023-11-26T07:22:50.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.789+0000 7f4016580640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4010070670 con 0x7f401006dfd0 2023-11-26T07:22:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.789+0000 7f400f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f401006dfd0 0x7f401006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.789+0000 7f4014b1d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f40100adcc0 0x7f40100ae0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59602/0 (socket says 172.21.15.31:59602) 2023-11-26T07:22:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.789+0000 7f400f7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f401006dfd0 0x7f401006e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41870/0 (socket says 172.21.15.31:41870) 2023-11-26T07:22:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.789+0000 7f4014b1d640 1 -- 172.21.15.31:0/1709483310 learned_addr learned my addr 172.21.15.31:0/1709483310 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.793+0000 7f400f7fe640 1 -- 172.21.15.31:0/1709483310 >> v1:172.21.15.97:6789/0 conn(0x7f4010143ae0 legacy=0x7f40100b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.793+0000 7f400f7fe640 1 -- 172.21.15.31:0/1709483310 >> v1:172.21.15.31:6789/0 conn(0x7f40100adcc0 legacy=0x7f40100ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.793+0000 7f400f7fe640 1 -- 172.21.15.31:0/1709483310 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4010070950 con 0x7f401006dfd0 2023-11-26T07:22:50.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.793+0000 7f400f7fe640 1 --2- 172.21.15.31:0/1709483310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401006dfd0 0x7f401006e440 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f400400ba60 tx=0x7f400401eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14ce3d358d27cd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.793+0000 7f400effd640 1 -- 172.21.15.31:0/1709483310 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40040a7bf0 con 0x7f401006dfd0 2023-11-26T07:22:50.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.793+0000 7f400effd640 1 -- 172.21.15.31:0/1709483310 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40040a7d50 con 0x7f401006dfd0 2023-11-26T07:22:50.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.793+0000 7f400effd640 1 -- 172.21.15.31:0/1709483310 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40040af660 con 0x7f401006dfd0 2023-11-26T07:22:50.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.793+0000 7f4016580640 1 -- 172.21.15.31:0/1709483310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401006dfd0 msgr2=0x7f401006e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.793+0000 7f4016580640 1 --2- 172.21.15.31:0/1709483310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401006dfd0 0x7f401006e440 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f400400ba60 tx=0x7f400401eb20 comp rx=0 tx=0).stop 2023-11-26T07:22:50.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 -- 172.21.15.31:0/1709483310 shutdown_connections 2023-11-26T07:22:50.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 --2- 172.21.15.31:0/1709483310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f401006dfd0 0x7f401006e440 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 -- 172.21.15.31:0/1709483310 >> 172.21.15.31:0/1709483310 conn(0x7f40100b0c40 msgr2=0x7f40100b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 -- 172.21.15.31:0/1709483310 shutdown_connections 2023-11-26T07:22:50.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 -- 172.21.15.31:0/1709483310 wait complete. 2023-11-26T07:22:50.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 Processor -- start 2023-11-26T07:22:50.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 -- start start 2023-11-26T07:22:50.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f401006dfd0 0x7f401009b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40100adcc0 0x7f4010156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4010143ae0 0x7f4010138590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401009a940 con 0x7f4010143ae0 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401009ac20 con 0x7f401006dfd0 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4016580640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401009af00 con 0x7f40100adcc0 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4014b1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4010143ae0 0x7f4010138590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4014b1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4010143ae0 0x7f4010138590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41880/0 (socket says 172.21.15.31:41880) 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4014b1d640 1 -- 172.21.15.31:0/1873913909 learned_addr learned my addr 172.21.15.31:0/1873913909 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f400ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f401006dfd0 0x7f401009b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f400f7fe640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40100adcc0 0x7f4010156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4014b1d640 1 -- 172.21.15.31:0/1873913909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40100adcc0 msgr2=0x7f4010156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4014b1d640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40100adcc0 0x7f4010156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4014b1d640 1 -- 172.21.15.31:0/1873913909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f401006dfd0 msgr2=0x7f401009b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4014b1d640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f401006dfd0 0x7f401009b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4014b1d640 1 -- 172.21.15.31:0/1873913909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400400b6c0 con 0x7f4010143ae0 2023-11-26T07:22:50.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.797+0000 7f4014b1d640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4010143ae0 0x7f4010138590 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f400800cbc0 tx=0x7f400800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f400cff9640 1 -- 172.21.15.31:0/1873913909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400800ad40 con 0x7f4010143ae0 2023-11-26T07:22:50.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f400cff9640 1 -- 172.21.15.31:0/1873913909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4008012400 con 0x7f4010143ae0 2023-11-26T07:22:50.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f400cff9640 1 -- 172.21.15.31:0/1873913909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400800aea0 con 0x7f4010143ae0 2023-11-26T07:22:50.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f4016580640 1 -- 172.21.15.31:0/1873913909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401009b1e0 con 0x7f4010143ae0 2023-11-26T07:22:50.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f4016580640 1 -- 172.21.15.31:0/1873913909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4010138ad0 con 0x7f4010143ae0 2023-11-26T07:22:50.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f4016580640 1 -- 172.21.15.31:0/1873913909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401004e850 con 0x7f4010143ae0 2023-11-26T07:22:50.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f400cff9640 1 -- 172.21.15.31:0/1873913909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f4008007070 con 0x7f4010143ae0 2023-11-26T07:22:50.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f400cff9640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3ffc066590 0x7f3ffc068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:50.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f400cff9640 1 -- 172.21.15.31:0/1873913909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4008083ec0 con 0x7f4010143ae0 2023-11-26T07:22:50.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f400ffff640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3ffc066590 0x7f3ffc068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:50.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.801+0000 7f400ffff640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3ffc066590 0x7f3ffc068a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4000007fd0 tx=0x7f4000009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:50.808 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.805+0000 7f400cff9640 1 -- 172.21.15.31:0/1873913909 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4008051270 con 0x7f4010143ae0 2023-11-26T07:22:50.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.965+0000 7f4016580640 1 -- 172.21.15.31:0/1873913909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f401009b690 con 0x7f4010143ae0 2023-11-26T07:22:50.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.965+0000 7f400cff9640 1 -- 172.21.15.31:0/1873913909 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16603 (secure 0 0 0) 0x7f4008017090 con 0x7f4010143ae0 2023-11-26T07:22:50.969 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:50.969 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:44.288769+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:20.941587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:21.520554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/683007803":"2023-11-27T07:22:41.280769+0000","172.21.15.97:6833/683007803":"2023-11-27T07:22:41.280769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 -- 172.21.15.31:0/1873913909 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3ffc066590 msgr2=0x7f3ffc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3ffc066590 0x7f3ffc068a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4000007fd0 tx=0x7f4000009040 comp rx=0 tx=0).stop 2023-11-26T07:22:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 -- 172.21.15.31:0/1873913909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4010143ae0 msgr2=0x7f4010138590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4010143ae0 0x7f4010138590 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f400800cbc0 tx=0x7f400800a430 comp rx=0 tx=0).stop 2023-11-26T07:22:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 -- 172.21.15.31:0/1873913909 shutdown_connections 2023-11-26T07:22:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3ffc066590 0x7f3ffc068a50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4010143ae0 0x7f4010138590 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f40100adcc0 0x7f4010156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 --2- 172.21.15.31:0/1873913909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f401006dfd0 0x7f401009b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:50.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 -- 172.21.15.31:0/1873913909 >> 172.21.15.31:0/1873913909 conn(0x7f40100b0c40 msgr2=0x7f40100b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 -- 172.21.15.31:0/1873913909 shutdown_connections 2023-11-26T07:22:50.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:50.969+0000 7f4016580640 1 -- 172.21.15.31:0/1873913909 wait complete. 2023-11-26T07:22:50.990 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:22:50.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:51.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.129+0000 7f3e1d89c640 1 Processor -- start 2023-11-26T07:22:51.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.129+0000 7f3e1d89c640 1 -- start start 2023-11-26T07:22:51.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e1d89c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18070730 0x7f3e1806bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e1d89c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3e18147040 0x7f3e1806c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e1d89c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e1815dba0 con 0x7f3e18146c60 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e1d89c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e1806d100 con 0x7f3e18147040 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e1d89c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1806c590 con 0x7f3e18070730 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e16ffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f3e18146c60 0x7f3e18070020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59610/0 (socket says 172.21.15.31:59610) 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e16ffd640 1 -- 172.21.15.31:0/4273044189 learned_addr learned my addr 172.21.15.31:0/4273044189 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e0ffff640 1 --2- 172.21.15.31:0/4273044189 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18070730 0x7f3e1806bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e177fe640 1 --2- 172.21.15.31:0/4273044189 >> v2:172.21.15.97:3300/0 conn(0x7f3e18147040 0x7f3e1806c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e167fc640 1 -- 172.21.15.31:0/4273044189 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278260273 0 0) 0x7f3e1815dba0 con 0x7f3e18146c60 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e167fc640 1 -- 172.21.15.31:0/4273044189 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e04003530 con 0x7f3e18146c60 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e0ffff640 1 -- 172.21.15.31:0/4273044189 >> v2:172.21.15.97:3300/0 conn(0x7f3e18147040 msgr2=0x7f3e1806c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:51.550 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e0ffff640 1 --2- 172.21.15.31:0/4273044189 >> v2:172.21.15.97:3300/0 conn(0x7f3e18147040 0x7f3e1806c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e0ffff640 1 -- 172.21.15.31:0/4273044189 >> v1:172.21.15.31:6789/0 conn(0x7f3e18146c60 legacy=0x7f3e18070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.133+0000 7f3e0ffff640 1 -- 172.21.15.31:0/4273044189 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1806c860 con 0x7f3e18070730 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e0ffff640 1 --2- 172.21.15.31:0/4273044189 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18070730 0x7f3e1806bb10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3e000097c0 tx=0x7f3e0001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=921c70a949e1aeea server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e167fc640 1 -- 172.21.15.31:0/4273044189 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e000ab650 con 0x7f3e18070730 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e167fc640 1 -- 172.21.15.31:0/4273044189 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e000ab7b0 con 0x7f3e18070730 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e167fc640 1 -- 172.21.15.31:0/4273044189 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e000abb40 con 0x7f3e18070730 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e1d89c640 1 -- 172.21.15.31:0/4273044189 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18070730 msgr2=0x7f3e1806bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e1d89c640 1 --2- 172.21.15.31:0/4273044189 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18070730 0x7f3e1806bb10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3e000097c0 tx=0x7f3e0001c450 comp rx=0 tx=0).stop 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e1d89c640 1 -- 172.21.15.31:0/4273044189 shutdown_connections 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e1d89c640 1 --2- 172.21.15.31:0/4273044189 >> v2:172.21.15.97:3300/0 conn(0x7f3e18147040 0x7f3e1806c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e1d89c640 1 --2- 172.21.15.31:0/4273044189 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18070730 0x7f3e1806bb10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e1d89c640 1 -- 172.21.15.31:0/4273044189 >> 172.21.15.31:0/4273044189 conn(0x7f3e180b2ea0 msgr2=0x7f3e180b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:51.551 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e1d89c640 1 -- 172.21.15.31:0/4273044189 shutdown_connections 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e1d89c640 1 -- 172.21.15.31:0/4273044189 wait complete. 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.137+0000 7f3e1d89c640 1 Processor -- start 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e1d89c640 1 -- start start 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e1d89c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e18070730 0x7f3e180ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e1d89c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18146c60 0x7f3e180ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e1d89c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e18147040 0x7f3e180ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e1d89c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e181439f0 con 0x7f3e18147040 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e1d89c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18143430 con 0x7f3e18070730 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e1d89c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18143710 con 0x7f3e18146c60 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e18147040 0x7f3e180ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e177fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e18147040 0x7f3e180ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41894/0 (socket says 172.21.15.31:41894) 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e177fe640 1 -- 172.21.15.31:0/3265133433 learned_addr learned my addr 172.21.15.31:0/3265133433 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e0ffff640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18146c60 0x7f3e180ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e16ffd640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e18070730 0x7f3e180ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e177fe640 1 -- 172.21.15.31:0/3265133433 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18146c60 msgr2=0x7f3e180ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e177fe640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18146c60 0x7f3e180ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e177fe640 1 -- 172.21.15.31:0/3265133433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e18070730 msgr2=0x7f3e180ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e177fe640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e18070730 0x7f3e180ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e177fe640 1 -- 172.21.15.31:0/3265133433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e00009420 con 0x7f3e18147040 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e0ffff640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18146c60 0x7f3e180ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e177fe640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e18147040 0x7f3e180ce5f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000efc0 tx=0x7f3e1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e14ff9640 1 -- 172.21.15.31:0/3265133433 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10009ba0 con 0x7f3e18147040 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e14ff9640 1 -- 172.21.15.31:0/3265133433 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1000f040 con 0x7f3e18147040 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e1d89c640 1 -- 172.21.15.31:0/3265133433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1813f2f0 con 0x7f3e18147040 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e14ff9640 1 -- 172.21.15.31:0/3265133433 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e100146c0 con 0x7f3e18147040 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e1d89c640 1 -- 172.21.15.31:0/3265133433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e180cb280 con 0x7f3e18147040 2023-11-26T07:22:51.553 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.141+0000 7f3e1d89c640 1 -- 172.21.15.31:0/3265133433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1804e850 con 0x7f3e18147040 2023-11-26T07:22:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.145+0000 7f3e14ff9640 1 -- 172.21.15.31:0/3265133433 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f3e10014820 con 0x7f3e18147040 2023-11-26T07:22:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.145+0000 7f3e14ff9640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3df8066590 0x7f3df8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.145+0000 7f3e14ff9640 1 -- 172.21.15.31:0/3265133433 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3e10084ba0 con 0x7f3e18147040 2023-11-26T07:22:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.145+0000 7f3e16ffd640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3df8066590 0x7f3df8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.145+0000 7f3e16ffd640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3df8066590 0x7f3df8068a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005890 tx=0x7f3e08009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.149+0000 7f3e14ff9640 1 -- 172.21.15.31:0/3265133433 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e10051e30 con 0x7f3e18147040 2023-11-26T07:22:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.337+0000 7f3e1d89c640 1 -- 172.21.15.31:0/3265133433 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e180d1f60 con 0x7f3e18147040 2023-11-26T07:22:51.554 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.337+0000 7f3e14ff9640 1 -- 172.21.15.31:0/3265133433 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f3e10018090 con 0x7f3e18147040 2023-11-26T07:22:51.554 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:51.555 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1684629063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1684629063},{"type":"v1","addr":"172.21.15.31:6833","nonce":1684629063}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/1262063204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1262063204},{"type":"v1","addr":"172.21.15.97:6834","nonce":1262063204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2863533780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2863533780},{"type":"v1","addr":"172.21.15.97:6835","nonce":2863533780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:22.526776+0000","modified":"2023-11-26T07:22:25.559499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7101":{"gid":7101,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3729395637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3729395637},{"type":"v1","addr":"172.21.15.31:6835","nonce":3729395637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:22:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 63 2023-11-26T07:22:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 -- 172.21.15.31:0/3265133433 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3df8066590 msgr2=0x7f3df8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3df8066590 0x7f3df8068a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005890 tx=0x7f3e08009040 comp rx=0 tx=0).stop 2023-11-26T07:22:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 -- 172.21.15.31:0/3265133433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e18147040 msgr2=0x7f3e180ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e18147040 0x7f3e180ce5f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000efc0 tx=0x7f3e1000c490 comp rx=0 tx=0).stop 2023-11-26T07:22:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 -- 172.21.15.31:0/3265133433 shutdown_connections 2023-11-26T07:22:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3df8066590 0x7f3df8068a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.556 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e18147040 0x7f3e180ce5f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3e18146c60 0x7f3e180ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 --2- 172.21.15.31:0/3265133433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e18070730 0x7f3e180ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 -- 172.21.15.31:0/3265133433 >> 172.21.15.31:0/3265133433 conn(0x7f3e180b2ea0 msgr2=0x7f3e181449f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:51.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 -- 172.21.15.31:0/3265133433 shutdown_connections 2023-11-26T07:22:51.557 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.341+0000 7f3e1d89c640 1 -- 172.21.15.31:0/3265133433 wait complete. 2023-11-26T07:22:51.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:51.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.697+0000 7f9dc8d2a640 1 Processor -- start 2023-11-26T07:22:51.701 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.697+0000 7f9dc8d2a640 1 -- start start 2023-11-26T07:22:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc8d2a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9dc4143ae0 0x7f9dc40b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc8d2a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc40ae7f0 con 0x7f9dc406dfd0 2023-11-26T07:22:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc8d2a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc40b2480 con 0x7f9dc40ae3d0 2023-11-26T07:22:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc8d2a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc4070670 con 0x7f9dc4143ae0 2023-11-26T07:22:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dbbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9dc406dfd0 0x7f9dc40adcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59620/0 (socket says 172.21.15.31:59620) 2023-11-26T07:22:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dbbfff640 1 -- 172.21.15.31:0/1635228516 learned_addr learned my addr 172.21.15.31:0/1635228516 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc259c640 1 --2- 172.21.15.31:0/1635228516 >> v2:172.21.15.97:3300/0 conn(0x7f9dc4143ae0 0x7f9dc40b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703415829 0 0) 0x7f9dc40ae7f0 con 0x7f9dc406dfd0 2023-11-26T07:22:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9db0003530 con 0x7f9dc406dfd0 2023-11-26T07:22:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2864960303 0 0) 0x7f9db0003530 con 0x7f9dc406dfd0 2023-11-26T07:22:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 >> v2:172.21.15.97:3300/0 conn(0x7f9dc4143ae0 msgr2=0x7f9dc40b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 --2- 172.21.15.31:0/1635228516 >> v2:172.21.15.97:3300/0 conn(0x7f9dc4143ae0 0x7f9dc40b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 >> v1:172.21.15.97:6789/0 conn(0x7f9dc40ae3d0 legacy=0x7f9dc4071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4070950 con 0x7f9dc406dfd0 2023-11-26T07:22:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f9dac001680 con 0x7f9dc406dfd0 2023-11-26T07:22:51.705 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 >> v1:172.21.15.31:6789/0 conn(0x7f9dc406dfd0 legacy=0x7f9dc40adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9db0003980 0x7f9db0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9db0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9db0005050 0x7f9db000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4070670 con 0x7f9db0004380 2023-11-26T07:22:51.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db000e1c0 con 0x7f9db0003980 2023-11-26T07:22:51.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db000e380 con 0x7f9db0005050 2023-11-26T07:22:51.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dbbfff640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9db0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc259c640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9db0003980 0x7f9db0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc2d9d640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9db0005050 0x7f9db000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.706 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dbbfff640 1 -- 172.21.15.31:0/1635228516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9db0005050 msgr2=0x7f9db000dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:51.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dbbfff640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9db0005050 0x7f9db000dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dbbfff640 1 -- 172.21.15.31:0/1635228516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9db0003980 msgr2=0x7f9db0003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:51.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dbbfff640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9db0003980 0x7f9db0003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dbbfff640 1 -- 172.21.15.31:0/1635228516 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db0003530 con 0x7f9db0004380 2023-11-26T07:22:51.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.701+0000 7f9dc2d9d640 1 -- 172.21.15.31:0/1635228516 reap_dead start 2023-11-26T07:22:51.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dbbfff640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9db0004850 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00c190 tx=0x7f9dac01cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531f192a70df61a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:51.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac005ae0 con 0x7f9db0004380 2023-11-26T07:22:51.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac005c60 con 0x7f9db0004380 2023-11-26T07:22:51.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dc1d9b640 1 -- 172.21.15.31:0/1635228516 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0a59d0 con 0x7f9db0004380 2023-11-26T07:22:51.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/1635228516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 msgr2=0x7f9db0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.708 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dc8d2a640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9db0004850 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9dac00c190 tx=0x7f9dac01cb10 comp rx=0 tx=0).stop 2023-11-26T07:22:51.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/1635228516 shutdown_connections 2023-11-26T07:22:51.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dc8d2a640 1 --2- 172.21.15.31:0/1635228516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9db0004850 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/1635228516 >> 172.21.15.31:0/1635228516 conn(0x7f9dc40b0c40 msgr2=0x7f9dc40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:51.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/1635228516 shutdown_connections 2023-11-26T07:22:51.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.705+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/1635228516 wait complete. 2023-11-26T07:22:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 Processor -- start 2023-11-26T07:22:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 -- start start 2023-11-26T07:22:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9dc40cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc40aea50 0x7f9dc40aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc415d000 0x7f9dc415d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40af3f0 con 0x7f9db0004380 2023-11-26T07:22:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4099b60 con 0x7f9dc40aea50 2023-11-26T07:22:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4099e40 con 0x7f9dc415d000 2023-11-26T07:22:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9dc40cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9dc40cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41914/0 (socket says 172.21.15.31:41914) 2023-11-26T07:22:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc259c640 1 -- 172.21.15.31:0/2541494075 learned_addr learned my addr 172.21.15.31:0/2541494075 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:51.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc2d9d640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc415d000 0x7f9dc415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dbbfff640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc40aea50 0x7f9dc40aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc259c640 1 -- 172.21.15.31:0/2541494075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc415d000 msgr2=0x7f9dc415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc259c640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc415d000 0x7f9dc415d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc259c640 1 -- 172.21.15.31:0/2541494075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc40aea50 msgr2=0x7f9dc40aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc259c640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc40aea50 0x7f9dc40aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc259c640 1 -- 172.21.15.31:0/2541494075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc409a120 con 0x7f9db0004380 2023-11-26T07:22:51.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc259c640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9dc40cd930 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9db40119d0 tx=0x7f9db4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:51.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dbaffd640 1 -- 172.21.15.31:0/2541494075 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db400fb50 con 0x7f9db0004380 2023-11-26T07:22:51.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dbaffd640 1 -- 172.21.15.31:0/2541494075 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db400fcb0 con 0x7f9db0004380 2023-11-26T07:22:51.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dbaffd640 1 -- 172.21.15.31:0/2541494075 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4014440 con 0x7f9db0004380 2023-11-26T07:22:51.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/2541494075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc409a400 con 0x7f9db0004380 2023-11-26T07:22:51.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/2541494075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc40cde70 con 0x7f9db0004380 2023-11-26T07:22:51.714 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.709+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/2541494075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc404e850 con 0x7f9db0004380 2023-11-26T07:22:51.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.713+0000 7f9dbaffd640 1 -- 172.21.15.31:0/2541494075 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f9db40145a0 con 0x7f9db0004380 2023-11-26T07:22:51.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.713+0000 7f9dbaffd640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9da4066590 0x7f9da4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:51.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.713+0000 7f9dbaffd640 1 -- 172.21.15.31:0/2541494075 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9db40848a0 con 0x7f9db0004380 2023-11-26T07:22:51.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.713+0000 7f9dbbfff640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9da4066590 0x7f9da4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:51.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.713+0000 7f9dbbfff640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9da4066590 0x7f9da4068a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0ab000 tx=0x7f9dac0b1040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.717+0000 7f9dbaffd640 1 -- 172.21.15.31:0/2541494075 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9db4051d00 con 0x7f9db0004380 2023-11-26T07:22:51.909 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.905+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/2541494075 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dc40b3920 con 0x7f9db0004380 2023-11-26T07:22:51.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dbaffd640 1 -- 172.21.15.31:0/2541494075 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f9db4022020 con 0x7f9db0004380 2023-11-26T07:22:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:51.910 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1684629063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1684629063},{"type":"v1","addr":"172.21.15.31:6833","nonce":1684629063}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/1262063204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1262063204},{"type":"v1","addr":"172.21.15.97:6834","nonce":1262063204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2863533780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2863533780},{"type":"v1","addr":"172.21.15.97:6835","nonce":2863533780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:22:22.526776+0000","modified":"2023-11-26T07:22:25.559499+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7101},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7101":{"gid":7101,"name":"c","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/3729395637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":3729395637},{"type":"v1","addr":"172.21.15.31:6835","nonce":3729395637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-26T07:22:51.911 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 63 2023-11-26T07:22:51.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/2541494075 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9da4066590 msgr2=0x7f9da4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.912 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9da4066590 0x7f9da4068a50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0ab000 tx=0x7f9dac0b1040 comp rx=0 tx=0).stop 2023-11-26T07:22:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/2541494075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 msgr2=0x7f9dc40cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9dc40cd930 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f9db40119d0 tx=0x7f9db4011e90 comp rx=0 tx=0).stop 2023-11-26T07:22:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/2541494075 shutdown_connections 2023-11-26T07:22:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9da4066590 0x7f9da4068a50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9dc415d000 0x7f9dc415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9dc40aea50 0x7f9dc40aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 --2- 172.21.15.31:0/2541494075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0004380 0x7f9dc40cd930 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/2541494075 >> 172.21.15.31:0/2541494075 conn(0x7f9dc40b0c40 msgr2=0x7f9dc415e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/2541494075 shutdown_connections 2023-11-26T07:22:51.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:51.909+0000 7f9dc8d2a640 1 -- 172.21.15.31:0/2541494075 wait complete. 2023-11-26T07:22:51.932 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:52.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fde74640 1 Processor -- start 2023-11-26T07:22:52.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fde74640 1 -- start start 2023-11-26T07:22:52.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fde74640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f69f80d3760 0x7f69f80d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fde74640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f69f80cfa20 0x7f69f80cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fde74640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f80d15f0 con 0x7f69f815d880 2023-11-26T07:22:52.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fde74640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f80d4110 con 0x7f69f80cfa20 2023-11-26T07:22:52.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fde74640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f80d0390 con 0x7f69f80d3760 2023-11-26T07:22:52.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fce72640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f69f815d880 0x7f69f80b6730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59626/0 (socket says 172.21.15.31:59626) 2023-11-26T07:22:52.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fce72640 1 -- 172.21.15.31:0/1334491022 learned_addr learned my addr 172.21.15.31:0/1334491022 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:52.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fd673640 1 --2- 172.21.15.31:0/1334491022 >> v2:172.21.15.31:3300/0 conn(0x7f69f80cfa20 0x7f69f80cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69f7fff640 1 --2- 172.21.15.31:0/1334491022 >> v2:172.21.15.97:3300/0 conn(0x7f69f80d3760 0x7f69f80d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fd673640 1 -- 172.21.15.31:0/1334491022 >> v2:172.21.15.97:3300/0 conn(0x7f69f80d3760 msgr2=0x7f69f80d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:52.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fd673640 1 --2- 172.21.15.31:0/1334491022 >> v2:172.21.15.97:3300/0 conn(0x7f69f80d3760 0x7f69f80d3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fd673640 1 -- 172.21.15.31:0/1334491022 >> v1:172.21.15.31:6789/0 conn(0x7f69f815d880 legacy=0x7f69f80b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.073+0000 7f69fd673640 1 -- 172.21.15.31:0/1334491022 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f80ce000 con 0x7f69f80cfa20 2023-11-26T07:22:52.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69fd673640 1 --2- 172.21.15.31:0/1334491022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80cfa20 0x7f69f80cfe50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f69f80cd850 tx=0x7f69f001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5378d5a57642ce8f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69f77fe640 1 -- 172.21.15.31:0/1334491022 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f00a59d0 con 0x7f69f80cfa20 2023-11-26T07:22:52.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69f77fe640 1 -- 172.21.15.31:0/1334491022 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f00a5b30 con 0x7f69f80cfa20 2023-11-26T07:22:52.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69f77fe640 1 -- 172.21.15.31:0/1334491022 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f00ad440 con 0x7f69f80cfa20 2023-11-26T07:22:52.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69fde74640 1 -- 172.21.15.31:0/1334491022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80cfa20 msgr2=0x7f69f80cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69fde74640 1 --2- 172.21.15.31:0/1334491022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80cfa20 0x7f69f80cfe50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f69f80cd850 tx=0x7f69f001c910 comp rx=0 tx=0).stop 2023-11-26T07:22:52.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69fde74640 1 -- 172.21.15.31:0/1334491022 shutdown_connections 2023-11-26T07:22:52.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69fde74640 1 --2- 172.21.15.31:0/1334491022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80cfa20 0x7f69f80cfe50 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69fde74640 1 --2- 172.21.15.31:0/1334491022 >> v2:172.21.15.97:3300/0 conn(0x7f69f80d3760 0x7f69f80d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69fde74640 1 -- 172.21.15.31:0/1334491022 >> 172.21.15.31:0/1334491022 conn(0x7f69f806b9f0 msgr2=0x7f69f806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:52.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69fde74640 1 -- 172.21.15.31:0/1334491022 shutdown_connections 2023-11-26T07:22:52.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.077+0000 7f69fde74640 1 -- 172.21.15.31:0/1334491022 wait complete. 2023-11-26T07:22:52.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fde74640 1 Processor -- start 2023-11-26T07:22:52.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fde74640 1 -- start start 2023-11-26T07:22:52.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fde74640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f69f80cfa20 0x7f69f81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fde74640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80d3760 0x7f69f80b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fde74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f815d880 0x7f69f815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fde74640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8156950 con 0x7f69f80d3760 2023-11-26T07:22:52.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fde74640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80cb440 con 0x7f69f815d880 2023-11-26T07:22:52.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fde74640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8155720 con 0x7f69f80cfa20 2023-11-26T07:22:52.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80d3760 0x7f69f80b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80d3760 0x7f69f80b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41928/0 (socket says 172.21.15.31:41928) 2023-11-26T07:22:52.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f7fff640 1 -- 172.21.15.31:0/715177834 learned_addr learned my addr 172.21.15.31:0/715177834 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:52.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fd673640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f815d880 0x7f69f815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fce72640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f69f80cfa20 0x7f69f81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f7fff640 1 -- 172.21.15.31:0/715177834 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f69f80cfa20 msgr2=0x7f69f81550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f7fff640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f69f80cfa20 0x7f69f81550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f7fff640 1 -- 172.21.15.31:0/715177834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f815d880 msgr2=0x7f69f815be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f7fff640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f815d880 0x7f69f815be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f7fff640 1 -- 172.21.15.31:0/715177834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0009a70 con 0x7f69f80d3760 2023-11-26T07:22:52.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f7fff640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80d3760 0x7f69f80b4d50 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f69e400e930 tx=0x7f69e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f57fa640 1 -- 172.21.15.31:0/715177834 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e4009690 con 0x7f69f80d3760 2023-11-26T07:22:52.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f57fa640 1 -- 172.21.15.31:0/715177834 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e400ca00 con 0x7f69f80d3760 2023-11-26T07:22:52.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fde74640 1 -- 172.21.15.31:0/715177834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f8155a00 con 0x7f69f80d3760 2023-11-26T07:22:52.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69f57fa640 1 -- 172.21.15.31:0/715177834 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e400cd70 con 0x7f69f80d3760 2023-11-26T07:22:52.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.081+0000 7f69fde74640 1 -- 172.21.15.31:0/715177834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f8155fc0 con 0x7f69f80d3760 2023-11-26T07:22:52.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.085+0000 7f69fde74640 1 -- 172.21.15.31:0/715177834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f8074a00 con 0x7f69f80d3760 2023-11-26T07:22:52.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.085+0000 7f69f57fa640 1 -- 172.21.15.31:0/715177834 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f69e4013470 con 0x7f69f80d3760 2023-11-26T07:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.085+0000 7f69f57fa640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f69dc066590 0x7f69dc068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.085+0000 7f69f57fa640 1 -- 172.21.15.31:0/715177834 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f69e40841a0 con 0x7f69f80d3760 2023-11-26T07:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.085+0000 7f69fce72640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f69dc066590 0x7f69dc068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.085+0000 7f69fce72640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f69dc066590 0x7f69dc068a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69ec005890 tx=0x7f69ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.089+0000 7f69f57fa640 1 -- 172.21.15.31:0/715177834 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f69e40516b0 con 0x7f69f80d3760 2023-11-26T07:22:52.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.249+0000 7f69fde74640 1 -- 172.21.15.31:0/715177834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69f80ce4f0 con 0x7f69f80d3760 2023-11-26T07:22:52.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.249+0000 7f69f57fa640 1 -- 172.21.15.31:0/715177834 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16603 (secure 0 0 0) 0x7f69e4054cd0 con 0x7f69f80d3760 2023-11-26T07:22:52.253 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:52.253 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":88,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:44.288769+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:22:20.941587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-26T07:22:21.520554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:6835/683007803":"2023-11-27T07:22:41.280769+0000","172.21.15.97:6833/683007803":"2023-11-27T07:22:41.280769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 -- 172.21.15.31:0/715177834 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f69dc066590 msgr2=0x7f69dc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f69dc066590 0x7f69dc068a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69ec005890 tx=0x7f69ec009040 comp rx=0 tx=0).stop 2023-11-26T07:22:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 -- 172.21.15.31:0/715177834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80d3760 msgr2=0x7f69f80b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80d3760 0x7f69f80b4d50 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f69e400e930 tx=0x7f69e400edf0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 -- 172.21.15.31:0/715177834 shutdown_connections 2023-11-26T07:22:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f69dc066590 0x7f69dc068a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69f815d880 0x7f69f815be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69f80d3760 0x7f69f80b4d50 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 --2- 172.21.15.31:0/715177834 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f69f80cfa20 0x7f69f81550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 -- 172.21.15.31:0/715177834 >> 172.21.15.31:0/715177834 conn(0x7f69f806b9f0 msgr2=0x7f69f815dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:52.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 -- 172.21.15.31:0/715177834 shutdown_connections 2023-11-26T07:22:52.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.253+0000 7f69fde74640 1 -- 172.21.15.31:0/715177834 wait complete. 2023-11-26T07:22:52.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:22:52.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.417+0000 7f4306c9c640 1 Processor -- start 2023-11-26T07:22:52.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.417+0000 7f4306c9c640 1 -- start start 2023-11-26T07:22:52.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.417+0000 7f4306c9c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f430006e3f0 0x7f430015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.417+0000 7f4306c9c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f43000ce5d0 0x7f430015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.417+0000 7f4306c9c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4300159630 con 0x7f43000cb9b0 2023-11-26T07:22:52.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.417+0000 7f4306c9c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430006e860 con 0x7f430006e3f0 2023-11-26T07:22:52.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.417+0000 7f4306c9c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430015bba0 con 0x7f43000ce5d0 2023-11-26T07:22:52.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.417+0000 7f42fffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f430006e3f0 0x7f430015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.417+0000 7f42fffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f430006e3f0 0x7f430015f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41942/0 (socket says 172.21.15.31:41942) 2023-11-26T07:22:52.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.417+0000 7f42fffff640 1 -- 172.21.15.31:0/1928128294 learned_addr learned my addr 172.21.15.31:0/1928128294 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:52.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.421+0000 7f4305239640 1 --2- 172.21.15.31:0/1928128294 >> v2:172.21.15.97:3300/0 conn(0x7f43000ce5d0 0x7f430015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.421+0000 7f42fffff640 1 -- 172.21.15.31:0/1928128294 >> v2:172.21.15.97:3300/0 conn(0x7f43000ce5d0 msgr2=0x7f430015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:52.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.421+0000 7f42fffff640 1 --2- 172.21.15.31:0/1928128294 >> v2:172.21.15.97:3300/0 conn(0x7f43000ce5d0 0x7f430015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.421+0000 7f42fffff640 1 -- 172.21.15.31:0/1928128294 >> v1:172.21.15.97:6789/0 conn(0x7f43000cb9b0 legacy=0x7f43000cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:52.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.421+0000 7f42fffff640 1 -- 172.21.15.31:0/1928128294 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430015f6d0 con 0x7f430006e3f0 2023-11-26T07:22:52.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.421+0000 7f4305239640 1 --2- 172.21.15.31:0/1928128294 >> v2:172.21.15.97:3300/0 conn(0x7f43000ce5d0 0x7f430015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.421+0000 7f42fffff640 1 --2- 172.21.15.31:0/1928128294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f430006e3f0 0x7f430015f190 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f43000ae970 tx=0x7f42ec01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1796ceabfd5a1b91 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.421+0000 7f42ff7fe640 1 -- 172.21.15.31:0/1928128294 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0ad640 con 0x7f430006e3f0 2023-11-26T07:22:52.425 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.421+0000 7f42ff7fe640 1 -- 172.21.15.31:0/1928128294 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec0ad7a0 con 0x7f430006e3f0 2023-11-26T07:22:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.421+0000 7f42ff7fe640 1 -- 172.21.15.31:0/1928128294 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0adb30 con 0x7f430006e3f0 2023-11-26T07:22:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 -- 172.21.15.31:0/1928128294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f430006e3f0 msgr2=0x7f430015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 --2- 172.21.15.31:0/1928128294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f430006e3f0 0x7f430015f190 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f43000ae970 tx=0x7f42ec01c4c0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 -- 172.21.15.31:0/1928128294 shutdown_connections 2023-11-26T07:22:52.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 --2- 172.21.15.31:0/1928128294 >> v2:172.21.15.97:3300/0 conn(0x7f43000ce5d0 0x7f430015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 --2- 172.21.15.31:0/1928128294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f430006e3f0 0x7f430015f190 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 -- 172.21.15.31:0/1928128294 >> 172.21.15.31:0/1928128294 conn(0x7f43000b0cb0 msgr2=0x7f43000b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 -- 172.21.15.31:0/1928128294 shutdown_connections 2023-11-26T07:22:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 -- 172.21.15.31:0/1928128294 wait complete. 2023-11-26T07:22:52.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 Processor -- start 2023-11-26T07:22:52.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 -- start start 2023-11-26T07:22:52.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430006e3f0 0x7f4300165d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43000cb9b0 0x7f43000b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43000ce5d0 0x7f430015d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4300156f80 con 0x7f43000cb9b0 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4300153130 con 0x7f430006e3f0 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4306c9c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43000ada70 con 0x7f43000ce5d0 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f42fffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43000cb9b0 0x7f43000b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f42fffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43000cb9b0 0x7f43000b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41954/0 (socket says 172.21.15.31:41954) 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f42fffff640 1 -- 172.21.15.31:0/3241779493 learned_addr learned my addr 172.21.15.31:0/3241779493 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4304a38640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430006e3f0 0x7f4300165d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4305239640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43000ce5d0 0x7f430015d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f42fffff640 1 -- 172.21.15.31:0/3241779493 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43000ce5d0 msgr2=0x7f430015d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f42fffff640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43000ce5d0 0x7f430015d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f42fffff640 1 -- 172.21.15.31:0/3241779493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430006e3f0 msgr2=0x7f4300165d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f42fffff640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430006e3f0 0x7f4300165d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f42fffff640 1 -- 172.21.15.31:0/3241779493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0094d0 con 0x7f43000cb9b0 2023-11-26T07:22:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f4304a38640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430006e3f0 0x7f4300165d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:52.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.425+0000 7f42fffff640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43000cb9b0 0x7f43000b20b0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0b0040 tx=0x7f42ec019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.429+0000 7f42fd7fa640 1 -- 172.21.15.31:0/3241779493 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0b6830 con 0x7f43000cb9b0 2023-11-26T07:22:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.429+0000 7f42fd7fa640 1 -- 172.21.15.31:0/3241779493 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec0a6040 con 0x7f43000cb9b0 2023-11-26T07:22:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.429+0000 7f42fd7fa640 1 -- 172.21.15.31:0/3241779493 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0b6bf0 con 0x7f43000cb9b0 2023-11-26T07:22:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.429+0000 7f4306c9c640 1 -- 172.21.15.31:0/3241779493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4300156ca0 con 0x7f43000cb9b0 2023-11-26T07:22:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.429+0000 7f4306c9c640 1 -- 172.21.15.31:0/3241779493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4300153790 con 0x7f43000cb9b0 2023-11-26T07:22:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.429+0000 7f4306c9c640 1 -- 172.21.15.31:0/3241779493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43000671f0 con 0x7f43000cb9b0 2023-11-26T07:22:52.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.429+0000 7f42fd7fa640 1 -- 172.21.15.31:0/3241779493 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f42ec0b4070 con 0x7f43000cb9b0 2023-11-26T07:22:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.429+0000 7f42fd7fa640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f42e4066590 0x7f42e4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.429+0000 7f42fd7fa640 1 -- 172.21.15.31:0/3241779493 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f42ec11eac0 con 0x7f43000cb9b0 2023-11-26T07:22:52.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.429+0000 7f4304a38640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f42e4066590 0x7f42e4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.433+0000 7f4304a38640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f42e4066590 0x7f42e4068a50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42f4003ec0 tx=0x7f42f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.433+0000 7f42fd7fa640 1 -- 172.21.15.31:0/3241779493 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f42ec0ebdc0 con 0x7f43000cb9b0 2023-11-26T07:22:52.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.625+0000 7f4306c9c640 1 -- 172.21.15.31:0/3241779493 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f43000671f0 con 0x7f43000cb9b0 2023-11-26T07:22:52.630 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-11-26T07:22:52.629+0000 7fe86499d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:22:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.629+0000 7f42fd7fa640 1 -- 172.21.15.31:0/3241779493 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7f42ec0b2090 con 0x7f43000cb9b0 2023-11-26T07:22:52.632 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:22:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 -- 172.21.15.31:0/3241779493 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f42e4066590 msgr2=0x7f42e4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f42e4066590 0x7f42e4068a50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f42f4003ec0 tx=0x7f42f4009040 comp rx=0 tx=0).stop 2023-11-26T07:22:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 -- 172.21.15.31:0/3241779493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43000cb9b0 msgr2=0x7f43000b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43000cb9b0 0x7f43000b20b0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0b0040 tx=0x7f42ec019260 comp rx=0 tx=0).stop 2023-11-26T07:22:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 -- 172.21.15.31:0/3241779493 shutdown_connections 2023-11-26T07:22:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f42e4066590 0x7f42e4068a50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f43000ce5d0 0x7f430015d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43000cb9b0 0x7f43000b20b0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 --2- 172.21.15.31:0/3241779493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f430006e3f0 0x7f4300165d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 -- 172.21.15.31:0/3241779493 >> 172.21.15.31:0/3241779493 conn(0x7f43000b0cb0 msgr2=0x7f43000b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 -- 172.21.15.31:0/3241779493 shutdown_connections 2023-11-26T07:22:52.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.633+0000 7f4306c9c640 1 -- 172.21.15.31:0/3241779493 wait complete. 2023-11-26T07:22:52.641 INFO:tasks.ceph.mds.c.smithi031.stderr: -2568> 2023-11-26T07:22:25.117+0000 7fc4fb4d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:22:52.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:22:52.688 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-11-26T07:22:52.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.801+0000 7f138e216640 1 Processor -- start 2023-11-26T07:22:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.801+0000 7f138e216640 1 -- start start 2023-11-26T07:22:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.801+0000 7f138e216640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13880b4210 0x7f13880b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.805 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.801+0000 7f138e216640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f138815db40 0x7f138815df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.801+0000 7f138e216640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1388147190 con 0x7f1388146820 2023-11-26T07:22:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.801+0000 7f138e216640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1388146c00 con 0x7f138815db40 2023-11-26T07:22:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.801+0000 7f138e216640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138806bf90 con 0x7f13880b4210 2023-11-26T07:22:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.801+0000 7f13877fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1388146820 0x7f1388070a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:59644/0 (socket says 172.21.15.31:59644) 2023-11-26T07:22:52.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.801+0000 7f13877fe640 1 -- 172.21.15.31:0/2968843731 learned_addr learned my addr 172.21.15.31:0/2968843731 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:52.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f1386ffd640 1 --2- 172.21.15.31:0/2968843731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13880b4210 0x7f13880b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f1387fff640 1 --2- 172.21.15.31:0/2968843731 >> v2:172.21.15.97:3300/0 conn(0x7f138815db40 0x7f138815df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f13867fc640 1 -- 172.21.15.31:0/2968843731 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093935165 0 0) 0x7f1388147190 con 0x7f1388146820 2023-11-26T07:22:52.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f13867fc640 1 -- 172.21.15.31:0/2968843731 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1370003530 con 0x7f1388146820 2023-11-26T07:22:52.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f1386ffd640 1 -- 172.21.15.31:0/2968843731 >> v2:172.21.15.97:3300/0 conn(0x7f138815db40 msgr2=0x7f138815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:52.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f1386ffd640 1 --2- 172.21.15.31:0/2968843731 >> v2:172.21.15.97:3300/0 conn(0x7f138815db40 0x7f138815df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f1386ffd640 1 -- 172.21.15.31:0/2968843731 >> v1:172.21.15.31:6789/0 conn(0x7f1388146820 legacy=0x7f1388070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.807 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f1386ffd640 1 -- 172.21.15.31:0/2968843731 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880b4bb0 con 0x7f13880b4210 2023-11-26T07:22:52.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f1386ffd640 1 --2- 172.21.15.31:0/2968843731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13880b4210 0x7f13880b4670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f137c0097c0 tx=0x7f137c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7171d3500a0ac1bc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f13867fc640 1 -- 172.21.15.31:0/2968843731 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c0ab650 con 0x7f13880b4210 2023-11-26T07:22:52.809 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f13867fc640 1 -- 172.21.15.31:0/2968843731 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c0ab7b0 con 0x7f13880b4210 2023-11-26T07:22:52.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.805+0000 7f13867fc640 1 -- 172.21.15.31:0/2968843731 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c0abb40 con 0x7f13880b4210 2023-11-26T07:22:52.810 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 -- 172.21.15.31:0/2968843731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13880b4210 msgr2=0x7f13880b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 --2- 172.21.15.31:0/2968843731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13880b4210 0x7f13880b4670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f137c0097c0 tx=0x7f137c01c450 comp rx=0 tx=0).stop 2023-11-26T07:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 -- 172.21.15.31:0/2968843731 shutdown_connections 2023-11-26T07:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 --2- 172.21.15.31:0/2968843731 >> v2:172.21.15.97:3300/0 conn(0x7f138815db40 0x7f138815df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 --2- 172.21.15.31:0/2968843731 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13880b4210 0x7f13880b4670 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 -- 172.21.15.31:0/2968843731 >> 172.21.15.31:0/2968843731 conn(0x7f13880b2f10 msgr2=0x7f13880b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 -- 172.21.15.31:0/2968843731 shutdown_connections 2023-11-26T07:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 -- 172.21.15.31:0/2968843731 wait complete. 2023-11-26T07:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 Processor -- start 2023-11-26T07:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 -- start start 2023-11-26T07:22:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13880b4210 0x7f13880ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1388146820 0x7f138815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815db40 0x7f138813f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13881412b0 con 0x7f138815db40 2023-11-26T07:22:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1388141a50 con 0x7f13880b4210 2023-11-26T07:22:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f138e216640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138815ba10 con 0x7f1388146820 2023-11-26T07:22:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f1387fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815db40 0x7f138813f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f13877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13880b4210 0x7f13880ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.814 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f1386ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1388146820 0x7f138815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f1386ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1388146820 0x7f138815ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:35862/0 (socket says 172.21.15.31:35862) 2023-11-26T07:22:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f1387fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815db40 0x7f138813f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41974/0 (socket says 172.21.15.31:41974) 2023-11-26T07:22:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.809+0000 7f1386ffd640 1 -- 172.21.15.31:0/3526476507 learned_addr learned my addr 172.21.15.31:0/3526476507 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f1386ffd640 1 -- 172.21.15.31:0/3526476507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13880b4210 msgr2=0x7f13880ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f1386ffd640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13880b4210 0x7f13880ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f1386ffd640 1 -- 172.21.15.31:0/3526476507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815db40 msgr2=0x7f138813f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f1386ffd640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815db40 0x7f138813f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f1386ffd640 1 -- 172.21.15.31:0/3526476507 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c009420 con 0x7f1388146820 2023-11-26T07:22:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f13877fe640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13880b4210 0x7f13880ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f1387fff640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815db40 0x7f138813f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f1386ffd640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1388146820 0x7f138815ace0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f137c0ad040 tx=0x7f137c01ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f136ffff640 1 -- 172.21.15.31:0/3526476507 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c0bf610 con 0x7f1388146820 2023-11-26T07:22:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f136ffff640 1 -- 172.21.15.31:0/3526476507 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c0bf7b0 con 0x7f1388146820 2023-11-26T07:22:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f138e216640 1 -- 172.21.15.31:0/3526476507 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138806d030 con 0x7f1388146820 2023-11-26T07:22:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f136ffff640 1 -- 172.21.15.31:0/3526476507 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c0bfb60 con 0x7f1388146820 2023-11-26T07:22:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f138e216640 1 -- 172.21.15.31:0/3526476507 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1388141490 con 0x7f1388146820 2023-11-26T07:22:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f138e216640 1 -- 172.21.15.31:0/3526476507 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138804e850 con 0x7f1388146820 2023-11-26T07:22:52.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f136ffff640 1 -- 172.21.15.31:0/3526476507 <== mon.2 v2:172.21.15.97:3301/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f137c0059a0 con 0x7f1388146820 2023-11-26T07:22:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f136ffff640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1374066590 0x7f1374068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f136ffff640 1 -- 172.21.15.31:0/3526476507 <== mon.2 v2:172.21.15.97:3301/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f137c11ef90 con 0x7f1388146820 2023-11-26T07:22:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.813+0000 7f13877fe640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1374066590 0x7f1374068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:52.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.817+0000 7f13877fe640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1374066590 0x7f1374068a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f13780011f0 tx=0x7f1378007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:52.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:52.817+0000 7f136ffff640 1 -- 172.21.15.31:0/3526476507 <== mon.2 v2:172.21.15.97:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f137c0ec2e0 con 0x7f1388146820 2023-11-26T07:22:53.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.009+0000 7f138e216640 1 -- 172.21.15.31:0/3526476507 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f13880d1f00 con 0x7f1388146820 2023-11-26T07:22:53.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.637+0000 7f136ffff640 1 -- 172.21.15.31:0/3526476507 <== mon.2 v2:172.21.15.97:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7f137c0ef900 con 0x7f1388146820 2023-11-26T07:22:53.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.637+0000 7f138e216640 1 -- 172.21.15.31:0/3526476507 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1374066590 msgr2=0x7f1374068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.637+0000 7f138e216640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1374066590 0x7f1374068a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f13780011f0 tx=0x7f1378007d40 comp rx=0 tx=0).stop 2023-11-26T07:22:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.637+0000 7f138e216640 1 -- 172.21.15.31:0/3526476507 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1388146820 msgr2=0x7f138815ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.637+0000 7f138e216640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1388146820 0x7f138815ace0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f137c0ad040 tx=0x7f137c01ac90 comp rx=0 tx=0).stop 2023-11-26T07:22:53.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.637+0000 7f138e216640 1 -- 172.21.15.31:0/3526476507 shutdown_connections 2023-11-26T07:22:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.637+0000 7f138e216640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1374066590 0x7f1374068a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.637+0000 7f138e216640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f138815db40 0x7f138813f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.637+0000 7f138e216640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1388146820 0x7f138815ace0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.637+0000 7f138e216640 1 --2- 172.21.15.31:0/3526476507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13880b4210 0x7f13880ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:53.645 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.641+0000 7f138e216640 1 -- 172.21.15.31:0/3526476507 >> 172.21.15.31:0/3526476507 conn(0x7f13880b2f10 msgr2=0x7f1388070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:53.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.641+0000 7f138e216640 1 -- 172.21.15.31:0/3526476507 shutdown_connections 2023-11-26T07:22:53.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.641+0000 7f138e216640 1 -- 172.21.15.31:0/3526476507 wait complete. 2023-11-26T07:22:53.666 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:22:53.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f492e0ea640 1 Processor -- start 2023-11-26T07:22:53.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f492e0ea640 1 -- start start 2023-11-26T07:22:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f492e0ea640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f492815d8f0 0x7f492815e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f492e0ea640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f49280d0510 0x7f49280d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f492e0ea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492815ecc0 con 0x7f49280ced30 2023-11-26T07:22:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f492e0ea640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49280d4420 con 0x7f49280d0510 2023-11-26T07:22:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f492e0ea640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49280cdfb0 con 0x7f492815d8f0 2023-11-26T07:22:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f4927fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f49280d0510 0x7f49280d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f4927fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f49280d0510 0x7f49280d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44320/0 (socket says 172.21.15.31:44320) 2023-11-26T07:22:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f4927fff640 1 -- 172.21.15.31:0/1014078011 learned_addr learned my addr 172.21.15.31:0/1014078011 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f49277fe640 1 --2- 172.21.15.31:0/1014078011 >> v2:172.21.15.97:3300/0 conn(0x7f492815d8f0 0x7f492815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f4927fff640 1 -- 172.21.15.31:0/1014078011 >> v2:172.21.15.97:3300/0 conn(0x7f492815d8f0 msgr2=0x7f492815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f4927fff640 1 --2- 172.21.15.31:0/1014078011 >> v2:172.21.15.97:3300/0 conn(0x7f492815d8f0 0x7f492815e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:53.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f4927fff640 1 -- 172.21.15.31:0/1014078011 >> v1:172.21.15.97:6789/0 conn(0x7f49280ced30 legacy=0x7f49280cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:53.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.865+0000 7f4927fff640 1 -- 172.21.15.31:0/1014078011 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49280d4140 con 0x7f49280d0510 2023-11-26T07:22:53.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f4927fff640 1 --2- 172.21.15.31:0/1014078011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49280d0510 0x7f49280d1000 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4920009e50 tx=0x7f492001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7862d2b05e62fc40 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f49267fc640 1 -- 172.21.15.31:0/1014078011 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49200a5770 con 0x7f49280d0510 2023-11-26T07:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f49267fc640 1 -- 172.21.15.31:0/1014078011 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49200a58d0 con 0x7f49280d0510 2023-11-26T07:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f49267fc640 1 -- 172.21.15.31:0/1014078011 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49200a5c60 con 0x7f49280d0510 2023-11-26T07:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f492e0ea640 1 -- 172.21.15.31:0/1014078011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49280d0510 msgr2=0x7f49280d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f492e0ea640 1 --2- 172.21.15.31:0/1014078011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49280d0510 0x7f49280d1000 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4920009e50 tx=0x7f492001c9a0 comp rx=0 tx=0).stop 2023-11-26T07:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f492e0ea640 1 -- 172.21.15.31:0/1014078011 shutdown_connections 2023-11-26T07:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f492e0ea640 1 --2- 172.21.15.31:0/1014078011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49280d0510 0x7f49280d1000 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f492e0ea640 1 --2- 172.21.15.31:0/1014078011 >> v2:172.21.15.97:3300/0 conn(0x7f492815d8f0 0x7f492815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f492e0ea640 1 -- 172.21.15.31:0/1014078011 >> 172.21.15.31:0/1014078011 conn(0x7f492809c750 msgr2=0x7f49280d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f492e0ea640 1 -- 172.21.15.31:0/1014078011 shutdown_connections 2023-11-26T07:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.869+0000 7f492e0ea640 1 -- 172.21.15.31:0/1014078011 wait complete. 2023-11-26T07:22:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f492e0ea640 1 Processor -- start 2023-11-26T07:22:53.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f492e0ea640 1 -- start start 2023-11-26T07:22:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f492e0ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49280ced30 0x7f49280ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f492e0ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49280d0510 0x7f4928157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f492e0ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492815d8f0 0x7f4928151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f492e0ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4928157c60 con 0x7f492815d8f0 2023-11-26T07:22:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f492e0ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49280cb7a0 con 0x7f49280d0510 2023-11-26T07:22:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f492e0ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4928151cb0 con 0x7f49280ced30 2023-11-26T07:22:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f4927fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492815d8f0 0x7f4928151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f4927fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492815d8f0 0x7f4928151770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44328/0 (socket says 172.21.15.31:44328) 2023-11-26T07:22:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f4927fff640 1 -- 172.21.15.31:0/3600256837 learned_addr learned my addr 172.21.15.31:0/3600256837 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f49277fe640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49280ced30 0x7f49280ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f4926ffd640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49280d0510 0x7f4928157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f4927fff640 1 -- 172.21.15.31:0/3600256837 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49280ced30 msgr2=0x7f49280ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f4927fff640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49280ced30 0x7f49280ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f4927fff640 1 -- 172.21.15.31:0/3600256837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49280d0510 msgr2=0x7f4928157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f4927fff640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49280d0510 0x7f4928157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f4927fff640 1 -- 172.21.15.31:0/3600256837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4920009b30 con 0x7f492815d8f0 2023-11-26T07:22:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f4927fff640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492815d8f0 0x7f4928151770 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f49200af040 tx=0x7f4920019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:53.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f490ffff640 1 -- 172.21.15.31:0/3600256837 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49200b8670 con 0x7f492815d8f0 2023-11-26T07:22:53.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f490ffff640 1 -- 172.21.15.31:0/3600256837 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49200b87d0 con 0x7f492815d8f0 2023-11-26T07:22:53.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f490ffff640 1 -- 172.21.15.31:0/3600256837 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49200b8b80 con 0x7f492815d8f0 2023-11-26T07:22:53.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f492e0ea640 1 -- 172.21.15.31:0/3600256837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49280cab90 con 0x7f492815d8f0 2023-11-26T07:22:53.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.873+0000 7f492e0ea640 1 -- 172.21.15.31:0/3600256837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49280cb150 con 0x7f492815d8f0 2023-11-26T07:22:53.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.877+0000 7f492e0ea640 1 -- 172.21.15.31:0/3600256837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49280cb910 con 0x7f492815d8f0 2023-11-26T07:22:53.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.877+0000 7f490ffff640 1 -- 172.21.15.31:0/3600256837 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f49200b8ce0 con 0x7f492815d8f0 2023-11-26T07:22:53.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.877+0000 7f490ffff640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4914066590 0x7f4914068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:53.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.877+0000 7f490ffff640 1 -- 172.21.15.31:0/3600256837 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f492011f6b0 con 0x7f492815d8f0 2023-11-26T07:22:53.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.877+0000 7f49277fe640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4914066590 0x7f4914068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.877+0000 7f49277fe640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4914066590 0x7f4914068a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f49280cd500 tx=0x7f4918008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:53.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:53.881+0000 7f490ffff640 1 -- 172.21.15.31:0/3600256837 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f49200ec9f0 con 0x7f492815d8f0 2023-11-26T07:22:54.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.037+0000 7f492e0ea640 1 -- 172.21.15.31:0/3600256837 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f492804e850 con 0x7f492815d8f0 2023-11-26T07:22:54.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.641+0000 7f490ffff640 1 -- 172.21.15.31:0/3600256837 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 181+0+0 (secure 0 0 0) 0x7f49200f0010 con 0x7f492815d8f0 2023-11-26T07:22:54.644 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:22:54.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 -- 172.21.15.31:0/3600256837 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4914066590 msgr2=0x7f4914068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4914066590 0x7f4914068a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f49280cd500 tx=0x7f4918008040 comp rx=0 tx=0).stop 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 -- 172.21.15.31:0/3600256837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492815d8f0 msgr2=0x7f4928151770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492815d8f0 0x7f4928151770 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f49200af040 tx=0x7f4920019740 comp rx=0 tx=0).stop 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 -- 172.21.15.31:0/3600256837 shutdown_connections 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4914066590 0x7f4914068a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f492815d8f0 0x7f4928151770 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49280d0510 0x7f4928157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 --2- 172.21.15.31:0/3600256837 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f49280ced30 0x7f49280ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 -- 172.21.15.31:0/3600256837 >> 172.21.15.31:0/3600256837 conn(0x7f492809c750 msgr2=0x7f49280b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 -- 172.21.15.31:0/3600256837 shutdown_connections 2023-11-26T07:22:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.645+0000 7f492e0ea640 1 -- 172.21.15.31:0/3600256837 wait complete. 2023-11-26T07:22:54.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:22:54.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.817+0000 7f58975f8640 1 Processor -- start 2023-11-26T07:22:54.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.817+0000 7f58975f8640 1 -- start start 2023-11-26T07:22:54.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f58975f8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f589015d8f0 0x7f589015e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:54.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f58975f8640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f58900ced30 0x7f58900cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f58975f8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589015ecc0 con 0x7f58900cf1e0 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f58975f8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58900d4420 con 0x7f589015d8f0 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f58975f8640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58900cdfb0 con 0x7f58900ced30 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f5895394640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f589015d8f0 0x7f589015e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f5895394640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f589015d8f0 0x7f589015e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44334/0 (socket says 172.21.15.31:44334) 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f5895394640 1 -- 172.21.15.31:0/604004400 learned_addr learned my addr 172.21.15.31:0/604004400 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f5895b95640 1 --1- 172.21.15.31:0/604004400 >> v1:172.21.15.31:6789/0 conn(0x7f58900cf1e0 0x7f58900d1000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49212/0 (socket says 172.21.15.31:49212) 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f5894b93640 1 --2- 172.21.15.31:0/604004400 >> v2:172.21.15.97:3300/0 conn(0x7f58900ced30 0x7f58900cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f5895394640 1 -- 172.21.15.31:0/604004400 >> v2:172.21.15.97:3300/0 conn(0x7f58900ced30 msgr2=0x7f58900cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f5895394640 1 --2- 172.21.15.31:0/604004400 >> v2:172.21.15.97:3300/0 conn(0x7f58900ced30 0x7f58900cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f5895394640 1 -- 172.21.15.31:0/604004400 >> v1:172.21.15.31:6789/0 conn(0x7f58900cf1e0 legacy=0x7f58900d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f5895394640 1 -- 172.21.15.31:0/604004400 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58900d4140 con 0x7f589015d8f0 2023-11-26T07:22:54.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.821+0000 7f5887fff640 1 -- 172.21.15.31:0/604004400 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623332703 0 0) 0x7f589015ecc0 con 0x7f58900cf1e0 2023-11-26T07:22:54.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f5895394640 1 --2- 172.21.15.31:0/604004400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f589015d8f0 0x7f589015e780 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f588c009870 tx=0x7f588c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6060a383515440 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f5887fff640 1 -- 172.21.15.31:0/604004400 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c0aa430 con 0x7f589015d8f0 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f5887fff640 1 -- 172.21.15.31:0/604004400 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c0aa590 con 0x7f589015d8f0 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f5887fff640 1 -- 172.21.15.31:0/604004400 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c0aa920 con 0x7f589015d8f0 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f58975f8640 1 -- 172.21.15.31:0/604004400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f589015d8f0 msgr2=0x7f589015e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f58975f8640 1 --2- 172.21.15.31:0/604004400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f589015d8f0 0x7f589015e780 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f588c009870 tx=0x7f588c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f58975f8640 1 -- 172.21.15.31:0/604004400 shutdown_connections 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f58975f8640 1 --2- 172.21.15.31:0/604004400 >> v2:172.21.15.97:3300/0 conn(0x7f58900ced30 0x7f58900cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f58975f8640 1 --2- 172.21.15.31:0/604004400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f589015d8f0 0x7f589015e780 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f58975f8640 1 -- 172.21.15.31:0/604004400 >> 172.21.15.31:0/604004400 conn(0x7f589009c750 msgr2=0x7f58900d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f58975f8640 1 -- 172.21.15.31:0/604004400 shutdown_connections 2023-11-26T07:22:54.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f58975f8640 1 -- 172.21.15.31:0/604004400 wait complete. 2023-11-26T07:22:54.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f58975f8640 1 Processor -- start 2023-11-26T07:22:54.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.825+0000 7f58975f8640 1 -- start start 2023-11-26T07:22:54.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f58975f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58900ced30 0x7f58900caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:54.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f58975f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58900cf1e0 0x7f58900c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:54.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f58975f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589015d8f0 0x7f5890157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f58975f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589015ad60 con 0x7f58900cf1e0 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f58975f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58900b4820 con 0x7f589015d8f0 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f58975f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589015aa80 con 0x7f58900ced30 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5894b93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58900cf1e0 0x7f58900c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5894b93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58900cf1e0 0x7f58900c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44336/0 (socket says 172.21.15.31:44336) 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5894b93640 1 -- 172.21.15.31:0/3775291469 learned_addr learned my addr 172.21.15.31:0/3775291469 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5895394640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58900ced30 0x7f58900caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5895b95640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589015d8f0 0x7f5890157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5894b93640 1 -- 172.21.15.31:0/3775291469 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58900ced30 msgr2=0x7f58900caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5894b93640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58900ced30 0x7f58900caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5894b93640 1 -- 172.21.15.31:0/3775291469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589015d8f0 msgr2=0x7f5890157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:54.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5894b93640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589015d8f0 0x7f5890157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5894b93640 1 -- 172.21.15.31:0/3775291469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c0094d0 con 0x7f58900cf1e0 2023-11-26T07:22:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5895394640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58900ced30 0x7f58900caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5895b95640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589015d8f0 0x7f5890157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5894b93640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58900cf1e0 0x7f58900c8aa0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f588000d5f0 tx=0x7f588000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5885ffb640 1 -- 172.21.15.31:0/3775291469 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58800125f0 con 0x7f58900cf1e0 2023-11-26T07:22:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5885ffb640 1 -- 172.21.15.31:0/3775291469 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5880012750 con 0x7f58900cf1e0 2023-11-26T07:22:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f58975f8640 1 -- 172.21.15.31:0/3775291469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58900b41d0 con 0x7f58900cf1e0 2023-11-26T07:22:54.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5885ffb640 1 -- 172.21.15.31:0/3775291469 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880012ae0 con 0x7f58900cf1e0 2023-11-26T07:22:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f58975f8640 1 -- 172.21.15.31:0/3775291469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58900c8fe0 con 0x7f58900cf1e0 2023-11-26T07:22:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f58975f8640 1 -- 172.21.15.31:0/3775291469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58900b4030 con 0x7f58900cf1e0 2023-11-26T07:22:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.829+0000 7f5885ffb640 1 -- 172.21.15.31:0/3775291469 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f5880012c40 con 0x7f58900cf1e0 2023-11-26T07:22:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.833+0000 7f5885ffb640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f58740665c0 0x7f5874068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:54.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.833+0000 7f5885ffb640 1 -- 172.21.15.31:0/3775291469 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f58800843d0 con 0x7f58900cf1e0 2023-11-26T07:22:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.833+0000 7f5895394640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f58740665c0 0x7f5874068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:54.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.833+0000 7f5895394640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f58740665c0 0x7f5874068a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f588c0ae000 tx=0x7f588c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:54.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.837+0000 7f5885ffb640 1 -- 172.21.15.31:0/3775291469 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f58900b4030 con 0x7f58900cf1e0 2023-11-26T07:22:54.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:54.993+0000 7f58975f8640 1 -- 172.21.15.31:0/3775291469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f58900b4030 con 0x7f58900cf1e0 2023-11-26T07:22:55.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f5885ffb640 1 -- 172.21.15.31:0/3775291469 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 169+0+0 (secure 0 0 0) 0x7f58900b4030 con 0x7f58900cf1e0 2023-11-26T07:22:55.647 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-11-26T07:22:55.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f58975f8640 1 -- 172.21.15.31:0/3775291469 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f58740665c0 msgr2=0x7f5874068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:55.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f58975f8640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f58740665c0 0x7f5874068a80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f588c0ae000 tx=0x7f588c0a7000 comp rx=0 tx=0).stop 2023-11-26T07:22:55.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f58975f8640 1 -- 172.21.15.31:0/3775291469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58900cf1e0 msgr2=0x7f58900c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:55.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f58975f8640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58900cf1e0 0x7f58900c8aa0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f588000d5f0 tx=0x7f588000dab0 comp rx=0 tx=0).stop 2023-11-26T07:22:55.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f58975f8640 1 -- 172.21.15.31:0/3775291469 shutdown_connections 2023-11-26T07:22:55.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f58975f8640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f58740665c0 0x7f5874068a80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:55.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f58975f8640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f589015d8f0 0x7f5890157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:55.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f58975f8640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58900cf1e0 0x7f58900c8aa0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:55.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f58975f8640 1 --2- 172.21.15.31:0/3775291469 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f58900ced30 0x7f58900caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:55.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.645+0000 7f58975f8640 1 -- 172.21.15.31:0/3775291469 >> 172.21.15.31:0/3775291469 conn(0x7f589009c750 msgr2=0x7f58900b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:55.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.649+0000 7f58975f8640 1 -- 172.21.15.31:0/3775291469 shutdown_connections 2023-11-26T07:22:55.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.649+0000 7f58975f8640 1 -- 172.21.15.31:0/3775291469 wait complete. 2023-11-26T07:22:55.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:22:55.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f112cd0e640 1 Processor -- start 2023-11-26T07:22:55.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f112cd0e640 1 -- start start 2023-11-26T07:22:55.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f112cd0e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f11280cded0 0x7f112813f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:55.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f112cd0e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f112813f7a0 0x7f11280cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f112cd0e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112813cf40 con 0x7f1128154180 2023-11-26T07:22:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f112cd0e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112813fbf0 con 0x7f11280cded0 2023-11-26T07:22:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f112cd0e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280cf0a0 con 0x7f112813f7a0 2023-11-26T07:22:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f111ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f11280cded0 0x7f112813f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:55.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f1126d9d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f112813f7a0 0x7f11280cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f111ffff640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f11280cded0 0x7f112813f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:43232/0 (socket says 172.21.15.31:43232) 2023-11-26T07:22:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f111ffff640 1 -- 172.21.15.31:0/2389349310 learned_addr learned my addr 172.21.15.31:0/2389349310 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f111ffff640 1 -- 172.21.15.31:0/2389349310 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f112813f7a0 msgr2=0x7f11280cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f111ffff640 1 --2- 172.21.15.31:0/2389349310 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f112813f7a0 0x7f11280cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f111ffff640 1 -- 172.21.15.31:0/2389349310 >> v1:172.21.15.97:6789/0 conn(0x7f1128154180 legacy=0x7f112815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:55.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.829+0000 7f111ffff640 1 -- 172.21.15.31:0/2389349310 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11281400e0 con 0x7f11280cded0 2023-11-26T07:22:55.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f111ffff640 1 --2- 172.21.15.31:0/2389349310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11280cded0 0x7f112813f260 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1110009870 tx=0x7f111001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4e54563dccba5c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f1125d9b640 1 -- 172.21.15.31:0/2389349310 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100ac410 con 0x7f11280cded0 2023-11-26T07:22:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f1125d9b640 1 -- 172.21.15.31:0/2389349310 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11100ac570 con 0x7f11280cded0 2023-11-26T07:22:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f1125d9b640 1 -- 172.21.15.31:0/2389349310 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100ac900 con 0x7f11280cded0 2023-11-26T07:22:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f112cd0e640 1 -- 172.21.15.31:0/2389349310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11280cded0 msgr2=0x7f112813f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:55.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f112cd0e640 1 --2- 172.21.15.31:0/2389349310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11280cded0 0x7f112813f260 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1110009870 tx=0x7f111001c540 comp rx=0 tx=0).stop 2023-11-26T07:22:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f112cd0e640 1 -- 172.21.15.31:0/2389349310 shutdown_connections 2023-11-26T07:22:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f112cd0e640 1 --2- 172.21.15.31:0/2389349310 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f112813f7a0 0x7f11280cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f112cd0e640 1 --2- 172.21.15.31:0/2389349310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11280cded0 0x7f112813f260 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f112cd0e640 1 -- 172.21.15.31:0/2389349310 >> 172.21.15.31:0/2389349310 conn(0x7f1128071d80 msgr2=0x7f112806a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f112cd0e640 1 -- 172.21.15.31:0/2389349310 shutdown_connections 2023-11-26T07:22:55.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.833+0000 7f112cd0e640 1 -- 172.21.15.31:0/2389349310 wait complete. 2023-11-26T07:22:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112cd0e640 1 Processor -- start 2023-11-26T07:22:55.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112cd0e640 1 -- start start 2023-11-26T07:22:55.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112cd0e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11280cded0 0x7f1128157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112cd0e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f112813f7a0 0x7f1128154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112cd0e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1128154180 0x7f1128154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112cd0e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1128152fc0 con 0x7f112813f7a0 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112cd0e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1128157880 con 0x7f11280cded0 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112cd0e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1128155520 con 0x7f1128154180 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f112813f7a0 0x7f1128154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f112813f7a0 0x7f1128154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44348/0 (socket says 172.21.15.31:44348) 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111ffff640 1 -- 172.21.15.31:0/1471881857 learned_addr learned my addr 172.21.15.31:0/1471881857 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112659c640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11280cded0 0x7f1128157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f1126d9d640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1128154180 0x7f1128154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111ffff640 1 -- 172.21.15.31:0/1471881857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1128154180 msgr2=0x7f1128154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111ffff640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1128154180 0x7f1128154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:55.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111ffff640 1 -- 172.21.15.31:0/1471881857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11280cded0 msgr2=0x7f1128157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:55.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111ffff640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11280cded0 0x7f1128157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:55.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111ffff640 1 -- 172.21.15.31:0/1471881857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11100094d0 con 0x7f112813f7a0 2023-11-26T07:22:55.842 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111ffff640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f112813f7a0 0x7f1128154970 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f11100af040 tx=0x7f11100192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111effd640 1 -- 172.21.15.31:0/1471881857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100bb070 con 0x7f112813f7a0 2023-11-26T07:22:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111effd640 1 -- 172.21.15.31:0/1471881857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11100ace70 con 0x7f112813f7a0 2023-11-26T07:22:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f111effd640 1 -- 172.21.15.31:0/1471881857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100ac410 con 0x7f112813f7a0 2023-11-26T07:22:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112cd0e640 1 -- 172.21.15.31:0/1471881857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1128155800 con 0x7f112813f7a0 2023-11-26T07:22:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.837+0000 7f112cd0e640 1 -- 172.21.15.31:0/1471881857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1128155dc0 con 0x7f112813f7a0 2023-11-26T07:22:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.841+0000 7f112cd0e640 1 -- 172.21.15.31:0/1471881857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112804e850 con 0x7f112813f7a0 2023-11-26T07:22:55.843 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.841+0000 7f111effd640 1 -- 172.21.15.31:0/1471881857 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f11100b65f0 con 0x7f112813f7a0 2023-11-26T07:22:55.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.841+0000 7f111effd640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1108066590 0x7f1108068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:55.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.841+0000 7f111effd640 1 -- 172.21.15.31:0/1471881857 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f111011eae0 con 0x7f112813f7a0 2023-11-26T07:22:55.845 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.841+0000 7f112659c640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1108066590 0x7f1108068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:55.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.845+0000 7f112659c640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1108066590 0x7f1108068a50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f112806b610 tx=0x7f1118009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:55.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:55.845+0000 7f111effd640 1 -- 172.21.15.31:0/1471881857 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f11100ec210 con 0x7f112813f7a0 2023-11-26T07:22:56.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.009+0000 7f112cd0e640 1 -- 172.21.15.31:0/1471881857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f11281325f0 con 0x7f112813f7a0 2023-11-26T07:22:56.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.013+0000 7f111effd640 1 -- 172.21.15.31:0/1471881857 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f11100ef830 con 0x7f112813f7a0 2023-11-26T07:22:56.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 -- 172.21.15.31:0/1471881857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1108066590 msgr2=0x7f1108068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1108066590 0x7f1108068a50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f112806b610 tx=0x7f1118009040 comp rx=0 tx=0).stop 2023-11-26T07:22:56.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 -- 172.21.15.31:0/1471881857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f112813f7a0 msgr2=0x7f1128154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f112813f7a0 0x7f1128154970 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f11100af040 tx=0x7f11100192e0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 -- 172.21.15.31:0/1471881857 shutdown_connections 2023-11-26T07:22:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1108066590 0x7f1108068a50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1128154180 0x7f1128154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f112813f7a0 0x7f1128154970 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 --2- 172.21.15.31:0/1471881857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f11280cded0 0x7f1128157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 -- 172.21.15.31:0/1471881857 >> 172.21.15.31:0/1471881857 conn(0x7f1128071d80 msgr2=0x7f11280cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 -- 172.21.15.31:0/1471881857 shutdown_connections 2023-11-26T07:22:56.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.017+0000 7f112cd0e640 1 -- 172.21.15.31:0/1471881857 wait complete. 2023-11-26T07:22:56.037 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2023-11-26T07:22:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f134cebb640 1 Processor -- start 2023-11-26T07:22:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f134cebb640 1 -- start start 2023-11-26T07:22:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f134cebb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f134815d310 0x7f134806b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f134cebb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13480ce710 0x7f13480d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f134cebb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134806c4f0 con 0x7f134806ba40 2023-11-26T07:22:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f134cebb640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13480d1810 con 0x7f13480ce710 2023-11-26T07:22:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f134cebb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f134806bf30 con 0x7f134815d310 2023-11-26T07:22:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f1347fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13480ce710 0x7f13480d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f13477fe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f134815d310 0x7f134806b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f1347fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f13480ce710 0x7f13480d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44354/0 (socket says 172.21.15.31:44354) 2023-11-26T07:22:56.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f1346ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f134806ba40 0x7f13480ce000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:44974/0 (socket says 172.21.15.31:44974) 2023-11-26T07:22:56.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f1347fff640 1 -- 172.21.15.31:0/2172862049 learned_addr learned my addr 172.21.15.31:0/2172862049 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:56.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f13477fe640 1 -- 172.21.15.31:0/2172862049 >> v2:172.21.15.31:3300/0 conn(0x7f13480ce710 msgr2=0x7f13480d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:56.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f13477fe640 1 --2- 172.21.15.31:0/2172862049 >> v2:172.21.15.31:3300/0 conn(0x7f13480ce710 0x7f13480d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f13477fe640 1 -- 172.21.15.31:0/2172862049 >> v1:172.21.15.97:6789/0 conn(0x7f134806ba40 legacy=0x7f13480ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f13477fe640 1 -- 172.21.15.31:0/2172862049 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134806c210 con 0x7f134815d310 2023-11-26T07:22:56.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f1347fff640 1 --2- 172.21.15.31:0/2172862049 >> v2:172.21.15.31:3300/0 conn(0x7f13480ce710 0x7f13480d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:22:56.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.185+0000 7f13467fc640 1 -- 172.21.15.31:0/2172862049 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158552014 0 0) 0x7f134806c4f0 con 0x7f134806ba40 2023-11-26T07:22:56.192 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f13477fe640 1 --2- 172.21.15.31:0/2172862049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f134815d310 0x7f134806b500 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f13380097c0 tx=0x7f133801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c42cc5a5a1bb4e server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:56.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f13467fc640 1 -- 172.21.15.31:0/2172862049 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13380ab600 con 0x7f134815d310 2023-11-26T07:22:56.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f13467fc640 1 -- 172.21.15.31:0/2172862049 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13380a7050 con 0x7f134815d310 2023-11-26T07:22:56.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f13467fc640 1 -- 172.21.15.31:0/2172862049 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13380ab980 con 0x7f134815d310 2023-11-26T07:22:56.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f134cebb640 1 -- 172.21.15.31:0/2172862049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f134815d310 msgr2=0x7f134806b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f134cebb640 1 --2- 172.21.15.31:0/2172862049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f134815d310 0x7f134806b500 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f13380097c0 tx=0x7f133801c3c0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f134cebb640 1 -- 172.21.15.31:0/2172862049 shutdown_connections 2023-11-26T07:22:56.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f134cebb640 1 --2- 172.21.15.31:0/2172862049 >> v2:172.21.15.31:3300/0 conn(0x7f13480ce710 0x7f13480d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f134cebb640 1 --2- 172.21.15.31:0/2172862049 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f134815d310 0x7f134806b500 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f134cebb640 1 -- 172.21.15.31:0/2172862049 >> 172.21.15.31:0/2172862049 conn(0x7f13480d00b0 msgr2=0x7f13480d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:56.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f134cebb640 1 -- 172.21.15.31:0/2172862049 shutdown_connections 2023-11-26T07:22:56.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.189+0000 7f134cebb640 1 -- 172.21.15.31:0/2172862049 wait complete. 2023-11-26T07:22:56.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f134cebb640 1 Processor -- start 2023-11-26T07:22:56.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f134cebb640 1 -- start start 2023-11-26T07:22:56.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f134cebb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f134806ba40 0x7f1348156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f134cebb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13480ce710 0x7f1348155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f134cebb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134815d310 0x7f13480cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f134cebb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480cbf10 con 0x7f134806ba40 2023-11-26T07:22:56.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f134cebb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13480ccfc0 con 0x7f134815d310 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f134cebb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13480cb300 con 0x7f13480ce710 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f13477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f134806ba40 0x7f1348156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f13477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f134806ba40 0x7f1348156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44362/0 (socket says 172.21.15.31:44362) 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f13477fe640 1 -- 172.21.15.31:0/662169143 learned_addr learned my addr 172.21.15.31:0/662169143 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f1347fff640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134815d310 0x7f13480cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f1346ffd640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13480ce710 0x7f1348155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f13477fe640 1 -- 172.21.15.31:0/662169143 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13480ce710 msgr2=0x7f1348155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f13477fe640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13480ce710 0x7f1348155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f13477fe640 1 -- 172.21.15.31:0/662169143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134815d310 msgr2=0x7f13480cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f13477fe640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134815d310 0x7f13480cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f13477fe640 1 -- 172.21.15.31:0/662169143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1340009b30 con 0x7f134806ba40 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f1347fff640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134815d310 0x7f13480cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:56.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f1346ffd640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13480ce710 0x7f1348155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:56.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.193+0000 7f13477fe640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f134806ba40 0x7f1348156920 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f13380ae040 tx=0x7f1338019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:56.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f132ffff640 1 -- 172.21.15.31:0/662169143 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338006e20 con 0x7f134806ba40 2023-11-26T07:22:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f132ffff640 1 -- 172.21.15.31:0/662169143 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13380b4410 con 0x7f134806ba40 2023-11-26T07:22:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f134cebb640 1 -- 172.21.15.31:0/662169143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1338009420 con 0x7f134806ba40 2023-11-26T07:22:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f132ffff640 1 -- 172.21.15.31:0/662169143 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13380b47a0 con 0x7f134806ba40 2023-11-26T07:22:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f134cebb640 1 -- 172.21.15.31:0/662169143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13480cb8c0 con 0x7f134806ba40 2023-11-26T07:22:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f134cebb640 1 -- 172.21.15.31:0/662169143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1348074a00 con 0x7f134806ba40 2023-11-26T07:22:56.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f132ffff640 1 -- 172.21.15.31:0/662169143 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f13380b2040 con 0x7f134806ba40 2023-11-26T07:22:56.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f132ffff640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1334066590 0x7f1334068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f132ffff640 1 -- 172.21.15.31:0/662169143 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f133811ddb0 con 0x7f134806ba40 2023-11-26T07:22:56.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f1346ffd640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1334066590 0x7f1334068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.197+0000 7f1346ffd640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1334066590 0x7f1334068a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f133c007fd0 tx=0x7f133c007c20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:56.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.201+0000 7f132ffff640 1 -- 172.21.15.31:0/662169143 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1338122070 con 0x7f134806ba40 2023-11-26T07:22:56.418 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.417+0000 7f134cebb640 1 -- 172.21.15.31:0/662169143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) v1 -- 0x7f13480d0ed0 con 0x7f134806ba40 2023-11-26T07:22:56.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.653+0000 7f132ffff640 1 -- 172.21.15.31:0/662169143 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v295) v1 ==== 146+0+0 (secure 0 0 0) 0x7f13380b0020 con 0x7f134806ba40 2023-11-26T07:22:56.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 -- 172.21.15.31:0/662169143 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1334066590 msgr2=0x7f1334068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1334066590 0x7f1334068a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f133c007fd0 tx=0x7f133c007c20 comp rx=0 tx=0).stop 2023-11-26T07:22:56.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 -- 172.21.15.31:0/662169143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f134806ba40 msgr2=0x7f1348156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f134806ba40 0x7f1348156920 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f13380ae040 tx=0x7f1338019160 comp rx=0 tx=0).stop 2023-11-26T07:22:56.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 -- 172.21.15.31:0/662169143 shutdown_connections 2023-11-26T07:22:56.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1334066590 0x7f1334068a50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f134815d310 0x7f13480cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f13480ce710 0x7f1348155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 --2- 172.21.15.31:0/662169143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f134806ba40 0x7f1348156920 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 -- 172.21.15.31:0/662169143 >> 172.21.15.31:0/662169143 conn(0x7f13480d00b0 msgr2=0x7f13480d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 -- 172.21.15.31:0/662169143 shutdown_connections 2023-11-26T07:22:56.662 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.657+0000 7f134cebb640 1 -- 172.21.15.31:0/662169143 wait complete. 2023-11-26T07:22:56.681 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2023-11-26T07:22:56.681 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2023-11-26T07:22:56.681 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:22:56.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:22:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.821+0000 7fcddb632640 1 Processor -- start 2023-11-26T07:22:56.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.821+0000 7fcddb632640 1 -- start start 2023-11-26T07:22:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcddb632640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcdd415d7b0 0x7fcdd40d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcddb632640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40cead0 0x7fcdd40d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcddb632640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcdd40d1eb0 0x7fcdd40cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcddb632640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd40d2340 con 0x7fcdd415d7b0 2023-11-26T07:22:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcddb632640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd40cefd0 con 0x7fcdd40d1eb0 2023-11-26T07:22:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcddb632640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd40cf2b0 con 0x7fcdd40cead0 2023-11-26T07:22:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd93ce640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcdd415d7b0 0x7fcdd40d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd93ce640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcdd415d7b0 0x7fcdd40d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44376/0 (socket says 172.21.15.31:44376) 2023-11-26T07:22:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd93ce640 1 -- 172.21.15.31:0/3742098937 learned_addr learned my addr 172.21.15.31:0/3742098937 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd8bcd640 1 --2- 172.21.15.31:0/3742098937 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40cead0 0x7fcdd40d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd9bcf640 1 --2- 172.21.15.31:0/3742098937 >> v2:172.21.15.97:3300/0 conn(0x7fcdd40d1eb0 0x7fcdd40cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd93ce640 1 -- 172.21.15.31:0/3742098937 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40cead0 msgr2=0x7fcdd40d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd93ce640 1 --2- 172.21.15.31:0/3742098937 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40cead0 0x7fcdd40d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd93ce640 1 -- 172.21.15.31:0/3742098937 >> v2:172.21.15.97:3300/0 conn(0x7fcdd40d1eb0 msgr2=0x7fcdd40cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd93ce640 1 --2- 172.21.15.31:0/3742098937 >> v2:172.21.15.97:3300/0 conn(0x7fcdd40d1eb0 0x7fcdd40cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd93ce640 1 -- 172.21.15.31:0/3742098937 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd406b3f0 con 0x7fcdd415d7b0 2023-11-26T07:22:56.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd8bcd640 1 --2- 172.21.15.31:0/3742098937 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40cead0 0x7fcdd40d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:56.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.825+0000 7fcdd93ce640 1 --2- 172.21.15.31:0/3742098937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 0x7fcdd40d10d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcdd408c870 tx=0x7fcdd001c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5e15d1c45f2e97 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcdcbfff640 1 -- 172.21.15.31:0/3742098937 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00ab600 con 0x7fcdd415d7b0 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcdcbfff640 1 -- 172.21.15.31:0/3742098937 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd00a7050 con 0x7fcdd415d7b0 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcdcbfff640 1 -- 172.21.15.31:0/3742098937 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd00ab980 con 0x7fcdd415d7b0 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 -- 172.21.15.31:0/3742098937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 msgr2=0x7fcdd40d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 --2- 172.21.15.31:0/3742098937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 0x7fcdd40d10d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fcdd408c870 tx=0x7fcdd001c0f0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 -- 172.21.15.31:0/3742098937 shutdown_connections 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 --2- 172.21.15.31:0/3742098937 >> v2:172.21.15.97:3300/0 conn(0x7fcdd40d1eb0 0x7fcdd40cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 --2- 172.21.15.31:0/3742098937 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40cead0 0x7fcdd40d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 --2- 172.21.15.31:0/3742098937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 0x7fcdd40d10d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 -- 172.21.15.31:0/3742098937 >> 172.21.15.31:0/3742098937 conn(0x7fcdd40d33b0 msgr2=0x7fcdd40d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:56.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 -- 172.21.15.31:0/3742098937 shutdown_connections 2023-11-26T07:22:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 -- 172.21.15.31:0/3742098937 wait complete. 2023-11-26T07:22:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 Processor -- start 2023-11-26T07:22:56.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 -- start start 2023-11-26T07:22:56.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd40cead0 0x7fcdd4155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40d1eb0 0x7fcdd40b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 0x7fcdd41536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd4154ba0 con 0x7fcdd415d7b0 2023-11-26T07:22:56.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd40c7da0 con 0x7fcdd40cead0 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcddb632640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd4154270 con 0x7fcdd40d1eb0 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcdd9bcf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 0x7fcdd41536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcdd9bcf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 0x7fcdd41536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44390/0 (socket says 172.21.15.31:44390) 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.829+0000 7fcdd9bcf640 1 -- 172.21.15.31:0/1430047600 learned_addr learned my addr 172.21.15.31:0/1430047600 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdd93ce640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd40cead0 0x7fcdd4155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdd8bcd640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40d1eb0 0x7fcdd40b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdd9bcf640 1 -- 172.21.15.31:0/1430047600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40d1eb0 msgr2=0x7fcdd40b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdd9bcf640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40d1eb0 0x7fcdd40b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdd9bcf640 1 -- 172.21.15.31:0/1430047600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd40cead0 msgr2=0x7fcdd4155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdd9bcf640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd40cead0 0x7fcdd4155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdd9bcf640 1 -- 172.21.15.31:0/1430047600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd00094d0 con 0x7fcdd415d7b0 2023-11-26T07:22:56.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdd93ce640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd40cead0 0x7fcdd4155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:56.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdd9bcf640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 0x7fcdd41536e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc007d90 tx=0x7fcdcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:56.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdc9ffb640 1 -- 172.21.15.31:0/1430047600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc014420 con 0x7fcdd415d7b0 2023-11-26T07:22:56.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdc9ffb640 1 -- 172.21.15.31:0/1430047600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc0145a0 con 0x7fcdd415d7b0 2023-11-26T07:22:56.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcddb632640 1 -- 172.21.15.31:0/1430047600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd4154550 con 0x7fcdd415d7b0 2023-11-26T07:22:56.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdc9ffb640 1 -- 172.21.15.31:0/1430047600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc014950 con 0x7fcdd415d7b0 2023-11-26T07:22:56.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcddb632640 1 -- 172.21.15.31:0/1430047600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd4155570 con 0x7fcdd415d7b0 2023-11-26T07:22:56.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcddb632640 1 -- 172.21.15.31:0/1430047600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd4154d10 con 0x7fcdd415d7b0 2023-11-26T07:22:56.837 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdc9ffb640 1 -- 172.21.15.31:0/1430047600 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fcdcc014ab0 con 0x7fcdd415d7b0 2023-11-26T07:22:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdc9ffb640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcdb80665c0 0x7fcdb8068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.833+0000 7fcdc9ffb640 1 -- 172.21.15.31:0/1430047600 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fcdcc084680 con 0x7fcdd415d7b0 2023-11-26T07:22:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.837+0000 7fcdd93ce640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcdb80665c0 0x7fcdb8068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:56.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.837+0000 7fcdd93ce640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcdb80665c0 0x7fcdb8068a80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00ad000 tx=0x7fcdd00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:56.841 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.837+0000 7fcdc9ffb640 1 -- 172.21.15.31:0/1430047600 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcdcc051bf0 con 0x7fcdd415d7b0 2023-11-26T07:22:56.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.993+0000 7fcddb632640 1 -- 172.21.15.31:0/1430047600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdd404e850 con 0x7fcdd415d7b0 2023-11-26T07:22:56.998 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcdc9ffb640 1 -- 172.21.15.31:0/1430047600 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+13031 (secure 0 0 0) 0x7fcdcc055210 con 0x7fcdd415d7b0 2023-11-26T07:22:56.998 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:56.999 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":91,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:22:55.644919+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6835/3729395637":"2023-11-27T07:22:52.630901+0000","172.21.15.31:6834/3729395637":"2023-11-27T07:22:52.630901+0000","172.21.15.97:6835/683007803":"2023-11-27T07:22:41.280769+0000","172.21.15.97:6833/683007803":"2023-11-27T07:22:41.280769+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:22:57.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 -- 172.21.15.31:0/1430047600 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcdb80665c0 msgr2=0x7fcdb8068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcdb80665c0 0x7fcdb8068a80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcdd00ad000 tx=0x7fcdd00a6000 comp rx=0 tx=0).stop 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 -- 172.21.15.31:0/1430047600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 msgr2=0x7fcdd41536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 0x7fcdd41536e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc007d90 tx=0x7fcdcc00e4b0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 -- 172.21.15.31:0/1430047600 shutdown_connections 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcdb80665c0 0x7fcdb8068a80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdd415d7b0 0x7fcdd41536e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcdd40d1eb0 0x7fcdd40b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 --2- 172.21.15.31:0/1430047600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdd40cead0 0x7fcdd4155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 -- 172.21.15.31:0/1430047600 >> 172.21.15.31:0/1430047600 conn(0x7fcdd40d33b0 msgr2=0x7fcdd40b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 -- 172.21.15.31:0/1430047600 shutdown_connections 2023-11-26T07:22:57.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:56.997+0000 7fcddb632640 1 -- 172.21.15.31:0/1430047600 wait complete. 2023-11-26T07:22:57.020 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:22:57.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:22:57.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5efb217640 1 Processor -- start 2023-11-26T07:22:57.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5efb217640 1 -- start start 2023-11-26T07:22:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5efb217640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5ef415b4b0 0x7f5ef40b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5efb217640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef40b0130 con 0x7f5ef40b1820 2023-11-26T07:22:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5efb217640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef40afb00 con 0x7f5ef406a5b0 2023-11-26T07:22:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5efb217640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef40b0b00 con 0x7f5ef415b4b0 2023-11-26T07:22:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5ef8fb3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5ef415b4b0 0x7f5ef40b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5ef8fb3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5ef415b4b0 0x7f5ef40b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44396/0 (socket says 172.21.15.31:44396) 2023-11-26T07:22:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5ef8fb3640 1 -- 172.21.15.31:0/3526812839 learned_addr learned my addr 172.21.15.31:0/3526812839 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:57.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5ef97b4640 1 --1- 172.21.15.31:0/3526812839 >> v1:172.21.15.31:6789/0 conn(0x7f5ef40b1820 0x7f5ef40ad030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49214/0 (socket says 172.21.15.31:49214) 2023-11-26T07:22:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5ef8fb3640 1 -- 172.21.15.31:0/3526812839 >> v1:172.21.15.97:6789/0 conn(0x7f5ef406a5b0 legacy=0x7f5ef40b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5ef8fb3640 1 -- 172.21.15.31:0/3526812839 >> v1:172.21.15.31:6789/0 conn(0x7f5ef40b1820 legacy=0x7f5ef40ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.165+0000 7f5ef8fb3640 1 -- 172.21.15.31:0/3526812839 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef406af70 con 0x7f5ef415b4b0 2023-11-26T07:22:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5ef8fb3640 1 --2- 172.21.15.31:0/3526812839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef415b4b0 0x7f5ef40b05c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009870 tx=0x7f5ee801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9732613afc53b2b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5ef37fe640 1 -- 172.21.15.31:0/3526812839 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80aa430 con 0x7f5ef415b4b0 2023-11-26T07:22:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5ef37fe640 1 -- 172.21.15.31:0/3526812839 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee80aa590 con 0x7f5ef415b4b0 2023-11-26T07:22:57.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5ef37fe640 1 -- 172.21.15.31:0/3526812839 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80aa920 con 0x7f5ef415b4b0 2023-11-26T07:22:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5efb217640 1 -- 172.21.15.31:0/3526812839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef415b4b0 msgr2=0x7f5ef40b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5efb217640 1 --2- 172.21.15.31:0/3526812839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef415b4b0 0x7f5ef40b05c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009870 tx=0x7f5ee801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5efb217640 1 -- 172.21.15.31:0/3526812839 shutdown_connections 2023-11-26T07:22:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5efb217640 1 --2- 172.21.15.31:0/3526812839 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef415b4b0 0x7f5ef40b05c0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5efb217640 1 -- 172.21.15.31:0/3526812839 >> 172.21.15.31:0/3526812839 conn(0x7f5ef40b1d50 msgr2=0x7f5ef40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:57.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5efb217640 1 -- 172.21.15.31:0/3526812839 shutdown_connections 2023-11-26T07:22:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.169+0000 7f5efb217640 1 -- 172.21.15.31:0/3526812839 wait complete. 2023-11-26T07:22:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5efb217640 1 Processor -- start 2023-11-26T07:22:57.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5efb217640 1 -- start start 2023-11-26T07:22:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5efb217640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef406a5b0 0x7f5ef414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5efb217640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ef40b1820 0x7f5ef4150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5efb217640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef415b4b0 0x7f5ef40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5efb217640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40c9bb0 con 0x7f5ef406a5b0 2023-11-26T07:22:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5efb217640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40c9720 con 0x7f5ef415b4b0 2023-11-26T07:22:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5efb217640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef414fd80 con 0x7f5ef40b1820 2023-11-26T07:22:57.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef8fb3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef406a5b0 0x7f5ef414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef8fb3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef406a5b0 0x7f5ef414f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44400/0 (socket says 172.21.15.31:44400) 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef8fb3640 1 -- 172.21.15.31:0/574148835 learned_addr learned my addr 172.21.15.31:0/574148835 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef97b4640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef415b4b0 0x7f5ef40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef3fff640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ef40b1820 0x7f5ef4150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef8fb3640 1 -- 172.21.15.31:0/574148835 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ef40b1820 msgr2=0x7f5ef4150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef8fb3640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ef40b1820 0x7f5ef4150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef8fb3640 1 -- 172.21.15.31:0/574148835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef415b4b0 msgr2=0x7f5ef40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef8fb3640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef415b4b0 0x7f5ef40cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef8fb3640 1 -- 172.21.15.31:0/574148835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee80094d0 con 0x7f5ef406a5b0 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef97b4640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef415b4b0 0x7f5ef40cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef8fb3640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef406a5b0 0x7f5ef414f840 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80ac040 tx=0x7f5ee801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef17fa640 1 -- 172.21.15.31:0/574148835 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80b38c0 con 0x7f5ef406a5b0 2023-11-26T07:22:57.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef17fa640 1 -- 172.21.15.31:0/574148835 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee80b3a40 con 0x7f5ef406a5b0 2023-11-26T07:22:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5ef17fa640 1 -- 172.21.15.31:0/574148835 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80b3df0 con 0x7f5ef406a5b0 2023-11-26T07:22:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5efb217640 1 -- 172.21.15.31:0/574148835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef40c8df0 con 0x7f5ef406a5b0 2023-11-26T07:22:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.173+0000 7f5efb217640 1 -- 172.21.15.31:0/574148835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef40c93b0 con 0x7f5ef406a5b0 2023-11-26T07:22:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.177+0000 7f5efb217640 1 -- 172.21.15.31:0/574148835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef404e850 con 0x7f5ef406a5b0 2023-11-26T07:22:57.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.177+0000 7f5ef17fa640 1 -- 172.21.15.31:0/574148835 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f5ee80b2380 con 0x7f5ef406a5b0 2023-11-26T07:22:57.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.177+0000 7f5ef17fa640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ed8066590 0x7f5ed8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.177+0000 7f5ef17fa640 1 -- 172.21.15.31:0/574148835 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5ee811e4f0 con 0x7f5ef406a5b0 2023-11-26T07:22:57.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.177+0000 7f5ef3fff640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ed8066590 0x7f5ed8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.177+0000 7f5ef3fff640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ed8066590 0x7f5ed8068a50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5ef41545d0 tx=0x7f5ee0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:57.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.181+0000 7f5ef17fa640 1 -- 172.21.15.31:0/574148835 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ee80eba90 con 0x7f5ef406a5b0 2023-11-26T07:22:57.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.341+0000 7f5efb217640 1 -- 172.21.15.31:0/574148835 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ef40b15e0 con 0x7f5ef406a5b0 2023-11-26T07:22:57.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.341+0000 7f5ef17fa640 1 -- 172.21.15.31:0/574148835 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ee80ef0b0 con 0x7f5ef406a5b0 2023-11-26T07:22:57.344 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:22:57.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.341+0000 7f5efb217640 1 -- 172.21.15.31:0/574148835 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ed8066590 msgr2=0x7f5ed8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.341+0000 7f5efb217640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ed8066590 0x7f5ed8068a50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5ef41545d0 tx=0x7f5ee0008040 comp rx=0 tx=0).stop 2023-11-26T07:22:57.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.341+0000 7f5efb217640 1 -- 172.21.15.31:0/574148835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef406a5b0 msgr2=0x7f5ef414f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.341+0000 7f5efb217640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef406a5b0 0x7f5ef414f840 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80ac040 tx=0x7f5ee801ad10 comp rx=0 tx=0).stop 2023-11-26T07:22:57.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.345+0000 7f5efb217640 1 -- 172.21.15.31:0/574148835 shutdown_connections 2023-11-26T07:22:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.345+0000 7f5efb217640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5ed8066590 0x7f5ed8068a50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.345+0000 7f5efb217640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ef415b4b0 0x7f5ef40cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.345+0000 7f5efb217640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5ef40b1820 0x7f5ef4150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.345+0000 7f5efb217640 1 --2- 172.21.15.31:0/574148835 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef406a5b0 0x7f5ef414f840 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.345+0000 7f5efb217640 1 -- 172.21.15.31:0/574148835 >> 172.21.15.31:0/574148835 conn(0x7f5ef40b1d50 msgr2=0x7f5ef406e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.345+0000 7f5efb217640 1 -- 172.21.15.31:0/574148835 shutdown_connections 2023-11-26T07:22:57.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.345+0000 7f5efb217640 1 -- 172.21.15.31:0/574148835 wait complete. 2023-11-26T07:22:57.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2023-11-26T07:22:57.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.505+0000 7fcffd668640 1 Processor -- start 2023-11-26T07:22:57.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.505+0000 7fcffd668640 1 -- start start 2023-11-26T07:22:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcffd668640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146860 0x7fcff80b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcffd668640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcff80b4750 0x7fcff815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcffd668640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff806c840 con 0x7fcff8146240 2023-11-26T07:22:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcffd668640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff80b4be0 con 0x7fcff80b4750 2023-11-26T07:22:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcffd668640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff8146620 con 0x7fcff8146860 2023-11-26T07:22:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcfeffff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146860 0x7fcff80b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcff6ffd640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fcff8146240 0x7fcff8161ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:44992/0 (socket says 172.21.15.31:44992) 2023-11-26T07:22:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcff6ffd640 1 -- 172.21.15.31:0/2368068829 learned_addr learned my addr 172.21.15.31:0/2368068829 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:57.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcff77fe640 1 --2- 172.21.15.31:0/2368068829 >> v2:172.21.15.97:3300/0 conn(0x7fcff80b4750 0x7fcff815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcfeffff640 1 -- 172.21.15.31:0/2368068829 >> v2:172.21.15.97:3300/0 conn(0x7fcff80b4750 msgr2=0x7fcff815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcfeffff640 1 --2- 172.21.15.31:0/2368068829 >> v2:172.21.15.97:3300/0 conn(0x7fcff80b4750 0x7fcff815daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcfeffff640 1 -- 172.21.15.31:0/2368068829 >> v1:172.21.15.97:6789/0 conn(0x7fcff8146240 legacy=0x7fcff8161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.509+0000 7fcfeffff640 1 -- 172.21.15.31:0/2368068829 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff806bc30 con 0x7fcff8146860 2023-11-26T07:22:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcfeffff640 1 --2- 172.21.15.31:0/2368068829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146860 0x7fcff80b4210 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00097c0 tx=0x7fcfe001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fc43612766577dc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcff67fc640 1 -- 172.21.15.31:0/2368068829 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00ab650 con 0x7fcff8146860 2023-11-26T07:22:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcff67fc640 1 -- 172.21.15.31:0/2368068829 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe00ab7b0 con 0x7fcff8146860 2023-11-26T07:22:57.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcff67fc640 1 -- 172.21.15.31:0/2368068829 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00abb40 con 0x7fcff8146860 2023-11-26T07:22:57.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 -- 172.21.15.31:0/2368068829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146860 msgr2=0x7fcff80b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 --2- 172.21.15.31:0/2368068829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146860 0x7fcff80b4210 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00097c0 tx=0x7fcfe001c450 comp rx=0 tx=0).stop 2023-11-26T07:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 -- 172.21.15.31:0/2368068829 shutdown_connections 2023-11-26T07:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 --2- 172.21.15.31:0/2368068829 >> v2:172.21.15.97:3300/0 conn(0x7fcff80b4750 0x7fcff815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 --2- 172.21.15.31:0/2368068829 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146860 0x7fcff80b4210 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 -- 172.21.15.31:0/2368068829 >> 172.21.15.31:0/2368068829 conn(0x7fcff806e950 msgr2=0x7fcff806ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 -- 172.21.15.31:0/2368068829 shutdown_connections 2023-11-26T07:22:57.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 -- 172.21.15.31:0/2368068829 wait complete. 2023-11-26T07:22:57.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 Processor -- start 2023-11-26T07:22:57.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 -- start start 2023-11-26T07:22:57.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff80b4750 0x7fcff813ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146240 0x7fcff8143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff8146860 0x7fcff8143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff8139490 con 0x7fcff8146860 2023-11-26T07:22:57.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff813cf50 con 0x7fcff80b4750 2023-11-26T07:22:57.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.513+0000 7fcffd668640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff813d1f0 con 0x7fcff8146240 2023-11-26T07:22:57.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcff77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff8146860 0x7fcff8143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcff77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff8146860 0x7fcff8143a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44404/0 (socket says 172.21.15.31:44404) 2023-11-26T07:22:57.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcff77fe640 1 -- 172.21.15.31:0/4258305699 learned_addr learned my addr 172.21.15.31:0/4258305699 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:57.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcff6ffd640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff80b4750 0x7fcff813ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcfeffff640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146240 0x7fcff8143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcff77fe640 1 -- 172.21.15.31:0/4258305699 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146240 msgr2=0x7fcff8143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcff77fe640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146240 0x7fcff8143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcff77fe640 1 -- 172.21.15.31:0/4258305699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff80b4750 msgr2=0x7fcff813ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:57.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcff77fe640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff80b4750 0x7fcff813ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:57.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcff77fe640 1 -- 172.21.15.31:0/4258305699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe0009420 con 0x7fcff8146860 2023-11-26T07:22:57.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcff77fe640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff8146860 0x7fcff8143a50 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcff000efc0 tx=0x7fcff000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:57.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcfef7fe640 1 -- 172.21.15.31:0/4258305699 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0014480 con 0x7fcff8146860 2023-11-26T07:22:57.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcfef7fe640 1 -- 172.21.15.31:0/4258305699 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff000f040 con 0x7fcff8146860 2023-11-26T07:22:57.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcfef7fe640 1 -- 172.21.15.31:0/4258305699 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0014820 con 0x7fcff8146860 2023-11-26T07:22:57.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcffd668640 1 -- 172.21.15.31:0/4258305699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff813d4d0 con 0x7fcff8146860 2023-11-26T07:22:57.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcffd668640 1 -- 172.21.15.31:0/4258305699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff813da90 con 0x7fcff8146860 2023-11-26T07:22:57.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcffd668640 1 -- 172.21.15.31:0/4258305699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff80b5c60 con 0x7fcff8146860 2023-11-26T07:22:57.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.517+0000 7fcfef7fe640 1 -- 172.21.15.31:0/4258305699 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fcff0014980 con 0x7fcff8146860 2023-11-26T07:22:57.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.521+0000 7fcfef7fe640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcfd80665c0 0x7fcfd8068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:57.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.521+0000 7fcfef7fe640 1 -- 172.21.15.31:0/4258305699 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fcff0084860 con 0x7fcff8146860 2023-11-26T07:22:57.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.521+0000 7fcff6ffd640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcfd80665c0 0x7fcfd8068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:57.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.521+0000 7fcff6ffd640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcfd80665c0 0x7fcfd8068a80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8005890 tx=0x7fcfe8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:57.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.525+0000 7fcfef7fe640 1 -- 172.21.15.31:0/4258305699 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcff0051e00 con 0x7fcff8146860 2023-11-26T07:22:57.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:57.737+0000 7fcffd668640 1 -- 172.21.15.31:0/4258305699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7fcff804e850 con 0x7fcff8146860 2023-11-26T07:22:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.665+0000 7fcfef7fe640 1 -- 172.21.15.31:0/4258305699 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v297) v1 ==== 152+0+0 (secure 0 0 0) 0x7fcff0055420 con 0x7fcff8146860 2023-11-26T07:22:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 -- 172.21.15.31:0/4258305699 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcfd80665c0 msgr2=0x7fcfd8068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcfd80665c0 0x7fcfd8068a80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8005890 tx=0x7fcfe8009040 comp rx=0 tx=0).stop 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 -- 172.21.15.31:0/4258305699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff8146860 msgr2=0x7fcff8143a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff8146860 0x7fcff8143a50 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fcff000efc0 tx=0x7fcff000c490 comp rx=0 tx=0).stop 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 -- 172.21.15.31:0/4258305699 shutdown_connections 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcfd80665c0 0x7fcfd8068a80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcff8146860 0x7fcff8143a50 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcff8146240 0x7fcff8143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 --2- 172.21.15.31:0/4258305699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcff80b4750 0x7fcff813ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 -- 172.21.15.31:0/4258305699 >> 172.21.15.31:0/4258305699 conn(0x7fcff806e950 msgr2=0x7fcff80b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 -- 172.21.15.31:0/4258305699 shutdown_connections 2023-11-26T07:22:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.669+0000 7fcffd668640 1 -- 172.21.15.31:0/4258305699 wait complete. 2023-11-26T07:22:58.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:22:58.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7df265a640 1 Processor -- start 2023-11-26T07:22:58.846 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7df265a640 1 -- start start 2023-11-26T07:22:58.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7df265a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7dec0cdfb0 0x7f7dec0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7df265a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7dec0cbd50 0x7f7dec0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7df265a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dec0b2710 con 0x7f7dec15a710 2023-11-26T07:22:58.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7df265a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dec0cca80 con 0x7f7dec0cdfb0 2023-11-26T07:22:58.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7df265a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dec0b29f0 con 0x7f7dec0cbd50 2023-11-26T07:22:58.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7debfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7dec0cdfb0 0x7f7dec0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7debfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7dec0cdfb0 0x7f7dec0abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44406/0 (socket says 172.21.15.31:44406) 2023-11-26T07:22:58.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7debfff640 1 -- 172.21.15.31:0/2932464079 learned_addr learned my addr 172.21.15.31:0/2932464079 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:58.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7df0bf7640 1 --2- 172.21.15.31:0/2932464079 >> v2:172.21.15.97:3300/0 conn(0x7f7dec0cbd50 0x7f7dec0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7debfff640 1 -- 172.21.15.31:0/2932464079 >> v2:172.21.15.97:3300/0 conn(0x7f7dec0cbd50 msgr2=0x7f7dec0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:58.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7debfff640 1 --2- 172.21.15.31:0/2932464079 >> v2:172.21.15.97:3300/0 conn(0x7f7dec0cbd50 0x7f7dec0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7debfff640 1 -- 172.21.15.31:0/2932464079 >> v1:172.21.15.97:6789/0 conn(0x7f7dec15a710 legacy=0x7f7dec0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7debfff640 1 -- 172.21.15.31:0/2932464079 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec0b2cd0 con 0x7f7dec0cdfb0 2023-11-26T07:22:58.849 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.845+0000 7f7df0bf7640 1 --2- 172.21.15.31:0/2932464079 >> v2:172.21.15.97:3300/0 conn(0x7f7dec0cbd50 0x7f7dec0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:22:58.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7debfff640 1 --2- 172.21.15.31:0/2932464079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cdfb0 0x7f7dec0abf90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0aead0 tx=0x7f7ddc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd12ede9173ab541 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:58.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7deaffd640 1 -- 172.21.15.31:0/2932464079 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0aa650 con 0x7f7dec0cdfb0 2023-11-26T07:22:58.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7deaffd640 1 -- 172.21.15.31:0/2932464079 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc0aa7b0 con 0x7f7dec0cdfb0 2023-11-26T07:22:58.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7deaffd640 1 -- 172.21.15.31:0/2932464079 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0aab40 con 0x7f7dec0cdfb0 2023-11-26T07:22:58.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7df265a640 1 -- 172.21.15.31:0/2932464079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cdfb0 msgr2=0x7f7dec0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7df265a640 1 --2- 172.21.15.31:0/2932464079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cdfb0 0x7f7dec0abf90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0aead0 tx=0x7f7ddc01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7df265a640 1 -- 172.21.15.31:0/2932464079 shutdown_connections 2023-11-26T07:22:58.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7df265a640 1 --2- 172.21.15.31:0/2932464079 >> v2:172.21.15.97:3300/0 conn(0x7f7dec0cbd50 0x7f7dec0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7df265a640 1 --2- 172.21.15.31:0/2932464079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cdfb0 0x7f7dec0abf90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7df265a640 1 -- 172.21.15.31:0/2932464079 >> 172.21.15.31:0/2932464079 conn(0x7f7dec0b0cb0 msgr2=0x7f7dec0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:58.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7df265a640 1 -- 172.21.15.31:0/2932464079 shutdown_connections 2023-11-26T07:22:58.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.849+0000 7f7df265a640 1 -- 172.21.15.31:0/2932464079 wait complete. 2023-11-26T07:22:58.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 Processor -- start 2023-11-26T07:22:58.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 -- start start 2023-11-26T07:22:58.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cbd50 0x7f7dec15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dec0cdfb0 0x7f7dec15c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dec15a710 0x7f7dec157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec157100 con 0x7f7dec0cbd50 2023-11-26T07:22:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec156810 con 0x7f7dec0cdfb0 2023-11-26T07:22:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec15cf50 con 0x7f7dec15a710 2023-11-26T07:22:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7debfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cbd50 0x7f7dec15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7debfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cbd50 0x7f7dec15d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44422/0 (socket says 172.21.15.31:44422) 2023-11-26T07:22:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7debfff640 1 -- 172.21.15.31:0/1941489582 learned_addr learned my addr 172.21.15.31:0/1941489582 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7deb7fe640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dec0cdfb0 0x7f7dec15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df0bf7640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dec15a710 0x7f7dec157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7debfff640 1 -- 172.21.15.31:0/1941489582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dec15a710 msgr2=0x7f7dec157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7debfff640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dec15a710 0x7f7dec157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7debfff640 1 -- 172.21.15.31:0/1941489582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dec0cdfb0 msgr2=0x7f7dec15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7debfff640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dec0cdfb0 0x7f7dec15c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7debfff640 1 -- 172.21.15.31:0/1941489582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0094d0 con 0x7f7dec0cbd50 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df0bf7640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dec15a710 0x7f7dec157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7debfff640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cbd50 0x7f7dec15d740 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0190e0 tx=0x7f7ddc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7de8ff9640 1 -- 172.21.15.31:0/1941489582 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0b3800 con 0x7f7dec0cbd50 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7de8ff9640 1 -- 172.21.15.31:0/1941489582 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc0b3960 con 0x7f7dec0cbd50 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7de8ff9640 1 -- 172.21.15.31:0/1941489582 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0b3d10 con 0x7f7dec0cbd50 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 -- 172.21.15.31:0/1941489582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dec156ab0 con 0x7f7dec0cbd50 2023-11-26T07:22:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 -- 172.21.15.31:0/1941489582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dec15cc60 con 0x7f7dec0cbd50 2023-11-26T07:22:58.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.853+0000 7f7df265a640 1 -- 172.21.15.31:0/1941489582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dec0671f0 con 0x7f7dec0cbd50 2023-11-26T07:22:58.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.857+0000 7f7de8ff9640 1 -- 172.21.15.31:0/1941489582 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f7ddc0c6390 con 0x7f7dec0cbd50 2023-11-26T07:22:58.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.857+0000 7f7de8ff9640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dd80665e0 0x7f7dd8068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:58.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.857+0000 7f7de8ff9640 1 -- 172.21.15.31:0/1941489582 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7ddc11eb00 con 0x7f7dec0cbd50 2023-11-26T07:22:58.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.857+0000 7f7deb7fe640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dd80665e0 0x7f7dd8068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:58.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.857+0000 7f7deb7fe640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dd80665e0 0x7f7dd8068aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0adc00 tx=0x7f7de0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:58.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:58.861+0000 7f7de8ff9640 1 -- 172.21.15.31:0/1941489582 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7ddc120e60 con 0x7f7dec0cbd50 2023-11-26T07:22:59.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.017+0000 7f7df265a640 1 -- 172.21.15.31:0/1941489582 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7dec06b460 con 0x7f7dec0cbd50 2023-11-26T07:22:59.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7de8ff9640 1 -- 172.21.15.31:0/1941489582 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7ddc0ec070 con 0x7f7dec0cbd50 2023-11-26T07:22:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 -- 172.21.15.31:0/1941489582 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dd80665e0 msgr2=0x7f7dd8068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dd80665e0 0x7f7dd8068aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0adc00 tx=0x7f7de0009040 comp rx=0 tx=0).stop 2023-11-26T07:22:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 -- 172.21.15.31:0/1941489582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cbd50 msgr2=0x7f7dec15d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cbd50 0x7f7dec15d740 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0190e0 tx=0x7f7ddc01ad10 comp rx=0 tx=0).stop 2023-11-26T07:22:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 -- 172.21.15.31:0/1941489582 shutdown_connections 2023-11-26T07:22:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dd80665e0 0x7f7dd8068aa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7dec15a710 0x7f7dec157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7dec0cdfb0 0x7f7dec15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 --2- 172.21.15.31:0/1941489582 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7dec0cbd50 0x7f7dec15d740 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 -- 172.21.15.31:0/1941489582 >> 172.21.15.31:0/1941489582 conn(0x7f7dec0b0cb0 msgr2=0x7f7dec0ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 -- 172.21.15.31:0/1941489582 shutdown_connections 2023-11-26T07:22:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.025+0000 7f7df265a640 1 -- 172.21.15.31:0/1941489582 wait complete. 2023-11-26T07:22:59.048 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:22:59.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:59.148 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:22:59.244 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:22:59.244 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:22:59.244 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:59.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:22:59.253 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:59.253 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:59.263 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:22:59.263 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:22:59.263 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:59.302 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:22:59.302 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:22:59.302 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:22:59.313 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:22:59.313 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-26T07:22:59.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:22:59.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d933f640 1 Processor -- start 2023-11-26T07:22:59.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d933f640 1 -- start start 2023-11-26T07:22:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d933f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30d4072100 0x7f30d415e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d933f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30d40724e0 0x7f30d40cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d933f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f30d40ce1b0 0x7f30d40ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d933f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d415b440 con 0x7f30d4072100 2023-11-26T07:22:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d933f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d413d3f0 con 0x7f30d40ce1b0 2023-11-26T07:22:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d933f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d415ea40 con 0x7f30d40724e0 2023-11-26T07:22:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30d4072100 0x7f30d415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f30d4072100 0x7f30d415e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44438/0 (socket says 172.21.15.31:44438) 2023-11-26T07:22:59.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.485+0000 7f30d3fff640 1 -- 172.21.15.31:0/2676080032 learned_addr learned my addr 172.21.15.31:0/2676080032 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d8b3e640 1 --2- 172.21.15.31:0/2676080032 >> v2:172.21.15.97:3300/0 conn(0x7f30d40ce1b0 0x7f30d40ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d3fff640 1 -- 172.21.15.31:0/2676080032 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30d40724e0 msgr2=0x7f30d40cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d3fff640 1 --2- 172.21.15.31:0/2676080032 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30d40724e0 0x7f30d40cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d3fff640 1 -- 172.21.15.31:0/2676080032 >> v2:172.21.15.97:3300/0 conn(0x7f30d40ce1b0 msgr2=0x7f30d40ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d3fff640 1 --2- 172.21.15.31:0/2676080032 >> v2:172.21.15.97:3300/0 conn(0x7f30d40ce1b0 0x7f30d40ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d3fff640 1 -- 172.21.15.31:0/2676080032 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d415a7c0 con 0x7f30d4072100 2023-11-26T07:22:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d8b3e640 1 --2- 172.21.15.31:0/2676080032 >> v2:172.21.15.97:3300/0 conn(0x7f30d40ce1b0 0x7f30d40ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:22:59.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d3fff640 1 --2- 172.21.15.31:0/2676080032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d4072100 0x7f30d415e500 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f30d406be70 tx=0x7f30c401e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de058464805b26c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d2ffd640 1 -- 172.21.15.31:0/2676080032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c40ac680 con 0x7f30d4072100 2023-11-26T07:22:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d2ffd640 1 -- 172.21.15.31:0/2676080032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c40ac7e0 con 0x7f30d4072100 2023-11-26T07:22:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d2ffd640 1 -- 172.21.15.31:0/2676080032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c40acb70 con 0x7f30d4072100 2023-11-26T07:22:59.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d933f640 1 -- 172.21.15.31:0/2676080032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d4072100 msgr2=0x7f30d415e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d933f640 1 --2- 172.21.15.31:0/2676080032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d4072100 0x7f30d415e500 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f30d406be70 tx=0x7f30c401e650 comp rx=0 tx=0).stop 2023-11-26T07:22:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d933f640 1 -- 172.21.15.31:0/2676080032 shutdown_connections 2023-11-26T07:22:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d933f640 1 --2- 172.21.15.31:0/2676080032 >> v2:172.21.15.97:3300/0 conn(0x7f30d40ce1b0 0x7f30d40ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d933f640 1 --2- 172.21.15.31:0/2676080032 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30d40724e0 0x7f30d40cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d933f640 1 --2- 172.21.15.31:0/2676080032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d4072100 0x7f30d415e500 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d933f640 1 -- 172.21.15.31:0/2676080032 >> 172.21.15.31:0/2676080032 conn(0x7f30d40b0e00 msgr2=0x7f30d40b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.489+0000 7f30d933f640 1 -- 172.21.15.31:0/2676080032 shutdown_connections 2023-11-26T07:22:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d933f640 1 -- 172.21.15.31:0/2676080032 wait complete. 2023-11-26T07:22:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d933f640 1 Processor -- start 2023-11-26T07:22:59.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d933f640 1 -- start start 2023-11-26T07:22:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d933f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30d4072100 0x7f30d4133880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d933f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40724e0 0x7f30d4133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d933f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40ce1b0 0x7f30d40afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d933f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4134850 con 0x7f30d40724e0 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d933f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40af670 con 0x7f30d40ce1b0 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d933f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4134300 con 0x7f30d4072100 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40724e0 0x7f30d4133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40724e0 0x7f30d4133dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44440/0 (socket says 172.21.15.31:44440) 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d37fe640 1 -- 172.21.15.31:0/1046675557 learned_addr learned my addr 172.21.15.31:0/1046675557 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d3fff640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30d4072100 0x7f30d4133880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d8b3e640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40ce1b0 0x7f30d40afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d37fe640 1 -- 172.21.15.31:0/1046675557 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30d4072100 msgr2=0x7f30d4133880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d37fe640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30d4072100 0x7f30d4133880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d37fe640 1 -- 172.21.15.31:0/1046675557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40ce1b0 msgr2=0x7f30d40afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d37fe640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40ce1b0 0x7f30d40afb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d37fe640 1 -- 172.21.15.31:0/1046675557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c40094d0 con 0x7f30d40724e0 2023-11-26T07:22:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.493+0000 7f30d37fe640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40724e0 0x7f30d4133dc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f30c800b5b0 tx=0x7f30c800ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.497+0000 7f30d0ff9640 1 -- 172.21.15.31:0/1046675557 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8010890 con 0x7f30d40724e0 2023-11-26T07:22:59.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.497+0000 7f30d0ff9640 1 -- 172.21.15.31:0/1046675557 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c8010a10 con 0x7f30d40724e0 2023-11-26T07:22:59.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.497+0000 7f30d0ff9640 1 -- 172.21.15.31:0/1046675557 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8010dc0 con 0x7f30d40724e0 2023-11-26T07:22:59.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.497+0000 7f30d933f640 1 -- 172.21.15.31:0/1046675557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d413c9a0 con 0x7f30d40724e0 2023-11-26T07:22:59.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.497+0000 7f30d933f640 1 -- 172.21.15.31:0/1046675557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d40af020 con 0x7f30d40724e0 2023-11-26T07:22:59.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.497+0000 7f30d933f640 1 -- 172.21.15.31:0/1046675557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d40aee80 con 0x7f30d40724e0 2023-11-26T07:22:59.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.497+0000 7f30d0ff9640 1 -- 172.21.15.31:0/1046675557 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f30c800f3e0 con 0x7f30d40724e0 2023-11-26T07:22:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.497+0000 7f30d0ff9640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f30c0066590 0x7f30c0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.497+0000 7f30d0ff9640 1 -- 172.21.15.31:0/1046675557 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f30c8083d80 con 0x7f30d40724e0 2023-11-26T07:22:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.497+0000 7f30d3fff640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f30c0066590 0x7f30c0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.501+0000 7f30d3fff640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f30c0066590 0x7f30c0068a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f30c40b0000 tx=0x7f30c40b7040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.501+0000 7f30d0ff9640 1 -- 172.21.15.31:0/1046675557 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f30d40aee80 con 0x7f30d40724e0 2023-11-26T07:22:59.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 -- 172.21.15.31:0/1046675557 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30d40aee80 con 0x7f30d40724e0 2023-11-26T07:22:59.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d0ff9640 1 -- 172.21.15.31:0/1046675557 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f30c801ee90 con 0x7f30d40724e0 2023-11-26T07:22:59.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:22:59.696 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6833/1684629063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1684629063},{"type":"v1","addr":"172.21.15.31:6833","nonce":1684629063}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6834/1262063204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1262063204},{"type":"v1","addr":"172.21.15.97:6834","nonce":1262063204}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6961,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2863533780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6833","nonce":2863533780},{"type":"v1","addr":"172.21.15.97:6835","nonce":2863533780}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":7494,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4007135054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":4007135054},{"type":"v1","addr":"172.21.15.31:6835","nonce":4007135054}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[]} 2023-11-26T07:22:59.696 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 65 2023-11-26T07:22:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 -- 172.21.15.31:0/1046675557 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f30c0066590 msgr2=0x7f30c0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f30c0066590 0x7f30c0068a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f30c40b0000 tx=0x7f30c40b7040 comp rx=0 tx=0).stop 2023-11-26T07:22:59.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 -- 172.21.15.31:0/1046675557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40724e0 msgr2=0x7f30d4133dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40724e0 0x7f30d4133dc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f30c800b5b0 tx=0x7f30c800ba70 comp rx=0 tx=0).stop 2023-11-26T07:22:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 -- 172.21.15.31:0/1046675557 shutdown_connections 2023-11-26T07:22:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f30c0066590 0x7f30c0068a50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d40ce1b0 0x7f30d40afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40724e0 0x7f30d4133dc0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 --2- 172.21.15.31:0/1046675557 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f30d4072100 0x7f30d4133880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 -- 172.21.15.31:0/1046675557 >> 172.21.15.31:0/1046675557 conn(0x7f30d40b0e00 msgr2=0x7f30d406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 -- 172.21.15.31:0/1046675557 shutdown_connections 2023-11-26T07:22:59.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.693+0000 7f30d933f640 1 -- 172.21.15.31:0/1046675557 wait complete. 2023-11-26T07:22:59.717 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-26T07:22:59.717 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-26T07:22:59.717 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:22:59.717 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:59.718 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-26T07:22:59.718 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-26T07:22:59.718 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:22:59.718 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:59.718 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-26T07:22:59.718 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-26T07:22:59.719 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:22:59.719 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:59.719 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-26T07:22:59.719 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-26T07:22:59.719 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:22:59.720 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:22:59.730 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:22:59.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-26T07:22:59.731 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:22:59.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-26T07:22:59.736 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:22:59.736 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-26T07:22:59.737 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:22:59.737 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-26T07:22:59.780 INFO:tasks.ceph.mds.d:Started 2023-11-26T07:22:59.780 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:59.781 INFO:tasks.ceph.mds.a:Started 2023-11-26T07:22:59.781 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:59.782 INFO:tasks.ceph.mds.b:Started 2023-11-26T07:22:59.782 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:59.783 INFO:tasks.ceph.mds.c:Started 2023-11-26T07:22:59.783 DEBUG:teuthology.parallel:result is None 2023-11-26T07:22:59.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-26T07:22:59.870 INFO:tasks.ceph.mds.a.smithi031.stdout:starting mds.a at 2023-11-26T07:22:59.871 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-11-26T07:22:59.871 INFO:tasks.ceph.mds.d.smithi097.stdout:starting mds.d at 2023-11-26T07:22:59.872 INFO:tasks.ceph.mds.b.smithi097.stdout:starting mds.b at 2023-11-26T07:22:59.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8102c9a640 1 Processor -- start 2023-11-26T07:22:59.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8102c9a640 1 -- start start 2023-11-26T07:22:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8102c9a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80fc15a710 0x7f80fc0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8102c9a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80fc0b2710 con 0x7f80fc0cdfb0 2023-11-26T07:22:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8102c9a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80fc0b29f0 con 0x7f80fc0cbd50 2023-11-26T07:22:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8102c9a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80fc0cca80 con 0x7f80fc15a710 2023-11-26T07:22:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8100a36640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f80fc0cdfb0 0x7f80fc0ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49232/0 (socket says 172.21.15.31:49232) 2023-11-26T07:22:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8100a36640 1 -- 172.21.15.31:0/1674623469 learned_addr learned my addr 172.21.15.31:0/1674623469 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:22:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524294118 0 0) 0x7f80fc0b2710 con 0x7f80fc0cdfb0 2023-11-26T07:22:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80ec003530 con 0x7f80fc0cdfb0 2023-11-26T07:22:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fbfff640 1 --2- 172.21.15.31:0/1674623469 >> v2:172.21.15.97:3300/0 conn(0x7f80fc15a710 0x7f80fc0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926899354 0 0) 0x7f80ec003530 con 0x7f80fc0cdfb0 2023-11-26T07:22:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 >> v2:172.21.15.97:3300/0 conn(0x7f80fc15a710 msgr2=0x7f80fc0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 --2- 172.21.15.31:0/1674623469 >> v2:172.21.15.97:3300/0 conn(0x7f80fc15a710 0x7f80fc0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 >> v1:172.21.15.97:6789/0 conn(0x7f80fc0cbd50 legacy=0x7f80fc0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc0b2cd0 con 0x7f80fc0cdfb0 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f80f0001680 con 0x7f80fc0cdfb0 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 >> v1:172.21.15.31:6789/0 conn(0x7f80fc0cdfb0 legacy=0x7f80fc0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80ec003980 0x7f80ec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80ec0047d0 0x7f80ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80ec005190 0x7f80ec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0cca80 con 0x7f80ec0047d0 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec00c030 con 0x7f80ec003980 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec00c1f0 con 0x7f80ec005190 2023-11-26T07:22:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fbfff640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80ec0047d0 0x7f80ec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8100a36640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80ec003980 0x7f80ec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8101237640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80ec005190 0x7f80ec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fbfff640 1 -- 172.21.15.31:0/1674623469 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80ec005190 msgr2=0x7f80ec00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fbfff640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80ec005190 0x7f80ec00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fbfff640 1 -- 172.21.15.31:0/1674623469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80ec003980 msgr2=0x7f80ec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:22:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fbfff640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80ec003980 0x7f80ec003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f80fbfff640 1 -- 172.21.15.31:0/1674623469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec003530 con 0x7f80ec0047d0 2023-11-26T07:22:59.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.929+0000 7f8101237640 1 -- 172.21.15.31:0/1674623469 reap_dead start 2023-11-26T07:22:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f80fbfff640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80ec0047d0 0x7f80ec004c50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009ab0 tx=0x7f80e801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bca989dd4eb0d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e80a6440 con 0x7f80ec0047d0 2023-11-26T07:22:59.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e80a65a0 con 0x7f80ec0047d0 2023-11-26T07:22:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f80fb7fe640 1 -- 172.21.15.31:0/1674623469 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e80a6950 con 0x7f80ec0047d0 2023-11-26T07:22:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f8102c9a640 1 -- 172.21.15.31:0/1674623469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80ec0047d0 msgr2=0x7f80ec004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f8102c9a640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80ec0047d0 0x7f80ec004c50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009ab0 tx=0x7f80e801ca50 comp rx=0 tx=0).stop 2023-11-26T07:22:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f8102c9a640 1 -- 172.21.15.31:0/1674623469 shutdown_connections 2023-11-26T07:22:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f8102c9a640 1 --2- 172.21.15.31:0/1674623469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80ec0047d0 0x7f80ec004c50 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f8102c9a640 1 -- 172.21.15.31:0/1674623469 >> 172.21.15.31:0/1674623469 conn(0x7f80fc0b0cb0 msgr2=0x7f80fc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:22:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f8102c9a640 1 -- 172.21.15.31:0/1674623469 shutdown_connections 2023-11-26T07:22:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.933+0000 7f8102c9a640 1 -- 172.21.15.31:0/1674623469 wait complete. 2023-11-26T07:22:59.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8102c9a640 1 Processor -- start 2023-11-26T07:22:59.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8102c9a640 1 -- start start 2023-11-26T07:22:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8102c9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80ec0047d0 0x7f80fc0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8102c9a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80fc15ab40 0x7f80fc0cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8102c9a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc1511d0 0x7f80fc151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8102c9a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc1521b0 con 0x7f80fc1511d0 2023-11-26T07:22:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8102c9a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc15c0a0 con 0x7f80ec0047d0 2023-11-26T07:22:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8102c9a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc15b000 con 0x7f80fc15ab40 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8101237640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc1511d0 0x7f80fc151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8101237640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc1511d0 0x7f80fc151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44480/0 (socket says 172.21.15.31:44480) 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8101237640 1 -- 172.21.15.31:0/3349799588 learned_addr learned my addr 172.21.15.31:0/3349799588 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8100a36640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80ec0047d0 0x7f80fc0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f80fbfff640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80fc15ab40 0x7f80fc0cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8101237640 1 -- 172.21.15.31:0/3349799588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80fc15ab40 msgr2=0x7f80fc0cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8101237640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80fc15ab40 0x7f80fc0cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8101237640 1 -- 172.21.15.31:0/3349799588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80ec0047d0 msgr2=0x7f80fc0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8101237640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80ec0047d0 0x7f80fc0cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8101237640 1 -- 172.21.15.31:0/3349799588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc15b2e0 con 0x7f80fc1511d0 2023-11-26T07:22:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8101237640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc1511d0 0x7f80fc151680 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0aebf0 tx=0x7f80f400fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f80f97fa640 1 -- 172.21.15.31:0/3349799588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f400fec0 con 0x7f80fc1511d0 2023-11-26T07:22:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f80f97fa640 1 -- 172.21.15.31:0/3349799588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f4003b90 con 0x7f80fc1511d0 2023-11-26T07:22:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f80f97fa640 1 -- 172.21.15.31:0/3349799588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f40050d0 con 0x7f80fc1511d0 2023-11-26T07:22:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8102c9a640 1 -- 172.21.15.31:0/3349799588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80fc15b5c0 con 0x7f80fc1511d0 2023-11-26T07:22:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.937+0000 7f8102c9a640 1 -- 172.21.15.31:0/3349799588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80fc15bb80 con 0x7f80fc1511d0 2023-11-26T07:22:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.941+0000 7f8102c9a640 1 -- 172.21.15.31:0/3349799588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80fc0671f0 con 0x7f80fc1511d0 2023-11-26T07:22:59.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.941+0000 7f80f97fa640 1 -- 172.21.15.31:0/3349799588 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f80f4005230 con 0x7f80fc1511d0 2023-11-26T07:22:59.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.941+0000 7f80f97fa640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80e0066590 0x7f80e0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:22:59.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.941+0000 7f80f97fa640 1 -- 172.21.15.31:0/3349799588 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f80f4083b40 con 0x7f80fc1511d0 2023-11-26T07:22:59.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.941+0000 7f8100a36640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80e0066590 0x7f80e0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:22:59.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.941+0000 7f8100a36640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80e0066590 0x7f80e0068a50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f80f0007cc0 tx=0x7f80f0001be0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:22:59.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:22:59.945+0000 7f80f97fa640 1 -- 172.21.15.31:0/3349799588 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f80f40511f0 con 0x7f80fc1511d0 2023-11-26T07:23:00.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.117+0000 7f8102c9a640 1 -- 172.21.15.31:0/3349799588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f80fc06b460 con 0x7f80fc1511d0 2023-11-26T07:23:00.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.685+0000 7f80f97fa640 1 -- 172.21.15.31:0/3349799588 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v93) v1 ==== 97+0+0 (secure 0 0 0) 0x7f80f4054810 con 0x7f80fc1511d0 2023-11-26T07:23:00.689 INFO:teuthology.orchestra.run.smithi031.stderr: removed all blocklist entries 2023-11-26T07:23:00.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 -- 172.21.15.31:0/3349799588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80e0066590 msgr2=0x7f80e0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80e0066590 0x7f80e0068a50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f80f0007cc0 tx=0x7f80f0001be0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 -- 172.21.15.31:0/3349799588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc1511d0 msgr2=0x7f80fc151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc1511d0 0x7f80fc151680 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0aebf0 tx=0x7f80f400fc70 comp rx=0 tx=0).stop 2023-11-26T07:23:00.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 -- 172.21.15.31:0/3349799588 shutdown_connections 2023-11-26T07:23:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80e0066590 0x7f80e0068a50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80fc1511d0 0x7f80fc151680 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80fc15ab40 0x7f80fc0cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 --2- 172.21.15.31:0/3349799588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80ec0047d0 0x7f80fc0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 -- 172.21.15.31:0/3349799588 >> 172.21.15.31:0/3349799588 conn(0x7f80fc0b0cb0 msgr2=0x7f80fc0b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 -- 172.21.15.31:0/3349799588 shutdown_connections 2023-11-26T07:23:00.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.689+0000 7f8102c9a640 1 -- 172.21.15.31:0/3349799588 wait complete. 2023-11-26T07:23:00.715 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-26T07:23:00.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:23:00.766 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:23:00.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbbca06640 1 Processor -- start 2023-11-26T07:23:00.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbbca06640 1 -- start start 2023-11-26T07:23:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbbca06640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffbb8159880 0x7ffbb806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbbca06640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbb8070ef0 0x7ffbb8071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbbca06640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb8071880 con 0x7ffbb80cb230 2023-11-26T07:23:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbbca06640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80b41a0 con 0x7ffbb8070ef0 2023-11-26T07:23:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbbca06640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80b3be0 con 0x7ffbb8159880 2023-11-26T07:23:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbb6d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbb8070ef0 0x7ffbb8071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbb6d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbb8070ef0 0x7ffbb8071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44484/0 (socket says 172.21.15.31:44484) 2023-11-26T07:23:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbb6d9d640 1 -- 172.21.15.31:0/2526480752 learned_addr learned my addr 172.21.15.31:0/2526480752 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbaffff640 1 --2- 172.21.15.31:0/2526480752 >> v2:172.21.15.97:3300/0 conn(0x7ffbb8159880 0x7ffbb806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbb6d9d640 1 -- 172.21.15.31:0/2526480752 >> v2:172.21.15.97:3300/0 conn(0x7ffbb8159880 msgr2=0x7ffbb806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbb6d9d640 1 --2- 172.21.15.31:0/2526480752 >> v2:172.21.15.97:3300/0 conn(0x7ffbb8159880 0x7ffbb806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbb6d9d640 1 -- 172.21.15.31:0/2526480752 >> v1:172.21.15.31:6789/0 conn(0x7ffbb80cb230 legacy=0x7ffbb80b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbb6d9d640 1 -- 172.21.15.31:0/2526480752 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb80b3ec0 con 0x7ffbb8070ef0 2023-11-26T07:23:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.861+0000 7ffbb5d9b640 1 -- 172.21.15.31:0/2526480752 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128760252 0 0) 0x7ffba8001120 con 0x7ffbb80cb230 2023-11-26T07:23:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbb6d9d640 1 --2- 172.21.15.31:0/2526480752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 0x7ffbb8071340 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80d0010 tx=0x7ffbb001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afbd77c0c1317f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbb5d9b640 1 -- 172.21.15.31:0/2526480752 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00a5660 con 0x7ffbb8070ef0 2023-11-26T07:23:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbb5d9b640 1 -- 172.21.15.31:0/2526480752 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb00a57c0 con 0x7ffbb8070ef0 2023-11-26T07:23:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbb5d9b640 1 -- 172.21.15.31:0/2526480752 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00a5b50 con 0x7ffbb8070ef0 2023-11-26T07:23:00.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbbca06640 1 -- 172.21.15.31:0/2526480752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 msgr2=0x7ffbb8071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbbca06640 1 --2- 172.21.15.31:0/2526480752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 0x7ffbb8071340 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80d0010 tx=0x7ffbb001c8b0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbbca06640 1 -- 172.21.15.31:0/2526480752 shutdown_connections 2023-11-26T07:23:00.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbbca06640 1 --2- 172.21.15.31:0/2526480752 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 0x7ffbb8071340 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbbca06640 1 --2- 172.21.15.31:0/2526480752 >> v2:172.21.15.97:3300/0 conn(0x7ffbb8159880 0x7ffbb806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbbca06640 1 -- 172.21.15.31:0/2526480752 >> 172.21.15.31:0/2526480752 conn(0x7ffbb80b0c40 msgr2=0x7ffbb80b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbbca06640 1 -- 172.21.15.31:0/2526480752 shutdown_connections 2023-11-26T07:23:00.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.865+0000 7ffbbca06640 1 -- 172.21.15.31:0/2526480752 wait complete. 2023-11-26T07:23:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbbca06640 1 Processor -- start 2023-11-26T07:23:00.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbbca06640 1 -- start start 2023-11-26T07:23:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbbca06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 0x7ffbb814f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbbca06640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80cb230 0x7ffbb80ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbbca06640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb8159880 0x7ffbb809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbbca06640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb814fc80 con 0x7ffbb8070ef0 2023-11-26T07:23:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbbca06640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb809ba60 con 0x7ffbb8159880 2023-11-26T07:23:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbbca06640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb809bd40 con 0x7ffbb80cb230 2023-11-26T07:23:00.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 0x7ffbb814f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 0x7ffbb814f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44500/0 (socket says 172.21.15.31:44500) 2023-11-26T07:23:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb659c640 1 -- 172.21.15.31:0/2888950875 learned_addr learned my addr 172.21.15.31:0/2888950875 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb6d9d640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb8159880 0x7ffbb809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb659c640 1 -- 172.21.15.31:0/2888950875 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80cb230 msgr2=0x7ffbb80ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb659c640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80cb230 0x7ffbb80ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb659c640 1 -- 172.21.15.31:0/2888950875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb8159880 msgr2=0x7ffbb809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb659c640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb8159880 0x7ffbb809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb659c640 1 -- 172.21.15.31:0/2888950875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb0009b30 con 0x7ffbb8070ef0 2023-11-26T07:23:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb6d9d640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb8159880 0x7ffbb809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:00.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbb659c640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 0x7ffbb814f150 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ffba800c750 tx=0x7ffba800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbaeffd640 1 -- 172.21.15.31:0/2888950875 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8005270 con 0x7ffbb8070ef0 2023-11-26T07:23:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbaeffd640 1 -- 172.21.15.31:0/2888950875 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba80053f0 con 0x7ffbb8070ef0 2023-11-26T07:23:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbbca06640 1 -- 172.21.15.31:0/2888950875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb809c020 con 0x7ffbb8070ef0 2023-11-26T07:23:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbaeffd640 1 -- 172.21.15.31:0/2888950875 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800a880 con 0x7ffbb8070ef0 2023-11-26T07:23:00.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.869+0000 7ffbbca06640 1 -- 172.21.15.31:0/2888950875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb80cbb40 con 0x7ffbb8070ef0 2023-11-26T07:23:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.873+0000 7ffbbca06640 1 -- 172.21.15.31:0/2888950875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb80cbd60 con 0x7ffbb8070ef0 2023-11-26T07:23:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.873+0000 7ffbaeffd640 1 -- 172.21.15.31:0/2888950875 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ffba800a9e0 con 0x7ffbb8070ef0 2023-11-26T07:23:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.873+0000 7ffbaeffd640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffb980665e0 0x7ffb98068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.873+0000 7ffbaeffd640 1 -- 172.21.15.31:0/2888950875 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffba80830a0 con 0x7ffbb8070ef0 2023-11-26T07:23:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.873+0000 7ffbaffff640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffb980665e0 0x7ffb98068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:00.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.873+0000 7ffbaffff640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffb980665e0 0x7ffb98068aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80ad940 tx=0x7ffba0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:00.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:00.877+0000 7ffbaeffd640 1 -- 172.21.15.31:0/2888950875 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffba8050840 con 0x7ffbb8070ef0 2023-11-26T07:23:01.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 -- 172.21.15.31:0/2888950875 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ffbb8071880 con 0x7ffbb8070ef0 2023-11-26T07:23:01.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbaeffd640 1 -- 172.21.15.31:0/2888950875 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ffbb8071880 con 0x7ffbb8070ef0 2023-11-26T07:23:01.091 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:01.091 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:01.091 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_dump": [ 2023-11-26T07:23:01.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.091 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.a", 2023-11-26T07:23:01.091 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlRJmWGBAAuUAS68YFt9PaqDV8RFi+6w==", 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.b", 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlIdH1JBAAmPlAQXq78ht4pJf6I7Cd5w==", 2023-11-26T07:23:01.092 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.c", 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlfuXoHxAAf7vKZr+LVZaN6HR3qiks5w==", 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mds.d", 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlWmgkLBAAojAzX06O7kbwmj154qUGog==", 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:23:01.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.0", 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDU8GJlc/bqEBAAueOQvSeK6MW+WOT746yQzA==", 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.1", 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDX8GJlEaW6FxAA8q4rEKSJtIl/Hr2uCNSMUg==", 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.2", 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDa8GJldV90HxAAQRgcLf8ABSU4Sfb3A1Xa4g==", 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.3", 2023-11-26T07:23:01.097 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDd8GJl2erPJhAAoC71eDwCOmskb+VCLYQyUw==", 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.4", 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDj8GJllq08LxAAGuz9+Ou/ybWxdkN3PSMaEQ==", 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.5", 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDm8GJlww4eNxAAjRaBq45nrNwJPlopyOhXNg==", 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.6", 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDq8GJl2jxiAhAAhu2OULy84YRJYxTmShoVBg==", 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "osd.7", 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDt8GJld8HNCBAA3deA4HKTqRggbVc73Y8efg==", 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow profile osd", 2023-11-26T07:23:01.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile osd", 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.0", 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA==", 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow r", 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.1", 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlEBAQOhAAx5TsMN17L2fNm+j3yqUIOQ==", 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.2", 2023-11-26T07:23:01.102 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJl/CN+NhAAeyry8gIOtxwLxCQb0t1eKw==", 2023-11-26T07:23:01.103 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:23:01.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:23:01.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:23:01.103 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:23:01.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.104 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.3", 2023-11-26T07:23:01.104 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDR8GJlA9e7BBAATe3WYA79PgfLc5UHgKGkpw==", 2023-11-26T07:23:01.104 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow", 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow r", 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow rw", 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow rwx" 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "client.admin", 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDP8GJlfT+4NhAACA0t23bQtWvdFLsB9X+X8w==", 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:23:01.105 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": "allow *", 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow *", 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.x", 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlnh/pExAA6qyPM7So2BLbxbGbquLZZA==", 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:23:01.106 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: "entity": "mgr.y", 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "AQDQ8GJlYhb/DxAAx1G3Ve23IdZ7cX9nPb2WhA==", 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: "caps": { 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": "allow *", 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": "allow profile mgr", 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": "allow *" 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:01.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:23:01.108 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 -- 172.21.15.31:0/2888950875 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffb980665e0 msgr2=0x7ffb98068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffb980665e0 0x7ffb98068aa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80ad940 tx=0x7ffba0008040 comp rx=0 tx=0).stop 2023-11-26T07:23:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 -- 172.21.15.31:0/2888950875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 msgr2=0x7ffbb814f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 0x7ffbb814f150 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ffba800c750 tx=0x7ffba800cc10 comp rx=0 tx=0).stop 2023-11-26T07:23:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 -- 172.21.15.31:0/2888950875 shutdown_connections 2023-11-26T07:23:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffb980665e0 0x7ffb98068aa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbb8159880 0x7ffbb809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffbb80cb230 0x7ffbb80ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 --2- 172.21.15.31:0/2888950875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbb8070ef0 0x7ffbb814f150 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 -- 172.21.15.31:0/2888950875 >> 172.21.15.31:0/2888950875 conn(0x7ffbb80b0c40 msgr2=0x7ffbb80ade90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 -- 172.21.15.31:0/2888950875 shutdown_connections 2023-11-26T07:23:01.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.089+0000 7ffbbca06640 1 -- 172.21.15.31:0/2888950875 wait complete. 2023-11-26T07:23:01.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:01.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6f571d640 1 Processor -- start 2023-11-26T07:23:01.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6f571d640 1 -- start start 2023-11-26T07:23:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6f571d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd6f00cead0 0x7fd6f00d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6f571d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f00d22c0 con 0x7fd6f00d1eb0 2023-11-26T07:23:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6f571d640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f00cefd0 con 0x7fd6f015d7b0 2023-11-26T07:23:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6f571d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f00d17e0 con 0x7fd6f00cead0 2023-11-26T07:23:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6e67fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd6f00cead0 0x7fd6f00d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6ef7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd6f00d1eb0 0x7fd6f006bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49238/0 (socket says 172.21.15.31:49238) 2023-11-26T07:23:01.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6ef7fe640 1 -- 172.21.15.31:0/510014095 learned_addr learned my addr 172.21.15.31:0/510014095 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:01.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6e67fc640 1 -- 172.21.15.31:0/510014095 >> v1:172.21.15.97:6789/0 conn(0x7fd6f015d7b0 legacy=0x7fd6f00d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:01.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6e67fc640 1 -- 172.21.15.31:0/510014095 >> v1:172.21.15.31:6789/0 conn(0x7fd6f00d1eb0 legacy=0x7fd6f006bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6e67fc640 1 -- 172.21.15.31:0/510014095 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f00cf2b0 con 0x7fd6f00cead0 2023-11-26T07:23:01.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.253+0000 7fd6ee7fc640 1 -- 172.21.15.31:0/510014095 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822537113 0 0) 0x7fd6f00d22c0 con 0x7fd6f00d1eb0 2023-11-26T07:23:01.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6e67fc640 1 --2- 172.21.15.31:0/510014095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f00cead0 0x7fd6f00d1970 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d801c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c01f47e58dfa72 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6ee7fc640 1 -- 172.21.15.31:0/510014095 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d80a5880 con 0x7fd6f00cead0 2023-11-26T07:23:01.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6ee7fc640 1 -- 172.21.15.31:0/510014095 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d80a59e0 con 0x7fd6f00cead0 2023-11-26T07:23:01.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6ee7fc640 1 -- 172.21.15.31:0/510014095 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d80a5d70 con 0x7fd6f00cead0 2023-11-26T07:23:01.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6f571d640 1 -- 172.21.15.31:0/510014095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f00cead0 msgr2=0x7fd6f00d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6f571d640 1 --2- 172.21.15.31:0/510014095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f00cead0 0x7fd6f00d1970 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d801c7c0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6f571d640 1 -- 172.21.15.31:0/510014095 shutdown_connections 2023-11-26T07:23:01.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6f571d640 1 --2- 172.21.15.31:0/510014095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f00cead0 0x7fd6f00d1970 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6f571d640 1 -- 172.21.15.31:0/510014095 >> 172.21.15.31:0/510014095 conn(0x7fd6f00d33b0 msgr2=0x7fd6f00d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6f571d640 1 -- 172.21.15.31:0/510014095 shutdown_connections 2023-11-26T07:23:01.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.257+0000 7fd6f571d640 1 -- 172.21.15.31:0/510014095 wait complete. 2023-11-26T07:23:01.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6f571d640 1 Processor -- start 2023-11-26T07:23:01.263 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6f571d640 1 -- start start 2023-11-26T07:23:01.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6f571d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd6f00cead0 0x7fd6f015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6f571d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6f00d1eb0 0x7fd6f0151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6f571d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f015d7b0 0x7fd6f0159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6f571d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f01524e0 con 0x7fd6f015d7b0 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6f571d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f015cbc0 con 0x7fd6f00d1eb0 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6f571d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00b6d30 con 0x7fd6f00cead0 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f015d7b0 0x7fd6f0159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f015d7b0 0x7fd6f0159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44520/0 (socket says 172.21.15.31:44520) 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6ef7fe640 1 -- 172.21.15.31:0/2191356879 learned_addr learned my addr 172.21.15.31:0/2191356879 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6e67fc640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6f00d1eb0 0x7fd6f0151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6eeffd640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd6f00cead0 0x7fd6f015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6ef7fe640 1 -- 172.21.15.31:0/2191356879 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd6f00cead0 msgr2=0x7fd6f015c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6ef7fe640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd6f00cead0 0x7fd6f015c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6ef7fe640 1 -- 172.21.15.31:0/2191356879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6f00d1eb0 msgr2=0x7fd6f0151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6ef7fe640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6f00d1eb0 0x7fd6f0151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6ef7fe640 1 -- 172.21.15.31:0/2191356879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d8009a70 con 0x7fd6f015d7b0 2023-11-26T07:23:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6ef7fe640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f015d7b0 0x7fd6f0159d70 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800c410 tx=0x7fd6e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6e7fff640 1 -- 172.21.15.31:0/2191356879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e80074e0 con 0x7fd6f015d7b0 2023-11-26T07:23:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6e7fff640 1 -- 172.21.15.31:0/2191356879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e8007640 con 0x7fd6f015d7b0 2023-11-26T07:23:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6f571d640 1 -- 172.21.15.31:0/2191356879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f015a2b0 con 0x7fd6f015d7b0 2023-11-26T07:23:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6e7fff640 1 -- 172.21.15.31:0/2191356879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8004a30 con 0x7fd6f015d7b0 2023-11-26T07:23:01.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.261+0000 7fd6f571d640 1 -- 172.21.15.31:0/2191356879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f00b66e0 con 0x7fd6f015d7b0 2023-11-26T07:23:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.265+0000 7fd6f571d640 1 -- 172.21.15.31:0/2191356879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f00b6540 con 0x7fd6f015d7b0 2023-11-26T07:23:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.265+0000 7fd6e7fff640 1 -- 172.21.15.31:0/2191356879 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fd6e8004b90 con 0x7fd6f015d7b0 2023-11-26T07:23:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.265+0000 7fd6e7fff640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd6d0066590 0x7fd6d0068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.265+0000 7fd6e7fff640 1 -- 172.21.15.31:0/2191356879 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd6e8083940 con 0x7fd6f015d7b0 2023-11-26T07:23:01.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.265+0000 7fd6eeffd640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd6d0066590 0x7fd6d0068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.265+0000 7fd6eeffd640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd6d0066590 0x7fd6d0068a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00b1d90 tx=0x7fd6e0009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.269+0000 7fd6e7fff640 1 -- 172.21.15.31:0/2191356879 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd6f00b6540 con 0x7fd6f015d7b0 2023-11-26T07:23:01.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.429+0000 7fd6f571d640 1 -- 172.21.15.31:0/2191356879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6f00b6540 con 0x7fd6f015d7b0 2023-11-26T07:23:01.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.429+0000 7fd6e7fff640 1 -- 172.21.15.31:0/2191356879 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+12774 (secure 0 0 0) 0x7fd6f00b6540 con 0x7fd6f015d7b0 2023-11-26T07:23:01.433 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:01.433 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":95,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:23:00.702042+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 -- 172.21.15.31:0/2191356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd6d0066590 msgr2=0x7fd6d0068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd6d0066590 0x7fd6d0068a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00b1d90 tx=0x7fd6e0009060 comp rx=0 tx=0).stop 2023-11-26T07:23:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 -- 172.21.15.31:0/2191356879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f015d7b0 msgr2=0x7fd6f0159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f015d7b0 0x7fd6f0159d70 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800c410 tx=0x7fd6e800c8d0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 -- 172.21.15.31:0/2191356879 shutdown_connections 2023-11-26T07:23:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd6d0066590 0x7fd6d0068a50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6f015d7b0 0x7fd6f0159d70 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6f00d1eb0 0x7fd6f0151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 --2- 172.21.15.31:0/2191356879 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd6f00cead0 0x7fd6f015c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 -- 172.21.15.31:0/2191356879 >> 172.21.15.31:0/2191356879 conn(0x7fd6f00d33b0 msgr2=0x7fd6f00b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 -- 172.21.15.31:0/2191356879 shutdown_connections 2023-11-26T07:23:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.433+0000 7fd6f571d640 1 -- 172.21.15.31:0/2191356879 wait complete. 2023-11-26T07:23:01.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:01.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:01.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.597+0000 7fcf2f86b640 1 Processor -- start 2023-11-26T07:23:01.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.597+0000 7fcf2f86b640 1 -- start start 2023-11-26T07:23:01.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2f86b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcf280cb9b0 0x7fcf2806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2f86b640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcf280cbd90 0x7fcf2815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2f86b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ce5d0 0x7fcf2815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2f86b640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2815bba0 con 0x7fcf280cb9b0 2023-11-26T07:23:01.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2f86b640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf280cdd50 con 0x7fcf280cbd90 2023-11-26T07:23:01.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2f86b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28159630 con 0x7fcf280ce5d0 2023-11-26T07:23:01.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2d607640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcf280cb9b0 0x7fcf2806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2d607640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcf280cb9b0 0x7fcf2806e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44524/0 (socket says 172.21.15.31:44524) 2023-11-26T07:23:01.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2d607640 1 -- 172.21.15.31:0/2574761269 learned_addr learned my addr 172.21.15.31:0/2574761269 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:01.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2ce06640 1 --2- 172.21.15.31:0/2574761269 >> v2:172.21.15.97:3300/0 conn(0x7fcf280cbd90 0x7fcf2815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2de08640 1 --2- 172.21.15.31:0/2574761269 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ce5d0 0x7fcf2815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2d607640 1 -- 172.21.15.31:0/2574761269 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ce5d0 msgr2=0x7fcf2815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2d607640 1 --2- 172.21.15.31:0/2574761269 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ce5d0 0x7fcf2815bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2d607640 1 -- 172.21.15.31:0/2574761269 >> v2:172.21.15.97:3300/0 conn(0x7fcf280cbd90 msgr2=0x7fcf2815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2d607640 1 --2- 172.21.15.31:0/2574761269 >> v2:172.21.15.97:3300/0 conn(0x7fcf280cbd90 0x7fcf2815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2d607640 1 -- 172.21.15.31:0/2574761269 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2815f6d0 con 0x7fcf280cb9b0 2023-11-26T07:23:01.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.601+0000 7fcf2de08640 1 --2- 172.21.15.31:0/2574761269 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ce5d0 0x7fcf2815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:01.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2d607640 1 --2- 172.21.15.31:0/2574761269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280cb9b0 0x7fcf2806e3f0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fcf280b5200 tx=0x7fcf2401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58bc6e301306e07 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf1ffff640 1 -- 172.21.15.31:0/2574761269 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf240ab670 con 0x7fcf280cb9b0 2023-11-26T07:23:01.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf1ffff640 1 -- 172.21.15.31:0/2574761269 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf240ab7d0 con 0x7fcf280cb9b0 2023-11-26T07:23:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf1ffff640 1 -- 172.21.15.31:0/2574761269 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf240abb60 con 0x7fcf280cb9b0 2023-11-26T07:23:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 -- 172.21.15.31:0/2574761269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280cb9b0 msgr2=0x7fcf2806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 --2- 172.21.15.31:0/2574761269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280cb9b0 0x7fcf2806e3f0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fcf280b5200 tx=0x7fcf2401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 -- 172.21.15.31:0/2574761269 shutdown_connections 2023-11-26T07:23:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 --2- 172.21.15.31:0/2574761269 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280ce5d0 0x7fcf2815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 --2- 172.21.15.31:0/2574761269 >> v2:172.21.15.97:3300/0 conn(0x7fcf280cbd90 0x7fcf2815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 --2- 172.21.15.31:0/2574761269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280cb9b0 0x7fcf2806e3f0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 -- 172.21.15.31:0/2574761269 >> 172.21.15.31:0/2574761269 conn(0x7fcf280b0cb0 msgr2=0x7fcf280b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 -- 172.21.15.31:0/2574761269 shutdown_connections 2023-11-26T07:23:01.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 -- 172.21.15.31:0/2574761269 wait complete. 2023-11-26T07:23:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 Processor -- start 2023-11-26T07:23:01.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.605+0000 7fcf2f86b640 1 -- start start 2023-11-26T07:23:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2f86b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf280cb9b0 0x7fcf280b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2f86b640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280cbd90 0x7fcf28150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2f86b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280ce5d0 0x7fcf2809cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2f86b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28165cb0 con 0x7fcf280ce5d0 2023-11-26T07:23:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2f86b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf280b08c0 con 0x7fcf280cb9b0 2023-11-26T07:23:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2f86b640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28165f00 con 0x7fcf280cbd90 2023-11-26T07:23:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2de08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280ce5d0 0x7fcf2809cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2de08640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280ce5d0 0x7fcf2809cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44526/0 (socket says 172.21.15.31:44526) 2023-11-26T07:23:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2de08640 1 -- 172.21.15.31:0/414883195 learned_addr learned my addr 172.21.15.31:0/414883195 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:01.612 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2d607640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf280cb9b0 0x7fcf280b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2ce06640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280cbd90 0x7fcf28150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2de08640 1 -- 172.21.15.31:0/414883195 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280cbd90 msgr2=0x7fcf28150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2de08640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280cbd90 0x7fcf28150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2de08640 1 -- 172.21.15.31:0/414883195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf280cb9b0 msgr2=0x7fcf280b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2de08640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf280cb9b0 0x7fcf280b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2de08640 1 -- 172.21.15.31:0/414883195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf240094d0 con 0x7fcf280ce5d0 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2de08640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280ce5d0 0x7fcf2809cc70 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000cd50 tx=0x7fcf2000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf1dffb640 1 -- 172.21.15.31:0/414883195 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000f680 con 0x7fcf280ce5d0 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf1dffb640 1 -- 172.21.15.31:0/414883195 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2000f800 con 0x7fcf280ce5d0 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf1dffb640 1 -- 172.21.15.31:0/414883195 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000fbb0 con 0x7fcf280ce5d0 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2f86b640 1 -- 172.21.15.31:0/414883195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf281661e0 con 0x7fcf280ce5d0 2023-11-26T07:23:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2f86b640 1 -- 172.21.15.31:0/414883195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf280aebd0 con 0x7fcf280ce5d0 2023-11-26T07:23:01.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf2f86b640 1 -- 172.21.15.31:0/414883195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf280671f0 con 0x7fcf280ce5d0 2023-11-26T07:23:01.614 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.609+0000 7fcf1dffb640 1 -- 172.21.15.31:0/414883195 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fcf2000b040 con 0x7fcf280ce5d0 2023-11-26T07:23:01.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.613+0000 7fcf1dffb640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf0c066590 0x7fcf0c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.613+0000 7fcf1dffb640 1 -- 172.21.15.31:0/414883195 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcf20084590 con 0x7fcf280ce5d0 2023-11-26T07:23:01.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.613+0000 7fcf2d607640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf0c066590 0x7fcf0c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.615 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.613+0000 7fcf2d607640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf0c066590 0x7fcf0c068a50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf240af000 tx=0x7fcf240b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.617+0000 7fcf1dffb640 1 -- 172.21.15.31:0/414883195 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcf20051bf0 con 0x7fcf280ce5d0 2023-11-26T07:23:01.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 -- 172.21.15.31:0/414883195 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf2806b460 con 0x7fcf280ce5d0 2023-11-26T07:23:01.779 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf1dffb640 1 -- 172.21.15.31:0/414883195 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf2001d0a0 con 0x7fcf280ce5d0 2023-11-26T07:23:01.779 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:23:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 -- 172.21.15.31:0/414883195 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf0c066590 msgr2=0x7fcf0c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf0c066590 0x7fcf0c068a50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf240af000 tx=0x7fcf240b6040 comp rx=0 tx=0).stop 2023-11-26T07:23:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 -- 172.21.15.31:0/414883195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280ce5d0 msgr2=0x7fcf2809cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.781 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280ce5d0 0x7fcf2809cc70 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000cd50 tx=0x7fcf2000a650 comp rx=0 tx=0).stop 2023-11-26T07:23:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 -- 172.21.15.31:0/414883195 shutdown_connections 2023-11-26T07:23:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf0c066590 0x7fcf0c068a50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf280ce5d0 0x7fcf2809cc70 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcf280cbd90 0x7fcf28150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 --2- 172.21.15.31:0/414883195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf280cb9b0 0x7fcf280b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 -- 172.21.15.31:0/414883195 >> 172.21.15.31:0/414883195 conn(0x7fcf280b0cb0 msgr2=0x7fcf2815d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 -- 172.21.15.31:0/414883195 shutdown_connections 2023-11-26T07:23:01.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.777+0000 7fcf2f86b640 1 -- 172.21.15.31:0/414883195 wait complete. 2023-11-26T07:23:01.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-26T07:23:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba7d006640 1 Processor -- start 2023-11-26T07:23:01.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba7d006640 1 -- start start 2023-11-26T07:23:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba7d006640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fba78143ae0 0x7fba780b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba7d006640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba7806dfd0 0x7fba7806e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba7d006640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7806e980 con 0x7fba780adcc0 2023-11-26T07:23:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba7d006640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba780ae7f0 con 0x7fba78143ae0 2023-11-26T07:23:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba7d006640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba780b2480 con 0x7fba7806dfd0 2023-11-26T07:23:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba76d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fba78143ae0 0x7fba780b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba7759e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fba780adcc0 0x7fba780ae0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49246/0 (socket says 172.21.15.31:49246) 2023-11-26T07:23:01.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba7759e640 1 -- 172.21.15.31:0/1614919681 learned_addr learned my addr 172.21.15.31:0/1614919681 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:01.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba76d9d640 1 -- 172.21.15.31:0/1614919681 >> v2:172.21.15.97:3300/0 conn(0x7fba7806dfd0 msgr2=0x7fba7806e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:01.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba6ffff640 1 --2- 172.21.15.31:0/1614919681 >> v2:172.21.15.97:3300/0 conn(0x7fba7806dfd0 0x7fba7806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba7659c640 1 -- 172.21.15.31:0/1614919681 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518821282 0 0) 0x7fba7806e980 con 0x7fba780adcc0 2023-11-26T07:23:01.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba76d9d640 1 --2- 172.21.15.31:0/1614919681 >> v2:172.21.15.97:3300/0 conn(0x7fba7806dfd0 0x7fba7806e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba76d9d640 1 -- 172.21.15.31:0/1614919681 >> v1:172.21.15.31:6789/0 conn(0x7fba780adcc0 legacy=0x7fba780ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba76d9d640 1 -- 172.21.15.31:0/1614919681 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba78070670 con 0x7fba78143ae0 2023-11-26T07:23:01.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.941+0000 7fba6ffff640 1 --2- 172.21.15.31:0/1614919681 >> v2:172.21.15.97:3300/0 conn(0x7fba7806dfd0 0x7fba7806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba76d9d640 1 --2- 172.21.15.31:0/1614919681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 0x7fba780b1f40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fba7806cbe0 tx=0x7fba6801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d13d8268f59d659 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7659c640 1 -- 172.21.15.31:0/1614919681 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba680a7070 con 0x7fba78143ae0 2023-11-26T07:23:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7659c640 1 -- 172.21.15.31:0/1614919681 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba680ab430 con 0x7fba78143ae0 2023-11-26T07:23:01.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7659c640 1 -- 172.21.15.31:0/1614919681 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba680ab7c0 con 0x7fba78143ae0 2023-11-26T07:23:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7d006640 1 -- 172.21.15.31:0/1614919681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 msgr2=0x7fba780b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7d006640 1 --2- 172.21.15.31:0/1614919681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 0x7fba780b1f40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fba7806cbe0 tx=0x7fba6801c430 comp rx=0 tx=0).stop 2023-11-26T07:23:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7d006640 1 -- 172.21.15.31:0/1614919681 shutdown_connections 2023-11-26T07:23:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7d006640 1 --2- 172.21.15.31:0/1614919681 >> v2:172.21.15.97:3300/0 conn(0x7fba7806dfd0 0x7fba7806e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7d006640 1 --2- 172.21.15.31:0/1614919681 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 0x7fba780b1f40 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7d006640 1 -- 172.21.15.31:0/1614919681 >> 172.21.15.31:0/1614919681 conn(0x7fba780b0c40 msgr2=0x7fba780b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7d006640 1 -- 172.21.15.31:0/1614919681 shutdown_connections 2023-11-26T07:23:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.945+0000 7fba7d006640 1 -- 172.21.15.31:0/1614919681 wait complete. 2023-11-26T07:23:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7d006640 1 Processor -- start 2023-11-26T07:23:01.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7d006640 1 -- start start 2023-11-26T07:23:01.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7d006640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba7806dfd0 0x7fba7813c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7d006640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba780adcc0 0x7fba7815b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7d006640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 0x7fba7815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7d006640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba78158030 con 0x7fba78143ae0 2023-11-26T07:23:01.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7d006640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7813dc40 con 0x7fba780adcc0 2023-11-26T07:23:01.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7d006640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba78162480 con 0x7fba7806dfd0 2023-11-26T07:23:01.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 0x7fba7815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 0x7fba7815b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44548/0 (socket says 172.21.15.31:44548) 2023-11-26T07:23:01.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7759e640 1 -- 172.21.15.31:0/490639836 learned_addr learned my addr 172.21.15.31:0/490639836 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:01.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba6ffff640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba780adcc0 0x7fba7815b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba76d9d640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba7806dfd0 0x7fba7813c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7759e640 1 -- 172.21.15.31:0/490639836 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba7806dfd0 msgr2=0x7fba7813c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7759e640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba7806dfd0 0x7fba7813c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7759e640 1 -- 172.21.15.31:0/490639836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba780adcc0 msgr2=0x7fba7815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:01.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7759e640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba780adcc0 0x7fba7815b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:01.954 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7759e640 1 -- 172.21.15.31:0/490639836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba68009420 con 0x7fba78143ae0 2023-11-26T07:23:01.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba76d9d640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba7806dfd0 0x7fba7813c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:01.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba6ffff640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba780adcc0 0x7fba7815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:01.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7759e640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 0x7fba7815b8b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fba70001570 tx=0x7fba7000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba6f7fe640 1 -- 172.21.15.31:0/490639836 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7000a640 con 0x7fba78143ae0 2023-11-26T07:23:01.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba6f7fe640 1 -- 172.21.15.31:0/490639836 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7000a7c0 con 0x7fba78143ae0 2023-11-26T07:23:01.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba6f7fe640 1 -- 172.21.15.31:0/490639836 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7000ab70 con 0x7fba78143ae0 2023-11-26T07:23:01.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7d006640 1 -- 172.21.15.31:0/490639836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7813de90 con 0x7fba78143ae0 2023-11-26T07:23:01.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.949+0000 7fba7d006640 1 -- 172.21.15.31:0/490639836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7813e450 con 0x7fba78143ae0 2023-11-26T07:23:01.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.953+0000 7fba7d006640 1 -- 172.21.15.31:0/490639836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba78162630 con 0x7fba78143ae0 2023-11-26T07:23:01.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.953+0000 7fba6f7fe640 1 -- 172.21.15.31:0/490639836 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fba7000acd0 con 0x7fba78143ae0 2023-11-26T07:23:01.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.953+0000 7fba6f7fe640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba58066590 0x7fba58068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:01.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.953+0000 7fba6f7fe640 1 -- 172.21.15.31:0/490639836 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fba700832b0 con 0x7fba78143ae0 2023-11-26T07:23:01.956 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.953+0000 7fba76d9d640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba58066590 0x7fba58068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:01.957 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.953+0000 7fba76d9d640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba58066590 0x7fba58068a50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fba680af000 tx=0x7fba680a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:01.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:01.957+0000 7fba6f7fe640 1 -- 172.21.15.31:0/490639836 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fba78162630 con 0x7fba78143ae0 2023-11-26T07:23:02.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 -- 172.21.15.31:0/490639836 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fba781625f0 con 0x7fba78143ae0 2023-11-26T07:23:02.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba6f7fe640 1 -- 172.21.15.31:0/490639836 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v95) v1 ==== 84+0+65 (secure 0 0 0) 0x7fba70018090 con 0x7fba78143ae0 2023-11-26T07:23:02.119 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:02.120 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:23:02.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:02.120 INFO:teuthology.orchestra.run.smithi031.stdout: "poolnum": 1, 2023-11-26T07:23:02.120 INFO:teuthology.orchestra.run.smithi031.stdout: "poolname": ".mgr" 2023-11-26T07:23:02.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:02.120 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 -- 172.21.15.31:0/490639836 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba58066590 msgr2=0x7fba58068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba58066590 0x7fba58068a50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fba680af000 tx=0x7fba680a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 -- 172.21.15.31:0/490639836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 msgr2=0x7fba7815b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 0x7fba7815b8b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fba70001570 tx=0x7fba7000cca0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 -- 172.21.15.31:0/490639836 shutdown_connections 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba58066590 0x7fba58068a50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba78143ae0 0x7fba7815b8b0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba780adcc0 0x7fba7815b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 --2- 172.21.15.31:0/490639836 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba7806dfd0 0x7fba7813c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 -- 172.21.15.31:0/490639836 >> 172.21.15.31:0/490639836 conn(0x7fba780b0c40 msgr2=0x7fba78164510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 -- 172.21.15.31:0/490639836 shutdown_connections 2023-11-26T07:23:02.122 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.117+0000 7fba7d006640 1 -- 172.21.15.31:0/490639836 wait complete. 2023-11-26T07:23:02.140 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-26T07:23:02.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-26T07:23:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.285+0000 7fa835faa640 1 Processor -- start 2023-11-26T07:23:02.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.285+0000 7fa835faa640 1 -- start start 2023-11-26T07:23:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa835faa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa83015a710 0x7fa8300abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa835faa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa8300cbd50 0x7fa8300cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa835faa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8300b2710 con 0x7fa8300cdfb0 2023-11-26T07:23:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa835faa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8300cca80 con 0x7fa8300cbd50 2023-11-26T07:23:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa835faa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8300b29f0 con 0x7fa83015a710 2023-11-26T07:23:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa82f7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa8300cdfb0 0x7fa8300ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49256/0 (socket says 172.21.15.31:49256) 2023-11-26T07:23:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa82f7fe640 1 -- 172.21.15.31:0/307296536 learned_addr learned my addr 172.21.15.31:0/307296536 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:02.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa82ffff640 1 --2- 172.21.15.31:0/307296536 >> v2:172.21.15.31:3300/0 conn(0x7fa8300cbd50 0x7fa8300cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:02.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa82effd640 1 --2- 172.21.15.31:0/307296536 >> v2:172.21.15.97:3300/0 conn(0x7fa83015a710 0x7fa8300abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:02.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa82ffff640 1 -- 172.21.15.31:0/307296536 >> v2:172.21.15.97:3300/0 conn(0x7fa83015a710 msgr2=0x7fa8300abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:02.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa82ffff640 1 --2- 172.21.15.31:0/307296536 >> v2:172.21.15.97:3300/0 conn(0x7fa83015a710 0x7fa8300abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa82e7fc640 1 -- 172.21.15.31:0/307296536 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118982978 0 0) 0x7fa8300b2710 con 0x7fa8300cdfb0 2023-11-26T07:23:02.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa82ffff640 1 -- 172.21.15.31:0/307296536 >> v1:172.21.15.31:6789/0 conn(0x7fa8300cdfb0 legacy=0x7fa8300ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:02.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa82ffff640 1 -- 172.21.15.31:0/307296536 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8300b2cd0 con 0x7fa8300cbd50 2023-11-26T07:23:02.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.289+0000 7fa82effd640 1 --2- 172.21.15.31:0/307296536 >> v2:172.21.15.97:3300/0 conn(0x7fa83015a710 0x7fa8300abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:02.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa82ffff640 1 --2- 172.21.15.31:0/307296536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8300cbd50 0x7fa8300cc1e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa8300d0040 tx=0x7fa82801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a7a57cba284410 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:02.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa82e7fc640 1 -- 172.21.15.31:0/307296536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280a59c0 con 0x7fa8300cbd50 2023-11-26T07:23:02.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa82e7fc640 1 -- 172.21.15.31:0/307296536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8280a5b20 con 0x7fa8300cbd50 2023-11-26T07:23:02.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa82e7fc640 1 -- 172.21.15.31:0/307296536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280ad440 con 0x7fa8300cbd50 2023-11-26T07:23:02.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa835faa640 1 -- 172.21.15.31:0/307296536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8300cbd50 msgr2=0x7fa8300cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:02.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa835faa640 1 --2- 172.21.15.31:0/307296536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8300cbd50 0x7fa8300cc1e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa8300d0040 tx=0x7fa82801c8e0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa835faa640 1 -- 172.21.15.31:0/307296536 shutdown_connections 2023-11-26T07:23:02.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa835faa640 1 --2- 172.21.15.31:0/307296536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8300cbd50 0x7fa8300cc1e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa835faa640 1 --2- 172.21.15.31:0/307296536 >> v2:172.21.15.97:3300/0 conn(0x7fa83015a710 0x7fa8300abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa835faa640 1 -- 172.21.15.31:0/307296536 >> 172.21.15.31:0/307296536 conn(0x7fa8300b0cb0 msgr2=0x7fa8300b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:02.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa835faa640 1 -- 172.21.15.31:0/307296536 shutdown_connections 2023-11-26T07:23:02.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa835faa640 1 -- 172.21.15.31:0/307296536 wait complete. 2023-11-26T07:23:02.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa835faa640 1 Processor -- start 2023-11-26T07:23:02.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.293+0000 7fa835faa640 1 -- start start 2023-11-26T07:23:02.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa835faa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8300cbd50 0x7fa83015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:02.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa835faa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8300cdfb0 0x7fa83014be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:02.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa835faa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83015a710 0x7fa830153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:02.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa835faa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83015c0c0 con 0x7fa83015a710 2023-11-26T07:23:02.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa835faa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83015b260 con 0x7fa8300cbd50 2023-11-26T07:23:02.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa835faa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83014c3b0 con 0x7fa8300cdfb0 2023-11-26T07:23:02.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83015a710 0x7fa830153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:02.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83015a710 0x7fa830153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44554/0 (socket says 172.21.15.31:44554) 2023-11-26T07:23:02.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82ffff640 1 -- 172.21.15.31:0/1775714847 learned_addr learned my addr 172.21.15.31:0/1775714847 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:02.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82f7fe640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8300cbd50 0x7fa83015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:02.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82effd640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8300cdfb0 0x7fa83014be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:02.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82ffff640 1 -- 172.21.15.31:0/1775714847 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8300cdfb0 msgr2=0x7fa83014be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:02.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82ffff640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8300cdfb0 0x7fa83014be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82ffff640 1 -- 172.21.15.31:0/1775714847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8300cbd50 msgr2=0x7fa83015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:02.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82ffff640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8300cbd50 0x7fa83015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:02.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82ffff640 1 -- 172.21.15.31:0/1775714847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa828009a70 con 0x7fa83015a710 2023-11-26T07:23:02.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82f7fe640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8300cbd50 0x7fa83015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:02.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa82ffff640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83015a710 0x7fa830153d10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa8280194f0 tx=0x7fa828019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:02.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa817fff640 1 -- 172.21.15.31:0/1775714847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280add70 con 0x7fa83015a710 2023-11-26T07:23:02.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa817fff640 1 -- 172.21.15.31:0/1775714847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8280b7410 con 0x7fa83015a710 2023-11-26T07:23:02.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa835faa640 1 -- 172.21.15.31:0/1775714847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83015b4b0 con 0x7fa83015a710 2023-11-26T07:23:02.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa817fff640 1 -- 172.21.15.31:0/1775714847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280b77c0 con 0x7fa83015a710 2023-11-26T07:23:02.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa835faa640 1 -- 172.21.15.31:0/1775714847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83015ba70 con 0x7fa83015a710 2023-11-26T07:23:02.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa835faa640 1 -- 172.21.15.31:0/1775714847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8300671f0 con 0x7fa83015a710 2023-11-26T07:23:02.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.297+0000 7fa817fff640 1 -- 172.21.15.31:0/1775714847 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fa8280b4050 con 0x7fa83015a710 2023-11-26T07:23:02.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.301+0000 7fa817fff640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa81c0665c0 0x7fa81c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:02.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.301+0000 7fa817fff640 1 -- 172.21.15.31:0/1775714847 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa82811eae0 con 0x7fa83015a710 2023-11-26T07:23:02.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.301+0000 7fa82f7fe640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa81c0665c0 0x7fa81c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:02.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.301+0000 7fa82f7fe640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa81c0665c0 0x7fa81c068a80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa820007fd0 tx=0x7fa820007c20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:02.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.305+0000 7fa817fff640 1 -- 172.21.15.31:0/1775714847 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa8280ec070 con 0x7fa83015a710 2023-11-26T07:23:02.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:02.465+0000 7fa835faa640 1 -- 172.21.15.31:0/1775714847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa83006b460 con 0x7fa83015a710 2023-11-26T07:23:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.085+0000 7fa817fff640 1 -- 172.21.15.31:0/1775714847 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v96) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa8280ef690 con 0x7fa83015a710 2023-11-26T07:23:03.089 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' created 2023-11-26T07:23:03.090 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 -- 172.21.15.31:0/1775714847 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa81c0665c0 msgr2=0x7fa81c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa81c0665c0 0x7fa81c068a80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa820007fd0 tx=0x7fa820007c20 comp rx=0 tx=0).stop 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 -- 172.21.15.31:0/1775714847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83015a710 msgr2=0x7fa830153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83015a710 0x7fa830153d10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa8280194f0 tx=0x7fa828019680 comp rx=0 tx=0).stop 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 -- 172.21.15.31:0/1775714847 shutdown_connections 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa81c0665c0 0x7fa81c068a80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83015a710 0x7fa830153d10 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa8300cdfb0 0x7fa83014be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 --2- 172.21.15.31:0/1775714847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa8300cbd50 0x7fa83015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 -- 172.21.15.31:0/1775714847 >> 172.21.15.31:0/1775714847 conn(0x7fa8300b0cb0 msgr2=0x7fa8300b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 -- 172.21.15.31:0/1775714847 shutdown_connections 2023-11-26T07:23:03.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.089+0000 7fa835faa640 1 -- 172.21.15.31:0/1775714847 wait complete. 2023-11-26T07:23:03.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-26T07:23:03.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe522ff5640 1 Processor -- start 2023-11-26T07:23:03.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe522ff5640 1 -- start start 2023-11-26T07:23:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe522ff5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe51c146820 0x7fe51c070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe522ff5640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe51c15eb50 0x7fe51c0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe522ff5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51c147190 con 0x7fe51c0b4210 2023-11-26T07:23:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe522ff5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51c0b4be0 con 0x7fe51c146820 2023-11-26T07:23:03.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe522ff5640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51c146c00 con 0x7fe51c15eb50 2023-11-26T07:23:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe520d91640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe51c146820 0x7fe51c070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe51bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe51c0b4210 0x7fe51c161180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:49260/0 (socket says 172.21.15.31:49260) 2023-11-26T07:23:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe520d91640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe51c146820 0x7fe51c070a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44558/0 (socket says 172.21.15.31:44558) 2023-11-26T07:23:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.265+0000 7fe51bfff640 1 -- 172.21.15.31:0/4226685464 learned_addr learned my addr 172.21.15.31:0/4226685464 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.269+0000 7fe521592640 1 --2- 172.21.15.31:0/4226685464 >> v2:172.21.15.97:3300/0 conn(0x7fe51c15eb50 0x7fe51c0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.269+0000 7fe520d91640 1 -- 172.21.15.31:0/4226685464 >> v2:172.21.15.97:3300/0 conn(0x7fe51c15eb50 msgr2=0x7fe51c0b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.269+0000 7fe520d91640 1 --2- 172.21.15.31:0/4226685464 >> v2:172.21.15.97:3300/0 conn(0x7fe51c15eb50 0x7fe51c0b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.269+0000 7fe520d91640 1 -- 172.21.15.31:0/4226685464 >> v1:172.21.15.31:6789/0 conn(0x7fe51c0b4210 legacy=0x7fe51c161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.269+0000 7fe520d91640 1 -- 172.21.15.31:0/4226685464 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c06d030 con 0x7fe51c146820 2023-11-26T07:23:03.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.269+0000 7fe51b7fe640 1 -- 172.21.15.31:0/4226685464 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941472193 0 0) 0x7fe51c147190 con 0x7fe51c0b4210 2023-11-26T07:23:03.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.269+0000 7fe520d91640 1 --2- 172.21.15.31:0/4226685464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 0x7fe51c070a50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0d22d0 tx=0x7fe51001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234f9c4671fba5c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:03.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.269+0000 7fe51b7fe640 1 -- 172.21.15.31:0/4226685464 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510004230 con 0x7fe51c146820 2023-11-26T07:23:03.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.269+0000 7fe51b7fe640 1 -- 172.21.15.31:0/4226685464 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5100a7050 con 0x7fe51c146820 2023-11-26T07:23:03.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.269+0000 7fe51b7fe640 1 -- 172.21.15.31:0/4226685464 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510003620 con 0x7fe51c146820 2023-11-26T07:23:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 -- 172.21.15.31:0/4226685464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 msgr2=0x7fe51c070a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 --2- 172.21.15.31:0/4226685464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 0x7fe51c070a50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0d22d0 tx=0x7fe51001c430 comp rx=0 tx=0).stop 2023-11-26T07:23:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 -- 172.21.15.31:0/4226685464 shutdown_connections 2023-11-26T07:23:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 --2- 172.21.15.31:0/4226685464 >> v2:172.21.15.97:3300/0 conn(0x7fe51c15eb50 0x7fe51c0b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:03.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 --2- 172.21.15.31:0/4226685464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 0x7fe51c070a50 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:03.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 -- 172.21.15.31:0/4226685464 >> 172.21.15.31:0/4226685464 conn(0x7fe51c0b2f10 msgr2=0x7fe51c0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:03.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 -- 172.21.15.31:0/4226685464 shutdown_connections 2023-11-26T07:23:03.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 -- 172.21.15.31:0/4226685464 wait complete. 2023-11-26T07:23:03.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 Processor -- start 2023-11-26T07:23:03.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 -- start start 2023-11-26T07:23:03.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c0b4210 0x7fe51c13a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 0x7fe51c13b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe51c15eb50 0x7fe51c13edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c13fe60 con 0x7fe51c146820 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c13ba60 con 0x7fe51c0b4210 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe522ff5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c13f2f0 con 0x7fe51c15eb50 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 0x7fe51c13b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe51bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 0x7fe51c13b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44566/0 (socket says 172.21.15.31:44566) 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe51bfff640 1 -- 172.21.15.31:0/2037300176 learned_addr learned my addr 172.21.15.31:0/2037300176 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe520d91640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c0b4210 0x7fe51c13a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe521592640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe51c15eb50 0x7fe51c13edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe51bfff640 1 -- 172.21.15.31:0/2037300176 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe51c15eb50 msgr2=0x7fe51c13edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:03.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe51bfff640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe51c15eb50 0x7fe51c13edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:03.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe51bfff640 1 -- 172.21.15.31:0/2037300176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c0b4210 msgr2=0x7fe51c13a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:03.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe51bfff640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c0b4210 0x7fe51c13a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:03.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe51bfff640 1 -- 172.21.15.31:0/2037300176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe510009420 con 0x7fe51c146820 2023-11-26T07:23:03.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.273+0000 7fe520d91640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c0b4210 0x7fe51c13a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:03.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.277+0000 7fe51bfff640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 0x7fe51c13b520 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe5080011f0 tx=0x7fe50800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:03.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.277+0000 7fe5197fa640 1 -- 172.21.15.31:0/2037300176 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5080074e0 con 0x7fe51c146820 2023-11-26T07:23:03.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.277+0000 7fe5197fa640 1 -- 172.21.15.31:0/2037300176 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe508007640 con 0x7fe51c146820 2023-11-26T07:23:03.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.277+0000 7fe5197fa640 1 -- 172.21.15.31:0/2037300176 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe508004a10 con 0x7fe51c146820 2023-11-26T07:23:03.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.277+0000 7fe522ff5640 1 -- 172.21.15.31:0/2037300176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c13f5c0 con 0x7fe51c146820 2023-11-26T07:23:03.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.277+0000 7fe522ff5640 1 -- 172.21.15.31:0/2037300176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c13fb80 con 0x7fe51c146820 2023-11-26T07:23:03.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.277+0000 7fe522ff5640 1 -- 172.21.15.31:0/2037300176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c04e850 con 0x7fe51c146820 2023-11-26T07:23:03.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.277+0000 7fe5197fa640 1 -- 172.21.15.31:0/2037300176 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fe50801b360 con 0x7fe51c146820 2023-11-26T07:23:03.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.277+0000 7fe5197fa640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066590 0x7fe500068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.277+0000 7fe5197fa640 1 -- 172.21.15.31:0/2037300176 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe5080833d0 con 0x7fe51c146820 2023-11-26T07:23:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.281+0000 7fe520d91640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066590 0x7fe500068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:03.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.281+0000 7fe520d91640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066590 0x7fe500068a50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5100ad000 tx=0x7fe5100a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:03.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.281+0000 7fe5197fa640 1 -- 172.21.15.31:0/2037300176 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe508050900 con 0x7fe51c146820 2023-11-26T07:23:03.446 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:03.445+0000 7fe522ff5640 1 -- 172.21.15.31:0/2037300176 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe51c15b990 con 0x7fe51c146820 2023-11-26T07:23:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.089+0000 7fe5197fa640 1 -- 172.21.15.31:0/2037300176 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v97) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe5080180d0 con 0x7fe51c146820 2023-11-26T07:23:04.091 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' created 2023-11-26T07:23:04.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 -- 172.21.15.31:0/2037300176 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066590 msgr2=0x7fe500068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:04.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066590 0x7fe500068a50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5100ad000 tx=0x7fe5100a6000 comp rx=0 tx=0).stop 2023-11-26T07:23:04.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 -- 172.21.15.31:0/2037300176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 msgr2=0x7fe51c13b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 0x7fe51c13b520 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fe5080011f0 tx=0x7fe50800c9b0 comp rx=0 tx=0).stop 2023-11-26T07:23:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 -- 172.21.15.31:0/2037300176 shutdown_connections 2023-11-26T07:23:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066590 0x7fe500068a50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe51c15eb50 0x7fe51c13edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51c146820 0x7fe51c13b520 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 --2- 172.21.15.31:0/2037300176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe51c0b4210 0x7fe51c13a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 -- 172.21.15.31:0/2037300176 >> 172.21.15.31:0/2037300176 conn(0x7fe51c0b2f10 msgr2=0x7fe51c15fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:04.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 -- 172.21.15.31:0/2037300176 shutdown_connections 2023-11-26T07:23:04.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.093+0000 7fe522ff5640 1 -- 172.21.15.31:0/2037300176 wait complete. 2023-11-26T07:23:04.136 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-26T07:23:04.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59f59e640 1 Processor -- start 2023-11-26T07:23:04.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59f59e640 1 -- start start 2023-11-26T07:23:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59f59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5a00cdf30 0x7fd5a00ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59f59e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a00d27f0 con 0x7fd5a00b8bf0 2023-11-26T07:23:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59f59e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a00ce850 con 0x7fd5a015d6b0 2023-11-26T07:23:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59f59e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a00d1be0 con 0x7fd5a00cdf30 2023-11-26T07:23:04.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59e59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5a00cdf30 0x7fd5a00ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59e59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd5a00cdf30 0x7fd5a00ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36166/0 (socket says 172.21.15.31:36166) 2023-11-26T07:23:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59ed9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd5a00b8bf0 0x7fd5a00b6390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60524/0 (socket says 172.21.15.31:60524) 2023-11-26T07:23:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59e59c640 1 -- 172.21.15.31:0/610146924 learned_addr learned my addr 172.21.15.31:0/610146924 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59e59c640 1 -- 172.21.15.31:0/610146924 >> v1:172.21.15.97:6789/0 conn(0x7fd5a015d6b0 legacy=0x7fd5a00d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59e59c640 1 -- 172.21.15.31:0/610146924 >> v1:172.21.15.31:6789/0 conn(0x7fd5a00b8bf0 legacy=0x7fd5a00b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59e59c640 1 -- 172.21.15.31:0/610146924 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a00d1ec0 con 0x7fd5a00cdf30 2023-11-26T07:23:04.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.285+0000 7fd59d59a640 1 -- 172.21.15.31:0/610146924 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184116406 0 0) 0x7fd5a00d27f0 con 0x7fd5a00b8bf0 2023-11-26T07:23:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59e59c640 1 --2- 172.21.15.31:0/610146924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a00cdf30 0x7fd5a00ce310 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd590009870 tx=0x7fd59001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7cbd89910ed3f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:04.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59d59a640 1 -- 172.21.15.31:0/610146924 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900a7070 con 0x7fd5a00cdf30 2023-11-26T07:23:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59d59a640 1 -- 172.21.15.31:0/610146924 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5900ab430 con 0x7fd5a00cdf30 2023-11-26T07:23:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59d59a640 1 -- 172.21.15.31:0/610146924 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900ab7c0 con 0x7fd5a00cdf30 2023-11-26T07:23:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59f59e640 1 -- 172.21.15.31:0/610146924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a00cdf30 msgr2=0x7fd5a00ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59f59e640 1 --2- 172.21.15.31:0/610146924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a00cdf30 0x7fd5a00ce310 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fd590009870 tx=0x7fd59001c540 comp rx=0 tx=0).stop 2023-11-26T07:23:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59f59e640 1 -- 172.21.15.31:0/610146924 shutdown_connections 2023-11-26T07:23:04.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59f59e640 1 --2- 172.21.15.31:0/610146924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a00cdf30 0x7fd5a00ce310 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59f59e640 1 -- 172.21.15.31:0/610146924 >> 172.21.15.31:0/610146924 conn(0x7fd5a0066470 msgr2=0x7fd5a0066880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59f59e640 1 -- 172.21.15.31:0/610146924 shutdown_connections 2023-11-26T07:23:04.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.289+0000 7fd59f59e640 1 -- 172.21.15.31:0/610146924 wait complete. 2023-11-26T07:23:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59f59e640 1 Processor -- start 2023-11-26T07:23:04.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59f59e640 1 -- start start 2023-11-26T07:23:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59f59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5a00b8bf0 0x7fd5a00cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a00cdf30 0x7fd5a014b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59f59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a015d6b0 0x7fd5a00b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59f59e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00cd070 con 0x7fd5a015d6b0 2023-11-26T07:23:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59f59e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a0155ba0 con 0x7fd5a00cdf30 2023-11-26T07:23:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59f59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a0155e80 con 0x7fd5a00b8bf0 2023-11-26T07:23:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a015d6b0 0x7fd5a00b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a015d6b0 0x7fd5a00b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36180/0 (socket says 172.21.15.31:36180) 2023-11-26T07:23:04.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59ed9d640 1 -- 172.21.15.31:0/3782252619 learned_addr learned my addr 172.21.15.31:0/3782252619 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59e59c640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5a00b8bf0 0x7fd5a00cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:04.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59dd9b640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a00cdf30 0x7fd5a014b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59ed9d640 1 -- 172.21.15.31:0/3782252619 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5a00b8bf0 msgr2=0x7fd5a00cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59ed9d640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5a00b8bf0 0x7fd5a00cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59ed9d640 1 -- 172.21.15.31:0/3782252619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a00cdf30 msgr2=0x7fd5a014b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59ed9d640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a00cdf30 0x7fd5a014b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59ed9d640 1 -- 172.21.15.31:0/3782252619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5900094d0 con 0x7fd5a015d6b0 2023-11-26T07:23:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59ed9d640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a015d6b0 0x7fd5a00b25e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd59800c780 tx=0x7fd59800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd58effd640 1 -- 172.21.15.31:0/3782252619 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59800ce90 con 0x7fd5a015d6b0 2023-11-26T07:23:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd58effd640 1 -- 172.21.15.31:0/3782252619 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59800a640 con 0x7fd5a015d6b0 2023-11-26T07:23:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd58effd640 1 -- 172.21.15.31:0/3782252619 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59800a9d0 con 0x7fd5a015d6b0 2023-11-26T07:23:04.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59f59e640 1 -- 172.21.15.31:0/3782252619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a00ccd90 con 0x7fd5a015d6b0 2023-11-26T07:23:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.293+0000 7fd59f59e640 1 -- 172.21.15.31:0/3782252619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a00cc9b0 con 0x7fd5a015d6b0 2023-11-26T07:23:04.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.297+0000 7fd59f59e640 1 -- 172.21.15.31:0/3782252619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a0074990 con 0x7fd5a015d6b0 2023-11-26T07:23:04.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.297+0000 7fd58effd640 1 -- 172.21.15.31:0/3782252619 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fd59800ab30 con 0x7fd5a015d6b0 2023-11-26T07:23:04.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.297+0000 7fd58effd640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd580066590 0x7fd580068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:04.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.297+0000 7fd58effd640 1 -- 172.21.15.31:0/3782252619 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd598083870 con 0x7fd5a015d6b0 2023-11-26T07:23:04.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.297+0000 7fd59e59c640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd580066590 0x7fd580068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:04.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.297+0000 7fd59e59c640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd580066590 0x7fd580068a50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5900af000 tx=0x7fd5900b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.301+0000 7fd58effd640 1 -- 172.21.15.31:0/3782252619 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd598050cd0 con 0x7fd5a015d6b0 2023-11-26T07:23:04.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:04.493+0000 7fd59f59e640 1 -- 172.21.15.31:0/3782252619 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd5a01f87c0 con 0x7fd5a015d6b0 2023-11-26T07:23:05.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.125+0000 7fd58effd640 1 -- 172.21.15.31:0/3782252619 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd598018020 con 0x7fd5a015d6b0 2023-11-26T07:23:05.131 INFO:teuthology.orchestra.run.smithi031.stderr:new fs with metadata pool 16 and data pool 17 2023-11-26T07:23:05.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 -- 172.21.15.31:0/3782252619 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd580066590 msgr2=0x7fd580068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd580066590 0x7fd580068a50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5900af000 tx=0x7fd5900b6040 comp rx=0 tx=0).stop 2023-11-26T07:23:05.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 -- 172.21.15.31:0/3782252619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a015d6b0 msgr2=0x7fd5a00b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a015d6b0 0x7fd5a00b25e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fd59800c780 tx=0x7fd59800cc40 comp rx=0 tx=0).stop 2023-11-26T07:23:05.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 -- 172.21.15.31:0/3782252619 shutdown_connections 2023-11-26T07:23:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd580066590 0x7fd580068a50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a015d6b0 0x7fd5a00b25e0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a00cdf30 0x7fd5a014b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 --2- 172.21.15.31:0/3782252619 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd5a00b8bf0 0x7fd5a00cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 -- 172.21.15.31:0/3782252619 >> 172.21.15.31:0/3782252619 conn(0x7fd5a0066470 msgr2=0x7fd5a00b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 -- 172.21.15.31:0/3782252619 shutdown_connections 2023-11-26T07:23:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.149+0000 7fd59f59e640 1 -- 172.21.15.31:0/3782252619 wait complete. 2023-11-26T07:23:05.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd99a3f8640 1 Processor -- start 2023-11-26T07:23:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd99a3f8640 1 -- start start 2023-11-26T07:23:05.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd99a3f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd99415d8b0 0x7fd9940b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd99a3f8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9940cecd0 0x7fd99406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd99a3f8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9940b0a70 con 0x7fd9940b5120 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd99a3f8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9940cddb0 con 0x7fd9940cecd0 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd99a3f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940b4100 con 0x7fd99415d8b0 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd998995640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9940cecd0 0x7fd99406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd9937fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fd9940b5120 0x7fd9940ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60530/0 (socket says 172.21.15.31:60530) 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd9937fe640 1 -- 172.21.15.31:0/3701898460 learned_addr learned my addr 172.21.15.31:0/3701898460 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd993fff640 1 --2- 172.21.15.31:0/3701898460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd99415d8b0 0x7fd9940b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd998995640 1 -- 172.21.15.31:0/3701898460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd99415d8b0 msgr2=0x7fd9940b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd998995640 1 --2- 172.21.15.31:0/3701898460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd99415d8b0 0x7fd9940b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd998995640 1 -- 172.21.15.31:0/3701898460 >> v1:172.21.15.31:6789/0 conn(0x7fd9940b5120 legacy=0x7fd9940ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd998995640 1 -- 172.21.15.31:0/3701898460 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940b43e0 con 0x7fd9940cecd0 2023-11-26T07:23:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.325+0000 7fd992ffd640 1 -- 172.21.15.31:0/3701898460 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 54781480 0 0) 0x7fd9940b0a70 con 0x7fd9940b5120 2023-11-26T07:23:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd998995640 1 --2- 172.21.15.31:0/3701898460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 0x7fd99406a5b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009d90 tx=0x7fd98c01f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8352d70fda016b3a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd992ffd640 1 -- 172.21.15.31:0/3701898460 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0af600 con 0x7fd9940cecd0 2023-11-26T07:23:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd992ffd640 1 -- 172.21.15.31:0/3701898460 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c0ab050 con 0x7fd9940cecd0 2023-11-26T07:23:05.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd992ffd640 1 -- 172.21.15.31:0/3701898460 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0af980 con 0x7fd9940cecd0 2023-11-26T07:23:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd99a3f8640 1 -- 172.21.15.31:0/3701898460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 msgr2=0x7fd99406a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd99a3f8640 1 --2- 172.21.15.31:0/3701898460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 0x7fd99406a5b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009d90 tx=0x7fd98c01f130 comp rx=0 tx=0).stop 2023-11-26T07:23:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd99a3f8640 1 -- 172.21.15.31:0/3701898460 shutdown_connections 2023-11-26T07:23:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd99a3f8640 1 --2- 172.21.15.31:0/3701898460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 0x7fd99406a5b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd99a3f8640 1 --2- 172.21.15.31:0/3701898460 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd99415d8b0 0x7fd9940b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd99a3f8640 1 -- 172.21.15.31:0/3701898460 >> 172.21.15.31:0/3701898460 conn(0x7fd9940b5570 msgr2=0x7fd9940b5980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:05.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd99a3f8640 1 -- 172.21.15.31:0/3701898460 shutdown_connections 2023-11-26T07:23:05.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.329+0000 7fd99a3f8640 1 -- 172.21.15.31:0/3701898460 wait complete. 2023-11-26T07:23:05.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd99a3f8640 1 Processor -- start 2023-11-26T07:23:05.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd99a3f8640 1 -- start start 2023-11-26T07:23:05.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd99a3f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9940b5120 0x7fd99414fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd99a3f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 0x7fd9940ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd99a3f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99415d8b0 0x7fd9941f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd99a3f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940ca7e0 con 0x7fd9940cecd0 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd99a3f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99415ee30 con 0x7fd99415d8b0 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd99a3f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99415eb50 con 0x7fd9940b5120 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 0x7fd9940ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 0x7fd9940ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36198/0 (socket says 172.21.15.31:36198) 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd9937fe640 1 -- 172.21.15.31:0/718937495 learned_addr learned my addr 172.21.15.31:0/718937495 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd993fff640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9940b5120 0x7fd99414fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd998995640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99415d8b0 0x7fd9941f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd9937fe640 1 -- 172.21.15.31:0/718937495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9940b5120 msgr2=0x7fd99414fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd9937fe640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9940b5120 0x7fd99414fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd9937fe640 1 -- 172.21.15.31:0/718937495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99415d8b0 msgr2=0x7fd9941f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd9937fe640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99415d8b0 0x7fd9941f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd9937fe640 1 -- 172.21.15.31:0/718937495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c009a70 con 0x7fd9940cecd0 2023-11-26T07:23:05.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd9937fe640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 0x7fd9940ad280 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c4b0 tx=0x7fd98800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:05.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd990ff9640 1 -- 172.21.15.31:0/718937495 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880074e0 con 0x7fd9940cecd0 2023-11-26T07:23:05.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd990ff9640 1 -- 172.21.15.31:0/718937495 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd988007660 con 0x7fd9940cecd0 2023-11-26T07:23:05.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd99a3f8640 1 -- 172.21.15.31:0/718937495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9940ca190 con 0x7fd9940cecd0 2023-11-26T07:23:05.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd990ff9640 1 -- 172.21.15.31:0/718937495 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd988004a30 con 0x7fd9940cecd0 2023-11-26T07:23:05.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.333+0000 7fd99a3f8640 1 -- 172.21.15.31:0/718937495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9941502f0 con 0x7fd9940cecd0 2023-11-26T07:23:05.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.337+0000 7fd99a3f8640 1 -- 172.21.15.31:0/718937495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9940ca950 con 0x7fd9940cecd0 2023-11-26T07:23:05.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.337+0000 7fd990ff9640 1 -- 172.21.15.31:0/718937495 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fd988004b90 con 0x7fd9940cecd0 2023-11-26T07:23:05.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.337+0000 7fd990ff9640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd980066590 0x7fd980068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.337+0000 7fd990ff9640 1 -- 172.21.15.31:0/718937495 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9880835a0 con 0x7fd9940cecd0 2023-11-26T07:23:05.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.337+0000 7fd993fff640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd980066590 0x7fd980068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.337+0000 7fd993fff640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd980066590 0x7fd980068a50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9840079c0 tx=0x7fd984008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:05.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.341+0000 7fd990ff9640 1 -- 172.21.15.31:0/718937495 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd9880509e0 con 0x7fd9940cecd0 2023-11-26T07:23:05.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.501+0000 7fd99a3f8640 1 -- 172.21.15.31:0/718937495 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd99404e850 con 0x7fd9940cecd0 2023-11-26T07:23:05.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd990ff9640 1 -- 172.21.15.31:0/718937495 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fd988054000 con 0x7fd9940cecd0 2023-11-26T07:23:05.506 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:05.507 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":98,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:23:04.496975+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:02.513363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:03.492889+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:05.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 -- 172.21.15.31:0/718937495 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd980066590 msgr2=0x7fd980068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd980066590 0x7fd980068a50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9840079c0 tx=0x7fd984008040 comp rx=0 tx=0).stop 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 -- 172.21.15.31:0/718937495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 msgr2=0x7fd9940ad280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 0x7fd9940ad280 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd98800c4b0 tx=0x7fd98800c970 comp rx=0 tx=0).stop 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 -- 172.21.15.31:0/718937495 shutdown_connections 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fd980066590 0x7fd980068a50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd99415d8b0 0x7fd9941f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9940cecd0 0x7fd9940ad280 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 --2- 172.21.15.31:0/718937495 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd9940b5120 0x7fd99414fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 -- 172.21.15.31:0/718937495 >> 172.21.15.31:0/718937495 conn(0x7fd9940b5570 msgr2=0x7fd9940ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 -- 172.21.15.31:0/718937495 shutdown_connections 2023-11-26T07:23:05.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.505+0000 7fd99a3f8640 1 -- 172.21.15.31:0/718937495 wait complete. 2023-11-26T07:23:05.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:05.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5513c1640 1 Processor -- start 2023-11-26T07:23:05.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5513c1640 1 -- start start 2023-11-26T07:23:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5513c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb54c144c80 0x7fb54c0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5513c1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb54c0ac460 0x7fb54c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5513c1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb54c070c20 0x7fb54c15eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5513c1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54c145060 con 0x7fb54c0ac460 2023-11-26T07:23:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5513c1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54c071fd0 con 0x7fb54c070c20 2023-11-26T07:23:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5513c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c0ac8c0 con 0x7fb54c144c80 2023-11-26T07:23:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5427fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb54c0ac460 0x7fb54c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5427fc640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb54c0ac460 0x7fb54c0706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36202/0 (socket says 172.21.15.31:36202) 2023-11-26T07:23:05.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5427fc640 1 -- 172.21.15.31:0/1085232378 learned_addr learned my addr 172.21.15.31:0/1085232378 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:05.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb54b7fe640 1 --2- 172.21.15.31:0/1085232378 >> v2:172.21.15.97:3300/0 conn(0x7fb54c070c20 0x7fb54c15eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb54affd640 1 --2- 172.21.15.31:0/1085232378 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb54c144c80 0x7fb54c0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5427fc640 1 -- 172.21.15.31:0/1085232378 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb54c144c80 msgr2=0x7fb54c0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:05.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5427fc640 1 --2- 172.21.15.31:0/1085232378 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb54c144c80 0x7fb54c0abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5427fc640 1 -- 172.21.15.31:0/1085232378 >> v2:172.21.15.97:3300/0 conn(0x7fb54c070c20 msgr2=0x7fb54c15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:05.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5427fc640 1 --2- 172.21.15.31:0/1085232378 >> v2:172.21.15.97:3300/0 conn(0x7fb54c070c20 0x7fb54c15eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb5427fc640 1 -- 172.21.15.31:0/1085232378 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c071450 con 0x7fb54c0ac460 2023-11-26T07:23:05.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.669+0000 7fb54b7fe640 1 --2- 172.21.15.31:0/1085232378 >> v2:172.21.15.97:3300/0 conn(0x7fb54c070c20 0x7fb54c15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:05.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb5427fc640 1 --2- 172.21.15.31:0/1085232378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c0ac460 0x7fb54c0706e0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb54c072620 tx=0x7fb53401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9bafc3eff279a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:05.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb54a7fc640 1 -- 172.21.15.31:0/1085232378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340ab600 con 0x7fb54c0ac460 2023-11-26T07:23:05.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb54a7fc640 1 -- 172.21.15.31:0/1085232378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5340a7050 con 0x7fb54c0ac460 2023-11-26T07:23:05.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb54a7fc640 1 -- 172.21.15.31:0/1085232378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340ab980 con 0x7fb54c0ac460 2023-11-26T07:23:05.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb5513c1640 1 -- 172.21.15.31:0/1085232378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c0ac460 msgr2=0x7fb54c0706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb5513c1640 1 --2- 172.21.15.31:0/1085232378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c0ac460 0x7fb54c0706e0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb54c072620 tx=0x7fb53401c210 comp rx=0 tx=0).stop 2023-11-26T07:23:05.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb5513c1640 1 -- 172.21.15.31:0/1085232378 shutdown_connections 2023-11-26T07:23:05.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb5513c1640 1 --2- 172.21.15.31:0/1085232378 >> v2:172.21.15.97:3300/0 conn(0x7fb54c070c20 0x7fb54c15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb5513c1640 1 --2- 172.21.15.31:0/1085232378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c0ac460 0x7fb54c0706e0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb5513c1640 1 --2- 172.21.15.31:0/1085232378 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb54c144c80 0x7fb54c0abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb5513c1640 1 -- 172.21.15.31:0/1085232378 >> 172.21.15.31:0/1085232378 conn(0x7fb54c0b0c40 msgr2=0x7fb54c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:05.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb5513c1640 1 -- 172.21.15.31:0/1085232378 shutdown_connections 2023-11-26T07:23:05.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.673+0000 7fb5513c1640 1 -- 172.21.15.31:0/1085232378 wait complete. 2023-11-26T07:23:05.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5513c1640 1 Processor -- start 2023-11-26T07:23:05.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5513c1640 1 -- start start 2023-11-26T07:23:05.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5513c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c070c20 0x7fb54c157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5513c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb54c0ac460 0x7fb54c09b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5513c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb54c144c80 0x7fb54c1fe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5513c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c0cd860 con 0x7fb54c070c20 2023-11-26T07:23:05.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5513c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c0b2040 con 0x7fb54c144c80 2023-11-26T07:23:05.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5513c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c09c940 con 0x7fb54c0ac460 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c070c20 0x7fb54c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c070c20 0x7fb54c157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36204/0 (socket says 172.21.15.31:36204) 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54affd640 1 -- 172.21.15.31:0/646590702 learned_addr learned my addr 172.21.15.31:0/646590702 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5427fc640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb54c0ac460 0x7fb54c09b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54b7fe640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb54c144c80 0x7fb54c1fe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54affd640 1 -- 172.21.15.31:0/646590702 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb54c0ac460 msgr2=0x7fb54c09b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54affd640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb54c0ac460 0x7fb54c09b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54affd640 1 -- 172.21.15.31:0/646590702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb54c144c80 msgr2=0x7fb54c1fe0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54affd640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb54c144c80 0x7fb54c1fe0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54affd640 1 -- 172.21.15.31:0/646590702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5340094d0 con 0x7fb54c070c20 2023-11-26T07:23:05.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54b7fe640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb54c144c80 0x7fb54c1fe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:05.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb54affd640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c070c20 0x7fb54c157750 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb53c007e70 tx=0x7fb53c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:05.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb543fff640 1 -- 172.21.15.31:0/646590702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c014620 con 0x7fb54c070c20 2023-11-26T07:23:05.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb543fff640 1 -- 172.21.15.31:0/646590702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c014780 con 0x7fb54c070c20 2023-11-26T07:23:05.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb543fff640 1 -- 172.21.15.31:0/646590702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c014b30 con 0x7fb54c070c20 2023-11-26T07:23:05.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5513c1640 1 -- 172.21.15.31:0/646590702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54c09a0f0 con 0x7fb54c070c20 2023-11-26T07:23:05.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.677+0000 7fb5513c1640 1 -- 172.21.15.31:0/646590702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54c099e10 con 0x7fb54c070c20 2023-11-26T07:23:05.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.681+0000 7fb5513c1640 1 -- 172.21.15.31:0/646590702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54c0b1ba0 con 0x7fb54c070c20 2023-11-26T07:23:05.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.681+0000 7fb543fff640 1 -- 172.21.15.31:0/646590702 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fb53c014c90 con 0x7fb54c070c20 2023-11-26T07:23:05.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.681+0000 7fb543fff640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb52c0665c0 0x7fb52c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:05.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.681+0000 7fb543fff640 1 -- 172.21.15.31:0/646590702 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb53c084880 con 0x7fb54c070c20 2023-11-26T07:23:05.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.681+0000 7fb5427fc640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb52c0665c0 0x7fb52c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:05.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.681+0000 7fb5427fc640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb52c0665c0 0x7fb52c068a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb5340ad000 tx=0x7fb534008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:05.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.685+0000 7fb543fff640 1 -- 172.21.15.31:0/646590702 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb53c051be0 con 0x7fb54c070c20 2023-11-26T07:23:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.845+0000 7fb5513c1640 1 -- 172.21.15.31:0/646590702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb54c04e850 con 0x7fb54c070c20 2023-11-26T07:23:05.848 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.845+0000 7fb543fff640 1 -- 172.21.15.31:0/646590702 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fb53c055200 con 0x7fb54c070c20 2023-11-26T07:23:05.848 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:05.849 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":98,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:23:04.496975+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:02.513363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:03.492889+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:05.850 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 -- 172.21.15.31:0/646590702 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb52c0665c0 msgr2=0x7fb52c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb52c0665c0 0x7fb52c068a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb5340ad000 tx=0x7fb534008040 comp rx=0 tx=0).stop 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 -- 172.21.15.31:0/646590702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c070c20 msgr2=0x7fb54c157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c070c20 0x7fb54c157750 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb53c007e70 tx=0x7fb53c00e4b0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 -- 172.21.15.31:0/646590702 shutdown_connections 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb52c0665c0 0x7fb52c068a80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb54c144c80 0x7fb54c1fe0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb54c0ac460 0x7fb54c09b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 --2- 172.21.15.31:0/646590702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb54c070c20 0x7fb54c157750 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 -- 172.21.15.31:0/646590702 >> 172.21.15.31:0/646590702 conn(0x7fb54c0b0c40 msgr2=0x7fb54c06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 -- 172.21.15.31:0/646590702 shutdown_connections 2023-11-26T07:23:05.851 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:05.849+0000 7fb5513c1640 1 -- 172.21.15.31:0/646590702 wait complete. 2023-11-26T07:23:05.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-26T07:23:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16aeee4640 1 Processor -- start 2023-11-26T07:23:06.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16aeee4640 1 -- start start 2023-11-26T07:23:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16aeee4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f16a80b5830 0x7f16a815e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16aeee4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 0x7f16a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16aeee4640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a80af630 con 0x7f16a80acd70 2023-11-26T07:23:06.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16aeee4640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a80acb10 con 0x7f16a80b5830 2023-11-26T07:23:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16aeee4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a80ae5b0 con 0x7f16a80ad8e0 2023-11-26T07:23:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16a7fff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f16a80acd70 0x7f16a80ad1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60544/0 (socket says 172.21.15.31:60544) 2023-11-26T07:23:06.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16a7fff640 1 -- 172.21.15.31:0/1107358582 learned_addr learned my addr 172.21.15.31:0/1107358582 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16acc80640 1 --2- 172.21.15.31:0/1107358582 >> v2:172.21.15.97:3300/0 conn(0x7f16a80b5830 0x7f16a815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.013+0000 7f16ad481640 1 --2- 172.21.15.31:0/1107358582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 0x7f16a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16a77fe640 1 -- 172.21.15.31:0/1107358582 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782138093 0 0) 0x7f1694001120 con 0x7f16a80acd70 2023-11-26T07:23:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16a77fe640 1 -- 172.21.15.31:0/1107358582 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1698003530 con 0x7f16a80acd70 2023-11-26T07:23:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16a77fe640 1 -- 172.21.15.31:0/1107358582 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 445879242 0 0) 0x7f1698003530 con 0x7f16a80acd70 2023-11-26T07:23:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16ad481640 1 -- 172.21.15.31:0/1107358582 >> v2:172.21.15.97:3300/0 conn(0x7f16a80b5830 msgr2=0x7f16a815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16ad481640 1 --2- 172.21.15.31:0/1107358582 >> v2:172.21.15.97:3300/0 conn(0x7f16a80b5830 0x7f16a815e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16ad481640 1 -- 172.21.15.31:0/1107358582 >> v1:172.21.15.31:6789/0 conn(0x7f16a80acd70 legacy=0x7f16a80ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16ad481640 1 -- 172.21.15.31:0/1107358582 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a80ae870 con 0x7f16a80ad8e0 2023-11-26T07:23:06.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16acc80640 1 --2- 172.21.15.31:0/1107358582 >> v2:172.21.15.97:3300/0 conn(0x7f16a80b5830 0x7f16a815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16ad481640 1 --2- 172.21.15.31:0/1107358582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 0x7f16a806d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009d90 tx=0x7f16a001c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9523f37f0200c43c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16a77fe640 1 -- 172.21.15.31:0/1107358582 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694001810 con 0x7f16a80ad8e0 2023-11-26T07:23:06.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16a77fe640 1 -- 172.21.15.31:0/1107358582 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a00a5670 con 0x7f16a80ad8e0 2023-11-26T07:23:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.017+0000 7f16a77fe640 1 -- 172.21.15.31:0/1107358582 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a00a59e0 con 0x7f16a80ad8e0 2023-11-26T07:23:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 -- 172.21.15.31:0/1107358582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 msgr2=0x7f16a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 --2- 172.21.15.31:0/1107358582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 0x7f16a806d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009d90 tx=0x7f16a001c870 comp rx=0 tx=0).stop 2023-11-26T07:23:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 -- 172.21.15.31:0/1107358582 shutdown_connections 2023-11-26T07:23:06.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 --2- 172.21.15.31:0/1107358582 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 0x7f16a806d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 --2- 172.21.15.31:0/1107358582 >> v2:172.21.15.97:3300/0 conn(0x7f16a80b5830 0x7f16a815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 -- 172.21.15.31:0/1107358582 >> 172.21.15.31:0/1107358582 conn(0x7f16a8071d80 msgr2=0x7f16a806a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 -- 172.21.15.31:0/1107358582 shutdown_connections 2023-11-26T07:23:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 -- 172.21.15.31:0/1107358582 wait complete. 2023-11-26T07:23:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 Processor -- start 2023-11-26T07:23:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 -- start start 2023-11-26T07:23:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16a80acd70 0x7f16a8152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 0x7f16a8155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a80b5830 0x7f16a814b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:06.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a80affc0 con 0x7f16a80b5830 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a814dfe0 con 0x7f16a80acd70 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16aeee4640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a814dd00 con 0x7f16a80ad8e0 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16ad481640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a80b5830 0x7f16a814b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16ad481640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a80b5830 0x7f16a814b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36212/0 (socket says 172.21.15.31:36212) 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16ad481640 1 -- 172.21.15.31:0/1725428572 learned_addr learned my addr 172.21.15.31:0/1725428572 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16acc80640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16a80acd70 0x7f16a8152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16a7fff640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 0x7f16a8155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16ad481640 1 -- 172.21.15.31:0/1725428572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 msgr2=0x7f16a8155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16ad481640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 0x7f16a8155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16ad481640 1 -- 172.21.15.31:0/1725428572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16a80acd70 msgr2=0x7f16a8152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:06.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16ad481640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16a80acd70 0x7f16a8152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.021+0000 7f16ad481640 1 -- 172.21.15.31:0/1725428572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c009420 con 0x7f16a80b5830 2023-11-26T07:23:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16ad481640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a80b5830 0x7f16a814b500 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f16a001af20 tx=0x7f16a0019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16a57fa640 1 -- 172.21.15.31:0/1725428572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a00a59f0 con 0x7f16a80b5830 2023-11-26T07:23:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16a57fa640 1 -- 172.21.15.31:0/1725428572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a00a5b50 con 0x7f16a80b5830 2023-11-26T07:23:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16a57fa640 1 -- 172.21.15.31:0/1725428572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0004040 con 0x7f16a80b5830 2023-11-26T07:23:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16aeee4640 1 -- 172.21.15.31:0/1725428572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a0009a70 con 0x7f16a80b5830 2023-11-26T07:23:06.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16aeee4640 1 -- 172.21.15.31:0/1725428572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a80afbe0 con 0x7f16a80b5830 2023-11-26T07:23:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16aeee4640 1 -- 172.21.15.31:0/1725428572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a80b0100 con 0x7f16a80b5830 2023-11-26T07:23:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16a57fa640 1 -- 172.21.15.31:0/1725428572 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f16a00b4070 con 0x7f16a80b5830 2023-11-26T07:23:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16a57fa640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f168c066590 0x7f168c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16a57fa640 1 -- 172.21.15.31:0/1725428572 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16a011e540 con 0x7f16a80b5830 2023-11-26T07:23:06.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.025+0000 7f16acc80640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f168c066590 0x7f168c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:06.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.029+0000 7f16acc80640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f168c066590 0x7f168c068a50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f169c009cf0 tx=0x7f169c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:06.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.029+0000 7f16a57fa640 1 -- 172.21.15.31:0/1725428572 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f16a80b0100 con 0x7f16a80b5830 2023-11-26T07:23:06.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:06.225+0000 7f16aeee4640 1 -- 172.21.15.31:0/1725428572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f16a80b0100 con 0x7f16a80b5830 2023-11-26T07:23:06.696 INFO:tasks.ceph.mds.c.smithi031.stderr:2023-11-26T07:23:06.693+0000 7f830d182640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-26T07:23:07.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.137+0000 7f16a57fa640 1 -- 172.21.15.31:0/1725428572 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) v1 ==== 118+0+0 (secure 0 0 0) 0x7f16a80b0100 con 0x7f16a80b5830 2023-11-26T07:23:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 -- 172.21.15.31:0/1725428572 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f168c066590 msgr2=0x7f168c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f168c066590 0x7f168c068a50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f169c009cf0 tx=0x7f169c009190 comp rx=0 tx=0).stop 2023-11-26T07:23:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 -- 172.21.15.31:0/1725428572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a80b5830 msgr2=0x7f16a814b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a80b5830 0x7f16a814b500 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f16a001af20 tx=0x7f16a0019610 comp rx=0 tx=0).stop 2023-11-26T07:23:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 -- 172.21.15.31:0/1725428572 shutdown_connections 2023-11-26T07:23:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f168c066590 0x7f168c068a50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16a80b5830 0x7f16a814b500 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f16a80ad8e0 0x7f16a8155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:07.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 --2- 172.21.15.31:0/1725428572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16a80acd70 0x7f16a8152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 -- 172.21.15.31:0/1725428572 >> 172.21.15.31:0/1725428572 conn(0x7f16a8071d80 msgr2=0x7f16a80b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 -- 172.21.15.31:0/1725428572 shutdown_connections 2023-11-26T07:23:07.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.141+0000 7f16aeee4640 1 -- 172.21.15.31:0/1725428572 wait complete. 2023-11-26T07:23:07.166 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-26T07:23:07.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-26T07:23:07.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.313+0000 7f4e7afd1640 1 Processor -- start 2023-11-26T07:23:07.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e7afd1640 1 -- start start 2023-11-26T07:23:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e7afd1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 0x7f4e7406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:07.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e7afd1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e74159630 con 0x7f4e740ce5d0 2023-11-26T07:23:07.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e7afd1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e740cd1d0 con 0x7f4e740cbd90 2023-11-26T07:23:07.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e7afd1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7415bba0 con 0x7f4e740cb9b0 2023-11-26T07:23:07.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e7956e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4e740ce5d0 0x7f4e7415bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60556/0 (socket says 172.21.15.31:60556) 2023-11-26T07:23:07.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e7956e640 1 -- 172.21.15.31:0/1652872844 learned_addr learned my addr 172.21.15.31:0/1652872844 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:07.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e78d6d640 1 --2- 172.21.15.31:0/1652872844 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 0x7f4e7406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e737fe640 1 -- 172.21.15.31:0/1652872844 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273136248 0 0) 0x7f4e74159630 con 0x7f4e740ce5d0 2023-11-26T07:23:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e737fe640 1 -- 172.21.15.31:0/1652872844 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e64003530 con 0x7f4e740ce5d0 2023-11-26T07:23:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e737fe640 1 -- 172.21.15.31:0/1652872844 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221394135 0 0) 0x7f4e740cd1d0 con 0x7f4e740cbd90 2023-11-26T07:23:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e737fe640 1 -- 172.21.15.31:0/1652872844 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e74159630 con 0x7f4e740cbd90 2023-11-26T07:23:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e78d6d640 1 -- 172.21.15.31:0/1652872844 >> v1:172.21.15.97:6789/0 conn(0x7f4e740cbd90 legacy=0x7f4e7415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e78d6d640 1 -- 172.21.15.31:0/1652872844 >> v1:172.21.15.31:6789/0 conn(0x7f4e740ce5d0 legacy=0x7f4e7415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:07.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.317+0000 7f4e78d6d640 1 -- 172.21.15.31:0/1652872844 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e740cd4b0 con 0x7f4e740cb9b0 2023-11-26T07:23:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e78d6d640 1 --2- 172.21.15.31:0/1652872844 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 0x7f4e7406e3f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097f0 tx=0x7f4e6801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c8c489cc9b71c62 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e737fe640 1 -- 172.21.15.31:0/1652872844 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e680ac410 con 0x7f4e740cb9b0 2023-11-26T07:23:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e737fe640 1 -- 172.21.15.31:0/1652872844 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e680ac570 con 0x7f4e740cb9b0 2023-11-26T07:23:07.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e737fe640 1 -- 172.21.15.31:0/1652872844 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e680ac900 con 0x7f4e740cb9b0 2023-11-26T07:23:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e7afd1640 1 -- 172.21.15.31:0/1652872844 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 msgr2=0x7f4e7406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e7afd1640 1 --2- 172.21.15.31:0/1652872844 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 0x7f4e7406e3f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097f0 tx=0x7f4e6801c4c0 comp rx=0 tx=0).stop 2023-11-26T07:23:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e7afd1640 1 -- 172.21.15.31:0/1652872844 shutdown_connections 2023-11-26T07:23:07.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e7afd1640 1 --2- 172.21.15.31:0/1652872844 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 0x7f4e7406e3f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e7afd1640 1 -- 172.21.15.31:0/1652872844 >> 172.21.15.31:0/1652872844 conn(0x7f4e740b0cb0 msgr2=0x7f4e740b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e7afd1640 1 -- 172.21.15.31:0/1652872844 shutdown_connections 2023-11-26T07:23:07.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.321+0000 7f4e7afd1640 1 -- 172.21.15.31:0/1652872844 wait complete. 2023-11-26T07:23:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 Processor -- start 2023-11-26T07:23:07.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 -- start start 2023-11-26T07:23:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 0x7f4e741555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e740cbd90 0x7f4e740ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e740ce5d0 0x7f4e74153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740ac770 con 0x7f4e740ce5d0 2023-11-26T07:23:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740aca50 con 0x7f4e740cbd90 2023-11-26T07:23:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740acd30 con 0x7f4e740cb9b0 2023-11-26T07:23:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7956e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e740ce5d0 0x7f4e74153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e78d6d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 0x7f4e741555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e78d6d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 0x7f4e741555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3301/0 says I am v2:172.21.15.31:44354/0 (socket says 172.21.15.31:44354) 2023-11-26T07:23:07.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e78d6d640 1 -- 172.21.15.31:0/2753047570 learned_addr learned my addr 172.21.15.31:0/2753047570 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7956e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e740ce5d0 0x7f4e74153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36228/0 (socket says 172.21.15.31:36228) 2023-11-26T07:23:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e73fff640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e740cbd90 0x7f4e740ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7956e640 1 -- 172.21.15.31:0/2753047570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 msgr2=0x7f4e741555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7956e640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 0x7f4e741555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7956e640 1 -- 172.21.15.31:0/2753047570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e740cbd90 msgr2=0x7f4e740ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7956e640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e740cbd90 0x7f4e740ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7956e640 1 -- 172.21.15.31:0/2753047570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68009420 con 0x7f4e740ce5d0 2023-11-26T07:23:07.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7956e640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e740ce5d0 0x7f4e74153820 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00ca70 tx=0x7f4e6c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e717fa640 1 -- 172.21.15.31:0/2753047570 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00ab70 con 0x7f4e740ce5d0 2023-11-26T07:23:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e717fa640 1 -- 172.21.15.31:0/2753047570 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c00acd0 con 0x7f4e740ce5d0 2023-11-26T07:23:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e717fa640 1 -- 172.21.15.31:0/2753047570 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c012640 con 0x7f4e740ce5d0 2023-11-26T07:23:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 -- 172.21.15.31:0/2753047570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e740ad010 con 0x7f4e740ce5d0 2023-11-26T07:23:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 -- 172.21.15.31:0/2753047570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e74155b30 con 0x7f4e740ce5d0 2023-11-26T07:23:07.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.325+0000 7f4e7afd1640 1 -- 172.21.15.31:0/2753047570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e740671f0 con 0x7f4e740ce5d0 2023-11-26T07:23:07.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.329+0000 7f4e717fa640 1 -- 172.21.15.31:0/2753047570 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f4e6c007070 con 0x7f4e740ce5d0 2023-11-26T07:23:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.329+0000 7f4e717fa640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4e58066590 0x7f4e58068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.329+0000 7f4e717fa640 1 -- 172.21.15.31:0/2753047570 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e6c083de0 con 0x7f4e740ce5d0 2023-11-26T07:23:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.329+0000 7f4e78d6d640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4e58066590 0x7f4e58068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:07.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.329+0000 7f4e78d6d640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4e58066590 0x7f4e58068a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4e6801a0f0 tx=0x7f4e68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:07.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.333+0000 7f4e717fa640 1 -- 172.21.15.31:0/2753047570 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e6c051200 con 0x7f4e740ce5d0 2023-11-26T07:23:07.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:07.525+0000 7f4e7afd1640 1 -- 172.21.15.31:0/2753047570 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4e7406b460 con 0x7f4e740ce5d0 2023-11-26T07:23:08.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e717fa640 1 -- 172.21.15.31:0/2753047570 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4e6c0170c0 con 0x7f4e740ce5d0 2023-11-26T07:23:08.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 -- 172.21.15.31:0/2753047570 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4e58066590 msgr2=0x7f4e58068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4e58066590 0x7f4e58068a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4e6801a0f0 tx=0x7f4e68008040 comp rx=0 tx=0).stop 2023-11-26T07:23:08.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 -- 172.21.15.31:0/2753047570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e740ce5d0 msgr2=0x7f4e74153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e740ce5d0 0x7f4e74153820 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00ca70 tx=0x7f4e6c00cf30 comp rx=0 tx=0).stop 2023-11-26T07:23:08.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 -- 172.21.15.31:0/2753047570 shutdown_connections 2023-11-26T07:23:08.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f4e58066590 0x7f4e58068a50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e740ce5d0 0x7f4e74153820 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e740cbd90 0x7f4e740ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 --2- 172.21.15.31:0/2753047570 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f4e740cb9b0 0x7f4e741555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 -- 172.21.15.31:0/2753047570 >> 172.21.15.31:0/2753047570 conn(0x7f4e740b0cb0 msgr2=0x7f4e7415d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:08.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 -- 172.21.15.31:0/2753047570 shutdown_connections 2023-11-26T07:23:08.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.141+0000 7f4e7afd1640 1 -- 172.21.15.31:0/2753047570 wait complete. 2023-11-26T07:23:08.169 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-26T07:23:08.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:08.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.313+0000 7f90c2165640 1 Processor -- start 2023-11-26T07:23:08.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.313+0000 7f90c2165640 1 -- start start 2023-11-26T07:23:08.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.313+0000 7f90c2165640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f90bc0aec30 0x7f90bc15c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.313+0000 7f90c2165640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90bc15c8f0 con 0x7f90bc071cc0 2023-11-26T07:23:08.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.313+0000 7f90c2165640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90bc06d190 con 0x7f90bc06cc20 2023-11-26T07:23:08.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.313+0000 7f90c2165640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc06ed80 con 0x7f90bc0aec30 2023-11-26T07:23:08.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.313+0000 7f90bb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f90bc0aec30 0x7f90bc15c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.313+0000 7f90bbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f90bc071cc0 0x7f90bc15cf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60558/0 (socket says 172.21.15.31:60558) 2023-11-26T07:23:08.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.313+0000 7f90bbfff640 1 -- 172.21.15.31:0/2003618754 learned_addr learned my addr 172.21.15.31:0/2003618754 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:08.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90bb7fe640 1 -- 172.21.15.31:0/2003618754 >> v1:172.21.15.97:6789/0 conn(0x7f90bc06cc20 legacy=0x7f90bc0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:08.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90bb7fe640 1 -- 172.21.15.31:0/2003618754 >> v1:172.21.15.31:6789/0 conn(0x7f90bc071cc0 legacy=0x7f90bc15cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90bb7fe640 1 -- 172.21.15.31:0/2003618754 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc06d470 con 0x7f90bc0aec30 2023-11-26T07:23:08.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90ba7fc640 1 -- 172.21.15.31:0/2003618754 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802981419 0 0) 0x7f90bc15c8f0 con 0x7f90bc071cc0 2023-11-26T07:23:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90bb7fe640 1 --2- 172.21.15.31:0/2003618754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0aec30 0x7f90bc15c3b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0097c0 tx=0x7f90ac01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d2c8f361c61bae server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90ba7fc640 1 -- 172.21.15.31:0/2003618754 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0ab600 con 0x7f90bc0aec30 2023-11-26T07:23:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90ba7fc640 1 -- 172.21.15.31:0/2003618754 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ac0a7050 con 0x7f90bc0aec30 2023-11-26T07:23:08.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90ba7fc640 1 -- 172.21.15.31:0/2003618754 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0ab980 con 0x7f90bc0aec30 2023-11-26T07:23:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90c2165640 1 -- 172.21.15.31:0/2003618754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0aec30 msgr2=0x7f90bc15c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90c2165640 1 --2- 172.21.15.31:0/2003618754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0aec30 0x7f90bc15c3b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0097c0 tx=0x7f90ac01c3f0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90c2165640 1 -- 172.21.15.31:0/2003618754 shutdown_connections 2023-11-26T07:23:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90c2165640 1 --2- 172.21.15.31:0/2003618754 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0aec30 0x7f90bc15c3b0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.317+0000 7f90c2165640 1 -- 172.21.15.31:0/2003618754 >> 172.21.15.31:0/2003618754 conn(0x7f90bc0b1e40 msgr2=0x7f90bc070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90c2165640 1 -- 172.21.15.31:0/2003618754 shutdown_connections 2023-11-26T07:23:08.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90c2165640 1 -- 172.21.15.31:0/2003618754 wait complete. 2023-11-26T07:23:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90c2165640 1 Processor -- start 2023-11-26T07:23:08.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90c2165640 1 -- start start 2023-11-26T07:23:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90c2165640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90bc06cc20 0x7f90bc14ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90c2165640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc071cc0 0x7f90bc1557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90c2165640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f90bc0aec30 0x7f90bc0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90c2165640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc14fe30 con 0x7f90bc071cc0 2023-11-26T07:23:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90c2165640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc14dfd0 con 0x7f90bc06cc20 2023-11-26T07:23:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90c2165640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc14e290 con 0x7f90bc0aec30 2023-11-26T07:23:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90baffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc071cc0 0x7f90bc1557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90baffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc071cc0 0x7f90bc1557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36252/0 (socket says 172.21.15.31:36252) 2023-11-26T07:23:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90baffd640 1 -- 172.21.15.31:0/3947292255 learned_addr learned my addr 172.21.15.31:0/3947292255 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90bb7fe640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90bc06cc20 0x7f90bc14ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90bbfff640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f90bc0aec30 0x7f90bc0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90baffd640 1 -- 172.21.15.31:0/3947292255 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f90bc0aec30 msgr2=0x7f90bc0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90baffd640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f90bc0aec30 0x7f90bc0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90baffd640 1 -- 172.21.15.31:0/3947292255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90bc06cc20 msgr2=0x7f90bc14ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90baffd640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90bc06cc20 0x7f90bc14ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.321+0000 7f90baffd640 1 -- 172.21.15.31:0/3947292255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac009420 con 0x7f90bc071cc0 2023-11-26T07:23:08.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90baffd640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc071cc0 0x7f90bc1557b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c3c0 tx=0x7f90b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90a3fff640 1 -- 172.21.15.31:0/3947292255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00052d0 con 0x7f90bc071cc0 2023-11-26T07:23:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90a3fff640 1 -- 172.21.15.31:0/3947292255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b0005430 con 0x7f90bc071cc0 2023-11-26T07:23:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90c2165640 1 -- 172.21.15.31:0/3947292255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc14e570 con 0x7f90bc071cc0 2023-11-26T07:23:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90a3fff640 1 -- 172.21.15.31:0/3947292255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0003840 con 0x7f90bc071cc0 2023-11-26T07:23:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90c2165640 1 -- 172.21.15.31:0/3947292255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc0b2ed0 con 0x7f90bc071cc0 2023-11-26T07:23:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90c2165640 1 -- 172.21.15.31:0/3947292255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc066ab0 con 0x7f90bc071cc0 2023-11-26T07:23:08.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90a3fff640 1 -- 172.21.15.31:0/3947292255 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f90b00039a0 con 0x7f90bc071cc0 2023-11-26T07:23:08.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90a3fff640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f90a8066590 0x7f90a8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90a3fff640 1 -- 172.21.15.31:0/3947292255 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90b00834b0 con 0x7f90bc071cc0 2023-11-26T07:23:08.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.325+0000 7f90bb7fe640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f90a8066590 0x7f90a8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.329+0000 7f90bb7fe640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f90a8066590 0x7f90a8068a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0ad000 tx=0x7f90ac0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.329+0000 7f90a3fff640 1 -- 172.21.15.31:0/3947292255 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f90b00508c0 con 0x7f90bc071cc0 2023-11-26T07:23:08.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.517+0000 7f90c2165640 1 -- 172.21.15.31:0/3947292255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90bc04e850 con 0x7f90bc071cc0 2023-11-26T07:23:08.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90a3fff640 1 -- 172.21.15.31:0/3947292255 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f90b0053ee0 con 0x7f90bc071cc0 2023-11-26T07:23:08.522 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:08.522 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3797004646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3797004646},{"type":"v1","addr":"172.21.15.97:6833","nonce":3797004646}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2675576819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2675576819},{"type":"v1","addr":"172.21.15.97:6835","nonce":2675576819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1710103664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1710103664},{"type":"v1","addr":"172.21.15.31:6834","nonce":1710103664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:04.497314+0000","modified":"2023-11-26T07:23:07.528539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1001580514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1001580514},{"type":"v1","addr":"172.21.15.31:6835","nonce":1001580514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:23:08.523 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 72 2023-11-26T07:23:08.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 -- 172.21.15.31:0/3947292255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f90a8066590 msgr2=0x7f90a8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f90a8066590 0x7f90a8068a50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0ad000 tx=0x7f90ac0a6000 comp rx=0 tx=0).stop 2023-11-26T07:23:08.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 -- 172.21.15.31:0/3947292255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc071cc0 msgr2=0x7f90bc1557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc071cc0 0x7f90bc1557b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c3c0 tx=0x7f90b000c880 comp rx=0 tx=0).stop 2023-11-26T07:23:08.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 -- 172.21.15.31:0/3947292255 shutdown_connections 2023-11-26T07:23:08.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f90a8066590 0x7f90a8068a50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f90bc0aec30 0x7f90bc0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc071cc0 0x7f90bc1557b0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 --2- 172.21.15.31:0/3947292255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90bc06cc20 0x7f90bc14ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 -- 172.21.15.31:0/3947292255 >> 172.21.15.31:0/3947292255 conn(0x7f90bc0b1e40 msgr2=0x7f90bc070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:08.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 -- 172.21.15.31:0/3947292255 shutdown_connections 2023-11-26T07:23:08.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.521+0000 7f90c2165640 1 -- 172.21.15.31:0/3947292255 wait complete. 2023-11-26T07:23:08.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:08.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f974232a640 1 Processor -- start 2023-11-26T07:23:08.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f974232a640 1 -- start start 2023-11-26T07:23:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f974232a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f973c0d2080 0x7f973c168160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f974232a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c0cefd0 con 0x7f973c15d7b0 2023-11-26T07:23:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f974232a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c0cf2b0 con 0x7f973c0cead0 2023-11-26T07:23:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f974232a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c0d17e0 con 0x7f973c0d2080 2023-11-26T07:23:08.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f97408c7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f973c0d2080 0x7f973c168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f973bfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f973c15d7b0 0x7f973c0d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60564/0 (socket says 172.21.15.31:60564) 2023-11-26T07:23:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f97408c7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f973c0d2080 0x7f973c168160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36254/0 (socket says 172.21.15.31:36254) 2023-11-26T07:23:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f973bfff640 1 -- 172.21.15.31:0/3349278587 learned_addr learned my addr 172.21.15.31:0/3349278587 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f97408c7640 1 -- 172.21.15.31:0/3349278587 >> v1:172.21.15.97:6789/0 conn(0x7f973c0cead0 legacy=0x7f973c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f97408c7640 1 -- 172.21.15.31:0/3349278587 >> v1:172.21.15.31:6789/0 conn(0x7f973c15d7b0 legacy=0x7f973c0d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f973affd640 1 -- 172.21.15.31:0/3349278587 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412194659 0 0) 0x7f973c0cefd0 con 0x7f973c15d7b0 2023-11-26T07:23:08.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.685+0000 7f97408c7640 1 -- 172.21.15.31:0/3349278587 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c06b3f0 con 0x7f973c0d2080 2023-11-26T07:23:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f97408c7640 1 --2- 172.21.15.31:0/3349278587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c0d2080 0x7f973c168160 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9734009d90 tx=0x7f973401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628a7a8158bb688 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f973affd640 1 -- 172.21.15.31:0/3349278587 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340a4bd0 con 0x7f973c0d2080 2023-11-26T07:23:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f973affd640 1 -- 172.21.15.31:0/3349278587 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97340a4d30 con 0x7f973c0d2080 2023-11-26T07:23:08.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f973affd640 1 -- 172.21.15.31:0/3349278587 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340a4ea0 con 0x7f973c0d2080 2023-11-26T07:23:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f974232a640 1 -- 172.21.15.31:0/3349278587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c0d2080 msgr2=0x7f973c168160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f974232a640 1 --2- 172.21.15.31:0/3349278587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c0d2080 0x7f973c168160 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9734009d90 tx=0x7f973401c3f0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f974232a640 1 -- 172.21.15.31:0/3349278587 shutdown_connections 2023-11-26T07:23:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f974232a640 1 --2- 172.21.15.31:0/3349278587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c0d2080 0x7f973c168160 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f974232a640 1 -- 172.21.15.31:0/3349278587 >> 172.21.15.31:0/3349278587 conn(0x7f973c0d33b0 msgr2=0x7f973c0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f974232a640 1 -- 172.21.15.31:0/3349278587 shutdown_connections 2023-11-26T07:23:08.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.689+0000 7f974232a640 1 -- 172.21.15.31:0/3349278587 wait complete. 2023-11-26T07:23:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f974232a640 1 Processor -- start 2023-11-26T07:23:08.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f974232a640 1 -- start start 2023-11-26T07:23:08.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f974232a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f973c0cead0 0x7f973c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f974232a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c0d2080 0x7f973c0c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f974232a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c15d7b0 0x7f973c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f974232a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c7eb0 con 0x7f973c15d7b0 2023-11-26T07:23:08.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f974232a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0cccd0 con 0x7f973c0d2080 2023-11-26T07:23:08.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f974232a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c8880 con 0x7f973c0cead0 2023-11-26T07:23:08.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f97408c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c15d7b0 0x7f973c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f97408c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c15d7b0 0x7f973c1529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36262/0 (socket says 172.21.15.31:36262) 2023-11-26T07:23:08.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f97408c7640 1 -- 172.21.15.31:0/3878823724 learned_addr learned my addr 172.21.15.31:0/3878823724 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:08.696 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f973b7fe640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c0d2080 0x7f973c0c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.697 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f973bfff640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f973c0cead0 0x7f973c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f97408c7640 1 -- 172.21.15.31:0/3878823724 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f973c0cead0 msgr2=0x7f973c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f97408c7640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f973c0cead0 0x7f973c0cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f97408c7640 1 -- 172.21.15.31:0/3878823724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c0d2080 msgr2=0x7f973c0c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f97408c7640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c0d2080 0x7f973c0c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f97408c7640 1 -- 172.21.15.31:0/3878823724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734009a70 con 0x7f973c15d7b0 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f97408c7640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c15d7b0 0x7f973c1529c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f973401a260 tx=0x7f97340194e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f9738ff9640 1 -- 172.21.15.31:0/3878823724 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340a4ea0 con 0x7f973c15d7b0 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f9738ff9640 1 -- 172.21.15.31:0/3878823724 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97340acd90 con 0x7f973c15d7b0 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f974232a640 1 -- 172.21.15.31:0/3878823724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c0c72a0 con 0x7f973c15d7b0 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f9738ff9640 1 -- 172.21.15.31:0/3878823724 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340b8670 con 0x7f973c15d7b0 2023-11-26T07:23:08.698 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.693+0000 7f974232a640 1 -- 172.21.15.31:0/3878823724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c0c7860 con 0x7f973c15d7b0 2023-11-26T07:23:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.697+0000 7f974232a640 1 -- 172.21.15.31:0/3878823724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c0c8020 con 0x7f973c15d7b0 2023-11-26T07:23:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.697+0000 7f9738ff9640 1 -- 172.21.15.31:0/3878823724 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f97340b87d0 con 0x7f973c15d7b0 2023-11-26T07:23:08.699 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.697+0000 7f9738ff9640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9728066590 0x7f9728068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.697+0000 7f9738ff9640 1 -- 172.21.15.31:0/3878823724 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f973411e6a0 con 0x7f973c15d7b0 2023-11-26T07:23:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.697+0000 7f973bfff640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9728066590 0x7f9728068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:08.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.697+0000 7f973bfff640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9728066590 0x7f9728068a50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f972c007fd0 tx=0x7f972c007200 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:08.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.701+0000 7f9738ff9640 1 -- 172.21.15.31:0/3878823724 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f97340ebae0 con 0x7f973c15d7b0 2023-11-26T07:23:08.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.861+0000 7f974232a640 1 -- 172.21.15.31:0/3878823724 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f973c04e850 con 0x7f973c15d7b0 2023-11-26T07:23:08.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.861+0000 7f9738ff9640 1 -- 172.21.15.31:0/3878823724 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16432 (secure 0 0 0) 0x7f97340ef100 con 0x7f973c15d7b0 2023-11-26T07:23:08.866 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:08.866 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":99,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:23:06.127026+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:02.513363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:03.492889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 -- 172.21.15.31:0/3878823724 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9728066590 msgr2=0x7f9728068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9728066590 0x7f9728068a50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f972c007fd0 tx=0x7f972c007200 comp rx=0 tx=0).stop 2023-11-26T07:23:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 -- 172.21.15.31:0/3878823724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c15d7b0 msgr2=0x7f973c1529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c15d7b0 0x7f973c1529c0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f973401a260 tx=0x7f97340194e0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 -- 172.21.15.31:0/3878823724 shutdown_connections 2023-11-26T07:23:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9728066590 0x7f9728068a50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f973c15d7b0 0x7f973c1529c0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f973c0d2080 0x7f973c0c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 --2- 172.21.15.31:0/3878823724 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f973c0cead0 0x7f973c0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 -- 172.21.15.31:0/3878823724 >> 172.21.15.31:0/3878823724 conn(0x7f973c0d33b0 msgr2=0x7f973c0d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 -- 172.21.15.31:0/3878823724 shutdown_connections 2023-11-26T07:23:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:08.865+0000 7f974232a640 1 -- 172.21.15.31:0/3878823724 wait complete. 2023-11-26T07:23:08.887 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-26T07:23:08.888 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:23:09.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03f905a640 1 Processor -- start 2023-11-26T07:23:09.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03f905a640 1 -- start start 2023-11-26T07:23:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03f905a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03f40cead0 0x7f03f40d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03f905a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f40d1eb0 0x7f03f406bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03f905a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f40d2300 con 0x7f03f415d7b0 2023-11-26T07:23:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03f905a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f40d17e0 con 0x7f03f40cead0 2023-11-26T07:23:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03f905a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f40cefd0 con 0x7f03f40d1eb0 2023-11-26T07:23:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03ebfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03f40cead0 0x7f03f40d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03ebfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f03f40cead0 0x7f03f40d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36272/0 (socket says 172.21.15.31:36272) 2023-11-26T07:23:09.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03f2d9d640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f03f415d7b0 0x7f03f40d10d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:57758/0 (socket says 172.21.15.31:57758) 2023-11-26T07:23:09.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03ebfff640 1 -- 172.21.15.31:0/2035676554 learned_addr learned my addr 172.21.15.31:0/2035676554 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:09.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03f359e640 1 --2- 172.21.15.31:0/2035676554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f40d1eb0 0x7f03f406bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03ebfff640 1 -- 172.21.15.31:0/2035676554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f40d1eb0 msgr2=0x7f03f406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:09.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03ebfff640 1 --2- 172.21.15.31:0/2035676554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f40d1eb0 0x7f03f406bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.033 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03ebfff640 1 -- 172.21.15.31:0/2035676554 >> v1:172.21.15.97:6789/0 conn(0x7f03f415d7b0 legacy=0x7f03f40d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03ebfff640 1 -- 172.21.15.31:0/2035676554 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f40cf2b0 con 0x7f03f40cead0 2023-11-26T07:23:09.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.029+0000 7f03f259c640 1 -- 172.21.15.31:0/2035676554 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915131660 0 0) 0x7f03e4001120 con 0x7f03f415d7b0 2023-11-26T07:23:09.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03ebfff640 1 --2- 172.21.15.31:0/2035676554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f40cead0 0x7f03f40d1970 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009870 tx=0x7f03dc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c30b2ce71d0b1b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f259c640 1 -- 172.21.15.31:0/2035676554 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0ab5f0 con 0x7f03f40cead0 2023-11-26T07:23:09.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f259c640 1 -- 172.21.15.31:0/2035676554 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc0ab750 con 0x7f03f40cead0 2023-11-26T07:23:09.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f259c640 1 -- 172.21.15.31:0/2035676554 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0abae0 con 0x7f03f40cead0 2023-11-26T07:23:09.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f905a640 1 -- 172.21.15.31:0/2035676554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f40cead0 msgr2=0x7f03f40d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f905a640 1 --2- 172.21.15.31:0/2035676554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f40cead0 0x7f03f40d1970 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009870 tx=0x7f03dc01c3f0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f905a640 1 -- 172.21.15.31:0/2035676554 shutdown_connections 2023-11-26T07:23:09.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f905a640 1 --2- 172.21.15.31:0/2035676554 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f40d1eb0 0x7f03f406bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f905a640 1 --2- 172.21.15.31:0/2035676554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f40cead0 0x7f03f40d1970 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f905a640 1 -- 172.21.15.31:0/2035676554 >> 172.21.15.31:0/2035676554 conn(0x7f03f40d33b0 msgr2=0x7f03f40d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:09.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f905a640 1 -- 172.21.15.31:0/2035676554 shutdown_connections 2023-11-26T07:23:09.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.033+0000 7f03f905a640 1 -- 172.21.15.31:0/2035676554 wait complete. 2023-11-26T07:23:09.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f905a640 1 Processor -- start 2023-11-26T07:23:09.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f905a640 1 -- start start 2023-11-26T07:23:09.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f905a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f40cead0 0x7f03f4142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f905a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f40d1eb0 0x7f03f40b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f905a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f415d7b0 0x7f03f41423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f905a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f4159010 con 0x7f03f415d7b0 2023-11-26T07:23:09.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f905a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f41557e0 con 0x7f03f40d1eb0 2023-11-26T07:23:09.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f905a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f4154f80 con 0x7f03f40cead0 2023-11-26T07:23:09.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f415d7b0 0x7f03f41423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f359e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f415d7b0 0x7f03f41423c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36284/0 (socket says 172.21.15.31:36284) 2023-11-26T07:23:09.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f359e640 1 -- 172.21.15.31:0/1914632961 learned_addr learned my addr 172.21.15.31:0/1914632961 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:09.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03ebfff640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f40d1eb0 0x7f03f40b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f359e640 1 -- 172.21.15.31:0/1914632961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f40cead0 msgr2=0x7f03f4142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f359e640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f40cead0 0x7f03f4142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f359e640 1 -- 172.21.15.31:0/1914632961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f40d1eb0 msgr2=0x7f03f40b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f359e640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f40d1eb0 0x7f03f40b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f359e640 1 -- 172.21.15.31:0/1914632961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc0094d0 con 0x7f03f415d7b0 2023-11-26T07:23:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03ebfff640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f40d1eb0 0x7f03f40b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f359e640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f415d7b0 0x7f03f41423c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00efc0 tx=0x7f03ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03eb7fe640 1 -- 172.21.15.31:0/1914632961 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec009b10 con 0x7f03f415d7b0 2023-11-26T07:23:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03eb7fe640 1 -- 172.21.15.31:0/1914632961 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ec00f040 con 0x7f03f415d7b0 2023-11-26T07:23:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03eb7fe640 1 -- 172.21.15.31:0/1914632961 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0146c0 con 0x7f03f415d7b0 2023-11-26T07:23:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f905a640 1 -- 172.21.15.31:0/1914632961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f40b64b0 con 0x7f03f415d7b0 2023-11-26T07:23:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.037+0000 7f03f905a640 1 -- 172.21.15.31:0/1914632961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f4155500 con 0x7f03f415d7b0 2023-11-26T07:23:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.041+0000 7f03f905a640 1 -- 172.21.15.31:0/1914632961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f404e850 con 0x7f03f415d7b0 2023-11-26T07:23:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.041+0000 7f03eb7fe640 1 -- 172.21.15.31:0/1914632961 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f03ec014820 con 0x7f03f415d7b0 2023-11-26T07:23:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.041+0000 7f03eb7fe640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f03d40665c0 0x7f03d4068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.041+0000 7f03eb7fe640 1 -- 172.21.15.31:0/1914632961 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03ec084aa0 con 0x7f03f415d7b0 2023-11-26T07:23:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.041+0000 7f03f2d9d640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f03d40665c0 0x7f03d4068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.041+0000 7f03f2d9d640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f03d40665c0 0x7f03d4068a80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03f40cda20 tx=0x7f03e4007970 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.045+0000 7f03eb7fe640 1 -- 172.21.15.31:0/1914632961 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03ec051d90 con 0x7f03f415d7b0 2023-11-26T07:23:09.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.201+0000 7f03f905a640 1 -- 172.21.15.31:0/1914632961 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f03f40b27f0 con 0x7f03d40665c0 2023-11-26T07:23:09.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.209+0000 7f03eb7fe640 1 -- 172.21.15.31:0/1914632961 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288884 (secure 0 0 0) 0x7f03f40b27f0 con 0x7f03d40665c0 2023-11-26T07:23:09.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:09.217 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:23:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 -- 172.21.15.31:0/1914632961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f03d40665c0 msgr2=0x7f03d4068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f03d40665c0 0x7f03d4068a80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03f40cda20 tx=0x7f03e4007970 comp rx=0 tx=0).stop 2023-11-26T07:23:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 -- 172.21.15.31:0/1914632961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f415d7b0 msgr2=0x7f03f41423c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f415d7b0 0x7f03f41423c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00efc0 tx=0x7f03ec00c490 comp rx=0 tx=0).stop 2023-11-26T07:23:09.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 -- 172.21.15.31:0/1914632961 shutdown_connections 2023-11-26T07:23:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f03d40665c0 0x7f03d4068a80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03f415d7b0 0x7f03f41423c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03f40d1eb0 0x7f03f40b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 --2- 172.21.15.31:0/1914632961 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f03f40cead0 0x7f03f4142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 -- 172.21.15.31:0/1914632961 >> 172.21.15.31:0/1914632961 conn(0x7f03f40d33b0 msgr2=0x7f03f4159f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 -- 172.21.15.31:0/1914632961 shutdown_connections 2023-11-26T07:23:09.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.213+0000 7f03f905a640 1 -- 172.21.15.31:0/1914632961 wait complete. 2023-11-26T07:23:09.236 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-11-26T07:23:08.209328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260292,"kb_used_data":10816,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714428,"statfs":{"total":773094113280,"available":772827574272,"internally_reserved":0,"allocated":11075584,"data_stored":5489085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.119955"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390193+0000","last_change":"2023-11-26T07:23:05.132846+0000","last_active":"2023-11-26T07:23:06.390193+0000","last_peered":"2023-11-26T07:23:06.390193+0000","last_clean":"2023-11-26T07:23:06.390193+0000","last_became_active":"2023-11-26T07:23:05.132557+0000","last_became_peered":"2023-11-26T07:23:05.132557+0000","last_unstale":"2023-11-26T07:23:06.390193+0000","last_undegraded":"2023-11-26T07:23:06.390193+0000","last_fullsized":"2023-11-26T07:23:06.390193+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:54.477719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.147578+0000","last_change":"2023-11-26T07:23:05.144465+0000","last_active":"2023-11-26T07:23:05.147578+0000","last_peered":"2023-11-26T07:23:05.147578+0000","last_clean":"2023-11-26T07:23:05.147578+0000","last_became_active":"2023-11-26T07:23:05.144176+0000","last_became_peered":"2023-11-26T07:23:05.144176+0000","last_unstale":"2023-11-26T07:23:05.147578+0000","last_undegraded":"2023-11-26T07:23:05.147578+0000","last_fullsized":"2023-11-26T07:23:05.147578+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:08:34.451843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410843+0000","last_change":"2023-11-26T07:23:05.144202+0000","last_active":"2023-11-26T07:23:06.410843+0000","last_peered":"2023-11-26T07:23:06.410843+0000","last_clean":"2023-11-26T07:23:06.410843+0000","last_became_active":"2023-11-26T07:23:05.143901+0000","last_became_peered":"2023-11-26T07:23:05.143901+0000","last_unstale":"2023-11-26T07:23:06.410843+0000","last_undegraded":"2023-11-26T07:23:06.410843+0000","last_fullsized":"2023-11-26T07:23:06.410843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:37:55.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390053+0000","last_change":"2023-11-26T07:23:05.127838+0000","last_active":"2023-11-26T07:23:06.390053+0000","last_peered":"2023-11-26T07:23:06.390053+0000","last_clean":"2023-11-26T07:23:06.390053+0000","last_became_active":"2023-11-26T07:23:05.127553+0000","last_became_peered":"2023-11-26T07:23:05.127553+0000","last_unstale":"2023-11-26T07:23:06.390053+0000","last_undegraded":"2023-11-26T07:23:06.390053+0000","last_fullsized":"2023-11-26T07:23:06.390053+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:33.155558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465380+0000","last_change":"2023-11-26T07:23:05.139886+0000","last_active":"2023-11-26T07:23:06.465380+0000","last_peered":"2023-11-26T07:23:06.465380+0000","last_clean":"2023-11-26T07:23:06.465380+0000","last_became_active":"2023-11-26T07:23:05.139541+0000","last_became_peered":"2023-11-26T07:23:05.139541+0000","last_unstale":"2023-11-26T07:23:06.465380+0000","last_undegraded":"2023-11-26T07:23:06.465380+0000","last_fullsized":"2023-11-26T07:23:06.465380+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:09:46.760185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139696+0000","last_change":"2023-11-26T07:23:05.144249+0000","last_active":"2023-11-26T07:23:06.139696+0000","last_peered":"2023-11-26T07:23:06.139696+0000","last_clean":"2023-11-26T07:23:06.139696+0000","last_became_active":"2023-11-26T07:23:05.143927+0000","last_became_peered":"2023-11-26T07:23:05.143927+0000","last_unstale":"2023-11-26T07:23:06.139696+0000","last_undegraded":"2023-11-26T07:23:06.139696+0000","last_fullsized":"2023-11-26T07:23:06.139696+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:47:07.166224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140825+0000","last_change":"2023-11-26T07:23:05.134065+0000","last_active":"2023-11-26T07:23:06.140825+0000","last_peered":"2023-11-26T07:23:06.140825+0000","last_clean":"2023-11-26T07:23:06.140825+0000","last_became_active":"2023-11-26T07:23:05.131455+0000","last_became_peered":"2023-11-26T07:23:05.131455+0000","last_unstale":"2023-11-26T07:23:06.140825+0000","last_undegraded":"2023-11-26T07:23:06.140825+0000","last_fullsized":"2023-11-26T07:23:06.140825+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:17:38.640369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410628+0000","last_change":"2023-11-26T07:23:05.161423+0000","last_active":"2023-11-26T07:23:06.410628+0000","last_peered":"2023-11-26T07:23:06.410628+0000","last_clean":"2023-11-26T07:23:06.410628+0000","last_became_active":"2023-11-26T07:23:05.161120+0000","last_became_peered":"2023-11-26T07:23:05.161120+0000","last_unstale":"2023-11-26T07:23:06.410628+0000","last_undegraded":"2023-11-26T07:23:06.410628+0000","last_fullsized":"2023-11-26T07:23:06.410628+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:12.122297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390186+0000","last_change":"2023-11-26T07:23:05.141806+0000","last_active":"2023-11-26T07:23:06.390186+0000","last_peered":"2023-11-26T07:23:06.390186+0000","last_clean":"2023-11-26T07:23:06.390186+0000","last_became_active":"2023-11-26T07:23:05.141509+0000","last_became_peered":"2023-11-26T07:23:05.141509+0000","last_unstale":"2023-11-26T07:23:06.390186+0000","last_undegraded":"2023-11-26T07:23:06.390186+0000","last_fullsized":"2023-11-26T07:23:06.390186+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:21:18.446192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389964+0000","last_change":"2023-11-26T07:23:05.141866+0000","last_active":"2023-11-26T07:23:06.389964+0000","last_peered":"2023-11-26T07:23:06.389964+0000","last_clean":"2023-11-26T07:23:06.389964+0000","last_became_active":"2023-11-26T07:23:05.141576+0000","last_became_peered":"2023-11-26T07:23:05.141576+0000","last_unstale":"2023-11-26T07:23:06.389964+0000","last_undegraded":"2023-11-26T07:23:06.389964+0000","last_fullsized":"2023-11-26T07:23:06.389964+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:53:41.971600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.133172+0000","last_change":"2023-11-26T07:23:05.127844+0000","last_active":"2023-11-26T07:23:05.133172+0000","last_peered":"2023-11-26T07:23:05.133172+0000","last_clean":"2023-11-26T07:23:05.133172+0000","last_became_active":"2023-11-26T07:23:05.127456+0000","last_became_peered":"2023-11-26T07:23:05.127456+0000","last_unstale":"2023-11-26T07:23:05.133172+0000","last_undegraded":"2023-11-26T07:23:05.133172+0000","last_fullsized":"2023-11-26T07:23:05.133172+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:06:14.987344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389982+0000","last_change":"2023-11-26T07:23:05.147915+0000","last_active":"2023-11-26T07:23:06.389982+0000","last_peered":"2023-11-26T07:23:06.389982+0000","last_clean":"2023-11-26T07:23:06.389982+0000","last_became_active":"2023-11-26T07:23:05.147628+0000","last_became_peered":"2023-11-26T07:23:05.147628+0000","last_unstale":"2023-11-26T07:23:06.389982+0000","last_undegraded":"2023-11-26T07:23:06.389982+0000","last_fullsized":"2023-11-26T07:23:06.389982+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:58:27.174766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410794+0000","last_change":"2023-11-26T07:23:05.148078+0000","last_active":"2023-11-26T07:23:06.410794+0000","last_peered":"2023-11-26T07:23:06.410794+0000","last_clean":"2023-11-26T07:23:06.410794+0000","last_became_active":"2023-11-26T07:23:05.147766+0000","last_became_peered":"2023-11-26T07:23:05.147766+0000","last_unstale":"2023-11-26T07:23:06.410794+0000","last_undegraded":"2023-11-26T07:23:06.410794+0000","last_fullsized":"2023-11-26T07:23:06.410794+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:41.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410510+0000","last_change":"2023-11-26T07:23:05.146986+0000","last_active":"2023-11-26T07:23:06.410510+0000","last_peered":"2023-11-26T07:23:06.410510+0000","last_clean":"2023-11-26T07:23:06.410510+0000","last_became_active":"2023-11-26T07:23:05.146710+0000","last_became_peered":"2023-11-26T07:23:05.146710+0000","last_unstale":"2023-11-26T07:23:06.410510+0000","last_undegraded":"2023-11-26T07:23:06.410510+0000","last_fullsized":"2023-11-26T07:23:06.410510+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:52:57.658909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139634+0000","last_change":"2023-11-26T07:23:05.145600+0000","last_active":"2023-11-26T07:23:06.139634+0000","last_peered":"2023-11-26T07:23:06.139634+0000","last_clean":"2023-11-26T07:23:06.139634+0000","last_became_active":"2023-11-26T07:23:05.145305+0000","last_became_peered":"2023-11-26T07:23:05.145305+0000","last_unstale":"2023-11-26T07:23:06.139634+0000","last_undegraded":"2023-11-26T07:23:06.139634+0000","last_fullsized":"2023-11-26T07:23:06.139634+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:15.966939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389603+0000","last_change":"2023-11-26T07:23:05.146778+0000","last_active":"2023-11-26T07:23:06.389603+0000","last_peered":"2023-11-26T07:23:06.389603+0000","last_clean":"2023-11-26T07:23:06.389603+0000","last_became_active":"2023-11-26T07:23:05.146276+0000","last_became_peered":"2023-11-26T07:23:05.146276+0000","last_unstale":"2023-11-26T07:23:06.389603+0000","last_undegraded":"2023-11-26T07:23:06.389603+0000","last_fullsized":"2023-11-26T07:23:06.389603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:21:41.468377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.133251+0000","last_change":"2023-11-26T07:23:05.127687+0000","last_active":"2023-11-26T07:23:05.133251+0000","last_peered":"2023-11-26T07:23:05.133251+0000","last_clean":"2023-11-26T07:23:05.133251+0000","last_became_active":"2023-11-26T07:23:05.127364+0000","last_became_peered":"2023-11-26T07:23:05.127364+0000","last_unstale":"2023-11-26T07:23:05.133251+0000","last_undegraded":"2023-11-26T07:23:05.133251+0000","last_fullsized":"2023-11-26T07:23:05.133251+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:46.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.141956+0000","last_change":"2023-11-26T07:23:05.127616+0000","last_active":"2023-11-26T07:23:06.141956+0000","last_peered":"2023-11-26T07:23:06.141956+0000","last_clean":"2023-11-26T07:23:06.141956+0000","last_became_active":"2023-11-26T07:23:05.126976+0000","last_became_peered":"2023-11-26T07:23:05.126976+0000","last_unstale":"2023-11-26T07:23:06.141956+0000","last_undegraded":"2023-11-26T07:23:06.141956+0000","last_fullsized":"2023-11-26T07:23:06.141956+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:25.410801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410937+0000","last_change":"2023-11-26T07:23:05.148473+0000","last_active":"2023-11-26T07:23:06.410937+0000","last_peered":"2023-11-26T07:23:06.410937+0000","last_clean":"2023-11-26T07:23:06.410937+0000","last_became_active":"2023-11-26T07:23:05.147711+0000","last_became_peered":"2023-11-26T07:23:05.147711+0000","last_unstale":"2023-11-26T07:23:06.410937+0000","last_undegraded":"2023-11-26T07:23:06.410937+0000","last_fullsized":"2023-11-26T07:23:06.410937+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:02.103968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.144941+0000","last_change":"2023-11-26T07:23:05.141222+0000","last_active":"2023-11-26T07:23:05.144941+0000","last_peered":"2023-11-26T07:23:05.144941+0000","last_clean":"2023-11-26T07:23:05.144941+0000","last_became_active":"2023-11-26T07:23:05.140938+0000","last_became_peered":"2023-11-26T07:23:05.140938+0000","last_unstale":"2023-11-26T07:23:05.144941+0000","last_undegraded":"2023-11-26T07:23:05.144941+0000","last_fullsized":"2023-11-26T07:23:05.144941+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:28:44.611417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.133648+0000","last_change":"2023-11-26T07:23:05.130893+0000","last_active":"2023-11-26T07:23:05.133648+0000","last_peered":"2023-11-26T07:23:05.133648+0000","last_clean":"2023-11-26T07:23:05.133648+0000","last_became_active":"2023-11-26T07:23:05.130586+0000","last_became_peered":"2023-11-26T07:23:05.130586+0000","last_unstale":"2023-11-26T07:23:05.133648+0000","last_undegraded":"2023-11-26T07:23:05.133648+0000","last_fullsized":"2023-11-26T07:23:05.133648+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:41.930667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139521+0000","last_change":"2023-11-26T07:23:05.129229+0000","last_active":"2023-11-26T07:23:05.139521+0000","last_peered":"2023-11-26T07:23:05.139521+0000","last_clean":"2023-11-26T07:23:05.139521+0000","last_became_active":"2023-11-26T07:23:05.128396+0000","last_became_peered":"2023-11-26T07:23:05.128396+0000","last_unstale":"2023-11-26T07:23:05.139521+0000","last_undegraded":"2023-11-26T07:23:05.139521+0000","last_fullsized":"2023-11-26T07:23:05.139521+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:48:06.630041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.147330+0000","last_change":"2023-11-26T07:23:05.142826+0000","last_active":"2023-11-26T07:23:05.147330+0000","last_peered":"2023-11-26T07:23:05.147330+0000","last_clean":"2023-11-26T07:23:05.147330+0000","last_became_active":"2023-11-26T07:23:05.142542+0000","last_became_peered":"2023-11-26T07:23:05.142542+0000","last_unstale":"2023-11-26T07:23:05.147330+0000","last_undegraded":"2023-11-26T07:23:05.147330+0000","last_fullsized":"2023-11-26T07:23:05.147330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:39:32.591740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.151384+0000","last_change":"2023-11-26T07:23:05.150532+0000","last_active":"2023-11-26T07:23:05.151384+0000","last_peered":"2023-11-26T07:23:05.151384+0000","last_clean":"2023-11-26T07:23:05.151384+0000","last_became_active":"2023-11-26T07:23:05.142740+0000","last_became_peered":"2023-11-26T07:23:05.142740+0000","last_unstale":"2023-11-26T07:23:05.151384+0000","last_undegraded":"2023-11-26T07:23:05.151384+0000","last_fullsized":"2023-11-26T07:23:05.151384+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:37:55.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.133738+0000","last_change":"2023-11-26T07:23:05.130516+0000","last_active":"2023-11-26T07:23:05.133738+0000","last_peered":"2023-11-26T07:23:05.133738+0000","last_clean":"2023-11-26T07:23:05.133738+0000","last_became_active":"2023-11-26T07:23:05.130169+0000","last_became_peered":"2023-11-26T07:23:05.130169+0000","last_unstale":"2023-11-26T07:23:05.133738+0000","last_undegraded":"2023-11-26T07:23:05.133738+0000","last_fullsized":"2023-11-26T07:23:05.133738+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:13:08.746625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390026+0000","last_change":"2023-11-26T07:23:05.135960+0000","last_active":"2023-11-26T07:23:06.390026+0000","last_peered":"2023-11-26T07:23:06.390026+0000","last_clean":"2023-11-26T07:23:06.390026+0000","last_became_active":"2023-11-26T07:23:05.135635+0000","last_became_peered":"2023-11-26T07:23:05.135635+0000","last_unstale":"2023-11-26T07:23:06.390026+0000","last_undegraded":"2023-11-26T07:23:06.390026+0000","last_fullsized":"2023-11-26T07:23:06.390026+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:20.430885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140134+0000","last_change":"2023-11-26T07:23:05.147446+0000","last_active":"2023-11-26T07:23:06.140134+0000","last_peered":"2023-11-26T07:23:06.140134+0000","last_clean":"2023-11-26T07:23:06.140134+0000","last_became_active":"2023-11-26T07:23:05.147185+0000","last_became_peered":"2023-11-26T07:23:05.147185+0000","last_unstale":"2023-11-26T07:23:06.140134+0000","last_undegraded":"2023-11-26T07:23:06.140134+0000","last_fullsized":"2023-11-26T07:23:06.140134+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:28:09.939466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139656+0000","last_change":"2023-11-26T07:23:05.129784+0000","last_active":"2023-11-26T07:23:05.139656+0000","last_peered":"2023-11-26T07:23:05.139656+0000","last_clean":"2023-11-26T07:23:05.139656+0000","last_became_active":"2023-11-26T07:23:05.129506+0000","last_became_peered":"2023-11-26T07:23:05.129506+0000","last_unstale":"2023-11-26T07:23:05.139656+0000","last_undegraded":"2023-11-26T07:23:05.139656+0000","last_fullsized":"2023-11-26T07:23:05.139656+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:53:28.488385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.147377+0000","last_change":"2023-11-26T07:23:05.144227+0000","last_active":"2023-11-26T07:23:05.147377+0000","last_peered":"2023-11-26T07:23:05.147377+0000","last_clean":"2023-11-26T07:23:05.147377+0000","last_became_active":"2023-11-26T07:23:05.143723+0000","last_became_peered":"2023-11-26T07:23:05.143723+0000","last_unstale":"2023-11-26T07:23:05.147377+0000","last_undegraded":"2023-11-26T07:23:05.147377+0000","last_fullsized":"2023-11-26T07:23:05.147377+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:50.220699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140506+0000","last_change":"2023-11-26T07:23:05.135631+0000","last_active":"2023-11-26T07:23:06.140506+0000","last_peered":"2023-11-26T07:23:06.140506+0000","last_clean":"2023-11-26T07:23:06.140506+0000","last_became_active":"2023-11-26T07:23:05.134335+0000","last_became_peered":"2023-11-26T07:23:05.134335+0000","last_unstale":"2023-11-26T07:23:06.140506+0000","last_undegraded":"2023-11-26T07:23:06.140506+0000","last_fullsized":"2023-11-26T07:23:06.140506+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:56.708048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389673+0000","last_change":"2023-11-26T07:23:05.136000+0000","last_active":"2023-11-26T07:23:06.389673+0000","last_peered":"2023-11-26T07:23:06.389673+0000","last_clean":"2023-11-26T07:23:06.389673+0000","last_became_active":"2023-11-26T07:23:05.135733+0000","last_became_peered":"2023-11-26T07:23:05.135733+0000","last_unstale":"2023-11-26T07:23:06.389673+0000","last_undegraded":"2023-11-26T07:23:06.389673+0000","last_fullsized":"2023-11-26T07:23:06.389673+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:18:27.600435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139634+0000","last_change":"2023-11-26T07:23:05.147344+0000","last_active":"2023-11-26T07:23:06.139634+0000","last_peered":"2023-11-26T07:23:06.139634+0000","last_clean":"2023-11-26T07:23:06.139634+0000","last_became_active":"2023-11-26T07:23:05.146985+0000","last_became_peered":"2023-11-26T07:23:05.146985+0000","last_unstale":"2023-11-26T07:23:06.139634+0000","last_undegraded":"2023-11-26T07:23:06.139634+0000","last_fullsized":"2023-11-26T07:23:06.139634+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:23:56.300262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140165+0000","last_change":"2023-11-26T07:23:05.153870+0000","last_active":"2023-11-26T07:23:06.140165+0000","last_peered":"2023-11-26T07:23:06.140165+0000","last_clean":"2023-11-26T07:23:06.140165+0000","last_became_active":"2023-11-26T07:23:05.153564+0000","last_became_peered":"2023-11-26T07:23:05.153564+0000","last_unstale":"2023-11-26T07:23:06.140165+0000","last_undegraded":"2023-11-26T07:23:06.140165+0000","last_fullsized":"2023-11-26T07:23:06.140165+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:26:16.296434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"98'4","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140756+0000","last_change":"2023-11-26T07:23:04.116235+0000","last_active":"2023-11-26T07:23:06.140756+0000","last_peered":"2023-11-26T07:23:06.140756+0000","last_clean":"2023-11-26T07:23:06.140756+0000","last_became_active":"2023-11-26T07:23:04.115926+0000","last_became_peered":"2023-11-26T07:23:04.115926+0000","last_unstale":"2023-11-26T07:23:06.140756+0000","last_undegraded":"2023-11-26T07:23:06.140756+0000","last_fullsized":"2023-11-26T07:23:06.140756+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:40.924680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.141983+0000","last_change":"2023-11-26T07:23:05.140365+0000","last_active":"2023-11-26T07:23:06.141983+0000","last_peered":"2023-11-26T07:23:06.141983+0000","last_clean":"2023-11-26T07:23:06.141983+0000","last_became_active":"2023-11-26T07:23:05.139709+0000","last_became_peered":"2023-11-26T07:23:05.139709+0000","last_unstale":"2023-11-26T07:23:06.141983+0000","last_undegraded":"2023-11-26T07:23:06.141983+0000","last_fullsized":"2023-11-26T07:23:06.141983+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:34:54.315569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"98'1","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410888+0000","last_change":"2023-11-26T07:23:04.117016+0000","last_active":"2023-11-26T07:23:06.410888+0000","last_peered":"2023-11-26T07:23:06.410888+0000","last_clean":"2023-11-26T07:23:06.410888+0000","last_became_active":"2023-11-26T07:23:04.116696+0000","last_became_peered":"2023-11-26T07:23:04.116696+0000","last_unstale":"2023-11-26T07:23:06.410888+0000","last_undegraded":"2023-11-26T07:23:06.410888+0000","last_fullsized":"2023-11-26T07:23:06.410888+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:12:26.540817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140009+0000","last_change":"2023-11-26T07:23:05.144556+0000","last_active":"2023-11-26T07:23:06.140009+0000","last_peered":"2023-11-26T07:23:06.140009+0000","last_clean":"2023-11-26T07:23:06.140009+0000","last_became_active":"2023-11-26T07:23:05.144288+0000","last_became_peered":"2023-11-26T07:23:05.144288+0000","last_unstale":"2023-11-26T07:23:06.140009+0000","last_undegraded":"2023-11-26T07:23:06.140009+0000","last_fullsized":"2023-11-26T07:23:06.140009+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:03:13.505342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"98'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465453+0000","last_change":"2023-11-26T07:23:04.121844+0000","last_active":"2023-11-26T07:23:06.465453+0000","last_peered":"2023-11-26T07:23:06.465453+0000","last_clean":"2023-11-26T07:23:06.465453+0000","last_became_active":"2023-11-26T07:23:04.120615+0000","last_became_peered":"2023-11-26T07:23:04.120615+0000","last_unstale":"2023-11-26T07:23:06.465453+0000","last_undegraded":"2023-11-26T07:23:06.465453+0000","last_fullsized":"2023-11-26T07:23:06.465453+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:09.053722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410985+0000","last_change":"2023-11-26T07:23:05.151228+0000","last_active":"2023-11-26T07:23:06.410985+0000","last_peered":"2023-11-26T07:23:06.410985+0000","last_clean":"2023-11-26T07:23:06.410985+0000","last_became_active":"2023-11-26T07:23:05.150837+0000","last_became_peered":"2023-11-26T07:23:05.150837+0000","last_unstale":"2023-11-26T07:23:06.410985+0000","last_undegraded":"2023-11-26T07:23:06.410985+0000","last_fullsized":"2023-11-26T07:23:06.410985+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:46.037771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"98'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465292+0000","last_change":"2023-11-26T07:23:04.122113+0000","last_active":"2023-11-26T07:23:06.465292+0000","last_peered":"2023-11-26T07:23:06.465292+0000","last_clean":"2023-11-26T07:23:06.465292+0000","last_became_active":"2023-11-26T07:23:04.119442+0000","last_became_peered":"2023-11-26T07:23:04.119442+0000","last_unstale":"2023-11-26T07:23:06.465292+0000","last_undegraded":"2023-11-26T07:23:06.465292+0000","last_fullsized":"2023-11-26T07:23:06.465292+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:14.681311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389646+0000","last_change":"2023-11-26T07:23:05.129737+0000","last_active":"2023-11-26T07:23:06.389646+0000","last_peered":"2023-11-26T07:23:06.389646+0000","last_clean":"2023-11-26T07:23:06.389646+0000","last_became_active":"2023-11-26T07:23:05.129423+0000","last_became_peered":"2023-11-26T07:23:05.129423+0000","last_unstale":"2023-11-26T07:23:06.389646+0000","last_undegraded":"2023-11-26T07:23:06.389646+0000","last_fullsized":"2023-11-26T07:23:06.389646+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:03:31.063138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"98'1","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140448+0000","last_change":"2023-11-26T07:23:04.119430+0000","last_active":"2023-11-26T07:23:06.140448+0000","last_peered":"2023-11-26T07:23:06.140448+0000","last_clean":"2023-11-26T07:23:06.140448+0000","last_became_active":"2023-11-26T07:23:04.118426+0000","last_became_peered":"2023-11-26T07:23:04.118426+0000","last_unstale":"2023-11-26T07:23:06.140448+0000","last_undegraded":"2023-11-26T07:23:06.140448+0000","last_fullsized":"2023-11-26T07:23:06.140448+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:12:26.540817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139139+0000","last_change":"2023-11-26T07:23:05.137129+0000","last_active":"2023-11-26T07:23:05.139139+0000","last_peered":"2023-11-26T07:23:05.139139+0000","last_clean":"2023-11-26T07:23:05.139139+0000","last_became_active":"2023-11-26T07:23:05.136818+0000","last_became_peered":"2023-11-26T07:23:05.136818+0000","last_unstale":"2023-11-26T07:23:05.139139+0000","last_undegraded":"2023-11-26T07:23:05.139139+0000","last_fullsized":"2023-11-26T07:23:05.139139+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:32:49.761866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"98'1","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.161276+0000","last_change":"2023-11-26T07:23:04.117349+0000","last_active":"2023-11-26T07:23:05.161276+0000","last_peered":"2023-11-26T07:23:05.161276+0000","last_clean":"2023-11-26T07:23:05.161276+0000","last_became_active":"2023-11-26T07:23:04.117029+0000","last_became_peered":"2023-11-26T07:23:04.117029+0000","last_unstale":"2023-11-26T07:23:05.161276+0000","last_undegraded":"2023-11-26T07:23:05.161276+0000","last_fullsized":"2023-11-26T07:23:05.161276+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:45.031785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389915+0000","last_change":"2023-11-26T07:23:05.146550+0000","last_active":"2023-11-26T07:23:06.389915+0000","last_peered":"2023-11-26T07:23:06.389915+0000","last_clean":"2023-11-26T07:23:06.389915+0000","last_became_active":"2023-11-26T07:23:05.145890+0000","last_became_peered":"2023-11-26T07:23:05.145890+0000","last_unstale":"2023-11-26T07:23:06.389915+0000","last_undegraded":"2023-11-26T07:23:06.389915+0000","last_fullsized":"2023-11-26T07:23:06.389915+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:43.433697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.120007+0000","last_change":"2023-11-26T07:23:04.116997+0000","last_active":"2023-11-26T07:23:05.120007+0000","last_peered":"2023-11-26T07:23:05.120007+0000","last_clean":"2023-11-26T07:23:05.120007+0000","last_became_active":"2023-11-26T07:23:04.116698+0000","last_became_peered":"2023-11-26T07:23:04.116698+0000","last_unstale":"2023-11-26T07:23:05.120007+0000","last_undegraded":"2023-11-26T07:23:05.120007+0000","last_fullsized":"2023-11-26T07:23:05.120007+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:52:56.652922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140777+0000","last_change":"2023-11-26T07:23:05.133574+0000","last_active":"2023-11-26T07:23:06.140777+0000","last_peered":"2023-11-26T07:23:06.140777+0000","last_clean":"2023-11-26T07:23:06.140777+0000","last_became_active":"2023-11-26T07:23:05.133234+0000","last_became_peered":"2023-11-26T07:23:05.133234+0000","last_unstale":"2023-11-26T07:23:06.140777+0000","last_undegraded":"2023-11-26T07:23:06.140777+0000","last_fullsized":"2023-11-26T07:23:06.140777+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:13:08.746625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389933+0000","last_change":"2023-11-26T07:23:04.120957+0000","last_active":"2023-11-26T07:23:06.389933+0000","last_peered":"2023-11-26T07:23:06.389933+0000","last_clean":"2023-11-26T07:23:06.389933+0000","last_became_active":"2023-11-26T07:23:04.117583+0000","last_became_peered":"2023-11-26T07:23:04.117583+0000","last_unstale":"2023-11-26T07:23:06.389933+0000","last_undegraded":"2023-11-26T07:23:06.389933+0000","last_fullsized":"2023-11-26T07:23:06.389933+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:54.448486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140873+0000","last_change":"2023-11-26T07:23:05.140844+0000","last_active":"2023-11-26T07:23:06.140873+0000","last_peered":"2023-11-26T07:23:06.140873+0000","last_clean":"2023-11-26T07:23:06.140873+0000","last_became_active":"2023-11-26T07:23:05.140417+0000","last_became_peered":"2023-11-26T07:23:05.140417+0000","last_unstale":"2023-11-26T07:23:06.140873+0000","last_undegraded":"2023-11-26T07:23:06.140873+0000","last_fullsized":"2023-11-26T07:23:06.140873+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:46.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465512+0000","last_change":"2023-11-26T07:23:04.123177+0000","last_active":"2023-11-26T07:23:06.465512+0000","last_peered":"2023-11-26T07:23:06.465512+0000","last_clean":"2023-11-26T07:23:06.465512+0000","last_became_active":"2023-11-26T07:23:04.122833+0000","last_became_peered":"2023-11-26T07:23:04.122833+0000","last_unstale":"2023-11-26T07:23:06.465512+0000","last_undegraded":"2023-11-26T07:23:06.465512+0000","last_fullsized":"2023-11-26T07:23:06.465512+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:33:32.019518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.142072+0000","last_change":"2023-11-26T07:23:05.136873+0000","last_active":"2023-11-26T07:23:06.142072+0000","last_peered":"2023-11-26T07:23:06.142072+0000","last_clean":"2023-11-26T07:23:06.142072+0000","last_became_active":"2023-11-26T07:23:05.136582+0000","last_became_peered":"2023-11-26T07:23:05.136582+0000","last_unstale":"2023-11-26T07:23:06.142072+0000","last_undegraded":"2023-11-26T07:23:06.142072+0000","last_fullsized":"2023-11-26T07:23:06.142072+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:55.454473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.117875+0000","last_change":"2023-11-26T07:23:04.117992+0000","last_active":"2023-11-26T07:23:05.117875+0000","last_peered":"2023-11-26T07:23:05.117875+0000","last_clean":"2023-11-26T07:23:05.117875+0000","last_became_active":"2023-11-26T07:23:04.117720+0000","last_became_peered":"2023-11-26T07:23:04.117720+0000","last_unstale":"2023-11-26T07:23:05.117875+0000","last_undegraded":"2023-11-26T07:23:05.117875+0000","last_fullsized":"2023-11-26T07:23:05.117875+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:24.404815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139636+0000","last_change":"2023-11-26T07:23:05.128563+0000","last_active":"2023-11-26T07:23:05.139636+0000","last_peered":"2023-11-26T07:23:05.139636+0000","last_clean":"2023-11-26T07:23:05.139636+0000","last_became_active":"2023-11-26T07:23:05.128266+0000","last_became_peered":"2023-11-26T07:23:05.128266+0000","last_unstale":"2023-11-26T07:23:05.139636+0000","last_undegraded":"2023-11-26T07:23:05.139636+0000","last_fullsized":"2023-11-26T07:23:05.139636+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:55.454473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"98'3","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389818+0000","last_change":"2023-11-26T07:23:04.120748+0000","last_active":"2023-11-26T07:23:06.389818+0000","last_peered":"2023-11-26T07:23:06.389818+0000","last_clean":"2023-11-26T07:23:06.389818+0000","last_became_active":"2023-11-26T07:23:04.120459+0000","last_became_peered":"2023-11-26T07:23:04.120459+0000","last_unstale":"2023-11-26T07:23:06.389818+0000","last_undegraded":"2023-11-26T07:23:06.389818+0000","last_fullsized":"2023-11-26T07:23:06.389818+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:02:06.833405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140631+0000","last_change":"2023-11-26T07:23:05.139974+0000","last_active":"2023-11-26T07:23:06.140631+0000","last_peered":"2023-11-26T07:23:06.140631+0000","last_clean":"2023-11-26T07:23:06.140631+0000","last_became_active":"2023-11-26T07:23:05.139686+0000","last_became_peered":"2023-11-26T07:23:05.139686+0000","last_unstale":"2023-11-26T07:23:06.140631+0000","last_undegraded":"2023-11-26T07:23:06.140631+0000","last_fullsized":"2023-11-26T07:23:06.140631+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:08:34.451843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"98'6","reported_seq":18,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390198+0000","last_change":"2023-11-26T07:23:04.125148+0000","last_active":"2023-11-26T07:23:06.390198+0000","last_peered":"2023-11-26T07:23:06.390198+0000","last_clean":"2023-11-26T07:23:06.390198+0000","last_became_active":"2023-11-26T07:23:04.124767+0000","last_became_peered":"2023-11-26T07:23:04.124767+0000","last_unstale":"2023-11-26T07:23:06.390198+0000","last_undegraded":"2023-11-26T07:23:06.390198+0000","last_fullsized":"2023-11-26T07:23:06.390198+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:06:32.768478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.144217+0000","last_change":"2023-11-26T07:23:05.140008+0000","last_active":"2023-11-26T07:23:05.144217+0000","last_peered":"2023-11-26T07:23:05.144217+0000","last_clean":"2023-11-26T07:23:05.144217+0000","last_became_active":"2023-11-26T07:23:05.139614+0000","last_became_peered":"2023-11-26T07:23:05.139614+0000","last_unstale":"2023-11-26T07:23:05.144217+0000","last_undegraded":"2023-11-26T07:23:05.144217+0000","last_fullsized":"2023-11-26T07:23:05.144217+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:41.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"98'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139758+0000","last_change":"2023-11-26T07:23:04.115520+0000","last_active":"2023-11-26T07:23:06.139758+0000","last_peered":"2023-11-26T07:23:06.139758+0000","last_clean":"2023-11-26T07:23:06.139758+0000","last_became_active":"2023-11-26T07:23:04.115161+0000","last_became_peered":"2023-11-26T07:23:04.115161+0000","last_unstale":"2023-11-26T07:23:06.139758+0000","last_undegraded":"2023-11-26T07:23:06.139758+0000","last_fullsized":"2023-11-26T07:23:06.139758+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:53:24.100793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.146701+0000","last_change":"2023-11-26T07:23:05.144906+0000","last_active":"2023-11-26T07:23:05.146701+0000","last_peered":"2023-11-26T07:23:05.146701+0000","last_clean":"2023-11-26T07:23:05.146701+0000","last_became_active":"2023-11-26T07:23:05.144644+0000","last_became_peered":"2023-11-26T07:23:05.144644+0000","last_unstale":"2023-11-26T07:23:05.146701+0000","last_undegraded":"2023-11-26T07:23:05.146701+0000","last_fullsized":"2023-11-26T07:23:05.146701+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:58:18.459826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.141696+0000","last_change":"2023-11-26T07:23:04.107900+0000","last_active":"2023-11-26T07:23:06.141696+0000","last_peered":"2023-11-26T07:23:06.141696+0000","last_clean":"2023-11-26T07:23:06.141696+0000","last_became_active":"2023-11-26T07:23:04.107344+0000","last_became_peered":"2023-11-26T07:23:04.107344+0000","last_unstale":"2023-11-26T07:23:06.141696+0000","last_undegraded":"2023-11-26T07:23:06.141696+0000","last_fullsized":"2023-11-26T07:23:06.141696+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:49.214712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139606+0000","last_change":"2023-11-26T07:23:05.128528+0000","last_active":"2023-11-26T07:23:05.139606+0000","last_peered":"2023-11-26T07:23:05.139606+0000","last_clean":"2023-11-26T07:23:05.139606+0000","last_became_active":"2023-11-26T07:23:05.128195+0000","last_became_peered":"2023-11-26T07:23:05.128195+0000","last_unstale":"2023-11-26T07:23:05.139606+0000","last_undegraded":"2023-11-26T07:23:05.139606+0000","last_fullsized":"2023-11-26T07:23:05.139606+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:54:34.159473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"98'8","reported_seq":20,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465342+0000","last_change":"2023-11-26T07:23:04.123499+0000","last_active":"2023-11-26T07:23:06.465342+0000","last_peered":"2023-11-26T07:23:06.465342+0000","last_clean":"2023-11-26T07:23:06.465342+0000","last_became_active":"2023-11-26T07:23:04.123193+0000","last_became_peered":"2023-11-26T07:23:04.123193+0000","last_unstale":"2023-11-26T07:23:06.465342+0000","last_undegraded":"2023-11-26T07:23:06.465342+0000","last_fullsized":"2023-11-26T07:23:06.465342+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:20:29.898632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465603+0000","last_change":"2023-11-26T07:23:05.142191+0000","last_active":"2023-11-26T07:23:06.465603+0000","last_peered":"2023-11-26T07:23:06.465603+0000","last_clean":"2023-11-26T07:23:06.465603+0000","last_became_active":"2023-11-26T07:23:05.141871+0000","last_became_peered":"2023-11-26T07:23:05.141871+0000","last_unstale":"2023-11-26T07:23:06.465603+0000","last_undegraded":"2023-11-26T07:23:06.465603+0000","last_fullsized":"2023-11-26T07:23:06.465603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:10:43.619217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"98'1","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390288+0000","last_change":"2023-11-26T07:23:04.118804+0000","last_active":"2023-11-26T07:23:06.390288+0000","last_peered":"2023-11-26T07:23:06.390288+0000","last_clean":"2023-11-26T07:23:06.390288+0000","last_became_active":"2023-11-26T07:23:04.117569+0000","last_became_peered":"2023-11-26T07:23:04.117569+0000","last_unstale":"2023-11-26T07:23:06.390288+0000","last_undegraded":"2023-11-26T07:23:06.390288+0000","last_fullsized":"2023-11-26T07:23:06.390288+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:20.217770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140255+0000","last_change":"2023-11-26T07:23:05.150851+0000","last_active":"2023-11-26T07:23:06.140255+0000","last_peered":"2023-11-26T07:23:06.140255+0000","last_clean":"2023-11-26T07:23:06.140255+0000","last_became_active":"2023-11-26T07:23:05.149804+0000","last_became_peered":"2023-11-26T07:23:05.149804+0000","last_unstale":"2023-11-26T07:23:06.140255+0000","last_undegraded":"2023-11-26T07:23:06.140255+0000","last_fullsized":"2023-11-26T07:23:06.140255+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:38:18.950490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"98'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465574+0000","last_change":"2023-11-26T07:23:04.113521+0000","last_active":"2023-11-26T07:23:06.465574+0000","last_peered":"2023-11-26T07:23:06.465574+0000","last_clean":"2023-11-26T07:23:06.465574+0000","last_became_active":"2023-11-26T07:23:04.113200+0000","last_became_peered":"2023-11-26T07:23:04.113200+0000","last_unstale":"2023-11-26T07:23:06.465574+0000","last_undegraded":"2023-11-26T07:23:06.465574+0000","last_fullsized":"2023-11-26T07:23:06.465574+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:29:45.912508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410767+0000","last_change":"2023-11-26T07:23:05.144415+0000","last_active":"2023-11-26T07:23:06.410767+0000","last_peered":"2023-11-26T07:23:06.410767+0000","last_clean":"2023-11-26T07:23:06.410767+0000","last_became_active":"2023-11-26T07:23:05.143499+0000","last_became_peered":"2023-11-26T07:23:05.143499+0000","last_unstale":"2023-11-26T07:23:06.410767+0000","last_undegraded":"2023-11-26T07:23:06.410767+0000","last_fullsized":"2023-11-26T07:23:06.410767+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:48:06.630041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"98'1","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.141858+0000","last_change":"2023-11-26T07:23:04.118905+0000","last_active":"2023-11-26T07:23:06.141858+0000","last_peered":"2023-11-26T07:23:06.141858+0000","last_clean":"2023-11-26T07:23:06.141858+0000","last_became_active":"2023-11-26T07:23:04.118510+0000","last_became_peered":"2023-11-26T07:23:04.118510+0000","last_unstale":"2023-11-26T07:23:06.141858+0000","last_undegraded":"2023-11-26T07:23:06.141858+0000","last_fullsized":"2023-11-26T07:23:06.141858+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:37:54.266592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.411054+0000","last_change":"2023-11-26T07:23:05.142827+0000","last_active":"2023-11-26T07:23:06.411054+0000","last_peered":"2023-11-26T07:23:06.411054+0000","last_clean":"2023-11-26T07:23:06.411054+0000","last_became_active":"2023-11-26T07:23:05.142577+0000","last_became_peered":"2023-11-26T07:23:05.142577+0000","last_unstale":"2023-11-26T07:23:06.411054+0000","last_undegraded":"2023-11-26T07:23:06.411054+0000","last_fullsized":"2023-11-26T07:23:06.411054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:34:54.315569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140446+0000","last_change":"2023-11-26T07:23:04.117690+0000","last_active":"2023-11-26T07:23:06.140446+0000","last_peered":"2023-11-26T07:23:06.140446+0000","last_clean":"2023-11-26T07:23:06.140446+0000","last_became_active":"2023-11-26T07:23:04.117410+0000","last_became_peered":"2023-11-26T07:23:04.117410+0000","last_unstale":"2023-11-26T07:23:06.140446+0000","last_undegraded":"2023-11-26T07:23:06.140446+0000","last_fullsized":"2023-11-26T07:23:06.140446+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:08:47.550078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410847+0000","last_change":"2023-11-26T07:23:05.141406+0000","last_active":"2023-11-26T07:23:06.410847+0000","last_peered":"2023-11-26T07:23:06.410847+0000","last_clean":"2023-11-26T07:23:06.410847+0000","last_became_active":"2023-11-26T07:23:05.140976+0000","last_became_peered":"2023-11-26T07:23:05.140976+0000","last_unstale":"2023-11-26T07:23:06.410847+0000","last_undegraded":"2023-11-26T07:23:06.410847+0000","last_fullsized":"2023-11-26T07:23:06.410847+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:54:34.159473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"98'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139857+0000","last_change":"2023-11-26T07:23:04.126106+0000","last_active":"2023-11-26T07:23:06.139857+0000","last_peered":"2023-11-26T07:23:06.139857+0000","last_clean":"2023-11-26T07:23:06.139857+0000","last_became_active":"2023-11-26T07:23:04.125831+0000","last_became_peered":"2023-11-26T07:23:04.125831+0000","last_unstale":"2023-11-26T07:23:06.139857+0000","last_undegraded":"2023-11-26T07:23:06.139857+0000","last_fullsized":"2023-11-26T07:23:06.139857+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:14:33.220876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389653+0000","last_change":"2023-11-26T07:23:05.143300+0000","last_active":"2023-11-26T07:23:06.389653+0000","last_peered":"2023-11-26T07:23:06.389653+0000","last_clean":"2023-11-26T07:23:06.389653+0000","last_became_active":"2023-11-26T07:23:05.143001+0000","last_became_peered":"2023-11-26T07:23:05.143001+0000","last_unstale":"2023-11-26T07:23:06.389653+0000","last_undegraded":"2023-11-26T07:23:06.389653+0000","last_fullsized":"2023-11-26T07:23:06.389653+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:35.400061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465219+0000","last_change":"2023-11-26T07:23:04.124425+0000","last_active":"2023-11-26T07:23:06.465219+0000","last_peered":"2023-11-26T07:23:06.465219+0000","last_clean":"2023-11-26T07:23:06.465219+0000","last_became_active":"2023-11-26T07:23:04.124162+0000","last_became_peered":"2023-11-26T07:23:04.124162+0000","last_unstale":"2023-11-26T07:23:06.465219+0000","last_undegraded":"2023-11-26T07:23:06.465219+0000","last_fullsized":"2023-11-26T07:23:06.465219+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:42:44.097894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410689+0000","last_change":"2023-11-26T07:23:05.148143+0000","last_active":"2023-11-26T07:23:06.410689+0000","last_peered":"2023-11-26T07:23:06.410689+0000","last_clean":"2023-11-26T07:23:06.410689+0000","last_became_active":"2023-11-26T07:23:05.147786+0000","last_became_peered":"2023-11-26T07:23:05.147786+0000","last_unstale":"2023-11-26T07:23:06.410689+0000","last_undegraded":"2023-11-26T07:23:06.410689+0000","last_fullsized":"2023-11-26T07:23:06.410689+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:58:18.459826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139652+0000","last_change":"2023-11-26T07:23:04.127017+0000","last_active":"2023-11-26T07:23:06.139652+0000","last_peered":"2023-11-26T07:23:06.139652+0000","last_clean":"2023-11-26T07:23:06.139652+0000","last_became_active":"2023-11-26T07:23:04.126670+0000","last_became_peered":"2023-11-26T07:23:04.126670+0000","last_unstale":"2023-11-26T07:23:06.139652+0000","last_undegraded":"2023-11-26T07:23:06.139652+0000","last_fullsized":"2023-11-26T07:23:06.139652+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:12.053806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139902+0000","last_change":"2023-11-26T07:23:05.131466+0000","last_active":"2023-11-26T07:23:05.139902+0000","last_peered":"2023-11-26T07:23:05.139902+0000","last_clean":"2023-11-26T07:23:05.139902+0000","last_became_active":"2023-11-26T07:23:05.131182+0000","last_became_peered":"2023-11-26T07:23:05.131182+0000","last_unstale":"2023-11-26T07:23:05.139902+0000","last_undegraded":"2023-11-26T07:23:05.139902+0000","last_fullsized":"2023-11-26T07:23:05.139902+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:02.103968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390060+0000","last_change":"2023-11-26T07:23:04.110115+0000","last_active":"2023-11-26T07:23:06.390060+0000","last_peered":"2023-11-26T07:23:06.390060+0000","last_clean":"2023-11-26T07:23:06.390060+0000","last_became_active":"2023-11-26T07:23:04.109456+0000","last_became_peered":"2023-11-26T07:23:04.109456+0000","last_unstale":"2023-11-26T07:23:06.390060+0000","last_undegraded":"2023-11-26T07:23:06.390060+0000","last_fullsized":"2023-11-26T07:23:06.390060+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:20:19.158427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.142067+0000","last_change":"2023-11-26T07:23:05.136849+0000","last_active":"2023-11-26T07:23:06.142067+0000","last_peered":"2023-11-26T07:23:06.142067+0000","last_clean":"2023-11-26T07:23:06.142067+0000","last_became_active":"2023-11-26T07:23:05.136552+0000","last_became_peered":"2023-11-26T07:23:05.136552+0000","last_unstale":"2023-11-26T07:23:06.142067+0000","last_undegraded":"2023-11-26T07:23:06.142067+0000","last_fullsized":"2023-11-26T07:23:06.142067+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:53:28.488385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"98'2","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.164921+0000","last_change":"2023-11-26T07:23:04.125258+0000","last_active":"2023-11-26T07:23:05.164921+0000","last_peered":"2023-11-26T07:23:05.164921+0000","last_clean":"2023-11-26T07:23:05.164921+0000","last_became_active":"2023-11-26T07:23:04.124963+0000","last_became_peered":"2023-11-26T07:23:04.124963+0000","last_unstale":"2023-11-26T07:23:05.164921+0000","last_undegraded":"2023-11-26T07:23:05.164921+0000","last_fullsized":"2023-11-26T07:23:05.164921+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:34:53.309582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390317+0000","last_change":"2023-11-26T07:23:05.131493+0000","last_active":"2023-11-26T07:23:06.390317+0000","last_peered":"2023-11-26T07:23:06.390317+0000","last_clean":"2023-11-26T07:23:06.390317+0000","last_became_active":"2023-11-26T07:23:05.131158+0000","last_became_peered":"2023-11-26T07:23:05.131158+0000","last_unstale":"2023-11-26T07:23:06.390317+0000","last_undegraded":"2023-11-26T07:23:06.390317+0000","last_fullsized":"2023-11-26T07:23:06.390317+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:28:39.194729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.142052+0000","last_change":"2023-11-26T07:23:04.110606+0000","last_active":"2023-11-26T07:23:06.142052+0000","last_peered":"2023-11-26T07:23:06.142052+0000","last_clean":"2023-11-26T07:23:06.142052+0000","last_became_active":"2023-11-26T07:23:04.110255+0000","last_became_peered":"2023-11-26T07:23:04.110255+0000","last_unstale":"2023-11-26T07:23:06.142052+0000","last_undegraded":"2023-11-26T07:23:06.142052+0000","last_fullsized":"2023-11-26T07:23:06.142052+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:54:33.153486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390138+0000","last_change":"2023-11-26T07:23:05.143408+0000","last_active":"2023-11-26T07:23:06.390138+0000","last_peered":"2023-11-26T07:23:06.390138+0000","last_clean":"2023-11-26T07:23:06.390138+0000","last_became_active":"2023-11-26T07:23:05.142937+0000","last_became_peered":"2023-11-26T07:23:05.142937+0000","last_unstale":"2023-11-26T07:23:06.390138+0000","last_undegraded":"2023-11-26T07:23:06.390138+0000","last_fullsized":"2023-11-26T07:23:06.390138+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:57.245618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389858+0000","last_change":"2023-11-26T07:23:04.108545+0000","last_active":"2023-11-26T07:23:06.389858+0000","last_peered":"2023-11-26T07:23:06.389858+0000","last_clean":"2023-11-26T07:23:06.389858+0000","last_became_active":"2023-11-26T07:23:04.108220+0000","last_became_peered":"2023-11-26T07:23:04.108220+0000","last_unstale":"2023-11-26T07:23:06.389858+0000","last_undegraded":"2023-11-26T07:23:06.389858+0000","last_fullsized":"2023-11-26T07:23:06.389858+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:09:15.559785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.141937+0000","last_change":"2023-11-26T07:23:05.130219+0000","last_active":"2023-11-26T07:23:06.141937+0000","last_peered":"2023-11-26T07:23:06.141937+0000","last_clean":"2023-11-26T07:23:06.141937+0000","last_became_active":"2023-11-26T07:23:05.129451+0000","last_became_peered":"2023-11-26T07:23:05.129451+0000","last_unstale":"2023-11-26T07:23:06.141937+0000","last_undegraded":"2023-11-26T07:23:06.141937+0000","last_fullsized":"2023-11-26T07:23:06.141937+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:46.037771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.118889+0000","last_change":"2023-11-26T07:23:04.129022+0000","last_active":"2023-11-26T07:23:05.118889+0000","last_peered":"2023-11-26T07:23:05.118889+0000","last_clean":"2023-11-26T07:23:05.118889+0000","last_became_active":"2023-11-26T07:23:04.128705+0000","last_became_peered":"2023-11-26T07:23:04.128705+0000","last_unstale":"2023-11-26T07:23:05.118889+0000","last_undegraded":"2023-11-26T07:23:05.118889+0000","last_fullsized":"2023-11-26T07:23:05.118889+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:59.160408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.141787+0000","last_change":"2023-11-26T07:23:05.138768+0000","last_active":"2023-11-26T07:23:05.141787+0000","last_peered":"2023-11-26T07:23:05.141787+0000","last_clean":"2023-11-26T07:23:05.141787+0000","last_became_active":"2023-11-26T07:23:05.138475+0000","last_became_peered":"2023-11-26T07:23:05.138475+0000","last_unstale":"2023-11-26T07:23:05.141787+0000","last_undegraded":"2023-11-26T07:23:05.141787+0000","last_fullsized":"2023-11-26T07:23:05.141787+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:56.708048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140547+0000","last_change":"2023-11-26T07:23:04.108594+0000","last_active":"2023-11-26T07:23:06.140547+0000","last_peered":"2023-11-26T07:23:06.140547+0000","last_clean":"2023-11-26T07:23:06.140547+0000","last_became_active":"2023-11-26T07:23:04.108285+0000","last_became_peered":"2023-11-26T07:23:04.108285+0000","last_unstale":"2023-11-26T07:23:06.140547+0000","last_undegraded":"2023-11-26T07:23:06.140547+0000","last_fullsized":"2023-11-26T07:23:06.140547+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:59:26.288227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.147359+0000","last_change":"2023-11-26T07:23:05.142869+0000","last_active":"2023-11-26T07:23:05.147359+0000","last_peered":"2023-11-26T07:23:05.147359+0000","last_clean":"2023-11-26T07:23:05.147359+0000","last_became_active":"2023-11-26T07:23:05.142536+0000","last_became_peered":"2023-11-26T07:23:05.142536+0000","last_unstale":"2023-11-26T07:23:05.147359+0000","last_undegraded":"2023-11-26T07:23:05.147359+0000","last_fullsized":"2023-11-26T07:23:05.147359+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:17:38.640369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.120660+0000","last_change":"2023-11-26T07:23:04.118603+0000","last_active":"2023-11-26T07:23:05.120660+0000","last_peered":"2023-11-26T07:23:05.120660+0000","last_clean":"2023-11-26T07:23:05.120660+0000","last_became_active":"2023-11-26T07:23:04.118302+0000","last_became_peered":"2023-11-26T07:23:04.118302+0000","last_unstale":"2023-11-26T07:23:05.120660+0000","last_undegraded":"2023-11-26T07:23:05.120660+0000","last_fullsized":"2023-11-26T07:23:05.120660+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:11.116310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410962+0000","last_change":"2023-11-26T07:23:05.146889+0000","last_active":"2023-11-26T07:23:06.410962+0000","last_peered":"2023-11-26T07:23:06.410962+0000","last_clean":"2023-11-26T07:23:06.410962+0000","last_became_active":"2023-11-26T07:23:05.146520+0000","last_became_peered":"2023-11-26T07:23:05.146520+0000","last_unstale":"2023-11-26T07:23:06.410962+0000","last_undegraded":"2023-11-26T07:23:06.410962+0000","last_fullsized":"2023-11-26T07:23:06.410962+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:25.410801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465337+0000","last_change":"2023-11-26T07:23:04.122675+0000","last_active":"2023-11-26T07:23:06.465337+0000","last_peered":"2023-11-26T07:23:06.465337+0000","last_clean":"2023-11-26T07:23:06.465337+0000","last_became_active":"2023-11-26T07:23:04.119010+0000","last_became_peered":"2023-11-26T07:23:04.119010+0000","last_unstale":"2023-11-26T07:23:06.465337+0000","last_undegraded":"2023-11-26T07:23:06.465337+0000","last_fullsized":"2023-11-26T07:23:06.465337+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:02:45.014974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389815+0000","last_change":"2023-11-26T07:23:05.135800+0000","last_active":"2023-11-26T07:23:06.389815+0000","last_peered":"2023-11-26T07:23:06.389815+0000","last_clean":"2023-11-26T07:23:06.389815+0000","last_became_active":"2023-11-26T07:23:05.135365+0000","last_became_peered":"2023-11-26T07:23:05.135365+0000","last_unstale":"2023-11-26T07:23:06.389815+0000","last_undegraded":"2023-11-26T07:23:06.389815+0000","last_fullsized":"2023-11-26T07:23:06.389815+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:24.633386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390032+0000","last_change":"2023-11-26T07:23:04.110306+0000","last_active":"2023-11-26T07:23:06.390032+0000","last_peered":"2023-11-26T07:23:06.390032+0000","last_clean":"2023-11-26T07:23:06.390032+0000","last_became_active":"2023-11-26T07:23:04.109827+0000","last_became_peered":"2023-11-26T07:23:04.109827+0000","last_unstale":"2023-11-26T07:23:06.390032+0000","last_undegraded":"2023-11-26T07:23:06.390032+0000","last_fullsized":"2023-11-26T07:23:06.390032+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:15:59.939594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.150741+0000","last_change":"2023-11-26T07:23:05.138755+0000","last_active":"2023-11-26T07:23:05.150741+0000","last_peered":"2023-11-26T07:23:05.150741+0000","last_clean":"2023-11-26T07:23:05.150741+0000","last_became_active":"2023-11-26T07:23:05.138425+0000","last_became_peered":"2023-11-26T07:23:05.138425+0000","last_unstale":"2023-11-26T07:23:05.150741+0000","last_undegraded":"2023-11-26T07:23:05.150741+0000","last_fullsized":"2023-11-26T07:23:05.150741+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:12.122297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140881+0000","last_change":"2023-11-26T07:23:04.120474+0000","last_active":"2023-11-26T07:23:06.140881+0000","last_peered":"2023-11-26T07:23:06.140881+0000","last_clean":"2023-11-26T07:23:06.140881+0000","last_became_active":"2023-11-26T07:23:04.120171+0000","last_became_peered":"2023-11-26T07:23:04.120171+0000","last_unstale":"2023-11-26T07:23:06.140881+0000","last_undegraded":"2023-11-26T07:23:06.140881+0000","last_fullsized":"2023-11-26T07:23:06.140881+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:43:01.110241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"69'128","reported_seq":269,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.135166+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:23:05.135166+0000","last_peered":"2023-11-26T07:23:05.135166+0000","last_clean":"2023-11-26T07:23:05.135166+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:23:05.135166+0000","last_undegraded":"2023-11-26T07:23:05.135166+0000","last_fullsized":"2023-11-26T07:23:05.135166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1822720,"data_stored":1125115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93600000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72099999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69899999999999995,"15min":0.68300000000000005},"min":{"1min":0.503,"5min":0.437,"15min":0.437},"max":{"1min":0.90100000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.47399999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93000000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.47999999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86399999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70499999999999996,"15min":0.72299999999999998},"min":{"1min":0.53100000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85699999999999998,"5min":1.016,"15min":1.016},"last":0.875}]},{"osd":3,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0089999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81499999999999995},"min":{"1min":0.64100000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68200000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.80000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63100000000000001,"15min":0.68400000000000005},"min":{"1min":0.45100000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71999999999999997,"15min":0.77000000000000002},"min":{"1min":0.56000000000000005,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95099999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.76000000000000001,"15min":0.83999999999999997},"min":{"1min":0.60799999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96799999999999997,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.84399999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70199999999999996,"15min":0.75600000000000001},"min":{"1min":0.501,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77400000000000002,"15min":0.82599999999999996},"min":{"1min":0.57199999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.69899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1236992,"data_stored":540501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.84799999999999998,"15min":0.77400000000000002},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82499999999999996,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":1.226,"15min":0.90800000000000003},"min":{"1min":0.499,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80600000000000005,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.59899999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.71599999999999997,"15min":0.74399999999999999},"min":{"1min":0.46700000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81100000000000005,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.79300000000000004,"15min":0.81999999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.96599999999999997,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.77900000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.78000000000000003,"15min":0.76000000000000001},"min":{"1min":0.52900000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91800000000000004,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.026,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.75900000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.751},"min":{"1min":0.57899999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97099999999999997,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.91600000000000004,"15min":0.86399999999999999},"min":{"1min":0.46100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82899999999999996,"5min":18.77,"15min":18.77},"last":0.65700000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81899999999999995,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66500000000000004,"15min":0.71299999999999997},"min":{"1min":0.48399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.92500000000000004,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.63600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.71299999999999997},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.048,"5min":1.508,"15min":1.508},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74199999999999999,"15min":0.78300000000000003},"min":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0900000000000001,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.746,"15min":0.78500000000000003},"min":{"1min":0.53800000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.98799999999999999,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.93300000000000005,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.69699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":541035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.55800000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.65500000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.627}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.68600000000000005},"min":{"1min":0.58099999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67800000000000005,"15min":0.69199999999999995},"min":{"1min":0.45200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.78300000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.96299999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64700000000000002,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.105,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.77100000000000002},"min":{"1min":0.65500000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.016}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.625,"15min":0.67200000000000004},"min":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.871},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.94299999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.755,"15min":0.76000000000000001},"min":{"1min":0.39900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.754,"15min":0.80700000000000005},"min":{"1min":0.48499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.0329999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":534131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66100000000000003,"15min":0.67900000000000005},"min":{"1min":0.41599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90300000000000002,"5min":1.016,"15min":1.016},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65400000000000003,"15min":0.67100000000000004},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.69899999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67800000000000005,"15min":0.68000000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69499999999999995,"15min":0.70999999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.077,"15min":1.077},"last":0.76900000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.55100000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.236,"5min":1.302,"15min":1.302},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.129,"5min":1.181,"15min":1.181},"last":0.71999999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72199999999999998,"15min":0.747},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75,"15min":0.78400000000000003},"min":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.323,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.85499999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67800000000000005,"15min":0.69599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.67300000000000004,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70799999999999996,"15min":0.73099999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.75700000000000001,"15min":0.78900000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.74199999999999999,"15min":0.746},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":4.3609999999999998,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77000000000000002,"15min":0.81000000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.754}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1236992,"data_stored":535813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.627,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51400000000000001,"15min":0.48999999999999999},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.70199999999999996,"5min":1.294,"15min":1.294},"last":0.439}]},{"osd":1,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59999999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.376},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64600000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.52500000000000002},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67600000000000005,"5min":1.456,"15min":1.456},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.73699999999999999,"5min":1.139,"15min":1.139},"last":0.56299999999999994}]},{"osd":4,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.54500000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91500000000000004,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.75900000000000001,"15min":0.77000000000000002},"min":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99199999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.55700000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89300000000000002,"5min":1.395,"15min":1.395},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.504,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95999999999999996,"5min":1.333,"15min":1.333},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.78800000000000003,"15min":0.80500000000000005},"min":{"1min":0.59699999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.89000000000000001,"5min":1.347,"15min":1.347},"last":0.80400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.78400000000000003,"15min":0.79700000000000004},"min":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94599999999999995,"5min":1.397,"15min":1.397},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.79200000000000004,"15min":0.80900000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95999999999999996,"5min":1.363,"15min":1.363},"last":0.73799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":540431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51500000000000001,"15min":0.63300000000000001},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73699999999999999,"5min":3.931,"15min":3.931},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.58399999999999996,"15min":0.69499999999999995},"min":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88700000000000001,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.44800000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55800000000000005,"15min":0.67800000000000005},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.871,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.57299999999999995,"15min":0.67700000000000005},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.40500000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.80000000000000004,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.66300000000000003,"15min":0.873},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70199999999999996,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.55200000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.91800000000000004},"min":{"1min":0.58299999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0149999999999999,"5min":4.431,"15min":4.431},"last":0.749},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.82299999999999995,"15min":0.89900000000000002},"min":{"1min":0.58299999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.284,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.80700000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2589999999999999,"5min":2.036,"15min":2.036},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.85399999999999998,"15min":0.95399999999999996},"min":{"1min":0.628,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.173,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.85699999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.96899999999999997},"min":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1100000000000001,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.181,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.83599999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.89400000000000002,"15min":0.97999999999999998},"min":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2110000000000001,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.877},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.81000000000000005,"15min":0.84399999999999997},"min":{"1min":0.59499999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.323,"5min":1.609,"15min":1.609},"last":0.89700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":540555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.56499999999999995,"5min":1.034,"15min":1.034},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.41399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68999999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.501}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.43099999999999999,"15min":0.45100000000000001},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65200000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64400000000000002,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.376}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54900000000000004,"15min":0.57899999999999996},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68000000000000005,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44600000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.52100000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0860000000000001,"5min":1.143,"15min":1.143},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0449999999999999,"5min":1.216,"15min":1.216},"last":0.82799999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.171,"5min":1.262,"15min":1.262},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.746,"15min":0.748},"min":{"1min":0.57099999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.343,"5min":1.54,"15min":1.54},"last":0.81799999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76700000000000002,"15min":0.73899999999999999},"min":{"1min":0.59699999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.77900000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.254,"5min":1.333,"15min":1.333},"last":0.85699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.751,"15min":0.75900000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2090000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.751,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.86899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1843200,"data_stored":1131504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.43099999999999999,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.57899999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.47999999999999998,"15min":0.47599999999999998},"min":{"1min":0.35599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.63800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.622,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48799999999999999,"15min":0.47799999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73799999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.38400000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.56299999999999994,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0289999999999999,"5min":1.365,"15min":1.365},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.80900000000000005},"min":{"1min":0.51300000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0980000000000001,"5min":1.181,"15min":1.181},"last":0.57599999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90300000000000002,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.755},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.76900000000000002},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.147,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.78500000000000003,"15min":0.80300000000000005},"min":{"1min":0.56100000000000005,"5min":0.503,"15min":0.503},"max":{"1min":0.91700000000000004,"5min":1.323,"15min":1.323},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.089,"5min":1.123,"15min":1.123},"last":0.622},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.82799999999999996,"15min":0.81799999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.111,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.65900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:23:09.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:23:09.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.421+0000 7f9b86194640 1 Processor -- start 2023-11-26T07:23:09.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.421+0000 7f9b86194640 1 -- start start 2023-11-26T07:23:09.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.421+0000 7f9b86194640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9b800715b0 0x7f9b80071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.421+0000 7f9b86194640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b80071f50 con 0x7f9b80159080 2023-11-26T07:23:09.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.421+0000 7f9b86194640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b800af1a0 con 0x7f9b8015b870 2023-11-26T07:23:09.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.421+0000 7f9b86194640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8015c450 con 0x7f9b800715b0 2023-11-26T07:23:09.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.421+0000 7f9b7effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9b800715b0 0x7f9b80071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.421+0000 7f9b7effd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9b800715b0 0x7f9b80071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36292/0 (socket says 172.21.15.31:36292) 2023-11-26T07:23:09.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.421+0000 7f9b7effd640 1 -- 172.21.15.31:0/2732068794 learned_addr learned my addr 172.21.15.31:0/2732068794 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:09.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.421+0000 7f9b7f7fe640 1 --1- 172.21.15.31:0/2732068794 >> v1:172.21.15.31:6789/0 conn(0x7f9b80159080 0x7f9b8006d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60580/0 (socket says 172.21.15.31:60580) 2023-11-26T07:23:09.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b7effd640 1 -- 172.21.15.31:0/2732068794 >> v1:172.21.15.97:6789/0 conn(0x7f9b8015b870 legacy=0x7f9b8015bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:09.426 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b7effd640 1 -- 172.21.15.31:0/2732068794 >> v1:172.21.15.31:6789/0 conn(0x7f9b80159080 legacy=0x7f9b8006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b7effd640 1 -- 172.21.15.31:0/2732068794 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8015c730 con 0x7f9b800715b0 2023-11-26T07:23:09.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b7effd640 1 --2- 172.21.15.31:0/2732068794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b800715b0 0x7f9b80071a10 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009870 tx=0x7f9b7401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a086d3b865530d09 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b7e7fc640 1 -- 172.21.15.31:0/2732068794 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740aa430 con 0x7f9b800715b0 2023-11-26T07:23:09.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b7e7fc640 1 -- 172.21.15.31:0/2732068794 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b740aa590 con 0x7f9b800715b0 2023-11-26T07:23:09.429 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b7e7fc640 1 -- 172.21.15.31:0/2732068794 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740aa920 con 0x7f9b800715b0 2023-11-26T07:23:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b86194640 1 -- 172.21.15.31:0/2732068794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b800715b0 msgr2=0x7f9b80071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b86194640 1 --2- 172.21.15.31:0/2732068794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b800715b0 0x7f9b80071a10 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009870 tx=0x7f9b7401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b86194640 1 -- 172.21.15.31:0/2732068794 shutdown_connections 2023-11-26T07:23:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b86194640 1 --2- 172.21.15.31:0/2732068794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b800715b0 0x7f9b80071a10 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b86194640 1 -- 172.21.15.31:0/2732068794 >> 172.21.15.31:0/2732068794 conn(0x7f9b800b1e40 msgr2=0x7f9b80070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b86194640 1 -- 172.21.15.31:0/2732068794 shutdown_connections 2023-11-26T07:23:09.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.425+0000 7f9b86194640 1 -- 172.21.15.31:0/2732068794 wait complete. 2023-11-26T07:23:09.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b86194640 1 Processor -- start 2023-11-26T07:23:09.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b86194640 1 -- start start 2023-11-26T07:23:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b86194640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800715b0 0x7f9b80152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b86194640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b80159080 0x7f9b8014e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b86194640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b8015b870 0x7f9b800b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b86194640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80152380 con 0x7f9b8015b870 2023-11-26T07:23:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b86194640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8014fc90 con 0x7f9b800715b0 2023-11-26T07:23:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b86194640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8014ff70 con 0x7f9b80159080 2023-11-26T07:23:09.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b8015b870 0x7f9b800b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b8015b870 0x7f9b800b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36296/0 (socket says 172.21.15.31:36296) 2023-11-26T07:23:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7ffff640 1 -- 172.21.15.31:0/2297820780 learned_addr learned my addr 172.21.15.31:0/2297820780 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7f7fe640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800715b0 0x7f9b80152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7effd640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b80159080 0x7f9b8014e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7ffff640 1 -- 172.21.15.31:0/2297820780 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b80159080 msgr2=0x7f9b8014e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7ffff640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b80159080 0x7f9b8014e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7ffff640 1 -- 172.21.15.31:0/2297820780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800715b0 msgr2=0x7f9b80152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7ffff640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800715b0 0x7f9b80152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7ffff640 1 -- 172.21.15.31:0/2297820780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b740094d0 con 0x7f9b8015b870 2023-11-26T07:23:09.434 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b7ffff640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b8015b870 0x7f9b800b4340 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800b970 tx=0x7f9b7800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b67fff640 1 -- 172.21.15.31:0/2297820780 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800cb40 con 0x7f9b8015b870 2023-11-26T07:23:09.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b67fff640 1 -- 172.21.15.31:0/2297820780 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7800cca0 con 0x7f9b8015b870 2023-11-26T07:23:09.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b67fff640 1 -- 172.21.15.31:0/2297820780 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78014680 con 0x7f9b8015b870 2023-11-26T07:23:09.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.429+0000 7f9b86194640 1 -- 172.21.15.31:0/2297820780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b80152620 con 0x7f9b8015b870 2023-11-26T07:23:09.435 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.433+0000 7f9b86194640 1 -- 172.21.15.31:0/2297820780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8014ecd0 con 0x7f9b8015b870 2023-11-26T07:23:09.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.433+0000 7f9b86194640 1 -- 172.21.15.31:0/2297820780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b80152860 con 0x7f9b8015b870 2023-11-26T07:23:09.436 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.433+0000 7f9b67fff640 1 -- 172.21.15.31:0/2297820780 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f9b7801d3b0 con 0x7f9b8015b870 2023-11-26T07:23:09.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.433+0000 7f9b67fff640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b6c066590 0x7f9b6c068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.433+0000 7f9b67fff640 1 -- 172.21.15.31:0/2297820780 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b78083f10 con 0x7f9b8015b870 2023-11-26T07:23:09.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.433+0000 7f9b7f7fe640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b6c066590 0x7f9b6c068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.437 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.433+0000 7f9b7f7fe640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b6c066590 0x7f9b6c068a50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9b70000f70 tx=0x7f9b70007cb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.437+0000 7f9b67fff640 1 -- 172.21.15.31:0/2297820780 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9b80152860 con 0x7f9b8015b870 2023-11-26T07:23:09.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.593+0000 7f9b86194640 1 -- 172.21.15.31:0/2297820780 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b800cf540 con 0x7f9b6c066590 2023-11-26T07:23:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.601+0000 7f9b67fff640 1 -- 172.21.15.31:0/2297820780 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288884 (secure 0 0 0) 0x7f9b800cf540 con 0x7f9b6c066590 2023-11-26T07:23:09.607 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:09.608 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:23:09.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 -- 172.21.15.31:0/2297820780 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b6c066590 msgr2=0x7f9b6c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b6c066590 0x7f9b6c068a50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9b70000f70 tx=0x7f9b70007cb0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 -- 172.21.15.31:0/2297820780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b8015b870 msgr2=0x7f9b800b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b8015b870 0x7f9b800b4340 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800b970 tx=0x7f9b7800be30 comp rx=0 tx=0).stop 2023-11-26T07:23:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 -- 172.21.15.31:0/2297820780 shutdown_connections 2023-11-26T07:23:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b6c066590 0x7f9b6c068a50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b8015b870 0x7f9b800b4340 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b80159080 0x7f9b8014e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 --2- 172.21.15.31:0/2297820780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b800715b0 0x7f9b80152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 -- 172.21.15.31:0/2297820780 >> 172.21.15.31:0/2297820780 conn(0x7f9b800b1e40 msgr2=0x7f9b800af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:09.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 -- 172.21.15.31:0/2297820780 shutdown_connections 2023-11-26T07:23:09.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.605+0000 7f9b86194640 1 -- 172.21.15.31:0/2297820780 wait complete. 2023-11-26T07:23:09.627 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-11-26T07:23:08.209328+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260292,"kb_used_data":10816,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714428,"statfs":{"total":773094113280,"available":772827574272,"internally_reserved":0,"allocated":11075584,"data_stored":5489085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.119955"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390193+0000","last_change":"2023-11-26T07:23:05.132846+0000","last_active":"2023-11-26T07:23:06.390193+0000","last_peered":"2023-11-26T07:23:06.390193+0000","last_clean":"2023-11-26T07:23:06.390193+0000","last_became_active":"2023-11-26T07:23:05.132557+0000","last_became_peered":"2023-11-26T07:23:05.132557+0000","last_unstale":"2023-11-26T07:23:06.390193+0000","last_undegraded":"2023-11-26T07:23:06.390193+0000","last_fullsized":"2023-11-26T07:23:06.390193+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:14:54.477719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.147578+0000","last_change":"2023-11-26T07:23:05.144465+0000","last_active":"2023-11-26T07:23:05.147578+0000","last_peered":"2023-11-26T07:23:05.147578+0000","last_clean":"2023-11-26T07:23:05.147578+0000","last_became_active":"2023-11-26T07:23:05.144176+0000","last_became_peered":"2023-11-26T07:23:05.144176+0000","last_unstale":"2023-11-26T07:23:05.147578+0000","last_undegraded":"2023-11-26T07:23:05.147578+0000","last_fullsized":"2023-11-26T07:23:05.147578+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:08:34.451843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410843+0000","last_change":"2023-11-26T07:23:05.144202+0000","last_active":"2023-11-26T07:23:06.410843+0000","last_peered":"2023-11-26T07:23:06.410843+0000","last_clean":"2023-11-26T07:23:06.410843+0000","last_became_active":"2023-11-26T07:23:05.143901+0000","last_became_peered":"2023-11-26T07:23:05.143901+0000","last_unstale":"2023-11-26T07:23:06.410843+0000","last_undegraded":"2023-11-26T07:23:06.410843+0000","last_fullsized":"2023-11-26T07:23:06.410843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:37:55.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390053+0000","last_change":"2023-11-26T07:23:05.127838+0000","last_active":"2023-11-26T07:23:06.390053+0000","last_peered":"2023-11-26T07:23:06.390053+0000","last_clean":"2023-11-26T07:23:06.390053+0000","last_became_active":"2023-11-26T07:23:05.127553+0000","last_became_peered":"2023-11-26T07:23:05.127553+0000","last_unstale":"2023-11-26T07:23:06.390053+0000","last_undegraded":"2023-11-26T07:23:06.390053+0000","last_fullsized":"2023-11-26T07:23:06.390053+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:08:33.155558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465380+0000","last_change":"2023-11-26T07:23:05.139886+0000","last_active":"2023-11-26T07:23:06.465380+0000","last_peered":"2023-11-26T07:23:06.465380+0000","last_clean":"2023-11-26T07:23:06.465380+0000","last_became_active":"2023-11-26T07:23:05.139541+0000","last_became_peered":"2023-11-26T07:23:05.139541+0000","last_unstale":"2023-11-26T07:23:06.465380+0000","last_undegraded":"2023-11-26T07:23:06.465380+0000","last_fullsized":"2023-11-26T07:23:06.465380+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:09:46.760185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139696+0000","last_change":"2023-11-26T07:23:05.144249+0000","last_active":"2023-11-26T07:23:06.139696+0000","last_peered":"2023-11-26T07:23:06.139696+0000","last_clean":"2023-11-26T07:23:06.139696+0000","last_became_active":"2023-11-26T07:23:05.143927+0000","last_became_peered":"2023-11-26T07:23:05.143927+0000","last_unstale":"2023-11-26T07:23:06.139696+0000","last_undegraded":"2023-11-26T07:23:06.139696+0000","last_fullsized":"2023-11-26T07:23:06.139696+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:47:07.166224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140825+0000","last_change":"2023-11-26T07:23:05.134065+0000","last_active":"2023-11-26T07:23:06.140825+0000","last_peered":"2023-11-26T07:23:06.140825+0000","last_clean":"2023-11-26T07:23:06.140825+0000","last_became_active":"2023-11-26T07:23:05.131455+0000","last_became_peered":"2023-11-26T07:23:05.131455+0000","last_unstale":"2023-11-26T07:23:06.140825+0000","last_undegraded":"2023-11-26T07:23:06.140825+0000","last_fullsized":"2023-11-26T07:23:06.140825+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:17:38.640369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410628+0000","last_change":"2023-11-26T07:23:05.161423+0000","last_active":"2023-11-26T07:23:06.410628+0000","last_peered":"2023-11-26T07:23:06.410628+0000","last_clean":"2023-11-26T07:23:06.410628+0000","last_became_active":"2023-11-26T07:23:05.161120+0000","last_became_peered":"2023-11-26T07:23:05.161120+0000","last_unstale":"2023-11-26T07:23:06.410628+0000","last_undegraded":"2023-11-26T07:23:06.410628+0000","last_fullsized":"2023-11-26T07:23:06.410628+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:12.122297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390186+0000","last_change":"2023-11-26T07:23:05.141806+0000","last_active":"2023-11-26T07:23:06.390186+0000","last_peered":"2023-11-26T07:23:06.390186+0000","last_clean":"2023-11-26T07:23:06.390186+0000","last_became_active":"2023-11-26T07:23:05.141509+0000","last_became_peered":"2023-11-26T07:23:05.141509+0000","last_unstale":"2023-11-26T07:23:06.390186+0000","last_undegraded":"2023-11-26T07:23:06.390186+0000","last_fullsized":"2023-11-26T07:23:06.390186+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:21:18.446192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389964+0000","last_change":"2023-11-26T07:23:05.141866+0000","last_active":"2023-11-26T07:23:06.389964+0000","last_peered":"2023-11-26T07:23:06.389964+0000","last_clean":"2023-11-26T07:23:06.389964+0000","last_became_active":"2023-11-26T07:23:05.141576+0000","last_became_peered":"2023-11-26T07:23:05.141576+0000","last_unstale":"2023-11-26T07:23:06.389964+0000","last_undegraded":"2023-11-26T07:23:06.389964+0000","last_fullsized":"2023-11-26T07:23:06.389964+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:53:41.971600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.133172+0000","last_change":"2023-11-26T07:23:05.127844+0000","last_active":"2023-11-26T07:23:05.133172+0000","last_peered":"2023-11-26T07:23:05.133172+0000","last_clean":"2023-11-26T07:23:05.133172+0000","last_became_active":"2023-11-26T07:23:05.127456+0000","last_became_peered":"2023-11-26T07:23:05.127456+0000","last_unstale":"2023-11-26T07:23:05.133172+0000","last_undegraded":"2023-11-26T07:23:05.133172+0000","last_fullsized":"2023-11-26T07:23:05.133172+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:06:14.987344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389982+0000","last_change":"2023-11-26T07:23:05.147915+0000","last_active":"2023-11-26T07:23:06.389982+0000","last_peered":"2023-11-26T07:23:06.389982+0000","last_clean":"2023-11-26T07:23:06.389982+0000","last_became_active":"2023-11-26T07:23:05.147628+0000","last_became_peered":"2023-11-26T07:23:05.147628+0000","last_unstale":"2023-11-26T07:23:06.389982+0000","last_undegraded":"2023-11-26T07:23:06.389982+0000","last_fullsized":"2023-11-26T07:23:06.389982+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:58:27.174766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410794+0000","last_change":"2023-11-26T07:23:05.148078+0000","last_active":"2023-11-26T07:23:06.410794+0000","last_peered":"2023-11-26T07:23:06.410794+0000","last_clean":"2023-11-26T07:23:06.410794+0000","last_became_active":"2023-11-26T07:23:05.147766+0000","last_became_peered":"2023-11-26T07:23:05.147766+0000","last_unstale":"2023-11-26T07:23:06.410794+0000","last_undegraded":"2023-11-26T07:23:06.410794+0000","last_fullsized":"2023-11-26T07:23:06.410794+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:41.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410510+0000","last_change":"2023-11-26T07:23:05.146986+0000","last_active":"2023-11-26T07:23:06.410510+0000","last_peered":"2023-11-26T07:23:06.410510+0000","last_clean":"2023-11-26T07:23:06.410510+0000","last_became_active":"2023-11-26T07:23:05.146710+0000","last_became_peered":"2023-11-26T07:23:05.146710+0000","last_unstale":"2023-11-26T07:23:06.410510+0000","last_undegraded":"2023-11-26T07:23:06.410510+0000","last_fullsized":"2023-11-26T07:23:06.410510+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:52:57.658909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139634+0000","last_change":"2023-11-26T07:23:05.145600+0000","last_active":"2023-11-26T07:23:06.139634+0000","last_peered":"2023-11-26T07:23:06.139634+0000","last_clean":"2023-11-26T07:23:06.139634+0000","last_became_active":"2023-11-26T07:23:05.145305+0000","last_became_peered":"2023-11-26T07:23:05.145305+0000","last_unstale":"2023-11-26T07:23:06.139634+0000","last_undegraded":"2023-11-26T07:23:06.139634+0000","last_fullsized":"2023-11-26T07:23:06.139634+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:50:15.966939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389603+0000","last_change":"2023-11-26T07:23:05.146778+0000","last_active":"2023-11-26T07:23:06.389603+0000","last_peered":"2023-11-26T07:23:06.389603+0000","last_clean":"2023-11-26T07:23:06.389603+0000","last_became_active":"2023-11-26T07:23:05.146276+0000","last_became_peered":"2023-11-26T07:23:05.146276+0000","last_unstale":"2023-11-26T07:23:06.389603+0000","last_undegraded":"2023-11-26T07:23:06.389603+0000","last_fullsized":"2023-11-26T07:23:06.389603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:21:41.468377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.133251+0000","last_change":"2023-11-26T07:23:05.127687+0000","last_active":"2023-11-26T07:23:05.133251+0000","last_peered":"2023-11-26T07:23:05.133251+0000","last_clean":"2023-11-26T07:23:05.133251+0000","last_became_active":"2023-11-26T07:23:05.127364+0000","last_became_peered":"2023-11-26T07:23:05.127364+0000","last_unstale":"2023-11-26T07:23:05.133251+0000","last_undegraded":"2023-11-26T07:23:05.133251+0000","last_fullsized":"2023-11-26T07:23:05.133251+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:46.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.141956+0000","last_change":"2023-11-26T07:23:05.127616+0000","last_active":"2023-11-26T07:23:06.141956+0000","last_peered":"2023-11-26T07:23:06.141956+0000","last_clean":"2023-11-26T07:23:06.141956+0000","last_became_active":"2023-11-26T07:23:05.126976+0000","last_became_peered":"2023-11-26T07:23:05.126976+0000","last_unstale":"2023-11-26T07:23:06.141956+0000","last_undegraded":"2023-11-26T07:23:06.141956+0000","last_fullsized":"2023-11-26T07:23:06.141956+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:25.410801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410937+0000","last_change":"2023-11-26T07:23:05.148473+0000","last_active":"2023-11-26T07:23:06.410937+0000","last_peered":"2023-11-26T07:23:06.410937+0000","last_clean":"2023-11-26T07:23:06.410937+0000","last_became_active":"2023-11-26T07:23:05.147711+0000","last_became_peered":"2023-11-26T07:23:05.147711+0000","last_unstale":"2023-11-26T07:23:06.410937+0000","last_undegraded":"2023-11-26T07:23:06.410937+0000","last_fullsized":"2023-11-26T07:23:06.410937+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:02.103968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.144941+0000","last_change":"2023-11-26T07:23:05.141222+0000","last_active":"2023-11-26T07:23:05.144941+0000","last_peered":"2023-11-26T07:23:05.144941+0000","last_clean":"2023-11-26T07:23:05.144941+0000","last_became_active":"2023-11-26T07:23:05.140938+0000","last_became_peered":"2023-11-26T07:23:05.140938+0000","last_unstale":"2023-11-26T07:23:05.144941+0000","last_undegraded":"2023-11-26T07:23:05.144941+0000","last_fullsized":"2023-11-26T07:23:05.144941+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:28:44.611417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.133648+0000","last_change":"2023-11-26T07:23:05.130893+0000","last_active":"2023-11-26T07:23:05.133648+0000","last_peered":"2023-11-26T07:23:05.133648+0000","last_clean":"2023-11-26T07:23:05.133648+0000","last_became_active":"2023-11-26T07:23:05.130586+0000","last_became_peered":"2023-11-26T07:23:05.130586+0000","last_unstale":"2023-11-26T07:23:05.133648+0000","last_undegraded":"2023-11-26T07:23:05.133648+0000","last_fullsized":"2023-11-26T07:23:05.133648+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:41.930667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139521+0000","last_change":"2023-11-26T07:23:05.129229+0000","last_active":"2023-11-26T07:23:05.139521+0000","last_peered":"2023-11-26T07:23:05.139521+0000","last_clean":"2023-11-26T07:23:05.139521+0000","last_became_active":"2023-11-26T07:23:05.128396+0000","last_became_peered":"2023-11-26T07:23:05.128396+0000","last_unstale":"2023-11-26T07:23:05.139521+0000","last_undegraded":"2023-11-26T07:23:05.139521+0000","last_fullsized":"2023-11-26T07:23:05.139521+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:48:06.630041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.147330+0000","last_change":"2023-11-26T07:23:05.142826+0000","last_active":"2023-11-26T07:23:05.147330+0000","last_peered":"2023-11-26T07:23:05.147330+0000","last_clean":"2023-11-26T07:23:05.147330+0000","last_became_active":"2023-11-26T07:23:05.142542+0000","last_became_peered":"2023-11-26T07:23:05.142542+0000","last_unstale":"2023-11-26T07:23:05.147330+0000","last_undegraded":"2023-11-26T07:23:05.147330+0000","last_fullsized":"2023-11-26T07:23:05.147330+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:39:32.591740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.151384+0000","last_change":"2023-11-26T07:23:05.150532+0000","last_active":"2023-11-26T07:23:05.151384+0000","last_peered":"2023-11-26T07:23:05.151384+0000","last_clean":"2023-11-26T07:23:05.151384+0000","last_became_active":"2023-11-26T07:23:05.142740+0000","last_became_peered":"2023-11-26T07:23:05.142740+0000","last_unstale":"2023-11-26T07:23:05.151384+0000","last_undegraded":"2023-11-26T07:23:05.151384+0000","last_fullsized":"2023-11-26T07:23:05.151384+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:37:55.272579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.133738+0000","last_change":"2023-11-26T07:23:05.130516+0000","last_active":"2023-11-26T07:23:05.133738+0000","last_peered":"2023-11-26T07:23:05.133738+0000","last_clean":"2023-11-26T07:23:05.133738+0000","last_became_active":"2023-11-26T07:23:05.130169+0000","last_became_peered":"2023-11-26T07:23:05.130169+0000","last_unstale":"2023-11-26T07:23:05.133738+0000","last_undegraded":"2023-11-26T07:23:05.133738+0000","last_fullsized":"2023-11-26T07:23:05.133738+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:13:08.746625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390026+0000","last_change":"2023-11-26T07:23:05.135960+0000","last_active":"2023-11-26T07:23:06.390026+0000","last_peered":"2023-11-26T07:23:06.390026+0000","last_clean":"2023-11-26T07:23:06.390026+0000","last_became_active":"2023-11-26T07:23:05.135635+0000","last_became_peered":"2023-11-26T07:23:05.135635+0000","last_unstale":"2023-11-26T07:23:06.390026+0000","last_undegraded":"2023-11-26T07:23:06.390026+0000","last_fullsized":"2023-11-26T07:23:06.390026+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:20.430885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140134+0000","last_change":"2023-11-26T07:23:05.147446+0000","last_active":"2023-11-26T07:23:06.140134+0000","last_peered":"2023-11-26T07:23:06.140134+0000","last_clean":"2023-11-26T07:23:06.140134+0000","last_became_active":"2023-11-26T07:23:05.147185+0000","last_became_peered":"2023-11-26T07:23:05.147185+0000","last_unstale":"2023-11-26T07:23:06.140134+0000","last_undegraded":"2023-11-26T07:23:06.140134+0000","last_fullsized":"2023-11-26T07:23:06.140134+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:28:09.939466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139656+0000","last_change":"2023-11-26T07:23:05.129784+0000","last_active":"2023-11-26T07:23:05.139656+0000","last_peered":"2023-11-26T07:23:05.139656+0000","last_clean":"2023-11-26T07:23:05.139656+0000","last_became_active":"2023-11-26T07:23:05.129506+0000","last_became_peered":"2023-11-26T07:23:05.129506+0000","last_unstale":"2023-11-26T07:23:05.139656+0000","last_undegraded":"2023-11-26T07:23:05.139656+0000","last_fullsized":"2023-11-26T07:23:05.139656+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:53:28.488385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.147377+0000","last_change":"2023-11-26T07:23:05.144227+0000","last_active":"2023-11-26T07:23:05.147377+0000","last_peered":"2023-11-26T07:23:05.147377+0000","last_clean":"2023-11-26T07:23:05.147377+0000","last_became_active":"2023-11-26T07:23:05.143723+0000","last_became_peered":"2023-11-26T07:23:05.143723+0000","last_unstale":"2023-11-26T07:23:05.147377+0000","last_undegraded":"2023-11-26T07:23:05.147377+0000","last_fullsized":"2023-11-26T07:23:05.147377+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:50.220699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140506+0000","last_change":"2023-11-26T07:23:05.135631+0000","last_active":"2023-11-26T07:23:06.140506+0000","last_peered":"2023-11-26T07:23:06.140506+0000","last_clean":"2023-11-26T07:23:06.140506+0000","last_became_active":"2023-11-26T07:23:05.134335+0000","last_became_peered":"2023-11-26T07:23:05.134335+0000","last_unstale":"2023-11-26T07:23:06.140506+0000","last_undegraded":"2023-11-26T07:23:06.140506+0000","last_fullsized":"2023-11-26T07:23:06.140506+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:56.708048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389673+0000","last_change":"2023-11-26T07:23:05.136000+0000","last_active":"2023-11-26T07:23:06.389673+0000","last_peered":"2023-11-26T07:23:06.389673+0000","last_clean":"2023-11-26T07:23:06.389673+0000","last_became_active":"2023-11-26T07:23:05.135733+0000","last_became_peered":"2023-11-26T07:23:05.135733+0000","last_unstale":"2023-11-26T07:23:06.389673+0000","last_undegraded":"2023-11-26T07:23:06.389673+0000","last_fullsized":"2023-11-26T07:23:06.389673+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:18:27.600435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139634+0000","last_change":"2023-11-26T07:23:05.147344+0000","last_active":"2023-11-26T07:23:06.139634+0000","last_peered":"2023-11-26T07:23:06.139634+0000","last_clean":"2023-11-26T07:23:06.139634+0000","last_became_active":"2023-11-26T07:23:05.146985+0000","last_became_peered":"2023-11-26T07:23:05.146985+0000","last_unstale":"2023-11-26T07:23:06.139634+0000","last_undegraded":"2023-11-26T07:23:06.139634+0000","last_fullsized":"2023-11-26T07:23:06.139634+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:23:56.300262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140165+0000","last_change":"2023-11-26T07:23:05.153870+0000","last_active":"2023-11-26T07:23:06.140165+0000","last_peered":"2023-11-26T07:23:06.140165+0000","last_clean":"2023-11-26T07:23:06.140165+0000","last_became_active":"2023-11-26T07:23:05.153564+0000","last_became_peered":"2023-11-26T07:23:05.153564+0000","last_unstale":"2023-11-26T07:23:06.140165+0000","last_undegraded":"2023-11-26T07:23:06.140165+0000","last_fullsized":"2023-11-26T07:23:06.140165+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:26:16.296434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"98'4","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140756+0000","last_change":"2023-11-26T07:23:04.116235+0000","last_active":"2023-11-26T07:23:06.140756+0000","last_peered":"2023-11-26T07:23:06.140756+0000","last_clean":"2023-11-26T07:23:06.140756+0000","last_became_active":"2023-11-26T07:23:04.115926+0000","last_became_peered":"2023-11-26T07:23:04.115926+0000","last_unstale":"2023-11-26T07:23:06.140756+0000","last_undegraded":"2023-11-26T07:23:06.140756+0000","last_fullsized":"2023-11-26T07:23:06.140756+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:15:40.924680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.141983+0000","last_change":"2023-11-26T07:23:05.140365+0000","last_active":"2023-11-26T07:23:06.141983+0000","last_peered":"2023-11-26T07:23:06.141983+0000","last_clean":"2023-11-26T07:23:06.141983+0000","last_became_active":"2023-11-26T07:23:05.139709+0000","last_became_peered":"2023-11-26T07:23:05.139709+0000","last_unstale":"2023-11-26T07:23:06.141983+0000","last_undegraded":"2023-11-26T07:23:06.141983+0000","last_fullsized":"2023-11-26T07:23:06.141983+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:34:54.315569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"98'1","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410888+0000","last_change":"2023-11-26T07:23:04.117016+0000","last_active":"2023-11-26T07:23:06.410888+0000","last_peered":"2023-11-26T07:23:06.410888+0000","last_clean":"2023-11-26T07:23:06.410888+0000","last_became_active":"2023-11-26T07:23:04.116696+0000","last_became_peered":"2023-11-26T07:23:04.116696+0000","last_unstale":"2023-11-26T07:23:06.410888+0000","last_undegraded":"2023-11-26T07:23:06.410888+0000","last_fullsized":"2023-11-26T07:23:06.410888+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:12:26.540817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140009+0000","last_change":"2023-11-26T07:23:05.144556+0000","last_active":"2023-11-26T07:23:06.140009+0000","last_peered":"2023-11-26T07:23:06.140009+0000","last_clean":"2023-11-26T07:23:06.140009+0000","last_became_active":"2023-11-26T07:23:05.144288+0000","last_became_peered":"2023-11-26T07:23:05.144288+0000","last_unstale":"2023-11-26T07:23:06.140009+0000","last_undegraded":"2023-11-26T07:23:06.140009+0000","last_fullsized":"2023-11-26T07:23:06.140009+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:03:13.505342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"98'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465453+0000","last_change":"2023-11-26T07:23:04.121844+0000","last_active":"2023-11-26T07:23:06.465453+0000","last_peered":"2023-11-26T07:23:06.465453+0000","last_clean":"2023-11-26T07:23:06.465453+0000","last_became_active":"2023-11-26T07:23:04.120615+0000","last_became_peered":"2023-11-26T07:23:04.120615+0000","last_unstale":"2023-11-26T07:23:06.465453+0000","last_undegraded":"2023-11-26T07:23:06.465453+0000","last_fullsized":"2023-11-26T07:23:06.465453+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:09.053722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410985+0000","last_change":"2023-11-26T07:23:05.151228+0000","last_active":"2023-11-26T07:23:06.410985+0000","last_peered":"2023-11-26T07:23:06.410985+0000","last_clean":"2023-11-26T07:23:06.410985+0000","last_became_active":"2023-11-26T07:23:05.150837+0000","last_became_peered":"2023-11-26T07:23:05.150837+0000","last_unstale":"2023-11-26T07:23:06.410985+0000","last_undegraded":"2023-11-26T07:23:06.410985+0000","last_fullsized":"2023-11-26T07:23:06.410985+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:46.037771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"98'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465292+0000","last_change":"2023-11-26T07:23:04.122113+0000","last_active":"2023-11-26T07:23:06.465292+0000","last_peered":"2023-11-26T07:23:06.465292+0000","last_clean":"2023-11-26T07:23:06.465292+0000","last_became_active":"2023-11-26T07:23:04.119442+0000","last_became_peered":"2023-11-26T07:23:04.119442+0000","last_unstale":"2023-11-26T07:23:06.465292+0000","last_undegraded":"2023-11-26T07:23:06.465292+0000","last_fullsized":"2023-11-26T07:23:06.465292+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:10:14.681311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389646+0000","last_change":"2023-11-26T07:23:05.129737+0000","last_active":"2023-11-26T07:23:06.389646+0000","last_peered":"2023-11-26T07:23:06.389646+0000","last_clean":"2023-11-26T07:23:06.389646+0000","last_became_active":"2023-11-26T07:23:05.129423+0000","last_became_peered":"2023-11-26T07:23:05.129423+0000","last_unstale":"2023-11-26T07:23:06.389646+0000","last_undegraded":"2023-11-26T07:23:06.389646+0000","last_fullsized":"2023-11-26T07:23:06.389646+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:03:31.063138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"98'1","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140448+0000","last_change":"2023-11-26T07:23:04.119430+0000","last_active":"2023-11-26T07:23:06.140448+0000","last_peered":"2023-11-26T07:23:06.140448+0000","last_clean":"2023-11-26T07:23:06.140448+0000","last_became_active":"2023-11-26T07:23:04.118426+0000","last_became_peered":"2023-11-26T07:23:04.118426+0000","last_unstale":"2023-11-26T07:23:06.140448+0000","last_undegraded":"2023-11-26T07:23:06.140448+0000","last_fullsized":"2023-11-26T07:23:06.140448+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:12:26.540817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139139+0000","last_change":"2023-11-26T07:23:05.137129+0000","last_active":"2023-11-26T07:23:05.139139+0000","last_peered":"2023-11-26T07:23:05.139139+0000","last_clean":"2023-11-26T07:23:05.139139+0000","last_became_active":"2023-11-26T07:23:05.136818+0000","last_became_peered":"2023-11-26T07:23:05.136818+0000","last_unstale":"2023-11-26T07:23:05.139139+0000","last_undegraded":"2023-11-26T07:23:05.139139+0000","last_fullsized":"2023-11-26T07:23:05.139139+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:32:49.761866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"98'1","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.161276+0000","last_change":"2023-11-26T07:23:04.117349+0000","last_active":"2023-11-26T07:23:05.161276+0000","last_peered":"2023-11-26T07:23:05.161276+0000","last_clean":"2023-11-26T07:23:05.161276+0000","last_became_active":"2023-11-26T07:23:04.117029+0000","last_became_peered":"2023-11-26T07:23:04.117029+0000","last_unstale":"2023-11-26T07:23:05.161276+0000","last_undegraded":"2023-11-26T07:23:05.161276+0000","last_fullsized":"2023-11-26T07:23:05.161276+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:45.031785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389915+0000","last_change":"2023-11-26T07:23:05.146550+0000","last_active":"2023-11-26T07:23:06.389915+0000","last_peered":"2023-11-26T07:23:06.389915+0000","last_clean":"2023-11-26T07:23:06.389915+0000","last_became_active":"2023-11-26T07:23:05.145890+0000","last_became_peered":"2023-11-26T07:23:05.145890+0000","last_unstale":"2023-11-26T07:23:06.389915+0000","last_undegraded":"2023-11-26T07:23:06.389915+0000","last_fullsized":"2023-11-26T07:23:06.389915+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:43.433697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.120007+0000","last_change":"2023-11-26T07:23:04.116997+0000","last_active":"2023-11-26T07:23:05.120007+0000","last_peered":"2023-11-26T07:23:05.120007+0000","last_clean":"2023-11-26T07:23:05.120007+0000","last_became_active":"2023-11-26T07:23:04.116698+0000","last_became_peered":"2023-11-26T07:23:04.116698+0000","last_unstale":"2023-11-26T07:23:05.120007+0000","last_undegraded":"2023-11-26T07:23:05.120007+0000","last_fullsized":"2023-11-26T07:23:05.120007+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:52:56.652922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140777+0000","last_change":"2023-11-26T07:23:05.133574+0000","last_active":"2023-11-26T07:23:06.140777+0000","last_peered":"2023-11-26T07:23:06.140777+0000","last_clean":"2023-11-26T07:23:06.140777+0000","last_became_active":"2023-11-26T07:23:05.133234+0000","last_became_peered":"2023-11-26T07:23:05.133234+0000","last_unstale":"2023-11-26T07:23:06.140777+0000","last_undegraded":"2023-11-26T07:23:06.140777+0000","last_fullsized":"2023-11-26T07:23:06.140777+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:13:08.746625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389933+0000","last_change":"2023-11-26T07:23:04.120957+0000","last_active":"2023-11-26T07:23:06.389933+0000","last_peered":"2023-11-26T07:23:06.389933+0000","last_clean":"2023-11-26T07:23:06.389933+0000","last_became_active":"2023-11-26T07:23:04.117583+0000","last_became_peered":"2023-11-26T07:23:04.117583+0000","last_unstale":"2023-11-26T07:23:06.389933+0000","last_undegraded":"2023-11-26T07:23:06.389933+0000","last_fullsized":"2023-11-26T07:23:06.389933+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:54.448486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140873+0000","last_change":"2023-11-26T07:23:05.140844+0000","last_active":"2023-11-26T07:23:06.140873+0000","last_peered":"2023-11-26T07:23:06.140873+0000","last_clean":"2023-11-26T07:23:06.140873+0000","last_became_active":"2023-11-26T07:23:05.140417+0000","last_became_peered":"2023-11-26T07:23:05.140417+0000","last_unstale":"2023-11-26T07:23:06.140873+0000","last_undegraded":"2023-11-26T07:23:06.140873+0000","last_fullsized":"2023-11-26T07:23:06.140873+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:55:46.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465512+0000","last_change":"2023-11-26T07:23:04.123177+0000","last_active":"2023-11-26T07:23:06.465512+0000","last_peered":"2023-11-26T07:23:06.465512+0000","last_clean":"2023-11-26T07:23:06.465512+0000","last_became_active":"2023-11-26T07:23:04.122833+0000","last_became_peered":"2023-11-26T07:23:04.122833+0000","last_unstale":"2023-11-26T07:23:06.465512+0000","last_undegraded":"2023-11-26T07:23:06.465512+0000","last_fullsized":"2023-11-26T07:23:06.465512+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:33:32.019518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.142072+0000","last_change":"2023-11-26T07:23:05.136873+0000","last_active":"2023-11-26T07:23:06.142072+0000","last_peered":"2023-11-26T07:23:06.142072+0000","last_clean":"2023-11-26T07:23:06.142072+0000","last_became_active":"2023-11-26T07:23:05.136582+0000","last_became_peered":"2023-11-26T07:23:05.136582+0000","last_unstale":"2023-11-26T07:23:06.142072+0000","last_undegraded":"2023-11-26T07:23:06.142072+0000","last_fullsized":"2023-11-26T07:23:06.142072+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:55.454473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.117875+0000","last_change":"2023-11-26T07:23:04.117992+0000","last_active":"2023-11-26T07:23:05.117875+0000","last_peered":"2023-11-26T07:23:05.117875+0000","last_clean":"2023-11-26T07:23:05.117875+0000","last_became_active":"2023-11-26T07:23:04.117720+0000","last_became_peered":"2023-11-26T07:23:04.117720+0000","last_unstale":"2023-11-26T07:23:05.117875+0000","last_undegraded":"2023-11-26T07:23:05.117875+0000","last_fullsized":"2023-11-26T07:23:05.117875+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:24.404815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139636+0000","last_change":"2023-11-26T07:23:05.128563+0000","last_active":"2023-11-26T07:23:05.139636+0000","last_peered":"2023-11-26T07:23:05.139636+0000","last_clean":"2023-11-26T07:23:05.139636+0000","last_became_active":"2023-11-26T07:23:05.128266+0000","last_became_peered":"2023-11-26T07:23:05.128266+0000","last_unstale":"2023-11-26T07:23:05.139636+0000","last_undegraded":"2023-11-26T07:23:05.139636+0000","last_fullsized":"2023-11-26T07:23:05.139636+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:01:55.454473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"98'3","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389818+0000","last_change":"2023-11-26T07:23:04.120748+0000","last_active":"2023-11-26T07:23:06.389818+0000","last_peered":"2023-11-26T07:23:06.389818+0000","last_clean":"2023-11-26T07:23:06.389818+0000","last_became_active":"2023-11-26T07:23:04.120459+0000","last_became_peered":"2023-11-26T07:23:04.120459+0000","last_unstale":"2023-11-26T07:23:06.389818+0000","last_undegraded":"2023-11-26T07:23:06.389818+0000","last_fullsized":"2023-11-26T07:23:06.389818+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:02:06.833405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140631+0000","last_change":"2023-11-26T07:23:05.139974+0000","last_active":"2023-11-26T07:23:06.140631+0000","last_peered":"2023-11-26T07:23:06.140631+0000","last_clean":"2023-11-26T07:23:06.140631+0000","last_became_active":"2023-11-26T07:23:05.139686+0000","last_became_peered":"2023-11-26T07:23:05.139686+0000","last_unstale":"2023-11-26T07:23:06.140631+0000","last_undegraded":"2023-11-26T07:23:06.140631+0000","last_fullsized":"2023-11-26T07:23:06.140631+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:08:34.451843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"98'6","reported_seq":18,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390198+0000","last_change":"2023-11-26T07:23:04.125148+0000","last_active":"2023-11-26T07:23:06.390198+0000","last_peered":"2023-11-26T07:23:06.390198+0000","last_clean":"2023-11-26T07:23:06.390198+0000","last_became_active":"2023-11-26T07:23:04.124767+0000","last_became_peered":"2023-11-26T07:23:04.124767+0000","last_unstale":"2023-11-26T07:23:06.390198+0000","last_undegraded":"2023-11-26T07:23:06.390198+0000","last_fullsized":"2023-11-26T07:23:06.390198+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:06:32.768478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.144217+0000","last_change":"2023-11-26T07:23:05.140008+0000","last_active":"2023-11-26T07:23:05.144217+0000","last_peered":"2023-11-26T07:23:05.144217+0000","last_clean":"2023-11-26T07:23:05.144217+0000","last_became_active":"2023-11-26T07:23:05.139614+0000","last_became_peered":"2023-11-26T07:23:05.139614+0000","last_unstale":"2023-11-26T07:23:05.144217+0000","last_undegraded":"2023-11-26T07:23:05.144217+0000","last_fullsized":"2023-11-26T07:23:05.144217+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:03:41.085829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"98'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139758+0000","last_change":"2023-11-26T07:23:04.115520+0000","last_active":"2023-11-26T07:23:06.139758+0000","last_peered":"2023-11-26T07:23:06.139758+0000","last_clean":"2023-11-26T07:23:06.139758+0000","last_became_active":"2023-11-26T07:23:04.115161+0000","last_became_peered":"2023-11-26T07:23:04.115161+0000","last_unstale":"2023-11-26T07:23:06.139758+0000","last_undegraded":"2023-11-26T07:23:06.139758+0000","last_fullsized":"2023-11-26T07:23:06.139758+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:53:24.100793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.146701+0000","last_change":"2023-11-26T07:23:05.144906+0000","last_active":"2023-11-26T07:23:05.146701+0000","last_peered":"2023-11-26T07:23:05.146701+0000","last_clean":"2023-11-26T07:23:05.146701+0000","last_became_active":"2023-11-26T07:23:05.144644+0000","last_became_peered":"2023-11-26T07:23:05.144644+0000","last_unstale":"2023-11-26T07:23:05.146701+0000","last_undegraded":"2023-11-26T07:23:05.146701+0000","last_fullsized":"2023-11-26T07:23:05.146701+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:58:18.459826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.141696+0000","last_change":"2023-11-26T07:23:04.107900+0000","last_active":"2023-11-26T07:23:06.141696+0000","last_peered":"2023-11-26T07:23:06.141696+0000","last_clean":"2023-11-26T07:23:06.141696+0000","last_became_active":"2023-11-26T07:23:04.107344+0000","last_became_peered":"2023-11-26T07:23:04.107344+0000","last_unstale":"2023-11-26T07:23:06.141696+0000","last_undegraded":"2023-11-26T07:23:06.141696+0000","last_fullsized":"2023-11-26T07:23:06.141696+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:19:49.214712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139606+0000","last_change":"2023-11-26T07:23:05.128528+0000","last_active":"2023-11-26T07:23:05.139606+0000","last_peered":"2023-11-26T07:23:05.139606+0000","last_clean":"2023-11-26T07:23:05.139606+0000","last_became_active":"2023-11-26T07:23:05.128195+0000","last_became_peered":"2023-11-26T07:23:05.128195+0000","last_unstale":"2023-11-26T07:23:05.139606+0000","last_undegraded":"2023-11-26T07:23:05.139606+0000","last_fullsized":"2023-11-26T07:23:05.139606+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:54:34.159473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"98'8","reported_seq":20,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465342+0000","last_change":"2023-11-26T07:23:04.123499+0000","last_active":"2023-11-26T07:23:06.465342+0000","last_peered":"2023-11-26T07:23:06.465342+0000","last_clean":"2023-11-26T07:23:06.465342+0000","last_became_active":"2023-11-26T07:23:04.123193+0000","last_became_peered":"2023-11-26T07:23:04.123193+0000","last_unstale":"2023-11-26T07:23:06.465342+0000","last_undegraded":"2023-11-26T07:23:06.465342+0000","last_fullsized":"2023-11-26T07:23:06.465342+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:20:29.898632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465603+0000","last_change":"2023-11-26T07:23:05.142191+0000","last_active":"2023-11-26T07:23:06.465603+0000","last_peered":"2023-11-26T07:23:06.465603+0000","last_clean":"2023-11-26T07:23:06.465603+0000","last_became_active":"2023-11-26T07:23:05.141871+0000","last_became_peered":"2023-11-26T07:23:05.141871+0000","last_unstale":"2023-11-26T07:23:06.465603+0000","last_undegraded":"2023-11-26T07:23:06.465603+0000","last_fullsized":"2023-11-26T07:23:06.465603+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:10:43.619217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"98'1","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390288+0000","last_change":"2023-11-26T07:23:04.118804+0000","last_active":"2023-11-26T07:23:06.390288+0000","last_peered":"2023-11-26T07:23:06.390288+0000","last_clean":"2023-11-26T07:23:06.390288+0000","last_became_active":"2023-11-26T07:23:04.117569+0000","last_became_peered":"2023-11-26T07:23:04.117569+0000","last_unstale":"2023-11-26T07:23:06.390288+0000","last_undegraded":"2023-11-26T07:23:06.390288+0000","last_fullsized":"2023-11-26T07:23:06.390288+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:59:20.217770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140255+0000","last_change":"2023-11-26T07:23:05.150851+0000","last_active":"2023-11-26T07:23:06.140255+0000","last_peered":"2023-11-26T07:23:06.140255+0000","last_clean":"2023-11-26T07:23:06.140255+0000","last_became_active":"2023-11-26T07:23:05.149804+0000","last_became_peered":"2023-11-26T07:23:05.149804+0000","last_unstale":"2023-11-26T07:23:06.140255+0000","last_undegraded":"2023-11-26T07:23:06.140255+0000","last_fullsized":"2023-11-26T07:23:06.140255+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:38:18.950490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"98'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465574+0000","last_change":"2023-11-26T07:23:04.113521+0000","last_active":"2023-11-26T07:23:06.465574+0000","last_peered":"2023-11-26T07:23:06.465574+0000","last_clean":"2023-11-26T07:23:06.465574+0000","last_became_active":"2023-11-26T07:23:04.113200+0000","last_became_peered":"2023-11-26T07:23:04.113200+0000","last_unstale":"2023-11-26T07:23:06.465574+0000","last_undegraded":"2023-11-26T07:23:06.465574+0000","last_fullsized":"2023-11-26T07:23:06.465574+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:29:45.912508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410767+0000","last_change":"2023-11-26T07:23:05.144415+0000","last_active":"2023-11-26T07:23:06.410767+0000","last_peered":"2023-11-26T07:23:06.410767+0000","last_clean":"2023-11-26T07:23:06.410767+0000","last_became_active":"2023-11-26T07:23:05.143499+0000","last_became_peered":"2023-11-26T07:23:05.143499+0000","last_unstale":"2023-11-26T07:23:06.410767+0000","last_undegraded":"2023-11-26T07:23:06.410767+0000","last_fullsized":"2023-11-26T07:23:06.410767+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:48:06.630041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"98'1","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.141858+0000","last_change":"2023-11-26T07:23:04.118905+0000","last_active":"2023-11-26T07:23:06.141858+0000","last_peered":"2023-11-26T07:23:06.141858+0000","last_clean":"2023-11-26T07:23:06.141858+0000","last_became_active":"2023-11-26T07:23:04.118510+0000","last_became_peered":"2023-11-26T07:23:04.118510+0000","last_unstale":"2023-11-26T07:23:06.141858+0000","last_undegraded":"2023-11-26T07:23:06.141858+0000","last_fullsized":"2023-11-26T07:23:06.141858+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T09:37:54.266592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.411054+0000","last_change":"2023-11-26T07:23:05.142827+0000","last_active":"2023-11-26T07:23:06.411054+0000","last_peered":"2023-11-26T07:23:06.411054+0000","last_clean":"2023-11-26T07:23:06.411054+0000","last_became_active":"2023-11-26T07:23:05.142577+0000","last_became_peered":"2023-11-26T07:23:05.142577+0000","last_unstale":"2023-11-26T07:23:06.411054+0000","last_undegraded":"2023-11-26T07:23:06.411054+0000","last_fullsized":"2023-11-26T07:23:06.411054+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:34:54.315569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140446+0000","last_change":"2023-11-26T07:23:04.117690+0000","last_active":"2023-11-26T07:23:06.140446+0000","last_peered":"2023-11-26T07:23:06.140446+0000","last_clean":"2023-11-26T07:23:06.140446+0000","last_became_active":"2023-11-26T07:23:04.117410+0000","last_became_peered":"2023-11-26T07:23:04.117410+0000","last_unstale":"2023-11-26T07:23:06.140446+0000","last_undegraded":"2023-11-26T07:23:06.140446+0000","last_fullsized":"2023-11-26T07:23:06.140446+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T19:08:47.550078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410847+0000","last_change":"2023-11-26T07:23:05.141406+0000","last_active":"2023-11-26T07:23:06.410847+0000","last_peered":"2023-11-26T07:23:06.410847+0000","last_clean":"2023-11-26T07:23:06.410847+0000","last_became_active":"2023-11-26T07:23:05.140976+0000","last_became_peered":"2023-11-26T07:23:05.140976+0000","last_unstale":"2023-11-26T07:23:06.410847+0000","last_undegraded":"2023-11-26T07:23:06.410847+0000","last_fullsized":"2023-11-26T07:23:06.410847+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:54:34.159473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"98'2","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139857+0000","last_change":"2023-11-26T07:23:04.126106+0000","last_active":"2023-11-26T07:23:06.139857+0000","last_peered":"2023-11-26T07:23:06.139857+0000","last_clean":"2023-11-26T07:23:06.139857+0000","last_became_active":"2023-11-26T07:23:04.125831+0000","last_became_peered":"2023-11-26T07:23:04.125831+0000","last_unstale":"2023-11-26T07:23:06.139857+0000","last_undegraded":"2023-11-26T07:23:06.139857+0000","last_fullsized":"2023-11-26T07:23:06.139857+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:14:33.220876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389653+0000","last_change":"2023-11-26T07:23:05.143300+0000","last_active":"2023-11-26T07:23:06.389653+0000","last_peered":"2023-11-26T07:23:06.389653+0000","last_clean":"2023-11-26T07:23:06.389653+0000","last_became_active":"2023-11-26T07:23:05.143001+0000","last_became_peered":"2023-11-26T07:23:05.143001+0000","last_unstale":"2023-11-26T07:23:06.389653+0000","last_undegraded":"2023-11-26T07:23:06.389653+0000","last_fullsized":"2023-11-26T07:23:06.389653+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:25:35.400061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465219+0000","last_change":"2023-11-26T07:23:04.124425+0000","last_active":"2023-11-26T07:23:06.465219+0000","last_peered":"2023-11-26T07:23:06.465219+0000","last_clean":"2023-11-26T07:23:06.465219+0000","last_became_active":"2023-11-26T07:23:04.124162+0000","last_became_peered":"2023-11-26T07:23:04.124162+0000","last_unstale":"2023-11-26T07:23:06.465219+0000","last_undegraded":"2023-11-26T07:23:06.465219+0000","last_fullsized":"2023-11-26T07:23:06.465219+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:42:44.097894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410689+0000","last_change":"2023-11-26T07:23:05.148143+0000","last_active":"2023-11-26T07:23:06.410689+0000","last_peered":"2023-11-26T07:23:06.410689+0000","last_clean":"2023-11-26T07:23:06.410689+0000","last_became_active":"2023-11-26T07:23:05.147786+0000","last_became_peered":"2023-11-26T07:23:05.147786+0000","last_unstale":"2023-11-26T07:23:06.410689+0000","last_undegraded":"2023-11-26T07:23:06.410689+0000","last_fullsized":"2023-11-26T07:23:06.410689+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:58:18.459826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.139652+0000","last_change":"2023-11-26T07:23:04.127017+0000","last_active":"2023-11-26T07:23:06.139652+0000","last_peered":"2023-11-26T07:23:06.139652+0000","last_clean":"2023-11-26T07:23:06.139652+0000","last_became_active":"2023-11-26T07:23:04.126670+0000","last_became_peered":"2023-11-26T07:23:04.126670+0000","last_unstale":"2023-11-26T07:23:06.139652+0000","last_undegraded":"2023-11-26T07:23:06.139652+0000","last_fullsized":"2023-11-26T07:23:06.139652+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:18:12.053806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.139902+0000","last_change":"2023-11-26T07:23:05.131466+0000","last_active":"2023-11-26T07:23:05.139902+0000","last_peered":"2023-11-26T07:23:05.139902+0000","last_clean":"2023-11-26T07:23:05.139902+0000","last_became_active":"2023-11-26T07:23:05.131182+0000","last_became_peered":"2023-11-26T07:23:05.131182+0000","last_unstale":"2023-11-26T07:23:05.139902+0000","last_undegraded":"2023-11-26T07:23:05.139902+0000","last_fullsized":"2023-11-26T07:23:05.139902+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:34:02.103968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390060+0000","last_change":"2023-11-26T07:23:04.110115+0000","last_active":"2023-11-26T07:23:06.390060+0000","last_peered":"2023-11-26T07:23:06.390060+0000","last_clean":"2023-11-26T07:23:06.390060+0000","last_became_active":"2023-11-26T07:23:04.109456+0000","last_became_peered":"2023-11-26T07:23:04.109456+0000","last_unstale":"2023-11-26T07:23:06.390060+0000","last_undegraded":"2023-11-26T07:23:06.390060+0000","last_fullsized":"2023-11-26T07:23:06.390060+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:20:19.158427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.142067+0000","last_change":"2023-11-26T07:23:05.136849+0000","last_active":"2023-11-26T07:23:06.142067+0000","last_peered":"2023-11-26T07:23:06.142067+0000","last_clean":"2023-11-26T07:23:06.142067+0000","last_became_active":"2023-11-26T07:23:05.136552+0000","last_became_peered":"2023-11-26T07:23:05.136552+0000","last_unstale":"2023-11-26T07:23:06.142067+0000","last_undegraded":"2023-11-26T07:23:06.142067+0000","last_fullsized":"2023-11-26T07:23:06.142067+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T17:53:28.488385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"98'2","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.164921+0000","last_change":"2023-11-26T07:23:04.125258+0000","last_active":"2023-11-26T07:23:05.164921+0000","last_peered":"2023-11-26T07:23:05.164921+0000","last_clean":"2023-11-26T07:23:05.164921+0000","last_became_active":"2023-11-26T07:23:04.124963+0000","last_became_peered":"2023-11-26T07:23:04.124963+0000","last_unstale":"2023-11-26T07:23:05.164921+0000","last_undegraded":"2023-11-26T07:23:05.164921+0000","last_fullsized":"2023-11-26T07:23:05.164921+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:34:53.309582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390317+0000","last_change":"2023-11-26T07:23:05.131493+0000","last_active":"2023-11-26T07:23:06.390317+0000","last_peered":"2023-11-26T07:23:06.390317+0000","last_clean":"2023-11-26T07:23:06.390317+0000","last_became_active":"2023-11-26T07:23:05.131158+0000","last_became_peered":"2023-11-26T07:23:05.131158+0000","last_unstale":"2023-11-26T07:23:06.390317+0000","last_undegraded":"2023-11-26T07:23:06.390317+0000","last_fullsized":"2023-11-26T07:23:06.390317+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T07:28:39.194729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.142052+0000","last_change":"2023-11-26T07:23:04.110606+0000","last_active":"2023-11-26T07:23:06.142052+0000","last_peered":"2023-11-26T07:23:06.142052+0000","last_clean":"2023-11-26T07:23:06.142052+0000","last_became_active":"2023-11-26T07:23:04.110255+0000","last_became_peered":"2023-11-26T07:23:04.110255+0000","last_unstale":"2023-11-26T07:23:06.142052+0000","last_undegraded":"2023-11-26T07:23:06.142052+0000","last_fullsized":"2023-11-26T07:23:06.142052+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:54:33.153486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390138+0000","last_change":"2023-11-26T07:23:05.143408+0000","last_active":"2023-11-26T07:23:06.390138+0000","last_peered":"2023-11-26T07:23:06.390138+0000","last_clean":"2023-11-26T07:23:06.390138+0000","last_became_active":"2023-11-26T07:23:05.142937+0000","last_became_peered":"2023-11-26T07:23:05.142937+0000","last_unstale":"2023-11-26T07:23:06.390138+0000","last_undegraded":"2023-11-26T07:23:06.390138+0000","last_fullsized":"2023-11-26T07:23:06.390138+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T16:29:57.245618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389858+0000","last_change":"2023-11-26T07:23:04.108545+0000","last_active":"2023-11-26T07:23:06.389858+0000","last_peered":"2023-11-26T07:23:06.389858+0000","last_clean":"2023-11-26T07:23:06.389858+0000","last_became_active":"2023-11-26T07:23:04.108220+0000","last_became_peered":"2023-11-26T07:23:04.108220+0000","last_unstale":"2023-11-26T07:23:06.389858+0000","last_undegraded":"2023-11-26T07:23:06.389858+0000","last_fullsized":"2023-11-26T07:23:06.389858+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:09:15.559785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.141937+0000","last_change":"2023-11-26T07:23:05.130219+0000","last_active":"2023-11-26T07:23:06.141937+0000","last_peered":"2023-11-26T07:23:06.141937+0000","last_clean":"2023-11-26T07:23:06.141937+0000","last_became_active":"2023-11-26T07:23:05.129451+0000","last_became_peered":"2023-11-26T07:23:05.129451+0000","last_unstale":"2023-11-26T07:23:06.141937+0000","last_undegraded":"2023-11-26T07:23:06.141937+0000","last_fullsized":"2023-11-26T07:23:06.141937+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:21:46.037771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.118889+0000","last_change":"2023-11-26T07:23:04.129022+0000","last_active":"2023-11-26T07:23:05.118889+0000","last_peered":"2023-11-26T07:23:05.118889+0000","last_clean":"2023-11-26T07:23:05.118889+0000","last_became_active":"2023-11-26T07:23:04.128705+0000","last_became_peered":"2023-11-26T07:23:04.128705+0000","last_unstale":"2023-11-26T07:23:05.118889+0000","last_undegraded":"2023-11-26T07:23:05.118889+0000","last_fullsized":"2023-11-26T07:23:05.118889+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T12:39:59.160408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.141787+0000","last_change":"2023-11-26T07:23:05.138768+0000","last_active":"2023-11-26T07:23:05.141787+0000","last_peered":"2023-11-26T07:23:05.141787+0000","last_clean":"2023-11-26T07:23:05.141787+0000","last_became_active":"2023-11-26T07:23:05.138475+0000","last_became_peered":"2023-11-26T07:23:05.138475+0000","last_unstale":"2023-11-26T07:23:05.141787+0000","last_undegraded":"2023-11-26T07:23:05.141787+0000","last_fullsized":"2023-11-26T07:23:05.141787+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:37:56.708048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140547+0000","last_change":"2023-11-26T07:23:04.108594+0000","last_active":"2023-11-26T07:23:06.140547+0000","last_peered":"2023-11-26T07:23:06.140547+0000","last_clean":"2023-11-26T07:23:06.140547+0000","last_became_active":"2023-11-26T07:23:04.108285+0000","last_became_peered":"2023-11-26T07:23:04.108285+0000","last_unstale":"2023-11-26T07:23:06.140547+0000","last_undegraded":"2023-11-26T07:23:06.140547+0000","last_fullsized":"2023-11-26T07:23:06.140547+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:59:26.288227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.147359+0000","last_change":"2023-11-26T07:23:05.142869+0000","last_active":"2023-11-26T07:23:05.147359+0000","last_peered":"2023-11-26T07:23:05.147359+0000","last_clean":"2023-11-26T07:23:05.147359+0000","last_became_active":"2023-11-26T07:23:05.142536+0000","last_became_peered":"2023-11-26T07:23:05.142536+0000","last_unstale":"2023-11-26T07:23:05.147359+0000","last_undegraded":"2023-11-26T07:23:05.147359+0000","last_fullsized":"2023-11-26T07:23:05.147359+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T11:17:38.640369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.120660+0000","last_change":"2023-11-26T07:23:04.118603+0000","last_active":"2023-11-26T07:23:05.120660+0000","last_peered":"2023-11-26T07:23:05.120660+0000","last_clean":"2023-11-26T07:23:05.120660+0000","last_became_active":"2023-11-26T07:23:04.118302+0000","last_became_peered":"2023-11-26T07:23:04.118302+0000","last_unstale":"2023-11-26T07:23:05.120660+0000","last_undegraded":"2023-11-26T07:23:05.120660+0000","last_fullsized":"2023-11-26T07:23:05.120660+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:11.116310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.410962+0000","last_change":"2023-11-26T07:23:05.146889+0000","last_active":"2023-11-26T07:23:06.410962+0000","last_peered":"2023-11-26T07:23:06.410962+0000","last_clean":"2023-11-26T07:23:06.410962+0000","last_became_active":"2023-11-26T07:23:05.146520+0000","last_became_peered":"2023-11-26T07:23:05.146520+0000","last_unstale":"2023-11-26T07:23:06.410962+0000","last_undegraded":"2023-11-26T07:23:06.410962+0000","last_fullsized":"2023-11-26T07:23:06.410962+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:29:25.410801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.465337+0000","last_change":"2023-11-26T07:23:04.122675+0000","last_active":"2023-11-26T07:23:06.465337+0000","last_peered":"2023-11-26T07:23:06.465337+0000","last_clean":"2023-11-26T07:23:06.465337+0000","last_became_active":"2023-11-26T07:23:04.119010+0000","last_became_peered":"2023-11-26T07:23:04.119010+0000","last_unstale":"2023-11-26T07:23:06.465337+0000","last_undegraded":"2023-11-26T07:23:06.465337+0000","last_fullsized":"2023-11-26T07:23:06.465337+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T10:02:45.014974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.389815+0000","last_change":"2023-11-26T07:23:05.135800+0000","last_active":"2023-11-26T07:23:06.389815+0000","last_peered":"2023-11-26T07:23:06.389815+0000","last_clean":"2023-11-26T07:23:06.389815+0000","last_became_active":"2023-11-26T07:23:05.135365+0000","last_became_peered":"2023-11-26T07:23:05.135365+0000","last_unstale":"2023-11-26T07:23:06.389815+0000","last_undegraded":"2023-11-26T07:23:06.389815+0000","last_fullsized":"2023-11-26T07:23:06.389815+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T18:33:24.633386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.390032+0000","last_change":"2023-11-26T07:23:04.110306+0000","last_active":"2023-11-26T07:23:06.390032+0000","last_peered":"2023-11-26T07:23:06.390032+0000","last_clean":"2023-11-26T07:23:06.390032+0000","last_became_active":"2023-11-26T07:23:04.109827+0000","last_became_peered":"2023-11-26T07:23:04.109827+0000","last_unstale":"2023-11-26T07:23:06.390032+0000","last_undegraded":"2023-11-26T07:23:06.390032+0000","last_fullsized":"2023-11-26T07:23:06.390032+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T08:15:59.939594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.150741+0000","last_change":"2023-11-26T07:23:05.138755+0000","last_active":"2023-11-26T07:23:05.150741+0000","last_peered":"2023-11-26T07:23:05.150741+0000","last_clean":"2023-11-26T07:23:05.150741+0000","last_became_active":"2023-11-26T07:23:05.138425+0000","last_became_peered":"2023-11-26T07:23:05.138425+0000","last_unstale":"2023-11-26T07:23:05.150741+0000","last_undegraded":"2023-11-26T07:23:05.150741+0000","last_fullsized":"2023-11-26T07:23:05.150741+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:04.088641+0000","last_clean_scrub_stamp":"2023-11-26T07:23:04.088641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:27:12.122297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-11-26T07:23:06.140881+0000","last_change":"2023-11-26T07:23:04.120474+0000","last_active":"2023-11-26T07:23:06.140881+0000","last_peered":"2023-11-26T07:23:06.140881+0000","last_clean":"2023-11-26T07:23:06.140881+0000","last_became_active":"2023-11-26T07:23:04.120171+0000","last_became_peered":"2023-11-26T07:23:04.120171+0000","last_unstale":"2023-11-26T07:23:06.140881+0000","last_undegraded":"2023-11-26T07:23:06.140881+0000","last_fullsized":"2023-11-26T07:23:06.140881+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:23:03.082654+0000","last_clean_scrub_stamp":"2023-11-26T07:23:03.082654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T15:43:01.110241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"69'128","reported_seq":269,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-11-26T07:23:05.135166+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:23:05.135166+0000","last_peered":"2023-11-26T07:23:05.135166+0000","last_clean":"2023-11-26T07:23:05.135166+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:23:05.135166+0000","last_undegraded":"2023-11-26T07:23:05.135166+0000","last_fullsized":"2023-11-26T07:23:05.135166+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1822720,"data_stored":1125115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.68500000000000005},"min":{"1min":0.42399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.93600000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.72099999999999997}]},{"osd":1,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69899999999999995,"15min":0.68300000000000005},"min":{"1min":0.503,"5min":0.437,"15min":0.437},"max":{"1min":0.90100000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.47399999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93000000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.74099999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.47999999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.86399999999999999,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70499999999999996,"15min":0.72299999999999998},"min":{"1min":0.53100000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85699999999999998,"5min":1.016,"15min":1.016},"last":0.875}]},{"osd":3,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0089999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.81499999999999995},"min":{"1min":0.64100000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68200000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.80000000000000004,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.63100000000000001,"15min":0.68400000000000005},"min":{"1min":0.45100000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84099999999999997,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71999999999999997,"15min":0.77000000000000002},"min":{"1min":0.56000000000000005,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95099999999999996,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.76000000000000001,"15min":0.83999999999999997},"min":{"1min":0.60799999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96799999999999997,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.84399999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70199999999999996,"15min":0.75600000000000001},"min":{"1min":0.501,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77400000000000002,"15min":0.82599999999999996},"min":{"1min":0.57199999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.69899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1236992,"data_stored":540501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.84799999999999998,"15min":0.77400000000000002},"min":{"1min":0.45900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82499999999999996,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":1.226,"15min":0.90800000000000003},"min":{"1min":0.499,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.80600000000000005,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.59899999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.71599999999999997,"15min":0.74399999999999999},"min":{"1min":0.46700000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81100000000000005,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.79300000000000004,"15min":0.81999999999999995},"min":{"1min":0.42899999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.96599999999999997,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.77900000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.78000000000000003,"15min":0.76000000000000001},"min":{"1min":0.52900000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91800000000000004,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":0.69199999999999995},"min":{"1min":0.47599999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.026,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.75900000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.81499999999999995,"15min":0.751},"min":{"1min":0.57899999999999996,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97099999999999997,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.91600000000000004,"15min":0.86399999999999999},"min":{"1min":0.46100000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82899999999999996,"5min":18.77,"15min":18.77},"last":0.65700000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81899999999999995,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66500000000000004,"15min":0.71299999999999997},"min":{"1min":0.48399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.92500000000000004,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.63600000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65900000000000003,"15min":0.71299999999999997},"min":{"1min":0.42099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.048,"5min":1.508,"15min":1.508},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74199999999999999,"15min":0.78300000000000003},"min":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0900000000000001,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.746,"15min":0.78500000000000003},"min":{"1min":0.53800000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.98799999999999999,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.77500000000000002},"min":{"1min":0.52200000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.93300000000000005,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.69699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":541035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64300000000000002,"15min":0.59299999999999997},"min":{"1min":0.39800000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.55800000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.65500000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.66900000000000004,"15min":0.64300000000000002},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67400000000000004,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.627}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.70299999999999996,"15min":0.68600000000000005},"min":{"1min":0.58099999999999996,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67800000000000005,"15min":0.69199999999999995},"min":{"1min":0.45200000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.78300000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.96299999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64700000000000002,"15min":0.68500000000000005},"min":{"1min":0.46300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.105,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.77100000000000002},"min":{"1min":0.65500000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.016}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.625,"15min":0.67200000000000004},"min":{"1min":0.35699999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.871},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.68300000000000005,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.434,"15min":0.434},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.94299999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.755,"15min":0.76000000000000001},"min":{"1min":0.39900000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.754,"15min":0.80700000000000005},"min":{"1min":0.48499999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.0329999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574923,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":534131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66100000000000003,"15min":0.67900000000000005},"min":{"1min":0.41599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90300000000000002,"5min":1.016,"15min":1.016},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65400000000000003,"15min":0.67100000000000004},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.69899999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67800000000000005,"15min":0.68000000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.996,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69499999999999995,"15min":0.70999999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.05,"5min":1.077,"15min":1.077},"last":0.76900000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.55100000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.236,"5min":1.302,"15min":1.302},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.71399999999999997,"15min":0.748},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.129,"5min":1.181,"15min":1.181},"last":0.71999999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72199999999999998,"15min":0.747},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75,"15min":0.78400000000000003},"min":{"1min":0.47599999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.323,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.85499999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67800000000000005,"15min":0.69599999999999995},"min":{"1min":0.39900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.67300000000000004,"15min":0.68400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":0.54500000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70799999999999996,"15min":0.73099999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.75700000000000001,"15min":0.78900000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.64400000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.74199999999999999,"15min":0.746},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":4.3609999999999998,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77000000000000002,"15min":0.81000000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.754}]}]},{"osd":3,"up_from":13,"seq":55834574921,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1236992,"data_stored":535813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47899999999999998,"15min":0.47499999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.627,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.51400000000000001,"15min":0.48999999999999999},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.70199999999999996,"5min":1.294,"15min":1.294},"last":0.439}]},{"osd":1,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59999999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.376},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.64600000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54200000000000004,"15min":0.52500000000000002},"min":{"1min":0.42399999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.67600000000000005,"5min":1.456,"15min":1.456},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.73699999999999999,"5min":1.139,"15min":1.139},"last":0.56299999999999994}]},{"osd":4,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.54500000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91500000000000004,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.75900000000000001,"15min":0.77000000000000002},"min":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99199999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.53100000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.55700000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.89300000000000002,"5min":1.395,"15min":1.395},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76600000000000001,"15min":0.77200000000000002},"min":{"1min":0.504,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86299999999999999,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78200000000000003,"15min":0.79000000000000004},"min":{"1min":0.57399999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95999999999999996,"5min":1.333,"15min":1.333},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.78800000000000003,"15min":0.80500000000000005},"min":{"1min":0.59699999999999998,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":0.89000000000000001,"5min":1.347,"15min":1.347},"last":0.80400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.78400000000000003,"15min":0.79700000000000004},"min":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94599999999999995,"5min":1.397,"15min":1.397},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.79200000000000004,"15min":0.80900000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.95999999999999996,"5min":1.363,"15min":1.363},"last":0.73799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574922,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":540431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.51500000000000001,"15min":0.63300000000000001},"min":{"1min":0.29399999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73699999999999999,"5min":3.931,"15min":3.931},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.58399999999999996,"15min":0.69499999999999995},"min":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88700000000000001,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.44800000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.55800000000000005,"15min":0.67800000000000005},"min":{"1min":0.35499999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.871,"5min":3.9929999999999999,"15min":3.9929999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.57299999999999995,"15min":0.67700000000000005},"min":{"1min":0.35499999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68100000000000005,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.40500000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.32900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.80000000000000004,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.66300000000000003,"15min":0.873},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.70199999999999996,"5min":8.0990000000000002,"15min":8.0990000000000002},"last":0.55200000000000005}]},{"osd":4,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.91800000000000004},"min":{"1min":0.58299999999999996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0149999999999999,"5min":4.431,"15min":4.431},"last":0.749},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.82299999999999995,"15min":0.89900000000000002},"min":{"1min":0.58299999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.284,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.80700000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2589999999999999,"5min":2.036,"15min":2.036},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.85399999999999998,"15min":0.95399999999999996},"min":{"1min":0.628,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.173,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.85699999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.84899999999999998,"15min":0.96899999999999997},"min":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1100000000000001,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81599999999999995,"15min":0.82399999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.181,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.83599999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.89400000000000002,"15min":0.97999999999999998},"min":{"1min":0.52600000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2110000000000001,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.877},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.81000000000000005,"15min":0.84399999999999997},"min":{"1min":0.59499999999999997,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.323,"5min":1.609,"15min":1.609},"last":0.89700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":540555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.442,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.56499999999999995,"5min":1.034,"15min":1.034},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.41399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68999999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.501}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.43099999999999999,"15min":0.45100000000000001},"min":{"1min":0.315,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65200000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.502,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64400000000000002,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.376}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.54900000000000004,"15min":0.57899999999999996},"min":{"1min":0.42899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.68000000000000005,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.49399999999999999,"15min":0.502},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44600000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.52100000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0860000000000001,"5min":1.143,"15min":1.143},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.72099999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.0449999999999999,"5min":1.216,"15min":1.216},"last":0.82799999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.48799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.171,"5min":1.262,"15min":1.262},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.746,"15min":0.748},"min":{"1min":0.57099999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.343,"5min":1.54,"15min":1.54},"last":0.81799999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76700000000000002,"15min":0.73899999999999999},"min":{"1min":0.59699999999999998,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.76600000000000001,"15min":0.77900000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.254,"5min":1.333,"15min":1.333},"last":0.85699999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.751,"15min":0.75900000000000001},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2090000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.751,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.86899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32984,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338856,"statfs":{"total":96636764160,"available":96602988544,"internally_reserved":0,"allocated":1843200,"data_stored":1131504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.43099999999999999,"15min":0.44900000000000001},"min":{"1min":0.26800000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.57899999999999996,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.47999999999999998,"15min":0.47599999999999998},"min":{"1min":0.35599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.63800000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.41599999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.622,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67000000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.44600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48799999999999999,"15min":0.47799999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.60199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.41699999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73799999999999999,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.38400000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.56299999999999994,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0289999999999999,"5min":1.365,"15min":1.365},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81799999999999995,"15min":0.80900000000000005},"min":{"1min":0.51300000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0980000000000001,"5min":1.181,"15min":1.181},"last":0.57599999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90300000000000002,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.755},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77300000000000002,"15min":0.76900000000000002},"min":{"1min":0.57099999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.147,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.78500000000000003,"15min":0.80300000000000005},"min":{"1min":0.56100000000000005,"5min":0.503,"15min":0.503},"max":{"1min":0.91700000000000004,"5min":1.323,"15min":1.323},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.089,"5min":1.123,"15min":1.123},"last":0.622},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.82799999999999996,"15min":0.81799999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.111,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.65900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:23:09.632 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-26T07:23:09.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-26T07:23:09.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6bad701640 1 Processor -- start 2023-11-26T07:23:09.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6bad701640 1 -- start start 2023-11-26T07:23:09.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6bad701640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f6ba8154940 0x7f6ba806da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6bad701640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba815e6a0 0x7f6ba80ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6bad701640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba80cd870 con 0x7f6ba806df50 2023-11-26T07:23:09.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6bad701640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba8154d20 con 0x7f6ba8154940 2023-11-26T07:23:09.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6bad701640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba815ecc0 con 0x7f6ba815e6a0 2023-11-26T07:23:09.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6ba67fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6ba806df50 0x7f6ba815df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60594/0 (socket says 172.21.15.31:60594) 2023-11-26T07:23:09.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6ba67fc640 1 -- 172.21.15.31:0/380710342 learned_addr learned my addr 172.21.15.31:0/380710342 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:09.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6ba6ffd640 1 --2- 172.21.15.31:0/380710342 >> v2:172.21.15.97:3300/0 conn(0x7f6ba8154940 0x7f6ba806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6ba77fe640 1 --2- 172.21.15.31:0/380710342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba815e6a0 0x7f6ba80ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6ba5ffb640 1 -- 172.21.15.31:0/380710342 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480045069 0 0) 0x7f6ba80cd870 con 0x7f6ba806df50 2023-11-26T07:23:09.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6ba5ffb640 1 -- 172.21.15.31:0/380710342 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b90003530 con 0x7f6ba806df50 2023-11-26T07:23:09.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6ba77fe640 1 -- 172.21.15.31:0/380710342 >> v2:172.21.15.97:3300/0 conn(0x7f6ba8154940 msgr2=0x7f6ba806da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:09.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6ba77fe640 1 --2- 172.21.15.31:0/380710342 >> v2:172.21.15.97:3300/0 conn(0x7f6ba8154940 0x7f6ba806da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6ba77fe640 1 -- 172.21.15.31:0/380710342 >> v1:172.21.15.31:6789/0 conn(0x7f6ba806df50 legacy=0x7f6ba815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.817+0000 7f6ba77fe640 1 -- 172.21.15.31:0/380710342 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba80ccf40 con 0x7f6ba815e6a0 2023-11-26T07:23:09.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6ba77fe640 1 --2- 172.21.15.31:0/380710342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba815e6a0 0x7f6ba80ad080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009d90 tx=0x7f6ba001c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3d6fd32dfb8bd58 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6ba5ffb640 1 -- 172.21.15.31:0/380710342 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00a5800 con 0x7f6ba815e6a0 2023-11-26T07:23:09.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6ba5ffb640 1 -- 172.21.15.31:0/380710342 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba00a5960 con 0x7f6ba815e6a0 2023-11-26T07:23:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6ba5ffb640 1 -- 172.21.15.31:0/380710342 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00a5cf0 con 0x7f6ba815e6a0 2023-11-26T07:23:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6bad701640 1 -- 172.21.15.31:0/380710342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba815e6a0 msgr2=0x7f6ba80ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6bad701640 1 --2- 172.21.15.31:0/380710342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba815e6a0 0x7f6ba80ad080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009d90 tx=0x7f6ba001c740 comp rx=0 tx=0).stop 2023-11-26T07:23:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6bad701640 1 -- 172.21.15.31:0/380710342 shutdown_connections 2023-11-26T07:23:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6bad701640 1 --2- 172.21.15.31:0/380710342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba815e6a0 0x7f6ba80ad080 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6bad701640 1 --2- 172.21.15.31:0/380710342 >> v2:172.21.15.97:3300/0 conn(0x7f6ba8154940 0x7f6ba806da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6bad701640 1 -- 172.21.15.31:0/380710342 >> 172.21.15.31:0/380710342 conn(0x7f6ba80b0cb0 msgr2=0x7f6ba80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6bad701640 1 -- 172.21.15.31:0/380710342 shutdown_connections 2023-11-26T07:23:09.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.821+0000 7f6bad701640 1 -- 172.21.15.31:0/380710342 wait complete. 2023-11-26T07:23:09.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6bad701640 1 Processor -- start 2023-11-26T07:23:09.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6bad701640 1 -- start start 2023-11-26T07:23:09.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6bad701640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba806df50 0x7f6ba8150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6bad701640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ba8154940 0x7f6ba80ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6bad701640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ba815e6a0 0x7f6ba809be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6bad701640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8150e90 con 0x7f6ba8154940 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6bad701640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81530f0 con 0x7f6ba815e6a0 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6bad701640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba81533d0 con 0x7f6ba806df50 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ba8154940 0x7f6ba80ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ba8154940 0x7f6ba80ac790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36306/0 (socket says 172.21.15.31:36306) 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba67fc640 1 -- 172.21.15.31:0/3976849120 learned_addr learned my addr 172.21.15.31:0/3976849120 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba6ffd640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba806df50 0x7f6ba8150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba77fe640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ba815e6a0 0x7f6ba809be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba67fc640 1 -- 172.21.15.31:0/3976849120 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba806df50 msgr2=0x7f6ba8150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba67fc640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba806df50 0x7f6ba8150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba67fc640 1 -- 172.21.15.31:0/3976849120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ba815e6a0 msgr2=0x7f6ba809be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba67fc640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ba815e6a0 0x7f6ba809be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba67fc640 1 -- 172.21.15.31:0/3976849120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba0009a70 con 0x7f6ba8154940 2023-11-26T07:23:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba6ffd640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba806df50 0x7f6ba8150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba77fe640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ba815e6a0 0x7f6ba809be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6ba67fc640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ba8154940 0x7f6ba80ac790 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c450 tx=0x7f6b9c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6b8f7fe640 1 -- 172.21.15.31:0/3976849120 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c005cf0 con 0x7f6ba8154940 2023-11-26T07:23:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6b8f7fe640 1 -- 172.21.15.31:0/3976849120 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c005e50 con 0x7f6ba8154940 2023-11-26T07:23:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6b8f7fe640 1 -- 172.21.15.31:0/3976849120 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c007290 con 0x7f6ba8154940 2023-11-26T07:23:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6bad701640 1 -- 172.21.15.31:0/3976849120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba81536b0 con 0x7f6ba8154940 2023-11-26T07:23:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.825+0000 7f6bad701640 1 -- 172.21.15.31:0/3976849120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba8153c70 con 0x7f6ba8154940 2023-11-26T07:23:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.829+0000 7f6bad701640 1 -- 172.21.15.31:0/3976849120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba80671f0 con 0x7f6ba8154940 2023-11-26T07:23:09.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.829+0000 7f6b8f7fe640 1 -- 172.21.15.31:0/3976849120 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f6b9c0073f0 con 0x7f6ba8154940 2023-11-26T07:23:09.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.829+0000 7f6b8f7fe640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6b94066590 0x7f6b94068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:09.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.829+0000 7f6b8f7fe640 1 -- 172.21.15.31:0/3976849120 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b9c083670 con 0x7f6ba8154940 2023-11-26T07:23:09.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.829+0000 7f6ba6ffd640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6b94066590 0x7f6b94068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:09.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.829+0000 7f6ba6ffd640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6b94066590 0x7f6b94068a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009ae0 tx=0x7f6b98008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:09.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:09.833+0000 7f6b8f7fe640 1 -- 172.21.15.31:0/3976849120 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b9c050940 con 0x7f6ba8154940 2023-11-26T07:23:10.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.045+0000 7f6bad701640 1 -- 172.21.15.31:0/3976849120 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6ba806b460 con 0x7f6ba8154940 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.049+0000 7f6b8f7fe640 1 -- 172.21.15.31:0/3976849120 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6b9c0170e0 con 0x7f6ba8154940 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDQ8GJl8FRCMBAAc0rEFWW7PIN0cwo/Kd/jMA== 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: caps mds = "allow" 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: caps mon = "allow r" 2023-11-26T07:23:10.052 INFO:teuthology.orchestra.run.smithi031.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-26T07:23:10.053 INFO:teuthology.orchestra.run.smithi031.stderr:updated caps for client.0 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 -- 172.21.15.31:0/3976849120 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6b94066590 msgr2=0x7f6b94068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6b94066590 0x7f6b94068a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009ae0 tx=0x7f6b98008040 comp rx=0 tx=0).stop 2023-11-26T07:23:10.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 -- 172.21.15.31:0/3976849120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ba8154940 msgr2=0x7f6ba80ac790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ba8154940 0x7f6ba80ac790 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c450 tx=0x7f6b9c00c910 comp rx=0 tx=0).stop 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 -- 172.21.15.31:0/3976849120 shutdown_connections 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6b94066590 0x7f6b94068a50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ba815e6a0 0x7f6ba809be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ba8154940 0x7f6ba80ac790 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 --2- 172.21.15.31:0/3976849120 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6ba806df50 0x7f6ba8150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 -- 172.21.15.31:0/3976849120 >> 172.21.15.31:0/3976849120 conn(0x7f6ba80b0cb0 msgr2=0x7f6ba80aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 -- 172.21.15.31:0/3976849120 shutdown_connections 2023-11-26T07:23:10.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.053+0000 7f6bad701640 1 -- 172.21.15.31:0/3976849120 wait complete. 2023-11-26T07:23:10.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5118e91640 1 Processor -- start 2023-11-26T07:23:10.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5118e91640 1 -- start start 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5118e91640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f511406dfd0 0x7f511406e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5118e91640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51140adcc0 0x7f51140ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5118e91640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511406e980 con 0x7f5114143ae0 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5118e91640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5114070670 con 0x7f51140adcc0 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5118e91640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5114070950 con 0x7f511406dfd0 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f511259c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5114143ae0 0x7f51140b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60606/0 (socket says 172.21.15.31:60606) 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f511259c640 1 -- 172.21.15.31:0/2303369416 learned_addr learned my addr 172.21.15.31:0/2303369416 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:10.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5112d9d640 1 --2- 172.21.15.31:0/2303369416 >> v2:172.21.15.31:3300/0 conn(0x7f51140adcc0 0x7f51140ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5111d9b640 1 --2- 172.21.15.31:0/2303369416 >> v2:172.21.15.97:3300/0 conn(0x7f511406dfd0 0x7f511406e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f511159a640 1 -- 172.21.15.31:0/2303369416 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344385726 0 0) 0x7f511406e980 con 0x7f5114143ae0 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f511159a640 1 -- 172.21.15.31:0/2303369416 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50fc003530 con 0x7f5114143ae0 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5112d9d640 1 -- 172.21.15.31:0/2303369416 >> v2:172.21.15.97:3300/0 conn(0x7f511406dfd0 msgr2=0x7f511406e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5112d9d640 1 --2- 172.21.15.31:0/2303369416 >> v2:172.21.15.97:3300/0 conn(0x7f511406dfd0 0x7f511406e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5112d9d640 1 -- 172.21.15.31:0/2303369416 >> v1:172.21.15.31:6789/0 conn(0x7f5114143ae0 legacy=0x7f51140b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:10.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.217+0000 7f5112d9d640 1 -- 172.21.15.31:0/2303369416 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114070c30 con 0x7f51140adcc0 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f5112d9d640 1 --2- 172.21.15.31:0/2303369416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51140adcc0 0x7f51140ae0e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f511406cbe0 tx=0x7f510c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d074608bcf14971 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f511159a640 1 -- 172.21.15.31:0/2303369416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c0a59a0 con 0x7f51140adcc0 2023-11-26T07:23:10.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f511159a640 1 -- 172.21.15.31:0/2303369416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c0a5b00 con 0x7f51140adcc0 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f511159a640 1 -- 172.21.15.31:0/2303369416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c0a5e90 con 0x7f51140adcc0 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f5118e91640 1 -- 172.21.15.31:0/2303369416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51140adcc0 msgr2=0x7f51140ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:10.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f5118e91640 1 --2- 172.21.15.31:0/2303369416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51140adcc0 0x7f51140ae0e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f511406cbe0 tx=0x7f510c01c8e0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f5118e91640 1 -- 172.21.15.31:0/2303369416 shutdown_connections 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f5118e91640 1 --2- 172.21.15.31:0/2303369416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51140adcc0 0x7f51140ae0e0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f5118e91640 1 --2- 172.21.15.31:0/2303369416 >> v2:172.21.15.97:3300/0 conn(0x7f511406dfd0 0x7f511406e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f5118e91640 1 -- 172.21.15.31:0/2303369416 >> 172.21.15.31:0/2303369416 conn(0x7f51140b0c40 msgr2=0x7f51140b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f5118e91640 1 -- 172.21.15.31:0/2303369416 shutdown_connections 2023-11-26T07:23:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.221+0000 7f5118e91640 1 -- 172.21.15.31:0/2303369416 wait complete. 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 Processor -- start 2023-11-26T07:23:10.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 -- start start 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f511406dfd0 0x7f511415bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51140adcc0 0x7f511415d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5114143ae0 0x7f5114158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511413c830 con 0x7f5114143ae0 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5114159450 con 0x7f51140adcc0 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51141f95e0 con 0x7f511406dfd0 2023-11-26T07:23:10.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5112d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5114143ae0 0x7f5114158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5112d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5114143ae0 0x7f5114158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36330/0 (socket says 172.21.15.31:36330) 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5112d9d640 1 -- 172.21.15.31:0/1096275392 learned_addr learned my addr 172.21.15.31:0/1096275392 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5111d9b640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51140adcc0 0x7f511415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f511259c640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f511406dfd0 0x7f511415bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5112d9d640 1 -- 172.21.15.31:0/1096275392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f511406dfd0 msgr2=0x7f511415bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5112d9d640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f511406dfd0 0x7f511415bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5112d9d640 1 -- 172.21.15.31:0/1096275392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51140adcc0 msgr2=0x7f511415d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5112d9d640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51140adcc0 0x7f511415d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5112d9d640 1 -- 172.21.15.31:0/1096275392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c009a70 con 0x7f5114143ae0 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5111d9b640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51140adcc0 0x7f511415d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5112d9d640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5114143ae0 0x7f5114158f10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f510c0194f0 tx=0x7f510c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5102ffd640 1 -- 172.21.15.31:0/1096275392 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c0add70 con 0x7f5114143ae0 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5102ffd640 1 -- 172.21.15.31:0/1096275392 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c009450 con 0x7f5114143ae0 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 -- 172.21.15.31:0/1096275392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51141f9880 con 0x7f5114143ae0 2023-11-26T07:23:10.230 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5102ffd640 1 -- 172.21.15.31:0/1096275392 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c0b9690 con 0x7f5114143ae0 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 -- 172.21.15.31:0/1096275392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51141f9e40 con 0x7f5114143ae0 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.225+0000 7f5118e91640 1 -- 172.21.15.31:0/1096275392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511404e850 con 0x7f5114143ae0 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.229+0000 7f5102ffd640 1 -- 172.21.15.31:0/1096275392 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f510c0b97f0 con 0x7f5114143ae0 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.229+0000 7f5102ffd640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50f4066590 0x7f50f4068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:10.231 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.229+0000 7f5102ffd640 1 -- 172.21.15.31:0/1096275392 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f510c11f800 con 0x7f5114143ae0 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.229+0000 7f511259c640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50f4066590 0x7f50f4068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.229+0000 7f511259c640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50f4066590 0x7f50f4068a50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f51040011f0 tx=0x7f5104009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.233+0000 7f5102ffd640 1 -- 172.21.15.31:0/1096275392 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f510c0ecb60 con 0x7f5114143ae0 2023-11-26T07:23:10.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.425+0000 7f5118e91640 1 -- 172.21.15.31:0/1096275392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f511406cc40 con 0x7f5114143ae0 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.425+0000 7f5102ffd640 1 -- 172.21.15.31:0/1096275392 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f510c0f0180 con 0x7f5114143ae0 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:10.428 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3797004646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3797004646},{"type":"v1","addr":"172.21.15.97:6833","nonce":3797004646}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2675576819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2675576819},{"type":"v1","addr":"172.21.15.97:6835","nonce":2675576819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1710103664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1710103664},{"type":"v1","addr":"172.21.15.31:6834","nonce":1710103664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:04.497314+0000","modified":"2023-11-26T07:23:07.528539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1001580514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1001580514},{"type":"v1","addr":"172.21.15.31:6835","nonce":1001580514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:23:10.429 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 72 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 -- 172.21.15.31:0/1096275392 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50f4066590 msgr2=0x7f50f4068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50f4066590 0x7f50f4068a50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f51040011f0 tx=0x7f5104009040 comp rx=0 tx=0).stop 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 -- 172.21.15.31:0/1096275392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5114143ae0 msgr2=0x7f5114158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5114143ae0 0x7f5114158f10 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f510c0194f0 tx=0x7f510c019680 comp rx=0 tx=0).stop 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 -- 172.21.15.31:0/1096275392 shutdown_connections 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50f4066590 0x7f50f4068a50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5114143ae0 0x7f5114158f10 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51140adcc0 0x7f511415d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 --2- 172.21.15.31:0/1096275392 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f511406dfd0 0x7f511415bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 -- 172.21.15.31:0/1096275392 >> 172.21.15.31:0/1096275392 conn(0x7f51140b0c40 msgr2=0x7f51140aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 -- 172.21.15.31:0/1096275392 shutdown_connections 2023-11-26T07:23:10.431 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.429+0000 7f5118e91640 1 -- 172.21.15.31:0/1096275392 wait complete. 2023-11-26T07:23:10.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-26T07:23:04.497314+0000', 'modified': '2023-11-26T07:23:07.528539+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7575': {'gid': 7575, 'name': 'c', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/1001580514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6833', 'nonce': 1001580514}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 1001580514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-26T07:23:10.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-26T07:23:10.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-26T07:23:10.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10ffd35640 1 Processor -- start 2023-11-26T07:23:10.999 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10ffd35640 1 -- start start 2023-11-26T07:23:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10ffd35640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f10f806e3f0 0x7f10f815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10ffd35640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80ce5d0 0x7f10f815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10ffd35640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f8159630 con 0x7f10f80cb9b0 2023-11-26T07:23:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10ffd35640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f806e860 con 0x7f10f806e3f0 2023-11-26T07:23:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10ffd35640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f815bba0 con 0x7f10f80ce5d0 2023-11-26T07:23:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10fdad1640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f10f80cb9b0 0x7f10f80cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60618/0 (socket says 172.21.15.31:60618) 2023-11-26T07:23:11.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10fdad1640 1 -- 172.21.15.31:0/1137859166 learned_addr learned my addr 172.21.15.31:0/1137859166 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10fe2d2640 1 --2- 172.21.15.31:0/1137859166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80ce5d0 0x7f10f815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10fd2d0640 1 --2- 172.21.15.31:0/1137859166 >> v2:172.21.15.97:3300/0 conn(0x7f10f806e3f0 0x7f10f815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10fcacf640 1 -- 172.21.15.31:0/1137859166 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3292390904 0 0) 0x7f10f8159630 con 0x7f10f80cb9b0 2023-11-26T07:23:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:10.997+0000 7f10fcacf640 1 -- 172.21.15.31:0/1137859166 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e4003530 con 0x7f10f80cb9b0 2023-11-26T07:23:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10fe2d2640 1 -- 172.21.15.31:0/1137859166 >> v2:172.21.15.97:3300/0 conn(0x7f10f806e3f0 msgr2=0x7f10f815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10fe2d2640 1 --2- 172.21.15.31:0/1137859166 >> v2:172.21.15.97:3300/0 conn(0x7f10f806e3f0 0x7f10f815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10fcacf640 1 -- 172.21.15.31:0/1137859166 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843558531 0 0) 0x7f10e4003530 con 0x7f10f80cb9b0 2023-11-26T07:23:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10fe2d2640 1 -- 172.21.15.31:0/1137859166 >> v1:172.21.15.31:6789/0 conn(0x7f10f80cb9b0 legacy=0x7f10f80cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10fe2d2640 1 -- 172.21.15.31:0/1137859166 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f815f6d0 con 0x7f10f80ce5d0 2023-11-26T07:23:11.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10fe2d2640 1 --2- 172.21.15.31:0/1137859166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80ce5d0 0x7f10f815bfd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009d90 tx=0x7f10f001c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f1566074fa2ae2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10fcacf640 1 -- 172.21.15.31:0/1137859166 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4001680 con 0x7f10f80ce5d0 2023-11-26T07:23:11.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10fcacf640 1 -- 172.21.15.31:0/1137859166 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f00a5540 con 0x7f10f80ce5d0 2023-11-26T07:23:11.005 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10fcacf640 1 -- 172.21.15.31:0/1137859166 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00a58b0 con 0x7f10f80ce5d0 2023-11-26T07:23:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10ffd35640 1 -- 172.21.15.31:0/1137859166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80ce5d0 msgr2=0x7f10f815bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.001+0000 7f10ffd35640 1 --2- 172.21.15.31:0/1137859166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80ce5d0 0x7f10f815bfd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009d90 tx=0x7f10f001c870 comp rx=0 tx=0).stop 2023-11-26T07:23:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 -- 172.21.15.31:0/1137859166 shutdown_connections 2023-11-26T07:23:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 --2- 172.21.15.31:0/1137859166 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80ce5d0 0x7f10f815bfd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 --2- 172.21.15.31:0/1137859166 >> v2:172.21.15.97:3300/0 conn(0x7f10f806e3f0 0x7f10f815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 -- 172.21.15.31:0/1137859166 >> 172.21.15.31:0/1137859166 conn(0x7f10f80b0cb0 msgr2=0x7f10f80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 -- 172.21.15.31:0/1137859166 shutdown_connections 2023-11-26T07:23:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 -- 172.21.15.31:0/1137859166 wait complete. 2023-11-26T07:23:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 Processor -- start 2023-11-26T07:23:11.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 -- start start 2023-11-26T07:23:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806e3f0 0x7f10f8153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80cb9b0 0x7f10f8158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80ce5d0 0x7f10f8154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8153570 con 0x7f10f80ce5d0 2023-11-26T07:23:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80af000 con 0x7f10f806e3f0 2023-11-26T07:23:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10ffd35640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f815ab70 con 0x7f10f80cb9b0 2023-11-26T07:23:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fe2d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80ce5d0 0x7f10f8154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fe2d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80ce5d0 0x7f10f8154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36338/0 (socket says 172.21.15.31:36338) 2023-11-26T07:23:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fe2d2640 1 -- 172.21.15.31:0/2586364595 learned_addr learned my addr 172.21.15.31:0/2586364595 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fd2d0640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80cb9b0 0x7f10f8158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fdad1640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806e3f0 0x7f10f8153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fe2d2640 1 -- 172.21.15.31:0/2586364595 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80cb9b0 msgr2=0x7f10f8158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fe2d2640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80cb9b0 0x7f10f8158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fe2d2640 1 -- 172.21.15.31:0/2586364595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806e3f0 msgr2=0x7f10f8153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fe2d2640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806e3f0 0x7f10f8153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fe2d2640 1 -- 172.21.15.31:0/2586364595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0009a70 con 0x7f10f80ce5d0 2023-11-26T07:23:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.005+0000 7f10fe2d2640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80ce5d0 0x7f10f8154f10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009d90 tx=0x7f10f0019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.009+0000 7f10ee7fc640 1 -- 172.21.15.31:0/2586364595 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0009450 con 0x7f10f80ce5d0 2023-11-26T07:23:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.009+0000 7f10ee7fc640 1 -- 172.21.15.31:0/2586364595 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f00ad440 con 0x7f10f80ce5d0 2023-11-26T07:23:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.009+0000 7f10ee7fc640 1 -- 172.21.15.31:0/2586364595 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00ad7b0 con 0x7f10f80ce5d0 2023-11-26T07:23:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.009+0000 7f10ffd35640 1 -- 172.21.15.31:0/2586364595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f815ae50 con 0x7f10f80ce5d0 2023-11-26T07:23:11.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.009+0000 7f10ffd35640 1 -- 172.21.15.31:0/2586364595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f815b410 con 0x7f10f80ce5d0 2023-11-26T07:23:11.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.009+0000 7f10ee7fc640 1 -- 172.21.15.31:0/2586364595 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f10f00ad910 con 0x7f10f80ce5d0 2023-11-26T07:23:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.009+0000 7f10ee7fc640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f10dc066590 0x7f10dc068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.009+0000 7f10ee7fc640 1 -- 172.21.15.31:0/2586364595 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10f011ef80 con 0x7f10f80ce5d0 2023-11-26T07:23:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.009+0000 7f10fdad1640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f10dc066590 0x7f10dc068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.009+0000 7f1103093000 1 Processor -- start 2023-11-26T07:23:11.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 -- start start 2023-11-26T07:23:11.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fdad1640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f10dc066590 0x7f10dc068a50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f10f4000fc0 tx=0x7f10f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ee9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7e9ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7e9f320 0x55d1b7e9f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7d09130 con 0x55d1b7ee8c90 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7cf69a0 con 0x55d1b7e9f320 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7cf3ed0 con 0x55d1b7ee95b0 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fdad1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ee9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fdad1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ee9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36346/0 (socket says 172.21.15.31:36346) 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fdad1640 1 -- 172.21.15.31:0/1137244735 learned_addr learned my addr 172.21.15.31:0/1137244735 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fd2d0640 1 --2- 172.21.15.31:0/1137244735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7e9ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fe2d2640 1 --2- 172.21.15.31:0/1137244735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7e9f320 0x55d1b7e9f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fdad1640 1 -- 172.21.15.31:0/1137244735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 msgr2=0x55d1b7e9ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1137244735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7e9ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fdad1640 1 -- 172.21.15.31:0/1137244735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7e9f320 msgr2=0x55d1b7e9f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1137244735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7e9f320 0x55d1b7e9f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.016 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fdad1640 1 -- 172.21.15.31:0/1137244735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1b79e7890 con 0x55d1b7ee8c90 2023-11-26T07:23:11.017 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1137244735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ee9070 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f10f4005b50 tx=0x7f10f4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3996351382951081 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10db7fe640 1 -- 172.21.15.31:0/1137244735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4007710 con 0x55d1b7ee8c90 2023-11-26T07:23:11.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10db7fe640 1 -- 172.21.15.31:0/1137244735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f400e9c0 con 0x55d1b7ee8c90 2023-11-26T07:23:11.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f10db7fe640 1 -- 172.21.15.31:0/1137244735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f400ed30 con 0x55d1b7ee8c90 2023-11-26T07:23:11.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 -- 172.21.15.31:0/1137244735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 msgr2=0x55d1b7ee9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 --2- 172.21.15.31:0/1137244735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ee9070 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f10f4005b50 tx=0x7f10f4011000 comp rx=0 tx=0).stop 2023-11-26T07:23:11.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 -- 172.21.15.31:0/1137244735 shutdown_connections 2023-11-26T07:23:11.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 --2- 172.21.15.31:0/1137244735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7e9f320 0x55d1b7e9f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 --2- 172.21.15.31:0/1137244735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7e9ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 --2- 172.21.15.31:0/1137244735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ee9070 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 -- 172.21.15.31:0/1137244735 >> 172.21.15.31:0/1137244735 conn(0x55d1b7e9b2d0 msgr2=0x55d1b7e9d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 -- 172.21.15.31:0/1137244735 shutdown_connections 2023-11-26T07:23:11.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 -- 172.21.15.31:0/1137244735 wait complete. 2023-11-26T07:23:11.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 -- start start 2023-11-26T07:23:11.019 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.013+0000 7f1103093000 1 Processor -- start 2023-11-26T07:23:11.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7e9f320 0x55d1b7ea3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.021 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ea4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7ee95b0 0x55d1b7eb7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7cef7c0 con 0x55d1b7ee8c90 2023-11-26T07:23:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7ca7bf0 con 0x55d1b7ee95b0 2023-11-26T07:23:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7cada90 con 0x55d1b7e9f320 2023-11-26T07:23:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 10 client.0 _ll_register_callbacks cb 0x55d1b7e8a3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:23:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fd2d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ea4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fd2d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ea4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36354/0 (socket says 172.21.15.31:36354) 2023-11-26T07:23:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fd2d0640 1 -- 172.21.15.31:0/1296719952 learned_addr learned my addr 172.21.15.31:0/1296719952 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7e9f320 0x55d1b7ea3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.022 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fe2d2640 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7ee95b0 0x55d1b7eb7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fd2d0640 1 -- 172.21.15.31:0/1296719952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7e9f320 msgr2=0x55d1b7ea3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fd2d0640 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7e9f320 0x55d1b7ea3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fd2d0640 1 -- 172.21.15.31:0/1296719952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7ee95b0 msgr2=0x55d1b7eb7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fd2d0640 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7ee95b0 0x55d1b7eb7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fd2d0640 1 -- 172.21.15.31:0/1296719952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f400f040 con 0x55d1b7ee8c90 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fd2d0640 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ea4280 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f10e80143d0 tx=0x7f10e8014890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c2c6beec03f6d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1296719952 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8004440 con 0x55d1b7ee8c90 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1296719952 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e801a030 con 0x55d1b7ee8c90 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 1 -- 172.21.15.31:0/1296719952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1b7cb1470 con 0x55d1b7ee8c90 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1296719952 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10e8003620 con 0x55d1b7ee8c90 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 10 client.7731 fetch_fsmap learned FSMap version 72 2023-11-26T07:23:11.023 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 1 -- 172.21.15.31:0/1296719952 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55d1b7cb0190 con 0x55d1b7ee8c90 2023-11-26T07:23:11.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1296719952 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d1b7cb1470 con 0x55d1b7ee8c90 2023-11-26T07:23:11.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1296719952 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10e8003780 con 0x55d1b7ee8c90 2023-11-26T07:23:11.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 10 client.7731 fetch_fsmap finished waiting for FSMap version 72 2023-11-26T07:23:11.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 10 client.7731 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:23:11.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 20 client.7731 populate_metadata read hostname 'smithi031' 2023-11-26T07:23:11.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.31:6833/1001580514,v1:172.21.15.31:6835/1001580514] conn(0x55d1b7eb8320 0x55d1b7efaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 4 client.7731 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-26T07:23:11.025 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f1103093000 1 -- 172.21.15.31:0/1296719952 --> [v2:172.21.15.31:6833/1001580514,v1:172.21.15.31:6835/1001580514] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1b7cb4b50 con 0x55d1b7eb8320 2023-11-26T07:23:11.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.31:6833/1001580514,v1:172.21.15.31:6835/1001580514] conn(0x55d1b7eb8320 0x55d1b7efaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10d8ff9640 10 client.7731 ms_handle_connect on v2:172.21.15.31:6833/1001580514 2023-11-26T07:23:11.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.017+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.31:6833/1001580514,v1:172.21.15.31:6835/1001580514] conn(0x55d1b7eb8320 0x55d1b7efaf50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fa3de15a004ed79 server_cookie=4eb62bddf92d0a in_seq=0 out_seq=0 2023-11-26T07:23:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1296719952 <== mds.0 v2:172.21.15.31:6833/1001580514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d1b7cb4b50 con 0x55d1b7eb8320 2023-11-26T07:23:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f10d8ff9640 10 client.7731 handle_command_reply: tid=0 2023-11-26T07:23:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 client.7731 shutdown 2023-11-26T07:23:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 -- 172.21.15.31:0/1296719952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 msgr2=0x55d1b7ea4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ea4280 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f10e80143d0 tx=0x7f10e8014890 comp rx=0 tx=0).stop 2023-11-26T07:23:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 -- 172.21.15.31:0/1296719952 shutdown_connections 2023-11-26T07:23:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.31:6833/1001580514,v1:172.21.15.31:6835/1001580514] conn(0x55d1b7eb8320 0x55d1b7efaf50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7ee95b0 0x55d1b7eb7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ea4280 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 --2- 172.21.15.31:0/1296719952 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7e9f320 0x55d1b7ea3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f10d8ff9640 0 client.7731 ms_handle_reset on v2:172.21.15.31:6833/1001580514 2023-11-26T07:23:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 -- 172.21.15.31:0/1296719952 >> 172.21.15.31:0/1296719952 conn(0x55d1b7e9b2d0 msgr2=0x55d1b7e9d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 -- 172.21.15.31:0/1296719952 shutdown_connections 2023-11-26T07:23:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 1 -- 172.21.15.31:0/1296719952 wait complete. 2023-11-26T07:23:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.021+0000 7f1103093000 20 client.7731 trim_cache size 0 max 16384 2023-11-26T07:23:11.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 Processor -- start 2023-11-26T07:23:11.039 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 -- start start 2023-11-26T07:23:11.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7e9f320 0x55d1b7ee7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ee6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7ee7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1b79a8940 con 0x55d1b7e9f320 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7cee4e0 con 0x55d1b7ee8c90 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7cb96c0 con 0x55d1b7ee95b0 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10fdad1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7e9f320 0x55d1b7ee7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10fdad1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7e9f320 0x55d1b7ee7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36370/0 (socket says 172.21.15.31:36370) 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10fdad1640 1 -- 172.21.15.31:0/1848708670 learned_addr learned my addr 172.21.15.31:0/1848708670 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10fdad1640 1 -- 172.21.15.31:0/1848708670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 msgr2=0x55d1b7ee7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1848708670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7ee7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10fdad1640 1 -- 172.21.15.31:0/1848708670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7ee8c90 msgr2=0x55d1b7ee6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1848708670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ee6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10fdad1640 1 -- 172.21.15.31:0/1848708670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1b7c96490 con 0x55d1b7e9f320 2023-11-26T07:23:11.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1848708670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7e9f320 0x55d1b7ee7c90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x55d1b7da5e70 tx=0x7f10f4017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f027c5cdc5bb6006 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10dbfff640 1 -- 172.21.15.31:0/1848708670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4007710 con 0x55d1b7e9f320 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10dbfff640 1 -- 172.21.15.31:0/1848708670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4019040 con 0x55d1b7e9f320 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f10dbfff640 1 -- 172.21.15.31:0/1848708670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f400ee00 con 0x55d1b7e9f320 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 -- 172.21.15.31:0/1848708670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7e9f320 msgr2=0x55d1b7ee7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 --2- 172.21.15.31:0/1848708670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7e9f320 0x55d1b7ee7c90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x55d1b7da5e70 tx=0x7f10f4017070 comp rx=0 tx=0).stop 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 -- 172.21.15.31:0/1848708670 shutdown_connections 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 --2- 172.21.15.31:0/1848708670 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7ee7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 --2- 172.21.15.31:0/1848708670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7ee8c90 0x55d1b7ee6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 --2- 172.21.15.31:0/1848708670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7e9f320 0x55d1b7ee7c90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 -- 172.21.15.31:0/1848708670 >> 172.21.15.31:0/1848708670 conn(0x55d1b7e9b2d0 msgr2=0x55d1b7ef6280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 -- 172.21.15.31:0/1848708670 shutdown_connections 2023-11-26T07:23:11.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.037+0000 7f1103093000 1 -- 172.21.15.31:0/1848708670 wait complete. 2023-11-26T07:23:11.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 -- start start 2023-11-26T07:23:11.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 Processor -- start 2023-11-26T07:23:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7e9f320 0x55d1b7ea0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7efbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7efc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7ca6a10 con 0x55d1b7ee8c90 2023-11-26T07:23:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7cf0070 con 0x55d1b7e9f320 2023-11-26T07:23:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1b7cc8e10 con 0x55d1b7ee95b0 2023-11-26T07:23:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 10 client.0 _ll_register_callbacks cb 0x55d1b7d3dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-26T07:23:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fd2d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7efbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fd2d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7efbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36376/0 (socket says 172.21.15.31:36376) 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fd2d0640 1 -- 172.21.15.31:0/1993815767 learned_addr learned my addr 172.21.15.31:0/1993815767 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fe2d2640 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7efc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7e9f320 0x55d1b7ea0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fd2d0640 1 -- 172.21.15.31:0/1993815767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 msgr2=0x55d1b7efc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fd2d0640 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7efc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fd2d0640 1 -- 172.21.15.31:0/1993815767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7e9f320 msgr2=0x55d1b7ea0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fd2d0640 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7e9f320 0x55d1b7ea0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fd2d0640 1 -- 172.21.15.31:0/1993815767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1b79e7890 con 0x55d1b7ee8c90 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fd2d0640 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7efbfa0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f10e8004000 tx=0x7f10e8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6360988528b5cc server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.31:3300/0 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1993815767 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8003620 con 0x55d1b7ee8c90 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1993815767 <== mon.0 v2:172.21.15.31:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e801a070 con 0x55d1b7ee8c90 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 -- 172.21.15.31:0/1993815767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1b7cee110 con 0x55d1b7ee8c90 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1993815767 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10e8003780 con 0x55d1b7ee8c90 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 10 client.7737 fetch_fsmap learned FSMap version 72 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 -- 172.21.15.31:0/1993815767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55d1b79e7cc0 con 0x55d1b7ee8c90 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1993815767 <== mon.0 v2:172.21.15.31:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d1b7cee110 con 0x55d1b7ee8c90 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1993815767 <== mon.0 v2:172.21.15.31:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f10e8004240 con 0x55d1b7ee8c90 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 10 client.7737 fetch_fsmap finished waiting for FSMap version 72 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 10 client.7737 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 20 client.7737 populate_metadata read hostname 'smithi031' 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.31:6833/1001580514,v1:172.21.15.31:6835/1001580514] conn(0x55d1b7f49010 0x55d1b7f493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 4 client.7737 mds_command: new command op to 7575 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f1103093000 1 -- 172.21.15.31:0/1993815767 --> [v2:172.21.15.31:6833/1001580514,v1:172.21.15.31:6835/1001580514] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d1b7d1d9a0 con 0x55d1b7f49010 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.041+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.31:6833/1001580514,v1:172.21.15.31:6835/1001580514] conn(0x55d1b7f49010 0x55d1b7f493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10fdad1640 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.31:6833/1001580514,v1:172.21.15.31:6835/1001580514] conn(0x55d1b7f49010 0x55d1b7f493f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a47bbcd8dd83793 server_cookie=decd74d88a3246c2 in_seq=0 out_seq=0 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10d8ff9640 10 client.7737 ms_handle_connect on v2:172.21.15.31:6833/1001580514 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10d8ff9640 1 -- 172.21.15.31:0/1993815767 <== mds.0 v2:172.21.15.31:6833/1001580514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55d1b7d1d9a0 con 0x55d1b7f49010 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10d8ff9640 10 client.7737 handle_command_reply: tid=0 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 client.7737 shutdown 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 -- 172.21.15.31:0/1993815767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 msgr2=0x55d1b7efbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7efbfa0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f10e8004000 tx=0x7f10e8004740 comp rx=0 tx=0).stop 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 -- 172.21.15.31:0/1993815767 shutdown_connections 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.31:6833/1001580514,v1:172.21.15.31:6835/1001580514] conn(0x55d1b7f49010 0x55d1b7f493f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x55d1b7ee95b0 0x55d1b7efc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55d1b7ee8c90 0x55d1b7efbfa0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 --2- 172.21.15.31:0/1993815767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d1b7e9f320 0x55d1b7ea0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10d8ff9640 0 client.7737 ms_handle_reset on v2:172.21.15.31:6833/1001580514 2023-11-26T07:23:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 -- 172.21.15.31:0/1993815767 >> 172.21.15.31:0/1993815767 conn(0x55d1b7e9b2d0 msgr2=0x55d1b7e9d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 -- 172.21.15.31:0/1993815767 shutdown_connections 2023-11-26T07:23:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 1 -- 172.21.15.31:0/1993815767 wait complete. 2023-11-26T07:23:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f1103093000 20 client.7737 trim_cache size 0 max 16384 2023-11-26T07:23:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:11.051 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_fsid": "4c2b4df1-71b1-4841-8807-af144d2610e7", 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "whoami": 0, 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 7575, 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "want_state": "up:active", 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "up:active", 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs", 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "rank_uptime": 5.9150294260000003, 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "mdsmap_epoch": 72, 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch": 99, 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "osdmap_epoch_barrier": 99, 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout: "uptime": 11.17048404 2023-11-26T07:23:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:11.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 -- 172.21.15.31:0/2586364595 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f10dc066590 msgr2=0x7f10dc068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f10dc066590 0x7f10dc068a50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f10f4000fc0 tx=0x7f10f4009040 comp rx=0 tx=0).stop 2023-11-26T07:23:11.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 -- 172.21.15.31:0/2586364595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80ce5d0 msgr2=0x7f10f8154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80ce5d0 0x7f10f8154f10 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009d90 tx=0x7f10f0019610 comp rx=0 tx=0).stop 2023-11-26T07:23:11.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 -- 172.21.15.31:0/2586364595 shutdown_connections 2023-11-26T07:23:11.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f10dc066590 0x7f10dc068a50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f80ce5d0 0x7f10f8154f10 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f10f80cb9b0 0x7f10f8158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 --2- 172.21.15.31:0/2586364595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f806e3f0 0x7f10f8153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 -- 172.21.15.31:0/2586364595 >> 172.21.15.31:0/2586364595 conn(0x7f10f80b0cb0 msgr2=0x7f10f815d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 -- 172.21.15.31:0/2586364595 shutdown_connections 2023-11-26T07:23:11.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.045+0000 7f10ffd35640 1 -- 172.21.15.31:0/2586364595 wait complete. 2023-11-26T07:23:11.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:23:11.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb125232640 1 Processor -- start 2023-11-26T07:23:11.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb125232640 1 -- start start 2023-11-26T07:23:11.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb125232640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb120143ae0 0x7fb1200b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb125232640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12006dfd0 0x7fb12006e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb125232640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb1200adcc0 0x7fb1200ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb125232640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1200b2480 con 0x7fb120143ae0 2023-11-26T07:23:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb125232640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12006e950 con 0x7fb1200adcc0 2023-11-26T07:23:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb125232640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb120070670 con 0x7fb12006dfd0 2023-11-26T07:23:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb11ed9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb120143ae0 0x7fb1200b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb11ed9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb120143ae0 0x7fb1200b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36388/0 (socket says 172.21.15.31:36388) 2023-11-26T07:23:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb11ed9d640 1 -- 172.21.15.31:0/3000077808 learned_addr learned my addr 172.21.15.31:0/3000077808 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb11f59e640 1 --2- 172.21.15.31:0/3000077808 >> v2:172.21.15.97:3300/0 conn(0x7fb1200adcc0 0x7fb1200ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb11e59c640 1 --2- 172.21.15.31:0/3000077808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12006dfd0 0x7fb12006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb11ed9d640 1 -- 172.21.15.31:0/3000077808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12006dfd0 msgr2=0x7fb12006e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb11ed9d640 1 --2- 172.21.15.31:0/3000077808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12006dfd0 0x7fb12006e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb11ed9d640 1 -- 172.21.15.31:0/3000077808 >> v2:172.21.15.97:3300/0 conn(0x7fb1200adcc0 msgr2=0x7fb1200ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:11.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb11ed9d640 1 --2- 172.21.15.31:0/3000077808 >> v2:172.21.15.97:3300/0 conn(0x7fb1200adcc0 0x7fb1200ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.214 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.209+0000 7fb11ed9d640 1 -- 172.21.15.31:0/3000077808 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb120070950 con 0x7fb120143ae0 2023-11-26T07:23:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb11ed9d640 1 --2- 172.21.15.31:0/3000077808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb120143ae0 0x7fb1200b1f40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb12006c950 tx=0x7fb11001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43d73f2ec1c57d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb11dd9b640 1 -- 172.21.15.31:0/3000077808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100aa5f0 con 0x7fb120143ae0 2023-11-26T07:23:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb11dd9b640 1 -- 172.21.15.31:0/3000077808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1100aa750 con 0x7fb120143ae0 2023-11-26T07:23:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb11dd9b640 1 -- 172.21.15.31:0/3000077808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100aaae0 con 0x7fb120143ae0 2023-11-26T07:23:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb125232640 1 -- 172.21.15.31:0/3000077808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb120143ae0 msgr2=0x7fb1200b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb125232640 1 --2- 172.21.15.31:0/3000077808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb120143ae0 0x7fb1200b1f40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb12006c950 tx=0x7fb11001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb125232640 1 -- 172.21.15.31:0/3000077808 shutdown_connections 2023-11-26T07:23:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb125232640 1 --2- 172.21.15.31:0/3000077808 >> v2:172.21.15.97:3300/0 conn(0x7fb1200adcc0 0x7fb1200ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb125232640 1 --2- 172.21.15.31:0/3000077808 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb12006dfd0 0x7fb12006e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb125232640 1 --2- 172.21.15.31:0/3000077808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb120143ae0 0x7fb1200b1f40 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb125232640 1 -- 172.21.15.31:0/3000077808 >> 172.21.15.31:0/3000077808 conn(0x7fb1200b0c40 msgr2=0x7fb1200b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb125232640 1 -- 172.21.15.31:0/3000077808 shutdown_connections 2023-11-26T07:23:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.213+0000 7fb125232640 1 -- 172.21.15.31:0/3000077808 wait complete. 2023-11-26T07:23:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb125232640 1 Processor -- start 2023-11-26T07:23:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb125232640 1 -- start start 2023-11-26T07:23:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb125232640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12006dfd0 0x7fb120159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb125232640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1200adcc0 0x7fb12009be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb125232640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb120143ae0 0x7fb1200b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb125232640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12013d170 con 0x7fb12006dfd0 2023-11-26T07:23:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb125232640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1201598a0 con 0x7fb1200adcc0 2023-11-26T07:23:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb125232640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb120138510 con 0x7fb120143ae0 2023-11-26T07:23:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12006dfd0 0x7fb120159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12006dfd0 0x7fb120159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36392/0 (socket says 172.21.15.31:36392) 2023-11-26T07:23:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11ed9d640 1 -- 172.21.15.31:0/312533020 learned_addr learned my addr 172.21.15.31:0/312533020 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11e59c640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1200adcc0 0x7fb12009be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11f59e640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb120143ae0 0x7fb1200b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11ed9d640 1 -- 172.21.15.31:0/312533020 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb120143ae0 msgr2=0x7fb1200b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11ed9d640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb120143ae0 0x7fb1200b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11ed9d640 1 -- 172.21.15.31:0/312533020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1200adcc0 msgr2=0x7fb12009be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11ed9d640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1200adcc0 0x7fb12009be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11ed9d640 1 -- 172.21.15.31:0/312533020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1100094d0 con 0x7fb12006dfd0 2023-11-26T07:23:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb11ed9d640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12006dfd0 0x7fb120159360 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb11001ab80 tx=0x7fb11001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb1077fe640 1 -- 172.21.15.31:0/312533020 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100bd5f0 con 0x7fb12006dfd0 2023-11-26T07:23:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb1077fe640 1 -- 172.21.15.31:0/312533020 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1100bd750 con 0x7fb12006dfd0 2023-11-26T07:23:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb1077fe640 1 -- 172.21.15.31:0/312533020 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100bdae0 con 0x7fb12006dfd0 2023-11-26T07:23:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb125232640 1 -- 172.21.15.31:0/312533020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12009c7f0 con 0x7fb12006dfd0 2023-11-26T07:23:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.217+0000 7fb125232640 1 -- 172.21.15.31:0/312533020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12013d6a0 con 0x7fb12006dfd0 2023-11-26T07:23:11.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.221+0000 7fb125232640 1 -- 172.21.15.31:0/312533020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1200ace40 con 0x7fb12006dfd0 2023-11-26T07:23:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.221+0000 7fb1077fe640 1 -- 172.21.15.31:0/312533020 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fb1100b33f0 con 0x7fb12006dfd0 2023-11-26T07:23:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.221+0000 7fb1077fe640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb10c0665c0 0x7fb10c068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.221+0000 7fb1077fe640 1 -- 172.21.15.31:0/312533020 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1100a7070 con 0x7fb12006dfd0 2023-11-26T07:23:11.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.221+0000 7fb11e59c640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb10c0665c0 0x7fb10c068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:11.225 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.221+0000 7fb11e59c640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb10c0665c0 0x7fb10c068a80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb1201f9720 tx=0x7fb114009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:11.228 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.225+0000 7fb1077fe640 1 -- 172.21.15.31:0/312533020 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb1100ec050 con 0x7fb12006dfd0 2023-11-26T07:23:11.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.417+0000 7fb125232640 1 -- 172.21.15.31:0/312533020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb12004e850 con 0x7fb12006dfd0 2023-11-26T07:23:11.421 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.417+0000 7fb1077fe640 1 -- 172.21.15.31:0/312533020 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb1100ef670 con 0x7fb12006dfd0 2023-11-26T07:23:11.421 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:23:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 -- 172.21.15.31:0/312533020 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb10c0665c0 msgr2=0x7fb10c068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb10c0665c0 0x7fb10c068a80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb1201f9720 tx=0x7fb114009190 comp rx=0 tx=0).stop 2023-11-26T07:23:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 -- 172.21.15.31:0/312533020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12006dfd0 msgr2=0x7fb120159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:11.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12006dfd0 0x7fb120159360 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb11001ab80 tx=0x7fb11001ad10 comp rx=0 tx=0).stop 2023-11-26T07:23:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 -- 172.21.15.31:0/312533020 shutdown_connections 2023-11-26T07:23:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb10c0665c0 0x7fb10c068a80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb120143ae0 0x7fb1200b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1200adcc0 0x7fb12009be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 --2- 172.21.15.31:0/312533020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb12006dfd0 0x7fb120159360 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 -- 172.21.15.31:0/312533020 >> 172.21.15.31:0/312533020 conn(0x7fb1200b0c40 msgr2=0x7fb1200b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 -- 172.21.15.31:0/312533020 shutdown_connections 2023-11-26T07:23:11.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:11.421+0000 7fb125232640 1 -- 172.21.15.31:0/312533020 wait complete. 2023-11-26T07:23:11.442 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:23:11.442 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:23:11.442 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-11-26T07:23:11.442 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-26T07:23:11.442 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:11.442 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:23:11.442 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:23:11.443 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:23:11.443 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:23:11.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:11.443 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-11-26T07:23:11.453 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:23:11.453 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:23:11.453 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:23:11.453 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:11.453 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-11-26T07:23:11.454 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42048sec preferred_lft 42048sec 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 02:42:1b:e4:a9:1e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:11.455 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:23:11.456 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:11.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:11.457 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:23:11.520 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-11-26T07:23:11.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:11.522 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-11-26T07:23:11.579 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-11-26T07:23:11.579 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:23:11.579 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-11-26T07:23:11.580 INFO:teuthology.orchestra.run.smithi031.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:23:11.580 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-11-26T07:23:11.580 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:23:11.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:11.580 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-11-26T07:23:11.580 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-11-26T07:23:11.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-26T07:23:11.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-26T07:23:11.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-26T07:23:11.581 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-11-26T07:23:11.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:11.677 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:23:11.687 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:23:11.687 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:23:11.688 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:23:11.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:11.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-11-26T07:23:11.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:23:11.750 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:11.801 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-26T07:23:11.802 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:23:11.802 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:11.853 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-26T07:23:11.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:11.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:23:11.945 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:23:11.945 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:23:11.960 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:23:11.960 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:23:11.960 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:23:11.960 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with new device syntax: 0@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:23:11.960 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:23:11.964 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:23:11.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:11.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:11.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:11.986 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:23:12.047 INFO:teuthology.orchestra.run.smithi031.stdout:client6542 2023-11-26T07:23:12.047 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:12.047 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:12.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-26T07:23:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6af2501640 1 Processor -- start 2023-11-26T07:23:12.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6af2501640 1 -- start start 2023-11-26T07:23:12.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6af2501640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6aec15d7b0 0x7f6aec0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6af2501640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec0d2080 0x7f6aec06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6af2501640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec0cefd0 con 0x7f6aec0cead0 2023-11-26T07:23:12.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6af2501640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec0cf2b0 con 0x7f6aec15d7b0 2023-11-26T07:23:12.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6af2501640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec06b3f0 con 0x7f6aec0d2080 2023-11-26T07:23:12.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6aebfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6aec15d7b0 0x7f6aec0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6aebfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6aec15d7b0 0x7f6aec0d10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36398/0 (socket says 172.21.15.31:36398) 2023-11-26T07:23:12.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6aebfff640 1 -- 172.21.15.31:0/1974628487 learned_addr learned my addr 172.21.15.31:0/1974628487 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:12.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6aeb7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6aec0cead0 0x7f6aec0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:60628/0 (socket says 172.21.15.31:60628) 2023-11-26T07:23:12.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6af0a9e640 1 --2- 172.21.15.31:0/1974628487 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec0d2080 0x7f6aec06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6aebfff640 1 -- 172.21.15.31:0/1974628487 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec0d2080 msgr2=0x7f6aec06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:12.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6aebfff640 1 --2- 172.21.15.31:0/1974628487 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec0d2080 0x7f6aec06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6aebfff640 1 -- 172.21.15.31:0/1974628487 >> v1:172.21.15.31:6789/0 conn(0x7f6aec0cead0 legacy=0x7f6aec0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6aebfff640 1 -- 172.21.15.31:0/1974628487 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec06b6d0 con 0x7f6aec15d7b0 2023-11-26T07:23:12.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6aeaffd640 1 -- 172.21.15.31:0/1974628487 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414716300 0 0) 0x7f6aec0cefd0 con 0x7f6aec0cead0 2023-11-26T07:23:12.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.289+0000 7f6af0a9e640 1 --2- 172.21.15.31:0/1974628487 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec0d2080 0x7f6aec06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:12.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6aebfff640 1 --2- 172.21.15.31:0/1974628487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15d7b0 0x7f6aec0d10d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009870 tx=0x7f6adc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e2921a1c42e41b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:12.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6aeaffd640 1 -- 172.21.15.31:0/1974628487 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0ab610 con 0x7f6aec15d7b0 2023-11-26T07:23:12.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6aeaffd640 1 -- 172.21.15.31:0/1974628487 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc0ab770 con 0x7f6aec15d7b0 2023-11-26T07:23:12.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6aeaffd640 1 -- 172.21.15.31:0/1974628487 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0abb00 con 0x7f6aec15d7b0 2023-11-26T07:23:12.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6af2501640 1 -- 172.21.15.31:0/1974628487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15d7b0 msgr2=0x7f6aec0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6af2501640 1 --2- 172.21.15.31:0/1974628487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15d7b0 0x7f6aec0d10d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009870 tx=0x7f6adc01c3f0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6af2501640 1 -- 172.21.15.31:0/1974628487 shutdown_connections 2023-11-26T07:23:12.296 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6af2501640 1 --2- 172.21.15.31:0/1974628487 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec0d2080 0x7f6aec06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6af2501640 1 --2- 172.21.15.31:0/1974628487 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec15d7b0 0x7f6aec0d10d0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6af2501640 1 -- 172.21.15.31:0/1974628487 >> 172.21.15.31:0/1974628487 conn(0x7f6aec0d33b0 msgr2=0x7f6aec0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6af2501640 1 -- 172.21.15.31:0/1974628487 shutdown_connections 2023-11-26T07:23:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6af2501640 1 -- 172.21.15.31:0/1974628487 wait complete. 2023-11-26T07:23:12.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.293+0000 7f6af2501640 1 Processor -- start 2023-11-26T07:23:12.298 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af2501640 1 -- start start 2023-11-26T07:23:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af2501640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0cead0 0x7f6aec0c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af2501640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0d2080 0x7f6aec0b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af2501640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec15d7b0 0x7f6aec0b8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af2501640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0b88c0 con 0x7f6aec0cead0 2023-11-26T07:23:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af2501640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0b8ba0 con 0x7f6aec0d2080 2023-11-26T07:23:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af2501640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec0b8e80 con 0x7f6aec15d7b0 2023-11-26T07:23:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6aebfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0cead0 0x7f6aec0c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6aebfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0cead0 0x7f6aec0c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36406/0 (socket says 172.21.15.31:36406) 2023-11-26T07:23:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6aebfff640 1 -- 172.21.15.31:0/1720233508 learned_addr learned my addr 172.21.15.31:0/1720233508 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:12.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6aeb7fe640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0d2080 0x7f6aec0b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af0a9e640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec15d7b0 0x7f6aec0b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6aebfff640 1 -- 172.21.15.31:0/1720233508 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec15d7b0 msgr2=0x7f6aec0b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6aebfff640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec15d7b0 0x7f6aec0b8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6aebfff640 1 -- 172.21.15.31:0/1720233508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0d2080 msgr2=0x7f6aec0b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6aebfff640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0d2080 0x7f6aec0b7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6aebfff640 1 -- 172.21.15.31:0/1720233508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc0094d0 con 0x7f6aec0cead0 2023-11-26T07:23:12.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af0a9e640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec15d7b0 0x7f6aec0b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:12.301 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6aebfff640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0cead0 0x7f6aec0c9af0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0ad040 tx=0x7f6adc01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:12.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6ae8ff9640 1 -- 172.21.15.31:0/1720233508 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0b4800 con 0x7f6aec0cead0 2023-11-26T07:23:12.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6ae8ff9640 1 -- 172.21.15.31:0/1720233508 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc0b4980 con 0x7f6aec0cead0 2023-11-26T07:23:12.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6ae8ff9640 1 -- 172.21.15.31:0/1720233508 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0b4d30 con 0x7f6aec0cead0 2023-11-26T07:23:12.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af2501640 1 -- 172.21.15.31:0/1720233508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec0b9160 con 0x7f6aec0cead0 2023-11-26T07:23:12.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af2501640 1 -- 172.21.15.31:0/1720233508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec1559e0 con 0x7f6aec0cead0 2023-11-26T07:23:12.302 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.297+0000 7f6af2501640 1 -- 172.21.15.31:0/1720233508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec04e850 con 0x7f6aec0cead0 2023-11-26T07:23:12.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.301+0000 7f6ae8ff9640 1 -- 172.21.15.31:0/1720233508 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f6adc0b3380 con 0x7f6aec0cead0 2023-11-26T07:23:12.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.301+0000 7f6ae8ff9640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6ad8066590 0x7f6ad8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:12.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.301+0000 7f6ae8ff9640 1 -- 172.21.15.31:0/1720233508 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6adc11e770 con 0x7f6aec0cead0 2023-11-26T07:23:12.303 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.301+0000 7f6aeb7fe640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6ad8066590 0x7f6ad8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:12.304 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.301+0000 7f6aeb7fe640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6ad8066590 0x7f6ad8068a50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6aec15a9b0 tx=0x7f6ae0007df0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:12.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.305+0000 7f6ae8ff9640 1 -- 172.21.15.31:0/1720233508 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6adc0eba70 con 0x7f6aec0cead0 2023-11-26T07:23:12.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.497+0000 7f6af2501640 1 -- 172.21.15.31:0/1720233508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6aec0b8a00 con 0x7f6aec0cead0 2023-11-26T07:23:12.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.497+0000 7f6ae8ff9640 1 -- 172.21.15.31:0/1720233508 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6aec0b8a00 con 0x7f6aec0cead0 2023-11-26T07:23:12.500 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-26T07:23:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 -- 172.21.15.31:0/1720233508 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6ad8066590 msgr2=0x7f6ad8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6ad8066590 0x7f6ad8068a50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6aec15a9b0 tx=0x7f6ae0007df0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 -- 172.21.15.31:0/1720233508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0cead0 msgr2=0x7f6aec0c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0cead0 0x7f6aec0c9af0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0ad040 tx=0x7f6adc01ac30 comp rx=0 tx=0).stop 2023-11-26T07:23:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 -- 172.21.15.31:0/1720233508 shutdown_connections 2023-11-26T07:23:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6ad8066590 0x7f6ad8068a50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f6aec15d7b0 0x7f6aec0b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6aec0d2080 0x7f6aec0b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 --2- 172.21.15.31:0/1720233508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6aec0cead0 0x7f6aec0c9af0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 -- 172.21.15.31:0/1720233508 >> 172.21.15.31:0/1720233508 conn(0x7f6aec0d33b0 msgr2=0x7f6aec0d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 -- 172.21.15.31:0/1720233508 shutdown_connections 2023-11-26T07:23:12.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:12.501+0000 7f6af2501640 1 -- 172.21.15.31:0/1720233508 wait complete. 2023-11-26T07:23:12.521 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-26T07:23:12.521 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-26T07:23:12.521 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2023-11-26T07:23:12.521 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-26T07:23:12.521 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:12.522 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-26T07:23:12.522 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-26T07:23:12.522 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-26T07:23:12.522 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:23:12.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.522 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2023-11-26T07:23:12.533 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-26T07:23:12.533 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-26T07:23:12.533 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2023-11-26T07:23:12.533 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:12.533 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2023-11-26T07:23:12.533 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:12.533 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:12.533 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:12.533 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42070sec preferred_lft 42070sec 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link 2023-11-26T07:23:12.534 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 02:42:61:bc:5b:3d brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:23:12.535 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-11-26T07:23:12.536 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:12.536 INFO:teuthology.orchestra.run.smithi097.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-26T07:23:12.536 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:23:12.536 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-11-26T07:23:12.536 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2023-11-26T07:23:12.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.537 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-26T07:23:12.601 INFO:teuthology.orchestra.run.smithi097.stdout:1 2023-11-26T07:23:12.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.604 DEBUG:teuthology.orchestra.run.smithi097:> route 2023-11-26T07:23:12.657 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2023-11-26T07:23:12.657 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-26T07:23:12.657 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-11-26T07:23:12.657 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-26T07:23:12.657 INFO:teuthology.orchestra.run.smithi097.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-11-26T07:23:12.657 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-26T07:23:12.657 INFO:teuthology.orchestra.run.smithi097.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:23:12.658 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:23:12.658 INFO:teuthology.orchestra.run.smithi097.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-11-26T07:23:12.658 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-26T07:23:12.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.658 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec bash -c ' 2023-11-26T07:23:12.658 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2023-11-26T07:23:12.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-26T07:23:12.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-26T07:23:12.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-26T07:23:12.658 DEBUG:teuthology.orchestra.run.smithi097:> ') 2023-11-26T07:23:12.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.756 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:23:12.766 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:23:12.766 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:23:12.767 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:23:12.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2023-11-26T07:23:12.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:23:12.830 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:12.881 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-26T07:23:12.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-26T07:23:12.881 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:12.932 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-26T07:23:12.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:12.933 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:23:13.028 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:23:13.028 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:23:13.042 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-26T07:23:13.042 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-26T07:23:13.042 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300,172.21.15.97:3301" 2023-11-26T07:23:13.042 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 1@4c2b4df1-71b1-4841-8807-af144d2610e7.cephfs=/ 2023-11-26T07:23:13.042 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.31:3300/172.21.15.97:3300/172.21.15.97:3301" will pass to kernel 2023-11-26T07:23:13.046 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-26T07:23:13.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:13.046 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:13.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:13.067 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-26T07:23:13.120 INFO:teuthology.orchestra.run.smithi097.stdout:client6551 2023-11-26T07:23:13.121 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:23:13.121 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:13.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-26T07:23:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.329+0000 7f66e8d0f640 1 Processor -- start 2023-11-26T07:23:13.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.329+0000 7f66e8d0f640 1 -- start start 2023-11-26T07:23:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66e8d0f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40cb9b0 0x7f66e406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66e8d0f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f66e40cbd90 0x7f66e415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66e8d0f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e4159630 con 0x7f66e40ce5d0 2023-11-26T07:23:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66e8d0f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e415bba0 con 0x7f66e40cbd90 2023-11-26T07:23:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66e8d0f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40cdd50 con 0x7f66e40cb9b0 2023-11-26T07:23:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66dbfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f66e40cbd90 0x7f66e415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66dbfff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f66e40cbd90 0x7f66e415f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36418/0 (socket says 172.21.15.31:36418) 2023-11-26T07:23:13.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66dbfff640 1 -- 172.21.15.31:0/2348715241 learned_addr learned my addr 172.21.15.31:0/2348715241 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66e259c640 1 --2- 172.21.15.31:0/2348715241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40cb9b0 0x7f66e406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66dbfff640 1 -- 172.21.15.31:0/2348715241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40cb9b0 msgr2=0x7f66e406e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66dbfff640 1 --2- 172.21.15.31:0/2348715241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40cb9b0 0x7f66e406e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66dbfff640 1 -- 172.21.15.31:0/2348715241 >> v1:172.21.15.97:6789/0 conn(0x7f66e40ce5d0 legacy=0x7f66e415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.333+0000 7f66dbfff640 1 -- 172.21.15.31:0/2348715241 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e415f6d0 con 0x7f66e40cbd90 2023-11-26T07:23:13.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66dbfff640 1 --2- 172.21.15.31:0/2348715241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 0x7f66e415f190 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009870 tx=0x7f66cc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf824cc6a8f72a38 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e1d9b640 1 -- 172.21.15.31:0/2348715241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0aa670 con 0x7f66e40cbd90 2023-11-26T07:23:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e1d9b640 1 -- 172.21.15.31:0/2348715241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc0aa7d0 con 0x7f66e40cbd90 2023-11-26T07:23:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e1d9b640 1 -- 172.21.15.31:0/2348715241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0aab60 con 0x7f66e40cbd90 2023-11-26T07:23:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 -- 172.21.15.31:0/2348715241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 msgr2=0x7f66e415f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 --2- 172.21.15.31:0/2348715241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 0x7f66e415f190 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009870 tx=0x7f66cc01c510 comp rx=0 tx=0).stop 2023-11-26T07:23:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 -- 172.21.15.31:0/2348715241 shutdown_connections 2023-11-26T07:23:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 --2- 172.21.15.31:0/2348715241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 0x7f66e415f190 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 --2- 172.21.15.31:0/2348715241 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40cb9b0 0x7f66e406e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 -- 172.21.15.31:0/2348715241 >> 172.21.15.31:0/2348715241 conn(0x7f66e40b0cb0 msgr2=0x7f66e40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:13.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 -- 172.21.15.31:0/2348715241 shutdown_connections 2023-11-26T07:23:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 -- 172.21.15.31:0/2348715241 wait complete. 2023-11-26T07:23:13.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 Processor -- start 2023-11-26T07:23:13.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 -- start start 2023-11-26T07:23:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66e40cb9b0 0x7f66e40b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.337+0000 7f66e8d0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 0x7f66e41567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e8d0f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40ce5d0 0x7f66e40b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e8d0f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4164db0 con 0x7f66e40cbd90 2023-11-26T07:23:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e8d0f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40b3610 con 0x7f66e40cb9b0 2023-11-26T07:23:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e8d0f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e41646f0 con 0x7f66e40ce5d0 2023-11-26T07:23:13.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 0x7f66e41567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 0x7f66e41567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36430/0 (socket says 172.21.15.31:36430) 2023-11-26T07:23:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66dbfff640 1 -- 172.21.15.31:0/281219127 learned_addr learned my addr 172.21.15.31:0/281219127 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e259c640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66e40cb9b0 0x7f66e40b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e2d9d640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40ce5d0 0x7f66e40b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66dbfff640 1 -- 172.21.15.31:0/281219127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40ce5d0 msgr2=0x7f66e40b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66dbfff640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40ce5d0 0x7f66e40b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66dbfff640 1 -- 172.21.15.31:0/281219127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66e40cb9b0 msgr2=0x7f66e40b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66dbfff640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66e40cb9b0 0x7f66e40b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66dbfff640 1 -- 172.21.15.31:0/281219127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc0094d0 con 0x7f66e40cbd90 2023-11-26T07:23:13.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e259c640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66e40cb9b0 0x7f66e40b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e2d9d640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40ce5d0 0x7f66e40b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66dbfff640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 0x7f66e41567e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f66cc01a140 tx=0x7f66cc01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66daffd640 1 -- 172.21.15.31:0/281219127 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0b3800 con 0x7f66e40cbd90 2023-11-26T07:23:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66daffd640 1 -- 172.21.15.31:0/281219127 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66cc0b3960 con 0x7f66e40cbd90 2023-11-26T07:23:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66daffd640 1 -- 172.21.15.31:0/281219127 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66cc0b3d10 con 0x7f66e40cbd90 2023-11-26T07:23:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e8d0f640 1 -- 172.21.15.31:0/281219127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e41649d0 con 0x7f66e40cbd90 2023-11-26T07:23:13.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e8d0f640 1 -- 172.21.15.31:0/281219127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e40b0630 con 0x7f66e40cbd90 2023-11-26T07:23:13.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66e8d0f640 1 -- 172.21.15.31:0/281219127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e40671f0 con 0x7f66e40cbd90 2023-11-26T07:23:13.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.341+0000 7f66daffd640 1 -- 172.21.15.31:0/281219127 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f66cc0c6440 con 0x7f66e40cbd90 2023-11-26T07:23:13.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.345+0000 7f66daffd640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f66c40665c0 0x7f66c4068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:13.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.345+0000 7f66daffd640 1 -- 172.21.15.31:0/281219127 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66cc11eea0 con 0x7f66e40cbd90 2023-11-26T07:23:13.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.345+0000 7f66e259c640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f66c40665c0 0x7f66c4068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:13.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.345+0000 7f66e259c640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f66c40665c0 0x7f66c4068a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f66e41531f0 tx=0x7f66d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:13.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.345+0000 7f66daffd640 1 -- 172.21.15.31:0/281219127 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f66cc0ec170 con 0x7f66e40cbd90 2023-11-26T07:23:13.552 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:13.549+0000 7f66e8d0f640 1 -- 172.21.15.31:0/281219127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f66e4164550 con 0x7f66e40cbd90 2023-11-26T07:23:14.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.181+0000 7f66daffd640 1 -- 172.21.15.31:0/281219127 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 136+0+0 (secure 0 0 0) 0x7f66e4164550 con 0x7f66e40cbd90 2023-11-26T07:23:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 -- 172.21.15.31:0/281219127 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f66c40665c0 msgr2=0x7f66c4068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f66c40665c0 0x7f66c4068a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f66e41531f0 tx=0x7f66d4008040 comp rx=0 tx=0).stop 2023-11-26T07:23:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 -- 172.21.15.31:0/281219127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 msgr2=0x7f66e41567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 0x7f66e41567e0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f66cc01a140 tx=0x7f66cc01ad50 comp rx=0 tx=0).stop 2023-11-26T07:23:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 -- 172.21.15.31:0/281219127 shutdown_connections 2023-11-26T07:23:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f66c40665c0 0x7f66c4068a80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f66e40ce5d0 0x7f66e40b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66e40cbd90 0x7f66e41567e0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 --2- 172.21.15.31:0/281219127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66e40cb9b0 0x7f66e40b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 -- 172.21.15.31:0/281219127 >> 172.21.15.31:0/281219127 conn(0x7f66e40b0cb0 msgr2=0x7f66e406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 -- 172.21.15.31:0/281219127 shutdown_connections 2023-11-26T07:23:14.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.185+0000 7f66e8d0f640 1 -- 172.21.15.31:0/281219127 wait complete. 2023-11-26T07:23:14.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-26T07:23:14.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.357+0000 7fa805a68640 1 Processor -- start 2023-11-26T07:23:14.361 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.357+0000 7fa805a68640 1 -- start start 2023-11-26T07:23:14.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa805a68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa800154940 0x7fa80006da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa805a68640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa80006df50 0x7fa80015df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa805a68640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8000cd870 con 0x7fa80015e4d0 2023-11-26T07:23:14.363 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa805a68640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa800154d20 con 0x7fa80006df50 2023-11-26T07:23:14.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa805a68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80015ecc0 con 0x7fa800154940 2023-11-26T07:23:14.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa7feffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa80006df50 0x7fa80015df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa7feffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa80006df50 0x7fa80015df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34990/0 (socket says 172.21.15.31:34990) 2023-11-26T07:23:14.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa7feffd640 1 -- 172.21.15.31:0/3404767375 learned_addr learned my addr 172.21.15.31:0/3404767375 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:14.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa7ff7fe640 1 --2- 172.21.15.31:0/3404767375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa800154940 0x7fa80006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa7feffd640 1 -- 172.21.15.31:0/3404767375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa800154940 msgr2=0x7fa80006da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:14.364 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa7feffd640 1 --2- 172.21.15.31:0/3404767375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa800154940 0x7fa80006da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa7feffd640 1 -- 172.21.15.31:0/3404767375 >> v1:172.21.15.97:6789/0 conn(0x7fa80015e4d0 legacy=0x7fa8000ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.365 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.361+0000 7fa7feffd640 1 -- 172.21.15.31:0/3404767375 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8000ccf40 con 0x7fa80006df50 2023-11-26T07:23:14.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa7feffd640 1 --2- 172.21.15.31:0/3404767375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 0x7fa80015df90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009870 tx=0x7fa7f401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe2b279d8dfcd77 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa7fe7fc640 1 -- 172.21.15.31:0/3404767375 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40aa670 con 0x7fa80006df50 2023-11-26T07:23:14.366 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa7fe7fc640 1 -- 172.21.15.31:0/3404767375 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f40aa7d0 con 0x7fa80006df50 2023-11-26T07:23:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa7fe7fc640 1 -- 172.21.15.31:0/3404767375 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40aab60 con 0x7fa80006df50 2023-11-26T07:23:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa805a68640 1 -- 172.21.15.31:0/3404767375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 msgr2=0x7fa80015df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.367 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa805a68640 1 --2- 172.21.15.31:0/3404767375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 0x7fa80015df90 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009870 tx=0x7fa7f401c510 comp rx=0 tx=0).stop 2023-11-26T07:23:14.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa805a68640 1 -- 172.21.15.31:0/3404767375 shutdown_connections 2023-11-26T07:23:14.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa805a68640 1 --2- 172.21.15.31:0/3404767375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 0x7fa80015df90 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa805a68640 1 --2- 172.21.15.31:0/3404767375 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa800154940 0x7fa80006da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa805a68640 1 -- 172.21.15.31:0/3404767375 >> 172.21.15.31:0/3404767375 conn(0x7fa8000b0cb0 msgr2=0x7fa8000b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa805a68640 1 -- 172.21.15.31:0/3404767375 shutdown_connections 2023-11-26T07:23:14.368 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa805a68640 1 -- 172.21.15.31:0/3404767375 wait complete. 2023-11-26T07:23:14.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa805a68640 1 Processor -- start 2023-11-26T07:23:14.369 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.365+0000 7fa805a68640 1 -- start start 2023-11-26T07:23:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa805a68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 0x7fa800071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa805a68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa800154940 0x7fa800150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa805a68640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa80015e4d0 0x7fa800150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.371 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa805a68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa800071730 con 0x7fa80006df50 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa805a68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa800070b20 con 0x7fa800154940 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa805a68640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa800070e00 con 0x7fa80015e4d0 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 0x7fa800071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 0x7fa800071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35002/0 (socket says 172.21.15.31:35002) 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7ff7fe640 1 -- 172.21.15.31:0/636358170 learned_addr learned my addr 172.21.15.31:0/636358170 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7feffd640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa800154940 0x7fa800150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7ff7fe640 1 -- 172.21.15.31:0/636358170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa80015e4d0 msgr2=0x7fa800150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7ff7fe640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa80015e4d0 0x7fa800150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7ff7fe640 1 -- 172.21.15.31:0/636358170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa800154940 msgr2=0x7fa800150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7ff7fe640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa800154940 0x7fa800150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7ff7fe640 1 -- 172.21.15.31:0/636358170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f40094d0 con 0x7fa80006df50 2023-11-26T07:23:14.372 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7ff7fe640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 0x7fa800071bc0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000d800 tx=0x7fa7f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7e7fff640 1 -- 172.21.15.31:0/636358170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000b6c0 con 0x7fa80006df50 2023-11-26T07:23:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7e7fff640 1 -- 172.21.15.31:0/636358170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f000b860 con 0x7fa80006df50 2023-11-26T07:23:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa805a68640 1 -- 172.21.15.31:0/636358170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8000710e0 con 0x7fa80006df50 2023-11-26T07:23:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7e7fff640 1 -- 172.21.15.31:0/636358170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000bc10 con 0x7fa80006df50 2023-11-26T07:23:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa805a68640 1 -- 172.21.15.31:0/636358170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa800070690 con 0x7fa80006df50 2023-11-26T07:23:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa805a68640 1 -- 172.21.15.31:0/636358170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8000671f0 con 0x7fa80006df50 2023-11-26T07:23:14.373 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.369+0000 7fa7e7fff640 1 -- 172.21.15.31:0/636358170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fa7f001e460 con 0x7fa80006df50 2023-11-26T07:23:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.373+0000 7fa7e7fff640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7ec0665c0 0x7fa7ec068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.373+0000 7fa7e7fff640 1 -- 172.21.15.31:0/636358170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7f0084600 con 0x7fa80006df50 2023-11-26T07:23:14.374 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.373+0000 7fa7feffd640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7ec0665c0 0x7fa7ec068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.375 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.373+0000 7fa7feffd640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7ec0665c0 0x7fa7ec068a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40ae000 tx=0x7fa7f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.378 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.377+0000 7fa7e7fff640 1 -- 172.21.15.31:0/636358170 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7f0051960 con 0x7fa80006df50 2023-11-26T07:23:14.558 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 -- 172.21.15.31:0/636358170 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8000cdce0 con 0x7fa7ec0665c0 2023-11-26T07:23:14.559 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa7e7fff640 1 -- 172.21.15.31:0/636358170 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa7f40ba080 con 0x7fa7ec0665c0 2023-11-26T07:23:14.560 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:23:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "cephfs" 2023-11-26T07:23:14.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:14.560 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:23:14.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 -- 172.21.15.31:0/636358170 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7ec0665c0 msgr2=0x7fa7ec068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7ec0665c0 0x7fa7ec068a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40ae000 tx=0x7fa7f4008040 comp rx=0 tx=0).stop 2023-11-26T07:23:14.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 -- 172.21.15.31:0/636358170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 msgr2=0x7fa800071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 0x7fa800071bc0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000d800 tx=0x7fa7f000dcc0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 -- 172.21.15.31:0/636358170 shutdown_connections 2023-11-26T07:23:14.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7ec0665c0 0x7fa7ec068a80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.562 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa80015e4d0 0x7fa800150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa800154940 0x7fa800150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 --2- 172.21.15.31:0/636358170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa80006df50 0x7fa800071bc0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 -- 172.21.15.31:0/636358170 >> 172.21.15.31:0/636358170 conn(0x7fa8000b0cb0 msgr2=0x7fa8000aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 -- 172.21.15.31:0/636358170 shutdown_connections 2023-11-26T07:23:14.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.557+0000 7fa805a68640 1 -- 172.21.15.31:0/636358170 wait complete. 2023-11-26T07:23:14.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-26T07:23:14.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3ed9c1640 1 Processor -- start 2023-11-26T07:23:14.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3ed9c1640 1 -- start start 2023-11-26T07:23:14.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3ed9c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e815d8f0 0x7ff3e815e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3ed9c1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3e80ced30 0x7ff3e80cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3ed9c1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff3e80cf1b0 0x7ff3e80d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3ed9c1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e80d4420 con 0x7ff3e80ced30 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3ed9c1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e815ecc0 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3ed9c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80cdfb0 con 0x7ff3e815d8f0 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3dffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3e80ced30 0x7ff3e80cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3dffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3e80ced30 0x7ff3e80cfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35014/0 (socket says 172.21.15.31:35014) 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3dffff640 1 -- 172.21.15.31:0/1063771204 learned_addr learned my addr 172.21.15.31:0/1063771204 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3e77fe640 1 --2- 172.21.15.31:0/1063771204 >> v2:172.21.15.97:3300/0 conn(0x7ff3e80cf1b0 0x7ff3e80d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3e6ffd640 1 --2- 172.21.15.31:0/1063771204 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e815d8f0 0x7ff3e815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3dffff640 1 -- 172.21.15.31:0/1063771204 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e815d8f0 msgr2=0x7ff3e815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3dffff640 1 --2- 172.21.15.31:0/1063771204 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e815d8f0 0x7ff3e815e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3dffff640 1 -- 172.21.15.31:0/1063771204 >> v2:172.21.15.97:3300/0 conn(0x7ff3e80cf1b0 msgr2=0x7ff3e80d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:14.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3dffff640 1 --2- 172.21.15.31:0/1063771204 >> v2:172.21.15.97:3300/0 conn(0x7ff3e80cf1b0 0x7ff3e80d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3dffff640 1 -- 172.21.15.31:0/1063771204 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e80d4140 con 0x7ff3e80ced30 2023-11-26T07:23:14.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.725+0000 7ff3e6ffd640 1 --2- 172.21.15.31:0/1063771204 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e815d8f0 0x7ff3e815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3dffff640 1 --2- 172.21.15.31:0/1063771204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80ced30 0x7ff3e80cfe00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80b21f0 tx=0x7ff3d001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62f8a497d85e7c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3e67fc640 1 -- 172.21.15.31:0/1063771204 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00aa5f0 con 0x7ff3e80ced30 2023-11-26T07:23:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3e67fc640 1 -- 172.21.15.31:0/1063771204 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d00aa750 con 0x7ff3e80ced30 2023-11-26T07:23:14.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3e67fc640 1 -- 172.21.15.31:0/1063771204 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00aaae0 con 0x7ff3e80ced30 2023-11-26T07:23:14.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1063771204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80ced30 msgr2=0x7ff3e80cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3ed9c1640 1 --2- 172.21.15.31:0/1063771204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80ced30 0x7ff3e80cfe00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80b21f0 tx=0x7ff3d001c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1063771204 shutdown_connections 2023-11-26T07:23:14.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3ed9c1640 1 --2- 172.21.15.31:0/1063771204 >> v2:172.21.15.97:3300/0 conn(0x7ff3e80cf1b0 0x7ff3e80d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3ed9c1640 1 --2- 172.21.15.31:0/1063771204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80ced30 0x7ff3e80cfe00 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3ed9c1640 1 --2- 172.21.15.31:0/1063771204 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e815d8f0 0x7ff3e815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1063771204 >> 172.21.15.31:0/1063771204 conn(0x7ff3e809c750 msgr2=0x7ff3e80d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1063771204 shutdown_connections 2023-11-26T07:23:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.729+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1063771204 wait complete. 2023-11-26T07:23:14.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3ed9c1640 1 Processor -- start 2023-11-26T07:23:14.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3ed9c1640 1 -- start start 2023-11-26T07:23:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3ed9c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e80ced30 0x7ff3e80cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3ed9c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80cf1b0 0x7ff3e80ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3ed9c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e815d8f0 0x7ff3e8153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3ed9c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80caa40 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3ed9c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8152b80 con 0x7ff3e815d8f0 2023-11-26T07:23:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3ed9c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80cace0 con 0x7ff3e80ced30 2023-11-26T07:23:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3dffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80cf1b0 0x7ff3e80ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3dffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80cf1b0 0x7ff3e80ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35026/0 (socket says 172.21.15.31:35026) 2023-11-26T07:23:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3dffff640 1 -- 172.21.15.31:0/1433924255 learned_addr learned my addr 172.21.15.31:0/1433924255 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:14.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3e6ffd640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e80ced30 0x7ff3e80cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3e77fe640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e815d8f0 0x7ff3e8153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3dffff640 1 -- 172.21.15.31:0/1433924255 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e80ced30 msgr2=0x7ff3e80cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3dffff640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e80ced30 0x7ff3e80cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3dffff640 1 -- 172.21.15.31:0/1433924255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e815d8f0 msgr2=0x7ff3e8153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3dffff640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e815d8f0 0x7ff3e8153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3dffff640 1 -- 172.21.15.31:0/1433924255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d00094d0 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3dffff640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80cf1b0 0x7ff3e80ca500 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff3d001ab80 tx=0x7ff3d001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3e4ff9640 1 -- 172.21.15.31:0/1433924255 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00b3800 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3e4ff9640 1 -- 172.21.15.31:0/1433924255 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d00b3980 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1433924255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e8152e40 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3e4ff9640 1 -- 172.21.15.31:0/1433924255 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00b3d30 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.733+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1433924255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e8153400 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.737+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1433924255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e81535f0 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.737+0000 7ff3e4ff9640 1 -- 172.21.15.31:0/1433924255 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7ff3d00b2380 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.737+0000 7ff3e4ff9640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff3c8066590 0x7ff3c8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:14.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.737+0000 7ff3e4ff9640 1 -- 172.21.15.31:0/1433924255 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3d00a7070 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.737+0000 7ff3e6ffd640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff3c8066590 0x7ff3c8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:14.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.737+0000 7ff3e6ffd640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff3c8066590 0x7ff3c8068a50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80b37f0 tx=0x7ff3d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:14.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.741+0000 7ff3e4ff9640 1 -- 172.21.15.31:0/1433924255 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff3d00eb390 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1433924255 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff3e804e850 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3e4ff9640 1 -- 172.21.15.31:0/1433924255 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff3d00ee9b0 con 0x7ff3e80cf1b0 2023-11-26T07:23:14.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1433924255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff3c8066590 msgr2=0x7ff3c8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff3c8066590 0x7ff3c8068a50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80b37f0 tx=0x7ff3d8008040 comp rx=0 tx=0).stop 2023-11-26T07:23:14.904 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1433924255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80cf1b0 msgr2=0x7ff3e80ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:14.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80cf1b0 0x7ff3e80ca500 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff3d001ab80 tx=0x7ff3d001ad10 comp rx=0 tx=0).stop 2023-11-26T07:23:14.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1433924255 shutdown_connections 2023-11-26T07:23:14.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff3c8066590 0x7ff3c8068a50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3e815d8f0 0x7ff3e8153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3e80cf1b0 0x7ff3e80ca500 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 --2- 172.21.15.31:0/1433924255 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff3e80ced30 0x7ff3e80cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:14.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1433924255 >> 172.21.15.31:0/1433924255 conn(0x7ff3e809c750 msgr2=0x7ff3e80b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:14.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1433924255 shutdown_connections 2023-11-26T07:23:14.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:14.901+0000 7ff3ed9c1640 1 -- 172.21.15.31:0/1433924255 wait complete. 2023-11-26T07:23:14.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:14.924 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103) 2023-11-26T07:23:14.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:14.951 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/fake) 2023-11-26T07:23:15.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:15.021 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2023-11-26T07:23:15.075 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-11-26T07:23:15.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:15.075 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103) 2023-11-26T07:23:15.141 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:15.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol66940/.meta 2023-11-26T07:23:15.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol66940 2023-11-26T07:23:15.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddf0a17640 1 Processor -- start 2023-11-26T07:23:15.403 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddf0a17640 1 -- start start 2023-11-26T07:23:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddf0a17640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0cb9b0 0x7fddec06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddf0a17640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fddec0ce5d0 0x7fddec15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddf0a17640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddec159630 con 0x7fddec0cbd90 2023-11-26T07:23:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddf0a17640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddec15bba0 con 0x7fddec0ce5d0 2023-11-26T07:23:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddf0a17640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddec0cdd50 con 0x7fddec0cb9b0 2023-11-26T07:23:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddea59c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0cb9b0 0x7fddec06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.405 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddead9d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fddec0ce5d0 0x7fddec15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fdde3fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fddec0cbd90 0x7fddec15f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:55076/0 (socket says 172.21.15.31:55076) 2023-11-26T07:23:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fdde3fff640 1 -- 172.21.15.31:0/3330012331 learned_addr learned my addr 172.21.15.31:0/3330012331 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddea59c640 1 -- 172.21.15.31:0/3330012331 >> v2:172.21.15.97:3300/0 conn(0x7fddec0ce5d0 msgr2=0x7fddec15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddea59c640 1 --2- 172.21.15.31:0/3330012331 >> v2:172.21.15.97:3300/0 conn(0x7fddec0ce5d0 0x7fddec15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddea59c640 1 -- 172.21.15.31:0/3330012331 >> v1:172.21.15.97:6789/0 conn(0x7fddec0cbd90 legacy=0x7fddec15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddea59c640 1 -- 172.21.15.31:0/3330012331 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec0cd1d0 con 0x7fddec0cb9b0 2023-11-26T07:23:15.406 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.401+0000 7fddead9d640 1 --2- 172.21.15.31:0/3330012331 >> v2:172.21.15.97:3300/0 conn(0x7fddec0ce5d0 0x7fddec15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:15.408 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.405+0000 7fddea59c640 1 --2- 172.21.15.31:0/3330012331 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0cb9b0 0x7fddec06e3f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0097c0 tx=0x7fdddc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f8b7f46a075988a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.405+0000 7fdde9d9b640 1 -- 172.21.15.31:0/3330012331 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0a7070 con 0x7fddec0cb9b0 2023-11-26T07:23:15.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.405+0000 7fdde9d9b640 1 -- 172.21.15.31:0/3330012331 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdddc0ab430 con 0x7fddec0cb9b0 2023-11-26T07:23:15.409 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.405+0000 7fdde9d9b640 1 -- 172.21.15.31:0/3330012331 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0ab7c0 con 0x7fddec0cb9b0 2023-11-26T07:23:15.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 -- 172.21.15.31:0/3330012331 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0cb9b0 msgr2=0x7fddec06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 --2- 172.21.15.31:0/3330012331 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0cb9b0 0x7fddec06e3f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0097c0 tx=0x7fdddc01c450 comp rx=0 tx=0).stop 2023-11-26T07:23:15.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 -- 172.21.15.31:0/3330012331 shutdown_connections 2023-11-26T07:23:15.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 --2- 172.21.15.31:0/3330012331 >> v2:172.21.15.97:3300/0 conn(0x7fddec0ce5d0 0x7fddec15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 --2- 172.21.15.31:0/3330012331 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0cb9b0 0x7fddec06e3f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 -- 172.21.15.31:0/3330012331 >> 172.21.15.31:0/3330012331 conn(0x7fddec0b0cb0 msgr2=0x7fddec0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:15.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 -- 172.21.15.31:0/3330012331 shutdown_connections 2023-11-26T07:23:15.410 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 -- 172.21.15.31:0/3330012331 wait complete. 2023-11-26T07:23:15.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 Processor -- start 2023-11-26T07:23:15.411 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 -- start start 2023-11-26T07:23:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddec0cb9b0 0x7fddec152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec0cbd90 0x7fddec0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0ce5d0 0x7fddec0ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec1650e0 con 0x7fddec0cbd90 2023-11-26T07:23:15.413 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec1536f0 con 0x7fddec0cb9b0 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddf0a17640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddec1647b0 con 0x7fddec0ce5d0 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fdde3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec0cbd90 0x7fddec0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fdde3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec0cbd90 0x7fddec0b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35038/0 (socket says 172.21.15.31:35038) 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fdde3fff640 1 -- 172.21.15.31:0/502180583 learned_addr learned my addr 172.21.15.31:0/502180583 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddea59c640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddec0cb9b0 0x7fddec152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fddead9d640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0ce5d0 0x7fddec0ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fdde3fff640 1 -- 172.21.15.31:0/502180583 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0ce5d0 msgr2=0x7fddec0ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fdde3fff640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0ce5d0 0x7fddec0ad0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fdde3fff640 1 -- 172.21.15.31:0/502180583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddec0cb9b0 msgr2=0x7fddec152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fdde3fff640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddec0cb9b0 0x7fddec152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.414 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.409+0000 7fdde3fff640 1 -- 172.21.15.31:0/502180583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde4009a70 con 0x7fddec0cbd90 2023-11-26T07:23:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fdde3fff640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec0cbd90 0x7fddec0b2a30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c500 tx=0x7fddd400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fdde2ffd640 1 -- 172.21.15.31:0/502180583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4005cf0 con 0x7fddec0cbd90 2023-11-26T07:23:15.415 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fdde2ffd640 1 -- 172.21.15.31:0/502180583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd4005e50 con 0x7fddec0cbd90 2023-11-26T07:23:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fddf0a17640 1 -- 172.21.15.31:0/502180583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc009420 con 0x7fddec0cbd90 2023-11-26T07:23:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fdde2ffd640 1 -- 172.21.15.31:0/502180583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd40072d0 con 0x7fddec0cbd90 2023-11-26T07:23:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fddf0a17640 1 -- 172.21.15.31:0/502180583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddec164d70 con 0x7fddec0cbd90 2023-11-26T07:23:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fddf0a17640 1 -- 172.21.15.31:0/502180583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddec0671f0 con 0x7fddec0cbd90 2023-11-26T07:23:15.416 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fdde2ffd640 1 -- 172.21.15.31:0/502180583 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fddd4007430 con 0x7fddec0cbd90 2023-11-26T07:23:15.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fdde2ffd640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fddcc0665c0 0x7fddcc068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fdde2ffd640 1 -- 172.21.15.31:0/502180583 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddd40834a0 con 0x7fddec0cbd90 2023-11-26T07:23:15.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fddea59c640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fddcc0665c0 0x7fddcc068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.417 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.413+0000 7fddea59c640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fddcc0665c0 0x7fddcc068a80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0af000 tx=0x7fdddc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.417+0000 7fdde2ffd640 1 -- 172.21.15.31:0/502180583 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fddd4050820 con 0x7fddec0cbd90 2023-11-26T07:23:15.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.601+0000 7fddf0a17640 1 -- 172.21.15.31:0/502180583 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol66940", "target": ["mon-mgr", ""]}) v1 -- 0x7fddec164920 con 0x7fddcc0665c0 2023-11-26T07:23:15.608 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:15.605+0000 7f1caf8af640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:15.608 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:15.605+0000 7f1caf8af640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:15.608 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:15.605+0000 7f1caf8af640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:15.608 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:15.605+0000 7f1caf8af640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:15.608 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:15.605+0000 7f1caf8af640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:15.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fdde2ffd640 1 -- 172.21.15.31:0/502180583 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7fddec164920 con 0x7fddcc0665c0 2023-11-26T07:23:15.744 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103 2023-11-26T07:23:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 -- 172.21.15.31:0/502180583 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fddcc0665c0 msgr2=0x7fddcc068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fddcc0665c0 0x7fddcc068a80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0af000 tx=0x7fdddc008040 comp rx=0 tx=0).stop 2023-11-26T07:23:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 -- 172.21.15.31:0/502180583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec0cbd90 msgr2=0x7fddec0b2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec0cbd90 0x7fddec0b2a30 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c500 tx=0x7fddd400c900 comp rx=0 tx=0).stop 2023-11-26T07:23:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 -- 172.21.15.31:0/502180583 shutdown_connections 2023-11-26T07:23:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fddcc0665c0 0x7fddcc068a80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fddec0ce5d0 0x7fddec0ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fddec0cbd90 0x7fddec0b2a30 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.746 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 --2- 172.21.15.31:0/502180583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fddec0cb9b0 0x7fddec152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 -- 172.21.15.31:0/502180583 >> 172.21.15.31:0/502180583 conn(0x7fddec0b0cb0 msgr2=0x7fddec158240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:15.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 -- 172.21.15.31:0/502180583 shutdown_connections 2023-11-26T07:23:15.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.741+0000 7fddf0a17640 1 -- 172.21.15.31:0/502180583 wait complete. 2023-11-26T07:23:15.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2023-11-26T07:23:15.910 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08cbc79640 1 Processor -- start 2023-11-26T07:23:15.911 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08cbc79640 1 -- start start 2023-11-26T07:23:15.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08cbc79640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f08c4159f00 0x7f08c40ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08cbc79640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08c40ae680 0x7f08c40cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08cbc79640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c4072950 con 0x7f08c40cd920 2023-11-26T07:23:15.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08cbc79640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c406e2b0 con 0x7f08c40ae680 2023-11-26T07:23:15.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08cbc79640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c406c6f0 con 0x7f08c4159f00 2023-11-26T07:23:15.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08c9214640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08c40ae680 0x7f08c40cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08c9214640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08c40ae680 0x7f08c40cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35064/0 (socket says 172.21.15.31:35064) 2023-11-26T07:23:15.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08c9214640 1 -- 172.21.15.31:0/633596892 learned_addr learned my addr 172.21.15.31:0/633596892 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:15.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08c9a15640 1 --2- 172.21.15.31:0/633596892 >> v2:172.21.15.97:3300/0 conn(0x7f08c4159f00 0x7f08c40ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08c9214640 1 -- 172.21.15.31:0/633596892 >> v2:172.21.15.97:3300/0 conn(0x7f08c4159f00 msgr2=0x7f08c40ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:15.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08c9214640 1 --2- 172.21.15.31:0/633596892 >> v2:172.21.15.97:3300/0 conn(0x7f08c4159f00 0x7f08c40ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08c9214640 1 -- 172.21.15.31:0/633596892 >> v1:172.21.15.97:6789/0 conn(0x7f08c40cd920 legacy=0x7f08c415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:15.914 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.909+0000 7f08c9214640 1 -- 172.21.15.31:0/633596892 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c406e550 con 0x7f08c40ae680 2023-11-26T07:23:15.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08c9214640 1 --2- 172.21.15.31:0/633596892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 0x7f08c40cd3e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f08c406c110 tx=0x7f08b401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461c70c6d1ef7b64 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08c8a13640 1 -- 172.21.15.31:0/633596892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40ac640 con 0x7f08c40ae680 2023-11-26T07:23:15.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08c8a13640 1 -- 172.21.15.31:0/633596892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b40ac7a0 con 0x7f08c40ae680 2023-11-26T07:23:15.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08c8a13640 1 -- 172.21.15.31:0/633596892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b40acb30 con 0x7f08c40ae680 2023-11-26T07:23:15.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08cbc79640 1 -- 172.21.15.31:0/633596892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 msgr2=0x7f08c40cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08cbc79640 1 --2- 172.21.15.31:0/633596892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 0x7f08c40cd3e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f08c406c110 tx=0x7f08b401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08cbc79640 1 -- 172.21.15.31:0/633596892 shutdown_connections 2023-11-26T07:23:15.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08cbc79640 1 --2- 172.21.15.31:0/633596892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 0x7f08c40cd3e0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08cbc79640 1 --2- 172.21.15.31:0/633596892 >> v2:172.21.15.97:3300/0 conn(0x7f08c4159f00 0x7f08c40ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08cbc79640 1 -- 172.21.15.31:0/633596892 >> 172.21.15.31:0/633596892 conn(0x7f08c40b0cb0 msgr2=0x7f08c40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:15.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08cbc79640 1 -- 172.21.15.31:0/633596892 shutdown_connections 2023-11-26T07:23:15.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.913+0000 7f08cbc79640 1 -- 172.21.15.31:0/633596892 wait complete. 2023-11-26T07:23:15.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08cbc79640 1 Processor -- start 2023-11-26T07:23:15.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08cbc79640 1 -- start start 2023-11-26T07:23:15.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08cbc79640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 0x7f08c4099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08cbc79640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08c40cd920 0x7f08c414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08cbc79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08c4152190 0x7f08c40afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08cbc79640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c41502e0 con 0x7f08c40ae680 2023-11-26T07:23:15.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08cbc79640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c4099f70 con 0x7f08c4152190 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08cbc79640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c409a210 con 0x7f08c40cd920 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08c9a15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 0x7f08c4099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08c9a15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 0x7f08c4099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35068/0 (socket says 172.21.15.31:35068) 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08c9a15640 1 -- 172.21.15.31:0/3027464741 learned_addr learned my addr 172.21.15.31:0/3027464741 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08c9214640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08c40cd920 0x7f08c414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08ca216640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08c4152190 0x7f08c40afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08c9a15640 1 -- 172.21.15.31:0/3027464741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08c40cd920 msgr2=0x7f08c414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08c9a15640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08c40cd920 0x7f08c414fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08c9a15640 1 -- 172.21.15.31:0/3027464741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08c4152190 msgr2=0x7f08c40afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08c9a15640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08c4152190 0x7f08c40afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:15.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08c9a15640 1 -- 172.21.15.31:0/3027464741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b40094d0 con 0x7f08c40ae680 2023-11-26T07:23:15.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08c9a15640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 0x7f08c4099a30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f08c000d5f0 tx=0x7f08c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08ba7fc640 1 -- 172.21.15.31:0/3027464741 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c00125f0 con 0x7f08c40ae680 2023-11-26T07:23:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08ba7fc640 1 -- 172.21.15.31:0/3027464741 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c0012750 con 0x7f08c40ae680 2023-11-26T07:23:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08cbc79640 1 -- 172.21.15.31:0/3027464741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c409a4f0 con 0x7f08c40ae680 2023-11-26T07:23:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08ba7fc640 1 -- 172.21.15.31:0/3027464741 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0012ae0 con 0x7f08c40ae680 2023-11-26T07:23:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.917+0000 7f08cbc79640 1 -- 172.21.15.31:0/3027464741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c4150810 con 0x7f08c40ae680 2023-11-26T07:23:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.921+0000 7f08cbc79640 1 -- 172.21.15.31:0/3027464741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c40671f0 con 0x7f08c40ae680 2023-11-26T07:23:15.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.921+0000 7f08ba7fc640 1 -- 172.21.15.31:0/3027464741 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f08c0012c40 con 0x7f08c40ae680 2023-11-26T07:23:15.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.921+0000 7f08ba7fc640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f08a8066590 0x7f08a8068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:15.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.921+0000 7f08ba7fc640 1 -- 172.21.15.31:0/3027464741 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08c0084420 con 0x7f08c40ae680 2023-11-26T07:23:15.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.921+0000 7f08c9214640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f08a8066590 0x7f08a8068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:15.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.921+0000 7f08c9214640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f08a8066590 0x7f08a8068a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08b40ae000 tx=0x7f08b40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:15.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:15.925+0000 7f08ba7fc640 1 -- 172.21.15.31:0/3027464741 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f08c0051790 con 0x7f08c40ae680 2023-11-26T07:23:16.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.109+0000 7f08cbc79640 1 -- 172.21.15.31:0/3027464741 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f08c409a730 con 0x7f08a8066590 2023-11-26T07:23:16.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.109+0000 7f08ba7fc640 1 -- 172.21.15.31:0/3027464741 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f08c409a730 con 0x7f08a8066590 2023-11-26T07:23:16.114 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:23:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subvol66940" 2023-11-26T07:23:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:16.114 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:23:16.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 -- 172.21.15.31:0/3027464741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f08a8066590 msgr2=0x7f08a8068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f08a8066590 0x7f08a8068a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08b40ae000 tx=0x7f08b40a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:16.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 -- 172.21.15.31:0/3027464741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 msgr2=0x7f08c4099a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 0x7f08c4099a30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f08c000d5f0 tx=0x7f08c000dab0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 -- 172.21.15.31:0/3027464741 shutdown_connections 2023-11-26T07:23:16.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f08a8066590 0x7f08a8068a50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08c4152190 0x7f08c40afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f08c40cd920 0x7f08c414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 --2- 172.21.15.31:0/3027464741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08c40ae680 0x7f08c4099a30 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 -- 172.21.15.31:0/3027464741 >> 172.21.15.31:0/3027464741 conn(0x7f08c40b0cb0 msgr2=0x7f08c4163cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:16.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 -- 172.21.15.31:0/3027464741 shutdown_connections 2023-11-26T07:23:16.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.113+0000 7f08cbc79640 1 -- 172.21.15.31:0/3027464741 wait complete. 2023-11-26T07:23:16.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol66940 2023-11-26T07:23:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.277+0000 7f7327ef7640 1 Processor -- start 2023-11-26T07:23:16.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.277+0000 7f7327ef7640 1 -- start start 2023-11-26T07:23:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7327ef7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f732006e3f0 0x7f732015f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7327ef7640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7320159630 con 0x7f73200ce5d0 2023-11-26T07:23:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7327ef7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732015f6d0 con 0x7f73200cb9b0 2023-11-26T07:23:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7327ef7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732006e860 con 0x7f732006e3f0 2023-11-26T07:23:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7326494640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f73200ce5d0 0x7f732015bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56284/0 (socket says 172.21.15.31:56284) 2023-11-26T07:23:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7326494640 1 -- 172.21.15.31:0/1555399145 learned_addr learned my addr 172.21.15.31:0/1555399145 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:16.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7325492640 1 --2- 172.21.15.31:0/1555399145 >> v2:172.21.15.97:3300/0 conn(0x7f732006e3f0 0x7f732015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7324c91640 1 -- 172.21.15.31:0/1555399145 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885091787 0 0) 0x7f7320159630 con 0x7f73200ce5d0 2023-11-26T07:23:16.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7324c91640 1 -- 172.21.15.31:0/1555399145 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f730c003530 con 0x7f73200ce5d0 2023-11-26T07:23:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7324c91640 1 -- 172.21.15.31:0/1555399145 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953096865 0 0) 0x7f732015f6d0 con 0x7f73200cb9b0 2023-11-26T07:23:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7324c91640 1 -- 172.21.15.31:0/1555399145 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7320159630 con 0x7f73200cb9b0 2023-11-26T07:23:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7325492640 1 -- 172.21.15.31:0/1555399145 >> v1:172.21.15.97:6789/0 conn(0x7f73200cb9b0 legacy=0x7f73200cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7325492640 1 -- 172.21.15.31:0/1555399145 >> v1:172.21.15.31:6789/0 conn(0x7f73200ce5d0 legacy=0x7f732015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7324c91640 1 -- 172.21.15.31:0/1555399145 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619381867 0 0) 0x7f730c003530 con 0x7f73200ce5d0 2023-11-26T07:23:16.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7325492640 1 -- 172.21.15.31:0/1555399145 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200cd1d0 con 0x7f732006e3f0 2023-11-26T07:23:16.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.281+0000 7f7325492640 1 --2- 172.21.15.31:0/1555399145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732006e3f0 0x7f732015f190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f73100097c0 tx=0x7f731001c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca6f84d59542001 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7324c91640 1 -- 172.21.15.31:0/1555399145 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318001a00 con 0x7f732006e3f0 2023-11-26T07:23:16.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7324c91640 1 -- 172.21.15.31:0/1555399145 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73100a7050 con 0x7f732006e3f0 2023-11-26T07:23:16.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7324c91640 1 -- 172.21.15.31:0/1555399145 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73100ab620 con 0x7f732006e3f0 2023-11-26T07:23:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 -- 172.21.15.31:0/1555399145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732006e3f0 msgr2=0x7f732015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 --2- 172.21.15.31:0/1555399145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732006e3f0 0x7f732015f190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f73100097c0 tx=0x7f731001c3a0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 -- 172.21.15.31:0/1555399145 shutdown_connections 2023-11-26T07:23:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 --2- 172.21.15.31:0/1555399145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f732006e3f0 0x7f732015f190 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 -- 172.21.15.31:0/1555399145 >> 172.21.15.31:0/1555399145 conn(0x7f73200b0cb0 msgr2=0x7f73200b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 -- 172.21.15.31:0/1555399145 shutdown_connections 2023-11-26T07:23:16.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 -- 172.21.15.31:0/1555399145 wait complete. 2023-11-26T07:23:16.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 Processor -- start 2023-11-26T07:23:16.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 -- start start 2023-11-26T07:23:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732006e3f0 0x7f7320151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73200cb9b0 0x7f732015de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73200ce5d0 0x7f732015e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7320153070 con 0x7f732006e3f0 2023-11-26T07:23:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7320153350 con 0x7f73200ce5d0 2023-11-26T07:23:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7327ef7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7320153630 con 0x7f73200cb9b0 2023-11-26T07:23:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.285+0000 7f7325c93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732006e3f0 0x7f7320151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7325c93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732006e3f0 0x7f7320151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35078/0 (socket says 172.21.15.31:35078) 2023-11-26T07:23:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7325c93640 1 -- 172.21.15.31:0/3035389349 learned_addr learned my addr 172.21.15.31:0/3035389349 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:16.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7326494640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73200ce5d0 0x7f732015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7325492640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73200cb9b0 0x7f732015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7325c93640 1 -- 172.21.15.31:0/3035389349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73200cb9b0 msgr2=0x7f732015de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7325c93640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73200cb9b0 0x7f732015de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7325c93640 1 -- 172.21.15.31:0/3035389349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73200ce5d0 msgr2=0x7f732015e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7325c93640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73200ce5d0 0x7f732015e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7325c93640 1 -- 172.21.15.31:0/3035389349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7310009420 con 0x7f732006e3f0 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7326494640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73200ce5d0 0x7f732015e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7325492640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73200cb9b0 0x7f732015de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7325c93640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732006e3f0 0x7f7320151930 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f731c00c490 tx=0x7f731c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f73167fc640 1 -- 172.21.15.31:0/3035389349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0052d0 con 0x7f732006e3f0 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f73167fc640 1 -- 172.21.15.31:0/3035389349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c005430 con 0x7f732006e3f0 2023-11-26T07:23:16.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f73167fc640 1 -- 172.21.15.31:0/3035389349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c012870 con 0x7f732006e3f0 2023-11-26T07:23:16.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7327ef7640 1 -- 172.21.15.31:0/3035389349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7320153910 con 0x7f732006e3f0 2023-11-26T07:23:16.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7327ef7640 1 -- 172.21.15.31:0/3035389349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7320151e70 con 0x7f732006e3f0 2023-11-26T07:23:16.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f7327ef7640 1 -- 172.21.15.31:0/3035389349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7320152060 con 0x7f732006e3f0 2023-11-26T07:23:16.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f73167fc640 1 -- 172.21.15.31:0/3035389349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f731c0129d0 con 0x7f732006e3f0 2023-11-26T07:23:16.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f73167fc640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7304066590 0x7f7304068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.289+0000 7f73167fc640 1 -- 172.21.15.31:0/3035389349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f731c083bf0 con 0x7f732006e3f0 2023-11-26T07:23:16.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.293+0000 7f7325492640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7304066590 0x7f7304068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.293+0000 7f7325492640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7304066590 0x7f7304068a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f73100ad000 tx=0x7f7310008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.293+0000 7f73167fc640 1 -- 172.21.15.31:0/3035389349 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f731c050f50 con 0x7f732006e3f0 2023-11-26T07:23:16.478 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.477+0000 7f7327ef7640 1 -- 172.21.15.31:0/3035389349 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol66940", "target": ["mon-mgr", ""]}) v1 -- 0x7f73200ccea0 con 0x7f7304066590 2023-11-26T07:23:16.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f73167fc640 1 -- 172.21.15.31:0/3035389349 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+667 (secure 0 0 0) 0x7f73100ab6e0 con 0x7f7304066590 2023-11-26T07:23:16.503 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "atime": "2023-11-26 07:23:14", 2023-11-26T07:23:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "undefined", 2023-11-26T07:23:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": "infinite", 2023-11-26T07:23:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 0, 2023-11-26T07:23:16.504 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-11-26 07:23:14", 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "ctime": "2023-11-26 07:23:15", 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "features": [ 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-clone", 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-autoprotect" 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 0, 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": 16877, 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_addrs": [ 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.31:6789", 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.97:6789", 2023-11-26T07:23:16.505 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.97:6790" 2023-11-26T07:23:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:23:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "mtime": "2023-11-26 07:23:14", 2023-11-26T07:23:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103", 2023-11-26T07:23:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_namespace": "", 2023-11-26T07:23:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete", 2023-11-26T07:23:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "subvolume", 2023-11-26T07:23:16.506 INFO:teuthology.orchestra.run.smithi031.stdout: "uid": 0 2023-11-26T07:23:16.506 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:16.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 -- 172.21.15.31:0/3035389349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7304066590 msgr2=0x7f7304068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7304066590 0x7f7304068a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f73100ad000 tx=0x7f7310008040 comp rx=0 tx=0).stop 2023-11-26T07:23:16.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 -- 172.21.15.31:0/3035389349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732006e3f0 msgr2=0x7f7320151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732006e3f0 0x7f7320151930 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f731c00c490 tx=0x7f731c00c950 comp rx=0 tx=0).stop 2023-11-26T07:23:16.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 -- 172.21.15.31:0/3035389349 shutdown_connections 2023-11-26T07:23:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7304066590 0x7f7304068a50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73200ce5d0 0x7f732015e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f73200cb9b0 0x7f732015de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 --2- 172.21.15.31:0/3035389349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f732006e3f0 0x7f7320151930 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 -- 172.21.15.31:0/3035389349 >> 172.21.15.31:0/3035389349 conn(0x7f73200b0cb0 msgr2=0x7f732015aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 -- 172.21.15.31:0/3035389349 shutdown_connections 2023-11-26T07:23:16.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.501+0000 7f7327ef7640 1 -- 172.21.15.31:0/3035389349 wait complete. 2023-11-26T07:23:16.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol66940 10485760 2023-11-26T07:23:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe74db65640 1 Processor -- start 2023-11-26T07:23:16.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe74db65640 1 -- start start 2023-11-26T07:23:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe74db65640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7480cdfb0 0x7fe7480abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe74db65640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe74815a710 0x7fe7480ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe74db65640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7480ce8d0 con 0x7fe7480cbd50 2023-11-26T07:23:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe74db65640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7480cca80 con 0x7fe7480cdfb0 2023-11-26T07:23:16.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe74db65640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480b2710 con 0x7fe74815a710 2023-11-26T07:23:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe746ffd640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe74815a710 0x7fe7480ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe7477fe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fe7480cdfb0 0x7fe7480abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe747fff640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fe7480cbd50 0x7fe7480cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:55114/0 (socket says 172.21.15.31:55114) 2023-11-26T07:23:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe747fff640 1 -- 172.21.15.31:0/2216187368 learned_addr learned my addr 172.21.15.31:0/2216187368 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:16.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe7467fc640 1 -- 172.21.15.31:0/2216187368 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923784866 0 0) 0x7fe7480ce8d0 con 0x7fe7480cbd50 2023-11-26T07:23:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe746ffd640 1 -- 172.21.15.31:0/2216187368 >> v2:172.21.15.97:3300/0 conn(0x7fe7480cdfb0 msgr2=0x7fe7480abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe746ffd640 1 --2- 172.21.15.31:0/2216187368 >> v2:172.21.15.97:3300/0 conn(0x7fe7480cdfb0 0x7fe7480abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe746ffd640 1 -- 172.21.15.31:0/2216187368 >> v1:172.21.15.97:6789/0 conn(0x7fe7480cbd50 legacy=0x7fe7480cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe746ffd640 1 -- 172.21.15.31:0/2216187368 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7480b29f0 con 0x7fe74815a710 2023-11-26T07:23:16.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.669+0000 7fe7477fe640 1 --2- 172.21.15.31:0/2216187368 >> v2:172.21.15.97:3300/0 conn(0x7fe7480cdfb0 0x7fe7480abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:16.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe746ffd640 1 --2- 172.21.15.31:0/2216187368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe74815a710 0x7fe7480ce390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0097c0 tx=0x7fe73c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b9a30870168d2a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe7467fc640 1 -- 172.21.15.31:0/2216187368 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0ab650 con 0x7fe74815a710 2023-11-26T07:23:16.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe7467fc640 1 -- 172.21.15.31:0/2216187368 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c0ab7b0 con 0x7fe74815a710 2023-11-26T07:23:16.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe7467fc640 1 -- 172.21.15.31:0/2216187368 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0abb40 con 0x7fe74815a710 2023-11-26T07:23:16.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe74db65640 1 -- 172.21.15.31:0/2216187368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe74815a710 msgr2=0x7fe7480ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe74db65640 1 --2- 172.21.15.31:0/2216187368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe74815a710 0x7fe7480ce390 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0097c0 tx=0x7fe73c01c450 comp rx=0 tx=0).stop 2023-11-26T07:23:16.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe74db65640 1 -- 172.21.15.31:0/2216187368 shutdown_connections 2023-11-26T07:23:16.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe74db65640 1 --2- 172.21.15.31:0/2216187368 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe74815a710 0x7fe7480ce390 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe74db65640 1 --2- 172.21.15.31:0/2216187368 >> v2:172.21.15.97:3300/0 conn(0x7fe7480cdfb0 0x7fe7480abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe74db65640 1 -- 172.21.15.31:0/2216187368 >> 172.21.15.31:0/2216187368 conn(0x7fe7480b0cb0 msgr2=0x7fe7480b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe74db65640 1 -- 172.21.15.31:0/2216187368 shutdown_connections 2023-11-26T07:23:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.673+0000 7fe74db65640 1 -- 172.21.15.31:0/2216187368 wait complete. 2023-11-26T07:23:16.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe74db65640 1 Processor -- start 2023-11-26T07:23:16.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe74db65640 1 -- start start 2023-11-26T07:23:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe74db65640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7480cbd50 0x7fe748149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe74db65640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7480cdfb0 0x7fe74814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe74db65640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815a710 0x7fe74815b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe74db65640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74814c600 con 0x7fe74815a710 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe74db65640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74814a170 con 0x7fe7480cbd50 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe74db65640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74815c320 con 0x7fe7480cdfb0 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe747fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815a710 0x7fe74815b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe747fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815a710 0x7fe74815b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35094/0 (socket says 172.21.15.31:35094) 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe747fff640 1 -- 172.21.15.31:0/1950178949 learned_addr learned my addr 172.21.15.31:0/1950178949 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe7477fe640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7480cbd50 0x7fe748149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe746ffd640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7480cdfb0 0x7fe74814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe747fff640 1 -- 172.21.15.31:0/1950178949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7480cdfb0 msgr2=0x7fe74814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe747fff640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7480cdfb0 0x7fe74814ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe747fff640 1 -- 172.21.15.31:0/1950178949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7480cbd50 msgr2=0x7fe748149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe747fff640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7480cbd50 0x7fe748149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe747fff640 1 -- 172.21.15.31:0/1950178949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe738009420 con 0x7fe74815a710 2023-11-26T07:23:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe747fff640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815a710 0x7fe74815b760 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe740001570 tx=0x7fe74000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe72ffff640 1 -- 172.21.15.31:0/1950178949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74000a640 con 0x7fe74815a710 2023-11-26T07:23:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe72ffff640 1 -- 172.21.15.31:0/1950178949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74000a7c0 con 0x7fe74815a710 2023-11-26T07:23:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe72ffff640 1 -- 172.21.15.31:0/1950178949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74000ab70 con 0x7fe74815a710 2023-11-26T07:23:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe74db65640 1 -- 172.21.15.31:0/1950178949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c009420 con 0x7fe74815a710 2023-11-26T07:23:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.677+0000 7fe74db65640 1 -- 172.21.15.31:0/1950178949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe748071000 con 0x7fe74815a710 2023-11-26T07:23:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.681+0000 7fe74db65640 1 -- 172.21.15.31:0/1950178949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7480671f0 con 0x7fe74815a710 2023-11-26T07:23:16.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.681+0000 7fe72ffff640 1 -- 172.21.15.31:0/1950178949 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7fe74000acd0 con 0x7fe74815a710 2023-11-26T07:23:16.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.681+0000 7fe72ffff640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe734066590 0x7fe734068a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:16.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.681+0000 7fe72ffff640 1 -- 172.21.15.31:0/1950178949 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7400833e0 con 0x7fe74815a710 2023-11-26T07:23:16.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.681+0000 7fe7477fe640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe734066590 0x7fe734068a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:16.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.681+0000 7fe7477fe640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe734066590 0x7fe734068a50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe738009cf0 tx=0x7fe738009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:16.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.685+0000 7fe72ffff640 1 -- 172.21.15.31:0/1950178949 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe740050800 con 0x7fe74815a710 2023-11-26T07:23:16.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.873+0000 7fe74db65640 1 -- 172.21.15.31:0/1950178949 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol66940", "new_size": "10485760", "target": ["mon-mgr", ""]}) v1 -- 0x7fe74806b460 con 0x7fe734066590 2023-11-26T07:23:16.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.897+0000 7fe72ffff640 1 -- 172.21.15.31:0/1950178949 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+128 (secure 0 0 0) 0x7fe74806b460 con 0x7fe734066590 2023-11-26T07:23:16.900 INFO:teuthology.orchestra.run.smithi031.stdout:[ 2023-11-26T07:23:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 0 2023-11-26T07:23:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": 10485760 2023-11-26T07:23:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2023-11-26T07:23:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "0.00" 2023-11-26T07:23:16.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:16.901 INFO:teuthology.orchestra.run.smithi031.stdout:] 2023-11-26T07:23:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 -- 172.21.15.31:0/1950178949 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe734066590 msgr2=0x7fe734068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe734066590 0x7fe734068a50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe738009cf0 tx=0x7fe738009190 comp rx=0 tx=0).stop 2023-11-26T07:23:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 -- 172.21.15.31:0/1950178949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815a710 msgr2=0x7fe74815b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815a710 0x7fe74815b760 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe740001570 tx=0x7fe74000cca0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 -- 172.21.15.31:0/1950178949 shutdown_connections 2023-11-26T07:23:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe734066590 0x7fe734068a50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe74815a710 0x7fe74815b760 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe7480cdfb0 0x7fe74814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 --2- 172.21.15.31:0/1950178949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7480cbd50 0x7fe748149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 -- 172.21.15.31:0/1950178949 >> 172.21.15.31:0/1950178949 conn(0x7fe7480b0cb0 msgr2=0x7fe7480b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 -- 172.21.15.31:0/1950178949 shutdown_connections 2023-11-26T07:23:16.903 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:16.901+0000 7fe74db65640 1 -- 172.21.15.31:0/1950178949 wait complete. 2023-11-26T07:23:16.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol66940 2023-11-26T07:23:17.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a6db80640 1 Processor -- start 2023-11-26T07:23:17.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a6db80640 1 -- start start 2023-11-26T07:23:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a6db80640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1a680cb9b0 0x7f1a6806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a6db80640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a680cbd90 0x7f1a6815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a6db80640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a68159630 con 0x7f1a680ce5d0 2023-11-26T07:23:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a6db80640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6815bba0 con 0x7f1a680cbd90 2023-11-26T07:23:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a6db80640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a680cdd50 con 0x7f1a680cb9b0 2023-11-26T07:23:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a66ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a680cbd90 0x7f1a6815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a66ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1a680cbd90 0x7f1a6815f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35108/0 (socket says 172.21.15.31:35108) 2023-11-26T07:23:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a66ffd640 1 -- 172.21.15.31:0/3690143549 learned_addr learned my addr 172.21.15.31:0/3690143549 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a67fff640 1 --1- 172.21.15.31:0/3690143549 >> v1:172.21.15.31:6789/0 conn(0x7f1a680ce5d0 0x7f1a6815bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56294/0 (socket says 172.21.15.31:56294) 2023-11-26T07:23:17.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a677fe640 1 --2- 172.21.15.31:0/3690143549 >> v2:172.21.15.97:3300/0 conn(0x7f1a680cb9b0 0x7f1a6806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a66ffd640 1 -- 172.21.15.31:0/3690143549 >> v2:172.21.15.97:3300/0 conn(0x7f1a680cb9b0 msgr2=0x7f1a6806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a66ffd640 1 --2- 172.21.15.31:0/3690143549 >> v2:172.21.15.97:3300/0 conn(0x7f1a680cb9b0 0x7f1a6806e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a66ffd640 1 -- 172.21.15.31:0/3690143549 >> v1:172.21.15.31:6789/0 conn(0x7f1a680ce5d0 legacy=0x7f1a6815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a66ffd640 1 -- 172.21.15.31:0/3690143549 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6815f6d0 con 0x7f1a680cbd90 2023-11-26T07:23:17.070 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.065+0000 7f1a667fc640 1 -- 172.21.15.31:0/3690143549 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474261258 0 0) 0x7f1a68159630 con 0x7f1a680ce5d0 2023-11-26T07:23:17.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a66ffd640 1 --2- 172.21.15.31:0/3690143549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680cbd90 0x7f1a6815f190 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1a6806c360 tx=0x7f1a5c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dcccd69aa852e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a667fc640 1 -- 172.21.15.31:0/3690143549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0aa430 con 0x7f1a680cbd90 2023-11-26T07:23:17.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a667fc640 1 -- 172.21.15.31:0/3690143549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c0aa590 con 0x7f1a680cbd90 2023-11-26T07:23:17.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a667fc640 1 -- 172.21.15.31:0/3690143549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0aa920 con 0x7f1a680cbd90 2023-11-26T07:23:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a6db80640 1 -- 172.21.15.31:0/3690143549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680cbd90 msgr2=0x7f1a6815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a6db80640 1 --2- 172.21.15.31:0/3690143549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680cbd90 0x7f1a6815f190 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f1a6806c360 tx=0x7f1a5c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a6db80640 1 -- 172.21.15.31:0/3690143549 shutdown_connections 2023-11-26T07:23:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a6db80640 1 --2- 172.21.15.31:0/3690143549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680cbd90 0x7f1a6815f190 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a6db80640 1 --2- 172.21.15.31:0/3690143549 >> v2:172.21.15.97:3300/0 conn(0x7f1a680cb9b0 0x7f1a6806e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a6db80640 1 -- 172.21.15.31:0/3690143549 >> 172.21.15.31:0/3690143549 conn(0x7f1a680b0cb0 msgr2=0x7f1a680b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a6db80640 1 -- 172.21.15.31:0/3690143549 shutdown_connections 2023-11-26T07:23:17.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.069+0000 7f1a6db80640 1 -- 172.21.15.31:0/3690143549 wait complete. 2023-11-26T07:23:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a6db80640 1 Processor -- start 2023-11-26T07:23:17.074 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a6db80640 1 -- start start 2023-11-26T07:23:17.076 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a6db80640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a680cb9b0 0x7f1a6814fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a6db80640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a680cbd90 0x7f1a68151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a6db80640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680ce5d0 0x7f1a6806c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a6db80640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a68156820 con 0x7f1a680ce5d0 2023-11-26T07:23:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a6db80640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a681595a0 con 0x7f1a680cb9b0 2023-11-26T07:23:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a6db80640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a68159880 con 0x7f1a680cbd90 2023-11-26T07:23:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a67fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680ce5d0 0x7f1a6806c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a67fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680ce5d0 0x7f1a6806c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35124/0 (socket says 172.21.15.31:35124) 2023-11-26T07:23:17.077 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a67fff640 1 -- 172.21.15.31:0/1128925805 learned_addr learned my addr 172.21.15.31:0/1128925805 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a66ffd640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a680cbd90 0x7f1a68151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a677fe640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a680cb9b0 0x7f1a6814fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a67fff640 1 -- 172.21.15.31:0/1128925805 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a680cbd90 msgr2=0x7f1a68151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a67fff640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a680cbd90 0x7f1a68151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a67fff640 1 -- 172.21.15.31:0/1128925805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a680cb9b0 msgr2=0x7f1a6814fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a67fff640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a680cb9b0 0x7f1a6814fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a67fff640 1 -- 172.21.15.31:0/1128925805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0094d0 con 0x7f1a680ce5d0 2023-11-26T07:23:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a677fe640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a680cb9b0 0x7f1a6814fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a67fff640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680ce5d0 0x7f1a6806c6f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000c780 tx=0x7f1a6000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.078 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a4ffff640 1 -- 172.21.15.31:0/1128925805 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000ce90 con 0x7f1a680ce5d0 2023-11-26T07:23:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a4ffff640 1 -- 172.21.15.31:0/1128925805 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6000a640 con 0x7f1a680ce5d0 2023-11-26T07:23:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a4ffff640 1 -- 172.21.15.31:0/1128925805 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000a9d0 con 0x7f1a680ce5d0 2023-11-26T07:23:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a6db80640 1 -- 172.21.15.31:0/1128925805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a68159b60 con 0x7f1a680ce5d0 2023-11-26T07:23:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.073+0000 7f1a6db80640 1 -- 172.21.15.31:0/1128925805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6806cc30 con 0x7f1a680ce5d0 2023-11-26T07:23:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.077+0000 7f1a6db80640 1 -- 172.21.15.31:0/1128925805 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a680671f0 con 0x7f1a680ce5d0 2023-11-26T07:23:17.079 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.077+0000 7f1a4ffff640 1 -- 172.21.15.31:0/1128925805 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 15) v1 ==== 85954+0+0 (secure 0 0 0) 0x7f1a6000ab30 con 0x7f1a680ce5d0 2023-11-26T07:23:17.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.077+0000 7f1a4ffff640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a540665c0 0x7f1a54068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.077+0000 7f1a4ffff640 1 -- 172.21.15.31:0/1128925805 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a600833a0 con 0x7f1a680ce5d0 2023-11-26T07:23:17.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.077+0000 7f1a677fe640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a540665c0 0x7f1a54068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.081 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.077+0000 7f1a677fe640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a540665c0 0x7f1a54068a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009bb0 tx=0x7f1a58008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.081+0000 7f1a4ffff640 1 -- 172.21.15.31:0/1128925805 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1a60050720 con 0x7f1a680ce5d0 2023-11-26T07:23:17.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.201+0000 7f1a4ffff640 1 -- 172.21.15.31:0/1128925805 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 16) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f1a60050af0 con 0x7f1a680ce5d0 2023-11-26T07:23:17.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.269+0000 7f1a6db80640 1 -- 172.21.15.31:0/1128925805 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol66940", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a68159710 con 0x7f1a540665c0 2023-11-26T07:23:17.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a4ffff640 1 -- 172.21.15.31:0/1128925805 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+660 (secure 0 0 0) 0x7f1a68159710 con 0x7f1a540665c0 2023-11-26T07:23:17.295 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: "atime": "2023-11-26 07:23:14", 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "0.00", 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": 10485760, 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 0, 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-11-26 07:23:14", 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: "ctime": "2023-11-26 07:23:16", 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: "features": [ 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-clone", 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-autoprotect" 2023-11-26T07:23:17.296 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 0, 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": 16877, 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_addrs": [ 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.31:6789", 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.97:6789", 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.97:6790" 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: "mtime": "2023-11-26 07:23:14", 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103", 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_namespace": "", 2023-11-26T07:23:17.297 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete", 2023-11-26T07:23:17.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "subvolume", 2023-11-26T07:23:17.298 INFO:teuthology.orchestra.run.smithi031.stdout: "uid": 0 2023-11-26T07:23:17.298 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 -- 172.21.15.31:0/1128925805 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a540665c0 msgr2=0x7f1a54068a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a540665c0 0x7f1a54068a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009bb0 tx=0x7f1a58008040 comp rx=0 tx=0).stop 2023-11-26T07:23:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 -- 172.21.15.31:0/1128925805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680ce5d0 msgr2=0x7f1a6806c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680ce5d0 0x7f1a6806c6f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000c780 tx=0x7f1a6000cc40 comp rx=0 tx=0).stop 2023-11-26T07:23:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 -- 172.21.15.31:0/1128925805 shutdown_connections 2023-11-26T07:23:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1a540665c0 0x7f1a54068a80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a680ce5d0 0x7f1a6806c6f0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1a680cbd90 0x7f1a68151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.299 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 --2- 172.21.15.31:0/1128925805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a680cb9b0 0x7f1a6814fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 -- 172.21.15.31:0/1128925805 >> 172.21.15.31:0/1128925805 conn(0x7f1a680b0cb0 msgr2=0x7f1a680b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:17.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.293+0000 7f1a6db80640 1 -- 172.21.15.31:0/1128925805 shutdown_connections 2023-11-26T07:23:17.300 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.297+0000 7f1a6db80640 1 -- 172.21.15.31:0/1128925805 wait complete. 2023-11-26T07:23:17.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol66940 --mode 777 --uid 1000 --gid 1000 2023-11-26T07:23:17.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.465+0000 7f9ed7da1640 1 Processor -- start 2023-11-26T07:23:17.469 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.465+0000 7f9ed7da1640 1 -- start start 2023-11-26T07:23:17.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed7da1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ed00b46a0 0x7f9ed015e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.470 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed7da1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed0147190 con 0x7f9ed00b4210 2023-11-26T07:23:17.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed7da1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed006d030 con 0x7f9ed0146820 2023-11-26T07:23:17.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed7da1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed0146c00 con 0x7f9ed00b46a0 2023-11-26T07:23:17.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed633e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ed00b46a0 0x7f9ed015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed633e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ed00b46a0 0x7f9ed015e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35138/0 (socket says 172.21.15.31:35138) 2023-11-26T07:23:17.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed633e640 1 -- 172.21.15.31:0/4041581853 learned_addr learned my addr 172.21.15.31:0/4041581853 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:17.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed533c640 1 --1- 172.21.15.31:0/4041581853 >> v1:172.21.15.31:6789/0 conn(0x7f9ed00b4210 0x7f9ed015db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56302/0 (socket says 172.21.15.31:56302) 2023-11-26T07:23:17.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed633e640 1 -- 172.21.15.31:0/4041581853 >> v1:172.21.15.97:6789/0 conn(0x7f9ed0146820 legacy=0x7f9ed0070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.471 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed633e640 1 -- 172.21.15.31:0/4041581853 >> v1:172.21.15.31:6789/0 conn(0x7f9ed00b4210 legacy=0x7f9ed015db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.472 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed633e640 1 -- 172.21.15.31:0/4041581853 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed006bf90 con 0x7f9ed00b46a0 2023-11-26T07:23:17.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.469+0000 7f9ed633e640 1 --2- 172.21.15.31:0/4041581853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed00b46a0 0x7f9ed015e250 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009e50 tx=0x7f9ec801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901c80053dc1efcf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed4b3b640 1 -- 172.21.15.31:0/4041581853 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80a58a0 con 0x7f9ed00b46a0 2023-11-26T07:23:17.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed4b3b640 1 -- 172.21.15.31:0/4041581853 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec80a5a00 con 0x7f9ed00b46a0 2023-11-26T07:23:17.474 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed4b3b640 1 -- 172.21.15.31:0/4041581853 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80a5d90 con 0x7f9ed00b46a0 2023-11-26T07:23:17.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 -- 172.21.15.31:0/4041581853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed00b46a0 msgr2=0x7f9ed015e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 --2- 172.21.15.31:0/4041581853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed00b46a0 0x7f9ed015e250 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009e50 tx=0x7f9ec801c9e0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 -- 172.21.15.31:0/4041581853 shutdown_connections 2023-11-26T07:23:17.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 --2- 172.21.15.31:0/4041581853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed00b46a0 0x7f9ed015e250 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 -- 172.21.15.31:0/4041581853 >> 172.21.15.31:0/4041581853 conn(0x7f9ed00b2f10 msgr2=0x7f9ed00b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:17.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 -- 172.21.15.31:0/4041581853 shutdown_connections 2023-11-26T07:23:17.475 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 -- 172.21.15.31:0/4041581853 wait complete. 2023-11-26T07:23:17.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 Processor -- start 2023-11-26T07:23:17.476 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 -- start start 2023-11-26T07:23:17.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ed00b4210 0x7f9ed0141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed00b46a0 0x7f9ed013fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed0146820 0x7f9ed0140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed0142b00 con 0x7f9ed0146820 2023-11-26T07:23:17.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed0141a10 con 0x7f9ed00b46a0 2023-11-26T07:23:17.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed7da1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed0141cb0 con 0x7f9ed00b4210 2023-11-26T07:23:17.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.473+0000 7f9ed633e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed0146820 0x7f9ed0140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed633e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed0146820 0x7f9ed0140220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35144/0 (socket says 172.21.15.31:35144) 2023-11-26T07:23:17.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed633e640 1 -- 172.21.15.31:0/2305731738 learned_addr learned my addr 172.21.15.31:0/2305731738 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:17.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed533c640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed00b46a0 0x7f9ed013fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed5b3d640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ed00b4210 0x7f9ed0141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed633e640 1 -- 172.21.15.31:0/2305731738 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ed00b4210 msgr2=0x7f9ed0141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed633e640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ed00b4210 0x7f9ed0141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed633e640 1 -- 172.21.15.31:0/2305731738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed00b46a0 msgr2=0x7f9ed013fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed633e640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed00b46a0 0x7f9ed013fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed633e640 1 -- 172.21.15.31:0/2305731738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec8009b30 con 0x7f9ed0146820 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed533c640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed00b46a0 0x7f9ed013fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed633e640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed0146820 0x7f9ed0140220 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80195f0 tx=0x7f9ec8019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ec67fc640 1 -- 172.21.15.31:0/2305731738 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80b8880 con 0x7f9ed0146820 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ec67fc640 1 -- 172.21.15.31:0/2305731738 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec80b8a00 con 0x7f9ed0146820 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ec67fc640 1 -- 172.21.15.31:0/2305731738 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80b8db0 con 0x7f9ed0146820 2023-11-26T07:23:17.480 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed7da1640 1 -- 172.21.15.31:0/2305731738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed0141f90 con 0x7f9ed0146820 2023-11-26T07:23:17.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed7da1640 1 -- 172.21.15.31:0/2305731738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed0141720 con 0x7f9ed0146820 2023-11-26T07:23:17.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ed7da1640 1 -- 172.21.15.31:0/2305731738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed006bb10 con 0x7f9ed0146820 2023-11-26T07:23:17.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ec67fc640 1 -- 172.21.15.31:0/2305731738 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 86005+0+0 (secure 0 0 0) 0x7f9ec80c3460 con 0x7f9ed0146820 2023-11-26T07:23:17.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ec67fc640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9eb4066620 0x7f9eb4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.477+0000 7f9ec67fc640 1 -- 172.21.15.31:0/2305731738 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ec811f9d0 con 0x7f9ed0146820 2023-11-26T07:23:17.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.481+0000 7f9ed5b3d640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9eb4066620 0x7f9eb4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.481+0000 7f9ed5b3d640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9eb4066620 0x7f9eb4068ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc000f70 tx=0x7f9ecc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.485+0000 7f9ec67fc640 1 -- 172.21.15.31:0/2305731738 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9ec80ecdc0 con 0x7f9ed0146820 2023-11-26T07:23:17.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.665+0000 7f9ed7da1640 1 -- 172.21.15.31:0/2305731738 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol66940", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) v1 -- 0x7f9ed006cb10 con 0x7f9eb4066620 2023-11-26T07:23:17.700 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.697+0000 7f9ec67fc640 1 -- 172.21.15.31:0/2305731738 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9ecc0012d0 con 0x7f9eb4066620 2023-11-26T07:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 -- 172.21.15.31:0/2305731738 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9eb4066620 msgr2=0x7f9eb4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9eb4066620 0x7f9eb4068ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc000f70 tx=0x7f9ecc009040 comp rx=0 tx=0).stop 2023-11-26T07:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 -- 172.21.15.31:0/2305731738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed0146820 msgr2=0x7f9ed0140220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed0146820 0x7f9ed0140220 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80195f0 tx=0x7f9ec8019780 comp rx=0 tx=0).stop 2023-11-26T07:23:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 -- 172.21.15.31:0/2305731738 shutdown_connections 2023-11-26T07:23:17.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9eb4066620 0x7f9eb4068ae0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ed0146820 0x7f9ed0140220 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ed00b46a0 0x7f9ed013fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 --2- 172.21.15.31:0/2305731738 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9ed00b4210 0x7f9ed0141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 -- 172.21.15.31:0/2305731738 >> 172.21.15.31:0/2305731738 conn(0x7f9ed00b2f10 msgr2=0x7f9ed0145790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:17.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 -- 172.21.15.31:0/2305731738 shutdown_connections 2023-11-26T07:23:17.704 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.701+0000 7f9ed7da1640 1 -- 172.21.15.31:0/2305731738 wait complete. 2023-11-26T07:23:17.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol66940 2023-11-26T07:23:17.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd351c640 1 Processor -- start 2023-11-26T07:23:17.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd351c640 1 -- start start 2023-11-26T07:23:17.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd351c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc0b07a0 0x7fabcc1619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd351c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fabcc0b15f0 0x7fabcc0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd351c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabcc161290 con 0x7fabcc161f20 2023-11-26T07:23:17.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd351c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabcc0b0b80 con 0x7fabcc0b15f0 2023-11-26T07:23:17.869 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd351c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc15ebc0 con 0x7fabcc0b07a0 2023-11-26T07:23:17.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd12b8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc0b07a0 0x7fabcc1619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd0ab7640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fabcc161f20 0x7fabcc0b0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:55126/0 (socket says 172.21.15.31:55126) 2023-11-26T07:23:17.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd1ab9640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fabcc0b15f0 0x7fabcc0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd0ab7640 1 -- 172.21.15.31:0/2067979779 learned_addr learned my addr 172.21.15.31:0/2067979779 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:17.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd12b8640 1 -- 172.21.15.31:0/2067979779 >> v2:172.21.15.97:3300/0 conn(0x7fabcc0b15f0 msgr2=0x7fabcc0cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:17.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd12b8640 1 --2- 172.21.15.31:0/2067979779 >> v2:172.21.15.97:3300/0 conn(0x7fabcc0b15f0 0x7fabcc0cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd12b8640 1 -- 172.21.15.31:0/2067979779 >> v1:172.21.15.97:6789/0 conn(0x7fabcc161f20 legacy=0x7fabcc0b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.871 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.865+0000 7fabd12b8640 1 -- 172.21.15.31:0/2067979779 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabcc161550 con 0x7fabcc0b07a0 2023-11-26T07:23:17.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabd12b8640 1 --2- 172.21.15.31:0/2067979779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc0b07a0 0x7fabcc1619e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fabc80097c0 tx=0x7fabc801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad4aaf884f6c55e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabc3fff640 1 -- 172.21.15.31:0/2067979779 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc80ab600 con 0x7fabcc0b07a0 2023-11-26T07:23:17.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabc3fff640 1 -- 172.21.15.31:0/2067979779 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc80a7050 con 0x7fabcc0b07a0 2023-11-26T07:23:17.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabc3fff640 1 -- 172.21.15.31:0/2067979779 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc80ab980 con 0x7fabcc0b07a0 2023-11-26T07:23:17.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabd351c640 1 -- 172.21.15.31:0/2067979779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc0b07a0 msgr2=0x7fabcc1619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabd351c640 1 --2- 172.21.15.31:0/2067979779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc0b07a0 0x7fabcc1619e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fabc80097c0 tx=0x7fabc801c480 comp rx=0 tx=0).stop 2023-11-26T07:23:17.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabd351c640 1 -- 172.21.15.31:0/2067979779 shutdown_connections 2023-11-26T07:23:17.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabd351c640 1 --2- 172.21.15.31:0/2067979779 >> v2:172.21.15.97:3300/0 conn(0x7fabcc0b15f0 0x7fabcc0cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabd351c640 1 --2- 172.21.15.31:0/2067979779 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc0b07a0 0x7fabcc1619e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabd351c640 1 -- 172.21.15.31:0/2067979779 >> 172.21.15.31:0/2067979779 conn(0x7fabcc0b1d50 msgr2=0x7fabcc0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.869+0000 7fabd351c640 1 -- 172.21.15.31:0/2067979779 shutdown_connections 2023-11-26T07:23:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd351c640 1 -- 172.21.15.31:0/2067979779 wait complete. 2023-11-26T07:23:17.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd351c640 1 Processor -- start 2023-11-26T07:23:17.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd351c640 1 -- start start 2023-11-26T07:23:17.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd351c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabcc0b07a0 0x7fabcc159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd351c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabcc0b15f0 0x7fabcc158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd351c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc161f20 0x7fabcc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd351c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc159420 con 0x7fabcc0b07a0 2023-11-26T07:23:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd351c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc0b3120 con 0x7fabcc0b15f0 2023-11-26T07:23:17.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd351c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabcc158840 con 0x7fabcc161f20 2023-11-26T07:23:17.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd12b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabcc0b07a0 0x7fabcc159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd12b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabcc0b07a0 0x7fabcc159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35150/0 (socket says 172.21.15.31:35150) 2023-11-26T07:23:17.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd12b8640 1 -- 172.21.15.31:0/1868372569 learned_addr learned my addr 172.21.15.31:0/1868372569 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:17.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd1ab9640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc161f20 0x7fabcc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd0ab7640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabcc0b15f0 0x7fabcc158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd12b8640 1 -- 172.21.15.31:0/1868372569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc161f20 msgr2=0x7fabcc0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd12b8640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc161f20 0x7fabcc0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd12b8640 1 -- 172.21.15.31:0/1868372569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabcc0b15f0 msgr2=0x7fabcc158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:17.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd12b8640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabcc0b15f0 0x7fabcc158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:17.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd12b8640 1 -- 172.21.15.31:0/1868372569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8009420 con 0x7fabcc0b07a0 2023-11-26T07:23:17.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd12b8640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabcc0b07a0 0x7fabcc159f10 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fabc80ae040 tx=0x7fabc8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabc1ffb640 1 -- 172.21.15.31:0/1868372569 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc80b4430 con 0x7fabcc0b07a0 2023-11-26T07:23:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabc1ffb640 1 -- 172.21.15.31:0/1868372569 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc80b4590 con 0x7fabcc0b07a0 2023-11-26T07:23:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabc1ffb640 1 -- 172.21.15.31:0/1868372569 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc80b4940 con 0x7fabcc0b07a0 2023-11-26T07:23:17.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.873+0000 7fabd351c640 1 -- 172.21.15.31:0/1868372569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabcc0b2df0 con 0x7fabcc0b07a0 2023-11-26T07:23:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.877+0000 7fabd351c640 1 -- 172.21.15.31:0/1868372569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabcc159950 con 0x7fabcc0b07a0 2023-11-26T07:23:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.877+0000 7fabd351c640 1 -- 172.21.15.31:0/1868372569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabcc04e850 con 0x7fabcc0b07a0 2023-11-26T07:23:17.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.877+0000 7fabc1ffb640 1 -- 172.21.15.31:0/1868372569 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fabc80b2040 con 0x7fabcc0b07a0 2023-11-26T07:23:17.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.877+0000 7fabc1ffb640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fabb0066620 0x7fabb0068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:17.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.877+0000 7fabc1ffb640 1 -- 172.21.15.31:0/1868372569 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabc811e0b0 con 0x7fabcc0b07a0 2023-11-26T07:23:17.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.877+0000 7fabd0ab7640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fabb0066620 0x7fabb0068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:17.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.881+0000 7fabd0ab7640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fabb0066620 0x7fabb0068ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fabcc160950 tx=0x7fabbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:17.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:17.881+0000 7fabc1ffb640 1 -- 172.21.15.31:0/1868372569 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fabc80eb1b0 con 0x7fabcc0b07a0 2023-11-26T07:23:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.061+0000 7fabd351c640 1 -- 172.21.15.31:0/1868372569 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol66940", "target": ["mon-mgr", ""]}) v1 -- 0x7fabcc0ccee0 con 0x7fabb0066620 2023-11-26T07:23:18.098 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabc1ffb640 1 -- 172.21.15.31:0/1868372569 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7fabbc001d80 con 0x7fabb0066620 2023-11-26T07:23:18.098 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103 2023-11-26T07:23:18.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 -- 172.21.15.31:0/1868372569 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fabb0066620 msgr2=0x7fabb0068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fabb0066620 0x7fabb0068ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fabcc160950 tx=0x7fabbc009040 comp rx=0 tx=0).stop 2023-11-26T07:23:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 -- 172.21.15.31:0/1868372569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabcc0b07a0 msgr2=0x7fabcc159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabcc0b07a0 0x7fabcc159f10 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fabc80ae040 tx=0x7fabc8019220 comp rx=0 tx=0).stop 2023-11-26T07:23:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 -- 172.21.15.31:0/1868372569 shutdown_connections 2023-11-26T07:23:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fabb0066620 0x7fabb0068ae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fabcc161f20 0x7fabcc0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fabcc0b15f0 0x7fabcc158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 --2- 172.21.15.31:0/1868372569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabcc0b07a0 0x7fabcc159f10 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 -- 172.21.15.31:0/1868372569 >> 172.21.15.31:0/1868372569 conn(0x7fabcc0b1d50 msgr2=0x7fabcc0c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:18.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 -- 172.21.15.31:0/1868372569 shutdown_connections 2023-11-26T07:23:18.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:18.097+0000 7fabd351c640 1 -- 172.21.15.31:0/1868372569 wait complete. 2023-11-26T07:23:18.119 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol66940 with 8 files each 1MB size under directory volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103 2023-11-26T07:23:18.120 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:18.120 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:18.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:18.129 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:23:18.204 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:23:18.204 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:23:18.204 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187805 s, 55.8 MB/s 2023-11-26T07:23:18.205 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:18.205 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:18.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:18.254 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:23:18.328 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:23:18.328 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:23:18.328 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179205 s, 58.5 MB/s 2023-11-26T07:23:18.329 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:18.329 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:18.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:18.378 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:23:18.448 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:23:18.448 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:23:18.448 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180822 s, 58.0 MB/s 2023-11-26T07:23:18.449 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:18.449 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:18.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:18.503 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:23:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:23:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:23:18.573 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182829 s, 57.4 MB/s 2023-11-26T07:23:18.574 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:18.574 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:18.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:18.626 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:23:18.694 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:23:18.694 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:23:18.694 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01622 s, 64.6 MB/s 2023-11-26T07:23:18.695 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:18.695 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:18.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:18.746 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:23:18.816 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:23:18.817 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:23:18.817 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181773 s, 57.7 MB/s 2023-11-26T07:23:18.818 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:18.818 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:18.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:18.874 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:23:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:23:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:23:18.950 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211232 s, 49.6 MB/s 2023-11-26T07:23:18.950 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:18.950 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:19.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:19.007 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2023-11-26T07:23:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-11-26T07:23:19.079 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-11-26T07:23:19.080 INFO:teuthology.orchestra.run.smithi031.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212402 s, 49.4 MB/s 2023-11-26T07:23:19.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol66940 subvol_snap62510 2023-11-26T07:23:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad25ddf640 1 Processor -- start 2023-11-26T07:23:19.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad25ddf640 1 -- start start 2023-11-26T07:23:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad25ddf640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fad200ce5d0 0x7fad2015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad25ddf640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad20159630 con 0x7fad200cb9b0 2023-11-26T07:23:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad25ddf640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad200cd1d0 con 0x7fad2006e3f0 2023-11-26T07:23:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad25ddf640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2006e860 con 0x7fad200ce5d0 2023-11-26T07:23:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1f7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fad200cb9b0 0x7fad200cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56316/0 (socket says 172.21.15.31:56316) 2023-11-26T07:23:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1f7fe640 1 -- 172.21.15.31:0/2402714342 learned_addr learned my addr 172.21.15.31:0/2402714342 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:19.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1ffff640 1 --2- 172.21.15.31:0/2402714342 >> v2:172.21.15.97:3300/0 conn(0x7fad200ce5d0 0x7fad2015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303450634 0 0) 0x7fad20159630 con 0x7fad200cb9b0 2023-11-26T07:23:19.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad08003530 con 0x7fad200cb9b0 2023-11-26T07:23:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667682127 0 0) 0x7fad200cd1d0 con 0x7fad2006e3f0 2023-11-26T07:23:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad20159630 con 0x7fad2006e3f0 2023-11-26T07:23:19.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4077764465 0 0) 0x7fad08003530 con 0x7fad200cb9b0 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 >> v2:172.21.15.97:3300/0 conn(0x7fad200ce5d0 msgr2=0x7fad2015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 --2- 172.21.15.31:0/2402714342 >> v2:172.21.15.97:3300/0 conn(0x7fad200ce5d0 0x7fad2015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 >> v1:172.21.15.97:6789/0 conn(0x7fad2006e3f0 legacy=0x7fad2015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200cd4b0 con 0x7fad200cb9b0 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fad10001680 con 0x7fad200cb9b0 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1ffff640 1 --2- 172.21.15.31:0/2402714342 >> v2:172.21.15.97:3300/0 conn(0x7fad200ce5d0 0x7fad2015bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 >> v1:172.21.15.31:6789/0 conn(0x7fad200cb9b0 legacy=0x7fad200cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad08003b60 0x7fad08003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080047d0 0x7fad08004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad08005190 0x7fad0800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2006e860 con 0x7fad08003b60 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0800c060 con 0x7fad080047d0 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0800c220 con 0x7fad08005190 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1f7fe640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad08003b60 0x7fad08003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1effd640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080047d0 0x7fad08004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1ffff640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad08005190 0x7fad0800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1f7fe640 1 -- 172.21.15.31:0/2402714342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad08005190 msgr2=0x7fad0800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1f7fe640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad08005190 0x7fad0800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1f7fe640 1 -- 172.21.15.31:0/2402714342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080047d0 msgr2=0x7fad08004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1f7fe640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080047d0 0x7fad08004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1f7fe640 1 -- 172.21.15.31:0/2402714342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad08003530 con 0x7fad08003b60 2023-11-26T07:23:19.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1effd640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad080047d0 0x7fad08004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:19.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.273+0000 7fad1ffff640 1 -- 172.21.15.31:0/2402714342 reap_dead start 2023-11-26T07:23:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad1f7fe640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad08003b60 0x7fad08003f60 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fad10007fa0 tx=0x7fad1001cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8d0bb199c236d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10005ae0 con 0x7fad08003b60 2023-11-26T07:23:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad10005c40 con 0x7fad08003b60 2023-11-26T07:23:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad1e7fc640 1 -- 172.21.15.31:0/2402714342 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10007290 con 0x7fad08003b60 2023-11-26T07:23:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad25ddf640 1 -- 172.21.15.31:0/2402714342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad08003b60 msgr2=0x7fad08003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad25ddf640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad08003b60 0x7fad08003f60 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fad10007fa0 tx=0x7fad1001cb00 comp rx=0 tx=0).stop 2023-11-26T07:23:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad25ddf640 1 -- 172.21.15.31:0/2402714342 shutdown_connections 2023-11-26T07:23:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad25ddf640 1 --2- 172.21.15.31:0/2402714342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad08003b60 0x7fad08003f60 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad25ddf640 1 -- 172.21.15.31:0/2402714342 >> 172.21.15.31:0/2402714342 conn(0x7fad200b0cb0 msgr2=0x7fad200b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad25ddf640 1 -- 172.21.15.31:0/2402714342 shutdown_connections 2023-11-26T07:23:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.277+0000 7fad25ddf640 1 -- 172.21.15.31:0/2402714342 wait complete. 2023-11-26T07:23:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad25ddf640 1 Processor -- start 2023-11-26T07:23:19.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad25ddf640 1 -- start start 2023-11-26T07:23:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad25ddf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad08003b60 0x7fad2014eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad25ddf640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad2014f3e0 0x7fad2009be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad25ddf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad200afdf0 0x7fad200b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad25ddf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200b07e0 con 0x7fad200afdf0 2023-11-26T07:23:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad25ddf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2014f0d0 con 0x7fad08003b60 2023-11-26T07:23:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad25ddf640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2009d070 con 0x7fad2014f3e0 2023-11-26T07:23:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad200afdf0 0x7fad200b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad200afdf0 0x7fad200b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35160/0 (socket says 172.21.15.31:35160) 2023-11-26T07:23:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1ffff640 1 -- 172.21.15.31:0/39356109 learned_addr learned my addr 172.21.15.31:0/39356109 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1f7fe640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad08003b60 0x7fad2014eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1effd640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad2014f3e0 0x7fad2009be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1ffff640 1 -- 172.21.15.31:0/39356109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad2014f3e0 msgr2=0x7fad2009be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1ffff640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad2014f3e0 0x7fad2009be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1ffff640 1 -- 172.21.15.31:0/39356109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad08003b60 msgr2=0x7fad2014eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1ffff640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad08003b60 0x7fad2014eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1ffff640 1 -- 172.21.15.31:0/39356109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2009c7d0 con 0x7fad200afdf0 2023-11-26T07:23:19.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad1ffff640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad200afdf0 0x7fad200b02a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fad200b43e0 tx=0x7fad1800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad07fff640 1 -- 172.21.15.31:0/39356109 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1800e040 con 0x7fad200afdf0 2023-11-26T07:23:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad07fff640 1 -- 172.21.15.31:0/39356109 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1800f040 con 0x7fad200afdf0 2023-11-26T07:23:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad07fff640 1 -- 172.21.15.31:0/39356109 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18003be0 con 0x7fad200afdf0 2023-11-26T07:23:19.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.281+0000 7fad25ddf640 1 -- 172.21.15.31:0/39356109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2009cab0 con 0x7fad200afdf0 2023-11-26T07:23:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.285+0000 7fad25ddf640 1 -- 172.21.15.31:0/39356109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad20156730 con 0x7fad200afdf0 2023-11-26T07:23:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.285+0000 7fad25ddf640 1 -- 172.21.15.31:0/39356109 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad200671f0 con 0x7fad200afdf0 2023-11-26T07:23:19.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.285+0000 7fad07fff640 1 -- 172.21.15.31:0/39356109 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fad18015430 con 0x7fad200afdf0 2023-11-26T07:23:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.285+0000 7fad07fff640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fad0c066620 0x7fad0c068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.285+0000 7fad07fff640 1 -- 172.21.15.31:0/39356109 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad18084c00 con 0x7fad200afdf0 2023-11-26T07:23:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.285+0000 7fad1f7fe640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fad0c066620 0x7fad0c068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.289+0000 7fad1f7fe640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fad0c066620 0x7fad0c068ae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fad100ab000 tx=0x7fad100b1040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:19.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.289+0000 7fad07fff640 1 -- 172.21.15.31:0/39356109 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fad18051e40 con 0x7fad200afdf0 2023-11-26T07:23:19.477 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.473+0000 7fad25ddf640 1 -- 172.21.15.31:0/39356109 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol66940", "snap_name": "subvol_snap62510", "target": ["mon-mgr", ""]}) v1 -- 0x7fad2006efa0 con 0x7fad0c066620 2023-11-26T07:23:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad07fff640 1 -- 172.21.15.31:0/39356109 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fad2006efa0 con 0x7fad0c066620 2023-11-26T07:23:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 -- 172.21.15.31:0/39356109 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fad0c066620 msgr2=0x7fad0c068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fad0c066620 0x7fad0c068ae0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fad100ab000 tx=0x7fad100b1040 comp rx=0 tx=0).stop 2023-11-26T07:23:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 -- 172.21.15.31:0/39356109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad200afdf0 msgr2=0x7fad200b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad200afdf0 0x7fad200b02a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fad200b43e0 tx=0x7fad1800f580 comp rx=0 tx=0).stop 2023-11-26T07:23:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 -- 172.21.15.31:0/39356109 shutdown_connections 2023-11-26T07:23:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fad0c066620 0x7fad0c068ae0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad200afdf0 0x7fad200b02a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fad2014f3e0 0x7fad2009be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 --2- 172.21.15.31:0/39356109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad08003b60 0x7fad2014eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 -- 172.21.15.31:0/39356109 >> 172.21.15.31:0/39356109 conn(0x7fad200b0cb0 msgr2=0x7fad2015f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 -- 172.21.15.31:0/39356109 shutdown_connections 2023-11-26T07:23:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.505+0000 7fad25ddf640 1 -- 172.21.15.31:0/39356109 wait complete. 2023-11-26T07:23:19.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol66940 subvol_snap62510 subvol_clone94340 2023-11-26T07:23:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.673+0000 7fa7ff957640 1 Processor -- start 2023-11-26T07:23:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.673+0000 7fa7ff957640 1 -- start start 2023-11-26T07:23:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.673+0000 7fa7ff957640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa7f80cb9b0 0x7fa7f806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.673+0000 7fa7ff957640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cbd90 0x7fa7f815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.673+0000 7fa7ff957640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f8159630 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.673+0000 7fa7ff957640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f815bba0 con 0x7fa7f80cb9b0 2023-11-26T07:23:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.673+0000 7fa7ff957640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80cdd50 con 0x7fa7f80cbd90 2023-11-26T07:23:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7fd6f3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa7f80cb9b0 0x7fa7f806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7fdef4640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa7f80ce5d0 0x7fa7f815bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56326/0 (socket says 172.21.15.31:56326) 2023-11-26T07:23:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7fdef4640 1 -- 172.21.15.31:0/3920226317 learned_addr learned my addr 172.21.15.31:0/3920226317 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7fcef2640 1 --2- 172.21.15.31:0/3920226317 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cbd90 0x7fa7f815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7fd6f3640 1 -- 172.21.15.31:0/3920226317 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cbd90 msgr2=0x7fa7f815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7fd6f3640 1 --2- 172.21.15.31:0/3920226317 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cbd90 0x7fa7f815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7fd6f3640 1 -- 172.21.15.31:0/3920226317 >> v1:172.21.15.31:6789/0 conn(0x7fa7f80ce5d0 legacy=0x7fa7f815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7fd6f3640 1 -- 172.21.15.31:0/3920226317 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f815f6d0 con 0x7fa7f80cb9b0 2023-11-26T07:23:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7effff640 1 -- 172.21.15.31:0/3920226317 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119294054 0 0) 0x7fa7f8159630 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7fd6f3640 1 --2- 172.21.15.31:0/3920226317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cb9b0 0x7fa7f806e3f0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40097c0 tx=0x7fa7f401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3fcdfeaedb6a6b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7effff640 1 -- 172.21.15.31:0/3920226317 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40ab650 con 0x7fa7f80cb9b0 2023-11-26T07:23:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7effff640 1 -- 172.21.15.31:0/3920226317 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f40ab7b0 con 0x7fa7f80cb9b0 2023-11-26T07:23:19.681 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.677+0000 7fa7effff640 1 -- 172.21.15.31:0/3920226317 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40abb40 con 0x7fa7f80cb9b0 2023-11-26T07:23:19.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 -- 172.21.15.31:0/3920226317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cb9b0 msgr2=0x7fa7f806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 --2- 172.21.15.31:0/3920226317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cb9b0 0x7fa7f806e3f0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40097c0 tx=0x7fa7f401c450 comp rx=0 tx=0).stop 2023-11-26T07:23:19.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 -- 172.21.15.31:0/3920226317 shutdown_connections 2023-11-26T07:23:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 --2- 172.21.15.31:0/3920226317 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cbd90 0x7fa7f815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 --2- 172.21.15.31:0/3920226317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80cb9b0 0x7fa7f806e3f0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 -- 172.21.15.31:0/3920226317 >> 172.21.15.31:0/3920226317 conn(0x7fa7f80b0cb0 msgr2=0x7fa7f80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 -- 172.21.15.31:0/3920226317 shutdown_connections 2023-11-26T07:23:19.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 -- 172.21.15.31:0/3920226317 wait complete. 2023-11-26T07:23:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 Processor -- start 2023-11-26T07:23:19.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 -- start start 2023-11-26T07:23:19.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cb9b0 0x7fa7f80ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f80cbd90 0x7fa7f8145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80ce5d0 0x7fa7f80af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80ad670 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f80ad3b0 con 0x7fa7f80cbd90 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7ff957640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f8145db0 con 0x7fa7f80cb9b0 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fdef4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80ce5d0 0x7fa7f80af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fdef4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80ce5d0 0x7fa7f80af9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35180/0 (socket says 172.21.15.31:35180) 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fdef4640 1 -- 172.21.15.31:0/144777975 learned_addr learned my addr 172.21.15.31:0/144777975 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fd6f3640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cb9b0 0x7fa7f80ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fcef2640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f80cbd90 0x7fa7f8145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fdef4640 1 -- 172.21.15.31:0/144777975 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cb9b0 msgr2=0x7fa7f80ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fdef4640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cb9b0 0x7fa7f80ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fdef4640 1 -- 172.21.15.31:0/144777975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f80cbd90 msgr2=0x7fa7f8145870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fdef4640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f80cbd90 0x7fa7f8145870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fdef4640 1 -- 172.21.15.31:0/144777975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f4009420 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.681+0000 7fa7fd6f3640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cb9b0 0x7fa7f80ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7fdef4640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80ce5d0 0x7fa7f80af9f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c780 tx=0x7fa7f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7edffb640 1 -- 172.21.15.31:0/144777975 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000ce90 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7edffb640 1 -- 172.21.15.31:0/144777975 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f000a640 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7edffb640 1 -- 172.21.15.31:0/144777975 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000a9d0 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7ff957640 1 -- 172.21.15.31:0/144777975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f80b1e70 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7ff957640 1 -- 172.21.15.31:0/144777975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f80b2430 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7ff957640 1 -- 172.21.15.31:0/144777975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f80671f0 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.688 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7edffb640 1 -- 172.21.15.31:0/144777975 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fa7f000ab30 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7edffb640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7dc066650 0x7fa7dc068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:19.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7edffb640 1 -- 172.21.15.31:0/144777975 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7f0083550 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.685+0000 7fa7fd6f3640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7dc066650 0x7fa7dc068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:19.689 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.689+0000 7fa7fd6f3640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7dc066650 0x7fa7dc068b10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40af000 tx=0x7fa7f40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:19.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.689+0000 7fa7edffb640 1 -- 172.21.15.31:0/144777975 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7f0050750 con 0x7fa7f80ce5d0 2023-11-26T07:23:19.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.877+0000 7fa7ff957640 1 -- 172.21.15.31:0/144777975 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol66940", "snap_name": "subvol_snap62510", "target_sub_name": "subvol_clone94340", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7f815caf0 con 0x7fa7dc066650 2023-11-26T07:23:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7edffb640 1 -- 172.21.15.31:0/144777975 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa7f40aba20 con 0x7fa7dc066650 2023-11-26T07:23:19.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 -- 172.21.15.31:0/144777975 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7dc066650 msgr2=0x7fa7dc068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7dc066650 0x7fa7dc068b10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40af000 tx=0x7fa7f40a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:19.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 -- 172.21.15.31:0/144777975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80ce5d0 msgr2=0x7fa7f80af9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80ce5d0 0x7fa7f80af9f0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000c780 tx=0x7fa7f000cc40 comp rx=0 tx=0).stop 2023-11-26T07:23:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 -- 172.21.15.31:0/144777975 shutdown_connections 2023-11-26T07:23:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa7dc066650 0x7fa7dc068b10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7f80ce5d0 0x7fa7f80af9f0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa7f80cbd90 0x7fa7f8145870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 --2- 172.21.15.31:0/144777975 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa7f80cb9b0 0x7fa7f80ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 -- 172.21.15.31:0/144777975 >> 172.21.15.31:0/144777975 conn(0x7fa7f80b0cb0 msgr2=0x7fa7f815e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 -- 172.21.15.31:0/144777975 shutdown_connections 2023-11-26T07:23:19.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:19.957+0000 7fa7ff957640 1 -- 172.21.15.31:0/144777975 wait complete. 2023-11-26T07:23:19.963 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:19.961+0000 7f1cb48b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:19.963 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:19.961+0000 7f1cb48b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:19.963 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:19.961+0000 7f1cb48b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:19.964 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:19.961+0000 7f1cb48b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:19.964 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:19.961+0000 7f1cb48b9640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:19.981 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:19.977+0000 7f1cb50ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:19.981 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:19.977+0000 7f1cb50ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:19.981 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:19.977+0000 7f1cb50ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:19.982 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:19.977+0000 7f1cb50ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:19.982 INFO:tasks.ceph.mgr.y.smithi031.stderr:2023-11-26T07:23:19.977+0000 7f1cb50ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-26T07:23:19.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone94340 2023-11-26T07:23:20.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.157+0000 7fe5195f5640 1 Processor -- start 2023-11-26T07:23:20.161 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.157+0000 7fe5195f5640 1 -- start start 2023-11-26T07:23:20.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe5195f5640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe51415d8f0 0x7fe51415e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe5195f5640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51415ecc0 con 0x7fe5140ced30 2023-11-26T07:23:20.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe5195f5640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5140cdfb0 con 0x7fe5140d0510 2023-11-26T07:23:20.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe5195f5640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5140d4420 con 0x7fe51415d8f0 2023-11-26T07:23:20.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe512ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe51415d8f0 0x7fe51415e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.162 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe5127fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fe5140ced30 0x7fe5140cfe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:56340/0 (socket says 172.21.15.31:56340) 2023-11-26T07:23:20.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe5127fc640 1 -- 172.21.15.31:0/3417494833 learned_addr learned my addr 172.21.15.31:0/3417494833 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:20.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe512ffd640 1 -- 172.21.15.31:0/3417494833 >> v1:172.21.15.97:6789/0 conn(0x7fe5140d0510 legacy=0x7fe5140d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe512ffd640 1 -- 172.21.15.31:0/3417494833 >> v1:172.21.15.31:6789/0 conn(0x7fe5140ced30 legacy=0x7fe5140cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe512ffd640 1 -- 172.21.15.31:0/3417494833 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140d4140 con 0x7fe51415d8f0 2023-11-26T07:23:20.163 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.161+0000 7fe511ffb640 1 -- 172.21.15.31:0/3417494833 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037894767 0 0) 0x7fe51415ecc0 con 0x7fe5140ced30 2023-11-26T07:23:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe512ffd640 1 --2- 172.21.15.31:0/3417494833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51415d8f0 0x7fe51415e780 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe5040097f0 tx=0x7fe50401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b29fd2e56c3455 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe511ffb640 1 -- 172.21.15.31:0/3417494833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040ab610 con 0x7fe51415d8f0 2023-11-26T07:23:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe511ffb640 1 -- 172.21.15.31:0/3417494833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5040ab770 con 0x7fe51415d8f0 2023-11-26T07:23:20.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe511ffb640 1 -- 172.21.15.31:0/3417494833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040abb00 con 0x7fe51415d8f0 2023-11-26T07:23:20.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe5195f5640 1 -- 172.21.15.31:0/3417494833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51415d8f0 msgr2=0x7fe51415e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe5195f5640 1 --2- 172.21.15.31:0/3417494833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51415d8f0 0x7fe51415e780 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe5040097f0 tx=0x7fe50401c410 comp rx=0 tx=0).stop 2023-11-26T07:23:20.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe5195f5640 1 -- 172.21.15.31:0/3417494833 shutdown_connections 2023-11-26T07:23:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe5195f5640 1 --2- 172.21.15.31:0/3417494833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe51415d8f0 0x7fe51415e780 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe5195f5640 1 -- 172.21.15.31:0/3417494833 >> 172.21.15.31:0/3417494833 conn(0x7fe51409c750 msgr2=0x7fe5140d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe5195f5640 1 -- 172.21.15.31:0/3417494833 shutdown_connections 2023-11-26T07:23:20.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe5195f5640 1 -- 172.21.15.31:0/3417494833 wait complete. 2023-11-26T07:23:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe5195f5640 1 Processor -- start 2023-11-26T07:23:20.169 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.165+0000 7fe5195f5640 1 -- start start 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5195f5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140ced30 0x7fe5140cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5195f5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5140d0510 0x7fe514153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5195f5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe51415d8f0 0x7fe5140cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5195f5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140caf10 con 0x7fe5140ced30 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5195f5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140cd370 con 0x7fe5140d0510 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5195f5640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe514159850 con 0x7fe51415d8f0 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe512ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140ced30 0x7fe5140cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe512ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140ced30 0x7fe5140cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35252/0 (socket says 172.21.15.31:35252) 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe512ffd640 1 -- 172.21.15.31:0/738699369 learned_addr learned my addr 172.21.15.31:0/738699369 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5127fc640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5140d0510 0x7fe514153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5137fe640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe51415d8f0 0x7fe5140cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe512ffd640 1 -- 172.21.15.31:0/738699369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe51415d8f0 msgr2=0x7fe5140cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe512ffd640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe51415d8f0 0x7fe5140cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe512ffd640 1 -- 172.21.15.31:0/738699369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5140d0510 msgr2=0x7fe514153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe512ffd640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5140d0510 0x7fe514153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe512ffd640 1 -- 172.21.15.31:0/738699369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe504009420 con 0x7fe5140ced30 2023-11-26T07:23:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe512ffd640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140ced30 0x7fe5140cce30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fe5040ad040 tx=0x7fe50401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/738699369 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040b5810 con 0x7fe5140ced30 2023-11-26T07:23:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/738699369 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5040b59b0 con 0x7fe5140ced30 2023-11-26T07:23:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/738699369 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040b5d60 con 0x7fe5140ced30 2023-11-26T07:23:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5195f5640 1 -- 172.21.15.31:0/738699369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe514159b30 con 0x7fe5140ced30 2023-11-26T07:23:20.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5195f5640 1 -- 172.21.15.31:0/738699369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5140cb1b0 con 0x7fe5140ced30 2023-11-26T07:23:20.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.169+0000 7fe5195f5640 1 -- 172.21.15.31:0/738699369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51404e850 con 0x7fe5140ced30 2023-11-26T07:23:20.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.173+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/738699369 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 16) v1 ==== 86005+0+0 (secure 0 0 0) 0x7fe5040bf410 con 0x7fe5140ced30 2023-11-26T07:23:20.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.173+0000 7fe4fb7fe640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066620 0x7fe500068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:20.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.173+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/738699369 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe50411ee10 con 0x7fe5140ced30 2023-11-26T07:23:20.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.173+0000 7fe5127fc640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066620 0x7fe500068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:20.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.173+0000 7fe5127fc640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066620 0x7fe500068ae0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe5140b5cf0 tx=0x7fe508007240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:20.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.177+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/738699369 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe504122030 con 0x7fe5140ced30 2023-11-26T07:23:20.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:20.381+0000 7fe5195f5640 1 -- 172.21.15.31:0/738699369 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone94340", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5140b29b0 con 0x7fe500066620 2023-11-26T07:23:21.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:21.273+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/738699369 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fe5040ec540 con 0x7fe5140ced30 2023-11-26T07:23:24.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.889+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/738699369 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7fe5080073d0 con 0x7fe500066620 2023-11-26T07:23:24.893 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-11-26T07:23:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete" 2023-11-26T07:23:24.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:24.893 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:24.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 -- 172.21.15.31:0/738699369 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066620 msgr2=0x7fe500068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066620 0x7fe500068ae0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe5140b5cf0 tx=0x7fe508007240 comp rx=0 tx=0).stop 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 -- 172.21.15.31:0/738699369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140ced30 msgr2=0x7fe5140cce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140ced30 0x7fe5140cce30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fe5040ad040 tx=0x7fe50401ac50 comp rx=0 tx=0).stop 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 -- 172.21.15.31:0/738699369 shutdown_connections 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fe500066620 0x7fe500068ae0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fe51415d8f0 0x7fe5140cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5140d0510 0x7fe514153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 --2- 172.21.15.31:0/738699369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5140ced30 0x7fe5140cce30 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 -- 172.21.15.31:0/738699369 >> 172.21.15.31:0/738699369 conn(0x7fe51409c750 msgr2=0x7fe5140d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 -- 172.21.15.31:0/738699369 shutdown_connections 2023-11-26T07:23:24.897 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:24.893+0000 7fe5195f5640 1 -- 172.21.15.31:0/738699369 wait complete. 2023-11-26T07:23:24.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:24.924 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone94340/.meta) 2023-11-26T07:23:24.947 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-11-26T07:23:24.947 INFO:teuthology.orchestra.run.smithi031.stdout:version = 2 2023-11-26T07:23:24.947 INFO:teuthology.orchestra.run.smithi031.stdout:type = clone 2023-11-26T07:23:24.947 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c 2023-11-26T07:23:24.948 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-11-26T07:23:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:24.948 INFO:teuthology.orchestra.run.smithi031.stdout:[source] 2023-11-26T07:23:24.948 INFO:teuthology.orchestra.run.smithi031.stdout:volume = cephfs 2023-11-26T07:23:24.948 INFO:teuthology.orchestra.run.smithi031.stdout:subvolume = subvol66940 2023-11-26T07:23:24.948 INFO:teuthology.orchestra.run.smithi031.stdout:snapshot = subvol_snap62510 2023-11-26T07:23:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:24.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol66940 2023-11-26T07:23:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.137+0000 7f59bf69c640 1 Processor -- start 2023-11-26T07:23:25.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.137+0000 7f59bf69c640 1 -- start start 2023-11-26T07:23:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bf69c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80cded0 0x7f59b813f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bf69c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f59b813f7a0 0x7f59b80cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bf69c640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b813cf40 con 0x7f59b8154180 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bf69c640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b813fc10 con 0x7f59b813f7a0 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bf69c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b80cf0a0 con 0x7f59b80cded0 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bd438640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f59b8154180 0x7f59b815aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57700/0 (socket says 172.21.15.31:57700) 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bd438640 1 -- 172.21.15.31:0/4213630194 learned_addr learned my addr 172.21.15.31:0/4213630194 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bdc39640 1 --2- 172.21.15.31:0/4213630194 >> v2:172.21.15.31:3300/0 conn(0x7f59b813f7a0 0x7f59b80cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bcc37640 1 --2- 172.21.15.31:0/4213630194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80cded0 0x7f59b813f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59affff640 1 -- 172.21.15.31:0/4213630194 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216071341 0 0) 0x7f59b813cf40 con 0x7f59b8154180 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59affff640 1 -- 172.21.15.31:0/4213630194 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59a4003530 con 0x7f59b8154180 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bdc39640 1 -- 172.21.15.31:0/4213630194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80cded0 msgr2=0x7f59b813f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bdc39640 1 --2- 172.21.15.31:0/4213630194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80cded0 0x7f59b813f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bdc39640 1 -- 172.21.15.31:0/4213630194 >> v1:172.21.15.31:6789/0 conn(0x7f59b8154180 legacy=0x7f59b815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.141+0000 7f59bdc39640 1 -- 172.21.15.31:0/4213630194 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b81400e0 con 0x7f59b813f7a0 2023-11-26T07:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bdc39640 1 --2- 172.21.15.31:0/4213630194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b813f7a0 0x7f59b80cbbb0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009dc0 tx=0x7f59b001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8ac7f9e2748756 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59affff640 1 -- 172.21.15.31:0/4213630194 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00a5770 con 0x7f59b813f7a0 2023-11-26T07:23:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59affff640 1 -- 172.21.15.31:0/4213630194 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b00a58d0 con 0x7f59b813f7a0 2023-11-26T07:23:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59affff640 1 -- 172.21.15.31:0/4213630194 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00a5c60 con 0x7f59b813f7a0 2023-11-26T07:23:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bf69c640 1 -- 172.21.15.31:0/4213630194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b813f7a0 msgr2=0x7f59b80cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bf69c640 1 --2- 172.21.15.31:0/4213630194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b813f7a0 0x7f59b80cbbb0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009dc0 tx=0x7f59b001c930 comp rx=0 tx=0).stop 2023-11-26T07:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bf69c640 1 -- 172.21.15.31:0/4213630194 shutdown_connections 2023-11-26T07:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bf69c640 1 --2- 172.21.15.31:0/4213630194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b813f7a0 0x7f59b80cbbb0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bf69c640 1 --2- 172.21.15.31:0/4213630194 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b80cded0 0x7f59b813f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bf69c640 1 -- 172.21.15.31:0/4213630194 >> 172.21.15.31:0/4213630194 conn(0x7f59b8071d80 msgr2=0x7f59b806a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bf69c640 1 -- 172.21.15.31:0/4213630194 shutdown_connections 2023-11-26T07:23:25.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bf69c640 1 -- 172.21.15.31:0/4213630194 wait complete. 2023-11-26T07:23:25.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bf69c640 1 Processor -- start 2023-11-26T07:23:25.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.145+0000 7f59bf69c640 1 -- start start 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bf69c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80cded0 0x7f59b813bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bf69c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b813f7a0 0x7f59b8157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bf69c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b8154180 0x7f59b81581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bf69c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b80b5460 con 0x7f59b80cded0 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bf69c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8159c90 con 0x7f59b813f7a0 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bf69c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8159f70 con 0x7f59b8154180 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bd438640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80cded0 0x7f59b813bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bd438640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80cded0 0x7f59b813bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35714/0 (socket says 172.21.15.31:35714) 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bd438640 1 -- 172.21.15.31:0/2273869735 learned_addr learned my addr 172.21.15.31:0/2273869735 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:25.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bdc39640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b8154180 0x7f59b81581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bcc37640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b813f7a0 0x7f59b8157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bd438640 1 -- 172.21.15.31:0/2273869735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b8154180 msgr2=0x7f59b81581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bd438640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b8154180 0x7f59b81581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bd438640 1 -- 172.21.15.31:0/2273869735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b813f7a0 msgr2=0x7f59b8157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bd438640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b813f7a0 0x7f59b8157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bd438640 1 -- 172.21.15.31:0/2273869735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b0009a70 con 0x7f59b80cded0 2023-11-26T07:23:25.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bd438640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80cded0 0x7f59b813bd40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f59b400c420 tx=0x7f59b400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59adffb640 1 -- 172.21.15.31:0/2273869735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4005cf0 con 0x7f59b80cded0 2023-11-26T07:23:25.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59adffb640 1 -- 172.21.15.31:0/2273869735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b4005e50 con 0x7f59b80cded0 2023-11-26T07:23:25.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59adffb640 1 -- 172.21.15.31:0/2273869735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40072d0 con 0x7f59b80cded0 2023-11-26T07:23:25.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bf69c640 1 -- 172.21.15.31:0/2273869735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b815a250 con 0x7f59b80cded0 2023-11-26T07:23:25.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bf69c640 1 -- 172.21.15.31:0/2273869735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b813c280 con 0x7f59b80cded0 2023-11-26T07:23:25.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.149+0000 7f59bf69c640 1 -- 172.21.15.31:0/2273869735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b804e850 con 0x7f59b80cded0 2023-11-26T07:23:25.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.153+0000 7f59adffb640 1 -- 172.21.15.31:0/2273869735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f59b4007430 con 0x7f59b80cded0 2023-11-26T07:23:25.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.153+0000 7f59adffb640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f599c066740 0x7f599c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.153+0000 7f59adffb640 1 -- 172.21.15.31:0/2273869735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59b4083800 con 0x7f59b80cded0 2023-11-26T07:23:25.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.153+0000 7f59bcc37640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f599c066740 0x7f599c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.153+0000 7f59bcc37640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f599c066740 0x7f599c068c00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009b60 tx=0x7f59a8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.159 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.157+0000 7f59adffb640 1 -- 172.21.15.31:0/2273869735 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f59b4050a30 con 0x7f59b80cded0 2023-11-26T07:23:25.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.341+0000 7f59bf69c640 1 -- 172.21.15.31:0/2273869735 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol66940", "target": ["mon-mgr", ""]}) v1 -- 0x7f59b80ced30 con 0x7f599c066740 2023-11-26T07:23:25.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59adffb640 1 -- 172.21.15.31:0/2273869735 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+67 (secure 0 0 0) 0x7f59a8011ac0 con 0x7f599c066740 2023-11-26T07:23:25.354 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 -- 172.21.15.31:0/2273869735 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f599c066740 msgr2=0x7f599c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f599c066740 0x7f599c068c00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009b60 tx=0x7f59a8009190 comp rx=0 tx=0).stop 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 -- 172.21.15.31:0/2273869735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80cded0 msgr2=0x7f59b813bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80cded0 0x7f59b813bd40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f59b400c420 tx=0x7f59b400c8e0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 -- 172.21.15.31:0/2273869735 shutdown_connections 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f599c066740 0x7f599c068c00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f59b8154180 0x7f59b81581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b813f7a0 0x7f59b8157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 --2- 172.21.15.31:0/2273869735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59b80cded0 0x7f59b813bd40 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 -- 172.21.15.31:0/2273869735 >> 172.21.15.31:0/2273869735 conn(0x7f59b8071d80 msgr2=0x7f59b806ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 -- 172.21.15.31:0/2273869735 shutdown_connections 2023-11-26T07:23:25.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.353+0000 7f59bf69c640 1 -- 172.21.15.31:0/2273869735 wait complete. 2023-11-26T07:23:25.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone94340 2023-11-26T07:23:25.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4bae9c640 1 Processor -- start 2023-11-26T07:23:25.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4bae9c640 1 -- start start 2023-11-26T07:23:25.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4bae9c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4b4159f00 0x7ff4b40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4bae9c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40ae680 0x7ff4b40ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4bae9c640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b4072950 con 0x7ff4b40cd3e0 2023-11-26T07:23:25.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4bae9c640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b406e2b0 con 0x7ff4b4159f00 2023-11-26T07:23:25.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4bae9c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b406c6f0 con 0x7ff4b40ae680 2023-11-26T07:23:25.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4b3fff640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40ae680 0x7ff4b40ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4b8c38640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff4b4159f00 0x7ff4b40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4b9439640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7ff4b40cd3e0 0x7ff4b415d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:41192/0 (socket says 172.21.15.31:41192) 2023-11-26T07:23:25.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4b9439640 1 -- 172.21.15.31:0/3530760982 learned_addr learned my addr 172.21.15.31:0/3530760982 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:25.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4b8c38640 1 -- 172.21.15.31:0/3530760982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40ae680 msgr2=0x7ff4b40ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:25.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4b8c38640 1 --2- 172.21.15.31:0/3530760982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40ae680 0x7ff4b40ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4b8c38640 1 -- 172.21.15.31:0/3530760982 >> v1:172.21.15.97:6789/0 conn(0x7ff4b40cd3e0 legacy=0x7ff4b415d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4b8c38640 1 -- 172.21.15.31:0/3530760982 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b406e550 con 0x7ff4b4159f00 2023-11-26T07:23:25.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.521+0000 7ff4b3fff640 1 --2- 172.21.15.31:0/3530760982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40ae680 0x7ff4b40ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:25.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4b8c38640 1 --2- 172.21.15.31:0/3530760982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b4159f00 0x7ff4b40cba20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80097c0 tx=0x7ff4a801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e6f49251e9b7157 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4b37fe640 1 -- 172.21.15.31:0/3530760982 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80ab650 con 0x7ff4b4159f00 2023-11-26T07:23:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4b37fe640 1 -- 172.21.15.31:0/3530760982 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a80ab7b0 con 0x7ff4b4159f00 2023-11-26T07:23:25.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4b37fe640 1 -- 172.21.15.31:0/3530760982 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80abb40 con 0x7ff4b4159f00 2023-11-26T07:23:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4bae9c640 1 -- 172.21.15.31:0/3530760982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b4159f00 msgr2=0x7ff4b40cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4bae9c640 1 --2- 172.21.15.31:0/3530760982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b4159f00 0x7ff4b40cba20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80097c0 tx=0x7ff4a801c450 comp rx=0 tx=0).stop 2023-11-26T07:23:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4bae9c640 1 -- 172.21.15.31:0/3530760982 shutdown_connections 2023-11-26T07:23:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4bae9c640 1 --2- 172.21.15.31:0/3530760982 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40ae680 0x7ff4b40ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4bae9c640 1 --2- 172.21.15.31:0/3530760982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b4159f00 0x7ff4b40cba20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4bae9c640 1 -- 172.21.15.31:0/3530760982 >> 172.21.15.31:0/3530760982 conn(0x7ff4b40b0cb0 msgr2=0x7ff4b40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4bae9c640 1 -- 172.21.15.31:0/3530760982 shutdown_connections 2023-11-26T07:23:25.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.525+0000 7ff4bae9c640 1 -- 172.21.15.31:0/3530760982 wait complete. 2023-11-26T07:23:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4bae9c640 1 Processor -- start 2023-11-26T07:23:25.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4bae9c640 1 -- start start 2023-11-26T07:23:25.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4bae9c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4b40ae680 0x7ff4b41400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4bae9c640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40cd3e0 0x7ff4b40b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4bae9c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b4159f00 0x7ff4b40b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4bae9c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40c9bf0 con 0x7ff4b40ae680 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4bae9c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40c8d40 con 0x7ff4b4159f00 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4bae9c640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40b3e60 con 0x7ff4b40cd3e0 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b8c38640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4b40ae680 0x7ff4b41400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b8c38640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4b40ae680 0x7ff4b41400e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35720/0 (socket says 172.21.15.31:35720) 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b8c38640 1 -- 172.21.15.31:0/1598836000 learned_addr learned my addr 172.21.15.31:0/1598836000 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b9439640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b4159f00 0x7ff4b40b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b3fff640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40cd3e0 0x7ff4b40b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b8c38640 1 -- 172.21.15.31:0/1598836000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40cd3e0 msgr2=0x7ff4b40b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b8c38640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40cd3e0 0x7ff4b40b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b8c38640 1 -- 172.21.15.31:0/1598836000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b4159f00 msgr2=0x7ff4b40b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b8c38640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b4159f00 0x7ff4b40b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b8c38640 1 -- 172.21.15.31:0/1598836000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a0009420 con 0x7ff4b40ae680 2023-11-26T07:23:25.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b8c38640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4b40ae680 0x7ff4b41400e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80ad040 tx=0x7ff4a801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b17fa640 1 -- 172.21.15.31:0/1598836000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80bf610 con 0x7ff4b40ae680 2023-11-26T07:23:25.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b17fa640 1 -- 172.21.15.31:0/1598836000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a80bf770 con 0x7ff4b40ae680 2023-11-26T07:23:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4b17fa640 1 -- 172.21.15.31:0/1598836000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80bfb20 con 0x7ff4b40ae680 2023-11-26T07:23:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4bae9c640 1 -- 172.21.15.31:0/1598836000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a8009420 con 0x7ff4b40ae680 2023-11-26T07:23:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.529+0000 7ff4bae9c640 1 -- 172.21.15.31:0/1598836000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b40c9250 con 0x7ff4b40ae680 2023-11-26T07:23:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.533+0000 7ff4bae9c640 1 -- 172.21.15.31:0/1598836000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40671f0 con 0x7ff4b40ae680 2023-11-26T07:23:25.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.533+0000 7ff4b17fa640 1 -- 172.21.15.31:0/1598836000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ff4a80b5400 con 0x7ff4b40ae680 2023-11-26T07:23:25.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.533+0000 7ff4b17fa640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff498066740 0x7ff498068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:25.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.533+0000 7ff4b17fa640 1 -- 172.21.15.31:0/1598836000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4a811ef70 con 0x7ff4b40ae680 2023-11-26T07:23:25.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.533+0000 7ff4b3fff640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff498066740 0x7ff498068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:25.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.533+0000 7ff4b3fff640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff498066740 0x7ff498068c00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40b3c60 tx=0x7ff4a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:25.540 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.537+0000 7ff4b17fa640 1 -- 172.21.15.31:0/1598836000 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff4a80ec060 con 0x7ff4b40ae680 2023-11-26T07:23:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.721+0000 7ff4bae9c640 1 -- 172.21.15.31:0/1598836000 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone94340", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4b40b23f0 con 0x7ff498066740 2023-11-26T07:23:25.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.725+0000 7ff4b17fa640 1 -- 172.21.15.31:0/1598836000 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+73 (secure 0 0 0) 0x7ff4b40b23f0 con 0x7ff498066740 2023-11-26T07:23:25.730 INFO:teuthology.orchestra.run.smithi031.stdout:/volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c 2023-11-26T07:23:25.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 -- 172.21.15.31:0/1598836000 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff498066740 msgr2=0x7ff498068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff498066740 0x7ff498068c00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40b3c60 tx=0x7ff4a0008040 comp rx=0 tx=0).stop 2023-11-26T07:23:25.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 -- 172.21.15.31:0/1598836000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4b40ae680 msgr2=0x7ff4b41400e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:25.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4b40ae680 0x7ff4b41400e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80ad040 tx=0x7ff4a801ac90 comp rx=0 tx=0).stop 2023-11-26T07:23:25.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 -- 172.21.15.31:0/1598836000 shutdown_connections 2023-11-26T07:23:25.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff498066740 0x7ff498068c00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4b4159f00 0x7ff4b40b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff4b40cd3e0 0x7ff4b40b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 --2- 172.21.15.31:0/1598836000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4b40ae680 0x7ff4b41400e0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:25.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 -- 172.21.15.31:0/1598836000 >> 172.21.15.31:0/1598836000 conn(0x7ff4b40b0cb0 msgr2=0x7ff4b4151bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:25.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 -- 172.21.15.31:0/1598836000 shutdown_connections 2023-11-26T07:23:25.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:25.729+0000 7ff4bae9c640 1 -- 172.21.15.31:0/1598836000 wait complete. 2023-11-26T07:23:25.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:25.751 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510) 2023-11-26T07:23:25.768 INFO:teuthology.orchestra.run.smithi031.stdout:9 2023-11-26T07:23:25.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:25.769 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c) 2023-11-26T07:23:25.841 INFO:teuthology.orchestra.run.smithi031.stdout:9 2023-11-26T07:23:25.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone94340 2023-11-26T07:23:26.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.033+0000 7f1e013f8640 1 Processor -- start 2023-11-26T07:23:26.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.033+0000 7f1e013f8640 1 -- start start 2023-11-26T07:23:26.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.033+0000 7f1e013f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cc460 0x7f1dfc0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.033+0000 7f1e013f8640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc0b2710 con 0x7f1dfc0cdfb0 2023-11-26T07:23:26.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.033+0000 7f1e013f8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc0b29f0 con 0x7f1dfc15a710 2023-11-26T07:23:26.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.033+0000 7f1e013f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0cca80 con 0x7f1dfc0cc460 2023-11-26T07:23:26.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.033+0000 7f1dfaffd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1dfc0cdfb0 0x7f1dfc0ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57714/0 (socket says 172.21.15.31:57714) 2023-11-26T07:23:26.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.033+0000 7f1dfaffd640 1 -- 172.21.15.31:0/2393490325 learned_addr learned my addr 172.21.15.31:0/2393490325 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:26.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfb7fe640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cc460 0x7f1dfc0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943343007 0 0) 0x7f1dfc0b2710 con 0x7f1dfc0cdfb0 2023-11-26T07:23:26.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1de8005740 con 0x7f1dfc0cdfb0 2023-11-26T07:23:26.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588227152 0 0) 0x7f1dfc0b29f0 con 0x7f1dfc15a710 2023-11-26T07:23:26.040 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dfc0b2710 con 0x7f1dfc15a710 2023-11-26T07:23:26.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 383511961 0 0) 0x7f1de8005740 con 0x7f1dfc0cdfb0 2023-11-26T07:23:26.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cc460 msgr2=0x7f1dfc0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:26.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cc460 0x7f1dfc0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 >> v1:172.21.15.97:6789/0 conn(0x7f1dfc15a710 legacy=0x7f1dfc0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc0b2cd0 con 0x7f1dfc0cdfb0 2023-11-26T07:23:26.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f1dec001680 con 0x7f1dfc0cdfb0 2023-11-26T07:23:26.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 >> v1:172.21.15.31:6789/0 conn(0x7f1dfc0cdfb0 legacy=0x7f1dfc0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8005d50 0x7f1de8006150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cc460 0x7f1de80069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1de8006f20 0x7f1de80073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0cca80 con 0x7f1de8006f20 2023-11-26T07:23:26.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de800dc80 con 0x7f1de8005d50 2023-11-26T07:23:26.042 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de800de40 con 0x7f1dfc0cc460 2023-11-26T07:23:26.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1df3fff640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cc460 0x7f1de80069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1dfaffd640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8005d50 0x7f1de8006150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1df3fff640 1 -- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8005d50 msgr2=0x7f1de8006150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:26.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1df3fff640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1de8005d50 0x7f1de8006150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1df3fff640 1 -- 172.21.15.31:0/2393490325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1de8006f20 msgr2=0x7f1de80073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:26.043 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1df3fff640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1de8006f20 0x7f1de80073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.044 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.037+0000 7f1df3fff640 1 -- 172.21.15.31:0/2393490325 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de8005740 con 0x7f1dfc0cc460 2023-11-26T07:23:26.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1df3fff640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cc460 0x7f1de80069e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc15ab10 tx=0x7f1de401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99d93c2a22978d24 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:26.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de40052d0 con 0x7f1dfc0cc460 2023-11-26T07:23:26.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de40a7050 con 0x7f1dfc0cc460 2023-11-26T07:23:26.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1dfa7fc640 1 -- 172.21.15.31:0/2393490325 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de4003600 con 0x7f1dfc0cc460 2023-11-26T07:23:26.045 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1e013f8640 1 -- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cc460 msgr2=0x7f1de80069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1e013f8640 1 --2- 172.21.15.31:0/2393490325 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cc460 0x7f1de80069e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc15ab10 tx=0x7f1de401c540 comp rx=0 tx=0).stop 2023-11-26T07:23:26.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1dfb7fe640 1 -- 172.21.15.31:0/2393490325 reap_dead start 2023-11-26T07:23:26.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1e013f8640 1 -- 172.21.15.31:0/2393490325 shutdown_connections 2023-11-26T07:23:26.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1e013f8640 1 -- 172.21.15.31:0/2393490325 >> 172.21.15.31:0/2393490325 conn(0x7f1dfc0b0cb0 msgr2=0x7f1dfc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:26.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1e013f8640 1 -- 172.21.15.31:0/2393490325 shutdown_connections 2023-11-26T07:23:26.046 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.041+0000 7f1e013f8640 1 -- 172.21.15.31:0/2393490325 wait complete. 2023-11-26T07:23:26.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 Processor -- start 2023-11-26T07:23:26.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 -- start start 2023-11-26T07:23:26.048 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dfc0cc460 0x7f1dfc0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cd100 0x7f1dfc0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dfc0caef0 0x7f1dfc156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0cb3b0 con 0x7f1dfc0cc460 2023-11-26T07:23:26.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc15bce0 con 0x7f1dfc0caef0 2023-11-26T07:23:26.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc15bfc0 con 0x7f1dfc0cd100 2023-11-26T07:23:26.049 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1dfaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dfc0cc460 0x7f1dfc0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1dfaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dfc0cc460 0x7f1dfc0c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35734/0 (socket says 172.21.15.31:35734) 2023-11-26T07:23:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1dfaffd640 1 -- 172.21.15.31:0/1366372066 learned_addr learned my addr 172.21.15.31:0/1366372066 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1df3fff640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cd100 0x7f1dfc0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1dfb7fe640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dfc0caef0 0x7f1dfc156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1dfaffd640 1 -- 172.21.15.31:0/1366372066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cd100 msgr2=0x7f1dfc0ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.050 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1dfaffd640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cd100 0x7f1dfc0ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1dfaffd640 1 -- 172.21.15.31:0/1366372066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dfc0caef0 msgr2=0x7f1dfc156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1dfaffd640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dfc0caef0 0x7f1dfc156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1dfaffd640 1 -- 172.21.15.31:0/1366372066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc15c2a0 con 0x7f1dfc0cc460 2023-11-26T07:23:26.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1dfaffd640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dfc0cc460 0x7f1dfc0c9190 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f1dec012470 tx=0x7f1dec012930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:26.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1df37fe640 1 -- 172.21.15.31:0/1366372066 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec014810 con 0x7f1dfc0cc460 2023-11-26T07:23:26.051 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1df37fe640 1 -- 172.21.15.31:0/1366372066 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec014970 con 0x7f1dfc0cc460 2023-11-26T07:23:26.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1df37fe640 1 -- 172.21.15.31:0/1366372066 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec014d20 con 0x7f1dfc0cc460 2023-11-26T07:23:26.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 -- 172.21.15.31:0/1366372066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc0cd560 con 0x7f1dfc0cc460 2023-11-26T07:23:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 -- 172.21.15.31:0/1366372066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc156e80 con 0x7f1dfc0cc460 2023-11-26T07:23:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.045+0000 7f1e013f8640 1 -- 172.21.15.31:0/1366372066 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc0671f0 con 0x7f1dfc0cc460 2023-11-26T07:23:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.049+0000 7f1df37fe640 1 -- 172.21.15.31:0/1366372066 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f1dec0153e0 con 0x7f1dfc0cc460 2023-11-26T07:23:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.049+0000 7f1df37fe640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1ddc066740 0x7f1ddc068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.049+0000 7f1df37fe640 1 -- 172.21.15.31:0/1366372066 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1dec086dd0 con 0x7f1dfc0cc460 2023-11-26T07:23:26.053 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.049+0000 7f1df3fff640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1ddc066740 0x7f1ddc068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:26.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.049+0000 7f1df3fff640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1ddc066740 0x7f1ddc068c00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1de40ac000 tx=0x7f1de40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:26.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.053+0000 7f1df37fe640 1 -- 172.21.15.31:0/1366372066 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1dec053ec0 con 0x7f1dfc0cc460 2023-11-26T07:23:26.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.237+0000 7f1e013f8640 1 -- 172.21.15.31:0/1366372066 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone94340", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dfc15bb40 con 0x7f1ddc066740 2023-11-26T07:23:26.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.245+0000 7f1df37fe640 1 -- 172.21.15.31:0/1366372066 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+705 (secure 0 0 0) 0x7f1dfc15bb40 con 0x7f1ddc066740 2023-11-26T07:23:26.252 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: "atime": "2023-11-26 07:23:14", 2023-11-26T07:23:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_pcent": "80.00", 2023-11-26T07:23:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_quota": 10485760, 2023-11-26T07:23:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "bytes_used": 8388608, 2023-11-26T07:23:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-11-26 07:23:19", 2023-11-26T07:23:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "ctime": "2023-11-26 07:23:20", 2023-11-26T07:23:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-11-26T07:23:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "features": [ 2023-11-26T07:23:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-clone", 2023-11-26T07:23:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-autoprotect", 2023-11-26T07:23:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot-retention" 2023-11-26T07:23:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:23:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "gid": 1000, 2023-11-26T07:23:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "mode": 16895, 2023-11-26T07:23:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_addrs": [ 2023-11-26T07:23:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.31:6789", 2023-11-26T07:23:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.97:6789", 2023-11-26T07:23:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "172.21.15.97:6790" 2023-11-26T07:23:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2023-11-26T07:23:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "mtime": "2023-11-26 07:23:19", 2023-11-26T07:23:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "path": "/volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c", 2023-11-26T07:23:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_namespace": "", 2023-11-26T07:23:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete", 2023-11-26T07:23:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "clone", 2023-11-26T07:23:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "uid": 1000 2023-11-26T07:23:26.256 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:26.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 -- 172.21.15.31:0/1366372066 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1ddc066740 msgr2=0x7f1ddc068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1ddc066740 0x7f1ddc068c00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1de40ac000 tx=0x7f1de40a6000 comp rx=0 tx=0).stop 2023-11-26T07:23:26.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 -- 172.21.15.31:0/1366372066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dfc0cc460 msgr2=0x7f1dfc0c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:26.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dfc0cc460 0x7f1dfc0c9190 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f1dec012470 tx=0x7f1dec012930 comp rx=0 tx=0).stop 2023-11-26T07:23:26.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 -- 172.21.15.31:0/1366372066 shutdown_connections 2023-11-26T07:23:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1ddc066740 0x7f1ddc068c00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1dfc0caef0 0x7f1dfc156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1dfc0cd100 0x7f1dfc0ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 --2- 172.21.15.31:0/1366372066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1dfc0cc460 0x7f1dfc0c9190 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 -- 172.21.15.31:0/1366372066 >> 172.21.15.31:0/1366372066 conn(0x7f1dfc0b0cb0 msgr2=0x7f1dfc0cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:26.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 -- 172.21.15.31:0/1366372066 shutdown_connections 2023-11-26T07:23:26.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:26.249+0000 7f1e013f8640 1 -- 172.21.15.31:0/1366372066 wait complete. 2023-11-26T07:23:26.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.272 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510) 2023-11-26T07:23:26.282 INFO:teuthology.orchestra.run.smithi031.stderr:volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510: ceph.quota.max_bytes: No such attribute 2023-11-26T07:23:26.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:23:26.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.284 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510) 2023-11-26T07:23:26.335 INFO:teuthology.orchestra.run.smithi031.stdout:cephfs_data 2023-11-26T07:23:26.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.336 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510) 2023-11-26T07:23:26.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.393 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510) 2023-11-26T07:23:26.445 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510 2023-11-26T07:23:26.445 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.2 2023-11-26T07:23:26.445 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.5 2023-11-26T07:23:26.446 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.4 2023-11-26T07:23:26.446 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.7 2023-11-26T07:23:26.446 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.6 2023-11-26T07:23:26.446 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.0 2023-11-26T07:23:26.446 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.3 2023-11-26T07:23:26.446 INFO:teuthology.orchestra.run.smithi031.stdout:volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.1 2023-11-26T07:23:26.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.447 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510) 2023-11-26T07:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:41ff 2023-11-26T07:23:26.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.500 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/) 2023-11-26T07:23:26.552 INFO:teuthology.orchestra.run.smithi031.stdout:41ff 2023-11-26T07:23:26.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.553 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510) 2023-11-26T07:23:26.606 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:26.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.607 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/) 2023-11-26T07:23:26.660 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:26.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.662 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510) 2023-11-26T07:23:26.718 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:26.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.720 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/) 2023-11-26T07:23:26.772 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:26.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.774 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510) 2023-11-26T07:23:26.827 INFO:teuthology.orchestra.run.smithi031.stdout:1700983399 2023-11-26T07:23:26.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.829 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/) 2023-11-26T07:23:26.883 INFO:teuthology.orchestra.run.smithi031.stdout:1700983399 2023-11-26T07:23:26.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.885 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.2) 2023-11-26T07:23:26.936 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:26.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.937 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.2) 2023-11-26T07:23:26.994 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:26.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:26.995 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.2) 2023-11-26T07:23:27.046 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:27.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.047 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.2) 2023-11-26T07:23:27.099 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:27.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.100 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.2) 2023-11-26T07:23:27.150 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:27.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.151 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.2) 2023-11-26T07:23:27.203 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:27.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.204 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.2) 2023-11-26T07:23:27.255 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:27.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.256 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.2) 2023-11-26T07:23:27.307 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:27.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.308 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.5) 2023-11-26T07:23:27.363 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:27.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.365 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.5) 2023-11-26T07:23:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:27.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.416 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.5) 2023-11-26T07:23:27.466 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:27.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.467 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.5) 2023-11-26T07:23:27.518 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:27.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.520 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.5) 2023-11-26T07:23:27.574 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:27.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.576 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.5) 2023-11-26T07:23:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:27.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.627 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.5) 2023-11-26T07:23:27.680 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:27.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.681 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.5) 2023-11-26T07:23:27.734 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:27.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.735 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.4) 2023-11-26T07:23:27.787 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:27.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.788 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.4) 2023-11-26T07:23:27.838 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:27.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.839 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.4) 2023-11-26T07:23:27.890 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:27.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.891 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.4) 2023-11-26T07:23:27.942 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:27.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.943 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.4) 2023-11-26T07:23:27.994 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:27.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:27.995 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.4) 2023-11-26T07:23:28.046 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:28.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.047 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.4) 2023-11-26T07:23:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:28.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.102 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.4) 2023-11-26T07:23:28.154 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:28.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.155 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.7) 2023-11-26T07:23:28.207 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:28.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.208 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.7) 2023-11-26T07:23:28.258 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:28.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.259 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.7) 2023-11-26T07:23:28.310 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:28.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.311 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.7) 2023-11-26T07:23:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:28.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.363 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.7) 2023-11-26T07:23:28.413 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:28.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.415 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.7) 2023-11-26T07:23:28.466 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:28.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.467 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.7) 2023-11-26T07:23:28.521 INFO:teuthology.orchestra.run.smithi031.stdout:1700983399 2023-11-26T07:23:28.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.522 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.7) 2023-11-26T07:23:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:1700983399 2023-11-26T07:23:28.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.576 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.6) 2023-11-26T07:23:28.631 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:28.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.632 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.6) 2023-11-26T07:23:28.686 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:28.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.688 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.6) 2023-11-26T07:23:28.738 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:28.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.739 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.6) 2023-11-26T07:23:28.790 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:28.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.791 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.6) 2023-11-26T07:23:28.842 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:28.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.844 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.6) 2023-11-26T07:23:28.894 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:28.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.895 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.6) 2023-11-26T07:23:28.949 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:28.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:28.951 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.6) 2023-11-26T07:23:29.003 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:29.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.004 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.0) 2023-11-26T07:23:29.055 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:29.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.056 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.0) 2023-11-26T07:23:29.106 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:29.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.108 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.0) 2023-11-26T07:23:29.158 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:29.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.159 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.0) 2023-11-26T07:23:29.210 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:29.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.211 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.0) 2023-11-26T07:23:29.262 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:29.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.263 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.0) 2023-11-26T07:23:29.318 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:29.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.319 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.0) 2023-11-26T07:23:29.374 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:29.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.375 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.0) 2023-11-26T07:23:29.426 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:29.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.428 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.3) 2023-11-26T07:23:29.484 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:29.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.485 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.3) 2023-11-26T07:23:29.535 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:29.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.536 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.3) 2023-11-26T07:23:29.586 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:29.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.588 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.3) 2023-11-26T07:23:29.642 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:29.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.643 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.3) 2023-11-26T07:23:29.694 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:29.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.695 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.3) 2023-11-26T07:23:29.746 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:29.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.747 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.3) 2023-11-26T07:23:29.801 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:29.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.803 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.3) 2023-11-26T07:23:29.855 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:29.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.856 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.1) 2023-11-26T07:23:29.911 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:29.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.912 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.1) 2023-11-26T07:23:29.962 INFO:teuthology.orchestra.run.smithi031.stdout:81b4 2023-11-26T07:23:29.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:29.963 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.1) 2023-11-26T07:23:30.015 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:30.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:30.017 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.1) 2023-11-26T07:23:30.066 INFO:teuthology.orchestra.run.smithi031.stdout:1000 2023-11-26T07:23:30.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:30.067 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.1) 2023-11-26T07:23:30.117 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:30.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:30.119 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.1) 2023-11-26T07:23:30.170 INFO:teuthology.orchestra.run.smithi031.stdout:1271 2023-11-26T07:23:30.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:30.171 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol66940/435cb1c0-796a-4a83-9236-db7c62d93103/.snap/subvol_snap62510/subvolume_file.1) 2023-11-26T07:23:30.225 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:30.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:30.226 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone94340/251c0876-fed6-4e2a-9dbc-dceda4d98f8c/subvolume_file.1) 2023-11-26T07:23:30.278 INFO:teuthology.orchestra.run.smithi031.stdout:1700983398 2023-11-26T07:23:30.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol66940 fake subvol_clone96731 2023-11-26T07:23:30.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63af879640 1 Processor -- start 2023-11-26T07:23:30.479 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63af879640 1 -- start start 2023-11-26T07:23:30.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63af879640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63a806e3f0 0x7f63a815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63af879640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63a80ce5d0 0x7f63a815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63af879640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a8159630 con 0x7f63a80cb9b0 2023-11-26T07:23:30.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63af879640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a815bba0 con 0x7f63a80ce5d0 2023-11-26T07:23:30.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63af879640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80cdd50 con 0x7f63a806e3f0 2023-11-26T07:23:30.481 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63ade16640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63a80ce5d0 0x7f63a815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63ade16640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63a80ce5d0 0x7f63a815bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35736/0 (socket says 172.21.15.31:35736) 2023-11-26T07:23:30.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63ade16640 1 -- 172.21.15.31:0/3502397683 learned_addr learned my addr 172.21.15.31:0/3502397683 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:30.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63ace14640 1 --2- 172.21.15.31:0/3502397683 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63a806e3f0 0x7f63a815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63ade16640 1 -- 172.21.15.31:0/3502397683 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63a806e3f0 msgr2=0x7f63a815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:30.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63ade16640 1 --2- 172.21.15.31:0/3502397683 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63a806e3f0 0x7f63a815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63ade16640 1 -- 172.21.15.31:0/3502397683 >> v1:172.21.15.97:6789/0 conn(0x7f63a80cb9b0 legacy=0x7f63a80cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.482 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.477+0000 7f63ade16640 1 -- 172.21.15.31:0/3502397683 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a815f6d0 con 0x7f63a80ce5d0 2023-11-26T07:23:30.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f63ade16640 1 --2- 172.21.15.31:0/3502397683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a80ce5d0 0x7f63a815bfd0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009e50 tx=0x7f63a001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d82b7aa0bc5c71f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.484 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f639ffff640 1 -- 172.21.15.31:0/3502397683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a00a57b0 con 0x7f63a80ce5d0 2023-11-26T07:23:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f639ffff640 1 -- 172.21.15.31:0/3502397683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a00a5910 con 0x7f63a80ce5d0 2023-11-26T07:23:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f639ffff640 1 -- 172.21.15.31:0/3502397683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a00a5ca0 con 0x7f63a80ce5d0 2023-11-26T07:23:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f63af879640 1 -- 172.21.15.31:0/3502397683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a80ce5d0 msgr2=0x7f63a815bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.485 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f63af879640 1 --2- 172.21.15.31:0/3502397683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a80ce5d0 0x7f63a815bfd0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009e50 tx=0x7f63a001ca10 comp rx=0 tx=0).stop 2023-11-26T07:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f63af879640 1 -- 172.21.15.31:0/3502397683 shutdown_connections 2023-11-26T07:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f63af879640 1 --2- 172.21.15.31:0/3502397683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a80ce5d0 0x7f63a815bfd0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f63af879640 1 --2- 172.21.15.31:0/3502397683 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63a806e3f0 0x7f63a815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f63af879640 1 -- 172.21.15.31:0/3502397683 >> 172.21.15.31:0/3502397683 conn(0x7f63a80b0cb0 msgr2=0x7f63a80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f63af879640 1 -- 172.21.15.31:0/3502397683 shutdown_connections 2023-11-26T07:23:30.486 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.481+0000 7f63af879640 1 -- 172.21.15.31:0/3502397683 wait complete. 2023-11-26T07:23:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63af879640 1 Processor -- start 2023-11-26T07:23:30.487 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63af879640 1 -- start start 2023-11-26T07:23:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63af879640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a806e3f0 0x7f63a814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63af879640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63a80cb9b0 0x7f63a8158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63af879640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a80ce5d0 0x7f63a80ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63af879640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a8150320 con 0x7f63a806e3f0 2023-11-26T07:23:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63af879640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a814f050 con 0x7f63a80ce5d0 2023-11-26T07:23:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63af879640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a8158b90 con 0x7f63a80cb9b0 2023-11-26T07:23:30.489 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ad615640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a806e3f0 0x7f63a814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ad615640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a806e3f0 0x7f63a814eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35748/0 (socket says 172.21.15.31:35748) 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ad615640 1 -- 172.21.15.31:0/3545686592 learned_addr learned my addr 172.21.15.31:0/3545686592 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ace14640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63a80cb9b0 0x7f63a8158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ade16640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a80ce5d0 0x7f63a80ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ad615640 1 -- 172.21.15.31:0/3545686592 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63a80cb9b0 msgr2=0x7f63a8158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ad615640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63a80cb9b0 0x7f63a8158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ad615640 1 -- 172.21.15.31:0/3545686592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a80ce5d0 msgr2=0x7f63a80ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ad615640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a80ce5d0 0x7f63a80ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ad615640 1 -- 172.21.15.31:0/3545686592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a0009b30 con 0x7f63a806e3f0 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ade16640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a80ce5d0 0x7f63a80ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:30.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63ad615640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a806e3f0 0x7f63a814eb10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f63a400c530 tx=0x7f63a400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f639dffb640 1 -- 172.21.15.31:0/3545686592 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4005220 con 0x7f63a806e3f0 2023-11-26T07:23:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f639dffb640 1 -- 172.21.15.31:0/3545686592 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a4005380 con 0x7f63a806e3f0 2023-11-26T07:23:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63af879640 1 -- 172.21.15.31:0/3545686592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a81536f0 con 0x7f63a806e3f0 2023-11-26T07:23:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f639dffb640 1 -- 172.21.15.31:0/3545686592 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4007270 con 0x7f63a806e3f0 2023-11-26T07:23:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.485+0000 7f63af879640 1 -- 172.21.15.31:0/3545686592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a814fc60 con 0x7f63a806e3f0 2023-11-26T07:23:30.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.489+0000 7f63af879640 1 -- 172.21.15.31:0/3545686592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a80671f0 con 0x7f63a806e3f0 2023-11-26T07:23:30.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.489+0000 7f639dffb640 1 -- 172.21.15.31:0/3545686592 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f63a40073d0 con 0x7f63a806e3f0 2023-11-26T07:23:30.492 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.489+0000 7f639dffb640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f638c066740 0x7f638c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.489+0000 7f639dffb640 1 -- 172.21.15.31:0/3545686592 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63a4083b00 con 0x7f63a806e3f0 2023-11-26T07:23:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.489+0000 7f63ace14640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f638c066740 0x7f638c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.489+0000 7f63ace14640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f638c066740 0x7f638c068c00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f63a80cc6f0 tx=0x7f6398009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.493+0000 7f639dffb640 1 -- 172.21.15.31:0/3545686592 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f63a4085e60 con 0x7f63a806e3f0 2023-11-26T07:23:30.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.685+0000 7f63af879640 1 -- 172.21.15.31:0/3545686592 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol66940", "snap_name": "fake", "target_sub_name": "subvol_clone96731", "target": ["mon-mgr", ""]}) v1 -- 0x7f63a80ccea0 con 0x7f638c066740 2023-11-26T07:23:30.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.741+0000 7f639dffb640 1 -- 172.21.15.31:0/3545686592 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6398011a30 con 0x7f638c066740 2023-11-26T07:23:30.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 -- 172.21.15.31:0/3545686592 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f638c066740 msgr2=0x7f638c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f638c066740 0x7f638c068c00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f63a80cc6f0 tx=0x7f6398009190 comp rx=0 tx=0).stop 2023-11-26T07:23:30.747 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 -- 172.21.15.31:0/3545686592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a806e3f0 msgr2=0x7f63a814eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a806e3f0 0x7f63a814eb10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f63a400c530 tx=0x7f63a400c9f0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 -- 172.21.15.31:0/3545686592 shutdown_connections 2023-11-26T07:23:30.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f638c066740 0x7f638c068c00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a80ce5d0 0x7f63a80ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63a80cb9b0 0x7f63a8158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 --2- 172.21.15.31:0/3545686592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63a806e3f0 0x7f63a814eb10 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.748 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 -- 172.21.15.31:0/3545686592 >> 172.21.15.31:0/3545686592 conn(0x7f63a80b0cb0 msgr2=0x7f63a80cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:30.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 -- 172.21.15.31:0/3545686592 shutdown_connections 2023-11-26T07:23:30.749 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.745+0000 7f63af879640 1 -- 172.21.15.31:0/3545686592 wait complete. 2023-11-26T07:23:30.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone96731 2023-11-26T07:23:30.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:23:30.826 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:23:30.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.913+0000 7fba4c279640 1 Processor -- start 2023-11-26T07:23:30.917 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.913+0000 7fba4c279640 1 -- start start 2023-11-26T07:23:30.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4c279640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fba440ce5d0 0x7fba4415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4c279640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba44159630 con 0x7fba4406e3f0 2023-11-26T07:23:30.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4c279640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba440cd1d0 con 0x7fba440cb9b0 2023-11-26T07:23:30.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4c279640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4406e860 con 0x7fba440ce5d0 2023-11-26T07:23:30.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49814640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fba4406e3f0 0x7fba4415f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57730/0 (socket says 172.21.15.31:57730) 2023-11-26T07:23:30.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49814640 1 -- 172.21.15.31:0/2804533762 learned_addr learned my addr 172.21.15.31:0/2804533762 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4a816640 1 --2- 172.21.15.31:0/2804533762 >> v2:172.21.15.97:3300/0 conn(0x7fba440ce5d0 0x7fba4415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003030758 0 0) 0x7fba44159630 con 0x7fba4406e3f0 2023-11-26T07:23:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba30003530 con 0x7fba4406e3f0 2023-11-26T07:23:30.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2079052045 0 0) 0x7fba30003530 con 0x7fba4406e3f0 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 >> v2:172.21.15.97:3300/0 conn(0x7fba440ce5d0 msgr2=0x7fba4415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 --2- 172.21.15.31:0/2804533762 >> v2:172.21.15.97:3300/0 conn(0x7fba440ce5d0 0x7fba4415bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 >> v1:172.21.15.97:6789/0 conn(0x7fba440cb9b0 legacy=0x7fba440cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba440cd4b0 con 0x7fba4406e3f0 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fba34001680 con 0x7fba4406e3f0 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 >> v1:172.21.15.31:6789/0 conn(0x7fba4406e3f0 legacy=0x7fba4415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba30003980 0x7fba30003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba30004380 0x7fba30004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba30005050 0x7fba3000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4406e860 con 0x7fba30003980 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3000bfb0 con 0x7fba30005050 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3000c170 con 0x7fba30004380 2023-11-26T07:23:30.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4a015640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba30003980 0x7fba30003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba49814640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba30004380 0x7fba30004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4a816640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba30005050 0x7fba3000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4a015640 1 -- 172.21.15.31:0/2804533762 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba30004380 msgr2=0x7fba30004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:30.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4a015640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba30004380 0x7fba30004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4a015640 1 -- 172.21.15.31:0/2804533762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba30005050 msgr2=0x7fba3000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:30.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4a015640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba30005050 0x7fba3000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4a015640 1 -- 172.21.15.31:0/2804533762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba30003530 con 0x7fba30003980 2023-11-26T07:23:30.922 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.917+0000 7fba4a816640 1 -- 172.21.15.31:0/2804533762 reap_dead start 2023-11-26T07:23:30.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba4a015640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba30003980 0x7fba30003d60 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fba40001860 tx=0x7fba4001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ce77c4f2fb284a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400ab600 con 0x7fba30003980 2023-11-26T07:23:30.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba400a7050 con 0x7fba30003980 2023-11-26T07:23:30.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba49013640 1 -- 172.21.15.31:0/2804533762 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400ab9e0 con 0x7fba30003980 2023-11-26T07:23:30.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba4c279640 1 -- 172.21.15.31:0/2804533762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba30003980 msgr2=0x7fba30003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba4c279640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba30003980 0x7fba30003d60 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fba40001860 tx=0x7fba4001c4e0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba4c279640 1 -- 172.21.15.31:0/2804533762 shutdown_connections 2023-11-26T07:23:30.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba4c279640 1 --2- 172.21.15.31:0/2804533762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba30003980 0x7fba30003d60 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba4c279640 1 -- 172.21.15.31:0/2804533762 >> 172.21.15.31:0/2804533762 conn(0x7fba440b0cb0 msgr2=0x7fba440b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:30.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba4c279640 1 -- 172.21.15.31:0/2804533762 shutdown_connections 2023-11-26T07:23:30.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba4c279640 1 -- 172.21.15.31:0/2804533762 wait complete. 2023-11-26T07:23:30.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.921+0000 7fba4c279640 1 Processor -- start 2023-11-26T07:23:30.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4c279640 1 -- start start 2023-11-26T07:23:30.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4c279640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba30003980 0x7fba4414e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4c279640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba44155c60 0x7fba4414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4c279640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba441f45a0 0x7fba441575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4c279640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba441560e0 con 0x7fba44155c60 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4c279640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba44159c20 con 0x7fba30003980 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4c279640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4414e590 con 0x7fba441f45a0 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba49814640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba44155c60 0x7fba4414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba49814640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba44155c60 0x7fba4414e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35756/0 (socket says 172.21.15.31:35756) 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba49814640 1 -- 172.21.15.31:0/2763393588 learned_addr learned my addr 172.21.15.31:0/2763393588 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba49814640 1 -- 172.21.15.31:0/2763393588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba441f45a0 msgr2=0x7fba441575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4a816640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba441f45a0 0x7fba441575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4a015640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba30003980 0x7fba4414e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba49814640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba441f45a0 0x7fba441575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba49814640 1 -- 172.21.15.31:0/2763393588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba30003980 msgr2=0x7fba4414e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba49814640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba30003980 0x7fba4414e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba49814640 1 -- 172.21.15.31:0/2763393588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44159e70 con 0x7fba44155c60 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4a816640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba441f45a0 0x7fba441575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4a015640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba30003980 0x7fba4414e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba49814640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba44155c60 0x7fba4414e050 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fba34004ed0 tx=0x7fba34007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba3affd640 1 -- 172.21.15.31:0/2763393588 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3400f730 con 0x7fba44155c60 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba3affd640 1 -- 172.21.15.31:0/2763393588 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3400f8b0 con 0x7fba44155c60 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba3affd640 1 -- 172.21.15.31:0/2763393588 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3400fc60 con 0x7fba44155c60 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4c279640 1 -- 172.21.15.31:0/2763393588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4415a150 con 0x7fba44155c60 2023-11-26T07:23:30.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4c279640 1 -- 172.21.15.31:0/2763393588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4415a710 con 0x7fba44155c60 2023-11-26T07:23:30.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.925+0000 7fba4c279640 1 -- 172.21.15.31:0/2763393588 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba440671f0 con 0x7fba44155c60 2023-11-26T07:23:30.930 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.929+0000 7fba3affd640 1 -- 172.21.15.31:0/2763393588 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fba3401d460 con 0x7fba44155c60 2023-11-26T07:23:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.929+0000 7fba3affd640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba28066740 0x7fba28068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.929+0000 7fba3affd640 1 -- 172.21.15.31:0/2763393588 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba34084320 con 0x7fba44155c60 2023-11-26T07:23:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.929+0000 7fba4a015640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba28066740 0x7fba28068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:30.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.929+0000 7fba4a015640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba28066740 0x7fba28068c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba400ad000 tx=0x7fba400a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:30.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:30.933+0000 7fba3affd640 1 -- 172.21.15.31:0/2763393588 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fba340514d0 con 0x7fba44155c60 2023-11-26T07:23:31.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:31.121+0000 7fba4c279640 1 -- 172.21.15.31:0/2763393588 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone96731", "target": ["mon-mgr", ""]}) v1 -- 0x7fba4415c7e0 con 0x7fba28066740 2023-11-26T07:23:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.912+0000 7fba3affd640 1 -- 172.21.15.31:0/2763393588 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+153 (secure 0 0 0) 0x7fba40006900 con 0x7fba28066740 2023-11-26T07:23:34.916 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:34.916 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-11-26T07:23:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "pending", 2023-11-26T07:23:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "source": { 2023-11-26T07:23:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "volume": "cephfs", 2023-11-26T07:23:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "subvolume": "subvol66940", 2023-11-26T07:23:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: "snapshot": "fake" 2023-11-26T07:23:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:34.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:34.917 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 -- 172.21.15.31:0/2763393588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba28066740 msgr2=0x7fba28068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba28066740 0x7fba28068c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fba400ad000 tx=0x7fba400a6000 comp rx=0 tx=0).stop 2023-11-26T07:23:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 -- 172.21.15.31:0/2763393588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba44155c60 msgr2=0x7fba4414e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba44155c60 0x7fba4414e050 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fba34004ed0 tx=0x7fba34007430 comp rx=0 tx=0).stop 2023-11-26T07:23:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 -- 172.21.15.31:0/2763393588 shutdown_connections 2023-11-26T07:23:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fba28066740 0x7fba28068c00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fba441f45a0 0x7fba441575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba44155c60 0x7fba4414e050 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 --2- 172.21.15.31:0/2763393588 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba30003980 0x7fba4414e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 -- 172.21.15.31:0/2763393588 >> 172.21.15.31:0/2763393588 conn(0x7fba440b0cb0 msgr2=0x7fba4415d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 -- 172.21.15.31:0/2763393588 shutdown_connections 2023-11-26T07:23:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:34.916+0000 7fba4c279640 1 -- 172.21.15.31:0/2763393588 wait complete. 2023-11-26T07:23:35.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone96731 2023-11-26T07:23:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f639b701640 1 Processor -- start 2023-11-26T07:23:36.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f639b701640 1 -- start start 2023-11-26T07:23:36.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f639b701640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63940cb9b0 0x7f639406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f639b701640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6394159630 con 0x7f63940ce5d0 2023-11-26T07:23:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f639b701640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63940cd1d0 con 0x7f63940cbd90 2023-11-26T07:23:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f639b701640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639415bba0 con 0x7f63940cb9b0 2023-11-26T07:23:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f6399c9e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f63940ce5d0 0x7f639415bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57226/0 (socket says 172.21.15.31:57226) 2023-11-26T07:23:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f6399c9e640 1 -- 172.21.15.31:0/2794376246 learned_addr learned my addr 172.21.15.31:0/2794376246 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f639949d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f63940cb9b0 0x7f639406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f639949d640 1 -- 172.21.15.31:0/2794376246 >> v1:172.21.15.97:6789/0 conn(0x7f63940cbd90 legacy=0x7f639415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f639949d640 1 -- 172.21.15.31:0/2794376246 >> v1:172.21.15.31:6789/0 conn(0x7f63940ce5d0 legacy=0x7f639415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f639949d640 1 -- 172.21.15.31:0/2794376246 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940cd4b0 con 0x7f63940cb9b0 2023-11-26T07:23:36.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.104+0000 7f638bfff640 1 -- 172.21.15.31:0/2794376246 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967713849 0 0) 0x7f6394159630 con 0x7f63940ce5d0 2023-11-26T07:23:36.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f639949d640 1 --2- 172.21.15.31:0/2794376246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940cb9b0 0x7f639406e3f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f63900097c0 tx=0x7f639001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7b4aa086d6740f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f638bfff640 1 -- 172.21.15.31:0/2794376246 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900ab600 con 0x7f63940cb9b0 2023-11-26T07:23:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f638bfff640 1 -- 172.21.15.31:0/2794376246 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63900a7050 con 0x7f63940cb9b0 2023-11-26T07:23:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f638bfff640 1 -- 172.21.15.31:0/2794376246 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900ab980 con 0x7f63940cb9b0 2023-11-26T07:23:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f639b701640 1 -- 172.21.15.31:0/2794376246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940cb9b0 msgr2=0x7f639406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f639b701640 1 --2- 172.21.15.31:0/2794376246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940cb9b0 0x7f639406e3f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f63900097c0 tx=0x7f639001c480 comp rx=0 tx=0).stop 2023-11-26T07:23:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f639b701640 1 -- 172.21.15.31:0/2794376246 shutdown_connections 2023-11-26T07:23:36.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f639b701640 1 --2- 172.21.15.31:0/2794376246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940cb9b0 0x7f639406e3f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:36.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f639b701640 1 -- 172.21.15.31:0/2794376246 >> 172.21.15.31:0/2794376246 conn(0x7f63940b0cb0 msgr2=0x7f63940b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:36.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f639b701640 1 -- 172.21.15.31:0/2794376246 shutdown_connections 2023-11-26T07:23:36.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.108+0000 7f639b701640 1 -- 172.21.15.31:0/2794376246 wait complete. 2023-11-26T07:23:36.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639b701640 1 Processor -- start 2023-11-26T07:23:36.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639b701640 1 -- start start 2023-11-26T07:23:36.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639b701640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63940cb9b0 0x7f6394155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:36.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639b701640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63940cbd90 0x7f6394152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:36.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639b701640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940ce5d0 0x7f639415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:36.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639b701640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6394153a50 con 0x7f63940ce5d0 2023-11-26T07:23:36.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639b701640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6394153d30 con 0x7f63940cb9b0 2023-11-26T07:23:36.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639b701640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6394154010 con 0x7f63940cbd90 2023-11-26T07:23:36.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6399c9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940ce5d0 0x7f639415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:36.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6399c9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940ce5d0 0x7f639415a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37440/0 (socket says 172.21.15.31:37440) 2023-11-26T07:23:36.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6399c9e640 1 -- 172.21.15.31:0/3062658426 learned_addr learned my addr 172.21.15.31:0/3062658426 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:36.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639949d640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63940cb9b0 0x7f6394155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:36.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6398c9c640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63940cbd90 0x7f6394152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:36.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6399c9e640 1 -- 172.21.15.31:0/3062658426 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63940cbd90 msgr2=0x7f6394152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:36.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6399c9e640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63940cbd90 0x7f6394152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:36.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6399c9e640 1 -- 172.21.15.31:0/3062658426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63940cb9b0 msgr2=0x7f6394155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:36.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6399c9e640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63940cb9b0 0x7f6394155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:36.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6399c9e640 1 -- 172.21.15.31:0/3062658426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390009420 con 0x7f63940ce5d0 2023-11-26T07:23:36.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6399c9e640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940ce5d0 0x7f639415a6b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f638c00c700 tx=0x7f638c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:36.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6389ffb640 1 -- 172.21.15.31:0/3062658426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00ce10 con 0x7f63940ce5d0 2023-11-26T07:23:36.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6389ffb640 1 -- 172.21.15.31:0/3062658426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c00a640 con 0x7f63940ce5d0 2023-11-26T07:23:36.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639b701640 1 -- 172.21.15.31:0/3062658426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63941542f0 con 0x7f63940ce5d0 2023-11-26T07:23:36.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f6389ffb640 1 -- 172.21.15.31:0/3062658426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c00aa70 con 0x7f63940ce5d0 2023-11-26T07:23:36.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.112+0000 7f639b701640 1 -- 172.21.15.31:0/3062658426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6394153740 con 0x7f63940ce5d0 2023-11-26T07:23:36.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.116+0000 7f639b701640 1 -- 172.21.15.31:0/3062658426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63940671f0 con 0x7f63940ce5d0 2023-11-26T07:23:36.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.116+0000 7f6389ffb640 1 -- 172.21.15.31:0/3062658426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f638c00abd0 con 0x7f63940ce5d0 2023-11-26T07:23:36.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.116+0000 7f6389ffb640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6378066740 0x7f6378068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:36.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.116+0000 7f6389ffb640 1 -- 172.21.15.31:0/3062658426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f638c083700 con 0x7f63940ce5d0 2023-11-26T07:23:36.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.116+0000 7f639949d640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6378066740 0x7f6378068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:36.121 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.116+0000 7f639949d640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6378066740 0x7f6378068c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f63900ad000 tx=0x7f63900a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:36.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.120+0000 7f6389ffb640 1 -- 172.21.15.31:0/3062658426 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f638c0508e0 con 0x7f63940ce5d0 2023-11-26T07:23:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:36.312+0000 7f639b701640 1 -- 172.21.15.31:0/3062658426 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone96731", "target": ["mon-mgr", ""]}) v1 -- 0x7f6394153ea0 con 0x7f6378066740 2023-11-26T07:23:39.889 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.884+0000 7f6389ffb640 1 -- 172.21.15.31:0/3062658426 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f6394153ea0 con 0x7f6378066740 2023-11-26T07:23:39.889 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "status": { 2023-11-26T07:23:39.889 INFO:teuthology.orchestra.run.smithi031.stdout: "state": "complete" 2023-11-26T07:23:39.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:39.890 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:39.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 -- 172.21.15.31:0/3062658426 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6378066740 msgr2=0x7f6378068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:39.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6378066740 0x7f6378068c00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f63900ad000 tx=0x7f63900a6000 comp rx=0 tx=0).stop 2023-11-26T07:23:39.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 -- 172.21.15.31:0/3062658426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940ce5d0 msgr2=0x7f639415a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:39.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940ce5d0 0x7f639415a6b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f638c00c700 tx=0x7f638c00cbc0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 -- 172.21.15.31:0/3062658426 shutdown_connections 2023-11-26T07:23:39.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f6378066740 0x7f6378068c00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63940ce5d0 0x7f639415a6b0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f63940cbd90 0x7f6394152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 --2- 172.21.15.31:0/3062658426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63940cb9b0 0x7f6394155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:39.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 -- 172.21.15.31:0/3062658426 >> 172.21.15.31:0/3062658426 conn(0x7f63940b0cb0 msgr2=0x7f639415d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:39.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 -- 172.21.15.31:0/3062658426 shutdown_connections 2023-11-26T07:23:39.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:39.888+0000 7f639b701640 1 -- 172.21.15.31:0/3062658426 wait complete. 2023-11-26T07:23:39.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:39.917 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone96731/.meta) 2023-11-26T07:23:39.940 INFO:teuthology.orchestra.run.smithi031.stdout:[GLOBAL] 2023-11-26T07:23:39.940 INFO:teuthology.orchestra.run.smithi031.stdout:version = 2 2023-11-26T07:23:39.940 INFO:teuthology.orchestra.run.smithi031.stdout:type = clone 2023-11-26T07:23:39.940 INFO:teuthology.orchestra.run.smithi031.stdout:path = /volumes/_nogroup/subvol_clone96731/5be5023a-1ab0-49f1-945f-cc4859c5fc36 2023-11-26T07:23:39.940 INFO:teuthology.orchestra.run.smithi031.stdout:state = complete 2023-11-26T07:23:39.940 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:39.940 INFO:teuthology.orchestra.run.smithi031.stdout:[source] 2023-11-26T07:23:39.940 INFO:teuthology.orchestra.run.smithi031.stdout:volume = cephfs 2023-11-26T07:23:39.940 INFO:teuthology.orchestra.run.smithi031.stdout:subvolume = subvol66940 2023-11-26T07:23:39.941 INFO:teuthology.orchestra.run.smithi031.stdout:snapshot = fake 2023-11-26T07:23:39.941 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:39.942 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol66940 subvol_snap62510 2023-11-26T07:23:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.132+0000 7f9b32691640 1 Processor -- start 2023-11-26T07:23:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.132+0000 7f9b32691640 1 -- start start 2023-11-26T07:23:40.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b32691640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f9b2c0cdfb0 0x7f9b2c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b32691640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2c0b2710 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b32691640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2c0b29f0 con 0x7f9b2c15a710 2023-11-26T07:23:40.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b32691640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c0cca80 con 0x7f9b2c0cdfb0 2023-11-26T07:23:40.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b30c2e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9b2c0cbd50 0x7f9b2c0cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57234/0 (socket says 172.21.15.31:57234) 2023-11-26T07:23:40.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b30c2e640 1 -- 172.21.15.31:0/841633822 learned_addr learned my addr 172.21.15.31:0/841633822 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b2bfff640 1 --2- 172.21.15.31:0/841633822 >> v2:172.21.15.97:3300/0 conn(0x7f9b2c0cdfb0 0x7f9b2c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b2affd640 1 -- 172.21.15.31:0/841633822 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102124832 0 0) 0x7f9b2c0b2710 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b2affd640 1 -- 172.21.15.31:0/841633822 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b14003530 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b2bfff640 1 -- 172.21.15.31:0/841633822 >> v1:172.21.15.97:6789/0 conn(0x7f9b2c15a710 legacy=0x7f9b2c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b2bfff640 1 -- 172.21.15.31:0/841633822 >> v1:172.21.15.31:6789/0 conn(0x7f9b2c0cbd50 legacy=0x7f9b2c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.136+0000 7f9b2bfff640 1 -- 172.21.15.31:0/841633822 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0b2cd0 con 0x7f9b2c0cdfb0 2023-11-26T07:23:40.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b2bfff640 1 --2- 172.21.15.31:0/841633822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cdfb0 0x7f9b2c0abf90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0097c0 tx=0x7f9b1c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f3b3f6f58a2924 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b2affd640 1 -- 172.21.15.31:0/841633822 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0ab670 con 0x7f9b2c0cdfb0 2023-11-26T07:23:40.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b2affd640 1 -- 172.21.15.31:0/841633822 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c0ab7d0 con 0x7f9b2c0cdfb0 2023-11-26T07:23:40.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b2affd640 1 -- 172.21.15.31:0/841633822 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0abb60 con 0x7f9b2c0cdfb0 2023-11-26T07:23:40.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b32691640 1 -- 172.21.15.31:0/841633822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cdfb0 msgr2=0x7f9b2c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b32691640 1 --2- 172.21.15.31:0/841633822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cdfb0 0x7f9b2c0abf90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0097c0 tx=0x7f9b1c01c450 comp rx=0 tx=0).stop 2023-11-26T07:23:40.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b32691640 1 -- 172.21.15.31:0/841633822 shutdown_connections 2023-11-26T07:23:40.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b32691640 1 --2- 172.21.15.31:0/841633822 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c0cdfb0 0x7f9b2c0abf90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.144 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b32691640 1 -- 172.21.15.31:0/841633822 >> 172.21.15.31:0/841633822 conn(0x7f9b2c0b0cb0 msgr2=0x7f9b2c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:40.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b32691640 1 -- 172.21.15.31:0/841633822 shutdown_connections 2023-11-26T07:23:40.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.140+0000 7f9b32691640 1 -- 172.21.15.31:0/841633822 wait complete. 2023-11-26T07:23:40.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 Processor -- start 2023-11-26T07:23:40.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 -- start start 2023-11-26T07:23:40.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b2c0cbd50 0x7f9b2c1548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b2c0cdfb0 0x7f9b2c149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c15a710 0x7f9b2c14a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c070ca0 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c070690 con 0x7f9b2c15a710 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c1530f0 con 0x7f9b2c0cdfb0 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b2c0cbd50 0x7f9b2c1548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b2c0cbd50 0x7f9b2c1548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37442/0 (socket says 172.21.15.31:37442) 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2bfff640 1 -- 172.21.15.31:0/3621718345 learned_addr learned my addr 172.21.15.31:0/3621718345 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2b7fe640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b2c0cdfb0 0x7f9b2c149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b30c2e640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c15a710 0x7f9b2c14a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2bfff640 1 -- 172.21.15.31:0/3621718345 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b2c0cdfb0 msgr2=0x7f9b2c149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2bfff640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b2c0cdfb0 0x7f9b2c149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2bfff640 1 -- 172.21.15.31:0/3621718345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c15a710 msgr2=0x7f9b2c14a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2bfff640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c15a710 0x7f9b2c14a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2bfff640 1 -- 172.21.15.31:0/3621718345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c009420 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2b7fe640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b2c0cdfb0 0x7f9b2c149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:40.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b2bfff640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b2c0cbd50 0x7f9b2c1548e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0ad040 tx=0x7f9b1c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b28ff9640 1 -- 172.21.15.31:0/3621718345 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0b58d0 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b28ff9640 1 -- 172.21.15.31:0/3621718345 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c0b5a50 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b28ff9640 1 -- 172.21.15.31:0/3621718345 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0b5e00 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 -- 172.21.15.31:0/3621718345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c1533d0 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 -- 172.21.15.31:0/3621718345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c152d10 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.144+0000 7f9b32691640 1 -- 172.21.15.31:0/3621718345 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c0671f0 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.148+0000 7f9b28ff9640 1 -- 172.21.15.31:0/3621718345 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f9b1c0bf410 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.151 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.148+0000 7f9b28ff9640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b18066740 0x7f9b18068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.148+0000 7f9b28ff9640 1 -- 172.21.15.31:0/3621718345 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b1c11f0b0 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.148+0000 7f9b2b7fe640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b18066740 0x7f9b18068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.148+0000 7f9b2b7fe640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b18066740 0x7f9b18068c00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c15d6b0 tx=0x7f9b20009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.152+0000 7f9b28ff9640 1 -- 172.21.15.31:0/3621718345 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9b1c0ec000 con 0x7f9b2c0cbd50 2023-11-26T07:23:40.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.336+0000 7f9b32691640 1 -- 172.21.15.31:0/3621718345 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol66940", "snap_name": "subvol_snap62510", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b2c070de0 con 0x7f9b18066740 2023-11-26T07:23:40.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b28ff9640 1 -- 172.21.15.31:0/3621718345 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+207 (secure 0 0 0) 0x7f9b20001350 con 0x7f9b18066740 2023-11-26T07:23:40.355 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "created_at": "2023-11-26 07:23:19.502655", 2023-11-26T07:23:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "data_pool": "cephfs_data", 2023-11-26T07:23:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "has_pending_clones": "yes", 2023-11-26T07:23:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: "pending_clones": [ 2023-11-26T07:23:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2023-11-26T07:23:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "subvol_clone94340" 2023-11-26T07:23:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2023-11-26T07:23:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2023-11-26T07:23:40.356 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:40.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 -- 172.21.15.31:0/3621718345 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b18066740 msgr2=0x7f9b18068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.357 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b18066740 0x7f9b18068c00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c15d6b0 tx=0x7f9b20009040 comp rx=0 tx=0).stop 2023-11-26T07:23:40.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 -- 172.21.15.31:0/3621718345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b2c0cbd50 msgr2=0x7f9b2c1548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b2c0cbd50 0x7f9b2c1548e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0ad040 tx=0x7f9b1c01ac90 comp rx=0 tx=0).stop 2023-11-26T07:23:40.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 -- 172.21.15.31:0/3621718345 shutdown_connections 2023-11-26T07:23:40.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f9b18066740 0x7f9b18068c00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b2c15a710 0x7f9b2c14a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9b2c0cdfb0 0x7f9b2c149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 --2- 172.21.15.31:0/3621718345 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b2c0cbd50 0x7f9b2c1548e0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 -- 172.21.15.31:0/3621718345 >> 172.21.15.31:0/3621718345 conn(0x7f9b2c0b0cb0 msgr2=0x7f9b2c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:40.358 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 -- 172.21.15.31:0/3621718345 shutdown_connections 2023-11-26T07:23:40.359 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.352+0000 7f9b32691640 1 -- 172.21.15.31:0/3621718345 wait complete. 2023-11-26T07:23:40.377 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-26T07:23:40.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:23:40.421 DEBUG:teuthology.orchestra.run.smithi097:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-26T07:23:40.507 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-26T07:23:40.507 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:40.507 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:40.517 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:40.517 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:40.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:40.755 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7effa1557640 1 Processor -- start 2023-11-26T07:23:40.756 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7effa1557640 1 -- start start 2023-11-26T07:23:40.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7effa1557640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff9c146c60 0x7eff9c070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7effa1557640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff9c147040 0x7eff9c070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7effa1557640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7eff9c06bb10 0x7eff9c06bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7effa1557640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c06d100 con 0x7eff9c146c60 2023-11-26T07:23:40.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7effa1557640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c15dba0 con 0x7eff9c06bb10 2023-11-26T07:23:40.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7effa1557640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c06c860 con 0x7eff9c147040 2023-11-26T07:23:40.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7eff9affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff9c146c60 0x7eff9c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7eff9affd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7eff9c146c60 0x7eff9c070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37454/0 (socket says 172.21.15.31:37454) 2023-11-26T07:23:40.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7eff9affd640 1 -- 172.21.15.31:0/2206977442 learned_addr learned my addr 172.21.15.31:0/2206977442 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:40.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7eff9affd640 1 -- 172.21.15.31:0/2206977442 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff9c147040 msgr2=0x7eff9c070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:40.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7eff9affd640 1 --2- 172.21.15.31:0/2206977442 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff9c147040 0x7eff9c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7eff9affd640 1 -- 172.21.15.31:0/2206977442 >> v2:172.21.15.97:3300/0 conn(0x7eff9c06bb10 msgr2=0x7eff9c06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:40.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.752+0000 7eff9b7fe640 1 --2- 172.21.15.31:0/2206977442 >> v2:172.21.15.97:3300/0 conn(0x7eff9c06bb10 0x7eff9c06bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.756+0000 7eff9affd640 1 --2- 172.21.15.31:0/2206977442 >> v2:172.21.15.97:3300/0 conn(0x7eff9c06bb10 0x7eff9c06bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.756+0000 7eff9affd640 1 -- 172.21.15.31:0/2206977442 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c06cb40 con 0x7eff9c146c60 2023-11-26T07:23:40.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.756+0000 7eff9b7fe640 1 --2- 172.21.15.31:0/2206977442 >> v2:172.21.15.97:3300/0 conn(0x7eff9c06bb10 0x7eff9c06bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:40.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.756+0000 7eff9affd640 1 --2- 172.21.15.31:0/2206977442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c146c60 0x7eff9c070020 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7eff9c071060 tx=0x7eff8c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868e135398f7a633 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.756+0000 7eff9a7fc640 1 -- 172.21.15.31:0/2206977442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0aa5f0 con 0x7eff9c146c60 2023-11-26T07:23:40.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.756+0000 7eff9a7fc640 1 -- 172.21.15.31:0/2206977442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c0aa750 con 0x7eff9c146c60 2023-11-26T07:23:40.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.756+0000 7eff9a7fc640 1 -- 172.21.15.31:0/2206977442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0aaae0 con 0x7eff9c146c60 2023-11-26T07:23:40.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.756+0000 7effa1557640 1 -- 172.21.15.31:0/2206977442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c146c60 msgr2=0x7eff9c070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.756+0000 7effa1557640 1 --2- 172.21.15.31:0/2206977442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c146c60 0x7eff9c070020 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7eff9c071060 tx=0x7eff8c01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 -- 172.21.15.31:0/2206977442 shutdown_connections 2023-11-26T07:23:40.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 --2- 172.21.15.31:0/2206977442 >> v2:172.21.15.97:3300/0 conn(0x7eff9c06bb10 0x7eff9c06bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 --2- 172.21.15.31:0/2206977442 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff9c147040 0x7eff9c070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 --2- 172.21.15.31:0/2206977442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c146c60 0x7eff9c070020 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 -- 172.21.15.31:0/2206977442 >> 172.21.15.31:0/2206977442 conn(0x7eff9c0b2ea0 msgr2=0x7eff9c0b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:40.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 -- 172.21.15.31:0/2206977442 shutdown_connections 2023-11-26T07:23:40.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 -- 172.21.15.31:0/2206977442 wait complete. 2023-11-26T07:23:40.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 Processor -- start 2023-11-26T07:23:40.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 -- start start 2023-11-26T07:23:40.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c06bb10 0x7eff9c13adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff9c146c60 0x7eff9c13b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c147040 0x7eff9c13bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c155420 con 0x7eff9c06bb10 2023-11-26T07:23:40.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c15a5b0 con 0x7eff9c147040 2023-11-26T07:23:40.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7effa1557640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c154dd0 con 0x7eff9c146c60 2023-11-26T07:23:40.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff9affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c06bb10 0x7eff9c13adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff9affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c06bb10 0x7eff9c13adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37460/0 (socket says 172.21.15.31:37460) 2023-11-26T07:23:40.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff9affd640 1 -- 172.21.15.31:0/1349413481 learned_addr learned my addr 172.21.15.31:0/1349413481 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:40.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff93fff640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff9c146c60 0x7eff9c13b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff9b7fe640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c147040 0x7eff9c13bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff9affd640 1 -- 172.21.15.31:0/1349413481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff9c146c60 msgr2=0x7eff9c13b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff9affd640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff9c146c60 0x7eff9c13b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff9affd640 1 -- 172.21.15.31:0/1349413481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c147040 msgr2=0x7eff9c13bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff9affd640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c147040 0x7eff9c13bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff9affd640 1 -- 172.21.15.31:0/1349413481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c0094d0 con 0x7eff9c06bb10 2023-11-26T07:23:40.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.760+0000 7eff9affd640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c06bb10 0x7eff9c13adc0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0ac040 tx=0x7eff8c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7eff937fe640 1 -- 172.21.15.31:0/1349413481 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0b3800 con 0x7eff9c06bb10 2023-11-26T07:23:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7eff937fe640 1 -- 172.21.15.31:0/1349413481 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c0b3980 con 0x7eff9c06bb10 2023-11-26T07:23:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7eff937fe640 1 -- 172.21.15.31:0/1349413481 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0b3d30 con 0x7eff9c06bb10 2023-11-26T07:23:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7effa1557640 1 -- 172.21.15.31:0/1349413481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c1550b0 con 0x7eff9c06bb10 2023-11-26T07:23:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7effa1557640 1 -- 172.21.15.31:0/1349413481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c13cc00 con 0x7eff9c06bb10 2023-11-26T07:23:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7effa1557640 1 -- 172.21.15.31:0/1349413481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c13cdf0 con 0x7eff9c06bb10 2023-11-26T07:23:40.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7eff937fe640 1 -- 172.21.15.31:0/1349413481 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7eff8c0b2380 con 0x7eff9c06bb10 2023-11-26T07:23:40.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7eff937fe640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7eff7c066740 0x7eff7c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:40.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7eff937fe640 1 -- 172.21.15.31:0/1349413481 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff8c0a7070 con 0x7eff9c06bb10 2023-11-26T07:23:40.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7eff93fff640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7eff7c066740 0x7eff7c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:40.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.764+0000 7eff93fff640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7eff7c066740 0x7eff7c068c00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7eff84006fd0 tx=0x7eff84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:40.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.768+0000 7eff937fe640 1 -- 172.21.15.31:0/1349413481 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7eff8c0ebcf0 con 0x7eff9c06bb10 2023-11-26T07:23:40.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 -- 172.21.15.31:0/1349413481 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff9c04e850 con 0x7eff9c06bb10 2023-11-26T07:23:40.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7eff937fe640 1 -- 172.21.15.31:0/1349413481 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16432 (secure 0 0 0) 0x7eff8c0c3090 con 0x7eff9c06bb10 2023-11-26T07:23:40.935 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:40.935 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":99,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:23:06.127026+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:02.513363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:03.492889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:40.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 -- 172.21.15.31:0/1349413481 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7eff7c066740 msgr2=0x7eff7c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7eff7c066740 0x7eff7c068c00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7eff84006fd0 tx=0x7eff84008040 comp rx=0 tx=0).stop 2023-11-26T07:23:40.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 -- 172.21.15.31:0/1349413481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c06bb10 msgr2=0x7eff9c13adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c06bb10 0x7eff9c13adc0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0ac040 tx=0x7eff8c01ad10 comp rx=0 tx=0).stop 2023-11-26T07:23:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 -- 172.21.15.31:0/1349413481 shutdown_connections 2023-11-26T07:23:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7eff7c066740 0x7eff7c068c00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7eff9c147040 0x7eff9c13bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7eff9c146c60 0x7eff9c13b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 --2- 172.21.15.31:0/1349413481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff9c06bb10 0x7eff9c13adc0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 -- 172.21.15.31:0/1349413481 >> 172.21.15.31:0/1349413481 conn(0x7eff9c0b2ea0 msgr2=0x7eff9c0d17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 -- 172.21.15.31:0/1349413481 shutdown_connections 2023-11-26T07:23:40.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:40.932+0000 7effa1557640 1 -- 172.21.15.31:0/1349413481 wait complete. 2023-11-26T07:23:40.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:23:02.513363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:23:03.492889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:40.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.100+0000 7f0171afe640 1 Processor -- start 2023-11-26T07:23:41.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f0171afe640 1 -- start start 2023-11-26T07:23:41.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f0171afe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f016c06e3f0 0x7f016c15f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f0171afe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c0ce5d0 0x7f016c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f0171afe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f016c159630 con 0x7f016c0cb9b0 2023-11-26T07:23:41.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f0171afe640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c06e860 con 0x7f016c06e3f0 2023-11-26T07:23:41.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f0171afe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c15bba0 con 0x7f016c0ce5d0 2023-11-26T07:23:41.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f016c0cb9b0 0x7f016c0cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57246/0 (socket says 172.21.15.31:57246) 2023-11-26T07:23:41.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016b7fe640 1 -- 172.21.15.31:0/3889912753 learned_addr learned my addr 172.21.15.31:0/3889912753 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016bfff640 1 --2- 172.21.15.31:0/3889912753 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c0ce5d0 0x7f016c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016affd640 1 --2- 172.21.15.31:0/3889912753 >> v2:172.21.15.97:3300/0 conn(0x7f016c06e3f0 0x7f016c15f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016a7fc640 1 -- 172.21.15.31:0/3889912753 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997670030 0 0) 0x7f016c159630 con 0x7f016c0cb9b0 2023-11-26T07:23:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016a7fc640 1 -- 172.21.15.31:0/3889912753 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0154003530 con 0x7f016c0cb9b0 2023-11-26T07:23:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016bfff640 1 -- 172.21.15.31:0/3889912753 >> v2:172.21.15.97:3300/0 conn(0x7f016c06e3f0 msgr2=0x7f016c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016bfff640 1 --2- 172.21.15.31:0/3889912753 >> v2:172.21.15.97:3300/0 conn(0x7f016c06e3f0 0x7f016c15f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016bfff640 1 -- 172.21.15.31:0/3889912753 >> v1:172.21.15.31:6789/0 conn(0x7f016c0cb9b0 legacy=0x7f016c0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016bfff640 1 -- 172.21.15.31:0/3889912753 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c15f6d0 con 0x7f016c0ce5d0 2023-11-26T07:23:41.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.104+0000 7f016affd640 1 --2- 172.21.15.31:0/3889912753 >> v2:172.21.15.97:3300/0 conn(0x7f016c06e3f0 0x7f016c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:41.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f016bfff640 1 --2- 172.21.15.31:0/3889912753 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c0ce5d0 0x7f016c15bfd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0164009d90 tx=0x7f016401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9215464d9d225c4a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:41.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f016a7fc640 1 -- 172.21.15.31:0/3889912753 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640a58e0 con 0x7f016c0ce5d0 2023-11-26T07:23:41.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f016a7fc640 1 -- 172.21.15.31:0/3889912753 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01640a5a40 con 0x7f016c0ce5d0 2023-11-26T07:23:41.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f016a7fc640 1 -- 172.21.15.31:0/3889912753 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640a5dd0 con 0x7f016c0ce5d0 2023-11-26T07:23:41.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f0171afe640 1 -- 172.21.15.31:0/3889912753 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c0ce5d0 msgr2=0x7f016c15bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f0171afe640 1 --2- 172.21.15.31:0/3889912753 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c0ce5d0 0x7f016c15bfd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0164009d90 tx=0x7f016401c950 comp rx=0 tx=0).stop 2023-11-26T07:23:41.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f0171afe640 1 -- 172.21.15.31:0/3889912753 shutdown_connections 2023-11-26T07:23:41.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f0171afe640 1 --2- 172.21.15.31:0/3889912753 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c0ce5d0 0x7f016c15bfd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f0171afe640 1 --2- 172.21.15.31:0/3889912753 >> v2:172.21.15.97:3300/0 conn(0x7f016c06e3f0 0x7f016c15f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f0171afe640 1 -- 172.21.15.31:0/3889912753 >> 172.21.15.31:0/3889912753 conn(0x7f016c0b0cb0 msgr2=0x7f016c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:41.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f0171afe640 1 -- 172.21.15.31:0/3889912753 shutdown_connections 2023-11-26T07:23:41.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.108+0000 7f0171afe640 1 -- 172.21.15.31:0/3889912753 wait complete. 2023-11-26T07:23:41.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 Processor -- start 2023-11-26T07:23:41.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 -- start start 2023-11-26T07:23:41.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c06e3f0 0x7f016c14fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f016c0cb9b0 0x7f016c154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f016c0ce5d0 0x7f016c0acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c14fff0 con 0x7f016c0cb9b0 2023-11-26T07:23:41.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0afc60 con 0x7f016c0ce5d0 2023-11-26T07:23:41.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0b1d60 con 0x7f016c06e3f0 2023-11-26T07:23:41.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f016c0cb9b0 0x7f016c154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f016c0cb9b0 0x7f016c154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37466/0 (socket says 172.21.15.31:37466) 2023-11-26T07:23:41.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016affd640 1 -- 172.21.15.31:0/40184611 learned_addr learned my addr 172.21.15.31:0/40184611 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:41.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016bfff640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f016c0ce5d0 0x7f016c0acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016b7fe640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c06e3f0 0x7f016c14fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016affd640 1 -- 172.21.15.31:0/40184611 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c06e3f0 msgr2=0x7f016c14fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016affd640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c06e3f0 0x7f016c14fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016affd640 1 -- 172.21.15.31:0/40184611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f016c0ce5d0 msgr2=0x7f016c0acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016affd640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f016c0ce5d0 0x7f016c0acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016affd640 1 -- 172.21.15.31:0/40184611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0160009420 con 0x7f016c0cb9b0 2023-11-26T07:23:41.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016b7fe640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c06e3f0 0x7f016c14fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:41.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f016affd640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f016c0cb9b0 0x7f016c154fd0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f016000e980 tx=0x7f016000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:41.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0153fff640 1 -- 172.21.15.31:0/40184611 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016000ca00 con 0x7f016c0cb9b0 2023-11-26T07:23:41.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0153fff640 1 -- 172.21.15.31:0/40184611 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016000cb60 con 0x7f016c0cb9b0 2023-11-26T07:23:41.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0153fff640 1 -- 172.21.15.31:0/40184611 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160013470 con 0x7f016c0cb9b0 2023-11-26T07:23:41.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 -- 172.21.15.31:0/40184611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0164009a70 con 0x7f016c0cb9b0 2023-11-26T07:23:41.118 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 -- 172.21.15.31:0/40184611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c155ab0 con 0x7f016c0cb9b0 2023-11-26T07:23:41.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.112+0000 7f0171afe640 1 -- 172.21.15.31:0/40184611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c0671f0 con 0x7f016c0cb9b0 2023-11-26T07:23:41.119 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.116+0000 7f0153fff640 1 -- 172.21.15.31:0/40184611 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f01600135d0 con 0x7f016c0cb9b0 2023-11-26T07:23:41.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.116+0000 7f0153fff640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0158066740 0x7f0158068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.116+0000 7f0153fff640 1 -- 172.21.15.31:0/40184611 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0160084460 con 0x7f016c0cb9b0 2023-11-26T07:23:41.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.116+0000 7f016b7fe640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0158066740 0x7f0158068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.116+0000 7f016b7fe640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0158066740 0x7f0158068c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f015c005890 tx=0x7f015c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:41.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.120+0000 7f0153fff640 1 -- 172.21.15.31:0/40184611 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0160051600 con 0x7f016c0cb9b0 2023-11-26T07:23:41.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 -- 172.21.15.31:0/40184611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f016c06b460 con 0x7f016c0cb9b0 2023-11-26T07:23:41.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0153fff640 1 -- 172.21.15.31:0/40184611 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0160054c20 con 0x7f016c0cb9b0 2023-11-26T07:23:41.283 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:23:41.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 -- 172.21.15.31:0/40184611 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0158066740 msgr2=0x7f0158068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0158066740 0x7f0158068c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f015c005890 tx=0x7f015c009040 comp rx=0 tx=0).stop 2023-11-26T07:23:41.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 -- 172.21.15.31:0/40184611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f016c0cb9b0 msgr2=0x7f016c154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f016c0cb9b0 0x7f016c154fd0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f016000e980 tx=0x7f016000ee40 comp rx=0 tx=0).stop 2023-11-26T07:23:41.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 -- 172.21.15.31:0/40184611 shutdown_connections 2023-11-26T07:23:41.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0158066740 0x7f0158068c00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f016c0ce5d0 0x7f016c0acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f016c0cb9b0 0x7f016c154fd0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 --2- 172.21.15.31:0/40184611 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f016c06e3f0 0x7f016c14fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 -- 172.21.15.31:0/40184611 >> 172.21.15.31:0/40184611 conn(0x7f016c0b0cb0 msgr2=0x7f016c071ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:41.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 -- 172.21.15.31:0/40184611 shutdown_connections 2023-11-26T07:23:41.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.280+0000 7f0171afe640 1 -- 172.21.15.31:0/40184611 wait complete. 2023-11-26T07:23:41.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:23:41.449 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.444+0000 7f770f9b8640 1 Processor -- start 2023-11-26T07:23:41.450 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770f9b8640 1 -- start start 2023-11-26T07:23:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770f9b8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77080b3c60 0x7f77080b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770f9b8640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77080b4040 0x7f77080ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770f9b8640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77080b2590 con 0x7f7708162190 2023-11-26T07:23:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770f9b8640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77080b2870 con 0x7f77080b4040 2023-11-26T07:23:41.452 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770f9b8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77080b2b50 con 0x7f77080b3c60 2023-11-26T07:23:41.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770cf53640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77080b4040 0x7f77080ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770cf53640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77080b4040 0x7f77080ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37474/0 (socket says 172.21.15.31:37474) 2023-11-26T07:23:41.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770cf53640 1 -- 172.21.15.31:0/3370914079 learned_addr learned my addr 172.21.15.31:0/3370914079 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:41.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770df55640 1 --1- 172.21.15.31:0/3370914079 >> v1:172.21.15.97:6789/0 conn(0x7f7708162190 0x7f7708165d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:35616/0 (socket says 172.21.15.31:35616) 2023-11-26T07:23:41.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770d754640 1 --2- 172.21.15.31:0/3370914079 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77080b3c60 0x7f77080b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770cf53640 1 -- 172.21.15.31:0/3370914079 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77080b3c60 msgr2=0x7f77080b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:41.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f76fffff640 1 -- 172.21.15.31:0/3370914079 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981097419 0 0) 0x7f77080b2590 con 0x7f7708162190 2023-11-26T07:23:41.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770cf53640 1 --2- 172.21.15.31:0/3370914079 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77080b3c60 0x7f77080b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770cf53640 1 -- 172.21.15.31:0/3370914079 >> v1:172.21.15.97:6789/0 conn(0x7f7708162190 legacy=0x7f7708165d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.448+0000 7f770cf53640 1 -- 172.21.15.31:0/3370914079 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77080b2e30 con 0x7f77080b4040 2023-11-26T07:23:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f770cf53640 1 --2- 172.21.15.31:0/3370914079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77080b4040 0x7f77080ce5e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009870 tx=0x7f76f801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32aa9c0df32cfd51 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f76fffff640 1 -- 172.21.15.31:0/3370914079 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80a6040 con 0x7f77080b4040 2023-11-26T07:23:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f76fffff640 1 -- 172.21.15.31:0/3370914079 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f80aa600 con 0x7f77080b4040 2023-11-26T07:23:41.455 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f76fffff640 1 -- 172.21.15.31:0/3370914079 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80aa990 con 0x7f77080b4040 2023-11-26T07:23:41.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f770f9b8640 1 -- 172.21.15.31:0/3370914079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77080b4040 msgr2=0x7f77080ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f770f9b8640 1 --2- 172.21.15.31:0/3370914079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77080b4040 0x7f77080ce5e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009870 tx=0x7f76f801c510 comp rx=0 tx=0).stop 2023-11-26T07:23:41.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f770f9b8640 1 -- 172.21.15.31:0/3370914079 shutdown_connections 2023-11-26T07:23:41.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f770f9b8640 1 --2- 172.21.15.31:0/3370914079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77080b4040 0x7f77080ce5e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f770f9b8640 1 --2- 172.21.15.31:0/3370914079 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77080b3c60 0x7f77080b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f770f9b8640 1 -- 172.21.15.31:0/3370914079 >> 172.21.15.31:0/3370914079 conn(0x7f77080b0cb0 msgr2=0x7f77080b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:41.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f770f9b8640 1 -- 172.21.15.31:0/3370914079 shutdown_connections 2023-11-26T07:23:41.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.452+0000 7f770f9b8640 1 -- 172.21.15.31:0/3370914079 wait complete. 2023-11-26T07:23:41.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 Processor -- start 2023-11-26T07:23:41.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 -- start start 2023-11-26T07:23:41.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77080b3c60 0x7f770809a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7708162190 0x7f7708147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.459 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77081586e0 0x7f7708148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770815b590 con 0x7f7708162190 2023-11-26T07:23:41.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770815afd0 con 0x7f77081586e0 2023-11-26T07:23:41.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770815b2b0 con 0x7f77080b3c60 2023-11-26T07:23:41.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770cf53640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7708162190 0x7f7708147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770cf53640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7708162190 0x7f7708147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37490/0 (socket says 172.21.15.31:37490) 2023-11-26T07:23:41.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770cf53640 1 -- 172.21.15.31:0/951142296 learned_addr learned my addr 172.21.15.31:0/951142296 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:41.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770df55640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77081586e0 0x7f7708148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770d754640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77080b3c60 0x7f770809a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770cf53640 1 -- 172.21.15.31:0/951142296 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77080b3c60 msgr2=0x7f770809a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.460 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770cf53640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77080b3c60 0x7f770809a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770cf53640 1 -- 172.21.15.31:0/951142296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77081586e0 msgr2=0x7f7708148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770cf53640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77081586e0 0x7f7708148300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770cf53640 1 -- 172.21.15.31:0/951142296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f80094d0 con 0x7f7708162190 2023-11-26T07:23:41.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770df55640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77081586e0 0x7f7708148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:41.461 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770cf53640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7708162190 0x7f7708147dc0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f76f801a140 tx=0x7f76f801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:41.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f76fdffb640 1 -- 172.21.15.31:0/951142296 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80b3600 con 0x7f7708162190 2023-11-26T07:23:41.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f76fdffb640 1 -- 172.21.15.31:0/951142296 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f80b3760 con 0x7f7708162190 2023-11-26T07:23:41.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f76fdffb640 1 -- 172.21.15.31:0/951142296 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80b3af0 con 0x7f7708162190 2023-11-26T07:23:41.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 -- 172.21.15.31:0/951142296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7708163330 con 0x7f7708162190 2023-11-26T07:23:41.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 -- 172.21.15.31:0/951142296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770809ac90 con 0x7f7708162190 2023-11-26T07:23:41.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.456+0000 7f770f9b8640 1 -- 172.21.15.31:0/951142296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77080671f0 con 0x7f7708162190 2023-11-26T07:23:41.462 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.460+0000 7f76fdffb640 1 -- 172.21.15.31:0/951142296 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f76f80b3c50 con 0x7f7708162190 2023-11-26T07:23:41.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.460+0000 7f76fdffb640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f76ec066740 0x7f76ec068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.460+0000 7f76fdffb640 1 -- 172.21.15.31:0/951142296 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76f811e660 con 0x7f7708162190 2023-11-26T07:23:41.463 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.460+0000 7f770d754640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f76ec066740 0x7f76ec068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.464 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.460+0000 7f770d754640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f76ec066740 0x7f76ec068c00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7704009af0 tx=0x7f7704008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:41.466 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.464+0000 7f76fdffb640 1 -- 172.21.15.31:0/951142296 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f76f80eb6d0 con 0x7f7708162190 2023-11-26T07:23:41.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 -- 172.21.15.31:0/951142296 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f770806b460 con 0x7f7708162190 2023-11-26T07:23:41.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f76fdffb640 1 -- 172.21.15.31:0/951142296 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76f80eecf0 con 0x7f7708162190 2023-11-26T07:23:41.622 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:23:41.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 -- 172.21.15.31:0/951142296 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f76ec066740 msgr2=0x7f76ec068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f76ec066740 0x7f76ec068c00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7704009af0 tx=0x7f7704008040 comp rx=0 tx=0).stop 2023-11-26T07:23:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 -- 172.21.15.31:0/951142296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7708162190 msgr2=0x7f7708147dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7708162190 0x7f7708147dc0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f76f801a140 tx=0x7f76f801ad50 comp rx=0 tx=0).stop 2023-11-26T07:23:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 -- 172.21.15.31:0/951142296 shutdown_connections 2023-11-26T07:23:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f76ec066740 0x7f76ec068c00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77081586e0 0x7f7708148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7708162190 0x7f7708147dc0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 --2- 172.21.15.31:0/951142296 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f77080b3c60 0x7f770809a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 -- 172.21.15.31:0/951142296 >> 172.21.15.31:0/951142296 conn(0x7f77080b0cb0 msgr2=0x7f77080ada40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 -- 172.21.15.31:0/951142296 shutdown_connections 2023-11-26T07:23:41.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.620+0000 7f770f9b8640 1 -- 172.21.15.31:0/951142296 wait complete. 2023-11-26T07:23:41.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:23:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.784+0000 7fdad02ea640 1 Processor -- start 2023-11-26T07:23:41.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.784+0000 7fdad02ea640 1 -- start start 2023-11-26T07:23:41.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdad02ea640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdac815a710 0x7fdac80abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdad02ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac80cbd50 0x7fdac80cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdad02ea640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80b2710 con 0x7fdac80cdfb0 2023-11-26T07:23:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdad02ea640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac80cca80 con 0x7fdac815a710 2023-11-26T07:23:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdad02ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac80b29f0 con 0x7fdac80cbd50 2023-11-26T07:23:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdacd885640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdac815a710 0x7fdac80abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdacd885640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdac815a710 0x7fdac80abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37506/0 (socket says 172.21.15.31:37506) 2023-11-26T07:23:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdacd885640 1 -- 172.21.15.31:0/2046691261 learned_addr learned my addr 172.21.15.31:0/2046691261 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:41.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdace887640 1 --2- 172.21.15.31:0/2046691261 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac80cbd50 0x7fdac80cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdacd885640 1 -- 172.21.15.31:0/2046691261 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac80cbd50 msgr2=0x7fdac80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdacd885640 1 --2- 172.21.15.31:0/2046691261 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac80cbd50 0x7fdac80cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdacd885640 1 -- 172.21.15.31:0/2046691261 >> v1:172.21.15.97:6789/0 conn(0x7fdac80cdfb0 legacy=0x7fdac80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdacd885640 1 -- 172.21.15.31:0/2046691261 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80b2cd0 con 0x7fdac815a710 2023-11-26T07:23:41.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.788+0000 7fdace887640 1 --2- 172.21.15.31:0/2046691261 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac80cbd50 0x7fdac80cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:41.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdacd885640 1 --2- 172.21.15.31:0/2046691261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac815a710 0x7fdac80abf90 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009870 tx=0x7fdab801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c8d326ee1c6689 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:41.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdacd084640 1 -- 172.21.15.31:0/2046691261 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80aa610 con 0x7fdac815a710 2023-11-26T07:23:41.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdacd084640 1 -- 172.21.15.31:0/2046691261 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab80a6040 con 0x7fdac815a710 2023-11-26T07:23:41.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdacd084640 1 -- 172.21.15.31:0/2046691261 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80aa990 con 0x7fdac815a710 2023-11-26T07:23:41.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdad02ea640 1 -- 172.21.15.31:0/2046691261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac815a710 msgr2=0x7fdac80abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdad02ea640 1 --2- 172.21.15.31:0/2046691261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac815a710 0x7fdac80abf90 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009870 tx=0x7fdab801c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdad02ea640 1 -- 172.21.15.31:0/2046691261 shutdown_connections 2023-11-26T07:23:41.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdad02ea640 1 --2- 172.21.15.31:0/2046691261 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac80cbd50 0x7fdac80cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdad02ea640 1 --2- 172.21.15.31:0/2046691261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac815a710 0x7fdac80abf90 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdad02ea640 1 -- 172.21.15.31:0/2046691261 >> 172.21.15.31:0/2046691261 conn(0x7fdac80b0cb0 msgr2=0x7fdac80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:41.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdad02ea640 1 -- 172.21.15.31:0/2046691261 shutdown_connections 2023-11-26T07:23:41.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdad02ea640 1 -- 172.21.15.31:0/2046691261 wait complete. 2023-11-26T07:23:41.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdad02ea640 1 Processor -- start 2023-11-26T07:23:41.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.792+0000 7fdad02ea640 1 -- start start 2023-11-26T07:23:41.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdad02ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac80cbd50 0x7fdac8153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdad02ea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac80cdfb0 0x7fdac8070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdad02ea640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac815a710 0x7fdac815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdad02ea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac81f68b0 con 0x7fdac80cdfb0 2023-11-26T07:23:41.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdad02ea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac815c690 con 0x7fdac80cbd50 2023-11-26T07:23:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdad02ea640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac815c970 con 0x7fdac815a710 2023-11-26T07:23:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdacd885640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac80cdfb0 0x7fdac8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdacd885640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac80cdfb0 0x7fdac8070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37508/0 (socket says 172.21.15.31:37508) 2023-11-26T07:23:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdacd885640 1 -- 172.21.15.31:0/312696430 learned_addr learned my addr 172.21.15.31:0/312696430 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdace887640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac815a710 0x7fdac815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdace086640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac80cbd50 0x7fdac8153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdacd885640 1 -- 172.21.15.31:0/312696430 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac815a710 msgr2=0x7fdac815d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdacd885640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac815a710 0x7fdac815d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdacd885640 1 -- 172.21.15.31:0/312696430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac80cbd50 msgr2=0x7fdac8153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdacd885640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac80cbd50 0x7fdac8153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdacd885640 1 -- 172.21.15.31:0/312696430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab80094d0 con 0x7fdac80cdfb0 2023-11-26T07:23:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdace086640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac80cbd50 0x7fdac8153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdacd885640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac80cdfb0 0x7fdac8070690 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fdab80ac040 tx=0x7fdab801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdabeffd640 1 -- 172.21.15.31:0/312696430 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80bd5f0 con 0x7fdac80cdfb0 2023-11-26T07:23:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdabeffd640 1 -- 172.21.15.31:0/312696430 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab80bd770 con 0x7fdac80cdfb0 2023-11-26T07:23:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdabeffd640 1 -- 172.21.15.31:0/312696430 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80bdb20 con 0x7fdac80cdfb0 2023-11-26T07:23:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdad02ea640 1 -- 172.21.15.31:0/312696430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac815cc50 con 0x7fdac80cdfb0 2023-11-26T07:23:41.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdad02ea640 1 -- 172.21.15.31:0/312696430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac8070bd0 con 0x7fdac80cdfb0 2023-11-26T07:23:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.796+0000 7fdad02ea640 1 -- 172.21.15.31:0/312696430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac80671f0 con 0x7fdac80cdfb0 2023-11-26T07:23:41.803 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.800+0000 7fdabeffd640 1 -- 172.21.15.31:0/312696430 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fdab80b33f0 con 0x7fdac80cdfb0 2023-11-26T07:23:41.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.800+0000 7fdabeffd640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdaac066790 0x7fdaac068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:41.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.800+0000 7fdabeffd640 1 -- 172.21.15.31:0/312696430 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdab811f2f0 con 0x7fdac80cdfb0 2023-11-26T07:23:41.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.800+0000 7fdace086640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdaac066790 0x7fdaac068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:41.804 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.800+0000 7fdace086640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdaac066790 0x7fdaac068c50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdac4003ec0 tx=0x7fdac4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:41.806 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.804+0000 7fdabeffd640 1 -- 172.21.15.31:0/312696430 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdab80ec400 con 0x7fdac80cdfb0 2023-11-26T07:23:41.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 -- 172.21.15.31:0/312696430 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdac815cab0 con 0x7fdac80cdfb0 2023-11-26T07:23:41.974 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdabeffd640 1 -- 172.21.15.31:0/312696430 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdac815cab0 con 0x7fdac80cdfb0 2023-11-26T07:23:41.974 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:23:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 -- 172.21.15.31:0/312696430 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdaac066790 msgr2=0x7fdaac068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdaac066790 0x7fdaac068c50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdac4003ec0 tx=0x7fdac4009040 comp rx=0 tx=0).stop 2023-11-26T07:23:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 -- 172.21.15.31:0/312696430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac80cdfb0 msgr2=0x7fdac8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac80cdfb0 0x7fdac8070690 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fdab80ac040 tx=0x7fdab801ad10 comp rx=0 tx=0).stop 2023-11-26T07:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 -- 172.21.15.31:0/312696430 shutdown_connections 2023-11-26T07:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdaac066790 0x7fdaac068c50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdac815a710 0x7fdac815d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdac80cdfb0 0x7fdac8070690 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 --2- 172.21.15.31:0/312696430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdac80cbd50 0x7fdac8153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 -- 172.21.15.31:0/312696430 >> 172.21.15.31:0/312696430 conn(0x7fdac80b0cb0 msgr2=0x7fdac80b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 -- 172.21.15.31:0/312696430 shutdown_connections 2023-11-26T07:23:41.977 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:41.972+0000 7fdad02ea640 1 -- 172.21.15.31:0/312696430 wait complete. 2023-11-26T07:23:41.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:23:42.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.136+0000 7f1b43dd1640 1 Processor -- start 2023-11-26T07:23:42.141 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.136+0000 7f1b43dd1640 1 -- start start 2023-11-26T07:23:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b43dd1640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f1b3c0cdfb0 0x7f1b3c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b43dd1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b3c0b2710 con 0x7f1b3c0cbd50 2023-11-26T07:23:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b43dd1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b3c0b29f0 con 0x7f1b3c15a710 2023-11-26T07:23:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b43dd1640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b3c0cca80 con 0x7f1b3c0cdfb0 2023-11-26T07:23:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b4236e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1b3c0cbd50 0x7f1b3c0cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57260/0 (socket says 172.21.15.31:57260) 2023-11-26T07:23:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b4236e640 1 -- 172.21.15.31:0/1252761268 learned_addr learned my addr 172.21.15.31:0/1252761268 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:42.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b41b6d640 1 --2- 172.21.15.31:0/1252761268 >> v2:172.21.15.97:3300/0 conn(0x7f1b3c0cdfb0 0x7f1b3c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187690135 0 0) 0x7f1b3c0b2710 con 0x7f1b3c0cbd50 2023-11-26T07:23:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b28003530 con 0x7f1b3c0cbd50 2023-11-26T07:23:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701560196 0 0) 0x7f1b3c0b29f0 con 0x7f1b3c15a710 2023-11-26T07:23:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b3c0b2710 con 0x7f1b3c15a710 2023-11-26T07:23:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2625662914 0 0) 0x7f1b28003530 con 0x7f1b3c0cbd50 2023-11-26T07:23:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 >> v2:172.21.15.97:3300/0 conn(0x7f1b3c0cdfb0 msgr2=0x7f1b3c0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 --2- 172.21.15.31:0/1252761268 >> v2:172.21.15.97:3300/0 conn(0x7f1b3c0cdfb0 0x7f1b3c0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 >> v1:172.21.15.97:6789/0 conn(0x7f1b3c15a710 legacy=0x7f1b3c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.145 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c0b2cd0 con 0x7f1b3c0cbd50 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f1b34001a00 con 0x7f1b3c0cbd50 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 >> v1:172.21.15.31:6789/0 conn(0x7f1b3c0cbd50 legacy=0x7f1b3c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b28003b60 0x7f1b28003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280047d0 0x7f1b28004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b2800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c0cca80 con 0x7f1b28005190 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2800c060 con 0x7f1b280047d0 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2800c220 con 0x7f1b28003b60 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b4236e640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b2800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b41b6d640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b28003b60 0x7f1b28003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b4136c640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280047d0 0x7f1b28004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.146 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b4236e640 1 -- 172.21.15.31:0/1252761268 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b28003b60 msgr2=0x7f1b28003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b4236e640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b28003b60 0x7f1b28003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b4236e640 1 -- 172.21.15.31:0/1252761268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280047d0 msgr2=0x7f1b28004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b4236e640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b280047d0 0x7f1b28004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.147 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.140+0000 7f1b4236e640 1 -- 172.21.15.31:0/1252761268 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b28003530 con 0x7f1b28005190 2023-11-26T07:23:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b4236e640 1 -- 172.21.15.31:0/1252761268 reap_dead start 2023-11-26T07:23:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b4236e640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b2800b990 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400a700 tx=0x7f1b3401f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c76348f37130c04 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b340b0070 con 0x7f1b28005190 2023-11-26T07:23:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b340ac3f0 con 0x7f1b28005190 2023-11-26T07:23:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b40b6b640 1 -- 172.21.15.31:0/1252761268 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b340ac7a0 con 0x7f1b28005190 2023-11-26T07:23:42.148 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b43dd1640 1 -- 172.21.15.31:0/1252761268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 msgr2=0x7f1b2800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b43dd1640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b2800b990 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400a700 tx=0x7f1b3401f940 comp rx=0 tx=0).stop 2023-11-26T07:23:42.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b43dd1640 1 -- 172.21.15.31:0/1252761268 shutdown_connections 2023-11-26T07:23:42.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b43dd1640 1 --2- 172.21.15.31:0/1252761268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b2800b990 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b43dd1640 1 -- 172.21.15.31:0/1252761268 >> 172.21.15.31:0/1252761268 conn(0x7f1b3c0b0cb0 msgr2=0x7f1b3c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:42.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b43dd1640 1 -- 172.21.15.31:0/1252761268 shutdown_connections 2023-11-26T07:23:42.149 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.144+0000 7f1b43dd1640 1 -- 172.21.15.31:0/1252761268 wait complete. 2023-11-26T07:23:42.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 Processor -- start 2023-11-26T07:23:42.150 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 -- start start 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b3c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b3c0b0520 0x7f1b3c15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c14db90 0x7f1b3c0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c15b3d0 con 0x7f1b28005190 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c15b6b0 con 0x7f1b3c14db90 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c15b990 con 0x7f1b3c0b0520 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b41b6d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b3c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b41b6d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b3c15c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37524/0 (socket says 172.21.15.31:37524) 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b41b6d640 1 -- 172.21.15.31:0/4213628903 learned_addr learned my addr 172.21.15.31:0/4213628903 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b41b6d640 1 -- 172.21.15.31:0/4213628903 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b3c0b0520 msgr2=0x7f1b3c15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b41b6d640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b3c0b0520 0x7f1b3c15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b41b6d640 1 -- 172.21.15.31:0/4213628903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c14db90 msgr2=0x7f1b3c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b41b6d640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c14db90 0x7f1b3c0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b41b6d640 1 -- 172.21.15.31:0/4213628903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3c15bc70 con 0x7f1b28005190 2023-11-26T07:23:42.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b41b6d640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b3c15c470 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1b38011960 tx=0x7f1b38011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:42.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b327fc640 1 -- 172.21.15.31:0/4213628903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800faf0 con 0x7f1b28005190 2023-11-26T07:23:42.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b327fc640 1 -- 172.21.15.31:0/4213628903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3800fc50 con 0x7f1b28005190 2023-11-26T07:23:42.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b327fc640 1 -- 172.21.15.31:0/4213628903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38014440 con 0x7f1b28005190 2023-11-26T07:23:42.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 -- 172.21.15.31:0/4213628903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3c15bf50 con 0x7f1b28005190 2023-11-26T07:23:42.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 -- 172.21.15.31:0/4213628903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3c14e010 con 0x7f1b28005190 2023-11-26T07:23:42.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.148+0000 7f1b43dd1640 1 -- 172.21.15.31:0/4213628903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3c0671f0 con 0x7f1b28005190 2023-11-26T07:23:42.154 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.152+0000 7f1b327fc640 1 -- 172.21.15.31:0/4213628903 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f1b380145a0 con 0x7f1b28005190 2023-11-26T07:23:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.152+0000 7f1b327fc640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1b20066770 0x7f1b20068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.152+0000 7f1b327fc640 1 -- 172.21.15.31:0/4213628903 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b38084c10 con 0x7f1b28005190 2023-11-26T07:23:42.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.152+0000 7f1b4136c640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1b20066770 0x7f1b20068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.152+0000 7f1b4136c640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1b20066770 0x7f1b20068c30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0b0400 tx=0x7f1b2c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:42.160 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.156+0000 7f1b327fc640 1 -- 172.21.15.31:0/4213628903 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1b38051cb0 con 0x7f1b28005190 2023-11-26T07:23:42.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 -- 172.21.15.31:0/4213628903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1b3c06b460 con 0x7f1b28005190 2023-11-26T07:23:42.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b327fc640 1 -- 172.21.15.31:0/4213628903 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1b38022020 con 0x7f1b28005190 2023-11-26T07:23:42.331 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:23:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 -- 172.21.15.31:0/4213628903 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1b20066770 msgr2=0x7f1b20068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1b20066770 0x7f1b20068c30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0b0400 tx=0x7f1b2c009040 comp rx=0 tx=0).stop 2023-11-26T07:23:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 -- 172.21.15.31:0/4213628903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 msgr2=0x7f1b3c15c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b3c15c470 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1b38011960 tx=0x7f1b38011e20 comp rx=0 tx=0).stop 2023-11-26T07:23:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 -- 172.21.15.31:0/4213628903 shutdown_connections 2023-11-26T07:23:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1b20066770 0x7f1b20068c30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b3c14db90 0x7f1b3c0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f1b3c0b0520 0x7f1b3c15ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 --2- 172.21.15.31:0/4213628903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b28005190 0x7f1b3c15c470 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 -- 172.21.15.31:0/4213628903 >> 172.21.15.31:0/4213628903 conn(0x7f1b3c0b0cb0 msgr2=0x7f1b3c15e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 -- 172.21.15.31:0/4213628903 shutdown_connections 2023-11-26T07:23:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.328+0000 7f1b43dd1640 1 -- 172.21.15.31:0/4213628903 wait complete. 2023-11-26T07:23:42.352 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-26T07:23:42.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:42.353 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-26T07:23:42.415 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:23:42.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:42.415 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-26T07:23:42.432 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:23:42.432 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:42.442 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:23:42.442 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:42.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:42.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.632+0000 7f465f1c1640 1 Processor -- start 2023-11-26T07:23:42.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.632+0000 7f465f1c1640 1 -- start start 2023-11-26T07:23:42.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.636+0000 7f465f1c1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4658144c80 0x7f46580abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.636+0000 7f465f1c1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4658145060 con 0x7f4658070df0 2023-11-26T07:23:42.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.636+0000 7f465f1c1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4658071fd0 con 0x7f46580ac460 2023-11-26T07:23:42.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.636+0000 7f465f1c1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46580ac8c0 con 0x7f4658144c80 2023-11-26T07:23:42.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.636+0000 7f465cf5d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4658144c80 0x7f46580abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.636+0000 7f465d75e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f4658070df0 0x7f465806c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57270/0 (socket says 172.21.15.31:57270) 2023-11-26T07:23:42.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.636+0000 7f465d75e640 1 -- 172.21.15.31:0/2037895672 learned_addr learned my addr 172.21.15.31:0/2037895672 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:42.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.636+0000 7f465cf5d640 1 -- 172.21.15.31:0/2037895672 >> v1:172.21.15.97:6789/0 conn(0x7f46580ac460 legacy=0x7f46580706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:42.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.636+0000 7f465cf5d640 1 -- 172.21.15.31:0/2037895672 >> v1:172.21.15.31:6789/0 conn(0x7f4658070df0 legacy=0x7f465806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.639 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.636+0000 7f465cf5d640 1 -- 172.21.15.31:0/2037895672 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4658071450 con 0x7f4658144c80 2023-11-26T07:23:42.641 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465cf5d640 1 --2- 172.21.15.31:0/2037895672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658144c80 0x7f46580abf20 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f464c0097c0 tx=0x7f464c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fd69019bfe8661 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:42.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f46577fe640 1 -- 172.21.15.31:0/2037895672 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0ab600 con 0x7f4658144c80 2023-11-26T07:23:42.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f46577fe640 1 -- 172.21.15.31:0/2037895672 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c0a7050 con 0x7f4658144c80 2023-11-26T07:23:42.642 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f46577fe640 1 -- 172.21.15.31:0/2037895672 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0ab980 con 0x7f4658144c80 2023-11-26T07:23:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 -- 172.21.15.31:0/2037895672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658144c80 msgr2=0x7f46580abf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 --2- 172.21.15.31:0/2037895672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658144c80 0x7f46580abf20 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f464c0097c0 tx=0x7f464c01c440 comp rx=0 tx=0).stop 2023-11-26T07:23:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 -- 172.21.15.31:0/2037895672 shutdown_connections 2023-11-26T07:23:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 --2- 172.21.15.31:0/2037895672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658144c80 0x7f46580abf20 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 -- 172.21.15.31:0/2037895672 >> 172.21.15.31:0/2037895672 conn(0x7f46580b0c40 msgr2=0x7f46580b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 -- 172.21.15.31:0/2037895672 shutdown_connections 2023-11-26T07:23:42.643 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 -- 172.21.15.31:0/2037895672 wait complete. 2023-11-26T07:23:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 Processor -- start 2023-11-26T07:23:42.644 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 -- start start 2023-11-26T07:23:42.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658070df0 0x7f465815a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46580ac460 0x7f465809b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4658144c80 0x7f465813c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465809bc20 con 0x7f4658070df0 2023-11-26T07:23:42.646 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465813e920 con 0x7f4658144c80 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.640+0000 7f465f1c1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465809ac00 con 0x7f46580ac460 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465cf5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658070df0 0x7f465815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465cf5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658070df0 0x7f465815a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37534/0 (socket says 172.21.15.31:37534) 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465cf5d640 1 -- 172.21.15.31:0/3381444488 learned_addr learned my addr 172.21.15.31:0/3381444488 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465d75e640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4658144c80 0x7f465813c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f4657fff640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46580ac460 0x7f465809b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465cf5d640 1 -- 172.21.15.31:0/3381444488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46580ac460 msgr2=0x7f465809b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465cf5d640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46580ac460 0x7f465809b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465cf5d640 1 -- 172.21.15.31:0/3381444488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4658144c80 msgr2=0x7f465813c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465cf5d640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4658144c80 0x7f465813c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465cf5d640 1 -- 172.21.15.31:0/3381444488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c009420 con 0x7f4658070df0 2023-11-26T07:23:42.647 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465d75e640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4658144c80 0x7f465813c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465cf5d640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658070df0 0x7f465815a090 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f464c019050 tx=0x7f464c0191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:42.648 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f46557fa640 1 -- 172.21.15.31:0/3381444488 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0b4610 con 0x7f4658070df0 2023-11-26T07:23:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f46557fa640 1 -- 172.21.15.31:0/3381444488 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c0b4790 con 0x7f4658070df0 2023-11-26T07:23:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f46557fa640 1 -- 172.21.15.31:0/3381444488 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0b4b40 con 0x7f4658070df0 2023-11-26T07:23:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465f1c1640 1 -- 172.21.15.31:0/3381444488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465809aee0 con 0x7f4658070df0 2023-11-26T07:23:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465f1c1640 1 -- 172.21.15.31:0/3381444488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465809a820 con 0x7f4658070df0 2023-11-26T07:23:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f465f1c1640 1 -- 172.21.15.31:0/3381444488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46580b1ba0 con 0x7f4658070df0 2023-11-26T07:23:42.649 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.644+0000 7f46557fa640 1 -- 172.21.15.31:0/3381444488 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f464c0b2040 con 0x7f4658070df0 2023-11-26T07:23:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.648+0000 7f46557fa640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f463c066770 0x7f463c068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.648+0000 7f46557fa640 1 -- 172.21.15.31:0/3381444488 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f464c11de00 con 0x7f4658070df0 2023-11-26T07:23:42.650 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.648+0000 7f4657fff640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f463c066770 0x7f463c068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.651 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.648+0000 7f4657fff640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f463c066770 0x7f463c068c30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f465815aba0 tx=0x7f4644009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:42.654 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.652+0000 7f46557fa640 1 -- 172.21.15.31:0/3381444488 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f464c0eade0 con 0x7f4658070df0 2023-11-26T07:23:42.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.812+0000 7f465f1c1640 1 -- 172.21.15.31:0/3381444488 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f465809aa10 con 0x7f4658070df0 2023-11-26T07:23:42.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.812+0000 7f46557fa640 1 -- 172.21.15.31:0/3381444488 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16432 (secure 0 0 0) 0x7f464c0ee400 con 0x7f4658070df0 2023-11-26T07:23:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:42.817 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":99,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:23:06.127026+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:02.513363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:03.492889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 -- 172.21.15.31:0/3381444488 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f463c066770 msgr2=0x7f463c068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f463c066770 0x7f463c068c30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f465815aba0 tx=0x7f4644009040 comp rx=0 tx=0).stop 2023-11-26T07:23:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 -- 172.21.15.31:0/3381444488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658070df0 msgr2=0x7f465815a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658070df0 0x7f465815a090 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f464c019050 tx=0x7f464c0191e0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 -- 172.21.15.31:0/3381444488 shutdown_connections 2023-11-26T07:23:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f463c066770 0x7f463c068c30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4658144c80 0x7f465813c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f46580ac460 0x7f465809b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 --2- 172.21.15.31:0/3381444488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4658070df0 0x7f465815a090 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 -- 172.21.15.31:0/3381444488 >> 172.21.15.31:0/3381444488 conn(0x7f46580b0c40 msgr2=0x7f46580afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 -- 172.21.15.31:0/3381444488 shutdown_connections 2023-11-26T07:23:42.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.816+0000 7f465f1c1640 1 -- 172.21.15.31:0/3381444488 wait complete. 2023-11-26T07:23:42.839 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:23:02.513363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:23:03.492889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:42.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:42.988 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.984+0000 7f150759e640 1 Processor -- start 2023-11-26T07:23:42.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.984+0000 7f150759e640 1 -- start start 2023-11-26T07:23:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f150759e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f15080b4aa0 0x7f15080b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f150759e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15080b3220 0x7f150806bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f150759e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15080b4050 con 0x7f15081465e0 2023-11-26T07:23:42.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f150759e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15080b3690 con 0x7f15080b3220 2023-11-26T07:23:42.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f150759e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150815e4c0 con 0x7f15080b4aa0 2023-11-26T07:23:42.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f150659c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f15081465e0 0x7f150815d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57276/0 (socket says 172.21.15.31:57276) 2023-11-26T07:23:42.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f150659c640 1 -- 172.21.15.31:0/1767762631 learned_addr learned my addr 172.21.15.31:0/1767762631 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:42.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f1506d9d640 1 --2- 172.21.15.31:0/1767762631 >> v2:172.21.15.31:3300/0 conn(0x7f15080b3220 0x7f150806bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f1505d9b640 1 --2- 172.21.15.31:0/1767762631 >> v2:172.21.15.97:3300/0 conn(0x7f15080b4aa0 0x7f15080b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:42.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f150559a640 1 -- 172.21.15.31:0/1767762631 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2327911200 0 0) 0x7f15080b4050 con 0x7f15081465e0 2023-11-26T07:23:42.992 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f150559a640 1 -- 172.21.15.31:0/1767762631 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f0003530 con 0x7f15081465e0 2023-11-26T07:23:42.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f1506d9d640 1 -- 172.21.15.31:0/1767762631 >> v2:172.21.15.97:3300/0 conn(0x7f15080b4aa0 msgr2=0x7f15080b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:42.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f1506d9d640 1 --2- 172.21.15.31:0/1767762631 >> v2:172.21.15.97:3300/0 conn(0x7f15080b4aa0 0x7f15080b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f1506d9d640 1 -- 172.21.15.31:0/1767762631 >> v1:172.21.15.31:6789/0 conn(0x7f15081465e0 legacy=0x7f150815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.988+0000 7f1506d9d640 1 -- 172.21.15.31:0/1767762631 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150815e7a0 con 0x7f15080b3220 2023-11-26T07:23:42.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f1506d9d640 1 --2- 172.21.15.31:0/1767762631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b3220 0x7f150806bb90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f15080b3e10 tx=0x7f150001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff73254e4363cc5b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:42.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150559a640 1 -- 172.21.15.31:0/1767762631 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000a58e0 con 0x7f15080b3220 2023-11-26T07:23:42.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150559a640 1 -- 172.21.15.31:0/1767762631 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15000a5a40 con 0x7f15080b3220 2023-11-26T07:23:42.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150559a640 1 -- 172.21.15.31:0/1767762631 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000a5dd0 con 0x7f15080b3220 2023-11-26T07:23:42.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150759e640 1 -- 172.21.15.31:0/1767762631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b3220 msgr2=0x7f150806bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:42.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150759e640 1 --2- 172.21.15.31:0/1767762631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b3220 0x7f150806bb90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f15080b3e10 tx=0x7f150001c820 comp rx=0 tx=0).stop 2023-11-26T07:23:42.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150759e640 1 -- 172.21.15.31:0/1767762631 shutdown_connections 2023-11-26T07:23:42.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150759e640 1 --2- 172.21.15.31:0/1767762631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b3220 0x7f150806bb90 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150759e640 1 --2- 172.21.15.31:0/1767762631 >> v2:172.21.15.97:3300/0 conn(0x7f15080b4aa0 0x7f15080b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:42.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150759e640 1 -- 172.21.15.31:0/1767762631 >> 172.21.15.31:0/1767762631 conn(0x7f1508070e30 msgr2=0x7f150806a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:42.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150759e640 1 -- 172.21.15.31:0/1767762631 shutdown_connections 2023-11-26T07:23:42.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150759e640 1 -- 172.21.15.31:0/1767762631 wait complete. 2023-11-26T07:23:42.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.992+0000 7f150759e640 1 Processor -- start 2023-11-26T07:23:42.997 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f150759e640 1 -- start start 2023-11-26T07:23:43.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f150759e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15080b3220 0x7f1508139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f150759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b4aa0 0x7f1508156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f150759e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15081465e0 0x7f1508139b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f150759e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150813c5b0 con 0x7f15080b4aa0 2023-11-26T07:23:43.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f150759e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1508156af0 con 0x7f15081465e0 2023-11-26T07:23:43.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f150759e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1508158240 con 0x7f15080b3220 2023-11-26T07:23:43.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f1505d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b4aa0 0x7f1508156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f1505d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b4aa0 0x7f1508156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37562/0 (socket says 172.21.15.31:37562) 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f1505d9b640 1 -- 172.21.15.31:0/1680663917 learned_addr learned my addr 172.21.15.31:0/1680663917 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f1505d9b640 1 -- 172.21.15.31:0/1680663917 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15080b3220 msgr2=0x7f1508139180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f1505d9b640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15080b3220 0x7f1508139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f1505d9b640 1 -- 172.21.15.31:0/1680663917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15081465e0 msgr2=0x7f1508139b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f1505d9b640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15081465e0 0x7f1508139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f1505d9b640 1 -- 172.21.15.31:0/1680663917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1500009a70 con 0x7f15080b4aa0 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f1505d9b640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b4aa0 0x7f1508156010 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00e870 tx=0x7f14fc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f14f6ffd640 1 -- 172.21.15.31:0/1680663917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc009510 con 0x7f15080b4aa0 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f14f6ffd640 1 -- 172.21.15.31:0/1680663917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc00eec0 con 0x7f15080b4aa0 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f14f6ffd640 1 -- 172.21.15.31:0/1680663917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc00cb40 con 0x7f15080b4aa0 2023-11-26T07:23:43.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f150759e640 1 -- 172.21.15.31:0/1680663917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150813c790 con 0x7f15080b4aa0 2023-11-26T07:23:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f150759e640 1 -- 172.21.15.31:0/1680663917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150813cd50 con 0x7f15080b4aa0 2023-11-26T07:23:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:42.996+0000 7f150759e640 1 -- 172.21.15.31:0/1680663917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1508074990 con 0x7f15080b4aa0 2023-11-26T07:23:43.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.000+0000 7f14f6ffd640 1 -- 172.21.15.31:0/1680663917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f14fc00cca0 con 0x7f15080b4aa0 2023-11-26T07:23:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.000+0000 7f14f6ffd640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f14e8066740 0x7f14e8068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.000+0000 7f14f6ffd640 1 -- 172.21.15.31:0/1680663917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14fc084430 con 0x7f15080b4aa0 2023-11-26T07:23:43.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.000+0000 7f150659c640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f14e8066740 0x7f14e8068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.000+0000 7f150659c640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f14e8066740 0x7f14e8068c00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14f8007fd0 tx=0x7f14f8007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.004+0000 7f14f6ffd640 1 -- 172.21.15.31:0/1680663917 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f14fc0514d0 con 0x7f15080b4aa0 2023-11-26T07:23:43.167 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.164+0000 7f150759e640 1 -- 172.21.15.31:0/1680663917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1508158520 con 0x7f15080b4aa0 2023-11-26T07:23:43.168 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.164+0000 7f14f6ffd640 1 -- 172.21.15.31:0/1680663917 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14fc054af0 con 0x7f15080b4aa0 2023-11-26T07:23:43.168 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:23:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 -- 172.21.15.31:0/1680663917 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f14e8066740 msgr2=0x7f14e8068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f14e8066740 0x7f14e8068c00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f14f8007fd0 tx=0x7f14f8007900 comp rx=0 tx=0).stop 2023-11-26T07:23:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 -- 172.21.15.31:0/1680663917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b4aa0 msgr2=0x7f1508156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b4aa0 0x7f1508156010 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f14fc00e870 tx=0x7f14fc00ec70 comp rx=0 tx=0).stop 2023-11-26T07:23:43.170 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 -- 172.21.15.31:0/1680663917 shutdown_connections 2023-11-26T07:23:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f14e8066740 0x7f14e8068c00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15081465e0 0x7f1508139b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15080b4aa0 0x7f1508156010 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 --2- 172.21.15.31:0/1680663917 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f15080b3220 0x7f1508139180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 -- 172.21.15.31:0/1680663917 >> 172.21.15.31:0/1680663917 conn(0x7f1508070e30 msgr2=0x7f150815fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 -- 172.21.15.31:0/1680663917 shutdown_connections 2023-11-26T07:23:43.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.168+0000 7f150759e640 1 -- 172.21.15.31:0/1680663917 wait complete. 2023-11-26T07:23:43.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:23:43.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa963d01640 1 Processor -- start 2023-11-26T07:23:43.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa963d01640 1 -- start start 2023-11-26T07:23:43.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa963d01640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa95c0b4210 0x7fa95c0b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa963d01640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c147190 con 0x7fa95c15db40 2023-11-26T07:23:43.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa963d01640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c06d030 con 0x7fa95c146820 2023-11-26T07:23:43.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa963d01640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c0b4be0 con 0x7fa95c0b4210 2023-11-26T07:23:43.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa96129c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa95c0b4210 0x7fa95c0b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa96229e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fa95c15db40 0x7fa95c15df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:57282/0 (socket says 172.21.15.31:57282) 2023-11-26T07:23:43.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa96229e640 1 -- 172.21.15.31:0/886228680 learned_addr learned my addr 172.21.15.31:0/886228680 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:43.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa96129c640 1 -- 172.21.15.31:0/886228680 >> v1:172.21.15.97:6789/0 conn(0x7fa95c146820 legacy=0x7fa95c070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:43.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa96129c640 1 -- 172.21.15.31:0/886228680 >> v1:172.21.15.31:6789/0 conn(0x7fa95c15db40 legacy=0x7fa95c15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa96129c640 1 -- 172.21.15.31:0/886228680 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c146c00 con 0x7fa95c0b4210 2023-11-26T07:23:43.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.332+0000 7fa960a9b640 1 -- 172.21.15.31:0/886228680 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514010046 0 0) 0x7fa95c147190 con 0x7fa95c15db40 2023-11-26T07:23:43.340 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa96129c640 1 --2- 172.21.15.31:0/886228680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0b4210 0x7fa95c0b46a0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0097c0 tx=0x7fa94c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc55312988cc4776 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa960a9b640 1 -- 172.21.15.31:0/886228680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0ab610 con 0x7fa95c0b4210 2023-11-26T07:23:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa960a9b640 1 -- 172.21.15.31:0/886228680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c0ab770 con 0x7fa95c0b4210 2023-11-26T07:23:43.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa960a9b640 1 -- 172.21.15.31:0/886228680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0abb00 con 0x7fa95c0b4210 2023-11-26T07:23:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa963d01640 1 -- 172.21.15.31:0/886228680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0b4210 msgr2=0x7fa95c0b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa963d01640 1 --2- 172.21.15.31:0/886228680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0b4210 0x7fa95c0b46a0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0097c0 tx=0x7fa94c01c410 comp rx=0 tx=0).stop 2023-11-26T07:23:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa963d01640 1 -- 172.21.15.31:0/886228680 shutdown_connections 2023-11-26T07:23:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa963d01640 1 --2- 172.21.15.31:0/886228680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c0b4210 0x7fa95c0b46a0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa963d01640 1 -- 172.21.15.31:0/886228680 >> 172.21.15.31:0/886228680 conn(0x7fa95c0b2f10 msgr2=0x7fa95c0b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa963d01640 1 -- 172.21.15.31:0/886228680 shutdown_connections 2023-11-26T07:23:43.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.336+0000 7fa963d01640 1 -- 172.21.15.31:0/886228680 wait complete. 2023-11-26T07:23:43.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa963d01640 1 Processor -- start 2023-11-26T07:23:43.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa963d01640 1 -- start start 2023-11-26T07:23:43.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa963d01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa95c0b4210 0x7fa95c15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa963d01640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa95c146820 0x7fa95c13b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa963d01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15db40 0x7fa95c0ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa963d01640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c15c530 con 0x7fa95c15db40 2023-11-26T07:23:43.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa963d01640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c15b920 con 0x7fa95c0b4210 2023-11-26T07:23:43.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa963d01640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c15bc00 con 0x7fa95c146820 2023-11-26T07:23:43.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa96229e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15db40 0x7fa95c0ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa96229e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15db40 0x7fa95c0ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37576/0 (socket says 172.21.15.31:37576) 2023-11-26T07:23:43.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa96229e640 1 -- 172.21.15.31:0/3675213157 learned_addr learned my addr 172.21.15.31:0/3675213157 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:43.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa961a9d640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa95c0b4210 0x7fa95c15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa96129c640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa95c146820 0x7fa95c13b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa96229e640 1 -- 172.21.15.31:0/3675213157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa95c146820 msgr2=0x7fa95c13b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa96229e640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa95c146820 0x7fa95c13b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa96229e640 1 -- 172.21.15.31:0/3675213157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa95c0b4210 msgr2=0x7fa95c15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa96229e640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa95c0b4210 0x7fa95c15b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa96229e640 1 -- 172.21.15.31:0/3675213157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c009420 con 0x7fa95c15db40 2023-11-26T07:23:43.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa96229e640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15db40 0x7fa95c0ca600 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa95400c780 tx=0x7fa95400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa9527fc640 1 -- 172.21.15.31:0/3675213157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95400ce90 con 0x7fa95c15db40 2023-11-26T07:23:43.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa9527fc640 1 -- 172.21.15.31:0/3675213157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95400a640 con 0x7fa95c15db40 2023-11-26T07:23:43.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa9527fc640 1 -- 172.21.15.31:0/3675213157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95400a9d0 con 0x7fa95c15db40 2023-11-26T07:23:43.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa963d01640 1 -- 172.21.15.31:0/3675213157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c15bee0 con 0x7fa95c15db40 2023-11-26T07:23:43.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.340+0000 7fa963d01640 1 -- 172.21.15.31:0/3675213157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c0cab40 con 0x7fa95c15db40 2023-11-26T07:23:43.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.344+0000 7fa963d01640 1 -- 172.21.15.31:0/3675213157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c15c6a0 con 0x7fa95c15db40 2023-11-26T07:23:43.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.344+0000 7fa9527fc640 1 -- 172.21.15.31:0/3675213157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fa95400ab30 con 0x7fa95c15db40 2023-11-26T07:23:43.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.344+0000 7fa9527fc640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa940066740 0x7fa940068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.344+0000 7fa9527fc640 1 -- 172.21.15.31:0/3675213157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa954083820 con 0x7fa95c15db40 2023-11-26T07:23:43.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.344+0000 7fa961a9d640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa940066740 0x7fa940068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.344+0000 7fa961a9d640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa940066740 0x7fa940068c00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa958007fd0 tx=0x7fa958007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.348+0000 7fa9527fc640 1 -- 172.21.15.31:0/3675213157 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa954050930 con 0x7fa95c15db40 2023-11-26T07:23:43.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.504+0000 7fa963d01640 1 -- 172.21.15.31:0/3675213157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa95c06bb10 con 0x7fa95c15db40 2023-11-26T07:23:43.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.504+0000 7fa9527fc640 1 -- 172.21.15.31:0/3675213157 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa954018020 con 0x7fa95c15db40 2023-11-26T07:23:43.509 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:23:43.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 -- 172.21.15.31:0/3675213157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa940066740 msgr2=0x7fa940068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa940066740 0x7fa940068c00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa958007fd0 tx=0x7fa958007e40 comp rx=0 tx=0).stop 2023-11-26T07:23:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 -- 172.21.15.31:0/3675213157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15db40 msgr2=0x7fa95c0ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15db40 0x7fa95c0ca600 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa95400c780 tx=0x7fa95400cc40 comp rx=0 tx=0).stop 2023-11-26T07:23:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 -- 172.21.15.31:0/3675213157 shutdown_connections 2023-11-26T07:23:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa940066740 0x7fa940068c00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa95c15db40 0x7fa95c0ca600 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa95c146820 0x7fa95c13b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 --2- 172.21.15.31:0/3675213157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa95c0b4210 0x7fa95c15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 -- 172.21.15.31:0/3675213157 >> 172.21.15.31:0/3675213157 conn(0x7fa95c0b2f10 msgr2=0x7fa95c13b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:43.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 -- 172.21.15.31:0/3675213157 shutdown_connections 2023-11-26T07:23:43.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.508+0000 7fa963d01640 1 -- 172.21.15.31:0/3675213157 wait complete. 2023-11-26T07:23:43.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:23:43.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.672+0000 7ff0c16bb640 1 Processor -- start 2023-11-26T07:23:43.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.672+0000 7ff0c16bb640 1 -- start start 2023-11-26T07:23:43.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0c16bb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0bc158cc0 0x7ff0bc071ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0c16bb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0bc0b4ab0 0x7ff0bc070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.679 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0c16bb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc0726d0 con 0x7ff0bc0720e0 2023-11-26T07:23:43.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0c16bb640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0bc15c100 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0c16bb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc1590a0 con 0x7ff0bc158cc0 2023-11-26T07:23:43.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0bb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0bc0b4ab0 0x7ff0bc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0bb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff0bc0b4ab0 0x7ff0bc070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37070/0 (socket says 172.21.15.31:37070) 2023-11-26T07:23:43.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0bb7fe640 1 -- 172.21.15.31:0/1611279236 learned_addr learned my addr 172.21.15.31:0/1611279236 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:43.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0baffd640 1 --2- 172.21.15.31:0/1611279236 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0bc158cc0 0x7ff0bc071ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0bb7fe640 1 -- 172.21.15.31:0/1611279236 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0bc158cc0 msgr2=0x7ff0bc071ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:43.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0bb7fe640 1 --2- 172.21.15.31:0/1611279236 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0bc158cc0 0x7ff0bc071ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0bb7fe640 1 -- 172.21.15.31:0/1611279236 >> v1:172.21.15.97:6789/0 conn(0x7ff0bc0720e0 legacy=0x7ff0bc0b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:43.680 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.676+0000 7ff0bb7fe640 1 -- 172.21.15.31:0/1611279236 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc15b7d0 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0bb7fe640 1 --2- 172.21.15.31:0/1611279236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 0x7ff0bc070470 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009e50 tx=0x7ff0b401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90e1b4f36116516 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0ba7fc640 1 -- 172.21.15.31:0/1611279236 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40a5820 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0ba7fc640 1 -- 172.21.15.31:0/1611279236 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b40a5980 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.682 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0ba7fc640 1 -- 172.21.15.31:0/1611279236 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40a5d10 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1611279236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 msgr2=0x7ff0bc070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0c16bb640 1 --2- 172.21.15.31:0/1611279236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 0x7ff0bc070470 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009e50 tx=0x7ff0b401c9a0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.683 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1611279236 shutdown_connections 2023-11-26T07:23:43.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0c16bb640 1 --2- 172.21.15.31:0/1611279236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 0x7ff0bc070470 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0c16bb640 1 --2- 172.21.15.31:0/1611279236 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0bc158cc0 0x7ff0bc071ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1611279236 >> 172.21.15.31:0/1611279236 conn(0x7ff0bc0b0d90 msgr2=0x7ff0bc0b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:43.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1611279236 shutdown_connections 2023-11-26T07:23:43.684 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1611279236 wait complete. 2023-11-26T07:23:43.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0c16bb640 1 Processor -- start 2023-11-26T07:23:43.685 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.680+0000 7ff0c16bb640 1 -- start start 2023-11-26T07:23:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0c16bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc0720e0 0x7ff0bc143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0c16bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 0x7ff0bc14cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0c16bb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0bc158cc0 0x7ff0bc0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.686 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0c16bb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc144400 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0c16bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc14d0a0 con 0x7ff0bc0720e0 2023-11-26T07:23:43.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0c16bb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc14bac0 con 0x7ff0bc158cc0 2023-11-26T07:23:43.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 0x7ff0bc14cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 0x7ff0bc14cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37076/0 (socket says 172.21.15.31:37076) 2023-11-26T07:23:43.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b3fff640 1 -- 172.21.15.31:0/1428095806 learned_addr learned my addr 172.21.15.31:0/1428095806 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:43.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0baffd640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc0720e0 0x7ff0bc143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.687 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0bb7fe640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0bc158cc0 0x7ff0bc0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b3fff640 1 -- 172.21.15.31:0/1428095806 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0bc158cc0 msgr2=0x7ff0bc0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b3fff640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0bc158cc0 0x7ff0bc0b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.691 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b3fff640 1 -- 172.21.15.31:0/1428095806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc0720e0 msgr2=0x7ff0bc143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b3fff640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc0720e0 0x7ff0bc143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b3fff640 1 -- 172.21.15.31:0/1428095806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4009b30 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b3fff640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 0x7ff0bc14cb60 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400c450 tx=0x7ff0a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b37fe640 1 -- 172.21.15.31:0/1428095806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4012830 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b37fe640 1 -- 172.21.15.31:0/1428095806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a40129b0 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1428095806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0bc0cd790 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b37fe640 1 -- 172.21.15.31:0/1428095806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4012d60 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1428095806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0bc14c080 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1428095806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0bc074990 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.684+0000 7ff0b37fe640 1 -- 172.21.15.31:0/1428095806 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ff0a401c460 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.688+0000 7ff0b37fe640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff09c066740 0x7ff09c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:43.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.688+0000 7ff0b37fe640 1 -- 172.21.15.31:0/1428095806 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0a4083fd0 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.688+0000 7ff0baffd640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff09c066740 0x7ff09c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:43.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.688+0000 7ff0baffd640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff09c066740 0x7ff09c068c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac006fd0 tx=0x7ff0ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:43.694 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.692+0000 7ff0b37fe640 1 -- 172.21.15.31:0/1428095806 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0a4051170 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.852 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.848+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1428095806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0bc15da40 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.848+0000 7ff0b37fe640 1 -- 172.21.15.31:0/1428095806 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0a4054790 con 0x7ff0bc0b4ab0 2023-11-26T07:23:43.853 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:23:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1428095806 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff09c066740 msgr2=0x7ff09c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff09c066740 0x7ff09c068c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac006fd0 tx=0x7ff0ac008040 comp rx=0 tx=0).stop 2023-11-26T07:23:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1428095806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 msgr2=0x7ff0bc14cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 0x7ff0bc14cb60 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400c450 tx=0x7ff0a400c910 comp rx=0 tx=0).stop 2023-11-26T07:23:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1428095806 shutdown_connections 2023-11-26T07:23:43.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff09c066740 0x7ff09c068c00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff0bc158cc0 0x7ff0bc0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0bc0b4ab0 0x7ff0bc14cb60 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 --2- 172.21.15.31:0/1428095806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc0720e0 0x7ff0bc143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1428095806 >> 172.21.15.31:0/1428095806 conn(0x7ff0bc0b0d90 msgr2=0x7ff0bc157e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1428095806 shutdown_connections 2023-11-26T07:23:43.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:43.852+0000 7ff0c16bb640 1 -- 172.21.15.31:0/1428095806 wait complete. 2023-11-26T07:23:43.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-26T07:23:44.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc53759e640 1 Processor -- start 2023-11-26T07:23:44.024 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc53759e640 1 -- start start 2023-11-26T07:23:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc53759e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc53815d2a0 0x7fc5380d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc53759e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5380d2c20 0x7fc5380b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc53759e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5380b5180 0x7fc53806b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc53759e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5380ce8c0 con 0x7fc5380d2c20 2023-11-26T07:23:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc53759e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5380b6250 con 0x7fc53815d2a0 2023-11-26T07:23:44.026 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc53759e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc538161cf0 con 0x7fc5380b5180 2023-11-26T07:23:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc535d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5380d2c20 0x7fc5380b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc535d9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5380d2c20 0x7fc5380b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37084/0 (socket says 172.21.15.31:37084) 2023-11-26T07:23:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.020+0000 7fc535d9b640 1 -- 172.21.15.31:0/3760644901 learned_addr learned my addr 172.21.15.31:0/3760644901 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc536d9d640 1 --2- 172.21.15.31:0/3760644901 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5380b5180 0x7fc53806b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc53659c640 1 --2- 172.21.15.31:0/3760644901 >> v2:172.21.15.97:3300/0 conn(0x7fc53815d2a0 0x7fc5380d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc535d9b640 1 -- 172.21.15.31:0/3760644901 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5380b5180 msgr2=0x7fc53806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc535d9b640 1 --2- 172.21.15.31:0/3760644901 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5380b5180 0x7fc53806b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc535d9b640 1 -- 172.21.15.31:0/3760644901 >> v2:172.21.15.97:3300/0 conn(0x7fc53815d2a0 msgr2=0x7fc5380d26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc535d9b640 1 --2- 172.21.15.31:0/3760644901 >> v2:172.21.15.97:3300/0 conn(0x7fc53815d2a0 0x7fc5380d26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc535d9b640 1 -- 172.21.15.31:0/3760644901 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5380b55a0 con 0x7fc5380d2c20 2023-11-26T07:23:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc536d9d640 1 --2- 172.21.15.31:0/3760644901 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5380b5180 0x7fc53806b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc53659c640 1 --2- 172.21.15.31:0/3760644901 >> v2:172.21.15.97:3300/0 conn(0x7fc53815d2a0 0x7fc5380d26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:44.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc535d9b640 1 --2- 172.21.15.31:0/3760644901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380d2c20 0x7fc5380b4c40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc5380cdb60 tx=0x7fc52c01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c432c430b117302 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc53559a640 1 -- 172.21.15.31:0/3760644901 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0ab600 con 0x7fc5380d2c20 2023-11-26T07:23:44.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc53559a640 1 -- 172.21.15.31:0/3760644901 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c0a7050 con 0x7fc5380d2c20 2023-11-26T07:23:44.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.024+0000 7fc53559a640 1 -- 172.21.15.31:0/3760644901 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0ab980 con 0x7fc5380d2c20 2023-11-26T07:23:44.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 -- 172.21.15.31:0/3760644901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380d2c20 msgr2=0x7fc5380b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 --2- 172.21.15.31:0/3760644901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380d2c20 0x7fc5380b4c40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc5380cdb60 tx=0x7fc52c01c210 comp rx=0 tx=0).stop 2023-11-26T07:23:44.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 -- 172.21.15.31:0/3760644901 shutdown_connections 2023-11-26T07:23:44.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 --2- 172.21.15.31:0/3760644901 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc5380b5180 0x7fc53806b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 --2- 172.21.15.31:0/3760644901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380d2c20 0x7fc5380b4c40 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 --2- 172.21.15.31:0/3760644901 >> v2:172.21.15.97:3300/0 conn(0x7fc53815d2a0 0x7fc5380d26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 -- 172.21.15.31:0/3760644901 >> 172.21.15.31:0/3760644901 conn(0x7fc5380668a0 msgr2=0x7fc5380cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 -- 172.21.15.31:0/3760644901 shutdown_connections 2023-11-26T07:23:44.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 -- 172.21.15.31:0/3760644901 wait complete. 2023-11-26T07:23:44.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 Processor -- start 2023-11-26T07:23:44.032 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 -- start start 2023-11-26T07:23:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380b5180 0x7fc5380c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5380d2c20 0x7fc5380c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc53815d2a0 0x7fc53815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc538156ef0 con 0x7fc5380b5180 2023-11-26T07:23:44.034 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5380c6ac0 con 0x7fc5380d2c20 2023-11-26T07:23:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53759e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc538156c10 con 0x7fc53815d2a0 2023-11-26T07:23:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380b5180 0x7fc5380c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380b5180 0x7fc5380c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37092/0 (socket says 172.21.15.31:37092) 2023-11-26T07:23:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53659c640 1 -- 172.21.15.31:0/2864303637 learned_addr learned my addr 172.21.15.31:0/2864303637 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc535d9b640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5380d2c20 0x7fc5380c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc536d9d640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc53815d2a0 0x7fc53815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53659c640 1 -- 172.21.15.31:0/2864303637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc53815d2a0 msgr2=0x7fc53815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53659c640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc53815d2a0 0x7fc53815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53659c640 1 -- 172.21.15.31:0/2864303637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5380d2c20 msgr2=0x7fc5380c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.035 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53659c640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5380d2c20 0x7fc5380c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.028+0000 7fc53659c640 1 -- 172.21.15.31:0/2864303637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c0094d0 con 0x7fc5380b5180 2023-11-26T07:23:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc535d9b640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5380d2c20 0x7fc5380c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc53659c640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380b5180 0x7fc5380c6040 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fc52800d800 tx=0x7fc52800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc526ffd640 1 -- 172.21.15.31:0/2864303637 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52800b6c0 con 0x7fc5380b5180 2023-11-26T07:23:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc526ffd640 1 -- 172.21.15.31:0/2864303637 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52800b820 con 0x7fc5380b5180 2023-11-26T07:23:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc526ffd640 1 -- 172.21.15.31:0/2864303637 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52800bbd0 con 0x7fc5380b5180 2023-11-26T07:23:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc53759e640 1 -- 172.21.15.31:0/2864303637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc538154fa0 con 0x7fc5380b5180 2023-11-26T07:23:44.036 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc53759e640 1 -- 172.21.15.31:0/2864303637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc538155560 con 0x7fc5380b5180 2023-11-26T07:23:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc53759e640 1 -- 172.21.15.31:0/2864303637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5381556f0 con 0x7fc5380b5180 2023-11-26T07:23:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc526ffd640 1 -- 172.21.15.31:0/2864303637 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fc52800bd30 con 0x7fc5380b5180 2023-11-26T07:23:44.037 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc526ffd640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc518066740 0x7fc518068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.032+0000 7fc526ffd640 1 -- 172.21.15.31:0/2864303637 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc528084770 con 0x7fc5380b5180 2023-11-26T07:23:44.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.036+0000 7fc535d9b640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc518066740 0x7fc518068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.038 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.036+0000 7fc535d9b640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc518066740 0x7fc518068c00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0ad000 tx=0x7fc52c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.041 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.036+0000 7fc526ffd640 1 -- 172.21.15.31:0/2864303637 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc528051860 con 0x7fc5380b5180 2023-11-26T07:23:44.212 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.208+0000 7fc53759e640 1 -- 172.21.15.31:0/2864303637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc538167d90 con 0x7fc5380b5180 2023-11-26T07:23:44.213 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.208+0000 7fc526ffd640 1 -- 172.21.15.31:0/2864303637 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc528054e80 con 0x7fc5380b5180 2023-11-26T07:23:44.213 INFO:teuthology.orchestra.run.smithi031.stderr:listed 0 entries 2023-11-26T07:23:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 -- 172.21.15.31:0/2864303637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc518066740 msgr2=0x7fc518068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc518066740 0x7fc518068c00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0ad000 tx=0x7fc52c008040 comp rx=0 tx=0).stop 2023-11-26T07:23:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 -- 172.21.15.31:0/2864303637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380b5180 msgr2=0x7fc5380c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380b5180 0x7fc5380c6040 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fc52800d800 tx=0x7fc52800dcc0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 -- 172.21.15.31:0/2864303637 shutdown_connections 2023-11-26T07:23:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc518066740 0x7fc518068c00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc53815d2a0 0x7fc53815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.215 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5380d2c20 0x7fc5380c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 --2- 172.21.15.31:0/2864303637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5380b5180 0x7fc5380c6040 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 -- 172.21.15.31:0/2864303637 >> 172.21.15.31:0/2864303637 conn(0x7fc5380668a0 msgr2=0x7fc5380b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:44.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 -- 172.21.15.31:0/2864303637 shutdown_connections 2023-11-26T07:23:44.216 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.212+0000 7fc53759e640 1 -- 172.21.15.31:0/2864303637 wait complete. 2023-11-26T07:23:44.234 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-26T07:23:44.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:44.234 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-26T07:23:44.264 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:23:44.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:44.265 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-26T07:23:44.318 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:44.318 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:44.328 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:23:44.328 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:23:44.329 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:44.369 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:23:44.370 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:44.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb56811d640 1 Processor -- start 2023-11-26T07:23:44.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb56811d640 1 -- start start 2023-11-26T07:23:44.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb56811d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb56015d7b0 0x7fb5600d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb56811d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 0x7fb56006bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb56811d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5600cefd0 con 0x7fb5600cead0 2023-11-26T07:23:44.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb56811d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5600cf2b0 con 0x7fb56015d7b0 2023-11-26T07:23:44.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb56811d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56006b3f0 con 0x7fb5600d2080 2023-11-26T07:23:44.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb5656b8640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb5600cead0 0x7fb5600d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38854/0 (socket says 172.21.15.31:38854) 2023-11-26T07:23:44.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb5656b8640 1 -- 172.21.15.31:0/2183364830 learned_addr learned my addr 172.21.15.31:0/2183364830 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:44.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb565eb9640 1 --2- 172.21.15.31:0/2183364830 >> v2:172.21.15.97:3300/0 conn(0x7fb56015d7b0 0x7fb5600d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb5666ba640 1 --2- 172.21.15.31:0/2183364830 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 0x7fb56006bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb564eb7640 1 -- 172.21.15.31:0/2183364830 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679257475 0 0) 0x7fb5600cefd0 con 0x7fb5600cead0 2023-11-26T07:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb564eb7640 1 -- 172.21.15.31:0/2183364830 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb54c003530 con 0x7fb5600cead0 2023-11-26T07:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb5666ba640 1 -- 172.21.15.31:0/2183364830 >> v2:172.21.15.97:3300/0 conn(0x7fb56015d7b0 msgr2=0x7fb5600d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb5666ba640 1 --2- 172.21.15.31:0/2183364830 >> v2:172.21.15.97:3300/0 conn(0x7fb56015d7b0 0x7fb5600d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.518 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb5666ba640 1 -- 172.21.15.31:0/2183364830 >> v1:172.21.15.31:6789/0 conn(0x7fb5600cead0 legacy=0x7fb5600d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb5666ba640 1 -- 172.21.15.31:0/2183364830 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56006b6d0 con 0x7fb5600d2080 2023-11-26T07:23:44.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.512+0000 7fb565eb9640 1 --2- 172.21.15.31:0/2183364830 >> v2:172.21.15.97:3300/0 conn(0x7fb56015d7b0 0x7fb5600d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:44.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.516+0000 7fb5666ba640 1 --2- 172.21.15.31:0/2183364830 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 0x7fb56006bed0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb558009d90 tx=0x7fb55801c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=289f01079b7dcfa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.516+0000 7fb564eb7640 1 -- 172.21.15.31:0/2183364830 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5580a58e0 con 0x7fb5600d2080 2023-11-26T07:23:44.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.516+0000 7fb564eb7640 1 -- 172.21.15.31:0/2183364830 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5580a5a40 con 0x7fb5600d2080 2023-11-26T07:23:44.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.516+0000 7fb564eb7640 1 -- 172.21.15.31:0/2183364830 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5580a5db0 con 0x7fb5600d2080 2023-11-26T07:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.516+0000 7fb56811d640 1 -- 172.21.15.31:0/2183364830 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 msgr2=0x7fb56006bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.516+0000 7fb56811d640 1 --2- 172.21.15.31:0/2183364830 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 0x7fb56006bed0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb558009d90 tx=0x7fb55801c800 comp rx=0 tx=0).stop 2023-11-26T07:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.516+0000 7fb56811d640 1 -- 172.21.15.31:0/2183364830 shutdown_connections 2023-11-26T07:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.516+0000 7fb56811d640 1 --2- 172.21.15.31:0/2183364830 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 0x7fb56006bed0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.516+0000 7fb56811d640 1 --2- 172.21.15.31:0/2183364830 >> v2:172.21.15.97:3300/0 conn(0x7fb56015d7b0 0x7fb5600d10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 -- 172.21.15.31:0/2183364830 >> 172.21.15.31:0/2183364830 conn(0x7fb5600d33b0 msgr2=0x7fb5600d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 -- 172.21.15.31:0/2183364830 shutdown_connections 2023-11-26T07:23:44.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 -- 172.21.15.31:0/2183364830 wait complete. 2023-11-26T07:23:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 Processor -- start 2023-11-26T07:23:44.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 -- start start 2023-11-26T07:23:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5600cead0 0x7fb560155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 0x7fb560153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56015d7b0 0x7fb5600c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600c9e70 con 0x7fb5600cead0 2023-11-26T07:23:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560155c20 con 0x7fb56015d7b0 2023-11-26T07:23:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600c9b90 con 0x7fb5600d2080 2023-11-26T07:23:44.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb565eb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5600cead0 0x7fb560155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb565eb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5600cead0 0x7fb560155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37102/0 (socket says 172.21.15.31:37102) 2023-11-26T07:23:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb565eb9640 1 -- 172.21.15.31:0/612893395 learned_addr learned my addr 172.21.15.31:0/612893395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb5666ba640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56015d7b0 0x7fb5600c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb5656b8640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 0x7fb560153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb565eb9640 1 -- 172.21.15.31:0/612893395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 msgr2=0x7fb560153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.526 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb565eb9640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 0x7fb560153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb565eb9640 1 -- 172.21.15.31:0/612893395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56015d7b0 msgr2=0x7fb5600c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb565eb9640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56015d7b0 0x7fb5600c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb565eb9640 1 -- 172.21.15.31:0/612893395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c009420 con 0x7fb5600cead0 2023-11-26T07:23:44.527 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb565eb9640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5600cead0 0x7fb560155490 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00ebe0 tx=0x7fb55c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb5567fc640 1 -- 172.21.15.31:0/612893395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c00eee0 con 0x7fb5600cead0 2023-11-26T07:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb5567fc640 1 -- 172.21.15.31:0/612893395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c00cc30 con 0x7fb5600cead0 2023-11-26T07:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.520+0000 7fb56811d640 1 -- 172.21.15.31:0/612893395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb558009a70 con 0x7fb5600cead0 2023-11-26T07:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.524+0000 7fb5567fc640 1 -- 172.21.15.31:0/612893395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c013470 con 0x7fb5600cead0 2023-11-26T07:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.524+0000 7fb56811d640 1 -- 172.21.15.31:0/612893395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5600c7d70 con 0x7fb5600cead0 2023-11-26T07:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.524+0000 7fb56811d640 1 -- 172.21.15.31:0/612893395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5600c78c0 con 0x7fb5600cead0 2023-11-26T07:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.524+0000 7fb5567fc640 1 -- 172.21.15.31:0/612893395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fb55c0135d0 con 0x7fb5600cead0 2023-11-26T07:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.524+0000 7fb5567fc640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb544066740 0x7fb544068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.524+0000 7fb5567fc640 1 -- 172.21.15.31:0/612893395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb55c0845b0 con 0x7fb5600cead0 2023-11-26T07:23:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.524+0000 7fb5656b8640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb544066740 0x7fb544068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.524+0000 7fb5656b8640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb544066740 0x7fb544068c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb5500011f0 tx=0x7fb550009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.528+0000 7fb5567fc640 1 -- 172.21.15.31:0/612893395 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb55c0516b0 con 0x7fb5600cead0 2023-11-26T07:23:44.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.716+0000 7fb56811d640 1 -- 172.21.15.31:0/612893395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb56004e850 con 0x7fb5600cead0 2023-11-26T07:23:44.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.716+0000 7fb5567fc640 1 -- 172.21.15.31:0/612893395 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fb55c054cd0 con 0x7fb5600cead0 2023-11-26T07:23:44.721 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:44.721 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3797004646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3797004646},{"type":"v1","addr":"172.21.15.97:6833","nonce":3797004646}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2675576819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2675576819},{"type":"v1","addr":"172.21.15.97:6835","nonce":2675576819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1710103664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1710103664},{"type":"v1","addr":"172.21.15.31:6834","nonce":1710103664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:04.497314+0000","modified":"2023-11-26T07:23:07.528539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1001580514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1001580514},{"type":"v1","addr":"172.21.15.31:6835","nonce":1001580514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:23:44.721 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 73 2023-11-26T07:23:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 -- 172.21.15.31:0/612893395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb544066740 msgr2=0x7fb544068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb544066740 0x7fb544068c00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb5500011f0 tx=0x7fb550009040 comp rx=0 tx=0).stop 2023-11-26T07:23:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 -- 172.21.15.31:0/612893395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5600cead0 msgr2=0x7fb560155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5600cead0 0x7fb560155490 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00ebe0 tx=0x7fb55c00c520 comp rx=0 tx=0).stop 2023-11-26T07:23:44.723 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 -- 172.21.15.31:0/612893395 shutdown_connections 2023-11-26T07:23:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb544066740 0x7fb544068c00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb56015d7b0 0x7fb5600c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb5600d2080 0x7fb560153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 --2- 172.21.15.31:0/612893395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5600cead0 0x7fb560155490 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.724 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 -- 172.21.15.31:0/612893395 >> 172.21.15.31:0/612893395 conn(0x7fb5600d33b0 msgr2=0x7fb5600d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 -- 172.21.15.31:0/612893395 shutdown_connections 2023-11-26T07:23:44.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.720+0000 7fb56811d640 1 -- 172.21.15.31:0/612893395 wait complete. 2023-11-26T07:23:44.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:44.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.884+0000 7fc3d1b1d640 1 Processor -- start 2023-11-26T07:23:44.888 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.884+0000 7fc3d1b1d640 1 -- start start 2023-11-26T07:23:44.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.884+0000 7fc3d1b1d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3cc15d7b0 0x7fc3cc0d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3d1b1d640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3cc0cead0 0x7fc3cc0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3d1b1d640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3cc0d22c0 con 0x7fc3cc0d1eb0 2023-11-26T07:23:44.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3d1b1d640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3cc0cefd0 con 0x7fc3cc15d7b0 2023-11-26T07:23:44.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3d1b1d640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3cc0cf2b0 con 0x7fc3cc0cead0 2023-11-26T07:23:44.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3cb7fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3cc15d7b0 0x7fc3cc0d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3cbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc3cc0d1eb0 0x7fc3cc0cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38858/0 (socket says 172.21.15.31:38858) 2023-11-26T07:23:44.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3cbfff640 1 -- 172.21.15.31:0/2548043469 learned_addr learned my addr 172.21.15.31:0/2548043469 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:44.891 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3caffd640 1 --2- 172.21.15.31:0/2548043469 >> v2:172.21.15.97:3300/0 conn(0x7fc3cc0cead0 0x7fc3cc0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3cb7fe640 1 -- 172.21.15.31:0/2548043469 >> v2:172.21.15.97:3300/0 conn(0x7fc3cc0cead0 msgr2=0x7fc3cc0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:44.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3cb7fe640 1 --2- 172.21.15.31:0/2548043469 >> v2:172.21.15.97:3300/0 conn(0x7fc3cc0cead0 0x7fc3cc0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3cb7fe640 1 -- 172.21.15.31:0/2548043469 >> v1:172.21.15.31:6789/0 conn(0x7fc3cc0d1eb0 legacy=0x7fc3cc0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3cb7fe640 1 -- 172.21.15.31:0/2548043469 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc06b3f0 con 0x7fc3cc15d7b0 2023-11-26T07:23:44.892 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3ca7fc640 1 -- 172.21.15.31:0/2548043469 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900383227 0 0) 0x7fc3cc0d22c0 con 0x7fc3cc0d1eb0 2023-11-26T07:23:44.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3cb7fe640 1 --2- 172.21.15.31:0/2548043469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc15d7b0 0x7fc3cc0d10d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc08ba60 tx=0x7fc3bc01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9040ffedf5c3244f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3ca7fc640 1 -- 172.21.15.31:0/2548043469 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0ac640 con 0x7fc3cc15d7b0 2023-11-26T07:23:44.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3ca7fc640 1 -- 172.21.15.31:0/2548043469 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc0ac7a0 con 0x7fc3cc15d7b0 2023-11-26T07:23:44.893 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.888+0000 7fc3ca7fc640 1 -- 172.21.15.31:0/2548043469 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0acb30 con 0x7fc3cc15d7b0 2023-11-26T07:23:44.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/2548043469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc15d7b0 msgr2=0x7fc3cc0d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 --2- 172.21.15.31:0/2548043469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc15d7b0 0x7fc3cc0d10d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc08ba60 tx=0x7fc3bc01c310 comp rx=0 tx=0).stop 2023-11-26T07:23:44.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/2548043469 shutdown_connections 2023-11-26T07:23:44.894 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 --2- 172.21.15.31:0/2548043469 >> v2:172.21.15.97:3300/0 conn(0x7fc3cc0cead0 0x7fc3cc0d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 --2- 172.21.15.31:0/2548043469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc15d7b0 0x7fc3cc0d10d0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/2548043469 >> 172.21.15.31:0/2548043469 conn(0x7fc3cc0d33b0 msgr2=0x7fc3cc0d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:44.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/2548043469 shutdown_connections 2023-11-26T07:23:44.895 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/2548043469 wait complete. 2023-11-26T07:23:44.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 Processor -- start 2023-11-26T07:23:44.896 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 -- start start 2023-11-26T07:23:44.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc0cead0 0x7fc3cc1569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc0d1eb0 0x7fc3cc0b8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3cc15d7b0 0x7fc3cc0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0b8e40 con 0x7fc3cc0cead0 2023-11-26T07:23:44.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0caf50 con 0x7fc3cc0d1eb0 2023-11-26T07:23:44.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3d1b1d640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0ca620 con 0x7fc3cc15d7b0 2023-11-26T07:23:44.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3cb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc0cead0 0x7fc3cc1569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.899 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3cb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc0cead0 0x7fc3cc1569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37128/0 (socket says 172.21.15.31:37128) 2023-11-26T07:23:44.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.892+0000 7fc3cb7fe640 1 -- 172.21.15.31:0/3896963813 learned_addr learned my addr 172.21.15.31:0/3896963813 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:44.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3caffd640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc0d1eb0 0x7fc3cc0b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3cbfff640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3cc15d7b0 0x7fc3cc0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3cb7fe640 1 -- 172.21.15.31:0/3896963813 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3cc15d7b0 msgr2=0x7fc3cc0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3cb7fe640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3cc15d7b0 0x7fc3cc0b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3cb7fe640 1 -- 172.21.15.31:0/3896963813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc0d1eb0 msgr2=0x7fc3cc0b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:44.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3cb7fe640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc0d1eb0 0x7fc3cc0b8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:44.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3cb7fe640 1 -- 172.21.15.31:0/3896963813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc009420 con 0x7fc3cc0cead0 2023-11-26T07:23:44.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3caffd640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc0d1eb0 0x7fc3cc0b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:44.900 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3cb7fe640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc0cead0 0x7fc3cc1569d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc018f20 tx=0x7fc3bc0190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3b3fff640 1 -- 172.21.15.31:0/3896963813 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc004230 con 0x7fc3cc0cead0 2023-11-26T07:23:44.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3b3fff640 1 -- 172.21.15.31:0/3896963813 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc004390 con 0x7fc3cc0cead0 2023-11-26T07:23:44.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3b3fff640 1 -- 172.21.15.31:0/3896963813 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc003820 con 0x7fc3cc0cead0 2023-11-26T07:23:44.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/3896963813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3cc0ca900 con 0x7fc3cc0cead0 2023-11-26T07:23:44.901 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/3896963813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3cc156650 con 0x7fc3cc0cead0 2023-11-26T07:23:44.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/3896963813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3cc04e850 con 0x7fc3cc0cead0 2023-11-26T07:23:44.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3b3fff640 1 -- 172.21.15.31:0/3896963813 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fc3bc0b3040 con 0x7fc3cc0cead0 2023-11-26T07:23:44.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3b3fff640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3b8066740 0x7fc3b8068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:44.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.896+0000 7fc3b3fff640 1 -- 172.21.15.31:0/3896963813 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3bc11e410 con 0x7fc3cc0cead0 2023-11-26T07:23:44.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.900+0000 7fc3caffd640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3b8066740 0x7fc3b8068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:44.902 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.900+0000 7fc3caffd640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3b8066740 0x7fc3b8068c00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc157270 tx=0x7fc3c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:44.905 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:44.900+0000 7fc3b3fff640 1 -- 172.21.15.31:0/3896963813 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc3bc0eb540 con 0x7fc3cc0cead0 2023-11-26T07:23:45.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.064+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/3896963813 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3cc1598b0 con 0x7fc3cc0cead0 2023-11-26T07:23:45.069 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.064+0000 7fc3b3fff640 1 -- 172.21.15.31:0/3896963813 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16432 (secure 0 0 0) 0x7fc3bc0b1090 con 0x7fc3cc0cead0 2023-11-26T07:23:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:45.070 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":99,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:23:06.127026+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:02.513363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:03.492889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/3896963813 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3b8066740 msgr2=0x7fc3b8068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3b8066740 0x7fc3b8068c00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc157270 tx=0x7fc3c0008040 comp rx=0 tx=0).stop 2023-11-26T07:23:45.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/3896963813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc0cead0 msgr2=0x7fc3cc1569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc0cead0 0x7fc3cc1569d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc018f20 tx=0x7fc3bc0190b0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/3896963813 shutdown_connections 2023-11-26T07:23:45.072 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3b8066740 0x7fc3b8068c00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3cc15d7b0 0x7fc3cc0b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc0d1eb0 0x7fc3cc0b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 --2- 172.21.15.31:0/3896963813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3cc0cead0 0x7fc3cc1569d0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/3896963813 >> 172.21.15.31:0/3896963813 conn(0x7fc3cc0d33b0 msgr2=0x7fc3cc0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:45.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/3896963813 shutdown_connections 2023-11-26T07:23:45.073 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.068+0000 7fc3d1b1d640 1 -- 172.21.15.31:0/3896963813 wait complete. 2023-11-26T07:23:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-26T07:17:20.370306+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-26T07:23:02.513363+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-26T07:23:03.492889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-26T07:23:45.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-26T07:23:45.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.236+0000 7f80e812a640 1 Processor -- start 2023-11-26T07:23:45.240 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.236+0000 7f80e812a640 1 -- start start 2023-11-26T07:23:45.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e812a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f80e00ce5d0 0x7f80e015bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e812a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e0159630 con 0x7f80e006e3f0 2023-11-26T07:23:45.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e812a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e00cd1d0 con 0x7f80e00cb9b0 2023-11-26T07:23:45.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e812a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e006e860 con 0x7f80e00ce5d0 2023-11-26T07:23:45.242 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e56c5640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f80e006e3f0 0x7f80e015f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38866/0 (socket says 172.21.15.31:38866) 2023-11-26T07:23:45.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e56c5640 1 -- 172.21.15.31:0/450876987 learned_addr learned my addr 172.21.15.31:0/450876987 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e66c7640 1 --2- 172.21.15.31:0/450876987 >> v2:172.21.15.97:3300/0 conn(0x7f80e00ce5d0 0x7f80e015bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449135796 0 0) 0x7f80e0159630 con 0x7f80e006e3f0 2023-11-26T07:23:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80cc003530 con 0x7f80e006e3f0 2023-11-26T07:23:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2138552497 0 0) 0x7f80cc003530 con 0x7f80e006e3f0 2023-11-26T07:23:45.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 >> v2:172.21.15.97:3300/0 conn(0x7f80e00ce5d0 msgr2=0x7f80e015bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 --2- 172.21.15.31:0/450876987 >> v2:172.21.15.97:3300/0 conn(0x7f80e00ce5d0 0x7f80e015bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 >> v1:172.21.15.97:6789/0 conn(0x7f80e00cb9b0 legacy=0x7f80e00cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e00cd4b0 con 0x7f80e006e3f0 2023-11-26T07:23:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7f80d0001680 con 0x7f80e006e3f0 2023-11-26T07:23:45.246 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 >> v1:172.21.15.31:6789/0 conn(0x7f80e006e3f0 legacy=0x7f80e015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80cc003980 0x7f80cc003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80cc004380 0x7f80cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80cc005050 0x7f80cc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e006e860 con 0x7f80cc005050 2023-11-26T07:23:45.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc00bfb0 con 0x7f80cc004380 2023-11-26T07:23:45.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc00c170 con 0x7f80cc003980 2023-11-26T07:23:45.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e66c7640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80cc005050 0x7f80cc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e5ec6640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80cc003980 0x7f80cc003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e56c5640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80cc004380 0x7f80cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e66c7640 1 -- 172.21.15.31:0/450876987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80cc003980 msgr2=0x7f80cc003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:45.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e66c7640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80cc003980 0x7f80cc003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e66c7640 1 -- 172.21.15.31:0/450876987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80cc004380 msgr2=0x7f80cc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:45.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e66c7640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80cc004380 0x7f80cc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.240+0000 7f80e66c7640 1 -- 172.21.15.31:0/450876987 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc003530 con 0x7f80cc005050 2023-11-26T07:23:45.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e66c7640 1 -- 172.21.15.31:0/450876987 reap_dead start 2023-11-26T07:23:45.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e66c7640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80cc005050 0x7f80cc00b990 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009fc0 tx=0x7f80d801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e697fdd27268cf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:45.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8003ba0 con 0x7f80cc005050 2023-11-26T07:23:45.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d800f040 con 0x7f80cc005050 2023-11-26T07:23:45.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e4ec4640 1 -- 172.21.15.31:0/450876987 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80a59a0 con 0x7f80cc005050 2023-11-26T07:23:45.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e812a640 1 -- 172.21.15.31:0/450876987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80cc005050 msgr2=0x7f80cc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.254 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e812a640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80cc005050 0x7f80cc00b990 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009fc0 tx=0x7f80d801cca0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e812a640 1 -- 172.21.15.31:0/450876987 shutdown_connections 2023-11-26T07:23:45.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e812a640 1 --2- 172.21.15.31:0/450876987 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80cc005050 0x7f80cc00b990 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e812a640 1 -- 172.21.15.31:0/450876987 >> 172.21.15.31:0/450876987 conn(0x7f80e00b0cb0 msgr2=0x7f80e00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:45.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e812a640 1 -- 172.21.15.31:0/450876987 shutdown_connections 2023-11-26T07:23:45.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e812a640 1 -- 172.21.15.31:0/450876987 wait complete. 2023-11-26T07:23:45.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e812a640 1 Processor -- start 2023-11-26T07:23:45.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.244+0000 7f80e812a640 1 -- start start 2023-11-26T07:23:45.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e812a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80cc005050 0x7f80e014f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e812a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e009c810 0x7f80e009a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e812a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e015a580 0x7f80e015aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e812a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00c8330 con 0x7f80e015a580 2023-11-26T07:23:45.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e812a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e014f5a0 con 0x7f80e009c810 2023-11-26T07:23:45.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e812a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e015af40 con 0x7f80cc005050 2023-11-26T07:23:45.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e66c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e015a580 0x7f80e015aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e66c7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e015a580 0x7f80e015aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37144/0 (socket says 172.21.15.31:37144) 2023-11-26T07:23:45.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e66c7640 1 -- 172.21.15.31:0/1119376204 learned_addr learned my addr 172.21.15.31:0/1119376204 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:45.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e56c5640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e009c810 0x7f80e009a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e66c7640 1 -- 172.21.15.31:0/1119376204 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80cc005050 msgr2=0x7f80e014f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e66c7640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80cc005050 0x7f80e014f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e66c7640 1 -- 172.21.15.31:0/1119376204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e009c810 msgr2=0x7f80e009a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e66c7640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e009c810 0x7f80e009a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e66c7640 1 -- 172.21.15.31:0/1119376204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e009ac90 con 0x7f80e015a580 2023-11-26T07:23:45.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e56c5640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e009c810 0x7f80e009a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e66c7640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e015a580 0x7f80e015aa00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f80d801c710 tx=0x7f80d8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80d67fc640 1 -- 172.21.15.31:0/1119376204 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8003db0 con 0x7f80e015a580 2023-11-26T07:23:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80d67fc640 1 -- 172.21.15.31:0/1119376204 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d8004030 con 0x7f80e015a580 2023-11-26T07:23:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80d67fc640 1 -- 172.21.15.31:0/1119376204 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d8004420 con 0x7f80e015a580 2023-11-26T07:23:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e812a640 1 -- 172.21.15.31:0/1119376204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e00ac0e0 con 0x7f80e015a580 2023-11-26T07:23:45.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e812a640 1 -- 172.21.15.31:0/1119376204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e00ac6a0 con 0x7f80e015a580 2023-11-26T07:23:45.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.248+0000 7f80e812a640 1 -- 172.21.15.31:0/1119376204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e00671f0 con 0x7f80e015a580 2023-11-26T07:23:45.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.252+0000 7f80d67fc640 1 -- 172.21.15.31:0/1119376204 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f80d8004580 con 0x7f80e015a580 2023-11-26T07:23:45.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.252+0000 7f80d67fc640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80c40667c0 0x7f80c4068c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.252+0000 7f80d67fc640 1 -- 172.21.15.31:0/1119376204 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80d811fbf0 con 0x7f80e015a580 2023-11-26T07:23:45.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.252+0000 7f80e5ec6640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80c40667c0 0x7f80c4068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.252+0000 7f80e5ec6640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80c40667c0 0x7f80c4068c80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f80dc007fd0 tx=0x7f80dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:45.261 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.256+0000 7f80d67fc640 1 -- 172.21.15.31:0/1119376204 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f80d8126080 con 0x7f80e015a580 2023-11-26T07:23:45.419 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.416+0000 7f80e812a640 1 -- 172.21.15.31:0/1119376204 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80e006b460 con 0x7f80e015a580 2023-11-26T07:23:45.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.416+0000 7f80d67fc640 1 -- 172.21.15.31:0/1119376204 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80d80ecf40 con 0x7f80e015a580 2023-11-26T07:23:45.420 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2023-11-26T07:23:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 -- 172.21.15.31:0/1119376204 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80c40667c0 msgr2=0x7f80c4068c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80c40667c0 0x7f80c4068c80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f80dc007fd0 tx=0x7f80dc009040 comp rx=0 tx=0).stop 2023-11-26T07:23:45.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 -- 172.21.15.31:0/1119376204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e015a580 msgr2=0x7f80e015aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e015a580 0x7f80e015aa00 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f80d801c710 tx=0x7f80d8009750 comp rx=0 tx=0).stop 2023-11-26T07:23:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 -- 172.21.15.31:0/1119376204 shutdown_connections 2023-11-26T07:23:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f80c40667c0 0x7f80c4068c80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80e015a580 0x7f80e015aa00 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80e009c810 0x7f80e009a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 --2- 172.21.15.31:0/1119376204 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f80cc005050 0x7f80e014f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 -- 172.21.15.31:0/1119376204 >> 172.21.15.31:0/1119376204 conn(0x7f80e00b0cb0 msgr2=0x7f80e00cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 -- 172.21.15.31:0/1119376204 shutdown_connections 2023-11-26T07:23:45.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.420+0000 7f80e812a640 1 -- 172.21.15.31:0/1119376204 wait complete. 2023-11-26T07:23:45.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-26T07:23:45.587 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f7269f72640 1 Processor -- start 2023-11-26T07:23:45.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f7269f72640 1 -- start start 2023-11-26T07:23:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f7269f72640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f72640cdfb0 0x7f72640abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f7269f72640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72640b2710 con 0x7f726415a710 2023-11-26T07:23:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f7269f72640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72640b29f0 con 0x7f72640cbd50 2023-11-26T07:23:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f7269f72640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72640cca80 con 0x7f72640cdfb0 2023-11-26T07:23:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f72637fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f72640cdfb0 0x7f72640abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f72637fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f72640cdfb0 0x7f72640abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37150/0 (socket says 172.21.15.31:37150) 2023-11-26T07:23:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f72637fe640 1 -- 172.21.15.31:0/2117849139 learned_addr learned my addr 172.21.15.31:0/2117849139 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f72637fe640 1 -- 172.21.15.31:0/2117849139 >> v1:172.21.15.97:6789/0 conn(0x7f72640cbd50 legacy=0x7f72640cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f72637fe640 1 -- 172.21.15.31:0/2117849139 >> v1:172.21.15.31:6789/0 conn(0x7f726415a710 legacy=0x7f72640ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f72637fe640 1 -- 172.21.15.31:0/2117849139 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640b2cd0 con 0x7f72640cdfb0 2023-11-26T07:23:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.584+0000 7f72627fc640 1 -- 172.21.15.31:0/2117849139 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057689962 0 0) 0x7f72640b2710 con 0x7f726415a710 2023-11-26T07:23:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f72637fe640 1 --2- 172.21.15.31:0/2117849139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72640cdfb0 0x7f72640abf90 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f7254009870 tx=0x7f725401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b358ef5d5a34ceaf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f72627fc640 1 -- 172.21.15.31:0/2117849139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72540a95f0 con 0x7f72640cdfb0 2023-11-26T07:23:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f72627fc640 1 -- 172.21.15.31:0/2117849139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72540a9750 con 0x7f72640cdfb0 2023-11-26T07:23:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f72627fc640 1 -- 172.21.15.31:0/2117849139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72540a9ae0 con 0x7f72640cdfb0 2023-11-26T07:23:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f7269f72640 1 -- 172.21.15.31:0/2117849139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72640cdfb0 msgr2=0x7f72640abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f7269f72640 1 --2- 172.21.15.31:0/2117849139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72640cdfb0 0x7f72640abf90 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f7254009870 tx=0x7f725401c510 comp rx=0 tx=0).stop 2023-11-26T07:23:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f7269f72640 1 -- 172.21.15.31:0/2117849139 shutdown_connections 2023-11-26T07:23:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f7269f72640 1 --2- 172.21.15.31:0/2117849139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72640cdfb0 0x7f72640abf90 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f7269f72640 1 -- 172.21.15.31:0/2117849139 >> 172.21.15.31:0/2117849139 conn(0x7f72640b0cb0 msgr2=0x7f72640b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:45.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f7269f72640 1 -- 172.21.15.31:0/2117849139 shutdown_connections 2023-11-26T07:23:45.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.588+0000 7f7269f72640 1 -- 172.21.15.31:0/2117849139 wait complete. 2023-11-26T07:23:45.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7269f72640 1 Processor -- start 2023-11-26T07:23:45.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7269f72640 1 -- start start 2023-11-26T07:23:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7269f72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72640cbd50 0x7f726414b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7269f72640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72640cdfb0 0x7f726415b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7269f72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726415a710 0x7f726415c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7269f72640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7264099f70 con 0x7f726415a710 2023-11-26T07:23:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7269f72640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72641f58a0 con 0x7f72640cbd50 2023-11-26T07:23:45.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7269f72640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726415c5d0 con 0x7f72640cdfb0 2023-11-26T07:23:45.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7263fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726415a710 0x7f726415c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7263fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726415a710 0x7f726415c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37158/0 (socket says 172.21.15.31:37158) 2023-11-26T07:23:45.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7263fff640 1 -- 172.21.15.31:0/3126938486 learned_addr learned my addr 172.21.15.31:0/3126938486 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:45.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f72637fe640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72640cbd50 0x7f726414b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7262ffd640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72640cdfb0 0x7f726415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7263fff640 1 -- 172.21.15.31:0/3126938486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72640cdfb0 msgr2=0x7f726415b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7263fff640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72640cdfb0 0x7f726415b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7263fff640 1 -- 172.21.15.31:0/3126938486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72640cbd50 msgr2=0x7f726414b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7263fff640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72640cbd50 0x7f726414b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7263fff640 1 -- 172.21.15.31:0/3126938486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72540094d0 con 0x7f726415a710 2023-11-26T07:23:45.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f7263fff640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726415a710 0x7f726415c8b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c770 tx=0x7f725c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:45.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.592+0000 7f724bfff640 1 -- 172.21.15.31:0/3126938486 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c003b80 con 0x7f726415a710 2023-11-26T07:23:45.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.596+0000 7f724bfff640 1 -- 172.21.15.31:0/3126938486 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c003ce0 con 0x7f726415a710 2023-11-26T07:23:45.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.596+0000 7f724bfff640 1 -- 172.21.15.31:0/3126938486 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c00aa90 con 0x7f726415a710 2023-11-26T07:23:45.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.596+0000 7f7269f72640 1 -- 172.21.15.31:0/3126938486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7264099bd0 con 0x7f726415a710 2023-11-26T07:23:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.596+0000 7f7269f72640 1 -- 172.21.15.31:0/3126938486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726415c2f0 con 0x7f726415a710 2023-11-26T07:23:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.596+0000 7f7269f72640 1 -- 172.21.15.31:0/3126938486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72640671f0 con 0x7f726415a710 2023-11-26T07:23:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.596+0000 7f724bfff640 1 -- 172.21.15.31:0/3126938486 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f725c00abf0 con 0x7f726415a710 2023-11-26T07:23:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.596+0000 7f724bfff640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7250066740 0x7f7250068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.596+0000 7f724bfff640 1 -- 172.21.15.31:0/3126938486 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f725c083f70 con 0x7f726415a710 2023-11-26T07:23:45.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.596+0000 7f72637fe640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7250066740 0x7f7250068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.596+0000 7f72637fe640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7250066740 0x7f7250068c00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f72540ae000 tx=0x7f72540b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:45.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.600+0000 7f724bfff640 1 -- 172.21.15.31:0/3126938486 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f725c0511a0 con 0x7f726415a710 2023-11-26T07:23:45.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.756+0000 7f7269f72640 1 -- 172.21.15.31:0/3126938486 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7264099a30 con 0x7f726415a710 2023-11-26T07:23:45.762 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.756+0000 7f724bfff640 1 -- 172.21.15.31:0/3126938486 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7264099a30 con 0x7f726415a710 2023-11-26T07:23:45.762 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2023-11-26T07:23:45.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 -- 172.21.15.31:0/3126938486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7250066740 msgr2=0x7f7250068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7250066740 0x7f7250068c00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f72540ae000 tx=0x7f72540b3040 comp rx=0 tx=0).stop 2023-11-26T07:23:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 -- 172.21.15.31:0/3126938486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726415a710 msgr2=0x7f726415c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726415a710 0x7f726415c8b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c770 tx=0x7f725c00cc30 comp rx=0 tx=0).stop 2023-11-26T07:23:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 -- 172.21.15.31:0/3126938486 shutdown_connections 2023-11-26T07:23:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7250066740 0x7f7250068c00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f726415a710 0x7f726415c8b0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f72640cdfb0 0x7f726415b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 --2- 172.21.15.31:0/3126938486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72640cbd50 0x7f726414b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 -- 172.21.15.31:0/3126938486 >> 172.21.15.31:0/3126938486 conn(0x7f72640b0cb0 msgr2=0x7f72640ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:45.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 -- 172.21.15.31:0/3126938486 shutdown_connections 2023-11-26T07:23:45.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.760+0000 7f7269f72640 1 -- 172.21.15.31:0/3126938486 wait complete. 2023-11-26T07:23:45.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-26T07:23:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf602e3640 1 Processor -- start 2023-11-26T07:23:45.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf602e3640 1 -- start start 2023-11-26T07:23:45.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf602e3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fdf580b5830 0x7fdf5815e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf602e3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdf580acd70 0x7fdf580ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf602e3640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf580af630 con 0x7fdf580ad710 2023-11-26T07:23:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf602e3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf580acb10 con 0x7fdf580acd70 2023-11-26T07:23:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf602e3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf580ae5b0 con 0x7fdf580b5830 2023-11-26T07:23:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf5d87e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdf580acd70 0x7fdf580ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf5e880640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fdf580ad710 0x7fdf58161200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38890/0 (socket says 172.21.15.31:38890) 2023-11-26T07:23:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf5e880640 1 -- 172.21.15.31:0/2257211270 learned_addr learned my addr 172.21.15.31:0/2257211270 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf5e07f640 1 --2- 172.21.15.31:0/2257211270 >> v2:172.21.15.97:3300/0 conn(0x7fdf580b5830 0x7fdf5815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf5d87e640 1 -- 172.21.15.31:0/2257211270 >> v2:172.21.15.97:3300/0 conn(0x7fdf580b5830 msgr2=0x7fdf5815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:45.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf5d87e640 1 --2- 172.21.15.31:0/2257211270 >> v2:172.21.15.97:3300/0 conn(0x7fdf580b5830 0x7fdf5815e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf5d87e640 1 -- 172.21.15.31:0/2257211270 >> v1:172.21.15.31:6789/0 conn(0x7fdf580ad710 legacy=0x7fdf58161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf5d87e640 1 -- 172.21.15.31:0/2257211270 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580ae870 con 0x7fdf580acd70 2023-11-26T07:23:45.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.928+0000 7fdf5e07f640 1 --2- 172.21.15.31:0/2257211270 >> v2:172.21.15.97:3300/0 conn(0x7fdf580b5830 0x7fdf5815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:45.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf5d87e640 1 --2- 172.21.15.31:0/2257211270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580acd70 0x7fdf580ad1d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fdf58070a60 tx=0x7fdf4801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb957168527127d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf5d07d640 1 -- 172.21.15.31:0/2257211270 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480ad680 con 0x7fdf580acd70 2023-11-26T07:23:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf5d07d640 1 -- 172.21.15.31:0/2257211270 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf480ad7e0 con 0x7fdf580acd70 2023-11-26T07:23:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf5d07d640 1 -- 172.21.15.31:0/2257211270 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf480adb70 con 0x7fdf580acd70 2023-11-26T07:23:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf602e3640 1 -- 172.21.15.31:0/2257211270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580acd70 msgr2=0x7fdf580ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf602e3640 1 --2- 172.21.15.31:0/2257211270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580acd70 0x7fdf580ad1d0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fdf58070a60 tx=0x7fdf4801c460 comp rx=0 tx=0).stop 2023-11-26T07:23:45.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf602e3640 1 -- 172.21.15.31:0/2257211270 shutdown_connections 2023-11-26T07:23:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf602e3640 1 --2- 172.21.15.31:0/2257211270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580acd70 0x7fdf580ad1d0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf602e3640 1 --2- 172.21.15.31:0/2257211270 >> v2:172.21.15.97:3300/0 conn(0x7fdf580b5830 0x7fdf5815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf602e3640 1 -- 172.21.15.31:0/2257211270 >> 172.21.15.31:0/2257211270 conn(0x7fdf58071d80 msgr2=0x7fdf5806a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf602e3640 1 -- 172.21.15.31:0/2257211270 shutdown_connections 2023-11-26T07:23:45.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.932+0000 7fdf602e3640 1 -- 172.21.15.31:0/2257211270 wait complete. 2023-11-26T07:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.936+0000 7fdf602e3640 1 Processor -- start 2023-11-26T07:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.936+0000 7fdf602e3640 1 -- start start 2023-11-26T07:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf602e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf580acd70 0x7fdf580a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf602e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf580ad710 0x7fdf5814dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf602e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580b5830 0x7fdf5815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf602e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf58150910 con 0x7fdf580b5830 2023-11-26T07:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf602e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf58159df0 con 0x7fdf580ad710 2023-11-26T07:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf602e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580ab3d0 con 0x7fdf580acd70 2023-11-26T07:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5e880640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580b5830 0x7fdf5815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5e880640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580b5830 0x7fdf5815a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37176/0 (socket says 172.21.15.31:37176) 2023-11-26T07:23:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5e880640 1 -- 172.21.15.31:0/756205504 learned_addr learned my addr 172.21.15.31:0/756205504 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5e07f640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf580acd70 0x7fdf580a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5d87e640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf580ad710 0x7fdf5814dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5e880640 1 -- 172.21.15.31:0/756205504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf580acd70 msgr2=0x7fdf580a86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5e880640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf580acd70 0x7fdf580a86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5e880640 1 -- 172.21.15.31:0/756205504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf580ad710 msgr2=0x7fdf5814dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5e880640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf580ad710 0x7fdf5814dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5e880640 1 -- 172.21.15.31:0/756205504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48009420 con 0x7fdf580b5830 2023-11-26T07:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf5e880640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580b5830 0x7fdf5815a280 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000cbc0 tx=0x7fdf5000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:45.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf4effd640 1 -- 172.21.15.31:0/756205504 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000ad40 con 0x7fdf580b5830 2023-11-26T07:23:45.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf4effd640 1 -- 172.21.15.31:0/756205504 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf50012400 con 0x7fdf580b5830 2023-11-26T07:23:45.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf4effd640 1 -- 172.21.15.31:0/756205504 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000aea0 con 0x7fdf580b5830 2023-11-26T07:23:45.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf602e3640 1 -- 172.21.15.31:0/756205504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf581594c0 con 0x7fdf580b5830 2023-11-26T07:23:45.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf602e3640 1 -- 172.21.15.31:0/756205504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf58159a80 con 0x7fdf580b5830 2023-11-26T07:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.940+0000 7fdf602e3640 1 -- 172.21.15.31:0/756205504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5806e7e0 con 0x7fdf580b5830 2023-11-26T07:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.944+0000 7fdf4effd640 1 -- 172.21.15.31:0/756205504 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fdf50007070 con 0x7fdf580b5830 2023-11-26T07:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.944+0000 7fdf4effd640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdf3c066770 0x7fdf3c068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:45.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.944+0000 7fdf4effd640 1 -- 172.21.15.31:0/756205504 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf50083ed0 con 0x7fdf580b5830 2023-11-26T07:23:45.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.944+0000 7fdf5e07f640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdf3c066770 0x7fdf3c068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:45.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.944+0000 7fdf5e07f640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdf3c066770 0x7fdf3c068c30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400a9a0 tx=0x7fdf54008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:45.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:45.948+0000 7fdf4effd640 1 -- 172.21.15.31:0/756205504 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdf50051100 con 0x7fdf580b5830 2023-11-26T07:23:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 -- 172.21.15.31:0/756205504 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf580af630 con 0x7fdf580b5830 2023-11-26T07:23:46.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf4effd640 1 -- 172.21.15.31:0/756205504 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf580af630 con 0x7fdf580b5830 2023-11-26T07:23:46.112 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 64 2023-11-26T07:23:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 -- 172.21.15.31:0/756205504 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdf3c066770 msgr2=0x7fdf3c068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdf3c066770 0x7fdf3c068c30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400a9a0 tx=0x7fdf54008040 comp rx=0 tx=0).stop 2023-11-26T07:23:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 -- 172.21.15.31:0/756205504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580b5830 msgr2=0x7fdf5815a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580b5830 0x7fdf5815a280 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000cbc0 tx=0x7fdf5000a430 comp rx=0 tx=0).stop 2023-11-26T07:23:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 -- 172.21.15.31:0/756205504 shutdown_connections 2023-11-26T07:23:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fdf3c066770 0x7fdf3c068c30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf580b5830 0x7fdf5815a280 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf580ad710 0x7fdf5814dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 --2- 172.21.15.31:0/756205504 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fdf580acd70 0x7fdf580a86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 -- 172.21.15.31:0/756205504 >> 172.21.15.31:0/756205504 conn(0x7fdf58071d80 msgr2=0x7fdf58161000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 -- 172.21.15.31:0/756205504 shutdown_connections 2023-11-26T07:23:46.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.108+0000 7fdf602e3640 1 -- 172.21.15.31:0/756205504 wait complete. 2023-11-26T07:23:46.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:46.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.272+0000 7fb64e832640 1 Processor -- start 2023-11-26T07:23:46.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.272+0000 7fb64e832640 1 -- start start 2023-11-26T07:23:46.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64e832640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb64815d510 0x7fb6480b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64e832640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6480715b0 0x7fb648071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64e832640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64806cf50 con 0x7fb6480b3cc0 2023-11-26T07:23:46.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64e832640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64815d8f0 con 0x7fb6480715b0 2023-11-26T07:23:46.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64e832640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb648071f80 con 0x7fb64815d510 2023-11-26T07:23:46.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64cdcf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6480715b0 0x7fb648071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64cdcf640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6480715b0 0x7fb648071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37184/0 (socket says 172.21.15.31:37184) 2023-11-26T07:23:46.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64cdcf640 1 -- 172.21.15.31:0/3588521099 learned_addr learned my addr 172.21.15.31:0/3588521099 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:46.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb647fff640 1 --2- 172.21.15.31:0/3588521099 >> v2:172.21.15.97:3300/0 conn(0x7fb64815d510 0x7fb6480b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64cdcf640 1 -- 172.21.15.31:0/3588521099 >> v2:172.21.15.97:3300/0 conn(0x7fb64815d510 msgr2=0x7fb6480b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:46.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64cdcf640 1 --2- 172.21.15.31:0/3588521099 >> v2:172.21.15.97:3300/0 conn(0x7fb64815d510 0x7fb6480b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64cdcf640 1 -- 172.21.15.31:0/3588521099 >> v1:172.21.15.97:6789/0 conn(0x7fb6480b3cc0 legacy=0x7fb64806d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.276+0000 7fb64cdcf640 1 -- 172.21.15.31:0/3588521099 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64806cc20 con 0x7fb6480715b0 2023-11-26T07:23:46.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64cdcf640 1 --2- 172.21.15.31:0/3588521099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480715b0 0x7fb648071a40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb64806b8f0 tx=0x7fb64001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7efc6695e019ec server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:46.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb646ffd640 1 -- 172.21.15.31:0/3588521099 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6400a5860 con 0x7fb6480715b0 2023-11-26T07:23:46.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb646ffd640 1 -- 172.21.15.31:0/3588521099 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6400a59c0 con 0x7fb6480715b0 2023-11-26T07:23:46.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb646ffd640 1 -- 172.21.15.31:0/3588521099 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6400a5d50 con 0x7fb6480715b0 2023-11-26T07:23:46.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64e832640 1 -- 172.21.15.31:0/3588521099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480715b0 msgr2=0x7fb648071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64e832640 1 --2- 172.21.15.31:0/3588521099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480715b0 0x7fb648071a40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fb64806b8f0 tx=0x7fb64001c9a0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64e832640 1 -- 172.21.15.31:0/3588521099 shutdown_connections 2023-11-26T07:23:46.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64e832640 1 --2- 172.21.15.31:0/3588521099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480715b0 0x7fb648071a40 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64e832640 1 --2- 172.21.15.31:0/3588521099 >> v2:172.21.15.97:3300/0 conn(0x7fb64815d510 0x7fb6480b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64e832640 1 -- 172.21.15.31:0/3588521099 >> 172.21.15.31:0/3588521099 conn(0x7fb6480b1e40 msgr2=0x7fb648070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:46.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64e832640 1 -- 172.21.15.31:0/3588521099 shutdown_connections 2023-11-26T07:23:46.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64e832640 1 -- 172.21.15.31:0/3588521099 wait complete. 2023-11-26T07:23:46.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64e832640 1 Processor -- start 2023-11-26T07:23:46.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.280+0000 7fb64e832640 1 -- start start 2023-11-26T07:23:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb64e832640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6480715b0 0x7fb64814f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb64e832640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480b3cc0 0x7fb6480ce400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb64e832640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64815d510 0x7fb648139d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb64e832640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480cdfe0 con 0x7fb6480b3cc0 2023-11-26T07:23:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb64e832640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480cd380 con 0x7fb6480715b0 2023-11-26T07:23:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb64e832640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6480cd640 con 0x7fb64815d510 2023-11-26T07:23:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb6477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480b3cc0 0x7fb6480ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb6477fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480b3cc0 0x7fb6480ce400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37188/0 (socket says 172.21.15.31:37188) 2023-11-26T07:23:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb6477fe640 1 -- 172.21.15.31:0/314189457 learned_addr learned my addr 172.21.15.31:0/314189457 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb647fff640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6480715b0 0x7fb64814f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.287 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb64cdcf640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64815d510 0x7fb648139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb6477fe640 1 -- 172.21.15.31:0/314189457 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64815d510 msgr2=0x7fb648139d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb6477fe640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64815d510 0x7fb648139d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb6477fe640 1 -- 172.21.15.31:0/314189457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6480715b0 msgr2=0x7fb64814f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb6477fe640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6480715b0 0x7fb64814f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb6477fe640 1 -- 172.21.15.31:0/314189457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb640009b30 con 0x7fb6480b3cc0 2023-11-26T07:23:46.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb6477fe640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480b3cc0 0x7fb6480ce400 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00c4d0 tx=0x7fb63c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.284+0000 7fb644ff9640 1 -- 172.21.15.31:0/314189457 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0052d0 con 0x7fb6480b3cc0 2023-11-26T07:23:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.288+0000 7fb644ff9640 1 -- 172.21.15.31:0/314189457 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c005430 con 0x7fb6480b3cc0 2023-11-26T07:23:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.288+0000 7fb64e832640 1 -- 172.21.15.31:0/314189457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6480cd920 con 0x7fb6480b3cc0 2023-11-26T07:23:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.288+0000 7fb644ff9640 1 -- 172.21.15.31:0/314189457 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c012830 con 0x7fb6480b3cc0 2023-11-26T07:23:46.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.288+0000 7fb64e832640 1 -- 172.21.15.31:0/314189457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6480ce940 con 0x7fb6480b3cc0 2023-11-26T07:23:46.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.288+0000 7fb64e832640 1 -- 172.21.15.31:0/314189457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6480ce120 con 0x7fb6480b3cc0 2023-11-26T07:23:46.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.288+0000 7fb644ff9640 1 -- 172.21.15.31:0/314189457 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fb63c012990 con 0x7fb6480b3cc0 2023-11-26T07:23:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.288+0000 7fb644ff9640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb634066740 0x7fb634068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.288+0000 7fb644ff9640 1 -- 172.21.15.31:0/314189457 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb63c083c40 con 0x7fb6480b3cc0 2023-11-26T07:23:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.288+0000 7fb647fff640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb634066740 0x7fb634068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.292+0000 7fb647fff640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb634066740 0x7fb634068c00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb6480cedd0 tx=0x7fb638008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:46.297 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.292+0000 7fb644ff9640 1 -- 172.21.15.31:0/314189457 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb6480ce120 con 0x7fb6480b3cc0 2023-11-26T07:23:46.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.488+0000 7fb64e832640 1 -- 172.21.15.31:0/314189457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6480ce120 con 0x7fb6480b3cc0 2023-11-26T07:23:46.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.488+0000 7fb644ff9640 1 -- 172.21.15.31:0/314189457 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fb6480ce120 con 0x7fb6480b3cc0 2023-11-26T07:23:46.494 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:46.494 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3797004646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3797004646},{"type":"v1","addr":"172.21.15.97:6833","nonce":3797004646}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2675576819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2675576819},{"type":"v1","addr":"172.21.15.97:6835","nonce":2675576819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1710103664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1710103664},{"type":"v1","addr":"172.21.15.31:6834","nonce":1710103664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:04.497314+0000","modified":"2023-11-26T07:23:07.528539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1001580514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1001580514},{"type":"v1","addr":"172.21.15.31:6835","nonce":1001580514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:23:46.494 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 73 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 -- 172.21.15.31:0/314189457 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb634066740 msgr2=0x7fb634068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb634066740 0x7fb634068c00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb6480cedd0 tx=0x7fb638008040 comp rx=0 tx=0).stop 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 -- 172.21.15.31:0/314189457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480b3cc0 msgr2=0x7fb6480ce400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480b3cc0 0x7fb6480ce400 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00c4d0 tx=0x7fb63c00c990 comp rx=0 tx=0).stop 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 -- 172.21.15.31:0/314189457 shutdown_connections 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb634066740 0x7fb634068c00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb64815d510 0x7fb648139d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6480b3cc0 0x7fb6480ce400 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 --2- 172.21.15.31:0/314189457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6480715b0 0x7fb64814f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 -- 172.21.15.31:0/314189457 >> 172.21.15.31:0/314189457 conn(0x7fb6480b1e40 msgr2=0x7fb6480b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:46.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 -- 172.21.15.31:0/314189457 shutdown_connections 2023-11-26T07:23:46.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.492+0000 7fb64e832640 1 -- 172.21.15.31:0/314189457 wait complete. 2023-11-26T07:23:46.515 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:46.658 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff308ebb640 1 Processor -- start 2023-11-26T07:23:46.659 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff308ebb640 1 -- start start 2023-11-26T07:23:46.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff308ebb640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff30415d7b0 0x7ff3040d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff308ebb640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3040cefd0 con 0x7ff3040cead0 2023-11-26T07:23:46.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff308ebb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3040cf2b0 con 0x7ff3040d2080 2023-11-26T07:23:46.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff308ebb640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30406b3f0 con 0x7ff30415d7b0 2023-11-26T07:23:46.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff2fbfff640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff3040cead0 0x7ff3040d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38892/0 (socket says 172.21.15.31:38892) 2023-11-26T07:23:46.660 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff2fbfff640 1 -- 172.21.15.31:0/1534599881 learned_addr learned my addr 172.21.15.31:0/1534599881 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:46.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff30259c640 1 --2- 172.21.15.31:0/1534599881 >> v2:172.21.15.97:3300/0 conn(0x7ff30415d7b0 0x7ff3040d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff301d9b640 1 -- 172.21.15.31:0/1534599881 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502100505 0 0) 0x7ff3040cefd0 con 0x7ff3040cead0 2023-11-26T07:23:46.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff301d9b640 1 -- 172.21.15.31:0/1534599881 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2f0003530 con 0x7ff3040cead0 2023-11-26T07:23:46.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff30259c640 1 -- 172.21.15.31:0/1534599881 >> v1:172.21.15.97:6789/0 conn(0x7ff3040d2080 legacy=0x7ff30406bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff30259c640 1 -- 172.21.15.31:0/1534599881 >> v1:172.21.15.31:6789/0 conn(0x7ff3040cead0 legacy=0x7ff3040d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.656+0000 7ff30259c640 1 -- 172.21.15.31:0/1534599881 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30406b6d0 con 0x7ff30415d7b0 2023-11-26T07:23:46.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff30259c640 1 --2- 172.21.15.31:0/1534599881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff30415d7b0 0x7ff3040d10d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40097c0 tx=0x7ff2f401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a1be1b4f08f8bd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:46.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff301d9b640 1 -- 172.21.15.31:0/1534599881 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4004230 con 0x7ff30415d7b0 2023-11-26T07:23:46.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff301d9b640 1 -- 172.21.15.31:0/1534599881 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f40a7050 con 0x7ff30415d7b0 2023-11-26T07:23:46.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff301d9b640 1 -- 172.21.15.31:0/1534599881 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4003620 con 0x7ff30415d7b0 2023-11-26T07:23:46.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff308ebb640 1 -- 172.21.15.31:0/1534599881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff30415d7b0 msgr2=0x7ff3040d10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff308ebb640 1 --2- 172.21.15.31:0/1534599881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff30415d7b0 0x7ff3040d10d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40097c0 tx=0x7ff2f401c330 comp rx=0 tx=0).stop 2023-11-26T07:23:46.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff308ebb640 1 -- 172.21.15.31:0/1534599881 shutdown_connections 2023-11-26T07:23:46.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff308ebb640 1 --2- 172.21.15.31:0/1534599881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff30415d7b0 0x7ff3040d10d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff308ebb640 1 -- 172.21.15.31:0/1534599881 >> 172.21.15.31:0/1534599881 conn(0x7ff3040d33b0 msgr2=0x7ff3040d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:46.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff308ebb640 1 -- 172.21.15.31:0/1534599881 shutdown_connections 2023-11-26T07:23:46.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.660+0000 7ff308ebb640 1 -- 172.21.15.31:0/1534599881 wait complete. 2023-11-26T07:23:46.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff308ebb640 1 Processor -- start 2023-11-26T07:23:46.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff308ebb640 1 -- start start 2023-11-26T07:23:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff308ebb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3040cead0 0x7ff3040b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff308ebb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3040d2080 0x7ff3040c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff308ebb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff30415d7b0 0x7ff30413f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff308ebb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c9a20 con 0x7ff3040cead0 2023-11-26T07:23:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff308ebb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c8980 con 0x7ff3040d2080 2023-11-26T07:23:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff308ebb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c8c60 con 0x7ff30415d7b0 2023-11-26T07:23:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff30259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3040cead0 0x7ff3040b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff30259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3040cead0 0x7ff3040b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37192/0 (socket says 172.21.15.31:37192) 2023-11-26T07:23:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff30259c640 1 -- 172.21.15.31:0/2650483152 learned_addr learned my addr 172.21.15.31:0/2650483152 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:46.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff2fbfff640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3040d2080 0x7ff3040c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff302d9d640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff30415d7b0 0x7ff30413f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff30259c640 1 -- 172.21.15.31:0/2650483152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff30415d7b0 msgr2=0x7ff30413f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff30259c640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff30415d7b0 0x7ff30413f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.670 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff30259c640 1 -- 172.21.15.31:0/2650483152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3040d2080 msgr2=0x7ff3040c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff30259c640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3040d2080 0x7ff3040c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff30259c640 1 -- 172.21.15.31:0/2650483152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4009420 con 0x7ff3040cead0 2023-11-26T07:23:46.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff302d9d640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff30415d7b0 0x7ff30413f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:46.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff30259c640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3040cead0 0x7ff3040b61d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40ae040 tx=0x7ff2f40190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:46.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff2faffd640 1 -- 172.21.15.31:0/2650483152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4003b70 con 0x7ff3040cead0 2023-11-26T07:23:46.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff2faffd640 1 -- 172.21.15.31:0/2650483152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f4003cd0 con 0x7ff3040cead0 2023-11-26T07:23:46.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff308ebb640 1 -- 172.21.15.31:0/2650483152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3040c8f40 con 0x7ff3040cead0 2023-11-26T07:23:46.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff2faffd640 1 -- 172.21.15.31:0/2650483152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40ab660 con 0x7ff3040cead0 2023-11-26T07:23:46.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.664+0000 7ff308ebb640 1 -- 172.21.15.31:0/2650483152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3040c9500 con 0x7ff3040cead0 2023-11-26T07:23:46.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.668+0000 7ff308ebb640 1 -- 172.21.15.31:0/2650483152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30404e850 con 0x7ff3040cead0 2023-11-26T07:23:46.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.668+0000 7ff2faffd640 1 -- 172.21.15.31:0/2650483152 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ff2f40b2040 con 0x7ff3040cead0 2023-11-26T07:23:46.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.668+0000 7ff2faffd640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff2e4066740 0x7ff2e4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:46.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.668+0000 7ff2faffd640 1 -- 172.21.15.31:0/2650483152 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2f411e180 con 0x7ff3040cead0 2023-11-26T07:23:46.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.668+0000 7ff2fbfff640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff2e4066740 0x7ff2e4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:46.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.668+0000 7ff2fbfff640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff2e4066740 0x7ff2e4068c00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3040c6a70 tx=0x7ff2ec007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:46.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.672+0000 7ff2faffd640 1 -- 172.21.15.31:0/2650483152 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff2f40eb160 con 0x7ff3040cead0 2023-11-26T07:23:46.835 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.832+0000 7ff308ebb640 1 -- 172.21.15.31:0/2650483152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3040c96f0 con 0x7ff3040cead0 2023-11-26T07:23:46.836 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.832+0000 7ff2faffd640 1 -- 172.21.15.31:0/2650483152 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16432 (secure 0 0 0) 0x7ff2f40ee780 con 0x7ff3040cead0 2023-11-26T07:23:46.837 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:46.837 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":99,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:23:06.127026+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:02.513363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:03.492889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:46.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 -- 172.21.15.31:0/2650483152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff2e4066740 msgr2=0x7ff2e4068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff2e4066740 0x7ff2e4068c00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3040c6a70 tx=0x7ff2ec007d40 comp rx=0 tx=0).stop 2023-11-26T07:23:46.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 -- 172.21.15.31:0/2650483152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3040cead0 msgr2=0x7ff3040b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:46.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3040cead0 0x7ff3040b61d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40ae040 tx=0x7ff2f40190d0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 -- 172.21.15.31:0/2650483152 shutdown_connections 2023-11-26T07:23:46.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff2e4066740 0x7ff2e4068c00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.839 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff30415d7b0 0x7ff30413f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3040d2080 0x7ff3040c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 --2- 172.21.15.31:0/2650483152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3040cead0 0x7ff3040b61d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:46.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 -- 172.21.15.31:0/2650483152 >> 172.21.15.31:0/2650483152 conn(0x7ff3040d33b0 msgr2=0x7ff3040cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:46.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 -- 172.21.15.31:0/2650483152 shutdown_connections 2023-11-26T07:23:46.840 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.836+0000 7ff308ebb640 1 -- 172.21.15.31:0/2650483152 wait complete. 2023-11-26T07:23:46.858 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:23:46.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:47.000 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.996+0000 7f50d7d65640 1 Processor -- start 2023-11-26T07:23:47.001 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:46.996+0000 7f50d7d65640 1 -- start start 2023-11-26T07:23:47.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d7d65640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f50d00b39c0 0x7f50d006c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d7d65640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d006cb70 0x7f50d0070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d7d65640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d006cf90 con 0x7f50d015f540 2023-11-26T07:23:47.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d7d65640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d00b2650 con 0x7f50d00b39c0 2023-11-26T07:23:47.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d7d65640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00b2920 con 0x7f50d006cb70 2023-11-26T07:23:47.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d5b01640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f50d015f540 0x7f50d00b1f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38894/0 (socket says 172.21.15.31:38894) 2023-11-26T07:23:47.002 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d5b01640 1 -- 172.21.15.31:0/3000490707 learned_addr learned my addr 172.21.15.31:0/3000490707 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:47.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d5300640 1 --2- 172.21.15.31:0/3000490707 >> v2:172.21.15.97:3300/0 conn(0x7f50d00b39c0 0x7f50d006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d6302640 1 --2- 172.21.15.31:0/3000490707 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d006cb70 0x7f50d0070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.003 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d4aff640 1 -- 172.21.15.31:0/3000490707 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506614875 0 0) 0x7f50d006cf90 con 0x7f50d015f540 2023-11-26T07:23:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d4aff640 1 -- 172.21.15.31:0/3000490707 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50bc003530 con 0x7f50d015f540 2023-11-26T07:23:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d4aff640 1 -- 172.21.15.31:0/3000490707 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1636902765 0 0) 0x7f50bc003530 con 0x7f50d015f540 2023-11-26T07:23:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d6302640 1 -- 172.21.15.31:0/3000490707 >> v2:172.21.15.97:3300/0 conn(0x7f50d00b39c0 msgr2=0x7f50d006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d6302640 1 --2- 172.21.15.31:0/3000490707 >> v2:172.21.15.97:3300/0 conn(0x7f50d00b39c0 0x7f50d006c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d6302640 1 -- 172.21.15.31:0/3000490707 >> v1:172.21.15.31:6789/0 conn(0x7f50d015f540 legacy=0x7f50d00b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.004 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.000+0000 7f50d6302640 1 -- 172.21.15.31:0/3000490707 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00b2c00 con 0x7f50d006cb70 2023-11-26T07:23:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d6302640 1 --2- 172.21.15.31:0/3000490707 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d006cb70 0x7f50d0070880 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009d90 tx=0x7f50c801c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2088a21952724ae5 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d4aff640 1 -- 172.21.15.31:0/3000490707 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc0016a0 con 0x7f50d006cb70 2023-11-26T07:23:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d4aff640 1 -- 172.21.15.31:0/3000490707 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c80a55b0 con 0x7f50d006cb70 2023-11-26T07:23:47.006 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d4aff640 1 -- 172.21.15.31:0/3000490707 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c80a5920 con 0x7f50d006cb70 2023-11-26T07:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d7d65640 1 -- 172.21.15.31:0/3000490707 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d006cb70 msgr2=0x7f50d0070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d7d65640 1 --2- 172.21.15.31:0/3000490707 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d006cb70 0x7f50d0070880 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009d90 tx=0x7f50c801c7b0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d7d65640 1 -- 172.21.15.31:0/3000490707 shutdown_connections 2023-11-26T07:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d7d65640 1 --2- 172.21.15.31:0/3000490707 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d006cb70 0x7f50d0070880 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d7d65640 1 --2- 172.21.15.31:0/3000490707 >> v2:172.21.15.97:3300/0 conn(0x7f50d00b39c0 0x7f50d006c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d7d65640 1 -- 172.21.15.31:0/3000490707 >> 172.21.15.31:0/3000490707 conn(0x7f50d00b0c40 msgr2=0x7f50d00b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:47.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d7d65640 1 -- 172.21.15.31:0/3000490707 shutdown_connections 2023-11-26T07:23:47.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d7d65640 1 -- 172.21.15.31:0/3000490707 wait complete. 2023-11-26T07:23:47.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d7d65640 1 Processor -- start 2023-11-26T07:23:47.009 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.004+0000 7f50d7d65640 1 -- start start 2023-11-26T07:23:47.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d7d65640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50d006cb70 0x7f50d00ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d7d65640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d00b39c0 0x7f50d015bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d7d65640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d015f540 0x7f50d00af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d7d65640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d015ae60 con 0x7f50d015f540 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d7d65640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d015c320 con 0x7f50d006cb70 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d7d65640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d00adf40 con 0x7f50d00b39c0 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d6302640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d015f540 0x7f50d00af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d6302640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d015f540 0x7f50d00af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37194/0 (socket says 172.21.15.31:37194) 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d6302640 1 -- 172.21.15.31:0/3830556788 learned_addr learned my addr 172.21.15.31:0/3830556788 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d5300640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d00b39c0 0x7f50d015bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d5b01640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50d006cb70 0x7f50d00ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d6302640 1 -- 172.21.15.31:0/3830556788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d00b39c0 msgr2=0x7f50d015bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d6302640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d00b39c0 0x7f50d015bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d6302640 1 -- 172.21.15.31:0/3830556788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50d006cb70 msgr2=0x7f50d00ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d6302640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50d006cb70 0x7f50d00ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d6302640 1 -- 172.21.15.31:0/3830556788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c8009a70 con 0x7f50d015f540 2023-11-26T07:23:47.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d5300640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d00b39c0 0x7f50d015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:47.012 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d6302640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d015f540 0x7f50d00af2d0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f50c801d050 tx=0x7f50c8019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:47.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50c67fc640 1 -- 172.21.15.31:0/3830556788 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8007000 con 0x7f50d015f540 2023-11-26T07:23:47.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50c67fc640 1 -- 172.21.15.31:0/3830556788 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8009450 con 0x7f50d015f540 2023-11-26T07:23:47.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50c67fc640 1 -- 172.21.15.31:0/3830556788 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c80ad870 con 0x7f50d015f540 2023-11-26T07:23:47.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d7d65640 1 -- 172.21.15.31:0/3830556788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d015e6b0 con 0x7f50d015f540 2023-11-26T07:23:47.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d7d65640 1 -- 172.21.15.31:0/3830556788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d015ec70 con 0x7f50d015f540 2023-11-26T07:23:47.013 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50d7d65640 1 -- 172.21.15.31:0/3830556788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d015afa0 con 0x7f50d015f540 2023-11-26T07:23:47.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.008+0000 7f50c67fc640 1 -- 172.21.15.31:0/3830556788 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f50c80ad9d0 con 0x7f50d015f540 2023-11-26T07:23:47.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.012+0000 7f50c67fc640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50b4066740 0x7f50b4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.014 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.012+0000 7f50c67fc640 1 -- 172.21.15.31:0/3830556788 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50c811f4f0 con 0x7f50d015f540 2023-11-26T07:23:47.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.012+0000 7f50d5b01640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50b4066740 0x7f50b4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.015 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.012+0000 7f50d5b01640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50b4066740 0x7f50b4068c00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f50cc001ec0 tx=0x7f50cc001cd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:47.018 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.016+0000 7f50c67fc640 1 -- 172.21.15.31:0/3830556788 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50d015afa0 con 0x7f50d015f540 2023-11-26T07:23:47.217 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.212+0000 7f50d7d65640 1 -- 172.21.15.31:0/3830556788 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50d015afa0 con 0x7f50d015f540 2023-11-26T07:23:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.212+0000 7f50c67fc640 1 -- 172.21.15.31:0/3830556788 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f50d015afa0 con 0x7f50d015f540 2023-11-26T07:23:47.218 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:47.218 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3797004646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3797004646},{"type":"v1","addr":"172.21.15.97:6833","nonce":3797004646}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2675576819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2675576819},{"type":"v1","addr":"172.21.15.97:6835","nonce":2675576819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1710103664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1710103664},{"type":"v1","addr":"172.21.15.31:6834","nonce":1710103664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:04.497314+0000","modified":"2023-11-26T07:23:07.528539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1001580514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1001580514},{"type":"v1","addr":"172.21.15.31:6835","nonce":1001580514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:23:47.218 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 73 2023-11-26T07:23:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 -- 172.21.15.31:0/3830556788 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50b4066740 msgr2=0x7f50b4068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50b4066740 0x7f50b4068c00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f50cc001ec0 tx=0x7f50cc001cd0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 -- 172.21.15.31:0/3830556788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d015f540 msgr2=0x7f50d00af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.220 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d015f540 0x7f50d00af2d0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f50c801d050 tx=0x7f50c8019550 comp rx=0 tx=0).stop 2023-11-26T07:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 -- 172.21.15.31:0/3830556788 shutdown_connections 2023-11-26T07:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f50b4066740 0x7f50b4068c00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50d015f540 0x7f50d00af2d0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f50d00b39c0 0x7f50d015bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 --2- 172.21.15.31:0/3830556788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50d006cb70 0x7f50d00ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 -- 172.21.15.31:0/3830556788 >> 172.21.15.31:0/3830556788 conn(0x7f50d00b0c40 msgr2=0x7f50d0159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 -- 172.21.15.31:0/3830556788 shutdown_connections 2023-11-26T07:23:47.221 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.216+0000 7f50d7d65640 1 -- 172.21.15.31:0/3830556788 wait complete. 2023-11-26T07:23:47.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:23:47.382 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e48232640 1 Processor -- start 2023-11-26T07:23:47.383 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e48232640 1 -- start start 2023-11-26T07:23:47.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e48232640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e400d1da0 0x7f2e400d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e48232640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2e401613a0 0x7f2e40161830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e48232640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e400ce1f0 con 0x7f2e40160d00 2023-11-26T07:23:47.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e48232640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e40161d70 con 0x7f2e401613a0 2023-11-26T07:23:47.384 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e48232640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400cea70 con 0x7f2e400d1da0 2023-11-26T07:23:47.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e45fce640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2e40160d00 0x7f2e400cef00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38906/0 (socket says 172.21.15.31:38906) 2023-11-26T07:23:47.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e45fce640 1 -- 172.21.15.31:0/2552375163 learned_addr learned my addr 172.21.15.31:0/2552375163 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:47.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e467cf640 1 --2- 172.21.15.31:0/2552375163 >> v2:172.21.15.31:3300/0 conn(0x7f2e401613a0 0x7f2e40161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e457cd640 1 --2- 172.21.15.31:0/2552375163 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e400d1da0 0x7f2e400d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.385 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e44fcc640 1 -- 172.21.15.31:0/2552375163 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701926754 0 0) 0x7f2e400ce1f0 con 0x7f2e40160d00 2023-11-26T07:23:47.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e44fcc640 1 -- 172.21.15.31:0/2552375163 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e2c003530 con 0x7f2e40160d00 2023-11-26T07:23:47.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e467cf640 1 -- 172.21.15.31:0/2552375163 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e400d1da0 msgr2=0x7f2e400d3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:47.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e467cf640 1 --2- 172.21.15.31:0/2552375163 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e400d1da0 0x7f2e400d3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e467cf640 1 -- 172.21.15.31:0/2552375163 >> v1:172.21.15.31:6789/0 conn(0x7f2e40160d00 legacy=0x7f2e400cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.380+0000 7f2e467cf640 1 -- 172.21.15.31:0/2552375163 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e400cf610 con 0x7f2e401613a0 2023-11-26T07:23:47.387 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e467cf640 1 --2- 172.21.15.31:0/2552375163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 0x7f2e40161830 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009d90 tx=0x7f2e3801f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6c6126610c8224 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:47.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e44fcc640 1 -- 172.21.15.31:0/2552375163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380af600 con 0x7f2e401613a0 2023-11-26T07:23:47.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e44fcc640 1 -- 172.21.15.31:0/2552375163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e380ab050 con 0x7f2e401613a0 2023-11-26T07:23:47.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e44fcc640 1 -- 172.21.15.31:0/2552375163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380af980 con 0x7f2e401613a0 2023-11-26T07:23:47.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e48232640 1 -- 172.21.15.31:0/2552375163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 msgr2=0x7f2e40161830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e48232640 1 --2- 172.21.15.31:0/2552375163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 0x7f2e40161830 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009d90 tx=0x7f2e3801f130 comp rx=0 tx=0).stop 2023-11-26T07:23:47.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e48232640 1 -- 172.21.15.31:0/2552375163 shutdown_connections 2023-11-26T07:23:47.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e48232640 1 --2- 172.21.15.31:0/2552375163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 0x7f2e40161830 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e48232640 1 --2- 172.21.15.31:0/2552375163 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e400d1da0 0x7f2e400d3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e48232640 1 -- 172.21.15.31:0/2552375163 >> 172.21.15.31:0/2552375163 conn(0x7f2e4008afc0 msgr2=0x7f2e400d3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:47.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e48232640 1 -- 172.21.15.31:0/2552375163 shutdown_connections 2023-11-26T07:23:47.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.384+0000 7f2e48232640 1 -- 172.21.15.31:0/2552375163 wait complete. 2023-11-26T07:23:47.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e48232640 1 Processor -- start 2023-11-26T07:23:47.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e48232640 1 -- start start 2023-11-26T07:23:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e48232640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e400d1da0 0x7f2e40158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e48232640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e40160d00 0x7f2e40159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e48232640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 0x7f2e401e8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e48232640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400ca0a0 con 0x7f2e401613a0 2023-11-26T07:23:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e48232640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e400c9e00 con 0x7f2e40160d00 2023-11-26T07:23:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e48232640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401e8360 con 0x7f2e400d1da0 2023-11-26T07:23:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e467cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 0x7f2e401e8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e467cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 0x7f2e401e8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37216/0 (socket says 172.21.15.31:37216) 2023-11-26T07:23:47.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e467cf640 1 -- 172.21.15.31:0/2339805642 learned_addr learned my addr 172.21.15.31:0/2339805642 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:47.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e457cd640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e40160d00 0x7f2e40159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e45fce640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e400d1da0 0x7f2e40158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e467cf640 1 -- 172.21.15.31:0/2339805642 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e400d1da0 msgr2=0x7f2e40158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e467cf640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e400d1da0 0x7f2e40158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e467cf640 1 -- 172.21.15.31:0/2339805642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e40160d00 msgr2=0x7f2e40159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e467cf640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e40160d00 0x7f2e40159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e467cf640 1 -- 172.21.15.31:0/2339805642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38009a70 con 0x7f2e401613a0 2023-11-26T07:23:47.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e467cf640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 0x7f2e401e8e40 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2e380b2040 tx=0x7f2e3801bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:47.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e367fc640 1 -- 172.21.15.31:0/2339805642 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e38009450 con 0x7f2e401613a0 2023-11-26T07:23:47.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e367fc640 1 -- 172.21.15.31:0/2339805642 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e380b9410 con 0x7f2e401613a0 2023-11-26T07:23:47.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e367fc640 1 -- 172.21.15.31:0/2339805642 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e380b97c0 con 0x7f2e401613a0 2023-11-26T07:23:47.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e48232640 1 -- 172.21.15.31:0/2339805642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e401e8640 con 0x7f2e401613a0 2023-11-26T07:23:47.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.388+0000 7f2e48232640 1 -- 172.21.15.31:0/2339805642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e40159130 con 0x7f2e401613a0 2023-11-26T07:23:47.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.392+0000 7f2e48232640 1 -- 172.21.15.31:0/2339805642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e401e81c0 con 0x7f2e401613a0 2023-11-26T07:23:47.395 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.392+0000 7f2e367fc640 1 -- 172.21.15.31:0/2339805642 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f2e380b6050 con 0x7f2e401613a0 2023-11-26T07:23:47.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.392+0000 7f2e367fc640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2e24066740 0x7f2e24068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.392+0000 7f2e367fc640 1 -- 172.21.15.31:0/2339805642 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e381228b0 con 0x7f2e401613a0 2023-11-26T07:23:47.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.392+0000 7f2e45fce640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2e24066740 0x7f2e24068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.392+0000 7f2e45fce640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2e24066740 0x7f2e24068c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0011f0 tx=0x7f2e3c007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:47.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.396+0000 7f2e367fc640 1 -- 172.21.15.31:0/2339805642 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2e401e81c0 con 0x7f2e401613a0 2023-11-26T07:23:47.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.588+0000 7f2e48232640 1 -- 172.21.15.31:0/2339805642 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e401e8b60 con 0x7f2e401613a0 2023-11-26T07:23:47.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.588+0000 7f2e367fc640 1 -- 172.21.15.31:0/2339805642 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f2e401e8b60 con 0x7f2e401613a0 2023-11-26T07:23:47.594 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:47.594 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3797004646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3797004646},{"type":"v1","addr":"172.21.15.97:6833","nonce":3797004646}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2675576819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2675576819},{"type":"v1","addr":"172.21.15.97:6835","nonce":2675576819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1710103664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1710103664},{"type":"v1","addr":"172.21.15.31:6834","nonce":1710103664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-26T07:23:04.497314+0000","modified":"2023-11-26T07:23:07.528539+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"c","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1001580514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1001580514},{"type":"v1","addr":"172.21.15.31:6835","nonce":1001580514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-11-26T07:23:47.594 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 73 2023-11-26T07:23:47.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 -- 172.21.15.31:0/2339805642 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2e24066740 msgr2=0x7f2e24068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2e24066740 0x7f2e24068c00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c0011f0 tx=0x7f2e3c007d40 comp rx=0 tx=0).stop 2023-11-26T07:23:47.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 -- 172.21.15.31:0/2339805642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 msgr2=0x7f2e401e8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 0x7f2e401e8e40 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2e380b2040 tx=0x7f2e3801bed0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 -- 172.21.15.31:0/2339805642 shutdown_connections 2023-11-26T07:23:47.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2e24066740 0x7f2e24068c00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e401613a0 0x7f2e401e8e40 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e40160d00 0x7f2e40159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 --2- 172.21.15.31:0/2339805642 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2e400d1da0 0x7f2e40158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 -- 172.21.15.31:0/2339805642 >> 172.21.15.31:0/2339805642 conn(0x7f2e4008afc0 msgr2=0x7f2e400cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:47.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 -- 172.21.15.31:0/2339805642 shutdown_connections 2023-11-26T07:23:47.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.592+0000 7f2e48232640 1 -- 172.21.15.31:0/2339805642 wait complete. 2023-11-26T07:23:47.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:47.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f33df640 1 Processor -- start 2023-11-26T07:23:47.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f33df640 1 -- start start 2023-11-26T07:23:47.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f33df640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fb6ec15d530 0x7fb6ec0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f33df640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ec0d12e0 con 0x7fb6ec0d04c0 2023-11-26T07:23:47.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f33df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ec0d15c0 con 0x7fb6ec0cf1b0 2023-11-26T07:23:47.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f33df640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec0d18a0 con 0x7fb6ec15d530 2023-11-26T07:23:47.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f197c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fb6ec0d04c0 0x7fb6ec1674d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38910/0 (socket says 172.21.15.31:38910) 2023-11-26T07:23:47.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f197c640 1 -- 172.21.15.31:0/2854319851 learned_addr learned my addr 172.21.15.31:0/2854319851 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f117b640 1 --2- 172.21.15.31:0/2854319851 >> v2:172.21.15.97:3300/0 conn(0x7fb6ec15d530 0x7fb6ec0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909920563 0 0) 0x7fb6ec0d12e0 con 0x7fb6ec0d04c0 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6d8003530 con 0x7fb6ec0d04c0 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2223106211 0 0) 0x7fb6d8003530 con 0x7fb6ec0d04c0 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 >> v2:172.21.15.97:3300/0 conn(0x7fb6ec15d530 msgr2=0x7fb6ec0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 --2- 172.21.15.31:0/2854319851 >> v2:172.21.15.97:3300/0 conn(0x7fb6ec15d530 0x7fb6ec0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 >> v1:172.21.15.97:6789/0 conn(0x7fb6ec0cf1b0 legacy=0x7fb6ec0cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0d1b80 con 0x7fb6ec0d04c0 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7fb6e4001a00 con 0x7fb6ec0d04c0 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 >> v1:172.21.15.31:6789/0 conn(0x7fb6ec0d04c0 legacy=0x7fb6ec1674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d80039d0 0x7fb6d8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.763 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6d80047d0 0x7fb6d8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6d8005190 0x7fb6d800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0d18a0 con 0x7fb6d80047d0 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d800c090 con 0x7fb6d80039d0 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d800c250 con 0x7fb6d8005190 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f097a640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6d80047d0 0x7fb6d8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f117b640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d80039d0 0x7fb6d8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.756+0000 7fb6f197c640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6d8005190 0x7fb6d800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f097a640 1 -- 172.21.15.31:0/2854319851 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6d8005190 msgr2=0x7fb6d800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f097a640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6d8005190 0x7fb6d800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f097a640 1 -- 172.21.15.31:0/2854319851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d80039d0 msgr2=0x7fb6d8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f097a640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d80039d0 0x7fb6d8003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.764 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f097a640 1 -- 172.21.15.31:0/2854319851 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d8003530 con 0x7fb6d80047d0 2023-11-26T07:23:47.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f117b640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d80039d0 0x7fb6d8003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:47.765 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f197c640 1 -- 172.21.15.31:0/2854319851 reap_dead start 2023-11-26T07:23:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f097a640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6d80047d0 0x7fb6d8004c50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc001750 tx=0x7fb6dc01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3f58a248ad8666 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0052d0 con 0x7fb6d80047d0 2023-11-26T07:23:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc0a7050 con 0x7fb6d80047d0 2023-11-26T07:23:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6e3fff640 1 -- 172.21.15.31:0/2854319851 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc003600 con 0x7fb6d80047d0 2023-11-26T07:23:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f33df640 1 -- 172.21.15.31:0/2854319851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6d80047d0 msgr2=0x7fb6d8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f33df640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6d80047d0 0x7fb6d8004c50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc001750 tx=0x7fb6dc01c3a0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f33df640 1 -- 172.21.15.31:0/2854319851 shutdown_connections 2023-11-26T07:23:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f33df640 1 --2- 172.21.15.31:0/2854319851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6d80047d0 0x7fb6d8004c50 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.760+0000 7fb6f33df640 1 -- 172.21.15.31:0/2854319851 >> 172.21.15.31:0/2854319851 conn(0x7fb6ec09c6d0 msgr2=0x7fb6ec0d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:47.766 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f33df640 1 -- 172.21.15.31:0/2854319851 shutdown_connections 2023-11-26T07:23:47.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f33df640 1 -- 172.21.15.31:0/2854319851 wait complete. 2023-11-26T07:23:47.767 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f33df640 1 Processor -- start 2023-11-26T07:23:47.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f33df640 1 -- start start 2023-11-26T07:23:47.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f33df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d80047d0 0x7fb6ec0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f33df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6ec06d200 0x7fb6ec0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.769 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f33df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec158770 0x7fb6ec0b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f33df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0cd370 con 0x7fb6ec158770 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f33df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0cb660 con 0x7fb6d80047d0 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f33df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0cb920 con 0x7fb6ec06d200 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f197c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec158770 0x7fb6ec0b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f197c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec158770 0x7fb6ec0b5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37238/0 (socket says 172.21.15.31:37238) 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f197c640 1 -- 172.21.15.31:0/2424299232 learned_addr learned my addr 172.21.15.31:0/2424299232 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f117b640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d80047d0 0x7fb6ec0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.764+0000 7fb6f097a640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6ec06d200 0x7fb6ec0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6f197c640 1 -- 172.21.15.31:0/2424299232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6ec06d200 msgr2=0x7fb6ec0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6f197c640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6ec06d200 0x7fb6ec0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6f197c640 1 -- 172.21.15.31:0/2424299232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d80047d0 msgr2=0x7fb6ec0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6f197c640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d80047d0 0x7fb6ec0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6f197c640 1 -- 172.21.15.31:0/2424299232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0cbc00 con 0x7fb6ec158770 2023-11-26T07:23:47.771 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6f197c640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec158770 0x7fb6ec0b5110 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec08c270 tx=0x7fb6e400a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6e1ffb640 1 -- 172.21.15.31:0/2424299232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400aca0 con 0x7fb6ec158770 2023-11-26T07:23:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6e1ffb640 1 -- 172.21.15.31:0/2424299232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e400ae40 con 0x7fb6ec158770 2023-11-26T07:23:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6e1ffb640 1 -- 172.21.15.31:0/2424299232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4013630 con 0x7fb6ec158770 2023-11-26T07:23:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6f33df640 1 -- 172.21.15.31:0/2424299232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ec0cbee0 con 0x7fb6ec158770 2023-11-26T07:23:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6f33df640 1 -- 172.21.15.31:0/2424299232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec158bf0 con 0x7fb6ec158770 2023-11-26T07:23:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6f33df640 1 -- 172.21.15.31:0/2424299232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ec158e10 con 0x7fb6ec158770 2023-11-26T07:23:47.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.768+0000 7fb6e1ffb640 1 -- 172.21.15.31:0/2424299232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fb6e4013790 con 0x7fb6ec158770 2023-11-26T07:23:47.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.772+0000 7fb6e1ffb640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb6d0066770 0x7fb6d0068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:47.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.772+0000 7fb6e1ffb640 1 -- 172.21.15.31:0/2424299232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6e4084840 con 0x7fb6ec158770 2023-11-26T07:23:47.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.772+0000 7fb6f117b640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb6d0066770 0x7fb6d0068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:47.775 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.772+0000 7fb6f117b640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb6d0066770 0x7fb6d0068c30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0643d0 tx=0x7fb6e8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:47.778 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.776+0000 7fb6e1ffb640 1 -- 172.21.15.31:0/2424299232 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb6e4051930 con 0x7fb6ec158770 2023-11-26T07:23:47.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.932+0000 7fb6f33df640 1 -- 172.21.15.31:0/2424299232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6ec04e850 con 0x7fb6ec158770 2023-11-26T07:23:47.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.932+0000 7fb6e1ffb640 1 -- 172.21.15.31:0/2424299232 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16432 (secure 0 0 0) 0x7fb6e4054f50 con 0x7fb6ec158770 2023-11-26T07:23:47.937 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:47.937 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":99,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:23:06.127026+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-26T07:23:02.513363+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-26T07:23:03.492889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:47.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 -- 172.21.15.31:0/2424299232 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb6d0066770 msgr2=0x7fb6d0068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb6d0066770 0x7fb6d0068c30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0643d0 tx=0x7fb6e8009190 comp rx=0 tx=0).stop 2023-11-26T07:23:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 -- 172.21.15.31:0/2424299232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec158770 msgr2=0x7fb6ec0b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec158770 0x7fb6ec0b5110 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec08c270 tx=0x7fb6e400a640 comp rx=0 tx=0).stop 2023-11-26T07:23:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 -- 172.21.15.31:0/2424299232 shutdown_connections 2023-11-26T07:23:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb6d0066770 0x7fb6d0068c30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6ec158770 0x7fb6ec0b5110 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6ec06d200 0x7fb6ec0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 --2- 172.21.15.31:0/2424299232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6d80047d0 0x7fb6ec0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:47.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 -- 172.21.15.31:0/2424299232 >> 172.21.15.31:0/2424299232 conn(0x7fb6ec09c6d0 msgr2=0x7fb6ec0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:47.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 -- 172.21.15.31:0/2424299232 shutdown_connections 2023-11-26T07:23:47.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:47.936+0000 7fb6f33df640 1 -- 172.21.15.31:0/2424299232 wait complete. 2023-11-26T07:23:47.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-26T07:23:48.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc036e74640 1 Processor -- start 2023-11-26T07:23:48.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc036e74640 1 -- start start 2023-11-26T07:23:48.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc036e74640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d1300 0x7fc0300cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc036e74640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc0300d0100 0x7fc0300b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc036e74640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0300d0590 con 0x7fc030160940 2023-11-26T07:23:48.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc036e74640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0300b6270 con 0x7fc0300d0100 2023-11-26T07:23:48.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc036e74640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc030160d20 con 0x7fc0300d1300 2023-11-26T07:23:48.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc035671640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d1300 0x7fc0300cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc036673640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc0300d0100 0x7fc0300b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc035e72640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc030160940 0x7fc0300d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:55174/0 (socket says 172.21.15.31:55174) 2023-11-26T07:23:48.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc035e72640 1 -- 172.21.15.31:0/365393282 learned_addr learned my addr 172.21.15.31:0/365393282 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:48.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc035671640 1 -- 172.21.15.31:0/365393282 >> v2:172.21.15.97:3300/0 conn(0x7fc0300d0100 msgr2=0x7fc0300b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:48.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc035671640 1 --2- 172.21.15.31:0/365393282 >> v2:172.21.15.97:3300/0 conn(0x7fc0300d0100 0x7fc0300b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc034e70640 1 -- 172.21.15.31:0/365393282 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572608303 0 0) 0x7fc0300d0590 con 0x7fc030160940 2023-11-26T07:23:48.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc035671640 1 -- 172.21.15.31:0/365393282 >> v1:172.21.15.97:6789/0 conn(0x7fc030160940 legacy=0x7fc0300d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.100+0000 7fc035671640 1 -- 172.21.15.31:0/365393282 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0300d2460 con 0x7fc0300d1300 2023-11-26T07:23:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc035671640 1 --2- 172.21.15.31:0/365393282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d1300 0x7fc0300cfbc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc0200097c0 tx=0x7fc02001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c598d17b5161d9ba server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc034e70640 1 -- 172.21.15.31:0/365393282 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0200ab600 con 0x7fc0300d1300 2023-11-26T07:23:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc034e70640 1 -- 172.21.15.31:0/365393282 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0200a7050 con 0x7fc0300d1300 2023-11-26T07:23:48.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc034e70640 1 -- 172.21.15.31:0/365393282 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0200ab980 con 0x7fc0300d1300 2023-11-26T07:23:48.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc036e74640 1 -- 172.21.15.31:0/365393282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d1300 msgr2=0x7fc0300cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc036e74640 1 --2- 172.21.15.31:0/365393282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d1300 0x7fc0300cfbc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc0200097c0 tx=0x7fc02001c480 comp rx=0 tx=0).stop 2023-11-26T07:23:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc036e74640 1 -- 172.21.15.31:0/365393282 shutdown_connections 2023-11-26T07:23:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc036e74640 1 --2- 172.21.15.31:0/365393282 >> v2:172.21.15.97:3300/0 conn(0x7fc0300d0100 0x7fc0300b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc036e74640 1 --2- 172.21.15.31:0/365393282 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d1300 0x7fc0300cfbc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc036e74640 1 -- 172.21.15.31:0/365393282 >> 172.21.15.31:0/365393282 conn(0x7fc0300668a0 msgr2=0x7fc0300b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc036e74640 1 -- 172.21.15.31:0/365393282 shutdown_connections 2023-11-26T07:23:48.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.104+0000 7fc036e74640 1 -- 172.21.15.31:0/365393282 wait complete. 2023-11-26T07:23:48.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036e74640 1 Processor -- start 2023-11-26T07:23:48.111 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036e74640 1 -- start start 2023-11-26T07:23:48.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036e74640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d0100 0x7fc030157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036e74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300d1300 0x7fc0300cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036e74640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc030160940 0x7fc0300d36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036e74640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0300d3c00 con 0x7fc030160940 2023-11-26T07:23:48.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036e74640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0300d3e70 con 0x7fc0300d1300 2023-11-26T07:23:48.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036e74640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03015c6c0 con 0x7fc0300d0100 2023-11-26T07:23:48.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036673640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc030160940 0x7fc0300d36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc035671640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300d1300 0x7fc0300cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036673640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc030160940 0x7fc0300d36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37244/0 (socket says 172.21.15.31:37244) 2023-11-26T07:23:48.113 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036673640 1 -- 172.21.15.31:0/3759473884 learned_addr learned my addr 172.21.15.31:0/3759473884 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc035e72640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d0100 0x7fc030157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036673640 1 -- 172.21.15.31:0/3759473884 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d0100 msgr2=0x7fc030157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036673640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d0100 0x7fc030157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036673640 1 -- 172.21.15.31:0/3759473884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300d1300 msgr2=0x7fc0300cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036673640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300d1300 0x7fc0300cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036673640 1 -- 172.21.15.31:0/3759473884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020009420 con 0x7fc030160940 2023-11-26T07:23:48.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036673640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc030160940 0x7fc0300d36c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fc02800efc0 tx=0x7fc02800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc0267fc640 1 -- 172.21.15.31:0/3759473884 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028014480 con 0x7fc030160940 2023-11-26T07:23:48.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc0267fc640 1 -- 172.21.15.31:0/3759473884 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02800f040 con 0x7fc030160940 2023-11-26T07:23:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc0267fc640 1 -- 172.21.15.31:0/3759473884 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028014820 con 0x7fc030160940 2023-11-26T07:23:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036e74640 1 -- 172.21.15.31:0/3759473884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03015c960 con 0x7fc030160940 2023-11-26T07:23:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.108+0000 7fc036e74640 1 -- 172.21.15.31:0/3759473884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03015cf20 con 0x7fc030160940 2023-11-26T07:23:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.112+0000 7fc036e74640 1 -- 172.21.15.31:0/3759473884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03015d110 con 0x7fc030160940 2023-11-26T07:23:48.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.112+0000 7fc0267fc640 1 -- 172.21.15.31:0/3759473884 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fc028014980 con 0x7fc030160940 2023-11-26T07:23:48.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.112+0000 7fc0267fc640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc014066740 0x7fc014068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.112+0000 7fc0267fc640 1 -- 172.21.15.31:0/3759473884 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc028084d50 con 0x7fc030160940 2023-11-26T07:23:48.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.112+0000 7fc035e72640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc014066740 0x7fc014068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.117 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.112+0000 7fc035e72640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc014066740 0x7fc014068c00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc02c000f70 tx=0x7fc02c007df0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.120 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.116+0000 7fc0267fc640 1 -- 172.21.15.31:0/3759473884 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc028051eb0 con 0x7fc030160940 2023-11-26T07:23:48.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.308+0000 7fc036e74640 1 -- 172.21.15.31:0/3759473884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc030074990 con 0x7fc030160940 2023-11-26T07:23:48.311 INFO:tasks.ceph.mon.a.smithi031.stderr:2023-11-26T07:23:48.308+0000 7fe86499d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-26T07:23:48.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.308+0000 7fc0267fc640 1 -- 172.21.15.31:0/3759473884 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc028018090 con 0x7fc030160940 2023-11-26T07:23:48.313 INFO:teuthology.orchestra.run.smithi031.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-26T07:23:48.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 -- 172.21.15.31:0/3759473884 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc014066740 msgr2=0x7fc014068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc014066740 0x7fc014068c00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc02c000f70 tx=0x7fc02c007df0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 -- 172.21.15.31:0/3759473884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc030160940 msgr2=0x7fc0300d36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc030160940 0x7fc0300d36c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fc02800efc0 tx=0x7fc02800c490 comp rx=0 tx=0).stop 2023-11-26T07:23:48.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 -- 172.21.15.31:0/3759473884 shutdown_connections 2023-11-26T07:23:48.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc014066740 0x7fc014068c00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc030160940 0x7fc0300d36c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0300d1300 0x7fc0300cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 --2- 172.21.15.31:0/3759473884 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc0300d0100 0x7fc030157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 -- 172.21.15.31:0/3759473884 >> 172.21.15.31:0/3759473884 conn(0x7fc0300668a0 msgr2=0x7fc0300b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:48.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 -- 172.21.15.31:0/3759473884 shutdown_connections 2023-11-26T07:23:48.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.312+0000 7fc036e74640 1 -- 172.21.15.31:0/3759473884 wait complete. 2023-11-26T07:23:48.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-26T07:23:48.382 INFO:tasks.ceph.mds.c.smithi031.stdout:starting mds.c at 2023-11-26T07:23:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.484+0000 7fc3b9c9a640 1 Processor -- start 2023-11-26T07:23:48.488 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.484+0000 7fc3b9c9a640 1 -- start start 2023-11-26T07:23:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.484+0000 7fc3b9c9a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3b40ced30 0x7fc3b40cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.484+0000 7fc3b9c9a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc3b40cf1e0 0x7fc3b40d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.484+0000 7fc3b9c9a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b40cdfb0 con 0x7fc3b415d8f0 2023-11-26T07:23:48.490 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.484+0000 7fc3b9c9a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b40d4420 con 0x7fc3b40ced30 2023-11-26T07:23:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b9c9a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b40d4140 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b2ffd640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc3b40ced30 0x7fc3b40cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b37fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc3b415d8f0 0x7fc3b415e780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:38922/0 (socket says 172.21.15.31:38922) 2023-11-26T07:23:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b37fe640 1 -- 172.21.15.31:0/1741919926 learned_addr learned my addr 172.21.15.31:0/1741919926 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b3fff640 1 --2- 172.21.15.31:0/1741919926 >> v2:172.21.15.97:3300/0 conn(0x7fc3b40cf1e0 0x7fc3b40d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b2ffd640 1 -- 172.21.15.31:0/1741919926 >> v2:172.21.15.97:3300/0 conn(0x7fc3b40cf1e0 msgr2=0x7fc3b40d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b2ffd640 1 --2- 172.21.15.31:0/1741919926 >> v2:172.21.15.97:3300/0 conn(0x7fc3b40cf1e0 0x7fc3b40d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b2ffd640 1 -- 172.21.15.31:0/1741919926 >> v1:172.21.15.31:6789/0 conn(0x7fc3b415d8f0 legacy=0x7fc3b415e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b2ffd640 1 -- 172.21.15.31:0/1741919926 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b40ce250 con 0x7fc3b40ced30 2023-11-26T07:23:48.491 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b27fc640 1 -- 172.21.15.31:0/1741919926 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309776330 0 0) 0x7fc3b40cdfb0 con 0x7fc3b415d8f0 2023-11-26T07:23:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b2ffd640 1 --2- 172.21.15.31:0/1741919926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40ced30 0x7fc3b40cfe00 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40b21f0 tx=0x7fc3a801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9dd5ca96071b0c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b27fc640 1 -- 172.21.15.31:0/1741919926 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80a7070 con 0x7fc3b40ced30 2023-11-26T07:23:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b27fc640 1 -- 172.21.15.31:0/1741919926 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a80ab430 con 0x7fc3b40ced30 2023-11-26T07:23:48.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.488+0000 7fc3b27fc640 1 -- 172.21.15.31:0/1741919926 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80ab7c0 con 0x7fc3b40ced30 2023-11-26T07:23:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/1741919926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40ced30 msgr2=0x7fc3b40cfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.494 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 --2- 172.21.15.31:0/1741919926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40ced30 0x7fc3b40cfe00 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40b21f0 tx=0x7fc3a801c430 comp rx=0 tx=0).stop 2023-11-26T07:23:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/1741919926 shutdown_connections 2023-11-26T07:23:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 --2- 172.21.15.31:0/1741919926 >> v2:172.21.15.97:3300/0 conn(0x7fc3b40cf1e0 0x7fc3b40d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 --2- 172.21.15.31:0/1741919926 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40ced30 0x7fc3b40cfe00 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/1741919926 >> 172.21.15.31:0/1741919926 conn(0x7fc3b409c750 msgr2=0x7fc3b40d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/1741919926 shutdown_connections 2023-11-26T07:23:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/1741919926 wait complete. 2023-11-26T07:23:48.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 Processor -- start 2023-11-26T07:23:48.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 -- start start 2023-11-26T07:23:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3b40ced30 0x7fc3b41547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40cf1e0 0x7fc3b41539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415d8f0 0x7fc3b4153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b415c320 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40b7750 con 0x7fc3b415d8f0 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b9c9a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40b7a30 con 0x7fc3b40ced30 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40cf1e0 0x7fc3b41539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40cf1e0 0x7fc3b41539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37284/0 (socket says 172.21.15.31:37284) 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b2ffd640 1 -- 172.21.15.31:0/884694103 learned_addr learned my addr 172.21.15.31:0/884694103 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b37fe640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3b40ced30 0x7fc3b41547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.492+0000 7fc3b3fff640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415d8f0 0x7fc3b4153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc3b2ffd640 1 -- 172.21.15.31:0/884694103 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3b40ced30 msgr2=0x7fc3b41547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc3b2ffd640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3b40ced30 0x7fc3b41547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc3b2ffd640 1 -- 172.21.15.31:0/884694103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415d8f0 msgr2=0x7fc3b4153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc3b2ffd640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415d8f0 0x7fc3b4153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:48.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc3b2ffd640 1 -- 172.21.15.31:0/884694103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a8009420 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc3b2ffd640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40cf1e0 0x7fc3b41539f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fc3a801aae0 tx=0x7fc3a801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc39bfff640 1 -- 172.21.15.31:0/884694103 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8004220 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc39bfff640 1 -- 172.21.15.31:0/884694103 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a80b9070 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/884694103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b40b7d10 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc39bfff640 1 -- 172.21.15.31:0/884694103 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8006e20 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/884694103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b40b7400 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/884694103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b404e850 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc39bfff640 1 -- 172.21.15.31:0/884694103 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fc3a8003620 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc39bfff640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3a0066740 0x7fc3a0068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:48.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.496+0000 7fc39bfff640 1 -- 172.21.15.31:0/884694103 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc3a80bb070 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.500+0000 7fc3b37fe640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3a0066740 0x7fc3a0068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:48.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.500+0000 7fc3b37fe640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3a0066740 0x7fc3a0068c00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4007fd0 tx=0x7fc3a4007d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:48.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.500+0000 7fc39bfff640 1 -- 172.21.15.31:0/884694103 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc3a80ebe90 con 0x7fc3b40cf1e0 2023-11-26T07:23:48.695 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:48.692+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/884694103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc3b40cf9d0 con 0x7fc3b40cf1e0 2023-11-26T07:23:49.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.312+0000 7fc39bfff640 1 -- 172.21.15.31:0/884694103 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc3a80ef4b0 con 0x7fc3b40cf1e0 2023-11-26T07:23:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.312+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/884694103 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3a0066740 msgr2=0x7fc3a0068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3a0066740 0x7fc3a0068c00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4007fd0 tx=0x7fc3a4007d60 comp rx=0 tx=0).stop 2023-11-26T07:23:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/884694103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40cf1e0 msgr2=0x7fc3b41539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40cf1e0 0x7fc3b41539f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fc3a801aae0 tx=0x7fc3a801ac70 comp rx=0 tx=0).stop 2023-11-26T07:23:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/884694103 shutdown_connections 2023-11-26T07:23:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc3a0066740 0x7fc3a0068c00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3b415d8f0 0x7fc3b4153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3b40cf1e0 0x7fc3b41539f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 --2- 172.21.15.31:0/884694103 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc3b40ced30 0x7fc3b41547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/884694103 >> 172.21.15.31:0/884694103 conn(0x7fc3b409c750 msgr2=0x7fc3b40734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/884694103 shutdown_connections 2023-11-26T07:23:49.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.316+0000 7fc3b9c9a640 1 -- 172.21.15.31:0/884694103 wait complete. 2023-11-26T07:23:49.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-26T07:23:49.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d7c32a640 1 Processor -- start 2023-11-26T07:23:49.495 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d7c32a640 1 -- start start 2023-11-26T07:23:49.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d7c32a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d7406e3f0 0x7f2d7415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d7c32a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d740ce5d0 0x7f2d7415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d7c32a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d74159630 con 0x7f2d740cb9b0 2023-11-26T07:23:49.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d7c32a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7415bba0 con 0x7f2d740ce5d0 2023-11-26T07:23:49.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d7c32a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740cdd50 con 0x7f2d7406e3f0 2023-11-26T07:23:49.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d7a8c7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f2d740ce5d0 0x7f2d7415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d798c5640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d7406e3f0 0x7f2d7415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d7a0c6640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7f2d740cb9b0 0x7f2d740cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:55206/0 (socket says 172.21.15.31:55206) 2023-11-26T07:23:49.498 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.492+0000 7f2d7a0c6640 1 -- 172.21.15.31:0/2341360110 learned_addr learned my addr 172.21.15.31:0/2341360110 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d7a8c7640 1 -- 172.21.15.31:0/2341360110 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d7406e3f0 msgr2=0x7f2d7415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d7a8c7640 1 --2- 172.21.15.31:0/2341360110 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d7406e3f0 0x7f2d7415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d7a8c7640 1 -- 172.21.15.31:0/2341360110 >> v1:172.21.15.97:6789/0 conn(0x7f2d740cb9b0 legacy=0x7f2d740cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d7a8c7640 1 -- 172.21.15.31:0/2341360110 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7415f6d0 con 0x7f2d740ce5d0 2023-11-26T07:23:49.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d798c5640 1 --2- 172.21.15.31:0/2341360110 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d7406e3f0 0x7f2d7415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:49.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d7a8c7640 1 --2- 172.21.15.31:0/2341360110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d740ce5d0 0x7f2d7415bfd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009d90 tx=0x7f2d6c01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb449a033ca71ccb server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:49.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d790c4640 1 -- 172.21.15.31:0/2341360110 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0a58e0 con 0x7f2d740ce5d0 2023-11-26T07:23:49.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d790c4640 1 -- 172.21.15.31:0/2341360110 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c0a5a40 con 0x7f2d740ce5d0 2023-11-26T07:23:49.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d790c4640 1 -- 172.21.15.31:0/2341360110 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0a5dd0 con 0x7f2d740ce5d0 2023-11-26T07:23:49.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d7c32a640 1 -- 172.21.15.31:0/2341360110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d740ce5d0 msgr2=0x7f2d7415bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.496+0000 7f2d7c32a640 1 --2- 172.21.15.31:0/2341360110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d740ce5d0 0x7f2d7415bfd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009d90 tx=0x7f2d6c01c950 comp rx=0 tx=0).stop 2023-11-26T07:23:49.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 -- 172.21.15.31:0/2341360110 shutdown_connections 2023-11-26T07:23:49.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 --2- 172.21.15.31:0/2341360110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d740ce5d0 0x7f2d7415bfd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 --2- 172.21.15.31:0/2341360110 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d7406e3f0 0x7f2d7415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 -- 172.21.15.31:0/2341360110 >> 172.21.15.31:0/2341360110 conn(0x7f2d740b0cb0 msgr2=0x7f2d740b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:49.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 -- 172.21.15.31:0/2341360110 shutdown_connections 2023-11-26T07:23:49.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 -- 172.21.15.31:0/2341360110 wait complete. 2023-11-26T07:23:49.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 Processor -- start 2023-11-26T07:23:49.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 -- start start 2023-11-26T07:23:49.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d7406e3f0 0x7f2d74147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d740cb9b0 0x7f2d74099a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d740ce5d0 0x7f2d7409bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7409aab0 con 0x7f2d740cb9b0 2023-11-26T07:23:49.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740abf90 con 0x7f2d7406e3f0 2023-11-26T07:23:49.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7c32a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d740ac830 con 0x7f2d740ce5d0 2023-11-26T07:23:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d798c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d740cb9b0 0x7f2d74099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d798c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d740cb9b0 0x7f2d74099a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37292/0 (socket says 172.21.15.31:37292) 2023-11-26T07:23:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d798c5640 1 -- 172.21.15.31:0/4202772842 learned_addr learned my addr 172.21.15.31:0/4202772842 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7a8c7640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d740ce5d0 0x7f2d7409bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d7a0c6640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d7406e3f0 0x7f2d74147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d798c5640 1 -- 172.21.15.31:0/4202772842 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d740ce5d0 msgr2=0x7f2d7409bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d798c5640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d740ce5d0 0x7f2d7409bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d798c5640 1 -- 172.21.15.31:0/4202772842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d7406e3f0 msgr2=0x7f2d74147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d798c5640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d7406e3f0 0x7f2d74147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d798c5640 1 -- 172.21.15.31:0/4202772842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64009420 con 0x7f2d740cb9b0 2023-11-26T07:23:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d798c5640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d740cb9b0 0x7f2d74099a30 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400ec10 tx=0x7f2d6400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:49.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.500+0000 7f2d6affd640 1 -- 172.21.15.31:0/4202772842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400cc30 con 0x7f2d740cb9b0 2023-11-26T07:23:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.504+0000 7f2d6affd640 1 -- 172.21.15.31:0/4202772842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400cd90 con 0x7f2d740cb9b0 2023-11-26T07:23:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.504+0000 7f2d7c32a640 1 -- 172.21.15.31:0/4202772842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c009a70 con 0x7f2d740cb9b0 2023-11-26T07:23:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.504+0000 7f2d6affd640 1 -- 172.21.15.31:0/4202772842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64013690 con 0x7f2d740cb9b0 2023-11-26T07:23:49.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.504+0000 7f2d7c32a640 1 -- 172.21.15.31:0/4202772842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7409a7d0 con 0x7f2d740cb9b0 2023-11-26T07:23:49.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.504+0000 7f2d7c32a640 1 -- 172.21.15.31:0/4202772842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d740671f0 con 0x7f2d740cb9b0 2023-11-26T07:23:49.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.504+0000 7f2d6affd640 1 -- 172.21.15.31:0/4202772842 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f2d640137f0 con 0x7f2d740cb9b0 2023-11-26T07:23:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.504+0000 7f2d6affd640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2d58066740 0x7f2d58068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.504+0000 7f2d6affd640 1 -- 172.21.15.31:0/4202772842 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2d640848f0 con 0x7f2d740cb9b0 2023-11-26T07:23:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.504+0000 7f2d7a0c6640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2d58066740 0x7f2d58068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:49.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.508+0000 7f2d7a0c6640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2d58066740 0x7f2d58068c00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000f70 tx=0x7f2d70007e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:49.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.508+0000 7f2d6affd640 1 -- 172.21.15.31:0/4202772842 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d64051ac0 con 0x7f2d740cb9b0 2023-11-26T07:23:49.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:49.668+0000 7f2d7c32a640 1 -- 172.21.15.31:0/4202772842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d7406b460 con 0x7f2d740cb9b0 2023-11-26T07:23:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.320+0000 7f2d6affd640 1 -- 172.21.15.31:0/4202772842 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2d640550e0 con 0x7f2d740cb9b0 2023-11-26T07:23:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_metadata' removed 2023-11-26T07:23:50.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 -- 172.21.15.31:0/4202772842 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2d58066740 msgr2=0x7f2d58068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2d58066740 0x7f2d58068c00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2d70000f70 tx=0x7f2d70007e90 comp rx=0 tx=0).stop 2023-11-26T07:23:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 -- 172.21.15.31:0/4202772842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d740cb9b0 msgr2=0x7f2d74099a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d740cb9b0 0x7f2d74099a30 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400ec10 tx=0x7f2d6400c520 comp rx=0 tx=0).stop 2023-11-26T07:23:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 -- 172.21.15.31:0/4202772842 shutdown_connections 2023-11-26T07:23:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f2d58066740 0x7f2d58068c00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f2d740ce5d0 0x7f2d7409bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d740cb9b0 0x7f2d74099a30 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 --2- 172.21.15.31:0/4202772842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d7406e3f0 0x7f2d74147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 -- 172.21.15.31:0/4202772842 >> 172.21.15.31:0/4202772842 conn(0x7f2d740b0cb0 msgr2=0x7f2d7406cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 -- 172.21.15.31:0/4202772842 shutdown_connections 2023-11-26T07:23:50.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.324+0000 7f2d7c32a640 1 -- 172.21.15.31:0/4202772842 wait complete. 2023-11-26T07:23:50.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-26T07:23:50.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.492+0000 7fd01d0e3640 1 Processor -- start 2023-11-26T07:23:50.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.492+0000 7fd01d0e3640 1 -- start start 2023-11-26T07:23:50.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd01d0e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018159f00 0x7fd0180ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd01d0e3640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0180ae680 0x7fd0180cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.499 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd01d0e3640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fd0180cd920 0x7fd01815d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd01d0e3640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd018072950 con 0x7fd0180ae680 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd01d0e3640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01806e2b0 con 0x7fd0180cd920 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd01d0e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01806c6f0 con 0x7fd018159f00 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd00ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0180ae680 0x7fd0180cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd00ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd0180ae680 0x7fd0180cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37306/0 (socket says 172.21.15.31:37306) 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd00ffff640 1 -- 172.21.15.31:0/479701861 learned_addr learned my addr 172.21.15.31:0/479701861 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd01759e640 1 --2- 172.21.15.31:0/479701861 >> v2:172.21.15.97:3300/0 conn(0x7fd0180cd920 0x7fd01815d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd016d9d640 1 --2- 172.21.15.31:0/479701861 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018159f00 0x7fd0180ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd00ffff640 1 -- 172.21.15.31:0/479701861 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018159f00 msgr2=0x7fd0180ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd00ffff640 1 --2- 172.21.15.31:0/479701861 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018159f00 0x7fd0180ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.500 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd00ffff640 1 -- 172.21.15.31:0/479701861 >> v2:172.21.15.97:3300/0 conn(0x7fd0180cd920 msgr2=0x7fd01815d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:50.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd00ffff640 1 --2- 172.21.15.31:0/479701861 >> v2:172.21.15.97:3300/0 conn(0x7fd0180cd920 0x7fd01815d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd00ffff640 1 -- 172.21.15.31:0/479701861 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01806e550 con 0x7fd0180ae680 2023-11-26T07:23:50.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd01759e640 1 --2- 172.21.15.31:0/479701861 >> v2:172.21.15.97:3300/0 conn(0x7fd0180cd920 0x7fd01815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:50.501 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.496+0000 7fd016d9d640 1 --2- 172.21.15.31:0/479701861 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018159f00 0x7fd0180ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:50.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd00ffff640 1 --2- 172.21.15.31:0/479701861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180ae680 0x7fd0180cd3e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd01815d7c0 tx=0x7fd00001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335901dd63af9a7f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01659c640 1 -- 172.21.15.31:0/479701861 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0000ab600 con 0x7fd0180ae680 2023-11-26T07:23:50.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01659c640 1 -- 172.21.15.31:0/479701861 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0000a7050 con 0x7fd0180ae680 2023-11-26T07:23:50.502 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01659c640 1 -- 172.21.15.31:0/479701861 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0000ab980 con 0x7fd0180ae680 2023-11-26T07:23:50.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 -- 172.21.15.31:0/479701861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180ae680 msgr2=0x7fd0180cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 --2- 172.21.15.31:0/479701861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180ae680 0x7fd0180cd3e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd01815d7c0 tx=0x7fd00001c500 comp rx=0 tx=0).stop 2023-11-26T07:23:50.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 -- 172.21.15.31:0/479701861 shutdown_connections 2023-11-26T07:23:50.503 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 --2- 172.21.15.31:0/479701861 >> v2:172.21.15.97:3300/0 conn(0x7fd0180cd920 0x7fd01815d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 --2- 172.21.15.31:0/479701861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180ae680 0x7fd0180cd3e0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 --2- 172.21.15.31:0/479701861 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018159f00 0x7fd0180ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 -- 172.21.15.31:0/479701861 >> 172.21.15.31:0/479701861 conn(0x7fd0180b0cb0 msgr2=0x7fd0180b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:50.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 -- 172.21.15.31:0/479701861 shutdown_connections 2023-11-26T07:23:50.504 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 -- 172.21.15.31:0/479701861 wait complete. 2023-11-26T07:23:50.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 Processor -- start 2023-11-26T07:23:50.505 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.500+0000 7fd01d0e3640 1 -- start start 2023-11-26T07:23:50.506 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd01d0e3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0180ae680 0x7fd018146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd01d0e3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180cd920 0x7fd018163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd01d0e3640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018156790 0x7fd018156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd01d0e3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0181fa590 con 0x7fd0180cd920 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd01d0e3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd018157180 con 0x7fd0180ae680 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd01d0e3640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0181f9a10 con 0x7fd018156790 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180cd920 0x7fd018163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180cd920 0x7fd018163ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37318/0 (socket says 172.21.15.31:37318) 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00ffff640 1 -- 172.21.15.31:0/940628217 learned_addr learned my addr 172.21.15.31:0/940628217 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd016d9d640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0180ae680 0x7fd018146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd01759e640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018156790 0x7fd018156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.507 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00ffff640 1 -- 172.21.15.31:0/940628217 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018156790 msgr2=0x7fd018156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00ffff640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018156790 0x7fd018156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00ffff640 1 -- 172.21.15.31:0/940628217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0180ae680 msgr2=0x7fd018146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:50.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00ffff640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0180ae680 0x7fd018146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:50.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00ffff640 1 -- 172.21.15.31:0/940628217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0000094d0 con 0x7fd0180cd920 2023-11-26T07:23:50.508 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00ffff640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180cd920 0x7fd018163ab0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd00001abb0 tx=0x7fd0000192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00f7fe640 1 -- 172.21.15.31:0/940628217 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0000b4640 con 0x7fd0180cd920 2023-11-26T07:23:50.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00f7fe640 1 -- 172.21.15.31:0/940628217 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0000b47a0 con 0x7fd0180cd920 2023-11-26T07:23:50.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00f7fe640 1 -- 172.21.15.31:0/940628217 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0000b4b50 con 0x7fd0180cd920 2023-11-26T07:23:50.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd01d0e3640 1 -- 172.21.15.31:0/940628217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0181f9cf0 con 0x7fd0180cd920 2023-11-26T07:23:50.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd01d0e3640 1 -- 172.21.15.31:0/940628217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0181fa2b0 con 0x7fd0180cd920 2023-11-26T07:23:50.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd01d0e3640 1 -- 172.21.15.31:0/940628217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0180671f0 con 0x7fd0180cd920 2023-11-26T07:23:50.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.504+0000 7fd00f7fe640 1 -- 172.21.15.31:0/940628217 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fd0000b2070 con 0x7fd0180cd920 2023-11-26T07:23:50.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.508+0000 7fd00f7fe640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcff8066740 0x7fcff8068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:50.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.508+0000 7fd00f7fe640 1 -- 172.21.15.31:0/940628217 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fd00011e350 con 0x7fd0180cd920 2023-11-26T07:23:50.510 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.508+0000 7fd016d9d640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcff8066740 0x7fcff8068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:50.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.508+0000 7fd016d9d640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcff8066740 0x7fcff8068c00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd008006fd0 tx=0x7fd008008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:50.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.512+0000 7fd00f7fe640 1 -- 172.21.15.31:0/940628217 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd0000eb490 con 0x7fd0180cd920 2023-11-26T07:23:50.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:50.668+0000 7fd01d0e3640 1 -- 172.21.15.31:0/940628217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd01806b460 con 0x7fd0180cd920 2023-11-26T07:23:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd00f7fe640 1 -- 172.21.15.31:0/940628217 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd0000eeab0 con 0x7fd0180cd920 2023-11-26T07:23:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'cephfs_data' removed 2023-11-26T07:23:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd01d0e3640 1 -- 172.21.15.31:0/940628217 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcff8066740 msgr2=0x7fcff8068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd01d0e3640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcff8066740 0x7fcff8068c00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd008006fd0 tx=0x7fd008008040 comp rx=0 tx=0).stop 2023-11-26T07:23:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd01d0e3640 1 -- 172.21.15.31:0/940628217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180cd920 msgr2=0x7fd018163ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd01d0e3640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180cd920 0x7fd018163ab0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd00001abb0 tx=0x7fd0000192a0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd01d0e3640 1 -- 172.21.15.31:0/940628217 shutdown_connections 2023-11-26T07:23:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd01d0e3640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcff8066740 0x7fcff8068c00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd01d0e3640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fd018156790 0x7fd018156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd01d0e3640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0180cd920 0x7fd018163ab0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd01d0e3640 1 --2- 172.21.15.31:0/940628217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0180ae680 0x7fd018146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.324+0000 7fd01d0e3640 1 -- 172.21.15.31:0/940628217 >> 172.21.15.31:0/940628217 conn(0x7fd0180b0cb0 msgr2=0x7fd0180af570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:51.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.328+0000 7fd01d0e3640 1 -- 172.21.15.31:0/940628217 shutdown_connections 2023-11-26T07:23:51.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.328+0000 7fd01d0e3640 1 -- 172.21.15.31:0/940628217 wait complete. 2023-11-26T07:23:51.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-26T07:23:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.504+0000 7f188bd65640 1 Processor -- start 2023-11-26T07:23:51.509 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.504+0000 7f188bd65640 1 -- start start 2023-11-26T07:23:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f188bd65640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 0x7f18840d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f188bd65640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1884161f20 0x7f18840cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f188bd65640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f18840d02f0 0x7f188415dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f188bd65640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18840d0f90 con 0x7f1884161f20 2023-11-26T07:23:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f188bd65640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18840d0740 con 0x7f18840d02f0 2023-11-26T07:23:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f188bd65640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18840d11a0 con 0x7f188415d7c0 2023-11-26T07:23:51.511 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f1889300640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1884161f20 0x7f18840cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f1889300640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1884161f20 0x7f18840cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37322/0 (socket says 172.21.15.31:37322) 2023-11-26T07:23:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f1889300640 1 -- 172.21.15.31:0/832252346 learned_addr learned my addr 172.21.15.31:0/832252346 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f1889b01640 1 --2- 172.21.15.31:0/832252346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 0x7f18840d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f188a302640 1 --2- 172.21.15.31:0/832252346 >> v2:172.21.15.97:3300/0 conn(0x7f18840d02f0 0x7f188415dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f1889300640 1 -- 172.21.15.31:0/832252346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 msgr2=0x7f18840d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f1889300640 1 --2- 172.21.15.31:0/832252346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 0x7f18840d3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f1889300640 1 -- 172.21.15.31:0/832252346 >> v2:172.21.15.97:3300/0 conn(0x7f18840d02f0 msgr2=0x7f188415dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f1889300640 1 --2- 172.21.15.31:0/832252346 >> v2:172.21.15.97:3300/0 conn(0x7f18840d02f0 0x7f188415dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.508+0000 7f1889300640 1 -- 172.21.15.31:0/832252346 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18840d1480 con 0x7f1884161f20 2023-11-26T07:23:51.514 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f1889300640 1 --2- 172.21.15.31:0/832252346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1884161f20 0x7f18840cfdb0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f188415ed40 tx=0x7f187401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28967294b85875b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f1888aff640 1 -- 172.21.15.31:0/832252346 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18740aa5f0 con 0x7f1884161f20 2023-11-26T07:23:51.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f1888aff640 1 -- 172.21.15.31:0/832252346 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18740aa750 con 0x7f1884161f20 2023-11-26T07:23:51.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f1888aff640 1 -- 172.21.15.31:0/832252346 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18740aaae0 con 0x7f1884161f20 2023-11-26T07:23:51.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 -- 172.21.15.31:0/832252346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1884161f20 msgr2=0x7f18840cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 --2- 172.21.15.31:0/832252346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1884161f20 0x7f18840cfdb0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f188415ed40 tx=0x7f187401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 -- 172.21.15.31:0/832252346 shutdown_connections 2023-11-26T07:23:51.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 --2- 172.21.15.31:0/832252346 >> v2:172.21.15.97:3300/0 conn(0x7f18840d02f0 0x7f188415dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 --2- 172.21.15.31:0/832252346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1884161f20 0x7f18840cfdb0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 --2- 172.21.15.31:0/832252346 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 0x7f18840d3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 -- 172.21.15.31:0/832252346 >> 172.21.15.31:0/832252346 conn(0x7f188408afc0 msgr2=0x7f18840d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:51.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 -- 172.21.15.31:0/832252346 shutdown_connections 2023-11-26T07:23:51.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 -- 172.21.15.31:0/832252346 wait complete. 2023-11-26T07:23:51.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 Processor -- start 2023-11-26T07:23:51.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.512+0000 7f188bd65640 1 -- start start 2023-11-26T07:23:51.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188bd65640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18840d02f0 0x7f1884157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188bd65640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 0x7f18841fa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188bd65640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18841518c0 0x7f1884156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188bd65640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18841fa6c0 con 0x7f18840d02f0 2023-11-26T07:23:51.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188bd65640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188415f420 con 0x7f18841518c0 2023-11-26T07:23:51.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188bd65640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188415f6e0 con 0x7f188415d7c0 2023-11-26T07:23:51.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889b01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18840d02f0 0x7f1884157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889b01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18840d02f0 0x7f1884157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37326/0 (socket says 172.21.15.31:37326) 2023-11-26T07:23:51.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889b01640 1 -- 172.21.15.31:0/1578923899 learned_addr learned my addr 172.21.15.31:0/1578923899 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889300640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 0x7f18841fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188a302640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18841518c0 0x7f1884156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889b01640 1 -- 172.21.15.31:0/1578923899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 msgr2=0x7f18841fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889b01640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 0x7f18841fa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889b01640 1 -- 172.21.15.31:0/1578923899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18841518c0 msgr2=0x7f1884156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889b01640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18841518c0 0x7f1884156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889b01640 1 -- 172.21.15.31:0/1578923899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1874009480 con 0x7f18840d02f0 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188a302640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18841518c0 0x7f1884156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889300640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 0x7f18841fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f1889b01640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18840d02f0 0x7f1884157a40 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f188000e930 tx=0x7f188000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f187a7fc640 1 -- 172.21.15.31:0/1578923899 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1880009690 con 0x7f18840d02f0 2023-11-26T07:23:51.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f187a7fc640 1 -- 172.21.15.31:0/1578923899 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188000ca00 con 0x7f18840d02f0 2023-11-26T07:23:51.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f187a7fc640 1 -- 172.21.15.31:0/1578923899 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188000cd70 con 0x7f18840d02f0 2023-11-26T07:23:51.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188bd65640 1 -- 172.21.15.31:0/1578923899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188415f9c0 con 0x7f18840d02f0 2023-11-26T07:23:51.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188bd65640 1 -- 172.21.15.31:0/1578923899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1884151d70 con 0x7f18840d02f0 2023-11-26T07:23:51.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f188bd65640 1 -- 172.21.15.31:0/1578923899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188404e850 con 0x7f18840d02f0 2023-11-26T07:23:51.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.516+0000 7f187a7fc640 1 -- 172.21.15.31:0/1578923899 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f1880013470 con 0x7f18840d02f0 2023-11-26T07:23:51.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.520+0000 7f187a7fc640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1868066770 0x7f1868068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.520+0000 7f187a7fc640 1 -- 172.21.15.31:0/1578923899 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f18800845e0 con 0x7f18840d02f0 2023-11-26T07:23:51.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.520+0000 7f1889300640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1868066770 0x7f1868068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.520+0000 7f1889300640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1868066770 0x7f1868068c30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18740ae000 tx=0x7f18740a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.524+0000 7f187a7fc640 1 -- 172.21.15.31:0/1578923899 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1880051a90 con 0x7f18840d02f0 2023-11-26T07:23:51.703 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.700+0000 7f188bd65640 1 -- 172.21.15.31:0/1578923899 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f188415cd40 con 0x7f18840d02f0 2023-11-26T07:23:51.707 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f187a7fc640 1 -- 172.21.15.31:0/1578923899 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f18800550b0 con 0x7f18840d02f0 2023-11-26T07:23:51.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 -- 172.21.15.31:0/1578923899 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1868066770 msgr2=0x7f1868068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.709 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1868066770 0x7f1868068c30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18740ae000 tx=0x7f18740a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 -- 172.21.15.31:0/1578923899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18840d02f0 msgr2=0x7f1884157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18840d02f0 0x7f1884157a40 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f188000e930 tx=0x7f188000edf0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 -- 172.21.15.31:0/1578923899 shutdown_connections 2023-11-26T07:23:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f1868066770 0x7f1868068c30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18841518c0 0x7f1884156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f188415d7c0 0x7f18841fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 --2- 172.21.15.31:0/1578923899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18840d02f0 0x7f1884157a40 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 -- 172.21.15.31:0/1578923899 >> 172.21.15.31:0/1578923899 conn(0x7f188408afc0 msgr2=0x7f18840b3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 -- 172.21.15.31:0/1578923899 shutdown_connections 2023-11-26T07:23:51.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.704+0000 7f188bd65640 1 -- 172.21.15.31:0/1578923899 wait complete. 2023-11-26T07:23:51.729 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2023-11-26T07:23:51.872 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.868+0000 7ff6c6c2f640 1 Processor -- start 2023-11-26T07:23:51.873 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.868+0000 7ff6c6c2f640 1 -- start start 2023-11-26T07:23:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c6c2f640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff6c00d3760 0x7ff6c00d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c6c2f640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6c00cfa20 0x7ff6c00cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c6c2f640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c00d15f0 con 0x7ff6c015d880 2023-11-26T07:23:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c6c2f640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00d4110 con 0x7ff6c00cfa20 2023-11-26T07:23:51.874 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c6c2f640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c00d0390 con 0x7ff6c00d3760 2023-11-26T07:23:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c642e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6c00cfa20 0x7ff6c00cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c542c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff6c00d3760 0x7ff6c00d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c642e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff6c00cfa20 0x7ff6c00cfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37328/0 (socket says 172.21.15.31:37328) 2023-11-26T07:23:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c642e640 1 -- 172.21.15.31:0/13004432 learned_addr learned my addr 172.21.15.31:0/13004432 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c542c640 1 -- 172.21.15.31:0/13004432 >> v2:172.21.15.31:3300/0 conn(0x7ff6c00cfa20 msgr2=0x7ff6c00cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:51.875 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c542c640 1 --2- 172.21.15.31:0/13004432 >> v2:172.21.15.31:3300/0 conn(0x7ff6c00cfa20 0x7ff6c00cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c542c640 1 -- 172.21.15.31:0/13004432 >> v1:172.21.15.97:6789/0 conn(0x7ff6c015d880 legacy=0x7ff6c00b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c4c2b640 1 -- 172.21.15.31:0/13004432 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877534557 0 0) 0x7ff6c00d15f0 con 0x7ff6c015d880 2023-11-26T07:23:51.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c542c640 1 -- 172.21.15.31:0/13004432 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c00ce000 con 0x7ff6c00d3760 2023-11-26T07:23:51.876 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.872+0000 7ff6c642e640 1 --2- 172.21.15.31:0/13004432 >> v2:172.21.15.31:3300/0 conn(0x7ff6c00cfa20 0x7ff6c00cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:51.877 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c542c640 1 --2- 172.21.15.31:0/13004432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00d3760 0x7ff6c00d3bd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00cd850 tx=0x7ff6b001c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893ff87268a2c0ed server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c4c2b640 1 -- 172.21.15.31:0/13004432 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00ab660 con 0x7ff6c00d3760 2023-11-26T07:23:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c4c2b640 1 -- 172.21.15.31:0/13004432 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b00ab7c0 con 0x7ff6c00d3760 2023-11-26T07:23:51.878 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c4c2b640 1 -- 172.21.15.31:0/13004432 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b00abb50 con 0x7ff6c00d3760 2023-11-26T07:23:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/13004432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00d3760 msgr2=0x7ff6c00d3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 --2- 172.21.15.31:0/13004432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00d3760 0x7ff6c00d3bd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00cd850 tx=0x7ff6b001c460 comp rx=0 tx=0).stop 2023-11-26T07:23:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/13004432 shutdown_connections 2023-11-26T07:23:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 --2- 172.21.15.31:0/13004432 >> v2:172.21.15.31:3300/0 conn(0x7ff6c00cfa20 0x7ff6c00cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 --2- 172.21.15.31:0/13004432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c00d3760 0x7ff6c00d3bd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/13004432 >> 172.21.15.31:0/13004432 conn(0x7ff6c006b9f0 msgr2=0x7ff6c006be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:51.879 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/13004432 shutdown_connections 2023-11-26T07:23:51.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/13004432 wait complete. 2023-11-26T07:23:51.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 Processor -- start 2023-11-26T07:23:51.880 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 -- start start 2023-11-26T07:23:51.881 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c00cfa20 0x7ff6c00b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6c00d3760 0x7ff6c0153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c015d880 0x7ff6c01565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00b8250 con 0x7ff6c00cfa20 2023-11-26T07:23:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00b7c20 con 0x7ff6c015d880 2023-11-26T07:23:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.876+0000 7ff6c6c2f640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c0159810 con 0x7ff6c00d3760 2023-11-26T07:23:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c5c2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c00cfa20 0x7ff6c00b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c5c2d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c00cfa20 0x7ff6c00b8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37342/0 (socket says 172.21.15.31:37342) 2023-11-26T07:23:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c5c2d640 1 -- 172.21.15.31:0/1020634937 learned_addr learned my addr 172.21.15.31:0/1020634937 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c642e640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c015d880 0x7ff6c01565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.882 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c542c640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6c00d3760 0x7ff6c0153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c5c2d640 1 -- 172.21.15.31:0/1020634937 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6c00d3760 msgr2=0x7ff6c0153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c5c2d640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6c00d3760 0x7ff6c0153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c5c2d640 1 -- 172.21.15.31:0/1020634937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c015d880 msgr2=0x7ff6c01565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:51.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c5c2d640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c015d880 0x7ff6c01565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:51.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c5c2d640 1 -- 172.21.15.31:0/1020634937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8009b30 con 0x7ff6c00cfa20 2023-11-26T07:23:51.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c5c2d640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c00cfa20 0x7ff6c00b8670 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc001ec0 tx=0x7ff6bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6b67fc640 1 -- 172.21.15.31:0/1020634937 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0054e0 con 0x7ff6c00cfa20 2023-11-26T07:23:51.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6b67fc640 1 -- 172.21.15.31:0/1020634937 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc003600 con 0x7ff6c00cfa20 2023-11-26T07:23:51.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6b67fc640 1 -- 172.21.15.31:0/1020634937 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0039b0 con 0x7ff6c00cfa20 2023-11-26T07:23:51.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/1020634937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b0009420 con 0x7ff6c00cfa20 2023-11-26T07:23:51.884 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/1020634937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c00b7e70 con 0x7ff6c00cfa20 2023-11-26T07:23:51.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/1020634937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c0074a00 con 0x7ff6c00cfa20 2023-11-26T07:23:51.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.880+0000 7ff6b67fc640 1 -- 172.21.15.31:0/1020634937 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ff6bc003b10 con 0x7ff6c00cfa20 2023-11-26T07:23:51.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.884+0000 7ff6b67fc640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff6a4066740 0x7ff6a4068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:51.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.884+0000 7ff6b67fc640 1 -- 172.21.15.31:0/1020634937 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff6bc083490 con 0x7ff6c00cfa20 2023-11-26T07:23:51.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.884+0000 7ff6c542c640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff6a4066740 0x7ff6a4068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:51.887 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.884+0000 7ff6c542c640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff6a4066740 0x7ff6a4068c00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00af000 tx=0x7ff6b00a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:51.890 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:51.888+0000 7ff6b67fc640 1 -- 172.21.15.31:0/1020634937 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6bc0509c0 con 0x7ff6c00cfa20 2023-11-26T07:23:52.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.104+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/1020634937 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) v1 -- 0x7ff6c00d11a0 con 0x7ff6c00cfa20 2023-11-26T07:23:52.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.324+0000 7ff6b67fc640 1 -- 172.21.15.31:0/1020634937 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v342) v1 ==== 149+0+0 (secure 0 0 0) 0x7ff6bc017020 con 0x7ff6c00cfa20 2023-11-26T07:23:52.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/1020634937 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff6a4066740 msgr2=0x7ff6a4068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff6a4066740 0x7ff6a4068c00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff6b00af000 tx=0x7ff6b00a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/1020634937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c00cfa20 msgr2=0x7ff6c00b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c00cfa20 0x7ff6c00b8670 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc001ec0 tx=0x7ff6bc00c8e0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/1020634937 shutdown_connections 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff6a4066740 0x7ff6a4068c00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6c015d880 0x7ff6c01565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff6c00d3760 0x7ff6c0153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 --2- 172.21.15.31:0/1020634937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6c00cfa20 0x7ff6c00b8670 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/1020634937 >> 172.21.15.31:0/1020634937 conn(0x7ff6c006b9f0 msgr2=0x7ff6c015f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/1020634937 shutdown_connections 2023-11-26T07:23:52.332 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:52.328+0000 7ff6c6c2f640 1 -- 172.21.15.31:0/1020634937 wait complete. 2023-11-26T07:23:52.352 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... FAIL 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner: 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner:FAIL: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs/test_volumes.py", line 7763, in test_subvolume_no_upgrade_v1_sanity 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner: self.assertEqual(snap_info["has_pending_clones"], "no") 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner:AssertionError: 'yes' != 'no' 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner:- yes 2023-11-26T07:23:52.353 INFO:tasks.cephfs_test_runner:+ no 2023-11-26T07:23:52.354 INFO:tasks.cephfs_test_runner: 2023-11-26T07:23:52.354 INFO:tasks.cephfs_test_runner: 2023-11-26T07:23:52.354 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-26T07:23:52.354 INFO:tasks.cephfs_test_runner:Ran 6 tests in 352.193s 2023-11-26T07:23:52.354 INFO:tasks.cephfs_test_runner: 2023-11-26T07:23:52.354 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2023-11-26T07:23:52.354 INFO:tasks.cephfs_test_runner: 2023-11-26T07:23:52.354 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-26T07:23:52.354 INFO:tasks.cephfs_test_runner:FAIL: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:23:52.354 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... 2023-11-26T07:23:52.355 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-26T07:23:52.355 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-26T07:23:52.355 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs/test_volumes.py", line 7763, in test_subvolume_no_upgrade_v1_sanity 2023-11-26T07:23:52.355 INFO:tasks.cephfs_test_runner: self.assertEqual(snap_info["has_pending_clones"], "no") 2023-11-26T07:23:52.355 INFO:tasks.cephfs_test_runner:AssertionError: 'yes' != 'no' 2023-11-26T07:23:52.355 INFO:tasks.cephfs_test_runner:- yes 2023-11-26T07:23:52.355 INFO:tasks.cephfs_test_runner:+ no 2023-11-26T07:23:52.355 INFO:tasks.cephfs_test_runner: 2023-11-26T07:23:52.355 INFO:tasks.cephfs_test_runner: 2023-11-26T07:23:52.355 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:23:52.473 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=fe001eef6bde4faba3c9651212b11a88 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_44c2237fe77713214148c1fd622c872f346f1b99/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2023-11-26T07:23:52.474 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-26T07:23:52.484 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-26T07:23:52.497 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-26T07:23:52.497 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:52.497 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:52.519 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:23:52.519 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-11-26T07:23:52.519 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:52.573 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357768k,nr_inodes=4089442,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7436 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-26T07:23:52.574 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:23:52.574 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:52.583 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:23:52.583 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-11-26T07:23:52.583 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-26T07:23:52.632 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354668k,nr_inodes=4088667,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3041 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275380k,nr_inodes=818845,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275384k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-26T07:23:52.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:52.633 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:23:52.641 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-26T07:23:52.642 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-26T07:23:52.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:52.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-26T07:23:52.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:52.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-26T07:23:52.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:52.784 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-11-26T07:23:52.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:52.944 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-11-26T07:23:52.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:52.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-11-26T07:23:53.021 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-11-26T07:23:53.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:23:53.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:53.024 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:23:53.033 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-26T07:23:53.033 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-26T07:23:53.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:53.034 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-26T07:23:53.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:53.100 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-26T07:23:53.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:53.186 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-11-26T07:23:53.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:53.355 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2023-11-26T07:23:53.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-26T07:23:53.365 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2023-11-26T07:23:53.432 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2023-11-26T07:23:53.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:23:53.435 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-26T07:23:53.449 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-26T07:23:53.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:23:53.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.592+0000 7f0128957640 1 Processor -- start 2023-11-26T07:23:53.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.592+0000 7f0128957640 1 -- start start 2023-11-26T07:23:53.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.592+0000 7f0128957640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f012415d510 0x7f01240b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.592+0000 7f0128957640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 0x7f012406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.592+0000 7f0128957640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f012406d920 0x7f01240715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.592+0000 7f0128957640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012415d8f0 con 0x7f012406d920 2023-11-26T07:23:53.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.592+0000 7f0128957640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012406cf50 con 0x7f012415d510 2023-11-26T07:23:53.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.592+0000 7f0128957640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012406cc20 con 0x7f01240b3cc0 2023-11-26T07:23:53.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0122d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f012406d920 0x7f01240715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f012259c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f012415d510 0x7f01240b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0122d9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f012406d920 0x7f01240715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37354/0 (socket says 172.21.15.31:37354) 2023-11-26T07:23:53.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0122d9d640 1 -- 172.21.15.31:0/351103676 learned_addr learned my addr 172.21.15.31:0/351103676 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f011bfff640 1 --2- 172.21.15.31:0/351103676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 0x7f012406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0122d9d640 1 -- 172.21.15.31:0/351103676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 msgr2=0x7f012406d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0122d9d640 1 --2- 172.21.15.31:0/351103676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 0x7f012406d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0122d9d640 1 -- 172.21.15.31:0/351103676 >> v2:172.21.15.97:3300/0 conn(0x7f012415d510 msgr2=0x7f01240b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0122d9d640 1 --2- 172.21.15.31:0/351103676 >> v2:172.21.15.97:3300/0 conn(0x7f012415d510 0x7f01240b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0122d9d640 1 -- 172.21.15.31:0/351103676 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01240b31d0 con 0x7f012406d920 2023-11-26T07:23:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0122d9d640 1 --2- 172.21.15.31:0/351103676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 0x7f01240715b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f012406b8f0 tx=0x7f011c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8736a7920dc6223 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0121d9b640 1 -- 172.21.15.31:0/351103676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c0a5730 con 0x7f012406d920 2023-11-26T07:23:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0121d9b640 1 -- 172.21.15.31:0/351103676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011c0a5890 con 0x7f012406d920 2023-11-26T07:23:53.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.596+0000 7f0121d9b640 1 -- 172.21.15.31:0/351103676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c0a5c20 con 0x7f012406d920 2023-11-26T07:23:53.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 -- 172.21.15.31:0/351103676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 msgr2=0x7f01240715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 --2- 172.21.15.31:0/351103676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 0x7f01240715b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f012406b8f0 tx=0x7f011c01c9a0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 -- 172.21.15.31:0/351103676 shutdown_connections 2023-11-26T07:23:53.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 --2- 172.21.15.31:0/351103676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 0x7f01240715b0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 --2- 172.21.15.31:0/351103676 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 0x7f012406d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 --2- 172.21.15.31:0/351103676 >> v2:172.21.15.97:3300/0 conn(0x7f012415d510 0x7f01240b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 -- 172.21.15.31:0/351103676 >> 172.21.15.31:0/351103676 conn(0x7f01240b1e40 msgr2=0x7f0124070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 -- 172.21.15.31:0/351103676 shutdown_connections 2023-11-26T07:23:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 -- 172.21.15.31:0/351103676 wait complete. 2023-11-26T07:23:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 Processor -- start 2023-11-26T07:23:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 -- start start 2023-11-26T07:23:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 0x7f0124155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 0x7f01240cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415d510 0x7f01241451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124156230 con 0x7f012406d920 2023-11-26T07:23:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0124155620 con 0x7f012415d510 2023-11-26T07:23:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0128957640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0124155900 con 0x7f01240b3cc0 2023-11-26T07:23:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f012259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 0x7f0124155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f012259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 0x7f0124155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37368/0 (socket says 172.21.15.31:37368) 2023-11-26T07:23:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f012259c640 1 -- 172.21.15.31:0/1490801665 learned_addr learned my addr 172.21.15.31:0/1490801665 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f0122d9d640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415d510 0x7f01241451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.600+0000 7f011bfff640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 0x7f01240cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f012259c640 1 -- 172.21.15.31:0/1490801665 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 msgr2=0x7f01240cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f012259c640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 0x7f01240cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f012259c640 1 -- 172.21.15.31:0/1490801665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415d510 msgr2=0x7f01241451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f012259c640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415d510 0x7f01241451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f012259c640 1 -- 172.21.15.31:0/1490801665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c009b30 con 0x7f012406d920 2023-11-26T07:23:53.608 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f011bfff640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 0x7f01240cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:53.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f0122d9d640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415d510 0x7f01241451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:53.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f012259c640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 0x7f0124155040 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f011400e870 tx=0x7f011400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f011affd640 1 -- 172.21.15.31:0/1490801665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114009510 con 0x7f012406d920 2023-11-26T07:23:53.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f011affd640 1 -- 172.21.15.31:0/1490801665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011400eec0 con 0x7f012406d920 2023-11-26T07:23:53.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f011affd640 1 -- 172.21.15.31:0/1490801665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011400cb40 con 0x7f012406d920 2023-11-26T07:23:53.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f0128957640 1 -- 172.21.15.31:0/1490801665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0124155be0 con 0x7f012406d920 2023-11-26T07:23:53.609 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f0128957640 1 -- 172.21.15.31:0/1490801665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01240cefc0 con 0x7f012406d920 2023-11-26T07:23:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f0128957640 1 -- 172.21.15.31:0/1490801665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012404e850 con 0x7f012406d920 2023-11-26T07:23:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f011affd640 1 -- 172.21.15.31:0/1490801665 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f011400cca0 con 0x7f012406d920 2023-11-26T07:23:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f011affd640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0104066740 0x7f0104068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f011affd640 1 -- 172.21.15.31:0/1490801665 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f01140840b0 con 0x7f012406d920 2023-11-26T07:23:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.604+0000 7f011bfff640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0104066740 0x7f0104068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.611 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.608+0000 7f011bfff640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0104066740 0x7f0104068c00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f01240cc720 tx=0x7f010c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.613 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.608+0000 7f011affd640 1 -- 172.21.15.31:0/1490801665 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0114051530 con 0x7f012406d920 2023-11-26T07:23:53.768 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.764+0000 7f0128957640 1 -- 172.21.15.31:0/1490801665 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0124160fb0 con 0x7f0104066740 2023-11-26T07:23:53.770 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f011affd640 1 -- 172.21.15.31:0/1490801665 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50198 (secure 0 0 0) 0x7f010c0119d0 con 0x7f0104066740 2023-11-26T07:23:53.771 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:53.771 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:23:53.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 -- 172.21.15.31:0/1490801665 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0104066740 msgr2=0x7f0104068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.772 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0104066740 0x7f0104068c00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f01240cc720 tx=0x7f010c008040 comp rx=0 tx=0).stop 2023-11-26T07:23:53.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 -- 172.21.15.31:0/1490801665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 msgr2=0x7f0124155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 0x7f0124155040 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f011400e870 tx=0x7f011400ec70 comp rx=0 tx=0).stop 2023-11-26T07:23:53.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 -- 172.21.15.31:0/1490801665 shutdown_connections 2023-11-26T07:23:53.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0104066740 0x7f0104068c00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f012415d510 0x7f01241451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f01240b3cc0 0x7f01240cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.773 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 --2- 172.21.15.31:0/1490801665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f012406d920 0x7f0124155040 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 -- 172.21.15.31:0/1490801665 >> 172.21.15.31:0/1490801665 conn(0x7f01240b1e40 msgr2=0x7f01240b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:53.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 -- 172.21.15.31:0/1490801665 shutdown_connections 2023-11-26T07:23:53.774 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.768+0000 7f0128957640 1 -- 172.21.15.31:0/1490801665 wait complete. 2023-11-26T07:23:53.791 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-11-26T07:23:52.219336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":276872,"kb_used_data":27424,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754697848,"statfs":{"total":773094113280,"available":772810596352,"internally_reserved":0,"allocated":28082176,"data_stored":22377341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17184176,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-65,"num_read_kb":-8237,"num_write":-184,"num_write_kb":-16866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001179"},"pg_stats":[{"pgid":"1.0","version":"69'128","reported_seq":273,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:23:50.331430+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:23:50.331430+0000","last_peered":"2023-11-26T07:23:50.331430+0000","last_clean":"2023-11-26T07:23:50.331430+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:23:50.331430+0000","last_undegraded":"2023-11-26T07:23:50.331430+0000","last_fullsized":"2023-11-26T07:23:50.331430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1871872,"data_stored":1148448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85599999999999998,"5min":1.008,"15min":1.008},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.94399999999999995},"last":0.51600000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.68300000000000005},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.90100000000000002,"5min":0.98799999999999999,"15min":1.0189999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.504,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86799999999999999,"5min":0.93000000000000005,"15min":1.1519999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0700000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":0.92400000000000004,"15min":1.016},"last":0.53300000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73099999999999998,"15min":0.73899999999999999},"min":{"1min":0.53300000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88500000000000001,"5min":1.0089999999999999,"15min":1.2270000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.63600000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93700000000000006,"5min":0.97999999999999998,"15min":1.1060000000000001},"last":0.59199999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.69499999999999995},"min":{"1min":0.46600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":1.0309999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.67700000000000005},"min":{"1min":0.47099999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.81100000000000005,"5min":0.91700000000000004,"15min":1.1759999999999999},"last":0.36299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69599999999999995,"15min":0.76200000000000001},"min":{"1min":0.52500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":1.5129999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.82999999999999996},"min":{"1min":0.60799999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86599999999999999,"5min":0.96799999999999997,"15min":1.5780000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.749},"min":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88800000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.442},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.81899999999999995},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.093},"last":0.495}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35472,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336368,"statfs":{"total":96636764160,"available":96600440832,"internally_reserved":0,"allocated":4395008,"data_stored":3692310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.83399999999999996,"15min":0.77000000000000002},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":1.2130000000000001,"15min":0.90400000000000003},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69899999999999995,"15min":0.73799999999999999},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":2.1880000000000002,"15min":3.1520000000000001},"last":0.624},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.81299999999999994},"min":{"1min":0.50900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.02,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.77300000000000002,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97099999999999997,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.69599999999999995},"min":{"1min":0.55000000000000004,"5min":0.47599999999999998,"15min":0.41699999999999998},"max":{"1min":0.95299999999999996,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.66100000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.751},"min":{"1min":0.504,"5min":0.499,"15min":0.46700000000000003},"max":{"1min":1.0529999999999999,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.89300000000000002,"15min":0.85699999999999998},"min":{"1min":0.55500000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92000000000000004,"5min":18.77,"15min":18.77},"last":0.59299999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.379,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0760000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.435},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.145,"5min":1.2070000000000001,"15min":1.2669999999999999},"last":0.503}]},{"osd":5,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64300000000000002,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.004,"5min":1.508,"15min":1.508},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72999999999999998,"15min":0.77900000000000003},"min":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1719999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73899999999999999,"15min":0.78300000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2250000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.77000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.099,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34452,"kb_used_data":3272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337388,"statfs":{"total":96636764160,"available":96601485312,"internally_reserved":0,"allocated":3350528,"data_stored":2644268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.60099999999999998},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0149999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.48599999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88100000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.83399999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.50700000000000001,"5min":0.42799999999999999,"15min":0.38300000000000001},"max":{"1min":1.0720000000000001,"5min":1.778,"15min":1.778},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68899999999999995,"15min":0.67200000000000004},"min":{"1min":0.45900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.1399999999999999,"5min":1.143,"15min":1.143},"last":0.624}]},{"osd":2,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.39600000000000002},"max":{"1min":1.0609999999999999,"5min":1.081,"15min":1.081},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.98999999999999999,"5min":1.823,"15min":1.823},"last":0.746}]},{"osd":3,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.53200000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.095,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.873}]},{"osd":4,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64400000000000002,"15min":0.68400000000000005},"min":{"1min":0.44600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93799999999999994,"5min":1.137,"15min":1.1859999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72699999999999998,"15min":0.76900000000000002},"min":{"1min":0.48099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.016,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.91300000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.871,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67900000000000005,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.434,"15min":0.434},"max":{"1min":1.1479999999999999,"5min":1.268,"15min":1.268},"last":0.93899999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.495,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0249999999999999,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.749,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.121,"5min":1.861,"15min":1.861},"last":0.84799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.67200000000000004},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":1.536,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.77400000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.747},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.51400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0249999999999999,"5min":1.05,"15min":1.077},"last":0.69699999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":1.236,"15min":1.302},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72099999999999997,"15min":0.751},"min":{"1min":0.57699999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.65400000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.873,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.752,"15min":0.78500000000000003},"min":{"1min":0.629,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.80300000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67800000000000005,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1020000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.45000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4550000000000001,"5min":2.544,"15min":2.544},"last":0.94499999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3129999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.879},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.749,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.96399999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.52700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.016,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77100000000000002,"15min":0.81000000000000005},"min":{"1min":0.55200000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.93100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36508,"kb_used_data":5328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335332,"statfs":{"total":96636764160,"available":96599379968,"internally_reserved":0,"allocated":5455872,"data_stored":4746819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47199999999999998,"15min":0.47299999999999998},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.74199999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.48999999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.62,"5min":1.294,"15min":1.294},"last":0.57999999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.442,"15min":0.442},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61199999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.626,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56000000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.39000000000000001,"5min":0.38,"15min":0.36499999999999999},"max":{"1min":0.66800000000000004,"5min":1.456,"15min":1.456},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.72599999999999998,"5min":1.139,"15min":1.139},"last":0.69699999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.46400000000000002,"15min":0.45300000000000001},"max":{"1min":1.042,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.091,"5min":1.2509999999999999,"15min":1.2629999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.57099999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0640000000000001,"5min":1.2649999999999999,"15min":1.395},"last":0.75},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.753,"15min":0.76800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0600000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77700000000000002,"15min":0.78900000000000003},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.0960000000000001,"15min":1.333},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.80000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.012,"5min":1.228,"15min":1.347},"last":0.93500000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.149,"5min":1.397,"15min":1.397},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.032,"5min":1.1859999999999999,"15min":1.363},"last":0.82499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35488,"kb_used_data":4308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336352,"statfs":{"total":96636764160,"available":96600424448,"internally_reserved":0,"allocated":4411392,"data_stored":3698441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.61899999999999999},"min":{"1min":0.28000000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.74199999999999999,"5min":1.036,"15min":3.931},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.67900000000000005},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":1.655,"15min":3.9649999999999999},"last":0.442}]},{"osd":1,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50700000000000001,"15min":0.66100000000000003},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.65500000000000003,"5min":1.073,"15min":3.9929999999999999},"last":0.504},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52600000000000002,"15min":0.66100000000000003},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73599999999999999,"5min":2.7650000000000001,"15min":4.0300000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.68400000000000005,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57399999999999995,"15min":0.84399999999999997},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76400000000000001,"5min":1.7909999999999999,"15min":8.0990000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78200000000000003,"15min":0.90600000000000003},"min":{"1min":0.60699999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0600000000000001,"5min":1.1240000000000001,"15min":4.431},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.88800000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99299999999999999,"5min":1.7050000000000001,"15min":4.1429999999999998},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92100000000000004,"5min":2.036,"15min":2.036},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81499999999999995,"15min":0.94099999999999995},"min":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.96699999999999997,"5min":1.8069999999999999,"15min":4.4790000000000001},"last":0.72999999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.95299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0129999999999999,"5min":1.7310000000000001,"15min":4.6859999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.54900000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.1180000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.96599999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0880000000000001,"5min":3.4740000000000002,"15min":4.7270000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.038,"5min":1.323,"15min":1.609},"last":0.58099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36504,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335336,"statfs":{"total":96636764160,"available":96599384064,"internally_reserved":0,"allocated":5451776,"data_stored":4741546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.441,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.505,"5min":1.034,"15min":1.034},"last":0.438},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51000000000000001,"15min":0.504},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.56799999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.45100000000000001},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.502},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.51900000000000002},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60499999999999998,"5min":0.71499999999999997,"15min":1.3979999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53600000000000003,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69499999999999995,"5min":0.82199999999999995,"15min":1.4810000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63700000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.68600000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47599999999999998},"max":{"1min":0.95599999999999996,"5min":1.143,"15min":1.143},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":1.0740000000000001,"15min":1.216},"last":0.69099999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.46600000000000003},"max":{"1min":0.90900000000000003,"5min":1.175,"15min":1.262},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89200000000000002,"5min":1.343,"15min":1.54},"last":0.84199999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.89100000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.56699999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.874,"5min":1.254,"15min":1.333},"last":0.73799999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.746,"15min":0.75700000000000001},"min":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92000000000000004,"5min":1.2090000000000001,"15min":1.5820000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91500000000000004,"5min":1.377,"15min":1.377},"last":0.621}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1867776,"data_stored":1147429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.25},"max":{"1min":0.55900000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68899999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39900000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29499999999999998},"max":{"1min":0.72099999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.59099999999999997,"5min":0.79700000000000004,"15min":0.85999999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.624,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.36099999999999999},"max":{"1min":0.67000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59099999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80400000000000005,"15min":0.80900000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95699999999999996,"5min":1.365,"15min":1.365},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81200000000000006,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0009999999999999,"5min":1.181,"15min":1.181},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.747,"15min":0.755},"min":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.78800000000000003,"15min":0.79700000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.747},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.79700000000000004},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.1040000000000001,"5min":1.323,"15min":1.323},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81799999999999995,"15min":0.82399999999999995},"min":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0569999999999999,"5min":1.1180000000000001,"15min":1.123},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.95799999999999996,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:23:53.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:23:53.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.932+0000 7fc64c757640 1 Processor -- start 2023-11-26T07:23:53.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.932+0000 7fc64c757640 1 -- start start 2023-11-26T07:23:53.937 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.932+0000 7fc64c757640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6440b3cc0 0x7fc64406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.932+0000 7fc64c757640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc64406d920 0x7fc6440715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.932+0000 7fc64c757640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64406cf50 con 0x7fc64415d510 2023-11-26T07:23:53.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc64c757640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64415d8f0 con 0x7fc64406d920 2023-11-26T07:23:53.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc64c757640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64406cc20 con 0x7fc6440b3cc0 2023-11-26T07:23:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc64acf4640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fc64406d920 0x7fc6440715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc64a4f3640 1 --1- >> v1:172.21.15.97:6789/0 conn(0x7fc64415d510 0x7fc6440b29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.97:6789/0 says I am v1:172.21.15.31:36994/0 (socket says 172.21.15.31:36994) 2023-11-26T07:23:53.939 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc649cf2640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6440b3cc0 0x7fc64406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc64a4f3640 1 -- 172.21.15.31:0/301168269 learned_addr learned my addr 172.21.15.31:0/301168269 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc6494f1640 1 -- 172.21.15.31:0/301168269 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881711985 0 0) 0x7fc64406cf50 con 0x7fc64415d510 2023-11-26T07:23:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc6494f1640 1 -- 172.21.15.31:0/301168269 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc630003530 con 0x7fc64415d510 2023-11-26T07:23:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc649cf2640 1 -- 172.21.15.31:0/301168269 >> v2:172.21.15.97:3300/0 conn(0x7fc64406d920 msgr2=0x7fc6440715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc649cf2640 1 --2- 172.21.15.31:0/301168269 >> v2:172.21.15.97:3300/0 conn(0x7fc64406d920 0x7fc6440715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc649cf2640 1 -- 172.21.15.31:0/301168269 >> v1:172.21.15.97:6789/0 conn(0x7fc64415d510 legacy=0x7fc6440b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc649cf2640 1 -- 172.21.15.31:0/301168269 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440b31d0 con 0x7fc6440b3cc0 2023-11-26T07:23:53.940 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc64acf4640 1 --2- 172.21.15.31:0/301168269 >> v2:172.21.15.97:3300/0 conn(0x7fc64406d920 0x7fc6440715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:53.941 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.936+0000 7fc649cf2640 1 --2- 172.21.15.31:0/301168269 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6440b3cc0 0x7fc64406d3e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc6340097c0 tx=0x7fc63401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48651a97a9d68726 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc6494f1640 1 -- 172.21.15.31:0/301168269 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340ad660 con 0x7fc6440b3cc0 2023-11-26T07:23:53.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc6494f1640 1 -- 172.21.15.31:0/301168269 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6340ad7c0 con 0x7fc6440b3cc0 2023-11-26T07:23:53.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc6494f1640 1 -- 172.21.15.31:0/301168269 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340adb50 con 0x7fc6440b3cc0 2023-11-26T07:23:53.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 -- 172.21.15.31:0/301168269 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6440b3cc0 msgr2=0x7fc64406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.942 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 --2- 172.21.15.31:0/301168269 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6440b3cc0 0x7fc64406d3e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc6340097c0 tx=0x7fc63401c450 comp rx=0 tx=0).stop 2023-11-26T07:23:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 -- 172.21.15.31:0/301168269 shutdown_connections 2023-11-26T07:23:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 --2- 172.21.15.31:0/301168269 >> v2:172.21.15.97:3300/0 conn(0x7fc64406d920 0x7fc6440715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 --2- 172.21.15.31:0/301168269 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc6440b3cc0 0x7fc64406d3e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 -- 172.21.15.31:0/301168269 >> 172.21.15.31:0/301168269 conn(0x7fc6440b1e40 msgr2=0x7fc644070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:53.943 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 -- 172.21.15.31:0/301168269 shutdown_connections 2023-11-26T07:23:53.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 -- 172.21.15.31:0/301168269 wait complete. 2023-11-26T07:23:53.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 Processor -- start 2023-11-26T07:23:53.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 -- start start 2023-11-26T07:23:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc64406d920 0x7fc64414bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6440b3cc0 0x7fc644151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64415d510 0x7fc644148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64414c490 con 0x7fc64415d510 2023-11-26T07:23:53.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440afb60 con 0x7fc6440b3cc0 2023-11-26T07:23:53.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64c757640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6440afe40 con 0x7fc64406d920 2023-11-26T07:23:53.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64acf4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64415d510 0x7fc644148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64acf4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64415d510 0x7fc644148ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55792/0 (socket says 172.21.15.31:55792) 2023-11-26T07:23:53.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.940+0000 7fc64acf4640 1 -- 172.21.15.31:0/2885493612 learned_addr learned my addr 172.21.15.31:0/2885493612 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:53.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64a4f3640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc64406d920 0x7fc64414bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc649cf2640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6440b3cc0 0x7fc644151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64acf4640 1 -- 172.21.15.31:0/2885493612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc64406d920 msgr2=0x7fc64414bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64acf4640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc64406d920 0x7fc64414bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64acf4640 1 -- 172.21.15.31:0/2885493612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6440b3cc0 msgr2=0x7fc644151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:53.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64acf4640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6440b3cc0 0x7fc644151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:53.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64acf4640 1 -- 172.21.15.31:0/2885493612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c009a70 con 0x7fc64415d510 2023-11-26T07:23:53.948 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64a4f3640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc64406d920 0x7fc64414bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64acf4640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64415d510 0x7fc644148ed0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00efc0 tx=0x7fc63c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc63affd640 1 -- 172.21.15.31:0/2885493612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00cdf0 con 0x7fc64415d510 2023-11-26T07:23:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc63affd640 1 -- 172.21.15.31:0/2885493612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63c00f040 con 0x7fc64415d510 2023-11-26T07:23:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc63affd640 1 -- 172.21.15.31:0/2885493612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63c003be0 con 0x7fc64415d510 2023-11-26T07:23:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64c757640 1 -- 172.21.15.31:0/2885493612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc634009420 con 0x7fc64415d510 2023-11-26T07:23:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64c757640 1 -- 172.21.15.31:0/2885493612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6440b0400 con 0x7fc64415d510 2023-11-26T07:23:53.949 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc64c757640 1 -- 172.21.15.31:0/2885493612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6440ae910 con 0x7fc64415d510 2023-11-26T07:23:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc63affd640 1 -- 172.21.15.31:0/2885493612 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fc63c014480 con 0x7fc64415d510 2023-11-26T07:23:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc63affd640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc628066740 0x7fc628068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:53.950 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.944+0000 7fc63affd640 1 -- 172.21.15.31:0/2885493612 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc63c084d30 con 0x7fc64415d510 2023-11-26T07:23:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.948+0000 7fc64a4f3640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc628066740 0x7fc628068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.948+0000 7fc64a4f3640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc628066740 0x7fc628068c00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6400011f0 tx=0x7fc640009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:53.953 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:53.948+0000 7fc63affd640 1 -- 172.21.15.31:0/2885493612 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc63c089070 con 0x7fc64415d510 2023-11-26T07:23:54.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.108+0000 7fc64c757640 1 -- 172.21.15.31:0/2885493612 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6440ae190 con 0x7fc628066740 2023-11-26T07:23:54.112 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.108+0000 7fc63affd640 1 -- 172.21.15.31:0/2885493612 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50198 (secure 0 0 0) 0x7fc6400013e0 con 0x7fc628066740 2023-11-26T07:23:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:23:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 -- 172.21.15.31:0/2885493612 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc628066740 msgr2=0x7fc628068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc628066740 0x7fc628068c00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc6400011f0 tx=0x7fc640009040 comp rx=0 tx=0).stop 2023-11-26T07:23:54.114 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 -- 172.21.15.31:0/2885493612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64415d510 msgr2=0x7fc644148ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64415d510 0x7fc644148ed0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00efc0 tx=0x7fc63c00c490 comp rx=0 tx=0).stop 2023-11-26T07:23:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 -- 172.21.15.31:0/2885493612 shutdown_connections 2023-11-26T07:23:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc628066740 0x7fc628068c00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64415d510 0x7fc644148ed0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6440b3cc0 0x7fc644151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 --2- 172.21.15.31:0/2885493612 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc64406d920 0x7fc64414bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.115 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 -- 172.21.15.31:0/2885493612 >> 172.21.15.31:0/2885493612 conn(0x7fc6440b1e40 msgr2=0x7fc644153480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 -- 172.21.15.31:0/2885493612 shutdown_connections 2023-11-26T07:23:54.116 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.112+0000 7fc64c757640 1 -- 172.21.15.31:0/2885493612 wait complete. 2023-11-26T07:23:54.133 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2023-11-26T07:23:52.219336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":276872,"kb_used_data":27424,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754697848,"statfs":{"total":773094113280,"available":772810596352,"internally_reserved":0,"allocated":28082176,"data_stored":22377341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17184176,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-65,"num_read_kb":-8237,"num_write":-184,"num_write_kb":-16866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001179"},"pg_stats":[{"pgid":"1.0","version":"69'128","reported_seq":273,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:23:50.331430+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:23:50.331430+0000","last_peered":"2023-11-26T07:23:50.331430+0000","last_clean":"2023-11-26T07:23:50.331430+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:23:50.331430+0000","last_undegraded":"2023-11-26T07:23:50.331430+0000","last_fullsized":"2023-11-26T07:23:50.331430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1871872,"data_stored":1148448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85599999999999998,"5min":1.008,"15min":1.008},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.94399999999999995},"last":0.51600000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.68300000000000005},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.90100000000000002,"5min":0.98799999999999999,"15min":1.0189999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.504,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86799999999999999,"5min":0.93000000000000005,"15min":1.1519999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0700000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":0.92400000000000004,"15min":1.016},"last":0.53300000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73099999999999998,"15min":0.73899999999999999},"min":{"1min":0.53300000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88500000000000001,"5min":1.0089999999999999,"15min":1.2270000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.63600000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93700000000000006,"5min":0.97999999999999998,"15min":1.1060000000000001},"last":0.59199999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.69499999999999995},"min":{"1min":0.46600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":1.0309999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.67700000000000005},"min":{"1min":0.47099999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.81100000000000005,"5min":0.91700000000000004,"15min":1.1759999999999999},"last":0.36299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69599999999999995,"15min":0.76200000000000001},"min":{"1min":0.52500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":1.5129999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.82999999999999996},"min":{"1min":0.60799999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86599999999999999,"5min":0.96799999999999997,"15min":1.5780000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.749},"min":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88800000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.442},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.81899999999999995},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.093},"last":0.495}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35472,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336368,"statfs":{"total":96636764160,"available":96600440832,"internally_reserved":0,"allocated":4395008,"data_stored":3692310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.83399999999999996,"15min":0.77000000000000002},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":1.2130000000000001,"15min":0.90400000000000003},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69899999999999995,"15min":0.73799999999999999},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":2.1880000000000002,"15min":3.1520000000000001},"last":0.624},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.81299999999999994},"min":{"1min":0.50900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.02,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.77300000000000002,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97099999999999997,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.69599999999999995},"min":{"1min":0.55000000000000004,"5min":0.47599999999999998,"15min":0.41699999999999998},"max":{"1min":0.95299999999999996,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.66100000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.751},"min":{"1min":0.504,"5min":0.499,"15min":0.46700000000000003},"max":{"1min":1.0529999999999999,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.89300000000000002,"15min":0.85699999999999998},"min":{"1min":0.55500000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92000000000000004,"5min":18.77,"15min":18.77},"last":0.59299999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.379,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0760000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.435},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.145,"5min":1.2070000000000001,"15min":1.2669999999999999},"last":0.503}]},{"osd":5,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64300000000000002,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.004,"5min":1.508,"15min":1.508},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72999999999999998,"15min":0.77900000000000003},"min":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1719999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73899999999999999,"15min":0.78300000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2250000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.77000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.099,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34452,"kb_used_data":3272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337388,"statfs":{"total":96636764160,"available":96601485312,"internally_reserved":0,"allocated":3350528,"data_stored":2644268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.60099999999999998},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0149999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.48599999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88100000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.83399999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.50700000000000001,"5min":0.42799999999999999,"15min":0.38300000000000001},"max":{"1min":1.0720000000000001,"5min":1.778,"15min":1.778},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68899999999999995,"15min":0.67200000000000004},"min":{"1min":0.45900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.1399999999999999,"5min":1.143,"15min":1.143},"last":0.624}]},{"osd":2,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.39600000000000002},"max":{"1min":1.0609999999999999,"5min":1.081,"15min":1.081},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.98999999999999999,"5min":1.823,"15min":1.823},"last":0.746}]},{"osd":3,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.53200000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.095,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.873}]},{"osd":4,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64400000000000002,"15min":0.68400000000000005},"min":{"1min":0.44600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93799999999999994,"5min":1.137,"15min":1.1859999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72699999999999998,"15min":0.76900000000000002},"min":{"1min":0.48099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.016,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.91300000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.871,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67900000000000005,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.434,"15min":0.434},"max":{"1min":1.1479999999999999,"5min":1.268,"15min":1.268},"last":0.93899999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.495,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0249999999999999,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.749,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.121,"5min":1.861,"15min":1.861},"last":0.84799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.67200000000000004},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":1.536,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.77400000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.747},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.51400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0249999999999999,"5min":1.05,"15min":1.077},"last":0.69699999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":1.236,"15min":1.302},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72099999999999997,"15min":0.751},"min":{"1min":0.57699999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.65400000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.873,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.752,"15min":0.78500000000000003},"min":{"1min":0.629,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.80300000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67800000000000005,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1020000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.45000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4550000000000001,"5min":2.544,"15min":2.544},"last":0.94499999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3129999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.879},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.749,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.96399999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.52700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.016,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77100000000000002,"15min":0.81000000000000005},"min":{"1min":0.55200000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.93100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36508,"kb_used_data":5328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335332,"statfs":{"total":96636764160,"available":96599379968,"internally_reserved":0,"allocated":5455872,"data_stored":4746819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47199999999999998,"15min":0.47299999999999998},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.74199999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.48999999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.62,"5min":1.294,"15min":1.294},"last":0.57999999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.442,"15min":0.442},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61199999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.626,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56000000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.39000000000000001,"5min":0.38,"15min":0.36499999999999999},"max":{"1min":0.66800000000000004,"5min":1.456,"15min":1.456},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.72599999999999998,"5min":1.139,"15min":1.139},"last":0.69699999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.46400000000000002,"15min":0.45300000000000001},"max":{"1min":1.042,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.091,"5min":1.2509999999999999,"15min":1.2629999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.57099999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0640000000000001,"5min":1.2649999999999999,"15min":1.395},"last":0.75},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.753,"15min":0.76800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0600000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77700000000000002,"15min":0.78900000000000003},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.0960000000000001,"15min":1.333},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.80000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.012,"5min":1.228,"15min":1.347},"last":0.93500000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.149,"5min":1.397,"15min":1.397},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.032,"5min":1.1859999999999999,"15min":1.363},"last":0.82499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35488,"kb_used_data":4308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336352,"statfs":{"total":96636764160,"available":96600424448,"internally_reserved":0,"allocated":4411392,"data_stored":3698441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.61899999999999999},"min":{"1min":0.28000000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.74199999999999999,"5min":1.036,"15min":3.931},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.67900000000000005},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":1.655,"15min":3.9649999999999999},"last":0.442}]},{"osd":1,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50700000000000001,"15min":0.66100000000000003},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.65500000000000003,"5min":1.073,"15min":3.9929999999999999},"last":0.504},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52600000000000002,"15min":0.66100000000000003},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73599999999999999,"5min":2.7650000000000001,"15min":4.0300000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.68400000000000005,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57399999999999995,"15min":0.84399999999999997},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76400000000000001,"5min":1.7909999999999999,"15min":8.0990000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78200000000000003,"15min":0.90600000000000003},"min":{"1min":0.60699999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0600000000000001,"5min":1.1240000000000001,"15min":4.431},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.88800000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99299999999999999,"5min":1.7050000000000001,"15min":4.1429999999999998},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92100000000000004,"5min":2.036,"15min":2.036},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81499999999999995,"15min":0.94099999999999995},"min":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.96699999999999997,"5min":1.8069999999999999,"15min":4.4790000000000001},"last":0.72999999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.95299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0129999999999999,"5min":1.7310000000000001,"15min":4.6859999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.54900000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.1180000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.96599999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0880000000000001,"5min":3.4740000000000002,"15min":4.7270000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.038,"5min":1.323,"15min":1.609},"last":0.58099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36504,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335336,"statfs":{"total":96636764160,"available":96599384064,"internally_reserved":0,"allocated":5451776,"data_stored":4741546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.441,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.505,"5min":1.034,"15min":1.034},"last":0.438},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51000000000000001,"15min":0.504},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.56799999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.45100000000000001},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.502},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.51900000000000002},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60499999999999998,"5min":0.71499999999999997,"15min":1.3979999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53600000000000003,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69499999999999995,"5min":0.82199999999999995,"15min":1.4810000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63700000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.68600000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47599999999999998},"max":{"1min":0.95599999999999996,"5min":1.143,"15min":1.143},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":1.0740000000000001,"15min":1.216},"last":0.69099999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.46600000000000003},"max":{"1min":0.90900000000000003,"5min":1.175,"15min":1.262},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89200000000000002,"5min":1.343,"15min":1.54},"last":0.84199999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.89100000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.56699999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.874,"5min":1.254,"15min":1.333},"last":0.73799999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.746,"15min":0.75700000000000001},"min":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92000000000000004,"5min":1.2090000000000001,"15min":1.5820000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91500000000000004,"5min":1.377,"15min":1.377},"last":0.621}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1867776,"data_stored":1147429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.25},"max":{"1min":0.55900000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68899999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39900000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29499999999999998},"max":{"1min":0.72099999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.59099999999999997,"5min":0.79700000000000004,"15min":0.85999999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.624,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.36099999999999999},"max":{"1min":0.67000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59099999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80400000000000005,"15min":0.80900000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95699999999999996,"5min":1.365,"15min":1.365},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81200000000000006,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0009999999999999,"5min":1.181,"15min":1.181},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.747,"15min":0.755},"min":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.78800000000000003,"15min":0.79700000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.747},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.79700000000000004},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.1040000000000001,"5min":1.323,"15min":1.323},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81799999999999995,"15min":0.82399999999999995},"min":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0569999999999999,"5min":1.1180000000000001,"15min":1.123},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.95799999999999996,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:23:54.135 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-26T07:23:54.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:23:54.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5dc17640 1 Processor -- start 2023-11-26T07:23:54.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5dc17640 1 -- start start 2023-11-26T07:23:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5dc17640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca5815b130 0x7fca58070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5dc17640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fca580cc680 0x7fca580ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5dc17640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca580b4410 0x7fca580b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5dc17640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca580b4dd0 con 0x7fca580b4410 2023-11-26T07:23:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5dc17640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca580709d0 con 0x7fca580cc680 2023-11-26T07:23:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5dc17640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca580706a0 con 0x7fca5815b130 2023-11-26T07:23:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5d416640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca580b4410 0x7fca580b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5d416640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fca580b4410 0x7fca580b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55798/0 (socket says 172.21.15.31:55798) 2023-11-26T07:23:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5d416640 1 -- 172.21.15.31:0/3993994567 learned_addr learned my addr 172.21.15.31:0/3993994567 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:54.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca57fff640 1 --2- 172.21.15.31:0/3993994567 >> v2:172.21.15.97:3300/0 conn(0x7fca580cc680 0x7fca580ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5cc15640 1 --2- 172.21.15.31:0/3993994567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca5815b130 0x7fca58070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5d416640 1 -- 172.21.15.31:0/3993994567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca5815b130 msgr2=0x7fca58070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5d416640 1 --2- 172.21.15.31:0/3993994567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca5815b130 0x7fca58070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5d416640 1 -- 172.21.15.31:0/3993994567 >> v2:172.21.15.97:3300/0 conn(0x7fca580cc680 msgr2=0x7fca580ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5d416640 1 --2- 172.21.15.31:0/3993994567 >> v2:172.21.15.97:3300/0 conn(0x7fca580cc680 0x7fca580ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.282 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.276+0000 7fca5d416640 1 -- 172.21.15.31:0/3993994567 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580cd690 con 0x7fca580b4410 2023-11-26T07:23:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5d416640 1 --2- 172.21.15.31:0/3993994567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca580b4410 0x7fca580b4890 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fca5806b590 tx=0x7fca5001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9483efdc7219eca server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca577fe640 1 -- 172.21.15.31:0/3993994567 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca500a5820 con 0x7fca580b4410 2023-11-26T07:23:54.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca577fe640 1 -- 172.21.15.31:0/3993994567 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca500a5980 con 0x7fca580b4410 2023-11-26T07:23:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca577fe640 1 -- 172.21.15.31:0/3993994567 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca500a5d10 con 0x7fca580b4410 2023-11-26T07:23:54.284 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5dc17640 1 -- 172.21.15.31:0/3993994567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca580b4410 msgr2=0x7fca580b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5dc17640 1 --2- 172.21.15.31:0/3993994567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca580b4410 0x7fca580b4890 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fca5806b590 tx=0x7fca5001c9a0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5dc17640 1 -- 172.21.15.31:0/3993994567 shutdown_connections 2023-11-26T07:23:54.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5dc17640 1 --2- 172.21.15.31:0/3993994567 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca580b4410 0x7fca580b4890 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5dc17640 1 --2- 172.21.15.31:0/3993994567 >> v2:172.21.15.97:3300/0 conn(0x7fca580cc680 0x7fca580ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5dc17640 1 --2- 172.21.15.31:0/3993994567 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca5815b130 0x7fca58070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5dc17640 1 -- 172.21.15.31:0/3993994567 >> 172.21.15.31:0/3993994567 conn(0x7fca580b0e00 msgr2=0x7fca580b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.285 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5dc17640 1 -- 172.21.15.31:0/3993994567 shutdown_connections 2023-11-26T07:23:54.286 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5dc17640 1 -- 172.21.15.31:0/3993994567 wait complete. 2023-11-26T07:23:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.280+0000 7fca5dc17640 1 Processor -- start 2023-11-26T07:23:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5dc17640 1 -- start start 2023-11-26T07:23:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5dc17640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca580b4410 0x7fca5814a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5dc17640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca580cc680 0x7fca58148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5dc17640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5815b130 0x7fca5814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5dc17640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5814d800 con 0x7fca5815b130 2023-11-26T07:23:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5dc17640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca58149e60 con 0x7fca580cc680 2023-11-26T07:23:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5dc17640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5806d140 con 0x7fca580b4410 2023-11-26T07:23:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5d416640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5815b130 0x7fca5814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5d416640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5815b130 0x7fca5814d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55802/0 (socket says 172.21.15.31:55802) 2023-11-26T07:23:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5d416640 1 -- 172.21.15.31:0/712081280 learned_addr learned my addr 172.21.15.31:0/712081280 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5cc15640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca580b4410 0x7fca5814a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca57fff640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca580cc680 0x7fca58148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5d416640 1 -- 172.21.15.31:0/712081280 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca580b4410 msgr2=0x7fca5814a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5d416640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca580b4410 0x7fca5814a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5d416640 1 -- 172.21.15.31:0/712081280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca580cc680 msgr2=0x7fca58148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5d416640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca580cc680 0x7fca58148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5d416640 1 -- 172.21.15.31:0/712081280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca50009b30 con 0x7fca5815b130 2023-11-26T07:23:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5d416640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5815b130 0x7fca5814d2c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fca500af040 tx=0x7fca50019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca557fa640 1 -- 172.21.15.31:0/712081280 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca500b8670 con 0x7fca5815b130 2023-11-26T07:23:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca557fa640 1 -- 172.21.15.31:0/712081280 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca500b87f0 con 0x7fca5815b130 2023-11-26T07:23:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca557fa640 1 -- 172.21.15.31:0/712081280 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca500b8ba0 con 0x7fca5815b130 2023-11-26T07:23:54.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5dc17640 1 -- 172.21.15.31:0/712081280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca58149250 con 0x7fca5815b130 2023-11-26T07:23:54.292 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5dc17640 1 -- 172.21.15.31:0/712081280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca58149810 con 0x7fca5815b130 2023-11-26T07:23:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.284+0000 7fca5dc17640 1 -- 172.21.15.31:0/712081280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca58149ff0 con 0x7fca5815b130 2023-11-26T07:23:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.288+0000 7fca557fa640 1 -- 172.21.15.31:0/712081280 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fca500b8d00 con 0x7fca5815b130 2023-11-26T07:23:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.288+0000 7fca557fa640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fca3c066770 0x7fca3c068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.288+0000 7fca557fa640 1 -- 172.21.15.31:0/712081280 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fca5011f540 con 0x7fca5815b130 2023-11-26T07:23:54.293 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.288+0000 7fca5cc15640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fca3c066770 0x7fca3c068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.294 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.288+0000 7fca5cc15640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fca3c066770 0x7fca3c068c30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009a40 tx=0x7fca4c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.295 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.292+0000 7fca557fa640 1 -- 172.21.15.31:0/712081280 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fca500ecbc0 con 0x7fca5815b130 2023-11-26T07:23:54.451 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.448+0000 7fca5dc17640 1 -- 172.21.15.31:0/712081280 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca58154560 con 0x7fca3c066770 2023-11-26T07:23:54.453 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.448+0000 7fca557fa640 1 -- 172.21.15.31:0/712081280 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50198 (secure 0 0 0) 0x7fca58154560 con 0x7fca3c066770 2023-11-26T07:23:54.454 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:54.455 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:23:54.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 -- 172.21.15.31:0/712081280 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fca3c066770 msgr2=0x7fca3c068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fca3c066770 0x7fca3c068c30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009a40 tx=0x7fca4c009190 comp rx=0 tx=0).stop 2023-11-26T07:23:54.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 -- 172.21.15.31:0/712081280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5815b130 msgr2=0x7fca5814d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.456 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5815b130 0x7fca5814d2c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fca500af040 tx=0x7fca50019740 comp rx=0 tx=0).stop 2023-11-26T07:23:54.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 -- 172.21.15.31:0/712081280 shutdown_connections 2023-11-26T07:23:54.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fca3c066770 0x7fca3c068c30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fca5815b130 0x7fca5814d2c0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca580cc680 0x7fca58148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 --2- 172.21.15.31:0/712081280 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fca580b4410 0x7fca5814a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.457 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 -- 172.21.15.31:0/712081280 >> 172.21.15.31:0/712081280 conn(0x7fca580b0e00 msgr2=0x7fca58153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 -- 172.21.15.31:0/712081280 shutdown_connections 2023-11-26T07:23:54.458 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.452+0000 7fca5dc17640 1 -- 172.21.15.31:0/712081280 wait complete. 2023-11-26T07:23:54.475 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":91,"stamp":"2023-11-26T07:23:54.219613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":276872,"kb_used_data":27424,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754697848,"statfs":{"total":773094113280,"available":772810596352,"internally_reserved":0,"allocated":28082176,"data_stored":22377341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17200346,"num_objects":-42,"num_object_clones":0,"num_object_copies":-84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-42,"num_whiteouts":0,"num_read":-65,"num_read_kb":-8237,"num_write":-195,"num_write_kb":-16888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001080"},"pg_stats":[{"pgid":"1.0","version":"69'128","reported_seq":273,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:23:50.331430+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:23:50.331430+0000","last_peered":"2023-11-26T07:23:50.331430+0000","last_clean":"2023-11-26T07:23:50.331430+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:23:50.331430+0000","last_undegraded":"2023-11-26T07:23:50.331430+0000","last_fullsized":"2023-11-26T07:23:50.331430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1871872,"data_stored":1148448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85599999999999998,"5min":1.008,"15min":1.008},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.94399999999999995},"last":0.51600000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.68300000000000005},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.90100000000000002,"5min":0.98799999999999999,"15min":1.0189999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.504,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86799999999999999,"5min":0.93000000000000005,"15min":1.1519999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0700000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":0.92400000000000004,"15min":1.016},"last":0.53300000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73099999999999998,"15min":0.73899999999999999},"min":{"1min":0.53300000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88500000000000001,"5min":1.0089999999999999,"15min":1.2270000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.63600000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93700000000000006,"5min":0.97999999999999998,"15min":1.1060000000000001},"last":0.59199999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.69499999999999995},"min":{"1min":0.46600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":1.0309999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.67700000000000005},"min":{"1min":0.47099999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.81100000000000005,"5min":0.91700000000000004,"15min":1.1759999999999999},"last":0.36299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69599999999999995,"15min":0.76200000000000001},"min":{"1min":0.52500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":1.5129999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.82999999999999996},"min":{"1min":0.60799999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86599999999999999,"5min":0.96799999999999997,"15min":1.5780000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.749},"min":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88800000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.442},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.81899999999999995},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.093},"last":0.495}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35472,"kb_used_data":4292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336368,"statfs":{"total":96636764160,"available":96600440832,"internally_reserved":0,"allocated":4395008,"data_stored":3692310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.83399999999999996,"15min":0.77000000000000002},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":1.2130000000000001,"15min":0.90400000000000003},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69899999999999995,"15min":0.73799999999999999},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":2.1880000000000002,"15min":3.1520000000000001},"last":0.624},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.81299999999999994},"min":{"1min":0.50900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.02,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.63600000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.77300000000000002,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97099999999999997,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.69599999999999995},"min":{"1min":0.55000000000000004,"5min":0.47599999999999998,"15min":0.41699999999999998},"max":{"1min":0.95299999999999996,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.66100000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.751},"min":{"1min":0.504,"5min":0.499,"15min":0.46700000000000003},"max":{"1min":1.0529999999999999,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.89300000000000002,"15min":0.85699999999999998},"min":{"1min":0.55500000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92000000000000004,"5min":18.77,"15min":18.77},"last":0.59299999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.379,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0760000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.435},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.145,"5min":1.2070000000000001,"15min":1.2669999999999999},"last":0.503}]},{"osd":5,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64300000000000002,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.004,"5min":1.508,"15min":1.508},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72999999999999998,"15min":0.77900000000000003},"min":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1719999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.57099999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73899999999999999,"15min":0.78300000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2250000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.77000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.099,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":34452,"kb_used_data":3272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337388,"statfs":{"total":96636764160,"available":96601485312,"internally_reserved":0,"allocated":3350528,"data_stored":2644268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.60099999999999998},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0149999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.48599999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88100000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.83399999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.50700000000000001,"5min":0.42799999999999999,"15min":0.38300000000000001},"max":{"1min":1.0720000000000001,"5min":1.778,"15min":1.778},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68899999999999995,"15min":0.67200000000000004},"min":{"1min":0.45900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.1399999999999999,"5min":1.143,"15min":1.143},"last":0.624}]},{"osd":2,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.39600000000000002},"max":{"1min":1.0609999999999999,"5min":1.081,"15min":1.081},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.98999999999999999,"5min":1.823,"15min":1.823},"last":0.746}]},{"osd":3,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.53200000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.095,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.873}]},{"osd":4,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64400000000000002,"15min":0.68400000000000005},"min":{"1min":0.44600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93799999999999994,"5min":1.137,"15min":1.1859999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72699999999999998,"15min":0.76900000000000002},"min":{"1min":0.48099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.016,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.91300000000000003}]},{"osd":6,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.871,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67900000000000005,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.434,"15min":0.434},"max":{"1min":1.1479999999999999,"5min":1.268,"15min":1.268},"last":0.93899999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.495,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0249999999999999,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.749,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.121,"5min":1.861,"15min":1.861},"last":0.84799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.67200000000000004},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":1.536,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.77400000000000002}]},{"osd":1,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.747},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.51400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0249999999999999,"5min":1.05,"15min":1.077},"last":0.69699999999999995}]},{"osd":2,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":1.236,"15min":1.302},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72099999999999997,"15min":0.751},"min":{"1min":0.57699999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.65400000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.873,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.752,"15min":0.78500000000000003},"min":{"1min":0.629,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.80300000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67800000000000005,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1020000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.45000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4550000000000001,"5min":2.544,"15min":2.544},"last":0.94499999999999995}]},{"osd":6,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3129999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.879},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.749,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.96399999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.52700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.016,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77100000000000002,"15min":0.81000000000000005},"min":{"1min":0.55200000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.93100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36508,"kb_used_data":5328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335332,"statfs":{"total":96636764160,"available":96599379968,"internally_reserved":0,"allocated":5455872,"data_stored":4746819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47199999999999998,"15min":0.47299999999999998},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.74199999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.48999999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.62,"5min":1.294,"15min":1.294},"last":0.57999999999999996}]},{"osd":1,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.442,"15min":0.442},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61199999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.626,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56000000000000005}]},{"osd":2,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.39000000000000001,"5min":0.38,"15min":0.36499999999999999},"max":{"1min":0.66800000000000004,"5min":1.456,"15min":1.456},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.72599999999999998,"5min":1.139,"15min":1.139},"last":0.69699999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.46400000000000002,"15min":0.45300000000000001},"max":{"1min":1.042,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.091,"5min":1.2509999999999999,"15min":1.2629999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.57099999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0640000000000001,"5min":1.2649999999999999,"15min":1.395},"last":0.75},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.753,"15min":0.76800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0600000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77700000000000002,"15min":0.78900000000000003},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.0960000000000001,"15min":1.333},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.80000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.012,"5min":1.228,"15min":1.347},"last":0.93500000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.149,"5min":1.397,"15min":1.397},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.032,"5min":1.1859999999999999,"15min":1.363},"last":0.82499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35488,"kb_used_data":4308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336352,"statfs":{"total":96636764160,"available":96600424448,"internally_reserved":0,"allocated":4411392,"data_stored":3698441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.61899999999999999},"min":{"1min":0.28000000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.74199999999999999,"5min":1.036,"15min":3.931},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.67900000000000005},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":1.655,"15min":3.9649999999999999},"last":0.442}]},{"osd":1,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50700000000000001,"15min":0.66100000000000003},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.65500000000000003,"5min":1.073,"15min":3.9929999999999999},"last":0.504},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52600000000000002,"15min":0.66100000000000003},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73599999999999999,"5min":2.7650000000000001,"15min":4.0300000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.68400000000000005,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57399999999999995,"15min":0.84399999999999997},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76400000000000001,"5min":1.7909999999999999,"15min":8.0990000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78200000000000003,"15min":0.90600000000000003},"min":{"1min":0.60699999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0600000000000001,"5min":1.1240000000000001,"15min":4.431},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.88800000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99299999999999999,"5min":1.7050000000000001,"15min":4.1429999999999998},"last":0.55400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92100000000000004,"5min":2.036,"15min":2.036},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81499999999999995,"15min":0.94099999999999995},"min":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.96699999999999997,"5min":1.8069999999999999,"15min":4.4790000000000001},"last":0.72999999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.95299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0129999999999999,"5min":1.7310000000000001,"15min":4.6859999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.54900000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.1180000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.96599999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0880000000000001,"5min":3.4740000000000002,"15min":4.7270000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.038,"5min":1.323,"15min":1.609},"last":0.58099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36504,"kb_used_data":5324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335336,"statfs":{"total":96636764160,"available":96599384064,"internally_reserved":0,"allocated":5451776,"data_stored":4741546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.441,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.505,"5min":1.034,"15min":1.034},"last":0.438},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51000000000000001,"15min":0.504},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.56799999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.45100000000000001},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.502},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.51900000000000002},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60499999999999998,"5min":0.71499999999999997,"15min":1.3979999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53600000000000003,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69499999999999995,"5min":0.82199999999999995,"15min":1.4810000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63700000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.68600000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47599999999999998},"max":{"1min":0.95599999999999996,"5min":1.143,"15min":1.143},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":1.0740000000000001,"15min":1.216},"last":0.69099999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.46600000000000003},"max":{"1min":0.90900000000000003,"5min":1.175,"15min":1.262},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89200000000000002,"5min":1.343,"15min":1.54},"last":0.84199999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.89100000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.56699999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.874,"5min":1.254,"15min":1.333},"last":0.73799999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.746,"15min":0.75700000000000001},"min":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92000000000000004,"5min":1.2090000000000001,"15min":1.5820000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91500000000000004,"5min":1.377,"15min":1.377},"last":0.621}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1867776,"data_stored":1147429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.25},"max":{"1min":0.55900000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68899999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39900000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29499999999999998},"max":{"1min":0.72099999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.59099999999999997,"5min":0.79700000000000004,"15min":0.85999999999999999},"last":0.56999999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.624,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.36099999999999999},"max":{"1min":0.67000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59099999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80400000000000005,"15min":0.80900000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95699999999999996,"5min":1.365,"15min":1.365},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81200000000000006,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0009999999999999,"5min":1.181,"15min":1.181},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.747,"15min":0.755},"min":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.78800000000000003,"15min":0.79700000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.747},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.79700000000000004},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.1040000000000001,"5min":1.323,"15min":1.323},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81799999999999995,"15min":0.82399999999999995},"min":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0569999999999999,"5min":1.1180000000000001,"15min":1.123},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.95799999999999996,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:23:54.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-26T07:23:54.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.616+0000 7f947f73e640 1 Processor -- start 2023-11-26T07:23:54.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.616+0000 7f947f73e640 1 -- start start 2023-11-26T07:23:54.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947f73e640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f94780b39c0 0x7f947806c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947f73e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f947806cb70 0x7f9478070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947f73e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947806cf90 con 0x7f947815f540 2023-11-26T07:23:54.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947f73e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780b2650 con 0x7f947806cb70 2023-11-26T07:23:54.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947f73e640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780b2920 con 0x7f94780b39c0 2023-11-26T07:23:54.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947dcdb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f947806cb70 0x7f9478070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947dcdb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f947806cb70 0x7f9478070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55804/0 (socket says 172.21.15.31:55804) 2023-11-26T07:23:54.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947dcdb640 1 -- 172.21.15.31:0/2642903535 learned_addr learned my addr 172.21.15.31:0/2642903535 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:54.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947ccd9640 1 --2- 172.21.15.31:0/2642903535 >> v2:172.21.15.97:3300/0 conn(0x7f94780b39c0 0x7f947806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947dcdb640 1 -- 172.21.15.31:0/2642903535 >> v2:172.21.15.97:3300/0 conn(0x7f94780b39c0 msgr2=0x7f947806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947dcdb640 1 --2- 172.21.15.31:0/2642903535 >> v2:172.21.15.97:3300/0 conn(0x7f94780b39c0 0x7f947806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947dcdb640 1 -- 172.21.15.31:0/2642903535 >> v1:172.21.15.97:6789/0 conn(0x7f947815f540 legacy=0x7f94780b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947dcdb640 1 -- 172.21.15.31:0/2642903535 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780b2c00 con 0x7f947806cb70 2023-11-26T07:23:54.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947ccd9640 1 --2- 172.21.15.31:0/2642903535 >> v2:172.21.15.97:3300/0 conn(0x7f94780b39c0 0x7f947806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:54.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.620+0000 7f947dcdb640 1 --2- 172.21.15.31:0/2642903535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947806cb70 0x7f9478070880 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f947806eb50 tx=0x7f947001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d229d7b42982826 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f946ffff640 1 -- 172.21.15.31:0/2642903535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94700a57d0 con 0x7f947806cb70 2023-11-26T07:23:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f946ffff640 1 -- 172.21.15.31:0/2642903535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94700a5930 con 0x7f947806cb70 2023-11-26T07:23:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f946ffff640 1 -- 172.21.15.31:0/2642903535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94700a5cc0 con 0x7f947806cb70 2023-11-26T07:23:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 -- 172.21.15.31:0/2642903535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947806cb70 msgr2=0x7f9478070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 --2- 172.21.15.31:0/2642903535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947806cb70 0x7f9478070880 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f947806eb50 tx=0x7f947001c910 comp rx=0 tx=0).stop 2023-11-26T07:23:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 -- 172.21.15.31:0/2642903535 shutdown_connections 2023-11-26T07:23:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 --2- 172.21.15.31:0/2642903535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f947806cb70 0x7f9478070880 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 --2- 172.21.15.31:0/2642903535 >> v2:172.21.15.97:3300/0 conn(0x7f94780b39c0 0x7f947806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 -- 172.21.15.31:0/2642903535 >> 172.21.15.31:0/2642903535 conn(0x7f94780b0c40 msgr2=0x7f94780b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 -- 172.21.15.31:0/2642903535 shutdown_connections 2023-11-26T07:23:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 -- 172.21.15.31:0/2642903535 wait complete. 2023-11-26T07:23:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 Processor -- start 2023-11-26T07:23:54.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 -- start start 2023-11-26T07:23:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f947806cb70 0x7f94780cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94780b39c0 0x7f94780cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f947815f540 0x7f9478159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780cd240 con 0x7f94780b39c0 2023-11-26T07:23:54.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780ccf60 con 0x7f947815f540 2023-11-26T07:23:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947f73e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94780cb330 con 0x7f947806cb70 2023-11-26T07:23:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947ccd9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94780b39c0 0x7f94780cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947ccd9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94780b39c0 0x7f94780cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55812/0 (socket says 172.21.15.31:55812) 2023-11-26T07:23:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947ccd9640 1 -- 172.21.15.31:0/1203525736 learned_addr learned my addr 172.21.15.31:0/1203525736 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.624+0000 7f947d4da640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f947806cb70 0x7f94780cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f947dcdb640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f947815f540 0x7f9478159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f947ccd9640 1 -- 172.21.15.31:0/1203525736 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f947806cb70 msgr2=0x7f94780cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f947ccd9640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f947806cb70 0x7f94780cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f947ccd9640 1 -- 172.21.15.31:0/1203525736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f947815f540 msgr2=0x7f9478159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f947ccd9640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f947815f540 0x7f9478159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f947ccd9640 1 -- 172.21.15.31:0/1203525736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9470009b30 con 0x7f94780b39c0 2023-11-26T07:23:54.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f947ccd9640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94780b39c0 0x7f94780cc610 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f946800d800 tx=0x7f946800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f946dffb640 1 -- 172.21.15.31:0/1203525736 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946800b6c0 con 0x7f94780b39c0 2023-11-26T07:23:54.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f946dffb640 1 -- 172.21.15.31:0/1203525736 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946800b820 con 0x7f94780b39c0 2023-11-26T07:23:54.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f946dffb640 1 -- 172.21.15.31:0/1203525736 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946800bbd0 con 0x7f94780b39c0 2023-11-26T07:23:54.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f947f73e640 1 -- 172.21.15.31:0/1203525736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94780cb610 con 0x7f94780b39c0 2023-11-26T07:23:54.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f947f73e640 1 -- 172.21.15.31:0/1203525736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94780cc350 con 0x7f94780b39c0 2023-11-26T07:23:54.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f947f73e640 1 -- 172.21.15.31:0/1203525736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947804e850 con 0x7f94780b39c0 2023-11-26T07:23:54.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f946dffb640 1 -- 172.21.15.31:0/1203525736 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f946800bd30 con 0x7f94780b39c0 2023-11-26T07:23:54.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f946dffb640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f945c066770 0x7f945c068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:54.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.628+0000 7f946dffb640 1 -- 172.21.15.31:0/1203525736 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9468084400 con 0x7f94780b39c0 2023-11-26T07:23:54.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.632+0000 7f947d4da640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f945c066770 0x7f945c068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:54.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.632+0000 7f947d4da640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f945c066770 0x7f945c068c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9474007fd0 tx=0x7f9474009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:54.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.632+0000 7f946dffb640 1 -- 172.21.15.31:0/1203525736 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9468051810 con 0x7f94780b39c0 2023-11-26T07:23:54.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.792+0000 7f947f73e640 1 -- 172.21.15.31:0/1203525736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94780ced70 con 0x7f94780b39c0 2023-11-26T07:23:54.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f946dffb640 1 -- 172.21.15.31:0/1203525736 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+12904 (secure 0 0 0) 0x7f9468054e30 con 0x7f94780b39c0 2023-11-26T07:23:54.798 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:23:54.798 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":102,"fsid":"4c2b4df1-71b1-4841-8807-af144d2610e7","created":"2023-11-26T07:17:07.220870+0000","modified":"2023-11-26T07:23:51.325506+0000","last_up_change":"2023-11-26T07:17:19.804088+0000","last_in_change":"2023-11-26T07:17:09.793096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-26T07:17:20.370306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ed44b69-8020-41c4-a5da-ef18bcf6728f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6817","nonce":4191585440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6819","nonce":4191585440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6823","nonce":4191585440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":4191585440},{"type":"v1","addr":"172.21.15.31:6821","nonce":4191585440}]},"public_addr":"172.21.15.31:6817/4191585440","cluster_addr":"172.21.15.31:6819/4191585440","heartbeat_back_addr":"172.21.15.31:6823/4191585440","heartbeat_front_addr":"172.21.15.31:6821/4191585440","state":["exists","up"]},{"osd":1,"uuid":"27567e0b-06aa-4c17-b136-1fc946605813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6801","nonce":845128234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6803","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6805","nonce":845128234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6811","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6813","nonce":845128234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6807","nonce":845128234},{"type":"v1","addr":"172.21.15.31:6809","nonce":845128234}]},"public_addr":"172.21.15.31:6801/845128234","cluster_addr":"172.21.15.31:6805/845128234","heartbeat_back_addr":"172.21.15.31:6813/845128234","heartbeat_front_addr":"172.21.15.31:6809/845128234","state":["exists","up"]},{"osd":2,"uuid":"bb48a666-f258-451b-93a0-90c77a44d49f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6804","nonce":2371417753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6808","nonce":2371417753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6815","nonce":2371417753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2371417753},{"type":"v1","addr":"172.21.15.31:6812","nonce":2371417753}]},"public_addr":"172.21.15.31:6804/2371417753","cluster_addr":"172.21.15.31:6808/2371417753","heartbeat_back_addr":"172.21.15.31:6815/2371417753","heartbeat_front_addr":"172.21.15.31:6812/2371417753","state":["exists","up"]},{"osd":3,"uuid":"5824e328-a264-4106-85ba-46428653e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6825","nonce":3143140069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6827","nonce":3143140069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6831","nonce":3143140069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3143140069},{"type":"v1","addr":"172.21.15.31:6829","nonce":3143140069}]},"public_addr":"172.21.15.31:6825/3143140069","cluster_addr":"172.21.15.31:6827/3143140069","heartbeat_back_addr":"172.21.15.31:6831/3143140069","heartbeat_front_addr":"172.21.15.31:6829/3143140069","state":["exists","up"]},{"osd":4,"uuid":"85a5d6b2-f995-4478-8add-968a03c0baa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6809","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6812","nonce":362016202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6816","nonce":362016202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6823","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6826","nonce":362016202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6819","nonce":362016202},{"type":"v1","addr":"172.21.15.97:6820","nonce":362016202}]},"public_addr":"172.21.15.97:6812/362016202","cluster_addr":"172.21.15.97:6816/362016202","heartbeat_back_addr":"172.21.15.97:6826/362016202","heartbeat_front_addr":"172.21.15.97:6820/362016202","state":["exists","up"]},{"osd":5,"uuid":"536f84f4-6ab7-494f-bfe9-0e88f03c1366","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6811","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6815","nonce":3669978555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6822","nonce":3669978555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6831","nonce":3669978555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6825","nonce":3669978555},{"type":"v1","addr":"172.21.15.97:6828","nonce":3669978555}]},"public_addr":"172.21.15.97:6815/3669978555","cluster_addr":"172.21.15.97:6822/3669978555","heartbeat_back_addr":"172.21.15.97:6831/3669978555","heartbeat_front_addr":"172.21.15.97:6828/3669978555","state":["exists","up"]},{"osd":6,"uuid":"f7a51918-df92-41a1-8128-0c83b2456051","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6801","nonce":28530448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6803","nonce":28530448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6807","nonce":28530448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":28530448},{"type":"v1","addr":"172.21.15.97:6805","nonce":28530448}]},"public_addr":"172.21.15.97:6801/28530448","cluster_addr":"172.21.15.97:6803/28530448","heartbeat_back_addr":"172.21.15.97:6807/28530448","heartbeat_front_addr":"172.21.15.97:6805/28530448","state":["exists","up"]},{"osd":7,"uuid":"be2367b1-f10f-410c-8eb2-4e56f59b2792","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6810","nonce":4245711455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6813","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6817","nonce":4245711455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6827","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6829","nonce":4245711455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6821","nonce":4245711455},{"type":"v1","addr":"172.21.15.97:6824","nonce":4245711455}]},"public_addr":"172.21.15.97:6810/4245711455","cluster_addr":"172.21.15.97:6817/4245711455","heartbeat_back_addr":"172.21.15.97:6829/4245711455","heartbeat_front_addr":"172.21.15.97:6824/4245711455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.304436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.514924+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.544635+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.433846+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.420288+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.623678+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.627841+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-26T07:17:15.543481+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6835/1001580514":"2023-11-27T07:23:48.312677+0000","172.21.15.31:6833/1001580514":"2023-11-27T07:23:48.312677+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-26T07:23:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 -- 172.21.15.31:0/1203525736 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f945c066770 msgr2=0x7f945c068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f945c066770 0x7f945c068c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9474007fd0 tx=0x7f9474009040 comp rx=0 tx=0).stop 2023-11-26T07:23:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 -- 172.21.15.31:0/1203525736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94780b39c0 msgr2=0x7f94780cc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:54.800 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94780b39c0 0x7f94780cc610 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f946800d800 tx=0x7f946800dcc0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 -- 172.21.15.31:0/1203525736 shutdown_connections 2023-11-26T07:23:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f945c066770 0x7f945c068c30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f947815f540 0x7f9478159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94780b39c0 0x7f94780cc610 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 --2- 172.21.15.31:0/1203525736 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f947806cb70 0x7f94780cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 -- 172.21.15.31:0/1203525736 >> 172.21.15.31:0/1203525736 conn(0x7f94780b0c40 msgr2=0x7f9478163010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:54.801 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 -- 172.21.15.31:0/1203525736 shutdown_connections 2023-11-26T07:23:54.802 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:54.796+0000 7f947f73e640 1 -- 172.21.15.31:0/1203525736 wait complete. 2023-11-26T07:23:55.819 INFO:tasks.ceph:Scrubbing osd.0 2023-11-26T07:23:55.820 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:23:55.978 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f5581032640 1 Processor -- start 2023-11-26T07:23:55.979 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f5581032640 1 -- start start 2023-11-26T07:23:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f5581032640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f557c15a710 0x7f557c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f5581032640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f557c0cbd50 0x7f557c0cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.980 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f5581032640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557c0b2710 con 0x7f557c0cdfb0 2023-11-26T07:23:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f5581032640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557c0cca80 con 0x7f557c15a710 2023-11-26T07:23:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f5581032640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0b29f0 con 0x7f557c0cbd50 2023-11-26T07:23:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f557a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f557c15a710 0x7f557c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f557b59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f557c0cbd50 0x7f557c0cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f557a59c640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f557c15a710 0x7f557c0abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:39926/0 (socket says 172.21.15.31:39926) 2023-11-26T07:23:55.981 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f557a59c640 1 -- 172.21.15.31:0/4130830315 learned_addr learned my addr 172.21.15.31:0/4130830315 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f557a59c640 1 -- 172.21.15.31:0/4130830315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f557c0cbd50 msgr2=0x7f557c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f557a59c640 1 --2- 172.21.15.31:0/4130830315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f557c0cbd50 0x7f557c0cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f557a59c640 1 -- 172.21.15.31:0/4130830315 >> v1:172.21.15.97:6789/0 conn(0x7f557c0cdfb0 legacy=0x7f557c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.982 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f557a59c640 1 -- 172.21.15.31:0/4130830315 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c0b2cd0 con 0x7f557c15a710 2023-11-26T07:23:55.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.976+0000 7f557b59e640 1 --2- 172.21.15.31:0/4130830315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f557c0cbd50 0x7f557c0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:55.983 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f557a59c640 1 --2- 172.21.15.31:0/4130830315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f557c15a710 0x7f557c0abf90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5570009870 tx=0x7f557001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87efdd0209703f8f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5579d9b640 1 -- 172.21.15.31:0/4130830315 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700a95f0 con 0x7f557c15a710 2023-11-26T07:23:55.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5579d9b640 1 -- 172.21.15.31:0/4130830315 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55700a9750 con 0x7f557c15a710 2023-11-26T07:23:55.984 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5579d9b640 1 -- 172.21.15.31:0/4130830315 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700a9ae0 con 0x7f557c15a710 2023-11-26T07:23:55.985 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5581032640 1 -- 172.21.15.31:0/4130830315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f557c15a710 msgr2=0x7f557c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5581032640 1 --2- 172.21.15.31:0/4130830315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f557c15a710 0x7f557c0abf90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5570009870 tx=0x7f557001c510 comp rx=0 tx=0).stop 2023-11-26T07:23:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5581032640 1 -- 172.21.15.31:0/4130830315 shutdown_connections 2023-11-26T07:23:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5581032640 1 --2- 172.21.15.31:0/4130830315 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f557c0cbd50 0x7f557c0cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5581032640 1 --2- 172.21.15.31:0/4130830315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f557c15a710 0x7f557c0abf90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5581032640 1 -- 172.21.15.31:0/4130830315 >> 172.21.15.31:0/4130830315 conn(0x7f557c0b0cb0 msgr2=0x7f557c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5581032640 1 -- 172.21.15.31:0/4130830315 shutdown_connections 2023-11-26T07:23:55.986 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.980+0000 7f5581032640 1 -- 172.21.15.31:0/4130830315 wait complete. 2023-11-26T07:23:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f5581032640 1 Processor -- start 2023-11-26T07:23:55.987 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f5581032640 1 -- start start 2023-11-26T07:23:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f5581032640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f557c0cbd50 0x7f557c152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f5581032640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f557c0cdfb0 0x7f557c154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f5581032640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f557c15a710 0x7f557c1552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.989 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f5581032640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c09bf00 con 0x7f557c15a710 2023-11-26T07:23:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f5581032640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c153530 con 0x7f557c0cdfb0 2023-11-26T07:23:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f5581032640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557c154070 con 0x7f557c0cbd50 2023-11-26T07:23:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f557c15a710 0x7f557c1552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557b59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f557c15a710 0x7f557c1552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55828/0 (socket says 172.21.15.31:55828) 2023-11-26T07:23:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557b59e640 1 -- 172.21.15.31:0/3789247281 learned_addr learned my addr 172.21.15.31:0/3789247281 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557a59c640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f557c0cdfb0 0x7f557c154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.990 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557ad9d640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f557c0cbd50 0x7f557c152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557b59e640 1 -- 172.21.15.31:0/3789247281 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f557c0cbd50 msgr2=0x7f557c152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557b59e640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f557c0cbd50 0x7f557c152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557b59e640 1 -- 172.21.15.31:0/3789247281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f557c0cdfb0 msgr2=0x7f557c154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557b59e640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f557c0cdfb0 0x7f557c154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557b59e640 1 -- 172.21.15.31:0/3789247281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5574009a70 con 0x7f557c15a710 2023-11-26T07:23:55.991 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f557b59e640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f557c15a710 0x7f557c1552c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f557400efc0 tx=0x7f5574009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f55637fe640 1 -- 172.21.15.31:0/3789247281 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574009d70 con 0x7f557c15a710 2023-11-26T07:23:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f55637fe640 1 -- 172.21.15.31:0/3789247281 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557400f040 con 0x7f557c15a710 2023-11-26T07:23:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f55637fe640 1 -- 172.21.15.31:0/3789247281 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55740039a0 con 0x7f557c15a710 2023-11-26T07:23:55.993 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.984+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55700094d0 con 0x7f557c15a710 2023-11-26T07:23:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557c153a20 con 0x7f557c15a710 2023-11-26T07:23:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f557c06d380 con 0x7f557c15a710 2023-11-26T07:23:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f55637fe640 1 -- 172.21.15.31:0/3789247281 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f55740144a0 con 0x7f557c15a710 2023-11-26T07:23:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f55637fe640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5568066740 0x7f5568068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f55637fe640 1 -- 172.21.15.31:0/3789247281 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5574084da0 con 0x7f557c15a710 2023-11-26T07:23:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f55637fe640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] conn(0x7f556806c5a0 0x7f556806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:55.994 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f55637fe640 1 -- 172.21.15.31:0/3789247281 --> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f556806f070 con 0x7f556806c5a0 2023-11-26T07:23:55.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f557ad9d640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5568066740 0x7f5568068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f55637fe640 1 -- 172.21.15.31:0/3789247281 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f557408a3f0 con 0x7f557c15a710 2023-11-26T07:23:55.995 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f557a59c640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] conn(0x7f556806c5a0 0x7f556806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.988+0000 7f557ad9d640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5568066740 0x7f5568068c00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f556c000f70 tx=0x7f556c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.992+0000 7f557a59c640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] conn(0x7f556806c5a0 0x7f556806e9c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:55.996 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:55.992+0000 7f55637fe640 1 -- 172.21.15.31:0/3789247281 <== osd.0 v2:172.21.15.31:6816/4191585440 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f556806f070 con 0x7f556806c5a0 2023-11-26T07:23:56.007 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 --> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f557c0671f0 con 0x7f556806c5a0 2023-11-26T07:23:56.008 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f55637fe640 1 -- 172.21.15.31:0/3789247281 <== osd.0 v2:172.21.15.31:6816/4191585440 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f557c0671f0 con 0x7f556806c5a0 2023-11-26T07:23:56.008 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:56.008 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:23:56.009 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:56.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] conn(0x7f556806c5a0 msgr2=0x7f556806e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:6816/4191585440,v1:172.21.15.31:6817/4191585440] conn(0x7f556806c5a0 0x7f556806e9c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5568066740 msgr2=0x7f5568068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5568066740 0x7f5568068c00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f556c000f70 tx=0x7f556c009040 comp rx=0 tx=0).stop 2023-11-26T07:23:56.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f557c15a710 msgr2=0x7f557c1552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.010 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 --2- 172.21.15.31:0/3789247281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f557c15a710 0x7f557c1552c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f557400efc0 tx=0x7f5574009f90 comp rx=0 tx=0).stop 2023-11-26T07:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f557b59e640 1 -- 172.21.15.31:0/3789247281 reap_dead start 2023-11-26T07:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 shutdown_connections 2023-11-26T07:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 >> 172.21.15.31:0/3789247281 conn(0x7f557c0b0cb0 msgr2=0x7f557c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 shutdown_connections 2023-11-26T07:23:56.011 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.004+0000 7f5581032640 1 -- 172.21.15.31:0/3789247281 wait complete. 2023-11-26T07:23:56.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-26T07:23:56.171 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.168+0000 7f708d1e7640 1 Processor -- start 2023-11-26T07:23:56.172 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.168+0000 7f708d1e7640 1 -- start start 2023-11-26T07:23:56.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.168+0000 7f708d1e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f708806e3f0 0x7f708815f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.173 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.168+0000 7f708d1e7640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7088159630 con 0x7f70880cb9b0 2023-11-26T07:23:56.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.168+0000 7f708d1e7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708815f6d0 con 0x7f70880ce5d0 2023-11-26T07:23:56.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.168+0000 7f708d1e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708815bba0 con 0x7f708806e3f0 2023-11-26T07:23:56.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.168+0000 7f7086d9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f70880cb9b0 0x7f70880cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52112/0 (socket says 172.21.15.31:52112) 2023-11-26T07:23:56.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.168+0000 7f7086d9d640 1 -- 172.21.15.31:0/3746563109 learned_addr learned my addr 172.21.15.31:0/3746563109 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:56.174 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f708659c640 1 --2- 172.21.15.31:0/3746563109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f708806e3f0 0x7f708815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.175 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f7085d9b640 1 -- 172.21.15.31:0/3746563109 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483934612 0 0) 0x7f7088159630 con 0x7f70880cb9b0 2023-11-26T07:23:56.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f7085d9b640 1 -- 172.21.15.31:0/3746563109 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7070003530 con 0x7f70880cb9b0 2023-11-26T07:23:56.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f7085d9b640 1 -- 172.21.15.31:0/3746563109 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67025592 0 0) 0x7f708815f6d0 con 0x7f70880ce5d0 2023-11-26T07:23:56.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f7085d9b640 1 -- 172.21.15.31:0/3746563109 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7088159630 con 0x7f70880ce5d0 2023-11-26T07:23:56.176 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f7085d9b640 1 -- 172.21.15.31:0/3746563109 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 785454364 0 0) 0x7f7070003530 con 0x7f70880cb9b0 2023-11-26T07:23:56.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f708659c640 1 -- 172.21.15.31:0/3746563109 >> v1:172.21.15.97:6789/0 conn(0x7f70880ce5d0 legacy=0x7f708815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f708659c640 1 -- 172.21.15.31:0/3746563109 >> v1:172.21.15.31:6789/0 conn(0x7f70880cb9b0 legacy=0x7f70880cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.177 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f708659c640 1 -- 172.21.15.31:0/3746563109 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880cd1d0 con 0x7f708806e3f0 2023-11-26T07:23:56.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f708659c640 1 --2- 172.21.15.31:0/3746563109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f708806e3f0 0x7f708815f190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f707c0097f0 tx=0x7f707c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c2ff6e591362396 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.178 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f7085d9b640 1 -- 172.21.15.31:0/3746563109 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078001680 con 0x7f708806e3f0 2023-11-26T07:23:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f7085d9b640 1 -- 172.21.15.31:0/3746563109 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c0a7050 con 0x7f708806e3f0 2023-11-26T07:23:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.172+0000 7f7085d9b640 1 -- 172.21.15.31:0/3746563109 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0ab600 con 0x7f708806e3f0 2023-11-26T07:23:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 -- 172.21.15.31:0/3746563109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f708806e3f0 msgr2=0x7f708815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 --2- 172.21.15.31:0/3746563109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f708806e3f0 0x7f708815f190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f707c0097f0 tx=0x7f707c01c3e0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.179 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 -- 172.21.15.31:0/3746563109 shutdown_connections 2023-11-26T07:23:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 --2- 172.21.15.31:0/3746563109 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f708806e3f0 0x7f708815f190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 -- 172.21.15.31:0/3746563109 >> 172.21.15.31:0/3746563109 conn(0x7f70880b0cb0 msgr2=0x7f70880b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 -- 172.21.15.31:0/3746563109 shutdown_connections 2023-11-26T07:23:56.180 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 -- 172.21.15.31:0/3746563109 wait complete. 2023-11-26T07:23:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 Processor -- start 2023-11-26T07:23:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 -- start start 2023-11-26T07:23:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f708806e3f0 0x7f70880b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70880cb9b0 0x7f708806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.183 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70880ce5d0 0x7f70881646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70880ac110 con 0x7f70880ce5d0 2023-11-26T07:23:56.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70880ac3d0 con 0x7f708806e3f0 2023-11-26T07:23:56.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708d1e7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70880ac6b0 con 0x7f70880cb9b0 2023-11-26T07:23:56.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70880ce5d0 0x7f70881646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.184 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708759e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70880ce5d0 0x7f70881646f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55844/0 (socket says 172.21.15.31:55844) 2023-11-26T07:23:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708759e640 1 -- 172.21.15.31:0/2217720382 learned_addr learned my addr 172.21.15.31:0/2217720382 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f7086d9d640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f708806e3f0 0x7f70880b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708659c640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70880cb9b0 0x7f708806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708759e640 1 -- 172.21.15.31:0/2217720382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70880cb9b0 msgr2=0x7f708806c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.185 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.176+0000 7f708759e640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70880cb9b0 0x7f708806c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f708759e640 1 -- 172.21.15.31:0/2217720382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f708806e3f0 msgr2=0x7f70880b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f708759e640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f708806e3f0 0x7f70880b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f708759e640 1 -- 172.21.15.31:0/2217720382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c009420 con 0x7f70880ce5d0 2023-11-26T07:23:56.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f708659c640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70880cb9b0 0x7f708806c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:56.186 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f708759e640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70880ce5d0 0x7f70881646f0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f708000c730 tx=0x7f708000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f706f7fe640 1 -- 172.21.15.31:0/2217720382 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080003b80 con 0x7f70880ce5d0 2023-11-26T07:23:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f706f7fe640 1 -- 172.21.15.31:0/2217720382 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7080003ce0 con 0x7f70880ce5d0 2023-11-26T07:23:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f708d1e7640 1 -- 172.21.15.31:0/2217720382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70880ac990 con 0x7f70880ce5d0 2023-11-26T07:23:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f706f7fe640 1 -- 172.21.15.31:0/2217720382 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708000aa90 con 0x7f70880ce5d0 2023-11-26T07:23:56.187 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f708d1e7640 1 -- 172.21.15.31:0/2217720382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708806ce90 con 0x7f70880ce5d0 2023-11-26T07:23:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f708d1e7640 1 -- 172.21.15.31:0/2217720382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70880671f0 con 0x7f70880ce5d0 2023-11-26T07:23:56.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f706f7fe640 1 -- 172.21.15.31:0/2217720382 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f708000abf0 con 0x7f70880ce5d0 2023-11-26T07:23:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f706f7fe640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7074066740 0x7f7074068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f706f7fe640 1 -- 172.21.15.31:0/2217720382 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7080083d50 con 0x7f70880ce5d0 2023-11-26T07:23:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.180+0000 7f7086d9d640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7074066740 0x7f7074068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.189 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.184+0000 7f7086d9d640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7074066740 0x7f7074068c00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7078000f70 tx=0x7f70780016c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.190 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.184+0000 7f706f7fe640 1 -- 172.21.15.31:0/2217720382 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7080051180 con 0x7f70880ce5d0 2023-11-26T07:23:56.346 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 -- 172.21.15.31:0/2217720382 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f70880ccea0 con 0x7f7074066740 2023-11-26T07:23:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f706f7fe640 1 -- 172.21.15.31:0/2217720382 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f707800fa50 con 0x7f7074066740 2023-11-26T07:23:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 0 to deep-scrub 2023-11-26T07:23:56.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 -- 172.21.15.31:0/2217720382 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7074066740 msgr2=0x7f7074068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7074066740 0x7f7074068c00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7078000f70 tx=0x7f70780016c0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 -- 172.21.15.31:0/2217720382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70880ce5d0 msgr2=0x7f70881646f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70880ce5d0 0x7f70881646f0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f708000c730 tx=0x7f708000cbf0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 -- 172.21.15.31:0/2217720382 shutdown_connections 2023-11-26T07:23:56.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7074066740 0x7f7074068c00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70880ce5d0 0x7f70881646f0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f70880cb9b0 0x7f708806c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 --2- 172.21.15.31:0/2217720382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f708806e3f0 0x7f70880b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 -- 172.21.15.31:0/2217720382 >> 172.21.15.31:0/2217720382 conn(0x7f70880b0cb0 msgr2=0x7f70880b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:56.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 -- 172.21.15.31:0/2217720382 shutdown_connections 2023-11-26T07:23:56.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.344+0000 7f708d1e7640 1 -- 172.21.15.31:0/2217720382 wait complete. 2023-11-26T07:23:56.368 INFO:tasks.ceph:Scrubbing osd.1 2023-11-26T07:23:56.369 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:23:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff21e365640 1 Processor -- start 2023-11-26T07:23:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff21e365640 1 -- start start 2023-11-26T07:23:56.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff21e365640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ff2180cb9b0 0x7ff21806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff21e365640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff218159630 con 0x7ff2180ce5d0 2023-11-26T07:23:56.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff21e365640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2180cd1d0 con 0x7ff2180cbd90 2023-11-26T07:23:56.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff21e365640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff21815bba0 con 0x7ff2180cb9b0 2023-11-26T07:23:56.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff21c902640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ff2180ce5d0 0x7ff21815bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52120/0 (socket says 172.21.15.31:52120) 2023-11-26T07:23:56.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff21c902640 1 -- 172.21.15.31:0/4219715600 learned_addr learned my addr 172.21.15.31:0/4219715600 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:56.519 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff217fff640 1 --2- 172.21.15.31:0/4219715600 >> v2:172.21.15.97:3300/0 conn(0x7ff2180cb9b0 0x7ff21806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190036255 0 0) 0x7ff218159630 con 0x7ff2180ce5d0 2023-11-26T07:23:56.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff200003530 con 0x7ff2180ce5d0 2023-11-26T07:23:56.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452548198 0 0) 0x7ff2180cd1d0 con 0x7ff2180cbd90 2023-11-26T07:23:56.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff218159630 con 0x7ff2180cbd90 2023-11-26T07:23:56.520 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369607736 0 0) 0x7ff200003530 con 0x7ff2180ce5d0 2023-11-26T07:23:56.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 >> v2:172.21.15.97:3300/0 conn(0x7ff2180cb9b0 msgr2=0x7ff21806e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:56.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff216ffd640 1 --2- 172.21.15.31:0/4219715600 >> v2:172.21.15.97:3300/0 conn(0x7ff2180cb9b0 0x7ff21806e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 >> v1:172.21.15.97:6789/0 conn(0x7ff2180cbd90 legacy=0x7ff21815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.521 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2180cd4b0 con 0x7ff2180ce5d0 2023-11-26T07:23:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.512+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7ff210001a00 con 0x7ff2180ce5d0 2023-11-26T07:23:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 >> v1:172.21.15.31:6789/0 conn(0x7ff2180ce5d0 legacy=0x7ff21815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff216ffd640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff200003b10 0x7ff200003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff216ffd640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 0x7ff200004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff216ffd640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200005190 0x7ff20000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.522 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20000bfe0 con 0x7ff2000047d0 2023-11-26T07:23:56.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20000c150 con 0x7ff200005190 2023-11-26T07:23:56.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20000c310 con 0x7ff200003b10 2023-11-26T07:23:56.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff217fff640 1 --2- 172.21.15.31:0/4219715600 >> v2:172.21.15.97:3300/0 conn(0x7ff2180cb9b0 0x7ff21806e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-26T07:23:56.523 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff2177fe640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 0x7ff200004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff21c902640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200005190 0x7ff20000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff217fff640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff200003b10 0x7ff200003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff2177fe640 1 -- 172.21.15.31:0/4219715600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff200003b10 msgr2=0x7ff200003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:56.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff2177fe640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff200003b10 0x7ff200003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.524 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff2177fe640 1 -- 172.21.15.31:0/4219715600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200005190 msgr2=0x7ff20000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:56.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff2177fe640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff200005190 0x7ff20000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff2177fe640 1 -- 172.21.15.31:0/4219715600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff200003530 con 0x7ff2000047d0 2023-11-26T07:23:56.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff21c902640 1 -- 172.21.15.31:0/4219715600 reap_dead start 2023-11-26T07:23:56.525 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff217fff640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff200003b10 0x7ff200003f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:56.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff2177fe640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 0x7ff200004c50 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0018d0 tx=0x7ff20c01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682163032f52daec server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0052d0 con 0x7ff2000047d0 2023-11-26T07:23:56.528 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20c005450 con 0x7ff2000047d0 2023-11-26T07:23:56.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.516+0000 7ff216ffd640 1 -- 172.21.15.31:0/4219715600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0a8910 con 0x7ff2000047d0 2023-11-26T07:23:56.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 -- 172.21.15.31:0/4219715600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 msgr2=0x7ff200004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 0x7ff200004c50 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0018d0 tx=0x7ff20c01ecb0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 -- 172.21.15.31:0/4219715600 shutdown_connections 2023-11-26T07:23:56.529 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 --2- 172.21.15.31:0/4219715600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 0x7ff200004c50 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 -- 172.21.15.31:0/4219715600 >> 172.21.15.31:0/4219715600 conn(0x7ff2180b0cb0 msgr2=0x7ff2180b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 -- 172.21.15.31:0/4219715600 shutdown_connections 2023-11-26T07:23:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 -- 172.21.15.31:0/4219715600 wait complete. 2023-11-26T07:23:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 Processor -- start 2023-11-26T07:23:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 -- start start 2023-11-26T07:23:56.530 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 0x7ff2180abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff218157890 0x7ff2180ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180acee0 0x7ff2181f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180ad370 con 0x7ff2000047d0 2023-11-26T07:23:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181f6940 con 0x7ff2180acee0 2023-11-26T07:23:56.531 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff21e365640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181f6c20 con 0x7ff218157890 2023-11-26T07:23:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff217fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 0x7ff2180abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff217fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 0x7ff2180abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55870/0 (socket says 172.21.15.31:55870) 2023-11-26T07:23:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.520+0000 7ff217fff640 1 -- 172.21.15.31:0/2668445977 learned_addr learned my addr 172.21.15.31:0/2668445977 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff217fff640 1 -- 172.21.15.31:0/2668445977 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff218157890 msgr2=0x7ff2180ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.532 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff217fff640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff218157890 0x7ff2180ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff217fff640 1 -- 172.21.15.31:0/2668445977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180acee0 msgr2=0x7ff2181f31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff21c902640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180acee0 0x7ff2181f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff217fff640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180acee0 0x7ff2181f31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff217fff640 1 -- 172.21.15.31:0/2668445977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2181f6f00 con 0x7ff2000047d0 2023-11-26T07:23:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff21c902640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2180acee0 0x7ff2181f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:56.533 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff217fff640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 0x7ff2180abf90 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff208011830 tx=0x7ff208011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff214ff9640 1 -- 172.21.15.31:0/2668445977 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20800f9c0 con 0x7ff2000047d0 2023-11-26T07:23:56.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff214ff9640 1 -- 172.21.15.31:0/2668445977 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20800fb20 con 0x7ff2000047d0 2023-11-26T07:23:56.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff214ff9640 1 -- 172.21.15.31:0/2668445977 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20800fe90 con 0x7ff2000047d0 2023-11-26T07:23:56.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2181f71e0 con 0x7ff2000047d0 2023-11-26T07:23:56.534 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21815d5a0 con 0x7ff2000047d0 2023-11-26T07:23:56.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff2180cfa50 con 0x7ff2000047d0 2023-11-26T07:23:56.535 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff214ff9640 1 -- 172.21.15.31:0/2668445977 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ff208015670 con 0x7ff2000047d0 2023-11-26T07:23:56.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff214ff9640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff204066740 0x7ff204068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff214ff9640 1 -- 172.21.15.31:0/2668445977 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff208084b40 con 0x7ff2000047d0 2023-11-26T07:23:56.536 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff2177fe640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff204066740 0x7ff204068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.524+0000 7ff214ff9640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] conn(0x7ff20406c5a0 0x7ff20406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.528+0000 7ff214ff9640 1 -- 172.21.15.31:0/2668445977 --> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff20406f070 con 0x7ff20406c5a0 2023-11-26T07:23:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.528+0000 7ff214ff9640 1 -- 172.21.15.31:0/2668445977 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2080892a0 con 0x7ff2000047d0 2023-11-26T07:23:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.528+0000 7ff21c902640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] conn(0x7ff20406c5a0 0x7ff20406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.537 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.528+0000 7ff2177fe640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff204066740 0x7ff204068c00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0b0000 tx=0x7ff20c0012e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.528+0000 7ff21c902640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] conn(0x7ff20406c5a0 0x7ff20406e9c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.538 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.528+0000 7ff214ff9640 1 -- 172.21.15.31:0/2668445977 <== osd.1 v2:172.21.15.31:6800/845128234 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff20406f070 con 0x7ff20406c5a0 2023-11-26T07:23:56.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.540+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 --> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff2180ccea0 con 0x7ff20406c5a0 2023-11-26T07:23:56.545 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.540+0000 7ff214ff9640 1 -- 172.21.15.31:0/2668445977 <== osd.1 v2:172.21.15.31:6800/845128234 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff210004b70 con 0x7ff20406c5a0 2023-11-26T07:23:56.545 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:23:56.546 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:56.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] conn(0x7ff20406c5a0 msgr2=0x7ff20406e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21e365640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:6800/845128234,v1:172.21.15.31:6801/845128234] conn(0x7ff20406c5a0 0x7ff20406e9c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.547 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff204066740 msgr2=0x7ff204068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21e365640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff204066740 0x7ff204068c00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0b0000 tx=0x7ff20c0012e0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 msgr2=0x7ff2180abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21e365640 1 --2- 172.21.15.31:0/2668445977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2000047d0 0x7ff2180abf90 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7ff208011830 tx=0x7ff208011cf0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21c902640 1 -- 172.21.15.31:0/2668445977 reap_dead start 2023-11-26T07:23:56.548 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 shutdown_connections 2023-11-26T07:23:56.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 >> 172.21.15.31:0/2668445977 conn(0x7ff2180b0cb0 msgr2=0x7ff21815b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:56.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 shutdown_connections 2023-11-26T07:23:56.549 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.544+0000 7ff21e365640 1 -- 172.21.15.31:0/2668445977 wait complete. 2023-11-26T07:23:56.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-26T07:23:56.710 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.704+0000 7fc12205a640 1 Processor -- start 2023-11-26T07:23:56.711 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.704+0000 7fc12205a640 1 -- start start 2023-11-26T07:23:56.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.704+0000 7fc12205a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c0cc460 0x7fc11c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.704+0000 7fc12205a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c0b2710 con 0x7fc11c0cdfb0 2023-11-26T07:23:56.712 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.704+0000 7fc12205a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c0b29f0 con 0x7fc11c15a710 2023-11-26T07:23:56.713 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.704+0000 7fc12205a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0cca80 con 0x7fc11c0cc460 2023-11-26T07:23:56.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11b7fe640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc11c0cdfb0 0x7fc11c0ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52124/0 (socket says 172.21.15.31:52124) 2023-11-26T07:23:56.715 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11b7fe640 1 -- 172.21.15.31:0/492355743 learned_addr learned my addr 172.21.15.31:0/492355743 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:56.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11bfff640 1 --2- 172.21.15.31:0/492355743 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c0cc460 0x7fc11c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.716 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11a7fc640 1 -- 172.21.15.31:0/492355743 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804437077 0 0) 0x7fc11c0b2710 con 0x7fc11c0cdfb0 2023-11-26T07:23:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11a7fc640 1 -- 172.21.15.31:0/492355743 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc104003530 con 0x7fc11c0cdfb0 2023-11-26T07:23:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11a7fc640 1 -- 172.21.15.31:0/492355743 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272315012 0 0) 0x7fc11c0b29f0 con 0x7fc11c15a710 2023-11-26T07:23:56.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11a7fc640 1 -- 172.21.15.31:0/492355743 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc11c0b2710 con 0x7fc11c15a710 2023-11-26T07:23:56.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11bfff640 1 -- 172.21.15.31:0/492355743 >> v1:172.21.15.97:6789/0 conn(0x7fc11c15a710 legacy=0x7fc11c0cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.718 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11a7fc640 1 -- 172.21.15.31:0/492355743 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3829481213 0 0) 0x7fc104003530 con 0x7fc11c0cdfb0 2023-11-26T07:23:56.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11bfff640 1 -- 172.21.15.31:0/492355743 >> v1:172.21.15.31:6789/0 conn(0x7fc11c0cdfb0 legacy=0x7fc11c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.719 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11bfff640 1 -- 172.21.15.31:0/492355743 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c0b2cd0 con 0x7fc11c0cc460 2023-11-26T07:23:56.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.708+0000 7fc11bfff640 1 --2- 172.21.15.31:0/492355743 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c0cc460 0x7fc11c0abf90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc114009dc0 tx=0x7fc11401c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a56c4bf43c4a397 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc11a7fc640 1 -- 172.21.15.31:0/492355743 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c001680 con 0x7fc11c0cc460 2023-11-26T07:23:56.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc11a7fc640 1 -- 172.21.15.31:0/492355743 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1140a56c0 con 0x7fc11c0cc460 2023-11-26T07:23:56.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc11a7fc640 1 -- 172.21.15.31:0/492355743 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140a5a30 con 0x7fc11c0cc460 2023-11-26T07:23:56.720 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 -- 172.21.15.31:0/492355743 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c0cc460 msgr2=0x7fc11c0abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 --2- 172.21.15.31:0/492355743 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c0cc460 0x7fc11c0abf90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc114009dc0 tx=0x7fc11401c860 comp rx=0 tx=0).stop 2023-11-26T07:23:56.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 -- 172.21.15.31:0/492355743 shutdown_connections 2023-11-26T07:23:56.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 --2- 172.21.15.31:0/492355743 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c0cc460 0x7fc11c0abf90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 -- 172.21.15.31:0/492355743 >> 172.21.15.31:0/492355743 conn(0x7fc11c0b0cb0 msgr2=0x7fc11c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:56.721 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 -- 172.21.15.31:0/492355743 shutdown_connections 2023-11-26T07:23:56.722 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 -- 172.21.15.31:0/492355743 wait complete. 2023-11-26T07:23:56.725 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 Processor -- start 2023-11-26T07:23:56.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 -- start start 2023-11-26T07:23:56.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc11c0cc460 0x7fc11c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.726 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11c0cdfb0 0x7fc11c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c15a710 0x7fc11c155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.727 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c15d650 con 0x7fc11c0cc460 2023-11-26T07:23:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c15c7a0 con 0x7fc11c0cdfb0 2023-11-26T07:23:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.712+0000 7fc12205a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c15da70 con 0x7fc11c15a710 2023-11-26T07:23:56.728 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc11c0cc460 0x7fc11c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc11c0cc460 0x7fc11c157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55872/0 (socket says 172.21.15.31:55872) 2023-11-26T07:23:56.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11b7fe640 1 -- 172.21.15.31:0/2150105273 learned_addr learned my addr 172.21.15.31:0/2150105273 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:56.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11bfff640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c15a710 0x7fc11c155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11affd640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11c0cdfb0 0x7fc11c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.730 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11b7fe640 1 -- 172.21.15.31:0/2150105273 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c15a710 msgr2=0x7fc11c155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11b7fe640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c15a710 0x7fc11c155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11b7fe640 1 -- 172.21.15.31:0/2150105273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11c0cdfb0 msgr2=0x7fc11c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11b7fe640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11c0cdfb0 0x7fc11c15c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11b7fe640 1 -- 172.21.15.31:0/2150105273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc114009a70 con 0x7fc11c0cc460 2023-11-26T07:23:56.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc11b7fe640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc11c0cc460 0x7fc11c157e00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c780 tx=0x7fc10c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc103fff640 1 -- 172.21.15.31:0/2150105273 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c005cf0 con 0x7fc11c0cc460 2023-11-26T07:23:56.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc103fff640 1 -- 172.21.15.31:0/2150105273 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c005e50 con 0x7fc11c0cc460 2023-11-26T07:23:56.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc103fff640 1 -- 172.21.15.31:0/2150105273 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c007270 con 0x7fc11c0cc460 2023-11-26T07:23:56.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc12205a640 1 -- 172.21.15.31:0/2150105273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c1562e0 con 0x7fc11c0cc460 2023-11-26T07:23:56.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc12205a640 1 -- 172.21.15.31:0/2150105273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c15ccb0 con 0x7fc11c0cc460 2023-11-26T07:23:56.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc12205a640 1 -- 172.21.15.31:0/2150105273 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c0671f0 con 0x7fc11c0cc460 2023-11-26T07:23:56.733 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.716+0000 7fc103fff640 1 -- 172.21.15.31:0/2150105273 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fc10c0073d0 con 0x7fc11c0cc460 2023-11-26T07:23:56.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.720+0000 7fc103fff640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc108066740 0x7fc108068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:56.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.720+0000 7fc103fff640 1 -- 172.21.15.31:0/2150105273 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc10c083690 con 0x7fc11c0cc460 2023-11-26T07:23:56.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.720+0000 7fc11affd640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc108066740 0x7fc108068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:56.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.720+0000 7fc11affd640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc108066740 0x7fc108068c00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc11c165680 tx=0x7fc110009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:56.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.724+0000 7fc103fff640 1 -- 172.21.15.31:0/2150105273 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc10c050bc0 con 0x7fc11c0cc460 2023-11-26T07:23:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 -- 172.21.15.31:0/2150105273 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc11c15d790 con 0x7fc108066740 2023-11-26T07:23:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc103fff640 1 -- 172.21.15.31:0/2150105273 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc11c15d790 con 0x7fc108066740 2023-11-26T07:23:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 1 to deep-scrub 2023-11-26T07:23:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 -- 172.21.15.31:0/2150105273 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc108066740 msgr2=0x7fc108068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc108066740 0x7fc108068c00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc11c165680 tx=0x7fc110009040 comp rx=0 tx=0).stop 2023-11-26T07:23:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 -- 172.21.15.31:0/2150105273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc11c0cc460 msgr2=0x7fc11c157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc11c0cc460 0x7fc11c157e00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c780 tx=0x7fc10c00cc40 comp rx=0 tx=0).stop 2023-11-26T07:23:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 -- 172.21.15.31:0/2150105273 shutdown_connections 2023-11-26T07:23:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fc108066740 0x7fc108068c00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fc11c15a710 0x7fc11c155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.885 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc11c0cdfb0 0x7fc11c15c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 --2- 172.21.15.31:0/2150105273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc11c0cc460 0x7fc11c157e00 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 -- 172.21.15.31:0/2150105273 >> 172.21.15.31:0/2150105273 conn(0x7fc11c0b0cb0 msgr2=0x7fc11c0b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 -- 172.21.15.31:0/2150105273 shutdown_connections 2023-11-26T07:23:56.886 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:56.880+0000 7fc12205a640 1 -- 172.21.15.31:0/2150105273 wait complete. 2023-11-26T07:23:56.904 INFO:tasks.ceph:Scrubbing osd.2 2023-11-26T07:23:56.904 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:23:57.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.048+0000 7f0a95b35640 1 Processor -- start 2023-11-26T07:23:57.052 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.048+0000 7f0a95b35640 1 -- start start 2023-11-26T07:23:57.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.048+0000 7f0a95b35640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a900cdfb0 0x7f0a900abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.048+0000 7f0a95b35640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a900cbd50 0x7f0a900cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.048+0000 7f0a95b35640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a900b2710 con 0x7f0a9015a710 2023-11-26T07:23:57.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.048+0000 7f0a95b35640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a900cca80 con 0x7f0a900cbd50 2023-11-26T07:23:57.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.048+0000 7f0a95b35640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900b29f0 con 0x7f0a900cdfb0 2023-11-26T07:23:57.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a900cbd50 0x7f0a900cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.054 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8ffff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0a900cbd50 0x7f0a900cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55880/0 (socket says 172.21.15.31:55880) 2023-11-26T07:23:57.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8effd640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f0a9015a710 0x7f0a900ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52128/0 (socket says 172.21.15.31:52128) 2023-11-26T07:23:57.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8ffff640 1 -- 172.21.15.31:0/3899215130 learned_addr learned my addr 172.21.15.31:0/3899215130 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:57.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8f7fe640 1 --2- 172.21.15.31:0/3899215130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a900cdfb0 0x7f0a900abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.055 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8e7fc640 1 -- 172.21.15.31:0/3899215130 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328293060 0 0) 0x7f0a900b2710 con 0x7f0a9015a710 2023-11-26T07:23:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8ffff640 1 -- 172.21.15.31:0/3899215130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a900cdfb0 msgr2=0x7f0a900abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8ffff640 1 --2- 172.21.15.31:0/3899215130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a900cdfb0 0x7f0a900abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8ffff640 1 -- 172.21.15.31:0/3899215130 >> v1:172.21.15.31:6789/0 conn(0x7f0a9015a710 legacy=0x7f0a900ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.056 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8ffff640 1 -- 172.21.15.31:0/3899215130 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a900b2cd0 con 0x7f0a900cbd50 2023-11-26T07:23:57.057 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8ffff640 1 --2- 172.21.15.31:0/3899215130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a900cbd50 0x7f0a900cc1e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009e50 tx=0x7f0a8801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1923dc0e1df96322 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8e7fc640 1 -- 172.21.15.31:0/3899215130 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a880a58d0 con 0x7f0a900cbd50 2023-11-26T07:23:57.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8e7fc640 1 -- 172.21.15.31:0/3899215130 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a880a5a30 con 0x7f0a900cbd50 2023-11-26T07:23:57.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.052+0000 7f0a8e7fc640 1 -- 172.21.15.31:0/3899215130 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a880a5dc0 con 0x7f0a900cbd50 2023-11-26T07:23:57.058 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 -- 172.21.15.31:0/3899215130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a900cbd50 msgr2=0x7f0a900cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 --2- 172.21.15.31:0/3899215130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a900cbd50 0x7f0a900cc1e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009e50 tx=0x7f0a8801c9e0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 -- 172.21.15.31:0/3899215130 shutdown_connections 2023-11-26T07:23:57.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 --2- 172.21.15.31:0/3899215130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a900cbd50 0x7f0a900cc1e0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 --2- 172.21.15.31:0/3899215130 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a900cdfb0 0x7f0a900abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 -- 172.21.15.31:0/3899215130 >> 172.21.15.31:0/3899215130 conn(0x7f0a900b0cb0 msgr2=0x7f0a900b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:57.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 -- 172.21.15.31:0/3899215130 shutdown_connections 2023-11-26T07:23:57.059 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 -- 172.21.15.31:0/3899215130 wait complete. 2023-11-26T07:23:57.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 Processor -- start 2023-11-26T07:23:57.060 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 -- start start 2023-11-26T07:23:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a900cbd50 0x7f0a90151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a900cdfb0 0x7f0a9015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.062 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a9015a710 0x7f0a9015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9015ab40 con 0x7f0a9015a710 2023-11-26T07:23:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a90155150 con 0x7f0a900cbd50 2023-11-26T07:23:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a95b35640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9015db10 con 0x7f0a900cdfb0 2023-11-26T07:23:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a8ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a9015a710 0x7f0a9015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a8ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a9015a710 0x7f0a9015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55882/0 (socket says 172.21.15.31:55882) 2023-11-26T07:23:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a8ffff640 1 -- 172.21.15.31:0/54530323 learned_addr learned my addr 172.21.15.31:0/54530323 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a8effd640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a900cdfb0 0x7f0a9015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.063 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a8f7fe640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a900cbd50 0x7f0a90151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a8ffff640 1 -- 172.21.15.31:0/54530323 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a900cdfb0 msgr2=0x7f0a9015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a8ffff640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f0a900cdfb0 0x7f0a9015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a8ffff640 1 -- 172.21.15.31:0/54530323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a900cbd50 msgr2=0x7f0a90151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.056+0000 7f0a8ffff640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a900cbd50 0x7f0a90151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a8ffff640 1 -- 172.21.15.31:0/54530323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a88009b30 con 0x7f0a9015a710 2023-11-26T07:23:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a8ffff640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a9015a710 0x7f0a9015d5d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0a880af040 tx=0x7f0a88019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.064 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a77fff640 1 -- 172.21.15.31:0/54530323 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a880b8670 con 0x7f0a9015a710 2023-11-26T07:23:57.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a77fff640 1 -- 172.21.15.31:0/54530323 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a880b87f0 con 0x7f0a9015a710 2023-11-26T07:23:57.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a77fff640 1 -- 172.21.15.31:0/54530323 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a880b8ba0 con 0x7f0a9015a710 2023-11-26T07:23:57.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a901553f0 con 0x7f0a9015a710 2023-11-26T07:23:57.065 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9015b000 con 0x7f0a9015a710 2023-11-26T07:23:57.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a9006b460 con 0x7f0a9015a710 2023-11-26T07:23:57.066 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a77fff640 1 -- 172.21.15.31:0/54530323 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f0a880b8d00 con 0x7f0a9015a710 2023-11-26T07:23:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a77fff640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0a7c066790 0x7f0a7c068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a77fff640 1 -- 172.21.15.31:0/54530323 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0a8811ea50 con 0x7f0a9015a710 2023-11-26T07:23:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a8f7fe640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0a7c066790 0x7f0a7c068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a77fff640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] conn(0x7f0a7c06c5f0 0x7f0a7c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a77fff640 1 -- 172.21.15.31:0/54530323 --> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0a7c06f0c0 con 0x7f0a7c06c5f0 2023-11-26T07:23:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.060+0000 7f0a77fff640 1 -- 172.21.15.31:0/54530323 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a881232a0 con 0x7f0a9015a710 2023-11-26T07:23:57.067 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.064+0000 7f0a8effd640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] conn(0x7f0a7c06c5f0 0x7f0a7c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.064+0000 7f0a8f7fe640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0a7c066790 0x7f0a7c068c50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009af0 tx=0x7f0a80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.064+0000 7f0a8effd640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] conn(0x7f0a7c06c5f0 0x7f0a7c06ea10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.068 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.064+0000 7f0a77fff640 1 -- 172.21.15.31:0/54530323 <== osd.2 v2:172.21.15.31:6802/2371417753 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0a7c06f0c0 con 0x7f0a7c06c5f0 2023-11-26T07:23:57.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 --> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0a90155580 con 0x7f0a7c06c5f0 2023-11-26T07:23:57.080 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a77fff640 1 -- 172.21.15.31:0/54530323 <== osd.2 v2:172.21.15.31:6802/2371417753 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0a90155580 con 0x7f0a7c06c5f0 2023-11-26T07:23:57.081 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:23:57.081 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:57.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] conn(0x7f0a7c06c5f0 msgr2=0x7f0a7c06ea10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.082 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a95b35640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:6802/2371417753,v1:172.21.15.31:6804/2371417753] conn(0x7f0a7c06c5f0 0x7f0a7c06ea10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0a7c066790 msgr2=0x7f0a7c068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a95b35640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f0a7c066790 0x7f0a7c068c50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009af0 tx=0x7f0a80008040 comp rx=0 tx=0).stop 2023-11-26T07:23:57.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a9015a710 msgr2=0x7f0a9015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a95b35640 1 --2- 172.21.15.31:0/54530323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a9015a710 0x7f0a9015d5d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0a880af040 tx=0x7f0a88019780 comp rx=0 tx=0).stop 2023-11-26T07:23:57.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a8ffff640 1 -- 172.21.15.31:0/54530323 reap_dead start 2023-11-26T07:23:57.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 shutdown_connections 2023-11-26T07:23:57.083 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 >> 172.21.15.31:0/54530323 conn(0x7f0a900b0cb0 msgr2=0x7f0a900ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:57.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.076+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 shutdown_connections 2023-11-26T07:23:57.084 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.080+0000 7f0a95b35640 1 -- 172.21.15.31:0/54530323 wait complete. 2023-11-26T07:23:57.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-26T07:23:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffab0ad6640 1 Processor -- start 2023-11-26T07:23:57.243 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffab0ad6640 1 -- start start 2023-11-26T07:23:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffab0ad6640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7ffaac15a710 0x7ffaac0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffab0ad6640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac0b2710 con 0x7ffaac0cdfb0 2023-11-26T07:23:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffab0ad6640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac0b29f0 con 0x7ffaac0cbd50 2023-11-26T07:23:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffab0ad6640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac0cca80 con 0x7ffaac15a710 2023-11-26T07:23:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffaaa59c640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7ffaac0cdfb0 0x7ffaac0ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52140/0 (socket says 172.21.15.31:52140) 2023-11-26T07:23:57.245 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffaaa59c640 1 -- 172.21.15.31:0/1124913294 learned_addr learned my addr 172.21.15.31:0/1124913294 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:57.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164622501 0 0) 0x7ffaac0b2710 con 0x7ffaac0cdfb0 2023-11-26T07:23:57.247 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa98005740 con 0x7ffaac0cdfb0 2023-11-26T07:23:57.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.240+0000 7ffaa3fff640 1 --2- 172.21.15.31:0/1124913294 >> v2:172.21.15.97:3300/0 conn(0x7ffaac15a710 0x7ffaac0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2939667552 0 0) 0x7ffa98005740 con 0x7ffaac0cdfb0 2023-11-26T07:23:57.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 >> v2:172.21.15.97:3300/0 conn(0x7ffaac15a710 msgr2=0x7ffaac0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:57.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 --2- 172.21.15.31:0/1124913294 >> v2:172.21.15.97:3300/0 conn(0x7ffaac15a710 0x7ffaac0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 >> v1:172.21.15.97:6789/0 conn(0x7ffaac0cbd50 legacy=0x7ffaac0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0b2cd0 con 0x7ffaac0cdfb0 2023-11-26T07:23:57.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 <== mon.0 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3125681778 0 0) 0x7ffa9c001680 con 0x7ffaac0cdfb0 2023-11-26T07:23:57.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa3fff640 1 --2- 172.21.15.31:0/1124913294 >> v2:172.21.15.97:3300/0 conn(0x7ffaac15a710 0x7ffaac0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:57.248 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 >> v1:172.21.15.31:6789/0 conn(0x7ffaac0cdfb0 legacy=0x7ffaac0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa98005be0 0x7ffa98006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa98006540 0x7ffa98006a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa98007200 0x7ffa9800ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac0cca80 con 0x7ffa98007200 2023-11-26T07:23:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa98010540 con 0x7ffa98005be0 2023-11-26T07:23:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa98010700 con 0x7ffa98006540 2023-11-26T07:23:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaaad9d640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa98007200 0x7ffa9800ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.249 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaa3fff640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa98006540 0x7ffa98006a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaaa59c640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa98005be0 0x7ffa98006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaaad9d640 1 -- 172.21.15.31:0/1124913294 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa98006540 msgr2=0x7ffa98006a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaaad9d640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa98006540 0x7ffa98006a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaaad9d640 1 -- 172.21.15.31:0/1124913294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa98005be0 msgr2=0x7ffa98006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaaad9d640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa98005be0 0x7ffa98006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.250 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaaad9d640 1 -- 172.21.15.31:0/1124913294 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa98005740 con 0x7ffa98007200 2023-11-26T07:23:57.251 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaaad9d640 1 -- 172.21.15.31:0/1124913294 reap_dead start 2023-11-26T07:23:57.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.244+0000 7ffaaad9d640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa98007200 0x7ffa9800ffa0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4001e70 tx=0x7ffaa401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc13d3e6833d2b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4003b80 con 0x7ffa98007200 2023-11-26T07:23:57.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa4003d20 con 0x7ffa98007200 2023-11-26T07:23:57.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffaa9d9b640 1 -- 172.21.15.31:0/1124913294 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4005260 con 0x7ffa98007200 2023-11-26T07:23:57.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1124913294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa98007200 msgr2=0x7ffa9800ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa98007200 0x7ffa9800ffa0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4001e70 tx=0x7ffaa401c760 comp rx=0 tx=0).stop 2023-11-26T07:23:57.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1124913294 shutdown_connections 2023-11-26T07:23:57.252 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 --2- 172.21.15.31:0/1124913294 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa98007200 0x7ffa9800ffa0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1124913294 >> 172.21.15.31:0/1124913294 conn(0x7ffaac0b0cb0 msgr2=0x7ffaac0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:57.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1124913294 shutdown_connections 2023-11-26T07:23:57.253 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1124913294 wait complete. 2023-11-26T07:23:57.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 Processor -- start 2023-11-26T07:23:57.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 -- start start 2023-11-26T07:23:57.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa98007200 0x7ffaac1ee600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaac070690 0x7ffaac1578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.255 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaac09b280 0x7ffaac155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac071730 con 0x7ffaac09b280 2023-11-26T07:23:57.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac070b20 con 0x7ffaac070690 2023-11-26T07:23:57.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.248+0000 7ffab0ad6640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac070e00 con 0x7ffa98007200 2023-11-26T07:23:57.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaaad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaac09b280 0x7ffaac155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaaad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaac09b280 0x7ffaac155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55906/0 (socket says 172.21.15.31:55906) 2023-11-26T07:23:57.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaaad9d640 1 -- 172.21.15.31:0/1428310450 learned_addr learned my addr 172.21.15.31:0/1428310450 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:57.256 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaa3fff640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaac070690 0x7ffaac1578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaaa59c640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa98007200 0x7ffaac1ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaaad9d640 1 -- 172.21.15.31:0/1428310450 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa98007200 msgr2=0x7ffaac1ee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaaad9d640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa98007200 0x7ffaac1ee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaaad9d640 1 -- 172.21.15.31:0/1428310450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaac070690 msgr2=0x7ffaac1578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaaad9d640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaac070690 0x7ffaac1578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaaad9d640 1 -- 172.21.15.31:0/1428310450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac0710e0 con 0x7ffaac09b280 2023-11-26T07:23:57.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaaad9d640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaac09b280 0x7ffaac155060 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ffaa401c590 tx=0x7ffa98007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.257 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaa2ffd640 1 -- 172.21.15.31:0/1428310450 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa40ac070 con 0x7ffaac09b280 2023-11-26T07:23:57.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaa2ffd640 1 -- 172.21.15.31:0/1428310450 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa4007dc0 con 0x7ffaac09b280 2023-11-26T07:23:57.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaa2ffd640 1 -- 172.21.15.31:0/1428310450 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4006d60 con 0x7ffaac09b280 2023-11-26T07:23:57.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1428310450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac0713c0 con 0x7ffaac09b280 2023-11-26T07:23:57.258 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1428310450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac157df0 con 0x7ffaac09b280 2023-11-26T07:23:57.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1428310450 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac0671f0 con 0x7ffaac09b280 2023-11-26T07:23:57.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.252+0000 7ffaa2ffd640 1 -- 172.21.15.31:0/1428310450 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ffaa4007030 con 0x7ffaac09b280 2023-11-26T07:23:57.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.256+0000 7ffaa2ffd640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffa8c066770 0x7ffa8c068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.259 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.256+0000 7ffaa2ffd640 1 -- 172.21.15.31:0/1428310450 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffaa411ce10 con 0x7ffaac09b280 2023-11-26T07:23:57.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.256+0000 7ffaaa59c640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffa8c066770 0x7ffa8c068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.260 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.256+0000 7ffaaa59c640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffa8c066770 0x7ffa8c068c30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c001bd0 tx=0x7ffa9c000ee0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.262 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.260+0000 7ffaa2ffd640 1 -- 172.21.15.31:0/1428310450 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffaa40ea1e0 con 0x7ffaac09b280 2023-11-26T07:23:57.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.416+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1428310450 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7ffaac14ef10 con 0x7ffa8c066770 2023-11-26T07:23:57.420 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.416+0000 7ffaa2ffd640 1 -- 172.21.15.31:0/1428310450 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffaac14ef10 con 0x7ffa8c066770 2023-11-26T07:23:57.421 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 2 to deep-scrub 2023-11-26T07:23:57.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1428310450 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffa8c066770 msgr2=0x7ffa8c068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.422 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffa8c066770 0x7ffa8c068c30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c001bd0 tx=0x7ffa9c000ee0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1428310450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaac09b280 msgr2=0x7ffaac155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaac09b280 0x7ffaac155060 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ffaa401c590 tx=0x7ffa98007a10 comp rx=0 tx=0).stop 2023-11-26T07:23:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1428310450 shutdown_connections 2023-11-26T07:23:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ffa8c066770 0x7ffa8c068c30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffaac09b280 0x7ffaac155060 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.423 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaac070690 0x7ffaac1578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 --2- 172.21.15.31:0/1428310450 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ffa98007200 0x7ffaac1ee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1428310450 >> 172.21.15.31:0/1428310450 conn(0x7ffaac0b0cb0 msgr2=0x7ffaac0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:57.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1428310450 shutdown_connections 2023-11-26T07:23:57.424 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.420+0000 7ffab0ad6640 1 -- 172.21.15.31:0/1428310450 wait complete. 2023-11-26T07:23:57.442 INFO:tasks.ceph:Scrubbing osd.3 2023-11-26T07:23:57.442 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:23:57.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.584+0000 7f5a5218a640 1 Processor -- start 2023-11-26T07:23:57.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a5218a640 1 -- start start 2023-11-26T07:23:57.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a5218a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5a4c06c250 0x7f5a4c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a5218a640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4c06cbf0 con 0x7f5a4c0b3340 2023-11-26T07:23:57.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a5218a640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4c15ef10 con 0x7f5a4c143e50 2023-11-26T07:23:57.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a5218a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4c0cecf0 con 0x7f5a4c06c250 2023-11-26T07:23:57.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a51989640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f5a4c0b3340 0x7f5a4c0b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52156/0 (socket says 172.21.15.31:52156) 2023-11-26T07:23:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a51989640 1 -- 172.21.15.31:0/2829931669 learned_addr learned my addr 172.21.15.31:0/2829931669 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a50987640 1 --2- 172.21.15.31:0/2829931669 >> v2:172.21.15.97:3300/0 conn(0x7f5a4c06c250 0x7f5a4c06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a43fff640 1 -- 172.21.15.31:0/2829931669 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948750483 0 0) 0x7f5a4c06cbf0 con 0x7f5a4c0b3340 2023-11-26T07:23:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a43fff640 1 -- 172.21.15.31:0/2829931669 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a38003530 con 0x7f5a4c0b3340 2023-11-26T07:23:57.593 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a43fff640 1 -- 172.21.15.31:0/2829931669 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232760576 0 0) 0x7f5a38003530 con 0x7f5a4c0b3340 2023-11-26T07:23:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a50987640 1 -- 172.21.15.31:0/2829931669 >> v1:172.21.15.97:6789/0 conn(0x7f5a4c143e50 legacy=0x7f5a4c1444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a50987640 1 -- 172.21.15.31:0/2829931669 >> v1:172.21.15.31:6789/0 conn(0x7f5a4c0b3340 legacy=0x7f5a4c0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.594 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.588+0000 7f5a50987640 1 -- 172.21.15.31:0/2829931669 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c15f1f0 con 0x7f5a4c06c250 2023-11-26T07:23:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a50987640 1 --2- 172.21.15.31:0/2829931669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c06c250 0x7f5a4c06c6b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0097c0 tx=0x7f5a3c01c190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4fb92190c7f570 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.595 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a43fff640 1 -- 172.21.15.31:0/2829931669 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a44001a00 con 0x7f5a4c06c250 2023-11-26T07:23:57.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a43fff640 1 -- 172.21.15.31:0/2829931669 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c0a7050 con 0x7f5a4c06c250 2023-11-26T07:23:57.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a43fff640 1 -- 172.21.15.31:0/2829931669 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c0ab600 con 0x7f5a4c06c250 2023-11-26T07:23:57.596 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a5218a640 1 -- 172.21.15.31:0/2829931669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c06c250 msgr2=0x7f5a4c06c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a5218a640 1 --2- 172.21.15.31:0/2829931669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c06c250 0x7f5a4c06c6b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0097c0 tx=0x7f5a3c01c190 comp rx=0 tx=0).stop 2023-11-26T07:23:57.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a5218a640 1 -- 172.21.15.31:0/2829931669 shutdown_connections 2023-11-26T07:23:57.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a5218a640 1 --2- 172.21.15.31:0/2829931669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c06c250 0x7f5a4c06c6b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a5218a640 1 -- 172.21.15.31:0/2829931669 >> 172.21.15.31:0/2829931669 conn(0x7f5a4c06df30 msgr2=0x7f5a4c06a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:57.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a5218a640 1 -- 172.21.15.31:0/2829931669 shutdown_connections 2023-11-26T07:23:57.597 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.592+0000 7f5a5218a640 1 -- 172.21.15.31:0/2829931669 wait complete. 2023-11-26T07:23:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 Processor -- start 2023-11-26T07:23:57.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 -- start start 2023-11-26T07:23:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c06c250 0x7f5a4c15b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c0b3340 0x7f5a4c1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a4c143e50 0x7f5a4c1355e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c159660 con 0x7f5a4c06c250 2023-11-26T07:23:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c15be70 con 0x7f5a4c0b3340 2023-11-26T07:23:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c15b1a0 con 0x7f5a4c143e50 2023-11-26T07:23:57.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a51188640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c06c250 0x7f5a4c15b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a51188640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c06c250 0x7f5a4c15b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55920/0 (socket says 172.21.15.31:55920) 2023-11-26T07:23:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a51188640 1 -- 172.21.15.31:0/596693711 learned_addr learned my addr 172.21.15.31:0/596693711 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a50987640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c0b3340 0x7f5a4c1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a51989640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a4c143e50 0x7f5a4c1355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a51188640 1 -- 172.21.15.31:0/596693711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a4c143e50 msgr2=0x7f5a4c1355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a51188640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f5a4c143e50 0x7f5a4c1355e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a51188640 1 -- 172.21.15.31:0/596693711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c0b3340 msgr2=0x7f5a4c1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a51188640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a4c0b3340 0x7f5a4c1574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a51188640 1 -- 172.21.15.31:0/596693711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c009420 con 0x7f5a4c06c250 2023-11-26T07:23:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a51188640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c06c250 0x7f5a4c15b930 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800c500 tx=0x7f5a4800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a41ffb640 1 -- 172.21.15.31:0/596693711 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48003600 con 0x7f5a4c06c250 2023-11-26T07:23:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a41ffb640 1 -- 172.21.15.31:0/596693711 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a48003760 con 0x7f5a4c06c250 2023-11-26T07:23:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c1598d0 con 0x7f5a4c06c250 2023-11-26T07:23:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a41ffb640 1 -- 172.21.15.31:0/596693711 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48003b10 con 0x7f5a4c06c250 2023-11-26T07:23:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c136db0 con 0x7f5a4c06c250 2023-11-26T07:23:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.596+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a4c15b7b0 con 0x7f5a4c06c250 2023-11-26T07:23:57.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a41ffb640 1 -- 172.21.15.31:0/596693711 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f5a48003c70 con 0x7f5a4c06c250 2023-11-26T07:23:57.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a41ffb640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5a30066740 0x7f5a30068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.604 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a41ffb640 1 -- 172.21.15.31:0/596693711 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5a480836f0 con 0x7f5a4c06c250 2023-11-26T07:23:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a50987640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5a30066740 0x7f5a30068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a41ffb640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] conn(0x7f5a3006c5a0 0x7f5a3006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a41ffb640 1 -- 172.21.15.31:0/596693711 --> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a3006f070 con 0x7f5a3006c5a0 2023-11-26T07:23:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a41ffb640 1 -- 172.21.15.31:0/596693711 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a48085cd0 con 0x7f5a4c06c250 2023-11-26T07:23:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a51989640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] conn(0x7f5a3006c5a0 0x7f5a3006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a50987640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5a30066740 0x7f5a30068c00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0ad000 tx=0x7f5a3c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.605 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a51989640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] conn(0x7f5a3006c5a0 0x7f5a3006e9c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.606 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.600+0000 7f5a41ffb640 1 -- 172.21.15.31:0/596693711 <== osd.3 v2:172.21.15.31:6824/3143140069 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5a3006f070 con 0x7f5a3006c5a0 2023-11-26T07:23:57.618 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 --> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5a4c144230 con 0x7f5a3006c5a0 2023-11-26T07:23:57.619 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a41ffb640 1 -- 172.21.15.31:0/596693711 <== osd.3 v2:172.21.15.31:6824/3143140069 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5a44004b70 con 0x7f5a3006c5a0 2023-11-26T07:23:57.619 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:57.619 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:23:57.619 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:57.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] conn(0x7f5a3006c5a0 msgr2=0x7f5a3006e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:6824/3143140069,v1:172.21.15.31:6825/3143140069] conn(0x7f5a3006c5a0 0x7f5a3006e9c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.620 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5a30066740 msgr2=0x7f5a30068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f5a30066740 0x7f5a30068c00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0ad000 tx=0x7f5a3c0a6000 comp rx=0 tx=0).stop 2023-11-26T07:23:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c06c250 msgr2=0x7f5a4c15b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 --2- 172.21.15.31:0/596693711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a4c06c250 0x7f5a4c15b930 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800c500 tx=0x7f5a4800c9c0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a51989640 1 -- 172.21.15.31:0/596693711 reap_dead start 2023-11-26T07:23:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 shutdown_connections 2023-11-26T07:23:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 >> 172.21.15.31:0/596693711 conn(0x7f5a4c06df30 msgr2=0x7f5a4c0adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 shutdown_connections 2023-11-26T07:23:57.621 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.616+0000 7f5a5218a640 1 -- 172.21.15.31:0/596693711 wait complete. 2023-11-26T07:23:57.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-26T07:23:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f515eefe640 1 Processor -- start 2023-11-26T07:23:57.782 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f515eefe640 1 -- start start 2023-11-26T07:23:57.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f515eefe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f5158144d50 0x7f51580b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f515eefe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51580b5160 0x7f515815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.783 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f515eefe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515815fcb0 con 0x7f515815f420 2023-11-26T07:23:57.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f515eefe640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515815f8b0 con 0x7f5158144d50 2023-11-26T07:23:57.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f515eefe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51580b5790 con 0x7f51580b5160 2023-11-26T07:23:57.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f515d49b640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f515815f420 0x7f5158070040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52164/0 (socket says 172.21.15.31:52164) 2023-11-26T07:23:57.784 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f515d49b640 1 -- 172.21.15.31:0/620793606 learned_addr learned my addr 172.21.15.31:0/620793606 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:57.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f5157fff640 1 --2- 172.21.15.31:0/620793606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51580b5160 0x7f515815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f515cc9a640 1 --2- 172.21.15.31:0/620793606 >> v2:172.21.15.97:3300/0 conn(0x7f5158144d50 0x7f51580b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f51577fe640 1 -- 172.21.15.31:0/620793606 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497712096 0 0) 0x7f515815fcb0 con 0x7f515815f420 2023-11-26T07:23:57.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f51577fe640 1 -- 172.21.15.31:0/620793606 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5148003530 con 0x7f515815f420 2023-11-26T07:23:57.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f5157fff640 1 -- 172.21.15.31:0/620793606 >> v2:172.21.15.97:3300/0 conn(0x7f5158144d50 msgr2=0x7f51580b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:57.785 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f5157fff640 1 --2- 172.21.15.31:0/620793606 >> v2:172.21.15.97:3300/0 conn(0x7f5158144d50 0x7f51580b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f5157fff640 1 -- 172.21.15.31:0/620793606 >> v1:172.21.15.31:6789/0 conn(0x7f515815f420 legacy=0x7f5158070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.786 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.780+0000 7f5157fff640 1 -- 172.21.15.31:0/620793606 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580b5a70 con 0x7f51580b5160 2023-11-26T07:23:57.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f5157fff640 1 --2- 172.21.15.31:0/620793606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51580b5160 0x7f515815eee0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f51440097c0 tx=0x7f514401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7345a2537433651f server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f51577fe640 1 -- 172.21.15.31:0/620793606 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440ab600 con 0x7f51580b5160 2023-11-26T07:23:57.787 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f51577fe640 1 -- 172.21.15.31:0/620793606 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51440a7050 con 0x7f51580b5160 2023-11-26T07:23:57.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f51577fe640 1 -- 172.21.15.31:0/620793606 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440ab980 con 0x7f51580b5160 2023-11-26T07:23:57.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f515eefe640 1 -- 172.21.15.31:0/620793606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51580b5160 msgr2=0x7f515815eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.788 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f515eefe640 1 --2- 172.21.15.31:0/620793606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51580b5160 0x7f515815eee0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f51440097c0 tx=0x7f514401c480 comp rx=0 tx=0).stop 2023-11-26T07:23:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f515eefe640 1 -- 172.21.15.31:0/620793606 shutdown_connections 2023-11-26T07:23:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f515eefe640 1 --2- 172.21.15.31:0/620793606 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f51580b5160 0x7f515815eee0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f515eefe640 1 --2- 172.21.15.31:0/620793606 >> v2:172.21.15.97:3300/0 conn(0x7f5158144d50 0x7f51580b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f515eefe640 1 -- 172.21.15.31:0/620793606 >> 172.21.15.31:0/620793606 conn(0x7f51580b2f10 msgr2=0x7f51580b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f515eefe640 1 -- 172.21.15.31:0/620793606 shutdown_connections 2023-11-26T07:23:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.784+0000 7f515eefe640 1 -- 172.21.15.31:0/620793606 wait complete. 2023-11-26T07:23:57.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 Processor -- start 2023-11-26T07:23:57.790 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 -- start start 2023-11-26T07:23:57.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51580b5160 0x7f5158165b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5158144d50 0x7f5158138bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.791 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f515815f420 0x7f5158165080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5158164c60 con 0x7f5158144d50 2023-11-26T07:23:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515806c7d0 con 0x7f51580b5160 2023-11-26T07:23:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51581660b0 con 0x7f515815f420 2023-11-26T07:23:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f5157fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5158144d50 0x7f5158138bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f5157fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5158144d50 0x7f5158138bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55922/0 (socket says 172.21.15.31:55922) 2023-11-26T07:23:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f5157fff640 1 -- 172.21.15.31:0/1981356879 learned_addr learned my addr 172.21.15.31:0/1981356879 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515cc9a640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51580b5160 0x7f5158165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.792 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515d49b640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f515815f420 0x7f5158165080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f5157fff640 1 -- 172.21.15.31:0/1981356879 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f515815f420 msgr2=0x7f5158165080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f5157fff640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f515815f420 0x7f5158165080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f5157fff640 1 -- 172.21.15.31:0/1981356879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51580b5160 msgr2=0x7f5158165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f5157fff640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51580b5160 0x7f5158165b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f5157fff640 1 -- 172.21.15.31:0/1981356879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5144009420 con 0x7f5158144d50 2023-11-26T07:23:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f5157fff640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5158144d50 0x7f5158138bb0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f51440ae040 tx=0x7f5144019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f51557fa640 1 -- 172.21.15.31:0/1981356879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440b4430 con 0x7f5158144d50 2023-11-26T07:23:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f51557fa640 1 -- 172.21.15.31:0/1981356879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51440b45b0 con 0x7f5158144d50 2023-11-26T07:23:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 -- 172.21.15.31:0/1981356879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515806d4e0 con 0x7f5158144d50 2023-11-26T07:23:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f51557fa640 1 -- 172.21.15.31:0/1981356879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440b4960 con 0x7f5158144d50 2023-11-26T07:23:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 -- 172.21.15.31:0/1981356879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5158164880 con 0x7f5158144d50 2023-11-26T07:23:57.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.788+0000 7f515eefe640 1 -- 172.21.15.31:0/1981356879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5158164da0 con 0x7f5158144d50 2023-11-26T07:23:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.792+0000 7f51557fa640 1 -- 172.21.15.31:0/1981356879 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f51440b2040 con 0x7f5158144d50 2023-11-26T07:23:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.792+0000 7f51557fa640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f513c066740 0x7f513c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.792+0000 7f51557fa640 1 -- 172.21.15.31:0/1981356879 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f514411df20 con 0x7f5158144d50 2023-11-26T07:23:57.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.792+0000 7f515cc9a640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f513c066740 0x7f513c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:57.797 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.792+0000 7f515cc9a640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f513c066740 0x7f513c068c00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f514c009af0 tx=0x7f514c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:57.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.796+0000 7f51557fa640 1 -- 172.21.15.31:0/1981356879 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5158164da0 con 0x7f5158144d50 2023-11-26T07:23:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 -- 172.21.15.31:0/1981356879 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5158168280 con 0x7f513c066740 2023-11-26T07:23:57.955 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f51557fa640 1 -- 172.21.15.31:0/1981356879 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f514c0119b0 con 0x7f513c066740 2023-11-26T07:23:57.956 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 3 to deep-scrub 2023-11-26T07:23:57.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 -- 172.21.15.31:0/1981356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f513c066740 msgr2=0x7f513c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f513c066740 0x7f513c068c00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f514c009af0 tx=0x7f514c008040 comp rx=0 tx=0).stop 2023-11-26T07:23:57.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 -- 172.21.15.31:0/1981356879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5158144d50 msgr2=0x7f5158138bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:57.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5158144d50 0x7f5158138bb0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f51440ae040 tx=0x7f5144019220 comp rx=0 tx=0).stop 2023-11-26T07:23:57.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 -- 172.21.15.31:0/1981356879 shutdown_connections 2023-11-26T07:23:57.958 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f513c066740 0x7f513c068c00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f515815f420 0x7f5158165080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5158144d50 0x7f5158138bb0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 --2- 172.21.15.31:0/1981356879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51580b5160 0x7f5158165b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.952+0000 7f515eefe640 1 -- 172.21.15.31:0/1981356879 >> 172.21.15.31:0/1981356879 conn(0x7f51580b2f10 msgr2=0x7f5158157ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.956+0000 7f515eefe640 1 -- 172.21.15.31:0/1981356879 shutdown_connections 2023-11-26T07:23:57.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:57.956+0000 7f515eefe640 1 -- 172.21.15.31:0/1981356879 wait complete. 2023-11-26T07:23:57.977 INFO:tasks.ceph:Scrubbing osd.4 2023-11-26T07:23:57.977 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:23:58.124 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.120+0000 7ff8fb59e640 1 Processor -- start 2023-11-26T07:23:58.125 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.120+0000 7ff8fb59e640 1 -- start start 2023-11-26T07:23:58.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fb59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8fc146b90 0x7ff8fc0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fb59e640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc06f510 con 0x7ff8fc06ef60 2023-11-26T07:23:58.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fb59e640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc0707f0 con 0x7ff8fc0b4480 2023-11-26T07:23:58.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fb59e640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc0b4910 con 0x7ff8fc146b90 2023-11-26T07:23:58.126 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fa59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8fc146b90 0x7ff8fc0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fa59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8fc146b90 0x7ff8fc0b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55936/0 (socket says 172.21.15.31:55936) 2023-11-26T07:23:58.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fa59c640 1 -- 172.21.15.31:0/1155598419 learned_addr learned my addr 172.21.15.31:0/1155598419 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:58.127 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fa59c640 1 -- 172.21.15.31:0/1155598419 >> v1:172.21.15.97:6789/0 conn(0x7ff8fc0b4480 legacy=0x7ff8fc06e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:58.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fa59c640 1 -- 172.21.15.31:0/1155598419 >> v1:172.21.15.31:6789/0 conn(0x7ff8fc06ef60 legacy=0x7ff8fc06b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fa59c640 1 -- 172.21.15.31:0/1155598419 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc146f70 con 0x7ff8fc146b90 2023-11-26T07:23:58.128 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8f9d9b640 1 -- 172.21.15.31:0/1155598419 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907080057 0 0) 0x7ff8fc06f510 con 0x7ff8fc06ef60 2023-11-26T07:23:58.129 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.124+0000 7ff8fa59c640 1 --2- 172.21.15.31:0/1155598419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8fc146b90 0x7ff8fc0b3f40 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009870 tx=0x7ff8ec01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66266c04b06337d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8f9d9b640 1 -- 172.21.15.31:0/1155598419 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0ab640 con 0x7ff8fc146b90 2023-11-26T07:23:58.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8f9d9b640 1 -- 172.21.15.31:0/1155598419 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec0a7070 con 0x7ff8fc146b90 2023-11-26T07:23:58.130 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8f9d9b640 1 -- 172.21.15.31:0/1155598419 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0ab9c0 con 0x7ff8fc146b90 2023-11-26T07:23:58.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 -- 172.21.15.31:0/1155598419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8fc146b90 msgr2=0x7ff8fc0b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 --2- 172.21.15.31:0/1155598419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8fc146b90 0x7ff8fc0b3f40 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009870 tx=0x7ff8ec01c440 comp rx=0 tx=0).stop 2023-11-26T07:23:58.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 -- 172.21.15.31:0/1155598419 shutdown_connections 2023-11-26T07:23:58.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 --2- 172.21.15.31:0/1155598419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8fc146b90 0x7ff8fc0b3f40 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 -- 172.21.15.31:0/1155598419 >> 172.21.15.31:0/1155598419 conn(0x7ff8fc0d0fb0 msgr2=0x7ff8fc0d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:58.131 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 -- 172.21.15.31:0/1155598419 shutdown_connections 2023-11-26T07:23:58.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 -- 172.21.15.31:0/1155598419 wait complete. 2023-11-26T07:23:58.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 Processor -- start 2023-11-26T07:23:58.132 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 -- start start 2023-11-26T07:23:58.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc06ef60 0x7ff8fc15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.133 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8fc0b4480 0x7ff8fc155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8fc146b90 0x7ff8fc155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc156100 con 0x7ff8fc146b90 2023-11-26T07:23:58.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc13ccd0 con 0x7ff8fc06ef60 2023-11-26T07:23:58.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fb59e640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc13cfb0 con 0x7ff8fc0b4480 2023-11-26T07:23:58.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8fc146b90 0x7ff8fc155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8fc146b90 0x7ff8fc155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55946/0 (socket says 172.21.15.31:55946) 2023-11-26T07:23:58.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.128+0000 7ff8fad9d640 1 -- 172.21.15.31:0/869532157 learned_addr learned my addr 172.21.15.31:0/869532157 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:58.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8fa59c640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc06ef60 0x7ff8fc15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.135 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8f3fff640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8fc0b4480 0x7ff8fc155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8fad9d640 1 -- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8fc0b4480 msgr2=0x7ff8fc155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8fad9d640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8fc0b4480 0x7ff8fc155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8fad9d640 1 -- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc06ef60 msgr2=0x7ff8fc15c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8fad9d640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff8fc06ef60 0x7ff8fc15c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.136 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8fad9d640 1 -- 172.21.15.31:0/869532157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec0094d0 con 0x7ff8fc146b90 2023-11-26T07:23:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8f3fff640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7ff8fc0b4480 0x7ff8fc155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8fad9d640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8fc146b90 0x7ff8fc155bc0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4001570 tx=0x7ff8f400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8f2ffd640 1 -- 172.21.15.31:0/869532157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4003db0 con 0x7ff8fc146b90 2023-11-26T07:23:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8f2ffd640 1 -- 172.21.15.31:0/869532157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f4005070 con 0x7ff8fc146b90 2023-11-26T07:23:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8f2ffd640 1 -- 172.21.15.31:0/869532157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4005420 con 0x7ff8fc146b90 2023-11-26T07:23:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc13d290 con 0x7ff8fc146b90 2023-11-26T07:23:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc13d850 con 0x7ff8fc146b90 2023-11-26T07:23:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8fc070c80 con 0x7ff8fc146b90 2023-11-26T07:23:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8f2ffd640 1 -- 172.21.15.31:0/869532157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7ff8f4005580 con 0x7ff8fc146b90 2023-11-26T07:23:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8f2ffd640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff8dc066770 0x7ff8dc068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.132+0000 7ff8f2ffd640 1 -- 172.21.15.31:0/869532157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff8f4083730 con 0x7ff8fc146b90 2023-11-26T07:23:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.136+0000 7ff8fa59c640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff8dc066770 0x7ff8dc068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.136+0000 7ff8f2ffd640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] conn(0x7ff8dc06c5d0 0x7ff8dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.136+0000 7ff8f2ffd640 1 -- 172.21.15.31:0/869532157 --> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8dc06f0a0 con 0x7ff8dc06c5d0 2023-11-26T07:23:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.136+0000 7ff8f2ffd640 1 -- 172.21.15.31:0/869532157 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8f4085d70 con 0x7ff8fc146b90 2023-11-26T07:23:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.136+0000 7ff8f3fff640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] conn(0x7ff8dc06c5d0 0x7ff8dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.136+0000 7ff8fa59c640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff8dc066770 0x7ff8dc068c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0af000 tx=0x7ff8ec0b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.136+0000 7ff8f3fff640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] conn(0x7ff8dc06c5d0 0x7ff8dc06e9f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.136+0000 7ff8f2ffd640 1 -- 172.21.15.31:0/869532157 <== osd.4 v2:172.21.15.97:6809/362016202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff8dc06f0a0 con 0x7ff8dc06c5d0 2023-11-26T07:23:58.152 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.148+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 --> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff8fc074990 con 0x7ff8dc06c5d0 2023-11-26T07:23:58.153 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.148+0000 7ff8f2ffd640 1 -- 172.21.15.31:0/869532157 <== osd.4 v2:172.21.15.97:6809/362016202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff8fc074990 con 0x7ff8dc06c5d0 2023-11-26T07:23:58.153 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:58.154 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:23:58.154 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.148+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] conn(0x7ff8dc06c5d0 msgr2=0x7ff8dc06e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.148+0000 7ff8fb59e640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.97:6809/362016202,v1:172.21.15.97:6812/362016202] conn(0x7ff8dc06c5d0 0x7ff8dc06e9f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.152+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff8dc066770 msgr2=0x7ff8dc068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.152+0000 7ff8fb59e640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7ff8dc066770 0x7ff8dc068c30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec0af000 tx=0x7ff8ec0b6040 comp rx=0 tx=0).stop 2023-11-26T07:23:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.152+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8fc146b90 msgr2=0x7ff8fc155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.155 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.152+0000 7ff8fb59e640 1 --2- 172.21.15.31:0/869532157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8fc146b90 0x7ff8fc155bc0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4001570 tx=0x7ff8f400cd20 comp rx=0 tx=0).stop 2023-11-26T07:23:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.152+0000 7ff8fad9d640 1 -- 172.21.15.31:0/869532157 reap_dead start 2023-11-26T07:23:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.152+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 shutdown_connections 2023-11-26T07:23:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.152+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 >> 172.21.15.31:0/869532157 conn(0x7ff8fc0d0fb0 msgr2=0x7ff8fc13ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.152+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 shutdown_connections 2023-11-26T07:23:58.156 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.152+0000 7ff8fb59e640 1 -- 172.21.15.31:0/869532157 wait complete. 2023-11-26T07:23:58.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-26T07:23:58.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.312+0000 7f7de3806640 1 Processor -- start 2023-11-26T07:23:58.316 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.312+0000 7f7de3806640 1 -- start start 2023-11-26T07:23:58.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.312+0000 7f7de3806640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f7ddc0cb9b0 0x7f7ddc06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.312+0000 7f7de3806640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ddc0ce5d0 0x7f7ddc15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.312+0000 7f7de3806640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc159630 con 0x7f7ddc0cbd90 2023-11-26T07:23:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.312+0000 7f7de3806640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc15bba0 con 0x7f7ddc0cb9b0 2023-11-26T07:23:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.312+0000 7f7de3806640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cdd50 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.312+0000 7f7de0da1640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f7ddc0cbd90 0x7f7ddc15f190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52180/0 (socket says 172.21.15.31:52180) 2023-11-26T07:23:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7de0da1640 1 -- 172.21.15.31:0/1858049744 learned_addr learned my addr 172.21.15.31:0/1858049744 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:58.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7de15a2640 1 --2- 172.21.15.31:0/1858049744 >> v2:172.21.15.97:3300/0 conn(0x7f7ddc0cb9b0 0x7f7ddc06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7de1da3640 1 --2- 172.21.15.31:0/1858049744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ddc0ce5d0 0x7f7ddc15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7de15a2640 1 -- 172.21.15.31:0/1858049744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ddc0ce5d0 msgr2=0x7f7ddc15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7de15a2640 1 --2- 172.21.15.31:0/1858049744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ddc0ce5d0 0x7f7ddc15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7de15a2640 1 -- 172.21.15.31:0/1858049744 >> v1:172.21.15.31:6789/0 conn(0x7f7ddc0cbd90 legacy=0x7f7ddc15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7dd3fff640 1 -- 172.21.15.31:0/1858049744 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674150319 0 0) 0x7f7ddc159630 con 0x7f7ddc0cbd90 2023-11-26T07:23:58.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7de15a2640 1 -- 172.21.15.31:0/1858049744 --> v2:172.21.15.97:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0cd1d0 con 0x7f7ddc0cb9b0 2023-11-26T07:23:58.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7de15a2640 1 --2- 172.21.15.31:0/1858049744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc0cb9b0 0x7f7ddc06e3f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80097c0 tx=0x7f7dd801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7bffb353b0e7c2c server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7dd3fff640 1 -- 172.21.15.31:0/1858049744 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80a7070 con 0x7f7ddc0cb9b0 2023-11-26T07:23:58.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7dd3fff640 1 -- 172.21.15.31:0/1858049744 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd80ab430 con 0x7f7ddc0cb9b0 2023-11-26T07:23:58.321 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.316+0000 7f7dd3fff640 1 -- 172.21.15.31:0/1858049744 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd80ab7c0 con 0x7f7ddc0cb9b0 2023-11-26T07:23:58.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 -- 172.21.15.31:0/1858049744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc0cb9b0 msgr2=0x7f7ddc06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.322 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 --2- 172.21.15.31:0/1858049744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc0cb9b0 0x7f7ddc06e3f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80097c0 tx=0x7f7dd801c450 comp rx=0 tx=0).stop 2023-11-26T07:23:58.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 -- 172.21.15.31:0/1858049744 shutdown_connections 2023-11-26T07:23:58.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 --2- 172.21.15.31:0/1858049744 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ddc0ce5d0 0x7f7ddc15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 --2- 172.21.15.31:0/1858049744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc0cb9b0 0x7f7ddc06e3f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 -- 172.21.15.31:0/1858049744 >> 172.21.15.31:0/1858049744 conn(0x7f7ddc0b0cb0 msgr2=0x7f7ddc0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:58.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 -- 172.21.15.31:0/1858049744 shutdown_connections 2023-11-26T07:23:58.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 -- 172.21.15.31:0/1858049744 wait complete. 2023-11-26T07:23:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 Processor -- start 2023-11-26T07:23:58.324 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 -- start start 2023-11-26T07:23:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc0cb9b0 0x7f7ddc150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.325 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ddc0cbd90 0x7f7ddc150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ddc0ce5d0 0x7f7ddc06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0b37b0 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc150640 con 0x7f7ddc0cb9b0 2023-11-26T07:23:58.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de3806640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc06c820 con 0x7f7ddc0cbd90 2023-11-26T07:23:58.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de1da3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ddc0ce5d0 0x7f7ddc06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de1da3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ddc0ce5d0 0x7f7ddc06cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55960/0 (socket says 172.21.15.31:55960) 2023-11-26T07:23:58.326 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de1da3640 1 -- 172.21.15.31:0/290853414 learned_addr learned my addr 172.21.15.31:0/290853414 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de15a2640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc0cb9b0 0x7f7ddc150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.320+0000 7f7de0da1640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ddc0cbd90 0x7f7ddc150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7de1da3640 1 -- 172.21.15.31:0/290853414 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ddc0cbd90 msgr2=0x7f7ddc150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7de1da3640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ddc0cbd90 0x7f7ddc150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7de1da3640 1 -- 172.21.15.31:0/290853414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc0cb9b0 msgr2=0x7f7ddc150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7de1da3640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc0cb9b0 0x7f7ddc150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.327 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7de1da3640 1 -- 172.21.15.31:0/290853414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8009420 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.328 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7de1da3640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ddc0ce5d0 0x7f7ddc06cd00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400efc0 tx=0x7f7dd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7dd1ffb640 1 -- 172.21.15.31:0/290853414 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4009ba0 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7dd1ffb640 1 -- 172.21.15.31:0/290853414 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd400f040 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7dd1ffb640 1 -- 172.21.15.31:0/290853414 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40146c0 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7de3806640 1 -- 172.21.15.31:0/290853414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc0b40a0 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7de3806640 1 -- 172.21.15.31:0/290853414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc0b3dc0 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7de3806640 1 -- 172.21.15.31:0/290853414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ddc0671f0 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.329 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.324+0000 7f7dd1ffb640 1 -- 172.21.15.31:0/290853414 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f7dd4014820 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.328+0000 7f7dd1ffb640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dc0066740 0x7f7dc0068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.328+0000 7f7dd1ffb640 1 -- 172.21.15.31:0/290853414 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7dd4084a10 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.330 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.328+0000 7f7de15a2640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dc0066740 0x7f7dc0068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.328+0000 7f7de15a2640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dc0066740 0x7f7dc0068c00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80af000 tx=0x7f7dd80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.332+0000 7f7dd1ffb640 1 -- 172.21.15.31:0/290853414 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7dd4051e70 con 0x7f7ddc0ce5d0 2023-11-26T07:23:58.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.488+0000 7f7de3806640 1 -- 172.21.15.31:0/290853414 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ddc0ccea0 con 0x7f7dc0066740 2023-11-26T07:23:58.493 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.488+0000 7f7dd1ffb640 1 -- 172.21.15.31:0/290853414 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7dd8004300 con 0x7f7dc0066740 2023-11-26T07:23:58.493 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 4 to deep-scrub 2023-11-26T07:23:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 -- 172.21.15.31:0/290853414 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dc0066740 msgr2=0x7f7dc0068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dc0066740 0x7f7dc0068c00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80af000 tx=0x7f7dd80a7000 comp rx=0 tx=0).stop 2023-11-26T07:23:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 -- 172.21.15.31:0/290853414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ddc0ce5d0 msgr2=0x7f7ddc06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ddc0ce5d0 0x7f7ddc06cd00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400efc0 tx=0x7f7dd400c490 comp rx=0 tx=0).stop 2023-11-26T07:23:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 -- 172.21.15.31:0/290853414 shutdown_connections 2023-11-26T07:23:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f7dc0066740 0x7f7dc0068c00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.496 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ddc0ce5d0 0x7f7ddc06cd00 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f7ddc0cbd90 0x7f7ddc150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 --2- 172.21.15.31:0/290853414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ddc0cb9b0 0x7f7ddc150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 -- 172.21.15.31:0/290853414 >> 172.21.15.31:0/290853414 conn(0x7f7ddc0b0cb0 msgr2=0x7f7ddc06e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 -- 172.21.15.31:0/290853414 shutdown_connections 2023-11-26T07:23:58.497 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.492+0000 7f7de3806640 1 -- 172.21.15.31:0/290853414 wait complete. 2023-11-26T07:23:58.514 INFO:tasks.ceph:Scrubbing osd.5 2023-11-26T07:23:58.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:23:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.656+0000 7fcfb25d1640 1 Processor -- start 2023-11-26T07:23:58.661 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.656+0000 7fcfb25d1640 1 -- start start 2023-11-26T07:23:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfb25d1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 0x7fcfac06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfb25d1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac159630 con 0x7fcfac0ce5d0 2023-11-26T07:23:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfb25d1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac0cd1d0 con 0x7fcfac0cbd90 2023-11-26T07:23:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfb25d1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac15bba0 con 0x7fcfac0cb9b0 2023-11-26T07:23:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfb0b6e640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fcfac0ce5d0 0x7fcfac15bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52190/0 (socket says 172.21.15.31:52190) 2023-11-26T07:23:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfb0b6e640 1 -- 172.21.15.31:0/1179125262 learned_addr learned my addr 172.21.15.31:0/1179125262 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:23:58.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfabfff640 1 --2- 172.21.15.31:0/1179125262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 0x7fcfac06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfaaffd640 1 -- 172.21.15.31:0/1179125262 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631897211 0 0) 0x7fcfac159630 con 0x7fcfac0ce5d0 2023-11-26T07:23:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfaaffd640 1 -- 172.21.15.31:0/1179125262 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf94003530 con 0x7fcfac0ce5d0 2023-11-26T07:23:58.664 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfaaffd640 1 -- 172.21.15.31:0/1179125262 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371804277 0 0) 0x7fcfac0cd1d0 con 0x7fcfac0cbd90 2023-11-26T07:23:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfaaffd640 1 -- 172.21.15.31:0/1179125262 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfac159630 con 0x7fcfac0cbd90 2023-11-26T07:23:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfabfff640 1 -- 172.21.15.31:0/1179125262 >> v1:172.21.15.97:6789/0 conn(0x7fcfac0cbd90 legacy=0x7fcfac15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfabfff640 1 -- 172.21.15.31:0/1179125262 >> v1:172.21.15.31:6789/0 conn(0x7fcfac0ce5d0 legacy=0x7fcfac15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfaaffd640 1 -- 172.21.15.31:0/1179125262 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1956630427 0 0) 0x7fcf94003530 con 0x7fcfac0ce5d0 2023-11-26T07:23:58.665 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.660+0000 7fcfabfff640 1 -- 172.21.15.31:0/1179125262 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac0cd4b0 con 0x7fcfac0cb9b0 2023-11-26T07:23:58.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfabfff640 1 --2- 172.21.15.31:0/1179125262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 0x7fcfac06e3f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0097f0 tx=0x7fcf9c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48cbcb89d4118383 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfaaffd640 1 -- 172.21.15.31:0/1179125262 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4001a00 con 0x7fcfac0cb9b0 2023-11-26T07:23:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfaaffd640 1 -- 172.21.15.31:0/1179125262 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c0a7050 con 0x7fcfac0cb9b0 2023-11-26T07:23:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfaaffd640 1 -- 172.21.15.31:0/1179125262 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0ab620 con 0x7fcfac0cb9b0 2023-11-26T07:23:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfb25d1640 1 -- 172.21.15.31:0/1179125262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 msgr2=0x7fcfac06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfb25d1640 1 --2- 172.21.15.31:0/1179125262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 0x7fcfac06e3f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0097f0 tx=0x7fcf9c01c3e0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfb25d1640 1 -- 172.21.15.31:0/1179125262 shutdown_connections 2023-11-26T07:23:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfb25d1640 1 --2- 172.21.15.31:0/1179125262 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 0x7fcfac06e3f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfb25d1640 1 -- 172.21.15.31:0/1179125262 >> 172.21.15.31:0/1179125262 conn(0x7fcfac0b0cb0 msgr2=0x7fcfac0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:58.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfb25d1640 1 -- 172.21.15.31:0/1179125262 shutdown_connections 2023-11-26T07:23:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfb25d1640 1 -- 172.21.15.31:0/1179125262 wait complete. 2023-11-26T07:23:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfb25d1640 1 Processor -- start 2023-11-26T07:23:58.669 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.664+0000 7fcfb25d1640 1 -- start start 2023-11-26T07:23:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb25d1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 0x7fcfac156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb25d1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac0cbd90 0x7fcfac1510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb25d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfac0ce5d0 0x7fcfac1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb25d1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac155d00 con 0x7fcfac0cbd90 2023-11-26T07:23:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb25d1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac1572a0 con 0x7fcfac0ce5d0 2023-11-26T07:23:58.671 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb25d1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac0ac0e0 con 0x7fcfac0cb9b0 2023-11-26T07:23:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac0cbd90 0x7fcfac1510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfab7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac0cbd90 0x7fcfac1510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55962/0 (socket says 172.21.15.31:55962) 2023-11-26T07:23:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfab7fe640 1 -- 172.21.15.31:0/3653653675 learned_addr learned my addr 172.21.15.31:0/3653653675 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfabfff640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 0x7fcfac156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb0b6e640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfac0ce5d0 0x7fcfac1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfab7fe640 1 -- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 msgr2=0x7fcfac156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfab7fe640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 0x7fcfac156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.672 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfab7fe640 1 -- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfac0ce5d0 msgr2=0x7fcfac1518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfab7fe640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfac0ce5d0 0x7fcfac1518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfab7fe640 1 -- 172.21.15.31:0/3653653675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c009420 con 0x7fcfac0cbd90 2023-11-26T07:23:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfabfff640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcfac0cb9b0 0x7fcfac156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb0b6e640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfac0ce5d0 0x7fcfac1518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:58.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfab7fe640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac0cbd90 0x7fcfac1510a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c490 tx=0x7fcfa000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.674 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfa8ff9640 1 -- 172.21.15.31:0/3653653675 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00052d0 con 0x7fcfac0cbd90 2023-11-26T07:23:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfa8ff9640 1 -- 172.21.15.31:0/3653653675 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa0005430 con 0x7fcfac0cbd90 2023-11-26T07:23:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac155fa0 con 0x7fcfac0cbd90 2023-11-26T07:23:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfa8ff9640 1 -- 172.21.15.31:0/3653653675 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0012870 con 0x7fcfac0cbd90 2023-11-26T07:23:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac156560 con 0x7fcfac0cbd90 2023-11-26T07:23:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.668+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcfac072700 con 0x7fcfac0cbd90 2023-11-26T07:23:58.675 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfa8ff9640 1 -- 172.21.15.31:0/3653653675 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fcfa00129d0 con 0x7fcfac0cbd90 2023-11-26T07:23:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfa8ff9640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf98066770 0x7fcf98068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfa8ff9640 1 -- 172.21.15.31:0/3653653675 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcfa0083bb0 con 0x7fcfac0cbd90 2023-11-26T07:23:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfa8ff9640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] conn(0x7fcf9806c5d0 0x7fcf9806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfa8ff9640 1 -- 172.21.15.31:0/3653653675 --> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf9806f0a0 con 0x7fcf9806c5d0 2023-11-26T07:23:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfa8ff9640 1 -- 172.21.15.31:0/3653653675 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfa00882a0 con 0x7fcfac0cbd90 2023-11-26T07:23:58.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfabfff640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf98066770 0x7fcf98068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfb0b6e640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] conn(0x7fcf9806c5d0 0x7fcf9806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfabfff640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf98066770 0x7fcf98068c30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0ad000 tx=0x7fcf9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfb0b6e640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] conn(0x7fcf9806c5d0 0x7fcf9806e9f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.672+0000 7fcfa8ff9640 1 -- 172.21.15.31:0/3653653675 <== osd.5 v2:172.21.15.97:6811/3669978555 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fcf9806f0a0 con 0x7fcf9806c5d0 2023-11-26T07:23:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 --> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcfac0ccea0 con 0x7fcf9806c5d0 2023-11-26T07:23:58.690 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfa8ff9640 1 -- 172.21.15.31:0/3653653675 <== osd.5 v2:172.21.15.97:6811/3669978555 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcfa40070e0 con 0x7fcf9806c5d0 2023-11-26T07:23:58.691 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:23:58.691 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] conn(0x7fcf9806c5d0 msgr2=0x7fcf9806e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.97:6811/3669978555,v1:172.21.15.97:6815/3669978555] conn(0x7fcf9806c5d0 0x7fcf9806e9f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf98066770 msgr2=0x7fcf98068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcf98066770 0x7fcf98068c30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0ad000 tx=0x7fcf9c008040 comp rx=0 tx=0).stop 2023-11-26T07:23:58.692 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac0cbd90 msgr2=0x7fcfac1510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 --2- 172.21.15.31:0/3653653675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcfac0cbd90 0x7fcfac1510a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c490 tx=0x7fcfa000c950 comp rx=0 tx=0).stop 2023-11-26T07:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb0b6e640 1 -- 172.21.15.31:0/3653653675 reap_dead start 2023-11-26T07:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 shutdown_connections 2023-11-26T07:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 >> 172.21.15.31:0/3653653675 conn(0x7fcfac0b0cb0 msgr2=0x7fcfac071ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 shutdown_connections 2023-11-26T07:23:58.693 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.688+0000 7fcfb25d1640 1 -- 172.21.15.31:0/3653653675 wait complete. 2023-11-26T07:23:58.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-26T07:23:58.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.848+0000 7f3664d57640 1 Processor -- start 2023-11-26T07:23:58.853 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.848+0000 7f3664d57640 1 -- start start 2023-11-26T07:23:58.854 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f3664d57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36600b07a0 0x7f36601619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f3664d57640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3660161f20 0x7f36600b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f3664d57640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f36600b2870 0x7f36600b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f3664d57640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36600b0b80 con 0x7f3660161f20 2023-11-26T07:23:58.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f3664d57640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3660161290 con 0x7f36600b2870 2023-11-26T07:23:58.855 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f3664d57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3660161550 con 0x7f36600b07a0 2023-11-26T07:23:58.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f365dd9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3660161f20 0x7f36600b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f365dd9b640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3660161f20 0x7f36600b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55968/0 (socket says 172.21.15.31:55968) 2023-11-26T07:23:58.856 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f365dd9b640 1 -- 172.21.15.31:0/3544027574 learned_addr learned my addr 172.21.15.31:0/3544027574 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f365e59c640 1 --2- 172.21.15.31:0/3544027574 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36600b07a0 0x7f36601619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f365ed9d640 1 --2- 172.21.15.31:0/3544027574 >> v2:172.21.15.97:3300/0 conn(0x7f36600b2870 0x7f36600b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f365dd9b640 1 -- 172.21.15.31:0/3544027574 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36600b07a0 msgr2=0x7f36601619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f365dd9b640 1 --2- 172.21.15.31:0/3544027574 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36600b07a0 0x7f36601619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f365dd9b640 1 -- 172.21.15.31:0/3544027574 >> v2:172.21.15.97:3300/0 conn(0x7f36600b2870 msgr2=0x7f36600b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f365dd9b640 1 --2- 172.21.15.31:0/3544027574 >> v2:172.21.15.97:3300/0 conn(0x7f36600b2870 0x7f36600b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.857 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.852+0000 7f365dd9b640 1 -- 172.21.15.31:0/3544027574 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36600c65a0 con 0x7f3660161f20 2023-11-26T07:23:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f365dd9b640 1 --2- 172.21.15.31:0/3544027574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3660161f20 0x7f36600b0ee0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f366006c650 tx=0x7f365401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ad4ec38b699274 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.858 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f365d59a640 1 -- 172.21.15.31:0/3544027574 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540aa5f0 con 0x7f3660161f20 2023-11-26T07:23:58.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f365d59a640 1 -- 172.21.15.31:0/3544027574 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36540aa750 con 0x7f3660161f20 2023-11-26T07:23:58.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f365d59a640 1 -- 172.21.15.31:0/3544027574 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540aaae0 con 0x7f3660161f20 2023-11-26T07:23:58.859 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 -- 172.21.15.31:0/3544027574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3660161f20 msgr2=0x7f36600b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 --2- 172.21.15.31:0/3544027574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3660161f20 0x7f36600b0ee0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f366006c650 tx=0x7f365401c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 -- 172.21.15.31:0/3544027574 shutdown_connections 2023-11-26T07:23:58.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 --2- 172.21.15.31:0/3544027574 >> v2:172.21.15.97:3300/0 conn(0x7f36600b2870 0x7f36600b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 --2- 172.21.15.31:0/3544027574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3660161f20 0x7f36600b0ee0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 --2- 172.21.15.31:0/3544027574 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36600b07a0 0x7f36601619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.860 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 -- 172.21.15.31:0/3544027574 >> 172.21.15.31:0/3544027574 conn(0x7f36600b1d50 msgr2=0x7f36600b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 -- 172.21.15.31:0/3544027574 shutdown_connections 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 -- 172.21.15.31:0/3544027574 wait complete. 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 Processor -- start 2023-11-26T07:23:58.861 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.856+0000 7f3664d57640 1 -- start start 2023-11-26T07:23:58.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f3664d57640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36600b07a0 0x7f36600af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f3664d57640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36600b2870 0x7f36600af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f3664d57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3660161f20 0x7f366014e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f3664d57640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3660159920 con 0x7f36600b2870 2023-11-26T07:23:58.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f3664d57640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36600b2e40 con 0x7f3660161f20 2023-11-26T07:23:58.863 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f3664d57640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36600afdb0 con 0x7f36600b07a0 2023-11-26T07:23:58.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36600b2870 0x7f36600af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36600b2870 0x7f36600af870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55978/0 (socket says 172.21.15.31:55978) 2023-11-26T07:23:58.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365dd9b640 1 -- 172.21.15.31:0/1783843395 learned_addr learned my addr 172.21.15.31:0/1783843395 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:58.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365e59c640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36600b07a0 0x7f36600af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365ed9d640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3660161f20 0x7f366014e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365dd9b640 1 -- 172.21.15.31:0/1783843395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36600b07a0 msgr2=0x7f36600af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365dd9b640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36600b07a0 0x7f36600af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.864 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365dd9b640 1 -- 172.21.15.31:0/1783843395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3660161f20 msgr2=0x7f366014e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:58.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365dd9b640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3660161f20 0x7f366014e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:58.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365dd9b640 1 -- 172.21.15.31:0/1783843395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36540094d0 con 0x7f36600b2870 2023-11-26T07:23:58.865 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f365dd9b640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36600b2870 0x7f36600af870 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f365401ab80 tx=0x7f365401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f364effd640 1 -- 172.21.15.31:0/1783843395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540b3800 con 0x7f36600b2870 2023-11-26T07:23:58.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f364effd640 1 -- 172.21.15.31:0/1783843395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36540b3980 con 0x7f36600b2870 2023-11-26T07:23:58.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f364effd640 1 -- 172.21.15.31:0/1783843395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540b3d30 con 0x7f36600b2870 2023-11-26T07:23:58.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f3664d57640 1 -- 172.21.15.31:0/1783843395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3660157070 con 0x7f36600b2870 2023-11-26T07:23:58.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f3664d57640 1 -- 172.21.15.31:0/1783843395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3660159540 con 0x7f36600b2870 2023-11-26T07:23:58.866 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.860+0000 7f3664d57640 1 -- 172.21.15.31:0/1783843395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3660159a60 con 0x7f36600b2870 2023-11-26T07:23:58.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.864+0000 7f364effd640 1 -- 172.21.15.31:0/1783843395 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f36540b2380 con 0x7f36600b2870 2023-11-26T07:23:58.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.864+0000 7f364effd640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3640066740 0x7f3640068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:58.867 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.864+0000 7f364effd640 1 -- 172.21.15.31:0/1783843395 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f36540a7070 con 0x7f36600b2870 2023-11-26T07:23:58.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.864+0000 7f365e59c640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3640066740 0x7f3640068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:58.868 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.864+0000 7f365e59c640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3640066740 0x7f3640068c00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f36601529f0 tx=0x7f3650008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:58.870 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:58.868+0000 7f364effd640 1 -- 172.21.15.31:0/1783843395 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3660159a60 con 0x7f36600b2870 2023-11-26T07:23:59.027 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 -- 172.21.15.31:0/1783843395 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f3660160de0 con 0x7f3640066740 2023-11-26T07:23:59.028 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f364effd640 1 -- 172.21.15.31:0/1783843395 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3660160de0 con 0x7f3640066740 2023-11-26T07:23:59.028 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 5 to deep-scrub 2023-11-26T07:23:59.029 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 -- 172.21.15.31:0/1783843395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3640066740 msgr2=0x7f3640068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3640066740 0x7f3640068c00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f36601529f0 tx=0x7f3650008040 comp rx=0 tx=0).stop 2023-11-26T07:23:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 -- 172.21.15.31:0/1783843395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36600b2870 msgr2=0x7f36600af870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36600b2870 0x7f36600af870 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f365401ab80 tx=0x7f365401ad10 comp rx=0 tx=0).stop 2023-11-26T07:23:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 -- 172.21.15.31:0/1783843395 shutdown_connections 2023-11-26T07:23:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3640066740 0x7f3640068c00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.030 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3660161f20 0x7f366014e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36600b2870 0x7f36600af870 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 --2- 172.21.15.31:0/1783843395 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f36600b07a0 0x7f36600af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.024+0000 7f3664d57640 1 -- 172.21.15.31:0/1783843395 >> 172.21.15.31:0/1783843395 conn(0x7f36600b1d50 msgr2=0x7f3660167c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.028+0000 7f3664d57640 1 -- 172.21.15.31:0/1783843395 shutdown_connections 2023-11-26T07:23:59.031 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.028+0000 7f3664d57640 1 -- 172.21.15.31:0/1783843395 wait complete. 2023-11-26T07:23:59.048 INFO:tasks.ceph:Scrubbing osd.6 2023-11-26T07:23:59.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:23:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f17909df640 1 Processor -- start 2023-11-26T07:23:59.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f17909df640 1 -- start start 2023-11-26T07:23:59.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f17909df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f178c0cb9b0 0x7f178c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f17909df640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f178c0ce5d0 0x7f178c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f17909df640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c159630 con 0x7f178c0cbd90 2023-11-26T07:23:59.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f17909df640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c15bba0 con 0x7f178c0ce5d0 2023-11-26T07:23:59.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f17909df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0cdd50 con 0x7f178c0cb9b0 2023-11-26T07:23:59.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f178ad9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f178c0ce5d0 0x7f178c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f178ad9d640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f178c0ce5d0 0x7f178c15bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55992/0 (socket says 172.21.15.31:55992) 2023-11-26T07:23:59.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f178ad9d640 1 -- 172.21.15.31:0/1752428038 learned_addr learned my addr 172.21.15.31:0/1752428038 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:59.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f178a59c640 1 --2- 172.21.15.31:0/1752428038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f178c0cb9b0 0x7f178c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f178ad9d640 1 -- 172.21.15.31:0/1752428038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f178c0cb9b0 msgr2=0x7f178c06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f178ad9d640 1 --2- 172.21.15.31:0/1752428038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f178c0cb9b0 0x7f178c06e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f178ad9d640 1 -- 172.21.15.31:0/1752428038 >> v1:172.21.15.97:6789/0 conn(0x7f178c0cbd90 legacy=0x7f178c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.192+0000 7f178ad9d640 1 -- 172.21.15.31:0/1752428038 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c0cd1d0 con 0x7f178c0ce5d0 2023-11-26T07:23:59.199 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f178a59c640 1 --2- 172.21.15.31:0/1752428038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f178c0cb9b0 0x7f178c06e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:59.200 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f178ad9d640 1 --2- 172.21.15.31:0/1752428038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f178c0ce5d0 0x7f178c15bfd0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f1784009e50 tx=0x7f178401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50eb3a0dc2f1d0b server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f1789d9b640 1 -- 172.21.15.31:0/1752428038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840a5800 con 0x7f178c0ce5d0 2023-11-26T07:23:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f1789d9b640 1 -- 172.21.15.31:0/1752428038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17840a5960 con 0x7f178c0ce5d0 2023-11-26T07:23:59.201 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f1789d9b640 1 -- 172.21.15.31:0/1752428038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840a5cf0 con 0x7f178c0ce5d0 2023-11-26T07:23:59.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f17909df640 1 -- 172.21.15.31:0/1752428038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f178c0ce5d0 msgr2=0x7f178c15bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f17909df640 1 --2- 172.21.15.31:0/1752428038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f178c0ce5d0 0x7f178c15bfd0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f1784009e50 tx=0x7f178401c9a0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f17909df640 1 -- 172.21.15.31:0/1752428038 shutdown_connections 2023-11-26T07:23:59.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f17909df640 1 --2- 172.21.15.31:0/1752428038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f178c0ce5d0 0x7f178c15bfd0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f17909df640 1 --2- 172.21.15.31:0/1752428038 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f178c0cb9b0 0x7f178c06e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f17909df640 1 -- 172.21.15.31:0/1752428038 >> 172.21.15.31:0/1752428038 conn(0x7f178c0b0cb0 msgr2=0x7f178c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.202 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f17909df640 1 -- 172.21.15.31:0/1752428038 shutdown_connections 2023-11-26T07:23:59.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.196+0000 7f17909df640 1 -- 172.21.15.31:0/1752428038 wait complete. 2023-11-26T07:23:59.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f17909df640 1 Processor -- start 2023-11-26T07:23:59.203 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f17909df640 1 -- start start 2023-11-26T07:23:59.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f17909df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f178c0cb9b0 0x7f178c09b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f17909df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f178c0cbd90 0x7f178c150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.204 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f17909df640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f178c0ce5d0 0x7f178c1506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f17909df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c14f620 con 0x7f178c0cbd90 2023-11-26T07:23:59.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f17909df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c1530d0 con 0x7f178c0cb9b0 2023-11-26T07:23:59.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f17909df640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c1524c0 con 0x7f178c0ce5d0 2023-11-26T07:23:59.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1783fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f178c0cbd90 0x7f178c150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1783fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f178c0cbd90 0x7f178c150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55994/0 (socket says 172.21.15.31:55994) 2023-11-26T07:23:59.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1783fff640 1 -- 172.21.15.31:0/2072195856 learned_addr learned my addr 172.21.15.31:0/2072195856 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:59.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f178a59c640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f178c0cb9b0 0x7f178c09b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.205 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f178ad9d640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f178c0ce5d0 0x7f178c1506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1783fff640 1 -- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f178c0ce5d0 msgr2=0x7f178c1506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1783fff640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f178c0ce5d0 0x7f178c1506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1783fff640 1 -- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f178c0cb9b0 msgr2=0x7f178c09b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1783fff640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f178c0cb9b0 0x7f178c09b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1783fff640 1 -- 172.21.15.31:0/2072195856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1784009b30 con 0x7f178c0cbd90 2023-11-26T07:23:59.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1783fff640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f178c0cbd90 0x7f178c150190 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f177400c450 tx=0x7f177400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.207 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1782ffd640 1 -- 172.21.15.31:0/2072195856 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774012830 con 0x7f178c0cbd90 2023-11-26T07:23:59.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1782ffd640 1 -- 172.21.15.31:0/2072195856 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17740129b0 con 0x7f178c0cbd90 2023-11-26T07:23:59.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f1782ffd640 1 -- 172.21.15.31:0/2072195856 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1774012d60 con 0x7f178c0cbd90 2023-11-26T07:23:59.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.200+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c0cc350 con 0x7f178c0cbd90 2023-11-26T07:23:59.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c152a80 con 0x7f178c0cbd90 2023-11-26T07:23:59.208 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f178c0cc1d0 con 0x7f178c0cbd90 2023-11-26T07:23:59.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f1782ffd640 1 -- 172.21.15.31:0/2072195856 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f177401c460 con 0x7f178c0cbd90 2023-11-26T07:23:59.209 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f1782ffd640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f176c066740 0x7f176c068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f1782ffd640 1 -- 172.21.15.31:0/2072195856 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1774083ba0 con 0x7f178c0cbd90 2023-11-26T07:23:59.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f178a59c640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f176c066740 0x7f176c068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f1782ffd640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] conn(0x7f176c06c5a0 0x7f176c06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f1782ffd640 1 -- 172.21.15.31:0/2072195856 --> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f176c06f070 con 0x7f176c06c5a0 2023-11-26T07:23:59.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f1782ffd640 1 -- 172.21.15.31:0/2072195856 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1774088220 con 0x7f178c0cbd90 2023-11-26T07:23:59.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f178ad9d640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] conn(0x7f176c06c5a0 0x7f176c06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.210 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.204+0000 7f178a59c640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f176c066740 0x7f176c068c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f178c15b430 tx=0x7f177c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.208+0000 7f178ad9d640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] conn(0x7f176c06c5a0 0x7f176c06e9c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.211 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.208+0000 7f1782ffd640 1 -- 172.21.15.31:0/2072195856 <== osd.6 v2:172.21.15.97:6800/28530448 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f176c06f070 con 0x7f176c06c5a0 2023-11-26T07:23:59.223 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 --> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f178c0ccea0 con 0x7f176c06c5a0 2023-11-26T07:23:59.224 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f1782ffd640 1 -- 172.21.15.31:0/2072195856 <== osd.6 v2:172.21.15.97:6800/28530448 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f17840091d0 con 0x7f176c06c5a0 2023-11-26T07:23:59.224 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:23:59.225 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] conn(0x7f176c06c5a0 msgr2=0x7f176c06e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.97:6800/28530448,v1:172.21.15.97:6801/28530448] conn(0x7f176c06c5a0 0x7f176c06e9c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f176c066740 msgr2=0x7f176c068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f176c066740 0x7f176c068c00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f178c15b430 tx=0x7f177c008040 comp rx=0 tx=0).stop 2023-11-26T07:23:59.226 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f178c0cbd90 msgr2=0x7f178c150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 --2- 172.21.15.31:0/2072195856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f178c0cbd90 0x7f178c150190 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f177400c450 tx=0x7f177400c910 comp rx=0 tx=0).stop 2023-11-26T07:23:59.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f178ad9d640 1 -- 172.21.15.31:0/2072195856 reap_dead start 2023-11-26T07:23:59.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 shutdown_connections 2023-11-26T07:23:59.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 >> 172.21.15.31:0/2072195856 conn(0x7f178c0b0cb0 msgr2=0x7f178c06e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 shutdown_connections 2023-11-26T07:23:59.227 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.220+0000 7f17909df640 1 -- 172.21.15.31:0/2072195856 wait complete. 2023-11-26T07:23:59.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-26T07:23:59.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bf0fe7640 1 Processor -- start 2023-11-26T07:23:59.386 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bf0fe7640 1 -- start start 2023-11-26T07:23:59.388 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bf0fe7640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3bec15d8f0 0x7f3bec15e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bf0fe7640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f3bec0ced30 0x7f3bec0cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bf0fe7640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec15ecc0 con 0x7f3bec0cf1e0 2023-11-26T07:23:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bf0fe7640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec0d4420 con 0x7f3bec15d8f0 2023-11-26T07:23:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bf0fe7640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec0cdfb0 con 0x7f3bec0ced30 2023-11-26T07:23:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bea59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3bec15d8f0 0x7f3bec15e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bea59c640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3bec15d8f0 0x7f3bec15e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56004/0 (socket says 172.21.15.31:56004) 2023-11-26T07:23:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bea59c640 1 -- 172.21.15.31:0/1657845919 learned_addr learned my addr 172.21.15.31:0/1657845919 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:59.389 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3be9d9b640 1 --2- 172.21.15.31:0/1657845919 >> v2:172.21.15.97:3300/0 conn(0x7f3bec0ced30 0x7f3bec0cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bea59c640 1 -- 172.21.15.31:0/1657845919 >> v2:172.21.15.97:3300/0 conn(0x7f3bec0ced30 msgr2=0x7f3bec0cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bea59c640 1 --2- 172.21.15.31:0/1657845919 >> v2:172.21.15.97:3300/0 conn(0x7f3bec0ced30 0x7f3bec0cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bea59c640 1 -- 172.21.15.31:0/1657845919 >> v1:172.21.15.97:6789/0 conn(0x7f3bec0cf1e0 legacy=0x7f3bec0d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.390 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.384+0000 7f3bea59c640 1 -- 172.21.15.31:0/1657845919 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0d4140 con 0x7f3bec15d8f0 2023-11-26T07:23:59.391 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3bea59c640 1 --2- 172.21.15.31:0/1657845919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 0x7f3bec15e780 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009870 tx=0x7f3bdc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c44c20c7b2e346a server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3be959a640 1 -- 172.21.15.31:0/1657845919 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0aa430 con 0x7f3bec15d8f0 2023-11-26T07:23:59.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3be959a640 1 -- 172.21.15.31:0/1657845919 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc0aa590 con 0x7f3bec15d8f0 2023-11-26T07:23:59.392 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3be959a640 1 -- 172.21.15.31:0/1657845919 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0aa920 con 0x7f3bec15d8f0 2023-11-26T07:23:59.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/1657845919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 msgr2=0x7f3bec15e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3bf0fe7640 1 --2- 172.21.15.31:0/1657845919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 0x7f3bec15e780 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009870 tx=0x7f3bdc01c4d0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/1657845919 shutdown_connections 2023-11-26T07:23:59.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3bf0fe7640 1 --2- 172.21.15.31:0/1657845919 >> v2:172.21.15.97:3300/0 conn(0x7f3bec0ced30 0x7f3bec0cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3bf0fe7640 1 --2- 172.21.15.31:0/1657845919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 0x7f3bec15e780 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.393 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/1657845919 >> 172.21.15.31:0/1657845919 conn(0x7f3bec09c750 msgr2=0x7f3bec0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/1657845919 shutdown_connections 2023-11-26T07:23:59.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.388+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/1657845919 wait complete. 2023-11-26T07:23:59.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bf0fe7640 1 Processor -- start 2023-11-26T07:23:59.394 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bf0fe7640 1 -- start start 2023-11-26T07:23:59.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bf0fe7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec0ced30 0x7f3bec1f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bf0fe7640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec0cf1e0 0x7f3bec15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bf0fe7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 0x7f3bec09d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bf0fe7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec148300 con 0x7f3bec15d8f0 2023-11-26T07:23:59.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bf0fe7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec1f8180 con 0x7f3bec0ced30 2023-11-26T07:23:59.396 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bf0fe7640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec1476f0 con 0x7f3bec0cf1e0 2023-11-26T07:23:59.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bead9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 0x7f3bec09d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bead9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 0x7f3bec09d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56018/0 (socket says 172.21.15.31:56018) 2023-11-26T07:23:59.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bead9d640 1 -- 172.21.15.31:0/996555713 learned_addr learned my addr 172.21.15.31:0/996555713 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:59.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3be9d9b640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec0cf1e0 0x7f3bec15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bea59c640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec0ced30 0x7f3bec1f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bead9d640 1 -- 172.21.15.31:0/996555713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec0cf1e0 msgr2=0x7f3bec15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.397 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bead9d640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec0cf1e0 0x7f3bec15a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bead9d640 1 -- 172.21.15.31:0/996555713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec0ced30 msgr2=0x7f3bec1f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bead9d640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec0ced30 0x7f3bec1f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.398 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bead9d640 1 -- 172.21.15.31:0/996555713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc0094d0 con 0x7f3bec15d8f0 2023-11-26T07:23:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bead9d640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 0x7f3bec09d1c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f3be400c730 tx=0x7f3be400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bdaffd640 1 -- 172.21.15.31:0/996555713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4003b80 con 0x7f3bec15d8f0 2023-11-26T07:23:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bdaffd640 1 -- 172.21.15.31:0/996555713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be4003ce0 con 0x7f3bec15d8f0 2023-11-26T07:23:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bdaffd640 1 -- 172.21.15.31:0/996555713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400aa30 con 0x7f3bec15d8f0 2023-11-26T07:23:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/996555713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec1479d0 con 0x7f3bec15d8f0 2023-11-26T07:23:59.399 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.392+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/996555713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec147f90 con 0x7f3bec15d8f0 2023-11-26T07:23:59.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.396+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/996555713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec0cdc20 con 0x7f3bec15d8f0 2023-11-26T07:23:59.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.396+0000 7f3bdaffd640 1 -- 172.21.15.31:0/996555713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f3be400ab90 con 0x7f3bec15d8f0 2023-11-26T07:23:59.400 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.396+0000 7f3bdaffd640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3bcc066770 0x7f3bcc068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.396+0000 7f3bdaffd640 1 -- 172.21.15.31:0/996555713 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3be4083c40 con 0x7f3bec15d8f0 2023-11-26T07:23:59.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.396+0000 7f3bea59c640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3bcc066770 0x7f3bcc068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.401 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.396+0000 7f3bea59c640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3bcc066770 0x7f3bcc068c30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0ae000 tx=0x7f3bdc0b5040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.404 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.400+0000 7f3bdaffd640 1 -- 172.21.15.31:0/996555713 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3be40510e0 con 0x7f3bec15d8f0 2023-11-26T07:23:59.560 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.556+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/996555713 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bec0b29b0 con 0x7f3bcc066770 2023-11-26T07:23:59.561 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.556+0000 7f3bdaffd640 1 -- 172.21.15.31:0/996555713 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3bdc0aa880 con 0x7f3bcc066770 2023-11-26T07:23:59.561 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 6 to deep-scrub 2023-11-26T07:23:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/996555713 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3bcc066770 msgr2=0x7f3bcc068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3bcc066770 0x7f3bcc068c30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc0ae000 tx=0x7f3bdc0b5040 comp rx=0 tx=0).stop 2023-11-26T07:23:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/996555713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 msgr2=0x7f3bec09d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 0x7f3bec09d1c0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f3be400c730 tx=0x7f3be400cbf0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.563 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/996555713 shutdown_connections 2023-11-26T07:23:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f3bcc066770 0x7f3bcc068c30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec15d8f0 0x7f3bec09d1c0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f3bec0cf1e0 0x7f3bec15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 --2- 172.21.15.31:0/996555713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3bec0ced30 0x7f3bec1f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/996555713 >> 172.21.15.31:0/996555713 conn(0x7f3bec09c750 msgr2=0x7f3bec15dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/996555713 shutdown_connections 2023-11-26T07:23:59.564 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.560+0000 7f3bf0fe7640 1 -- 172.21.15.31:0/996555713 wait complete. 2023-11-26T07:23:59.582 INFO:tasks.ceph:Scrubbing osd.7 2023-11-26T07:23:59.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-26T07:23:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.724+0000 7fb6bada1640 1 Processor -- start 2023-11-26T07:23:59.729 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.724+0000 7fb6bada1640 1 -- start start 2023-11-26T07:23:59.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.728+0000 7fb6bada1640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6b406e3f0 0x7fb6b415f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.728+0000 7fb6bada1640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b4159630 con 0x7fb6b40ce5d0 2023-11-26T07:23:59.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.728+0000 7fb6bada1640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6b415f6d0 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.728+0000 7fb6bada1640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6b406e860 con 0x7fb6b406e3f0 2023-11-26T07:23:59.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.728+0000 7fb6b3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6b406e3f0 0x7fb6b415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.731 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.728+0000 7fb6b3fff640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb6b406e3f0 0x7fb6b415f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56024/0 (socket says 172.21.15.31:56024) 2023-11-26T07:23:59.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.728+0000 7fb6b3fff640 1 -- 172.21.15.31:0/470337200 learned_addr learned my addr 172.21.15.31:0/470337200 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:59.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.728+0000 7fb6b3fff640 1 -- 172.21.15.31:0/470337200 >> v1:172.21.15.97:6789/0 conn(0x7fb6b40cb9b0 legacy=0x7fb6b40cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.728+0000 7fb6b3fff640 1 -- 172.21.15.31:0/470337200 >> v1:172.21.15.31:6789/0 conn(0x7fb6b40ce5d0 legacy=0x7fb6b415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.732 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.728+0000 7fb6b3fff640 1 -- 172.21.15.31:0/470337200 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b40cd1d0 con 0x7fb6b406e3f0 2023-11-26T07:23:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6b3fff640 1 --2- 172.21.15.31:0/470337200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b406e3f0 0x7fb6b415f190 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009870 tx=0x7fb6a001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22942315ccfdedf server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6b37fe640 1 -- 172.21.15.31:0/470337200 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00ab600 con 0x7fb6b406e3f0 2023-11-26T07:23:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6b37fe640 1 -- 172.21.15.31:0/470337200 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a00a7050 con 0x7fb6b406e3f0 2023-11-26T07:23:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6b37fe640 1 -- 172.21.15.31:0/470337200 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00ab980 con 0x7fb6b406e3f0 2023-11-26T07:23:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6bada1640 1 -- 172.21.15.31:0/470337200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b406e3f0 msgr2=0x7fb6b415f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6bada1640 1 --2- 172.21.15.31:0/470337200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b406e3f0 0x7fb6b415f190 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009870 tx=0x7fb6a001c3d0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6bada1640 1 -- 172.21.15.31:0/470337200 shutdown_connections 2023-11-26T07:23:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6bada1640 1 --2- 172.21.15.31:0/470337200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b406e3f0 0x7fb6b415f190 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6bada1640 1 -- 172.21.15.31:0/470337200 >> 172.21.15.31:0/470337200 conn(0x7fb6b40b0cb0 msgr2=0x7fb6b40b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6bada1640 1 -- 172.21.15.31:0/470337200 shutdown_connections 2023-11-26T07:23:59.736 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6bada1640 1 -- 172.21.15.31:0/470337200 wait complete. 2023-11-26T07:23:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6bada1640 1 Processor -- start 2023-11-26T07:23:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.732+0000 7fb6bada1640 1 -- start start 2023-11-26T07:23:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6bada1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b406e3f0 0x7fb6b40afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6bada1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b40cb9b0 0x7fb6b41581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6bada1640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6b40ce5d0 0x7fb6b41586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6bada1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40b2970 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6bada1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40b1d60 con 0x7fb6b406e3f0 2023-11-26T07:23:59.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6bada1640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b40b2040 con 0x7fb6b40ce5d0 2023-11-26T07:23:59.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b40cb9b0 0x7fb6b41581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.739 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b40cb9b0 0x7fb6b41581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56038/0 (socket says 172.21.15.31:56038) 2023-11-26T07:23:59.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b3fff640 1 -- 172.21.15.31:0/4225894068 learned_addr learned my addr 172.21.15.31:0/4225894068 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:59.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b8b3d640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b406e3f0 0x7fb6b40afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b933e640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6b40ce5d0 0x7fb6b41586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b3fff640 1 -- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6b40ce5d0 msgr2=0x7fb6b41586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b3fff640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fb6b40ce5d0 0x7fb6b41586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b3fff640 1 -- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b406e3f0 msgr2=0x7fb6b40afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b3fff640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b406e3f0 0x7fb6b40afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.740 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b3fff640 1 -- 172.21.15.31:0/4225894068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a00094d0 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b8b3d640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6b406e3f0 0x7fb6b40afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:59.741 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b3fff640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b40cb9b0 0x7fb6b41581b0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00ae040 tx=0x7fb6a0019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b17fa640 1 -- 172.21.15.31:0/4225894068 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00aa8b0 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b17fa640 1 -- 172.21.15.31:0/4225894068 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a00aaa10 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b40b2320 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b17fa640 1 -- 172.21.15.31:0/4225894068 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00aada0 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b40b0150 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb6b40cda70 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.742 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.736+0000 7fb6b17fa640 1 -- 172.21.15.31:0/4225894068 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fb6a00b2070 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.743 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.740+0000 7fb6b17fa640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb698066740 0x7fb698068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.740+0000 7fb6b17fa640 1 -- 172.21.15.31:0/4225894068 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb6a011e4e0 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.740+0000 7fb6b8b3d640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb698066740 0x7fb698068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.740+0000 7fb6b17fa640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] conn(0x7fb69806c5a0 0x7fb69806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.740+0000 7fb6b17fa640 1 -- 172.21.15.31:0/4225894068 --> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb69806f070 con 0x7fb69806c5a0 2023-11-26T07:23:59.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.740+0000 7fb6b17fa640 1 -- 172.21.15.31:0/4225894068 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6a0120b20 con 0x7fb6b40cb9b0 2023-11-26T07:23:59.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.740+0000 7fb6b933e640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] conn(0x7fb69806c5a0 0x7fb69806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.744 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.740+0000 7fb6b8b3d640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb698066740 0x7fb698068c00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40cb7a0 tx=0x7fb6a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.740+0000 7fb6b933e640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] conn(0x7fb69806c5a0 0x7fb69806e9c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.745 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.740+0000 7fb6b17fa640 1 -- 172.21.15.31:0/4225894068 <== osd.7 v2:172.21.15.97:6808/4245711455 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb69806f070 con 0x7fb69806c5a0 2023-11-26T07:23:59.757 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.752+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 --> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb6b40cd790 con 0x7fb69806c5a0 2023-11-26T07:23:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6b17fa640 1 -- 172.21.15.31:0/4225894068 <== osd.7 v2:172.21.15.97:6808/4245711455 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb6ac009c20 con 0x7fb69806c5a0 2023-11-26T07:23:59.758 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2023-11-26T07:23:59.758 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-26T07:23:59.758 INFO:teuthology.orchestra.run.smithi031.stdout:} 2023-11-26T07:23:59.759 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] conn(0x7fb69806c5a0 msgr2=0x7fb69806e9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6bada1640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.97:6808/4245711455,v1:172.21.15.97:6810/4245711455] conn(0x7fb69806c5a0 0x7fb69806e9c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb698066740 msgr2=0x7fb698068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6bada1640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fb698066740 0x7fb698068c00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40cb7a0 tx=0x7fb6a8009040 comp rx=0 tx=0).stop 2023-11-26T07:23:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b40cb9b0 msgr2=0x7fb6b41581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6bada1640 1 --2- 172.21.15.31:0/4225894068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b40cb9b0 0x7fb6b41581b0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00ae040 tx=0x7fb6a0019170 comp rx=0 tx=0).stop 2023-11-26T07:23:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6b933e640 1 -- 172.21.15.31:0/4225894068 reap_dead start 2023-11-26T07:23:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 shutdown_connections 2023-11-26T07:23:59.760 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 >> 172.21.15.31:0/4225894068 conn(0x7fb6b40b0cb0 msgr2=0x7fb6b415a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 shutdown_connections 2023-11-26T07:23:59.761 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.756+0000 7fb6bada1640 1 -- 172.21.15.31:0/4225894068 wait complete. 2023-11-26T07:23:59.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-26T07:23:59.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.916+0000 7fcd5c5aa640 1 Processor -- start 2023-11-26T07:23:59.921 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.916+0000 7fcd5c5aa640 1 -- start start 2023-11-26T07:23:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5c5aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54146820 0x7fcd54070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5c5aa640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fcd540b4210 0x7fcd540b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5c5aa640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcd54161180 0x7fcd541615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5c5aa640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd54147190 con 0x7fcd54161180 2023-11-26T07:23:59.923 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5c5aa640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd5406d030 con 0x7fcd540b4210 2023-11-26T07:23:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5c5aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540b4bb0 con 0x7fcd54146820 2023-11-26T07:23:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5ab47640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcd54161180 0x7fcd541615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5ab47640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcd54161180 0x7fcd541615d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56054/0 (socket says 172.21.15.31:56054) 2023-11-26T07:23:59.924 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5ab47640 1 -- 172.21.15.31:0/1073062499 learned_addr learned my addr 172.21.15.31:0/1073062499 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd59b45640 1 --2- 172.21.15.31:0/1073062499 >> v2:172.21.15.97:3300/0 conn(0x7fcd540b4210 0x7fcd540b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5a346640 1 --2- 172.21.15.31:0/1073062499 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54146820 0x7fcd54070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5ab47640 1 -- 172.21.15.31:0/1073062499 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54146820 msgr2=0x7fcd54070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5ab47640 1 --2- 172.21.15.31:0/1073062499 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54146820 0x7fcd54070a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5ab47640 1 -- 172.21.15.31:0/1073062499 >> v2:172.21.15.97:3300/0 conn(0x7fcd540b4210 msgr2=0x7fcd540b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:23:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5ab47640 1 --2- 172.21.15.31:0/1073062499 >> v2:172.21.15.97:3300/0 conn(0x7fcd540b4210 0x7fcd540b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5ab47640 1 -- 172.21.15.31:0/1073062499 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd54146c00 con 0x7fcd54161180 2023-11-26T07:23:59.925 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.920+0000 7fcd5a346640 1 --2- 172.21.15.31:0/1073062499 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54146820 0x7fcd54070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:23:59.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5ab47640 1 --2- 172.21.15.31:0/1073062499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54161180 0x7fcd541615d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fcd540b3fd0 tx=0x7fcd4c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387482687069e8ea server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd59344640 1 -- 172.21.15.31:0/1073062499 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0a5820 con 0x7fcd54161180 2023-11-26T07:23:59.926 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd59344640 1 -- 172.21.15.31:0/1073062499 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c0a5980 con 0x7fcd54161180 2023-11-26T07:23:59.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd59344640 1 -- 172.21.15.31:0/1073062499 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0a5d10 con 0x7fcd54161180 2023-11-26T07:23:59.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/1073062499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54161180 msgr2=0x7fcd541615d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.927 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 --2- 172.21.15.31:0/1073062499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54161180 0x7fcd541615d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fcd540b3fd0 tx=0x7fcd4c01c9a0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/1073062499 shutdown_connections 2023-11-26T07:23:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 --2- 172.21.15.31:0/1073062499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54161180 0x7fcd541615d0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 --2- 172.21.15.31:0/1073062499 >> v2:172.21.15.97:3300/0 conn(0x7fcd540b4210 0x7fcd540b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 --2- 172.21.15.31:0/1073062499 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54146820 0x7fcd54070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/1073062499 >> 172.21.15.31:0/1073062499 conn(0x7fcd540b2f10 msgr2=0x7fcd540b3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:23:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/1073062499 shutdown_connections 2023-11-26T07:23:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/1073062499 wait complete. 2023-11-26T07:23:59.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 Processor -- start 2023-11-26T07:23:59.929 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.924+0000 7fcd5c5aa640 1 -- start start 2023-11-26T07:23:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5c5aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd540b4210 0x7fcd5415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5c5aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54146820 0x7fcd54164640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5c5aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54161180 0x7fcd541f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5c5aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415bad0 con 0x7fcd54146820 2023-11-26T07:23:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5c5aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd541f7270 con 0x7fcd540b4210 2023-11-26T07:23:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5c5aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5415b410 con 0x7fcd54161180 2023-11-26T07:23:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd59b45640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54146820 0x7fcd54164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd59b45640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54146820 0x7fcd54164640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56064/0 (socket says 172.21.15.31:56064) 2023-11-26T07:23:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd59b45640 1 -- 172.21.15.31:0/3155584065 learned_addr learned my addr 172.21.15.31:0/3155584065 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:23:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5a346640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd540b4210 0x7fcd5415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5ab47640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54161180 0x7fcd541f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd59b45640 1 -- 172.21.15.31:0/3155584065 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54161180 msgr2=0x7fcd541f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.932 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd59b45640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54161180 0x7fcd541f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd59b45640 1 -- 172.21.15.31:0/3155584065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd540b4210 msgr2=0x7fcd5415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:23:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd59b45640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd540b4210 0x7fcd5415bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:23:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd59b45640 1 -- 172.21.15.31:0/3155584065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c009b30 con 0x7fcd54146820 2023-11-26T07:23:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5ab47640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54161180 0x7fcd541f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:23:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd59b45640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54146820 0x7fcd54164640 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400e8b0 tx=0x7fcd4400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd4affd640 1 -- 172.21.15.31:0/3155584065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400c940 con 0x7fcd54146820 2023-11-26T07:23:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd4affd640 1 -- 172.21.15.31:0/3155584065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4400caa0 con 0x7fcd54146820 2023-11-26T07:23:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd4affd640 1 -- 172.21.15.31:0/3155584065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400ce30 con 0x7fcd54146820 2023-11-26T07:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/3155584065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5415b6f0 con 0x7fcd54146820 2023-11-26T07:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/3155584065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd54168020 con 0x7fcd54146820 2023-11-26T07:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/3155584065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5415b270 con 0x7fcd54146820 2023-11-26T07:23:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.928+0000 7fcd4affd640 1 -- 172.21.15.31:0/3155584065 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fcd44013470 con 0x7fcd54146820 2023-11-26T07:23:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.932+0000 7fcd4affd640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcd38066740 0x7fcd38068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:23:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.932+0000 7fcd4affd640 1 -- 172.21.15.31:0/3155584065 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcd44084120 con 0x7fcd54146820 2023-11-26T07:23:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.932+0000 7fcd5a346640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcd38066740 0x7fcd38068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:23:59.935 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.932+0000 7fcd5a346640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcd38066740 0x7fcd38068c00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd54145c80 tx=0x7fcd50008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:23:59.938 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:23:59.936+0000 7fcd4affd640 1 -- 172.21.15.31:0/3155584065 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcd5415b270 con 0x7fcd54146820 2023-11-26T07:24:00.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.096+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/3155584065 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd5406cb10 con 0x7fcd38066740 2023-11-26T07:24:00.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.096+0000 7fcd4affd640 1 -- 172.21.15.31:0/3155584065 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcd5000c590 con 0x7fcd38066740 2023-11-26T07:24:00.101 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 7 to deep-scrub 2023-11-26T07:24:00.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/3155584065 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcd38066740 msgr2=0x7fcd38068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcd38066740 0x7fcd38068c00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcd54145c80 tx=0x7fcd50008040 comp rx=0 tx=0).stop 2023-11-26T07:24:00.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/3155584065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54146820 msgr2=0x7fcd54164640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54146820 0x7fcd54164640 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400e8b0 tx=0x7fcd4400ed70 comp rx=0 tx=0).stop 2023-11-26T07:24:00.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/3155584065 shutdown_connections 2023-11-26T07:24:00.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fcd38066740 0x7fcd38068c00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fcd54161180 0x7fcd541f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd54146820 0x7fcd54164640 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 --2- 172.21.15.31:0/3155584065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd540b4210 0x7fcd5415bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/3155584065 >> 172.21.15.31:0/3155584065 conn(0x7fcd540b2f10 msgr2=0x7fcd5406e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:00.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/3155584065 shutdown_connections 2023-11-26T07:24:00.104 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.100+0000 7fcd5c5aa640 1 -- 172.21.15.31:0/3155584065 wait complete. 2023-11-26T07:24:00.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:24:00.265 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.260+0000 7f65e51aa640 1 Processor -- start 2023-11-26T07:24:00.266 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65e51aa640 1 -- start start 2023-11-26T07:24:00.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65e51aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e006d920 0x7f65e0070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65e51aa640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e0145060 con 0x7f65e0144c80 2023-11-26T07:24:00.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65e51aa640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e0071fd0 con 0x7f65e00ac630 2023-11-26T07:24:00.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65e51aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0071450 con 0x7f65e006d920 2023-11-26T07:24:00.267 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65ded9d640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f65e0144c80 0x7f65e00abf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:52218/0 (socket says 172.21.15.31:52218) 2023-11-26T07:24:00.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65ded9d640 1 -- 172.21.15.31:0/2143063394 learned_addr learned my addr 172.21.15.31:0/2143063394 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:24:00.268 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65df59e640 1 --2- 172.21.15.31:0/2143063394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e006d920 0x7f65e0070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65ddd9b640 1 -- 172.21.15.31:0/2143063394 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185631212 0 0) 0x7f65e0145060 con 0x7f65e0144c80 2023-11-26T07:24:00.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65ddd9b640 1 -- 172.21.15.31:0/2143063394 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65c8003530 con 0x7f65e0144c80 2023-11-26T07:24:00.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65ddd9b640 1 -- 172.21.15.31:0/2143063394 <== mon.1 v1:172.21.15.97:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656683760 0 0) 0x7f65e0071fd0 con 0x7f65e00ac630 2023-11-26T07:24:00.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65ddd9b640 1 -- 172.21.15.31:0/2143063394 --> v1:172.21.15.97:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e0145060 con 0x7f65e00ac630 2023-11-26T07:24:00.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65ddd9b640 1 -- 172.21.15.31:0/2143063394 <== mon.0 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1575292200 0 0) 0x7f65c8003530 con 0x7f65e0144c80 2023-11-26T07:24:00.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65df59e640 1 -- 172.21.15.31:0/2143063394 >> v1:172.21.15.97:6789/0 conn(0x7f65e00ac630 legacy=0x7f65e00706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65df59e640 1 -- 172.21.15.31:0/2143063394 >> v1:172.21.15.31:6789/0 conn(0x7f65e0144c80 legacy=0x7f65e00abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.269 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.264+0000 7f65df59e640 1 -- 172.21.15.31:0/2143063394 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0071730 con 0x7f65e006d920 2023-11-26T07:24:00.270 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65df59e640 1 --2- 172.21.15.31:0/2143063394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e006d920 0x7f65e0070df0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009dc0 tx=0x7f65d801c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46a68ce1a678264 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.271 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65ddd9b640 1 -- 172.21.15.31:0/2143063394 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0001680 con 0x7f65e006d920 2023-11-26T07:24:00.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65ddd9b640 1 -- 172.21.15.31:0/2143063394 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d80a56a0 con 0x7f65e006d920 2023-11-26T07:24:00.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65ddd9b640 1 -- 172.21.15.31:0/2143063394 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80a5a30 con 0x7f65e006d920 2023-11-26T07:24:00.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65e51aa640 1 -- 172.21.15.31:0/2143063394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e006d920 msgr2=0x7f65e0070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65e51aa640 1 --2- 172.21.15.31:0/2143063394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e006d920 0x7f65e0070df0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009dc0 tx=0x7f65d801c860 comp rx=0 tx=0).stop 2023-11-26T07:24:00.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65e51aa640 1 -- 172.21.15.31:0/2143063394 shutdown_connections 2023-11-26T07:24:00.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65e51aa640 1 --2- 172.21.15.31:0/2143063394 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e006d920 0x7f65e0070df0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.272 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65e51aa640 1 -- 172.21.15.31:0/2143063394 >> 172.21.15.31:0/2143063394 conn(0x7f65e00b0c40 msgr2=0x7f65e00b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:00.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65e51aa640 1 -- 172.21.15.31:0/2143063394 shutdown_connections 2023-11-26T07:24:00.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.268+0000 7f65e51aa640 1 -- 172.21.15.31:0/2143063394 wait complete. 2023-11-26T07:24:00.273 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 Processor -- start 2023-11-26T07:24:00.274 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 -- start start 2023-11-26T07:24:00.275 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e006d920 0x7f65e0158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e00ac630 0x7f65e01595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e0144c80 0x7f65e0159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e013a390 con 0x7f65e0144c80 2023-11-26T07:24:00.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e015e700 con 0x7f65e006d920 2023-11-26T07:24:00.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e009ad60 con 0x7f65e00ac630 2023-11-26T07:24:00.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65df59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e0144c80 0x7f65e0159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.276 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65df59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e0144c80 0x7f65e0159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56066/0 (socket says 172.21.15.31:56066) 2023-11-26T07:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65df59e640 1 -- 172.21.15.31:0/1710558343 learned_addr learned my addr 172.21.15.31:0/1710558343 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65ded9d640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e006d920 0x7f65e0158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65de59c640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e00ac630 0x7f65e01595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65df59e640 1 -- 172.21.15.31:0/1710558343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e00ac630 msgr2=0x7f65e01595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65df59e640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e00ac630 0x7f65e01595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65df59e640 1 -- 172.21.15.31:0/1710558343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e006d920 msgr2=0x7f65e0158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65df59e640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e006d920 0x7f65e0158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.277 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65df59e640 1 -- 172.21.15.31:0/1710558343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d8009a70 con 0x7f65e0144c80 2023-11-26T07:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65de59c640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e00ac630 0x7f65e01595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:00.278 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65df59e640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e0144c80 0x7f65e0159af0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f65d801cd40 tx=0x7f65d8019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65c77fe640 1 -- 172.21.15.31:0/1710558343 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80b9650 con 0x7f65e0144c80 2023-11-26T07:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65c77fe640 1 -- 172.21.15.31:0/1710558343 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d80b97b0 con 0x7f65e0144c80 2023-11-26T07:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65c77fe640 1 -- 172.21.15.31:0/1710558343 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80b9b20 con 0x7f65e0144c80 2023-11-26T07:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 -- 172.21.15.31:0/1710558343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e009b040 con 0x7f65e0144c80 2023-11-26T07:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 -- 172.21.15.31:0/1710558343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e0159330 con 0x7f65e0144c80 2023-11-26T07:24:00.279 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.272+0000 7f65e51aa640 1 -- 172.21.15.31:0/1710558343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e00b1ba0 con 0x7f65e0144c80 2023-11-26T07:24:00.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.276+0000 7f65c77fe640 1 -- 172.21.15.31:0/1710558343 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f65d80b9c80 con 0x7f65e0144c80 2023-11-26T07:24:00.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.276+0000 7f65c77fe640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f65cc066740 0x7f65cc068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:00.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.276+0000 7f65c77fe640 1 -- 172.21.15.31:0/1710558343 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f65d811f160 con 0x7f65e0144c80 2023-11-26T07:24:00.280 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.276+0000 7f65ded9d640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f65cc066740 0x7f65cc068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:00.281 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.276+0000 7f65ded9d640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f65cc066740 0x7f65cc068c00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65d0000fe0 tx=0x7f65d0001cc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:00.283 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.280+0000 7f65c77fe640 1 -- 172.21.15.31:0/1710558343 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f65d80ec5e0 con 0x7f65e0144c80 2023-11-26T07:24:00.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.436+0000 7f65e51aa640 1 -- 172.21.15.31:0/1710558343 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65e009b560 con 0x7f65cc066740 2023-11-26T07:24:00.440 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.436+0000 7f65c77fe640 1 -- 172.21.15.31:0/1710558343 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50210 (secure 0 0 0) 0x7f65e009b560 con 0x7f65cc066740 2023-11-26T07:24:00.441 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:24:00.442 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:24:00.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 -- 172.21.15.31:0/1710558343 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f65cc066740 msgr2=0x7f65cc068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f65cc066740 0x7f65cc068c00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65d0000fe0 tx=0x7f65d0001cc0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 -- 172.21.15.31:0/1710558343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e0144c80 msgr2=0x7f65e0159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:00.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e0144c80 0x7f65e0159af0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f65d801cd40 tx=0x7f65d8019600 comp rx=0 tx=0).stop 2023-11-26T07:24:00.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 -- 172.21.15.31:0/1710558343 shutdown_connections 2023-11-26T07:24:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f65cc066740 0x7f65cc068c00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65e0144c80 0x7f65e0159af0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f65e00ac630 0x7f65e01595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 --2- 172.21.15.31:0/1710558343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65e006d920 0x7f65e0158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 -- 172.21.15.31:0/1710558343 >> 172.21.15.31:0/1710558343 conn(0x7f65e00b0c40 msgr2=0x7f65e00b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 -- 172.21.15.31:0/1710558343 shutdown_connections 2023-11-26T07:24:00.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:00.440+0000 7f65e51aa640 1 -- 172.21.15.31:0/1710558343 wait complete. 2023-11-26T07:24:00.462 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-11-26T07:24:00.220475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260592,"kb_used_data":11140,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714128,"statfs":{"total":773094113280,"available":772827267072,"internally_reserved":0,"allocated":11407360,"data_stored":5644357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16777869,"num_objects":-19,"num_object_clones":0,"num_object_copies":-38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-19,"num_whiteouts":0,"num_read":-60,"num_read_kb":-8232,"num_write":-31,"num_write_kb":-16393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.894149"},"pg_stats":[{"pgid":"1.0","version":"69'128","reported_seq":273,"reported_epoch":101,"state":"active+clean","last_fresh":"2023-11-26T07:23:50.331430+0000","last_change":"2023-11-26T07:17:22.381511+0000","last_active":"2023-11-26T07:23:50.331430+0000","last_peered":"2023-11-26T07:23:50.331430+0000","last_clean":"2023-11-26T07:23:50.331430+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:23:50.331430+0000","last_undegraded":"2023-11-26T07:23:50.331430+0000","last_fullsized":"2023-11-26T07:23:50.331430+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-26T07:17:20.806918+0000","last_clean_scrub_stamp":"2023-11-26T07:17:20.806918+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T14:55:58.959160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1871872,"data_stored":1148448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85599999999999998,"5min":1.008,"15min":1.008},"last":0.627},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.94399999999999995},"last":0.68200000000000005}]},{"osd":1,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.68300000000000005},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.90100000000000002,"5min":0.98799999999999999,"15min":1.0189999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.504,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86799999999999999,"5min":0.93000000000000005,"15min":1.1519999999999999},"last":0.70899999999999996}]},{"osd":2,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0700000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":0.92400000000000004,"15min":1.016},"last":0.54900000000000004}]},{"osd":3,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73099999999999998,"15min":0.73899999999999999},"min":{"1min":0.53300000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88500000000000001,"5min":1.0089999999999999,"15min":1.2270000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.63600000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93700000000000006,"5min":0.97999999999999998,"15min":1.1060000000000001},"last":0.81499999999999995}]},{"osd":4,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.69499999999999995},"min":{"1min":0.46600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":1.0309999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.67700000000000005},"min":{"1min":0.47099999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.81100000000000005,"5min":0.91700000000000004,"15min":1.1759999999999999},"last":0.53300000000000003}]},{"osd":5,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69599999999999995,"15min":0.76200000000000001},"min":{"1min":0.52500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":1.5129999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.82999999999999996},"min":{"1min":0.60799999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86599999999999999,"5min":0.96799999999999997,"15min":1.5780000000000001},"last":0.752}]},{"osd":6,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.749},"min":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88800000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.81899999999999995},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.093},"last":0.72099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.83399999999999996,"15min":0.77000000000000002},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":1.2130000000000001,"15min":0.90400000000000003},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.59899999999999998}]},{"osd":1,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69899999999999995,"15min":0.73799999999999999},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":2.1880000000000002,"15min":3.1520000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.81299999999999994},"min":{"1min":0.50900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.02,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.74299999999999999}]},{"osd":2,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.77300000000000002,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97099999999999997,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.69599999999999995},"min":{"1min":0.55000000000000004,"5min":0.47599999999999998,"15min":0.41699999999999998},"max":{"1min":0.95299999999999996,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.76600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.751},"min":{"1min":0.504,"5min":0.499,"15min":0.46700000000000003},"max":{"1min":1.0529999999999999,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.89300000000000002,"15min":0.85699999999999998},"min":{"1min":0.55500000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92000000000000004,"5min":18.77,"15min":18.77},"last":0.66900000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.379,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0760000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.145,"5min":1.2070000000000001,"15min":1.2669999999999999},"last":0.65000000000000002}]},{"osd":5,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64300000000000002,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.004,"5min":1.508,"15min":1.508},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72999999999999998,"15min":0.77900000000000003},"min":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1719999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.57299999999999995}]},{"osd":7,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73899999999999999,"15min":0.78300000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2250000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.77000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.099,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.71199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.60099999999999998},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0149999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.48599999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88100000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56899999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.50700000000000001,"5min":0.42799999999999999,"15min":0.38300000000000001},"max":{"1min":1.0720000000000001,"5min":1.778,"15min":1.778},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68899999999999995,"15min":0.67200000000000004},"min":{"1min":0.45900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.1399999999999999,"5min":1.143,"15min":1.143},"last":0.751}]},{"osd":2,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.39600000000000002},"max":{"1min":1.0609999999999999,"5min":1.081,"15min":1.081},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.98999999999999999,"5min":1.823,"15min":1.823},"last":0.72599999999999998}]},{"osd":3,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.53200000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.095,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.91800000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64400000000000002,"15min":0.68400000000000005},"min":{"1min":0.44600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93799999999999994,"5min":1.137,"15min":1.1859999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72699999999999998,"15min":0.76900000000000002},"min":{"1min":0.48099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.016,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.98799999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.871,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67900000000000005,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.434,"15min":0.434},"max":{"1min":1.1479999999999999,"5min":1.268,"15min":1.268},"last":0.83799999999999997}]},{"osd":7,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.495,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0249999999999999,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.036},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.749,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.121,"5min":1.861,"15min":1.861},"last":0.90500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.67200000000000004},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":1.536,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.89000000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.51400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0249999999999999,"5min":1.05,"15min":1.077},"last":0.873}]},{"osd":2,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":1.236,"15min":1.302},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72099999999999997,"15min":0.751},"min":{"1min":0.57699999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.90400000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.873,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.752,"15min":0.78500000000000003},"min":{"1min":0.629,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.95299999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67800000000000005,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1020000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.45000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4550000000000001,"5min":2.544,"15min":2.544},"last":0.74199999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3129999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.749,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.83099999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.52700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.016,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77100000000000002,"15min":0.81000000000000005},"min":{"1min":0.55200000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.91800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47199999999999998,"15min":0.47299999999999998},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.74199999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.48999999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.62,"5min":1.294,"15min":1.294},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.442,"15min":0.442},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61199999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.626,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.41699999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.39000000000000001,"5min":0.38,"15min":0.36499999999999999},"max":{"1min":0.66800000000000004,"5min":1.456,"15min":1.456},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.72599999999999998,"5min":1.139,"15min":1.139},"last":0.47699999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.46400000000000002,"15min":0.45300000000000001},"max":{"1min":1.042,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.091,"5min":1.2509999999999999,"15min":1.2629999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.57099999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0640000000000001,"5min":1.2649999999999999,"15min":1.395},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.753,"15min":0.76800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0600000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77700000000000002,"15min":0.78900000000000003},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.0960000000000001,"15min":1.333},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.80000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.012,"5min":1.228,"15min":1.347},"last":0.64300000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.149,"5min":1.397,"15min":1.397},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.032,"5min":1.1859999999999999,"15min":1.363},"last":0.78800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.61899999999999999},"min":{"1min":0.28000000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.74199999999999999,"5min":1.036,"15min":3.931},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.67900000000000005},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":1.655,"15min":3.9649999999999999},"last":0.45200000000000001}]},{"osd":1,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50700000000000001,"15min":0.66100000000000003},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.65500000000000003,"5min":1.073,"15min":3.9929999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52600000000000002,"15min":0.66100000000000003},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73599999999999999,"5min":2.7650000000000001,"15min":4.0300000000000002},"last":0.38600000000000001}]},{"osd":3,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.68400000000000005,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57399999999999995,"15min":0.84399999999999997},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76400000000000001,"5min":1.7909999999999999,"15min":8.0990000000000002},"last":0.46899999999999997}]},{"osd":4,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78200000000000003,"15min":0.90600000000000003},"min":{"1min":0.60699999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0600000000000001,"5min":1.1240000000000001,"15min":4.431},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.88800000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99299999999999999,"5min":1.7050000000000001,"15min":4.1429999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92100000000000004,"5min":2.036,"15min":2.036},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81499999999999995,"15min":0.94099999999999995},"min":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.96699999999999997,"5min":1.8069999999999999,"15min":4.4790000000000001},"last":0.70799999999999996}]},{"osd":6,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.95299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0129999999999999,"5min":1.7310000000000001,"15min":4.6859999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.54900000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.1180000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.96599999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0880000000000001,"5min":3.4740000000000002,"15min":4.7270000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.038,"5min":1.323,"15min":1.609},"last":0.55200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.441,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.505,"5min":1.034,"15min":1.034},"last":0.313},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51000000000000001,"15min":0.504},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.56799999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.47199999999999998}]},{"osd":2,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.45100000000000001},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.51900000000000002},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60499999999999998,"5min":0.71499999999999997,"15min":1.3979999999999999},"last":0.36199999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53600000000000003,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69499999999999995,"5min":0.82199999999999995,"15min":1.4810000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63700000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.42799999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.68600000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47599999999999998},"max":{"1min":0.95599999999999996,"5min":1.143,"15min":1.143},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":1.0740000000000001,"15min":1.216},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.46600000000000003},"max":{"1min":0.90900000000000003,"5min":1.175,"15min":1.262},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89200000000000002,"5min":1.343,"15min":1.54},"last":0.79800000000000004}]},{"osd":6,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.89100000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.56699999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.874,"5min":1.254,"15min":1.333},"last":0.76900000000000002}]},{"osd":7,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.746,"15min":0.75700000000000001},"min":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92000000000000004,"5min":1.2090000000000001,"15min":1.5820000000000001},"last":0.747},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91500000000000004,"5min":1.377,"15min":1.377},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1867776,"data_stored":1147429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.25},"max":{"1min":0.55900000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68899999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29499999999999998},"max":{"1min":0.72099999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.59099999999999997,"5min":0.79700000000000004,"15min":0.85999999999999999},"last":0.57499999999999996}]},{"osd":3,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.624,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.36099999999999999},"max":{"1min":0.67000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54400000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80400000000000005,"15min":0.80900000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95699999999999996,"5min":1.365,"15min":1.365},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81200000000000006,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0009999999999999,"5min":1.181,"15min":1.181},"last":0.85099999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.747,"15min":0.755},"min":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.73999999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.78800000000000003,"15min":0.79700000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.79700000000000004},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.1040000000000001,"5min":1.323,"15min":1.323},"last":0.75700000000000001}]},{"osd":7,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81799999999999995,"15min":0.82399999999999995},"min":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0569999999999999,"5min":1.1180000000000001,"15min":1.123},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.95799999999999996,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.82099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:24:00.465 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-26T07:17:20.806918+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=26, tm_hour=7, tm_min=17, tm_sec=20, tm_wday=6, tm_yday=330, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=26, tm_hour=7, tm_min=23, tm_sec=54, tm_wday=6, tm_yday=330, tm_isdst=0) 2023-11-26T07:24:00.465 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-26T07:24:00.872 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:00.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:20.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-26T07:24:20.622 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f92bb640 1 Processor -- start 2023-11-26T07:24:20.623 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f92bb640 1 -- start start 2023-11-26T07:24:20.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f92bb640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79f406d920 0x7f79f4070df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f92bb640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f4145060 con 0x7f79f40ac630 2023-11-26T07:24:20.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f92bb640 1 -- --> v1:172.21.15.97:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f4071fd0 con 0x7f79f4144c80 2023-11-26T07:24:20.624 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f92bb640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f4071450 con 0x7f79f406d920 2023-11-26T07:24:20.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f37fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79f406d920 0x7f79f4070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79ea7fc640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f79f40ac630 0x7f79f40706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46628/0 (socket says 172.21.15.31:46628) 2023-11-26T07:24:20.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79ea7fc640 1 -- 172.21.15.31:0/1291487442 learned_addr learned my addr 172.21.15.31:0/1291487442 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:24:20.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f37fe640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f79f406d920 0x7f79f4070df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36570/0 (socket says 172.21.15.31:36570) 2023-11-26T07:24:20.625 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f37fe640 1 -- 172.21.15.31:0/1291487442 >> v1:172.21.15.97:6789/0 conn(0x7f79f4144c80 legacy=0x7f79f40abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f37fe640 1 -- 172.21.15.31:0/1291487442 >> v1:172.21.15.31:6789/0 conn(0x7f79f40ac630 legacy=0x7f79f40706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f37fe640 1 -- 172.21.15.31:0/1291487442 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f4071730 con 0x7f79f406d920 2023-11-26T07:24:20.626 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.620+0000 7f79f27fc640 1 -- 172.21.15.31:0/1291487442 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657074584 0 0) 0x7f79f4145060 con 0x7f79f40ac630 2023-11-26T07:24:20.627 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f37fe640 1 --2- 172.21.15.31:0/1291487442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 0x7f79f4070df0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009d90 tx=0x7f79ec01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24bbfec4d1f4a89 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f27fc640 1 -- 172.21.15.31:0/1291487442 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0a59e0 con 0x7f79f406d920 2023-11-26T07:24:20.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f27fc640 1 -- 172.21.15.31:0/1291487442 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec0a5b40 con 0x7f79f406d920 2023-11-26T07:24:20.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f27fc640 1 -- 172.21.15.31:0/1291487442 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0ad440 con 0x7f79f406d920 2023-11-26T07:24:20.628 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f92bb640 1 -- 172.21.15.31:0/1291487442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 msgr2=0x7f79f4070df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f92bb640 1 --2- 172.21.15.31:0/1291487442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 0x7f79f4070df0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009d90 tx=0x7f79ec01c920 comp rx=0 tx=0).stop 2023-11-26T07:24:20.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f92bb640 1 -- 172.21.15.31:0/1291487442 shutdown_connections 2023-11-26T07:24:20.629 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f92bb640 1 --2- 172.21.15.31:0/1291487442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 0x7f79f4070df0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f92bb640 1 -- 172.21.15.31:0/1291487442 >> 172.21.15.31:0/1291487442 conn(0x7f79f40b0c40 msgr2=0x7f79f40b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:20.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f92bb640 1 -- 172.21.15.31:0/1291487442 shutdown_connections 2023-11-26T07:24:20.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.624+0000 7f79f92bb640 1 -- 172.21.15.31:0/1291487442 wait complete. 2023-11-26T07:24:20.630 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f92bb640 1 Processor -- start 2023-11-26T07:24:20.631 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f92bb640 1 -- start start 2023-11-26T07:24:20.632 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f92bb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 0x7f79f4159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f92bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40ac630 0x7f79f413a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f92bb640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79f4144c80 0x7f79f415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f92bb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f413bde0 con 0x7f79f406d920 2023-11-26T07:24:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f92bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f413c060 con 0x7f79f40ac630 2023-11-26T07:24:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f92bb640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f413c340 con 0x7f79f4144c80 2023-11-26T07:24:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 0x7f79f4159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 0x7f79f4159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36586/0 (socket says 172.21.15.31:36586) 2023-11-26T07:24:20.633 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f2ffd640 1 -- 172.21.15.31:0/883884405 learned_addr learned my addr 172.21.15.31:0/883884405 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:24:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79ea7fc640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40ac630 0x7f79f413a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f37fe640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79f4144c80 0x7f79f415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f2ffd640 1 -- 172.21.15.31:0/883884405 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79f4144c80 msgr2=0x7f79f415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f2ffd640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79f4144c80 0x7f79f415aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f2ffd640 1 -- 172.21.15.31:0/883884405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40ac630 msgr2=0x7f79f413a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f2ffd640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40ac630 0x7f79f413a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.634 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f2ffd640 1 -- 172.21.15.31:0/883884405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec009a70 con 0x7f79f406d920 2023-11-26T07:24:20.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f37fe640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79f4144c80 0x7f79f415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-26T07:24:20.635 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f2ffd640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 0x7f79f4159f70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f79e400c3c0 tx=0x7f79e400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79ebfff640 1 -- 172.21.15.31:0/883884405 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40052d0 con 0x7f79f406d920 2023-11-26T07:24:20.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79ebfff640 1 -- 172.21.15.31:0/883884405 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e4005430 con 0x7f79f406d920 2023-11-26T07:24:20.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79ebfff640 1 -- 172.21.15.31:0/883884405 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4003840 con 0x7f79f406d920 2023-11-26T07:24:20.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f92bb640 1 -- 172.21.15.31:0/883884405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f413c620 con 0x7f79f406d920 2023-11-26T07:24:20.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.628+0000 7f79f92bb640 1 -- 172.21.15.31:0/883884405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f413cbe0 con 0x7f79f406d920 2023-11-26T07:24:20.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.632+0000 7f79f92bb640 1 -- 172.21.15.31:0/883884405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f40b1ba0 con 0x7f79f406d920 2023-11-26T07:24:20.636 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.632+0000 7f79ebfff640 1 -- 172.21.15.31:0/883884405 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f79e40039a0 con 0x7f79f406d920 2023-11-26T07:24:20.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.632+0000 7f79ebfff640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f79d8066740 0x7f79d8068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.632+0000 7f79ebfff640 1 -- 172.21.15.31:0/883884405 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f79e4083590 con 0x7f79f406d920 2023-11-26T07:24:20.637 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.632+0000 7f79ea7fc640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f79d8066740 0x7f79d8068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.638 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.632+0000 7f79ea7fc640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f79d8066740 0x7f79d8068c00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f79f409c0f0 tx=0x7f79dc007ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.640 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.636+0000 7f79ebfff640 1 -- 172.21.15.31:0/883884405 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79e4050980 con 0x7f79f406d920 2023-11-26T07:24:20.793 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.792+0000 7f79f92bb640 1 -- 172.21.15.31:0/883884405 --> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79f4066dd0 con 0x7f79d8066740 2023-11-26T07:24:20.795 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.792+0000 7f79ebfff640 1 -- 172.21.15.31:0/883884405 <== mgr.4110 v2:172.21.15.31:6836/179823611 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50154 (secure 0 0 0) 0x7f79f4066dd0 con 0x7f79d8066740 2023-11-26T07:24:20.796 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:24:20.797 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 -- 172.21.15.31:0/883884405 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f79d8066740 msgr2=0x7f79d8068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f79d8066740 0x7f79d8068c00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f79f409c0f0 tx=0x7f79dc007ba0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 -- 172.21.15.31:0/883884405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 msgr2=0x7f79f4159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 0x7f79f4159f70 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f79e400c3c0 tx=0x7f79e400c880 comp rx=0 tx=0).stop 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 -- 172.21.15.31:0/883884405 shutdown_connections 2023-11-26T07:24:20.798 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f79d8066740 0x7f79d8068c00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f79f4144c80 0x7f79f415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f79f40ac630 0x7f79f413a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 --2- 172.21.15.31:0/883884405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f406d920 0x7f79f4159f70 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 -- 172.21.15.31:0/883884405 >> 172.21.15.31:0/883884405 conn(0x7f79f40b0c40 msgr2=0x7f79f415ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 -- 172.21.15.31:0/883884405 shutdown_connections 2023-11-26T07:24:20.799 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.796+0000 7f79f92bb640 1 -- 172.21.15.31:0/883884405 wait complete. 2023-11-26T07:24:20.817 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-11-26T07:24:20.225006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":260580,"kb_used_data":11140,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714140,"statfs":{"total":773094113280,"available":772827279360,"internally_reserved":0,"allocated":11407360,"data_stored":5644357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002825"},"pg_stats":[{"pgid":"1.0","version":"102'130","reported_seq":284,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-26T07:24:02.291918+0000","last_change":"2023-11-26T07:24:02.291946+0000","last_active":"2023-11-26T07:24:02.291918+0000","last_peered":"2023-11-26T07:24:02.291918+0000","last_clean":"2023-11-26T07:24:02.291918+0000","last_became_active":"2023-11-26T07:17:22.380066+0000","last_became_peered":"2023-11-26T07:17:22.380066+0000","last_unstale":"2023-11-26T07:24:02.291918+0000","last_undegraded":"2023-11-26T07:24:02.291918+0000","last_fullsized":"2023-11-26T07:24:02.291918+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'130","last_scrub_stamp":"2023-11-26T07:24:02.291818+0000","last_deep_scrub":"102'130","last_deep_scrub_stamp":"2023-11-26T07:24:02.291818+0000","last_clean_scrub_stamp":"2023-11-26T07:24:02.291818+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-27T13:26:33.603239+0000","scrub_duration":0.0083140760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574936,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33004,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338836,"statfs":{"total":96636764160,"available":96602968064,"internally_reserved":0,"allocated":1871872,"data_stored":1148448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.48399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85599999999999998,"5min":1.008,"15min":1.008},"last":0.751},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":0.94399999999999995},"last":0.871}]},{"osd":1,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69999999999999996,"15min":0.68300000000000005},"min":{"1min":0.47399999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.90100000000000002,"5min":0.98799999999999999,"15min":1.0189999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67900000000000005,"15min":0.70899999999999996},"min":{"1min":0.504,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86799999999999999,"5min":0.93000000000000005,"15min":1.1519999999999999},"last":0.83599999999999997}]},{"osd":2,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0700000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70299999999999996,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":0.875,"5min":0.92400000000000004,"15min":1.016},"last":0.61699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73099999999999998,"15min":0.73899999999999999},"min":{"1min":0.53300000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.88500000000000001,"5min":1.0089999999999999,"15min":1.2270000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.63600000000000001,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93700000000000006,"5min":0.97999999999999998,"15min":1.1060000000000001},"last":0.91000000000000003}]},{"osd":4,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.69499999999999995},"min":{"1min":0.46600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.90400000000000003,"5min":0.91800000000000004,"15min":1.0309999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61099999999999999,"15min":0.67700000000000005},"min":{"1min":0.47099999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.81100000000000005,"5min":0.91700000000000004,"15min":1.1759999999999999},"last":0.85399999999999998}]},{"osd":5,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69599999999999995,"15min":0.76200000000000001},"min":{"1min":0.52500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.95899999999999996,"15min":1.5129999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.82999999999999996},"min":{"1min":0.60799999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.86599999999999999,"5min":0.96799999999999997,"15min":1.5780000000000001},"last":0.72399999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.749},"min":{"1min":0.53200000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.88800000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.81899999999999995},"min":{"1min":0.63,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92600000000000005,"5min":1.093,"15min":1.093},"last":0.92900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.83399999999999996,"15min":0.77000000000000002},"min":{"1min":0.51400000000000001,"5min":0.40100000000000002,"15min":0.39800000000000002},"max":{"1min":0.93400000000000005,"5min":17.222000000000001,"15min":17.222000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":1.2130000000000001,"15min":0.90400000000000003},"min":{"1min":0.443,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93899999999999995,"5min":54.737000000000002,"15min":54.737000000000002},"last":0.78000000000000003}]},{"osd":1,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69899999999999995,"15min":0.73799999999999999},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.38700000000000001},"max":{"1min":0.81699999999999995,"5min":2.1880000000000002,"15min":3.1520000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.77100000000000002,"15min":0.81299999999999994},"min":{"1min":0.50900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.02,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":0.89000000000000001}]},{"osd":2,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.77300000000000002,"15min":0.75800000000000001},"min":{"1min":0.56499999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97099999999999997,"5min":6.8479999999999999,"15min":6.8479999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.69599999999999995},"min":{"1min":0.55000000000000004,"5min":0.47599999999999998,"15min":0.41699999999999998},"max":{"1min":0.95299999999999996,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":0.87}]},{"osd":3,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.751},"min":{"1min":0.504,"5min":0.499,"15min":0.46700000000000003},"max":{"1min":1.0529999999999999,"5min":9.4870000000000001,"15min":9.4870000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.89300000000000002,"15min":0.85699999999999998},"min":{"1min":0.55500000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92000000000000004,"5min":18.77,"15min":18.77},"last":0.80000000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.379,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0760000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65400000000000003,"15min":0.70899999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.145,"5min":1.2070000000000001,"15min":1.2669999999999999},"last":0.57099999999999995}]},{"osd":5,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64300000000000002,"15min":0.70799999999999996},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.376},"max":{"1min":1.004,"5min":1.508,"15min":1.508},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72999999999999998,"15min":0.77900000000000003},"min":{"1min":0.45400000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1719999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.624}]},{"osd":7,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73899999999999999,"15min":0.78300000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.2250000000000001,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.71099999999999997,"15min":0.77000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.099,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.59199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.66800000000000004,"15min":0.60099999999999998},"min":{"1min":0.46400000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0149999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.48599999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88100000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.62}]},{"osd":1,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.50700000000000001,"5min":0.42799999999999999,"15min":0.38300000000000001},"max":{"1min":1.0720000000000001,"5min":1.778,"15min":1.778},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68899999999999995,"15min":0.67200000000000004},"min":{"1min":0.45900000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.1399999999999999,"5min":1.143,"15min":1.143},"last":0.752}]},{"osd":2,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.68700000000000006},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.39600000000000002},"max":{"1min":1.0609999999999999,"5min":1.081,"15min":1.081},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67400000000000004,"15min":0.69099999999999995},"min":{"1min":0.436,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.98999999999999999,"5min":1.823,"15min":1.823},"last":0.69499999999999995}]},{"osd":3,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.53200000000000003,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.095,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.77600000000000002}]},{"osd":4,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64400000000000002,"15min":0.68400000000000005},"min":{"1min":0.44600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.93799999999999994,"5min":1.137,"15min":1.1859999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72699999999999998,"15min":0.76900000000000002},"min":{"1min":0.48099999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.016,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.72499999999999998}]},{"osd":6,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.871,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67900000000000005,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.434,"15min":0.434},"max":{"1min":1.1479999999999999,"5min":1.268,"15min":1.268},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Nov 26 07:23:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.495,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0249999999999999,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.749,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.121,"5min":1.861,"15min":1.861},"last":0.91500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66400000000000003,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65700000000000003,"15min":0.67200000000000004},"min":{"1min":0.40899999999999997,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":1.536,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.79200000000000004}]},{"osd":1,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.68999999999999995,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.69399999999999995,"15min":0.70999999999999996},"min":{"1min":0.51400000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.0249999999999999,"5min":1.05,"15min":1.077},"last":0.872}]},{"osd":2,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.99199999999999999,"5min":1.236,"15min":1.302},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72099999999999997,"15min":0.751},"min":{"1min":0.57699999999999996,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":0.73699999999999999}]},{"osd":3,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.74099999999999999},"min":{"1min":0.49199999999999999,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.873,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.752,"15min":0.78500000000000003},"min":{"1min":0.629,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67800000000000005,"15min":0.69599999999999995},"min":{"1min":0.49299999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1020000000000001,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.45000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.4550000000000001,"5min":2.544,"15min":2.544},"last":0.71199999999999997}]},{"osd":6,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.72999999999999998},"min":{"1min":0.48299999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.3129999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.749,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.436,"15min":0.436},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.90800000000000003}]},{"osd":7,"last update":"Sun Nov 26 07:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.74099999999999999,"15min":0.745},"min":{"1min":0.52700000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.016,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.77100000000000002,"15min":0.81000000000000005},"min":{"1min":0.55200000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.58899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47199999999999998,"15min":0.47299999999999998},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66000000000000003,"5min":0.72999999999999998,"15min":0.74199999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.51200000000000001,"15min":0.48999999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.62,"5min":1.294,"15min":1.294},"last":0.56899999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.442,"15min":0.442},"min":{"1min":0.28999999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.61199999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34499999999999997,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.626,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54000000000000004,"15min":0.52400000000000002},"min":{"1min":0.39000000000000001,"5min":0.38,"15min":0.36499999999999999},"max":{"1min":0.66800000000000004,"5min":1.456,"15min":1.456},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.72599999999999998,"5min":1.139,"15min":1.139},"last":0.47499999999999998}]},{"osd":4,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.46400000000000002,"15min":0.45300000000000001},"max":{"1min":1.042,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.53100000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.091,"5min":1.2509999999999999,"15min":1.2629999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.746,"15min":0.77200000000000002},"min":{"1min":0.57099999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0640000000000001,"5min":1.2649999999999999,"15min":1.395},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.753,"15min":0.76800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0600000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77700000000000002,"15min":0.78900000000000003},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.085,"5min":1.0960000000000001,"15min":1.333},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.80000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.012,"5min":1.228,"15min":1.347},"last":0.73799999999999999}]},{"osd":7,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77400000000000002,"15min":0.79400000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.149,"5min":1.397,"15min":1.397},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.032,"5min":1.1859999999999999,"15min":1.363},"last":0.81299999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.61899999999999999},"min":{"1min":0.28000000000000003,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.74199999999999999,"5min":1.036,"15min":3.931},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53600000000000003,"15min":0.67900000000000005},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79100000000000004,"5min":1.655,"15min":3.9649999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50700000000000001,"15min":0.66100000000000003},"min":{"1min":0.33800000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.65500000000000003,"5min":1.073,"15min":3.9929999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52600000000000002,"15min":0.66100000000000003},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.73599999999999999,"5min":2.7650000000000001,"15min":4.0300000000000002},"last":0.64400000000000002}]},{"osd":3,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.318},"max":{"1min":0.68400000000000005,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.625},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57399999999999995,"15min":0.84399999999999997},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.76400000000000001,"5min":1.7909999999999999,"15min":8.0990000000000002},"last":0.74399999999999999}]},{"osd":4,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78200000000000003,"15min":0.90600000000000003},"min":{"1min":0.60699999999999998,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0600000000000001,"5min":1.1240000000000001,"15min":4.431},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.88800000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99299999999999999,"5min":1.7050000000000001,"15min":4.1429999999999998},"last":0.88900000000000001}]},{"osd":5,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76200000000000001,"15min":0.78600000000000003},"min":{"1min":0.501,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92100000000000004,"5min":2.036,"15min":2.036},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81499999999999995,"15min":0.94099999999999995},"min":{"1min":0.55900000000000005,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.96699999999999997,"5min":1.8069999999999999,"15min":4.4790000000000001},"last":0.85899999999999999}]},{"osd":6,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80100000000000005,"15min":0.95299999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0129999999999999,"5min":1.7310000000000001,"15min":4.6859999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.54900000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.1180000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.83399999999999996}]},{"osd":7,"last update":"Sun Nov 26 07:23:32 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.85199999999999998,"15min":0.96599999999999997},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0880000000000001,"5min":3.4740000000000002,"15min":4.7270000000000003},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.038,"5min":1.323,"15min":1.609},"last":0.91500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":558080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.441,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":0.505,"5min":1.034,"15min":1.034},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.51000000000000001,"15min":0.504},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.56799999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.45100000000000001},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.28699999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.49199999999999999,"15min":0.51900000000000002},"min":{"1min":0.376,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.60499999999999998,"5min":0.71499999999999997,"15min":1.3979999999999999},"last":0.58699999999999997}]},{"osd":3,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53600000000000003,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.69499999999999995,"5min":0.82199999999999995,"15min":1.4810000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.48199999999999998,"15min":0.498},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63700000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.68600000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47599999999999998},"max":{"1min":0.95599999999999996,"5min":1.143,"15min":1.143},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":1.0740000000000001,"15min":1.216},"last":0.86699999999999999}]},{"osd":5,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71299999999999997,"15min":0.70699999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.46600000000000003},"max":{"1min":0.90900000000000003,"5min":1.175,"15min":1.262},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.89200000000000002,"5min":1.343,"15min":1.54},"last":0.746}]},{"osd":6,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.73999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.89100000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.56699999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.874,"5min":1.254,"15min":1.333},"last":0.80400000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.746,"15min":0.75700000000000001},"min":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.92000000000000004,"5min":1.2090000000000001,"15min":1.5820000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.752},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91500000000000004,"5min":1.377,"15min":1.377},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1867776,"data_stored":1147429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.42099999999999999,"15min":0.44500000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.25},"max":{"1min":0.55900000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.377},{"interface":"front","average":{"1min":0.434,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.307,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.68899999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.40300000000000002}]},{"osd":2,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.29499999999999998},"max":{"1min":0.72099999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.36499999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.59099999999999997,"5min":0.79700000000000004,"15min":0.85999999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48399999999999999,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.624,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.36099999999999999},"max":{"1min":0.67000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55000000000000004}]},{"osd":4,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80400000000000005,"15min":0.80900000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95699999999999996,"5min":1.365,"15min":1.365},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81200000000000006,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0009999999999999,"5min":1.181,"15min":1.181},"last":0.80400000000000005}]},{"osd":5,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.747,"15min":0.755},"min":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46999999999999997},"max":{"1min":0.97499999999999998,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.629}]},{"osd":6,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.78800000000000003,"15min":0.79700000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.94299999999999995,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.79700000000000004},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.503},"max":{"1min":1.1040000000000001,"5min":1.323,"15min":1.323},"last":0.67900000000000005}]},{"osd":7,"last update":"Sun Nov 26 07:23:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81799999999999995,"15min":0.82399999999999995},"min":{"1min":0.56599999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0569999999999999,"5min":1.1180000000000001,"15min":1.123},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82199999999999995,"15min":0.81599999999999995},"min":{"1min":0.56100000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.95799999999999996,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.86299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-26T07:24:20.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-26T07:24:20.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984fafe640 1 Processor -- start 2023-11-26T07:24:20.959 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984fafe640 1 -- start start 2023-11-26T07:24:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984fafe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9848161f20 0x7f98480cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.960 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984fafe640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7f98480d02f0 0x7f98480cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984fafe640 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98480d0740 con 0x7f984815d7c0 2023-11-26T07:24:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984fafe640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98480d0f90 con 0x7f98480d02f0 2023-11-26T07:24:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984fafe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98480d11a0 con 0x7f9848161f20 2023-11-26T07:24:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984d89a640 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f984815d7c0 0x7f98480d3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:46636/0 (socket says 172.21.15.31:46636) 2023-11-26T07:24:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984d89a640 1 -- 172.21.15.31:0/934881452 learned_addr learned my addr 172.21.15.31:0/934881452 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-11-26T07:24:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984d099640 1 --2- 172.21.15.31:0/934881452 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9848161f20 0x7f98480cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984e09b640 1 --2- 172.21.15.31:0/934881452 >> v2:172.21.15.97:3300/0 conn(0x7f98480d02f0 0x7f98480cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984c898640 1 -- 172.21.15.31:0/934881452 <== mon.0 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897139545 0 0) 0x7f9844001120 con 0x7f984815d7c0 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984c898640 1 -- 172.21.15.31:0/934881452 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9834003530 con 0x7f984815d7c0 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984d099640 1 -- 172.21.15.31:0/934881452 >> v2:172.21.15.97:3300/0 conn(0x7f98480d02f0 msgr2=0x7f98480cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:20.962 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984d099640 1 --2- 172.21.15.31:0/934881452 >> v2:172.21.15.97:3300/0 conn(0x7f98480d02f0 0x7f98480cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984d099640 1 -- 172.21.15.31:0/934881452 >> v1:172.21.15.31:6789/0 conn(0x7f984815d7c0 legacy=0x7f98480d3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.963 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.956+0000 7f984d099640 1 -- 172.21.15.31:0/934881452 --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98480d1480 con 0x7f9848161f20 2023-11-26T07:24:20.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984d099640 1 --2- 172.21.15.31:0/934881452 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9848161f20 0x7f98480cfdb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98380097a0 tx=0x7f983801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=492c5b9e59b5a753 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984c898640 1 -- 172.21.15.31:0/934881452 <== mon.2 v2:172.21.15.97:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98380ac600 con 0x7f9848161f20 2023-11-26T07:24:20.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984c898640 1 -- 172.21.15.31:0/934881452 <== mon.2 v2:172.21.15.97:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98380a8050 con 0x7f9848161f20 2023-11-26T07:24:20.964 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984c898640 1 -- 172.21.15.31:0/934881452 <== mon.2 v2:172.21.15.97:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98380ac960 con 0x7f9848161f20 2023-11-26T07:24:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984fafe640 1 -- 172.21.15.31:0/934881452 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9848161f20 msgr2=0x7f98480cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984fafe640 1 --2- 172.21.15.31:0/934881452 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9848161f20 0x7f98480cfdb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98380097a0 tx=0x7f983801c3a0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984fafe640 1 -- 172.21.15.31:0/934881452 shutdown_connections 2023-11-26T07:24:20.965 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984fafe640 1 --2- 172.21.15.31:0/934881452 >> v2:172.21.15.97:3300/0 conn(0x7f98480d02f0 0x7f98480cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984fafe640 1 --2- 172.21.15.31:0/934881452 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f9848161f20 0x7f98480cfdb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984fafe640 1 -- 172.21.15.31:0/934881452 >> 172.21.15.31:0/934881452 conn(0x7f984808afc0 msgr2=0x7f98480d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984fafe640 1 -- 172.21.15.31:0/934881452 shutdown_connections 2023-11-26T07:24:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.960+0000 7f984fafe640 1 -- 172.21.15.31:0/934881452 wait complete. 2023-11-26T07:24:20.966 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984fafe640 1 Processor -- start 2023-11-26T07:24:20.967 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984fafe640 1 -- start start 2023-11-26T07:24:20.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984fafe640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f98480d02f0 0x7f984815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.968 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984fafe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f984815d7c0 0x7f9848158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984fafe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9848155970 0x7f98481f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984fafe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9848160710 con 0x7f9848155970 2023-11-26T07:24:20.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984fafe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98481f6690 con 0x7f984815d7c0 2023-11-26T07:24:20.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984fafe640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9848155e20 con 0x7f98480d02f0 2023-11-26T07:24:20.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984e09b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9848155970 0x7f98481f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984e09b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9848155970 0x7f98481f7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36588/0 (socket says 172.21.15.31:36588) 2023-11-26T07:24:20.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984e09b640 1 -- 172.21.15.31:0/2141073777 learned_addr learned my addr 172.21.15.31:0/2141073777 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:24:20.969 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984d099640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f984815d7c0 0x7f9848158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984d89a640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f98480d02f0 0x7f984815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984e09b640 1 -- 172.21.15.31:0/2141073777 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f98480d02f0 msgr2=0x7f984815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984e09b640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f98480d02f0 0x7f984815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984e09b640 1 -- 172.21.15.31:0/2141073777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f984815d7c0 msgr2=0x7f9848158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:20.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984e09b640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f984815d7c0 0x7f9848158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:20.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984e09b640 1 -- 172.21.15.31:0/2141073777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380093b0 con 0x7f9848155970 2023-11-26T07:24:20.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984d89a640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f98480d02f0 0x7f984815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-26T07:24:20.970 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984e09b640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9848155970 0x7f98481f7960 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9840009dd0 tx=0x7f984000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f983e7fc640 1 -- 172.21.15.31:0/2141073777 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840014480 con 0x7f9848155970 2023-11-26T07:24:20.971 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f983e7fc640 1 -- 172.21.15.31:0/2141073777 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984000f040 con 0x7f9848155970 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f983e7fc640 1 -- 172.21.15.31:0/2141073777 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840014840 con 0x7f9848155970 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984fafe640 1 -- 172.21.15.31:0/2141073777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984815f450 con 0x7f9848155970 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.964+0000 7f984fafe640 1 -- 172.21.15.31:0/2141073777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984815fa10 con 0x7f9848155970 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.968+0000 7f984fafe640 1 -- 172.21.15.31:0/2141073777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984808b5d0 con 0x7f9848155970 2023-11-26T07:24:20.972 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.968+0000 7f983e7fc640 1 -- 172.21.15.31:0/2141073777 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7f98400149a0 con 0x7f9848155970 2023-11-26T07:24:20.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.968+0000 7f983e7fc640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f982c066770 0x7f982c068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:20.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.968+0000 7f983e7fc640 1 -- 172.21.15.31:0/2141073777 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9840084f00 con 0x7f9848155970 2023-11-26T07:24:20.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.968+0000 7f984d89a640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f982c066770 0x7f982c068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:20.973 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.968+0000 7f984d89a640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f982c066770 0x7f982c068c30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9844005890 tx=0x7f9844009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:20.976 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:20.972+0000 7f983e7fc640 1 -- 172.21.15.31:0/2141073777 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98400521e0 con 0x7f9848155970 2023-11-26T07:24:21.134 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.132+0000 7f984fafe640 1 -- 172.21.15.31:0/2141073777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f98481f64b0 con 0x7f9848155970 2023-11-26T07:24:21.139 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.136+0000 7f983e7fc640 1 -- 172.21.15.31:0/2141073777 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f9840055800 con 0x7f9848155970 2023-11-26T07:24:21.140 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.136+0000 7f983e7fc640 1 -- 172.21.15.31:0/2141073777 <== mon.0 v2:172.21.15.31:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f984005c110 con 0x7f9848155970 2023-11-26T07:24:21.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 -- 172.21.15.31:0/2141073777 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f982c066770 msgr2=0x7f982c068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f982c066770 0x7f982c068c30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9844005890 tx=0x7f9844009040 comp rx=0 tx=0).stop 2023-11-26T07:24:21.142 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 -- 172.21.15.31:0/2141073777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9848155970 msgr2=0x7f98481f7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9848155970 0x7f98481f7960 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9840009dd0 tx=0x7f984000c490 comp rx=0 tx=0).stop 2023-11-26T07:24:21.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 -- 172.21.15.31:0/2141073777 shutdown_connections 2023-11-26T07:24:21.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7f982c066770 0x7f982c068c30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9848155970 0x7f98481f7960 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f984815d7c0 0x7f9848158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 --2- 172.21.15.31:0/2141073777 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7f98480d02f0 0x7f984815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 -- 172.21.15.31:0/2141073777 >> 172.21.15.31:0/2141073777 conn(0x7f984808afc0 msgr2=0x7f98480cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:21.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 -- 172.21.15.31:0/2141073777 shutdown_connections 2023-11-26T07:24:21.143 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.140+0000 7f984fafe640 1 -- 172.21.15.31:0/2141073777 wait complete. 2023-11-26T07:24:21.162 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-26T07:24:21.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-26T07:24:21.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.300+0000 7fa90525a640 1 Processor -- start 2023-11-26T07:24:21.305 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa90525a640 1 -- start start 2023-11-26T07:24:21.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa90525a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa900144c80 0x7fa9000abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa90525a640 1 --2- >> v2:172.21.15.97:3300/0 conn(0x7fa9000ac460 0x7fa9000706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa90525a640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa900070c20 0x7fa90015eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.307 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa90525a640 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa900145060 con 0x7fa900070c20 2023-11-26T07:24:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa90525a640 1 -- --> v2:172.21.15.97:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa900071fd0 con 0x7fa9000ac460 2023-11-26T07:24:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa90525a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9000ac8c0 con 0x7fa900144c80 2023-11-26T07:24:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa8ff59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa900070c20 0x7fa90015eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa8ff59e640 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa900070c20 0x7fa90015eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36600/0 (socket says 172.21.15.31:36600) 2023-11-26T07:24:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa8ff59e640 1 -- 172.21.15.31:0/2491674452 learned_addr learned my addr 172.21.15.31:0/2491674452 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:24:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa8fed9d640 1 --2- 172.21.15.31:0/2491674452 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa900144c80 0x7fa9000abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.308 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa8f7fff640 1 --2- 172.21.15.31:0/2491674452 >> v2:172.21.15.97:3300/0 conn(0x7fa9000ac460 0x7fa9000706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa8ff59e640 1 -- 172.21.15.31:0/2491674452 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa900144c80 msgr2=0x7fa9000abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa8ff59e640 1 --2- 172.21.15.31:0/2491674452 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa900144c80 0x7fa9000abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa8ff59e640 1 -- 172.21.15.31:0/2491674452 >> v2:172.21.15.97:3300/0 conn(0x7fa9000ac460 msgr2=0x7fa9000706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-26T07:24:21.309 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa8ff59e640 1 --2- 172.21.15.31:0/2491674452 >> v2:172.21.15.97:3300/0 conn(0x7fa9000ac460 0x7fa9000706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.304+0000 7fa8ff59e640 1 -- 172.21.15.31:0/2491674452 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa900071450 con 0x7fa900070c20 2023-11-26T07:24:21.310 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa8ff59e640 1 --2- 172.21.15.31:0/2491674452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900070c20 0x7fa90015eed0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa90006bf70 tx=0x7fa8f801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61a6721266b811d server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa8fe59c640 1 -- 172.21.15.31:0/2491674452 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80a5560 con 0x7fa900070c20 2023-11-26T07:24:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa8fe59c640 1 -- 172.21.15.31:0/2491674452 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa8f80a56c0 con 0x7fa900070c20 2023-11-26T07:24:21.311 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa8fe59c640 1 -- 172.21.15.31:0/2491674452 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80a5a70 con 0x7fa900070c20 2023-11-26T07:24:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa90525a640 1 -- 172.21.15.31:0/2491674452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900070c20 msgr2=0x7fa90015eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa90525a640 1 --2- 172.21.15.31:0/2491674452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900070c20 0x7fa90015eed0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa90006bf70 tx=0x7fa8f801c6e0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa90525a640 1 -- 172.21.15.31:0/2491674452 shutdown_connections 2023-11-26T07:24:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa90525a640 1 --2- 172.21.15.31:0/2491674452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900070c20 0x7fa90015eed0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa90525a640 1 --2- 172.21.15.31:0/2491674452 >> v2:172.21.15.97:3300/0 conn(0x7fa9000ac460 0x7fa9000706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa90525a640 1 --2- 172.21.15.31:0/2491674452 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa900144c80 0x7fa9000abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.312 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa90525a640 1 -- 172.21.15.31:0/2491674452 >> 172.21.15.31:0/2491674452 conn(0x7fa9000b0c40 msgr2=0x7fa9000b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:21.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa90525a640 1 -- 172.21.15.31:0/2491674452 shutdown_connections 2023-11-26T07:24:21.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa90525a640 1 -- 172.21.15.31:0/2491674452 wait complete. 2023-11-26T07:24:21.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.308+0000 7fa90525a640 1 Processor -- start 2023-11-26T07:24:21.313 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa90525a640 1 -- start start 2023-11-26T07:24:21.314 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa90525a640 1 --2- >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa900070c20 0x7fa900158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa90525a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9000ac460 0x7fa9001415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa90525a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900144c80 0x7fa90015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa90525a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9001410a0 con 0x7fa900144c80 2023-11-26T07:24:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa90525a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa900141ae0 con 0x7fa9000ac460 2023-11-26T07:24:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa90525a640 1 -- --> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa900157760 con 0x7fa900070c20 2023-11-26T07:24:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8ff59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900144c80 0x7fa90015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8ff59e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900144c80 0x7fa90015ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36602/0 (socket says 172.21.15.31:36602) 2023-11-26T07:24:21.315 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8ff59e640 1 -- 172.21.15.31:0/3810737709 learned_addr learned my addr 172.21.15.31:0/3810737709 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-11-26T07:24:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8fed9d640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa900070c20 0x7fa900158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8f7fff640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9000ac460 0x7fa9001415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8ff59e640 1 -- 172.21.15.31:0/3810737709 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa900070c20 msgr2=0x7fa900158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8ff59e640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa900070c20 0x7fa900158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8ff59e640 1 -- 172.21.15.31:0/3810737709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9000ac460 msgr2=0x7fa9001415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8ff59e640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9000ac460 0x7fa9001415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.317 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8ff59e640 1 -- 172.21.15.31:0/3810737709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f8009b30 con 0x7fa900144c80 2023-11-26T07:24:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8ff59e640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900144c80 0x7fa90015ad00 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa8f801ad90 tx=0x7fa8f8019480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8f77fe640 1 -- 172.21.15.31:0/3810737709 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80a5560 con 0x7fa900144c80 2023-11-26T07:24:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8f77fe640 1 -- 172.21.15.31:0/3810737709 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa8f80b8690 con 0x7fa900144c80 2023-11-26T07:24:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa8f77fe640 1 -- 172.21.15.31:0/3810737709 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80b8a60 con 0x7fa900144c80 2023-11-26T07:24:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa90525a640 1 -- 172.21.15.31:0/3810737709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa900157a40 con 0x7fa900144c80 2023-11-26T07:24:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa90525a640 1 -- 172.21.15.31:0/3810737709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa900158000 con 0x7fa900144c80 2023-11-26T07:24:21.318 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.312+0000 7fa90525a640 1 -- 172.21.15.31:0/3810737709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9000b1ba0 con 0x7fa900144c80 2023-11-26T07:24:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.316+0000 7fa8f77fe640 1 -- 172.21.15.31:0/3810737709 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 86107+0+0 (secure 0 0 0) 0x7fa8f80b8bc0 con 0x7fa900144c80 2023-11-26T07:24:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.316+0000 7fa8f77fe640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa8e0066740 0x7fa8e0068c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-26T07:24:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.316+0000 7fa8f77fe640 1 -- 172.21.15.31:0/3810737709 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa8f811f1f0 con 0x7fa900144c80 2023-11-26T07:24:21.319 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.316+0000 7fa8fed9d640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa8e0066740 0x7fa8e0068c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-26T07:24:21.320 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.316+0000 7fa8fed9d640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa8e0066740 0x7fa8e0068c00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa900159320 tx=0x7fa8f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-26T07:24:21.323 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.320+0000 7fa8f77fe640 1 -- 172.21.15.31:0/3810737709 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa8f80ec5e0 con 0x7fa900144c80 2023-11-26T07:24:21.512 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.508+0000 7fa90525a640 1 -- 172.21.15.31:0/3810737709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa90004e850 con 0x7fa900144c80 2023-11-26T07:24:21.513 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.508+0000 7fa8f77fe640 1 -- 172.21.15.31:0/3810737709 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fa8f80bce90 con 0x7fa900144c80 2023-11-26T07:24:21.513 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-11-26T07:24:21.514 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/3797004646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":3797004646},{"type":"v1","addr":"172.21.15.97:6833","nonce":3797004646}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":7180,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/2675576819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":2675576819},{"type":"v1","addr":"172.21.15.97:6835","nonce":2675576819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/1710103664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":1710103664},{"type":"v1","addr":"172.21.15.31:6834","nonce":1710103664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8043,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/3586296606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":3586296606},{"type":"v1","addr":"172.21.15.31:6835","nonce":3586296606}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[]} 2023-11-26T07:24:21.514 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 75 2023-11-26T07:24:21.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 -- 172.21.15.31:0/3810737709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa8e0066740 msgr2=0x7fa8e0068c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa8e0066740 0x7fa8e0068c00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa900159320 tx=0x7fa8f0008040 comp rx=0 tx=0).stop 2023-11-26T07:24:21.515 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 -- 172.21.15.31:0/3810737709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900144c80 msgr2=0x7fa90015ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-26T07:24:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900144c80 0x7fa90015ad00 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa8f801ad90 tx=0x7fa8f8019480 comp rx=0 tx=0).stop 2023-11-26T07:24:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 -- 172.21.15.31:0/3810737709 shutdown_connections 2023-11-26T07:24:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.31:6836/179823611,v1:172.21.15.31:6837/179823611] conn(0x7fa8e0066740 0x7fa8e0068c00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa900144c80 0x7fa90015ad00 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9000ac460 0x7fa9001415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 --2- 172.21.15.31:0/3810737709 >> [v2:172.21.15.97:3301/0,v1:172.21.15.97:6790/0] conn(0x7fa900070c20 0x7fa900158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-26T07:24:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 -- 172.21.15.31:0/3810737709 >> 172.21.15.31:0/3810737709 conn(0x7fa9000b0c40 msgr2=0x7fa9000aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-26T07:24:21.516 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 -- 172.21.15.31:0/3810737709 shutdown_connections 2023-11-26T07:24:21.517 INFO:teuthology.orchestra.run.smithi031.stderr:2023-11-26T07:24:21.512+0000 7fa90525a640 1 -- 172.21.15.31:0/3810737709 wait complete. 2023-11-26T07:24:21.535 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-26T07:24:21.535 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-26T07:24:21.535 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-26T07:24:21.535 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:21.547 INFO:tasks.ceph.mds.a:Stopped 2023-11-26T07:24:21.547 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-26T07:24:21.548 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:21.601 INFO:tasks.ceph.mds.c:Stopped 2023-11-26T07:24:21.601 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-26T07:24:21.601 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:21.616 INFO:tasks.ceph.mds.b:Stopped 2023-11-26T07:24:21.616 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-26T07:24:21.617 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:21.674 INFO:tasks.ceph.mds.d:Stopped 2023-11-26T07:24:21.674 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-26T07:24:21.674 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-26T07:24:21.675 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:21.742 INFO:tasks.ceph.osd.0:Stopped 2023-11-26T07:24:21.742 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-26T07:24:21.742 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:21.813 INFO:tasks.ceph.osd.1:Stopped 2023-11-26T07:24:21.813 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-26T07:24:21.813 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:21.897 INFO:tasks.ceph.osd.2:Stopped 2023-11-26T07:24:21.897 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-26T07:24:21.897 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:23.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-11-26T07:24:27.999 INFO:tasks.ceph.osd.3:Stopped 2023-11-26T07:24:27.999 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-26T07:24:28.000 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:28.066 INFO:tasks.ceph.osd.4:Stopped 2023-11-26T07:24:28.067 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-26T07:24:28.067 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:28.883 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-11-26T07:24:28.884 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-11-26T07:24:30.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:30.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:34.169 INFO:tasks.ceph.osd.5:Stopped 2023-11-26T07:24:34.169 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-26T07:24:34.169 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:34.225 INFO:tasks.ceph.osd.6:Stopped 2023-11-26T07:24:34.225 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-26T07:24:34.225 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:34.313 INFO:tasks.ceph.osd.7:Stopped 2023-11-26T07:24:34.313 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-26T07:24:34.314 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-26T07:24:34.314 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:34.376 INFO:tasks.ceph.mgr.y:Stopped 2023-11-26T07:24:34.376 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-26T07:24:34.376 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:34.431 INFO:tasks.ceph.mgr.x:Stopped 2023-11-26T07:24:34.431 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-26T07:24:34.432 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-26T07:24:34.432 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:34.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~6s 2023-11-26T07:24:34.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-11-26T07:24:34.452 INFO:tasks.ceph.mon.a:Stopped 2023-11-26T07:24:34.452 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-26T07:24:34.452 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:34.468 INFO:tasks.ceph.mon.b:Stopped 2023-11-26T07:24:34.469 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-26T07:24:34.469 INFO:teuthology.orchestra.run:waiting for 300 2023-11-26T07:24:34.533 INFO:tasks.ceph.mon.c:Stopped 2023-11-26T07:24:34.533 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-26T07:24:34.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-26T07:24:34.570 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:24:34.570 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-26T07:24:34.972 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:24:34.972 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-26T07:24:35.160 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:24:35.161 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-26T07:24:35.319 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:24:35.319 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-26T07:24:35.477 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:24:35.478 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-26T07:24:35.711 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:24:35.711 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-26T07:24:35.921 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:24:35.921 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-26T07:24:36.137 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:24:36.138 DEBUG:teuthology.orchestra.run.smithi097:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-26T07:24:36.296 INFO:tasks.ceph:Archiving mon data... 2023-11-26T07:24:36.297 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348/data/mon.a.tgz 2023-11-26T07:24:36.298 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2023-11-26T07:24:36.307 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.0M5OUNIXKg 2023-11-26T07:24:36.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.0M5OUNIXKg 2023-11-26T07:24:36.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.0M5OUNIXKg 2023-11-26T07:24:36.582 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.0M5OUNIXKg is 750KB 2023-11-26T07:24:36.656 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.0M5OUNIXKg 2023-11-26T07:24:36.664 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348/data/mon.b.tgz 2023-11-26T07:24:36.665 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2023-11-26T07:24:36.674 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.wyLFaQqZ0b 2023-11-26T07:24:36.674 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.wyLFaQqZ0b 2023-11-26T07:24:36.871 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.wyLFaQqZ0b 2023-11-26T07:24:36.944 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.wyLFaQqZ0b is 756KB 2023-11-26T07:24:37.019 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.wyLFaQqZ0b 2023-11-26T07:24:37.027 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348/data/mon.c.tgz 2023-11-26T07:24:37.027 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2023-11-26T07:24:37.075 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.siJO3QSu1z 2023-11-26T07:24:37.076 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.siJO3QSu1z 2023-11-26T07:24:37.265 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.siJO3QSu1z 2023-11-26T07:24:37.340 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.siJO3QSu1z is 756KB 2023-11-26T07:24:37.410 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.siJO3QSu1z 2023-11-26T07:24:37.419 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-26T07:24:37.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-26T07:24:37.423 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-26T07:24:37.479 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-26T07:24:37.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-26T07:24:37.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-26T07:24:37.525 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-26T07:24:37.560 INFO:teuthology.orchestra.run.smithi097.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-26T07:24:37.566 INFO:tasks.ceph:Archiving crash dumps... 2023-11-26T07:24:37.568 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348/remote/smithi031/crash 2023-11-26T07:24:37.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-26T07:24:37.598 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348/remote/smithi097/crash 2023-11-26T07:24:37.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-26T07:24:37.626 INFO:tasks.ceph:Shutting down logrotate 2023-11-26T07:24:37.626 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:37.644 DEBUG:teuthology.orchestra.run.smithi097:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:37.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:37.732 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-26T07:24:37.770 INFO:tasks.ceph:Compressing logs... 2023-11-26T07:24:37.771 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-26T07:24:37.776 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-26T07:24:37.802 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56065.log 2023-11-26T07:24:37.803 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54021.log 2023-11-26T07:24:37.804 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44537.log 2023-11-26T07:24:37.804 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56065.log.gz 2023-11-26T07:24:37.805 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51578.log 2023-11-26T07:24:37.805 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54021.log.gz 2023-11-26T07:24:37.806 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44459.log 2023-11-26T07:24:37.806 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44537.log.gz 2023-11-26T07:24:37.807 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51578.log: gzip -5 89.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51578.log.gz -- 2023-11-26T07:24:37.807 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.53048.log 2023-11-26T07:24:37.808 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48348.log 2023-11-26T07:24:37.808 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44459.log.gz 2023-11-26T07:24:37.809 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53048.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.53048.log.gz --verbose 2023-11-26T07:24:37.809 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph-client.admin.55643.log 2023-11-26T07:24:37.810 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48296.log 2023-11-26T07:24:37.810 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48348.log.gz 2023-11-26T07:24:37.811 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55643.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55643.log.gz 2023-11-26T07:24:37.811 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56855.log 2023-11-26T07:24:37.812 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46863.log 2023-11-26T07:24:37.812 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48296.log.gz 2023-11-26T07:24:37.813 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56855.log: gzip -5 --verbose 88.1% -- -- replaced with /var/log/ceph/ceph-client.admin.56855.log.gz /var/log/ceph/ceph-client.admin.54728.log 2023-11-26T07:24:37.813 INFO:teuthology.orchestra.run.smithi031.stderr: 2023-11-26T07:24:37.814 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50312.log 2023-11-26T07:24:37.814 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46863.log.gz 2023-11-26T07:24:37.815 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54728.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54728.log.gz 2023-11-26T07:24:37.815 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54047.log 2023-11-26T07:24:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50312.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50312.log.gzgzip 2023-11-26T07:24:37.816 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43361.log 2023-11-26T07:24:37.817 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54047.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54047.log.gzgzip 2023-11-26T07:24:37.817 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54214.log 2023-11-26T07:24:37.818 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43361.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.58314.log -- replaced with /var/log/ceph/ceph-client.admin.43361.log.gz 2023-11-26T07:24:37.818 INFO:teuthology.orchestra.run.smithi031.stderr: 2023-11-26T07:24:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54214.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54214.log.gz 2023-11-26T07:24:37.819 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55326.log 2023-11-26T07:24:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58465.log 2023-11-26T07:24:37.820 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58314.log.gz 2023-11-26T07:24:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55326.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55326.log.gz 2023-11-26T07:24:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58366.log 2023-11-26T07:24:37.822 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56611.log 89.6% 2023-11-26T07:24:37.822 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58465.log.gz 2023-11-26T07:24:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58366.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58366.log.gz 2023-11-26T07:24:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51945.log 2023-11-26T07:24:37.824 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49112.log 2023-11-26T07:24:37.824 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56611.log.gz 2023-11-26T07:24:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51945.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51945.log.gz 2023-11-26T07:24:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53528.log 2023-11-26T07:24:37.826 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49112.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.49112.log.gz 2023-11-26T07:24:37.826 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.52826.log 2023-11-26T07:24:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53528.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53528.log.gz 2023-11-26T07:24:37.827 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51310.log 2023-11-26T07:24:37.828 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52826.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-26T07:24:37.828 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52826.log.gz 2023-11-26T07:24:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51310.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51310.log.gz 2023-11-26T07:24:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57071.log 2023-11-26T07:24:37.830 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58777.log 2023-11-26T07:24:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57071.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57071.log.gz 2023-11-26T07:24:37.831 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49647.log 2023-11-26T07:24:37.832 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45442.log 2023-11-26T07:24:37.832 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58777.log.gz 2023-11-26T07:24:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49647.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49647.log.gz 2023-11-26T07:24:37.833 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47760.log 2023-11-26T07:24:37.834 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45442.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.45442.log.gz --verbose 2023-11-26T07:24:37.834 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph-client.admin.46837.log 2023-11-26T07:24:37.835 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47760.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47760.log.gz 2023-11-26T07:24:37.835 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57840.log 2023-11-26T07:24:37.836 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55805.log 87.8% 2023-11-26T07:24:37.836 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46837.log.gz 2023-11-26T07:24:37.837 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57840.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.57840.log.gz 2023-11-26T07:24:37.837 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47076.log 2023-11-26T07:24:37.841 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55805.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55805.log.gz 2023-11-26T07:24:37.841 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50572.log 2023-11-26T07:24:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54311.log 2023-11-26T07:24:37.843 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47076.log.gz 2023-11-26T07:24:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52104.log 2023-11-26T07:24:37.843 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50572.log.gz 2023-11-26T07:24:37.843 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58411.log 2023-11-26T07:24:37.844 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54311.log.gz 2023-11-26T07:24:37.844 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52104.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52104.log.gz 2023-11-26T07:24:37.844 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49595.log 2023-11-26T07:24:37.844 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44297.log 2023-11-26T07:24:37.845 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48445.log 2023-11-26T07:24:37.845 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58411.log.gz 2023-11-26T07:24:37.846 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47590.log 2023-11-26T07:24:37.846 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49595.log.gz 2023-11-26T07:24:37.848 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47400.log 2023-11-26T07:24:37.848 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48445.log.gz 2023-11-26T07:24:37.848 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47590.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47590.log.gz 2023-11-26T07:24:37.848 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54785.log 2023-11-26T07:24:37.849 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44165.log 2023-11-26T07:24:37.849 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.1.44297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44297.log.gz 2023-11-26T07:24:37.849 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38284.log 2023-11-26T07:24:37.849 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.3.44165.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.44165.log.gz 2023-11-26T07:24:37.849 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44570.log 2023-11-26T07:24:37.850 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44441.log 2023-11-26T07:24:37.850 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38284.log.gz 2023-11-26T07:24:37.850 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.1.44570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44570.log.gz 2023-11-26T07:24:37.850 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-26T07:24:37.851 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.1.44441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44441.log.gz 2023-11-26T07:24:37.852 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-26T07:24:37.852 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38188.log 2023-11-26T07:24:37.853 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53719.log 2023-11-26T07:24:37.853 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47400.log.gz 2023-11-26T07:24:37.853 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46058.log 2023-11-26T07:24:37.854 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54785.log.gz 2023-11-26T07:24:37.854 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46349.log 2023-11-26T07:24:37.854 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53719.log.gz 2023-11-26T07:24:37.854 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46058.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46058.log.gz 2023-11-26T07:24:37.854 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44070.log 2023-11-26T07:24:37.854 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57097.log 2023-11-26T07:24:37.854 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2023-11-26T07:24:37.854 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55055.log 2023-11-26T07:24:37.855 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44070.log.gz 2023-11-26T07:24:37.857 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49512.log 2023-11-26T07:24:37.857 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57097.log.gz 2023-11-26T07:24:37.857 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53474.log 2023-11-26T07:24:37.857 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55055.log.gz 2023-11-26T07:24:37.857 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54467.log 2023-11-26T07:24:37.858 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49512.log.gz 2023-11-26T07:24:37.859 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38212.log 2023-11-26T07:24:37.859 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38188.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-26T07:24:37.859 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38188.log.gz 2023-11-26T07:24:37.859 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38332.log 2023-11-26T07:24:37.859 INFO:teuthology.orchestra.run.smithi097.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38212.log.gz 2023-11-26T07:24:37.859 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38260.log 2023-11-26T07:24:37.859 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38332.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38332.log.gz 2023-11-26T07:24:37.860 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38164.log 2023-11-26T07:24:37.860 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38260.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-26T07:24:37.860 INFO:teuthology.orchestra.run.smithi097.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38260.log.gz 2023-11-26T07:24:37.860 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38164.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38164.log.gz 2023-11-26T07:24:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44711.log 2023-11-26T07:24:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-26T07:24:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.1.44711.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44711.log.gz 2023-11-26T07:24:37.862 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44864.log 2023-11-26T07:24:37.863 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53474.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53474.log.gz 2023-11-26T07:24:37.863 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55726.log 2023-11-26T07:24:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54467.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-26T07:24:37.864 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54467.log.gz 2023-11-26T07:24:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54363.log 2023-11-26T07:24:37.864 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55726.log.gz 2023-11-26T07:24:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51336.log 2023-11-26T07:24:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54363.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54363.log.gz 2023-11-26T07:24:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45260.log 2023-11-26T07:24:37.864 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51336.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51336.log.gz 2023-11-26T07:24:37.865 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52656.log 2023-11-26T07:24:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38236.log 2023-11-26T07:24:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.1.44864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44864.log.gz 2023-11-26T07:24:37.867 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-26T07:24:37.867 INFO:teuthology.orchestra.run.smithi097.stderr: 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-26T07:24:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.admin.38236.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38236.log.gz 2023-11-26T07:24:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-26T07:24:37.868 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-26T07:24:37.869 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45260.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45260.log.gz 2023-11-26T07:24:37.869 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57003.log 2023-11-26T07:24:37.869 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52656.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52656.log.gz 2023-11-26T07:24:37.869 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48114.log 2023-11-26T07:24:37.869 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.57003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57003.log.gz 2023-11-26T07:24:37.869 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50520.log 2023-11-26T07:24:37.870 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47788.log 2023-11-26T07:24:37.870 INFO:teuthology.orchestra.run.smithi031.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48114.log.gz 2023-11-26T07:24:37.870 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50520.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50520.log.gz 2023-11-26T07:24:37.870 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58080.log 2023-11-26T07:24:37.871 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-26T07:24:37.872 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46941.log 2023-11-26T07:24:37.872 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47788.log.gz 2023-11-26T07:24:37.872 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58080.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58080.log.gz 2023-11-26T07:24:37.872 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44263.log 2023-11-26T07:24:37.873 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46401.log 2023-11-26T07:24:37.874 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46941.log.gz 2023-11-26T07:24:37.874 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-26T07:24:37.875 INFO:teuthology.orchestra.run.smithi097.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-26T07:24:37.875 INFO:teuthology.orchestra.run.smithi097.stderr: 93.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-26T07:24:37.875 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38308.log 2023-11-26T07:24:37.876 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44263.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44263.log.gz 2023-11-26T07:24:37.876 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46084.log 2023-11-26T07:24:37.876 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46401.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46401.log.gz 2023-11-26T07:24:37.876 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45052.log 2023-11-26T07:24:37.878 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.38308.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38308.log.gz 2023-11-26T07:24:37.878 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-26T07:24:37.878 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46084.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46084.log.gz 2023-11-26T07:24:37.878 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48140.log 2023-11-26T07:24:37.879 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45052.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45052.log.gz 2023-11-26T07:24:37.879 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51500.log 2023-11-26T07:24:37.880 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46915.log 2023-11-26T07:24:37.880 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48140.log.gz 2023-11-26T07:24:37.880 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51500.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51500.log.gz 2023-11-26T07:24:37.880 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48497.log 2023-11-26T07:24:37.881 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46915.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46915.log.gz 2023-11-26T07:24:37.881 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50036.log 2023-11-26T07:24:37.882 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48497.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48497.log.gz 2023-11-26T07:24:37.882 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45652.log 2023-11-26T07:24:37.883 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50036.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50036.log.gz 2023-11-26T07:24:37.883 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51206.log 2023-11-26T07:24:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45652.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45652.log.gz 2023-11-26T07:24:37.884 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49008.log 2023-11-26T07:24:37.885 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44997.log 2023-11-26T07:24:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51206.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51206.log.gz 2023-11-26T07:24:37.885 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51064.log 2023-11-26T07:24:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49008.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49008.log.gz 2023-11-26T07:24:37.886 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45977.log 2023-11-26T07:24:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51064.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51064.log.gz 2023-11-26T07:24:37.887 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50955.log 2023-11-26T07:24:37.888 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45977.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45977.log.gz 2023-11-26T07:24:37.888 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52364.log 2023-11-26T07:24:37.889 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50955.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50955.log.gz 2023-11-26T07:24:37.890 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44080.log 2023-11-26T07:24:37.890 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56475.log 2023-11-26T07:24:37.891 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52364.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52364.log.gz 2023-11-26T07:24:37.891 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55591.log 2023-11-26T07:24:37.891 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.1.44997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44997.log.gz 2023-11-26T07:24:37.892 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56475.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56475.log.gz 2023-11-26T07:24:37.892 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49703.log 2023-11-26T07:24:37.893 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55591.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55591.log.gz 2023-11-26T07:24:37.893 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59011.log 2023-11-26T07:24:37.894 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49703.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49703.log.gz 2023-11-26T07:24:37.894 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47983.log 2023-11-26T07:24:37.895 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59011.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59011.log.gz 2023-11-26T07:24:37.895 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54623.log 2023-11-26T07:24:37.896 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47983.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47983.log.gz 2023-11-26T07:24:37.896 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48954.log 2023-11-26T07:24:37.897 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/ceph-client.1.44080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44080.log.gz 2023-11-26T07:24:37.897 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54623.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54623.log.gz 2023-11-26T07:24:37.898 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55700.log 2023-11-26T07:24:37.898 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48954.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48954.log.gz 2023-11-26T07:24:37.898 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57176.log 2023-11-26T07:24:37.899 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55700.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55700.log.gz 2023-11-26T07:24:37.899 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54649.log 2023-11-26T07:24:37.900 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57176.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57176.log.gz 2023-11-26T07:24:37.900 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54162.log 2023-11-26T07:24:37.901 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54649.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54649.log.gz 2023-11-26T07:24:37.902 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48009.log 2023-11-26T07:24:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54162.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54162.log.gz 2023-11-26T07:24:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59116.log 2023-11-26T07:24:37.904 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48009.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48009.log.gz 2023-11-26T07:24:37.904 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51836.log 2023-11-26T07:24:37.905 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53969.log 2023-11-26T07:24:37.905 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59116.log.gz 2023-11-26T07:24:37.906 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51836.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51836.log.gz 2023-11-26T07:24:37.906 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44485.log 2023-11-26T07:24:37.907 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53969.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53969.log.gz 2023-11-26T07:24:37.907 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53861.log 2023-11-26T07:24:37.908 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44485.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44485.log.gz 2023-11-26T07:24:37.908 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45546.log 2023-11-26T07:24:37.909 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53861.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.53861.log.gz 2023-11-26T07:24:37.909 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47190.log 2023-11-26T07:24:37.910 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45546.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45546.log.gz 2023-11-26T07:24:37.911 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58236.log 2023-11-26T07:24:37.912 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47190.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47190.log.gz 2023-11-26T07:24:37.912 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49060.log 2023-11-26T07:24:37.913 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58236.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58236.log.gz 2023-11-26T07:24:37.913 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46375.log 2023-11-26T07:24:37.913 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49060.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49060.log.gz 2023-11-26T07:24:37.914 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54899.log 2023-11-26T07:24:37.915 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46375.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46375.log.gz 2023-11-26T07:24:37.915 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53300.log 2023-11-26T07:24:37.916 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54899.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54899.log.gz 2023-11-26T07:24:37.916 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57488.log 2023-11-26T07:24:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.53300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53300.log.gz 2023-11-26T07:24:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38199.log 2023-11-26T07:24:37.918 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57488.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57488.log.gz 2023-11-26T07:24:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44270.log 2023-11-26T07:24:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38199.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38199.log.gz 2023-11-26T07:24:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51526.log 2023-11-26T07:24:37.921 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44270.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44270.log.gz 2023-11-26T07:24:37.921 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46609.log 2023-11-26T07:24:37.921 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51526.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51526.log.gz 2023-11-26T07:24:37.922 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51914.log 2023-11-26T07:24:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46609.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46609.log.gz 2023-11-26T07:24:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51008.log 2023-11-26T07:24:37.924 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51914.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.51914.log.gz 2023-11-26T07:24:37.924 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44049.log 2023-11-26T07:24:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51008.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51008.log.gz 2023-11-26T07:24:37.925 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55081.log 2023-11-26T07:24:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44049.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44049.log.gz 2023-11-26T07:24:37.926 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50832.log 2023-11-26T07:24:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55081.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55081.log.gz 2023-11-26T07:24:37.927 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46453.log 2023-11-26T07:24:37.928 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50832.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50832.log.gz 2023-11-26T07:24:37.928 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-26T07:24:37.930 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46453.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46453.log.gz 2023-11-26T07:24:37.930 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50157.log 2023-11-26T07:24:37.931 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51612.log 2023-11-26T07:24:37.932 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50157.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50157.log.gz 2023-11-26T07:24:37.932 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55109.log 2023-11-26T07:24:37.933 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-26T07:24:37.933 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54558.log 2023-11-26T07:24:37.934 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51612.log.gz 2023-11-26T07:24:37.935 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55109.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55109.log.gz 2023-11-26T07:24:37.935 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44849.log 2023-11-26T07:24:37.936 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54558.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54558.log.gz 2023-11-26T07:24:37.937 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54415.log 2023-11-26T07:24:37.938 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51888.log 2023-11-26T07:24:37.938 INFO:teuthology.orchestra.run.smithi031.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44849.log.gz 2023-11-26T07:24:37.939 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54415.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54415.log.gz 2023-11-26T07:24:37.939 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58751.log 2023-11-26T07:24:37.940 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55539.log 2023-11-26T07:24:37.941 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51888.log.gz 2023-11-26T07:24:37.942 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58751.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58751.log.gz 2023-11-26T07:24:37.942 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52130.log 2023-11-26T07:24:37.943 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55539.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55539.log.gz 2023-11-26T07:24:37.943 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51362.log 2023-11-26T07:24:37.944 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45390.log 2023-11-26T07:24:37.945 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52130.log.gz 2023-11-26T07:24:37.946 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51362.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51362.log.gz 2023-11-26T07:24:37.946 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44823.log 2023-11-26T07:24:37.947 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45390.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45390.log.gz 2023-11-26T07:24:37.948 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56957.log 2023-11-26T07:24:37.949 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44823.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44823.log.gz 2023-11-26T07:24:37.949 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56447.log 2023-11-26T07:24:37.950 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56957.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56957.log.gz 2023-11-26T07:24:37.951 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59092.log 2023-11-26T07:24:37.952 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52747.log 2023-11-26T07:24:37.952 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56447.log.gz 2023-11-26T07:24:37.953 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59092.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59092.log.gz 2023-11-26T07:24:37.953 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55378.log 2023-11-26T07:24:37.954 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52747.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52747.log.gz 2023-11-26T07:24:37.955 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50695.log 2023-11-26T07:24:37.956 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46427.log 2023-11-26T07:24:37.956 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55378.log.gz 2023-11-26T07:24:37.957 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50695.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50695.log.gz 2023-11-26T07:24:37.958 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50416.log 2023-11-26T07:24:37.958 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46427.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46427.log.gz 2023-11-26T07:24:37.959 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46531.log 2023-11-26T07:24:37.960 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47452.log 2023-11-26T07:24:37.961 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50416.log.gz 2023-11-26T07:24:37.961 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46531.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46531.log.gz 2023-11-26T07:24:37.962 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-26T07:24:37.963 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47452.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47452.log.gz 2023-11-26T07:24:37.963 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58621.log 2023-11-26T07:24:37.965 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54977.log 2023-11-26T07:24:37.966 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56091.log 2023-11-26T07:24:37.967 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58621.log.gz 2023-11-26T07:24:37.968 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49380.log 2023-11-26T07:24:37.968 INFO:teuthology.orchestra.run.smithi031.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54977.log.gz 2023-11-26T07:24:37.970 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56091.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56091.log.gz 2023-11-26T07:24:37.970 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53126.log 2023-11-26T07:24:37.971 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49380.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49380.log.gz 2023-11-26T07:24:37.971 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49408.log 2023-11-26T07:24:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48218.log 2023-11-26T07:24:37.973 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53126.log.gz 2023-11-26T07:24:37.974 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48374.log 2023-11-26T07:24:37.975 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49408.log.gz 2023-11-26T07:24:37.975 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48218.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48218.log.gz 2023-11-26T07:24:37.976 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55283.log 2023-11-26T07:24:37.977 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48374.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48374.log.gz 2023-11-26T07:24:37.978 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58699.log 2023-11-26T07:24:37.978 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.55283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55283.log.gz 2023-11-26T07:24:37.979 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51697.log 2023-11-26T07:24:37.980 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48634.log 2023-11-26T07:24:37.981 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58699.log.gz 2023-11-26T07:24:37.982 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49569.log 2023-11-26T07:24:37.982 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51697.log.gz 2023-11-26T07:24:37.983 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43862.log 2023-11-26T07:24:37.984 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48634.log.gz 2023-11-26T07:24:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56697.log 2023-11-26T07:24:37.985 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49569.log.gz 2023-11-26T07:24:37.986 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54441.log 2023-11-26T07:24:37.986 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43862.log.gz 2023-11-26T07:24:37.988 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54951.log 2023-11-26T07:24:37.988 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56697.log.gz 2023-11-26T07:24:37.989 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48270.log 2023-11-26T07:24:37.989 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54441.log.gz 2023-11-26T07:24:37.991 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59142.log 2023-11-26T07:24:37.991 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54951.log.gz 2023-11-26T07:24:37.992 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47528.log 2023-11-26T07:24:37.992 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48270.log.gz 2023-11-26T07:24:37.993 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50468.log 2023-11-26T07:24:37.994 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59142.log.gz 2023-11-26T07:24:37.995 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53074.log 2023-11-26T07:24:37.995 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47528.log.gz 2023-11-26T07:24:37.997 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47873.log 2023-11-26T07:24:37.997 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50468.log.gz 2023-11-26T07:24:37.998 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45678.log 2023-11-26T07:24:37.998 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53074.log.gz 2023-11-26T07:24:37.999 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52880.log 2023-11-26T07:24:37.999 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47873.log.gz 2023-11-26T07:24:38.001 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44771.log 2023-11-26T07:24:38.001 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45678.log.gz 2023-11-26T07:24:38.002 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56162.log 2023-11-26T07:24:38.002 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52880.log.gz 2023-11-26T07:24:38.004 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47847.log 2023-11-26T07:24:38.004 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44771.log.gz 2023-11-26T07:24:38.005 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47731.log 2023-11-26T07:24:38.005 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56162.log.gz 2023-11-26T07:24:38.006 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49621.log 2023-11-26T07:24:38.007 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47847.log.gz 2023-11-26T07:24:38.008 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49822.log 2023-11-26T07:24:38.008 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47731.log.gz 2023-11-26T07:24:38.010 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48757.log 2023-11-26T07:24:38.010 INFO:teuthology.orchestra.run.smithi031.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49621.log.gz 2023-11-26T07:24:38.011 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48322.log 2023-11-26T07:24:38.011 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49822.log.gz 2023-11-26T07:24:38.013 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56529.log 2023-11-26T07:24:38.013 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48757.log.gz 2023-11-26T07:24:38.014 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51258.log 2023-11-26T07:24:38.014 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48322.log.gz 2023-11-26T07:24:38.015 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48035.log 2023-11-26T07:24:38.016 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56529.log.gz 2023-11-26T07:24:38.017 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50390.log 2023-11-26T07:24:38.017 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51258.log.gz 2023-11-26T07:24:38.018 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49486.log 2023-11-26T07:24:38.018 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48035.log.gz 2023-11-26T07:24:38.020 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55987.log 2023-11-26T07:24:38.020 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50390.log.gz 2023-11-26T07:24:38.021 INFO:teuthology.orchestra.run.smithi097.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-26T07:24:38.021 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52051.log 2023-11-26T07:24:38.021 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49486.log.gz 2023-11-26T07:24:38.023 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55987.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55987.log.gz 2023-11-26T07:24:38.023 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52435.log 2023-11-26T07:24:38.025 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46297.log 2023-11-26T07:24:38.025 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52051.log.gz 2023-11-26T07:24:38.026 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50364.log 2023-11-26T07:24:38.026 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52435.log.gz 2023-11-26T07:24:38.027 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46297.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46297.log.gz 2023-11-26T07:24:38.027 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45023.log 2023-11-26T07:24:38.029 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57019.log 2023-11-26T07:24:38.029 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50364.log.gz 2023-11-26T07:24:38.030 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.45023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45023.log.gz 2023-11-26T07:24:38.030 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46889.log 2023-11-26T07:24:38.032 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57019.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57019.log.gz 2023-11-26T07:24:38.032 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43627.log 2023-11-26T07:24:38.033 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52591.log 2023-11-26T07:24:38.034 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46889.log.gz 2023-11-26T07:24:38.035 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43627.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.43627.log.gz -5 2023-11-26T07:24:38.035 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52513.log 2023-11-26T07:24:38.036 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58543.log 2023-11-26T07:24:38.037 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52591.log.gz 2023-11-26T07:24:38.038 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50864.log 2023-11-26T07:24:38.038 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52513.log.gz 2023-11-26T07:24:38.039 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47925.log 2023-11-26T07:24:38.039 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58543.log.gz 2023-11-26T07:24:38.041 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57697.log 2023-11-26T07:24:38.041 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50864.log.gz 2023-11-26T07:24:38.042 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44050.log 2023-11-26T07:24:38.042 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47925.log.gz 2023-11-26T07:24:38.044 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54337.log 2023-11-26T07:24:38.044 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57697.log.gz 2023-11-26T07:24:38.045 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47020.log 2023-11-26T07:24:38.045 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44050.log.gz 2023-11-26T07:24:38.046 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46323.log 2023-11-26T07:24:38.047 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54337.log.gz 2023-11-26T07:24:38.048 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53663.log 2023-11-26T07:24:38.048 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47020.log.gz 2023-11-26T07:24:38.049 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53559.log 2023-11-26T07:24:38.050 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46323.log.gz 2023-11-26T07:24:38.051 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46219.log 2023-11-26T07:24:38.051 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53663.log.gz 2023-11-26T07:24:38.052 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52025.log 2023-11-26T07:24:38.053 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53559.log.gz 2023-11-26T07:24:38.054 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57949.log 2023-11-26T07:24:38.054 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46219.log.gz 2023-11-26T07:24:38.055 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53775.log 2023-11-26T07:24:38.055 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52025.log.gz 2023-11-26T07:24:38.056 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57949.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57949.log.gz 2023-11-26T07:24:38.057 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49302.log 2023-11-26T07:24:38.058 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58106.log 2023-11-26T07:24:38.058 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53775.log.gz 2023-11-26T07:24:38.060 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46661.log 2023-11-26T07:24:38.060 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49302.log.gz 2023-11-26T07:24:38.061 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58106.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.58106.log.gz -- 2023-11-26T07:24:38.061 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.53342.log 2023-11-26T07:24:38.063 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54389.log 2023-11-26T07:24:38.063 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46661.log.gz 2023-11-26T07:24:38.064 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57254.log 2023-11-26T07:24:38.064 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53342.log.gz 2023-11-26T07:24:38.066 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37964.log 2023-11-26T07:24:38.066 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54389.log.gz 2023-11-26T07:24:38.067 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45756.log 2023-11-26T07:24:38.067 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57254.log.gz 2023-11-26T07:24:38.069 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50669.log 2023-11-26T07:24:38.069 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37964.log.gz 2023-11-26T07:24:38.070 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50747.log 2023-11-26T07:24:38.070 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45756.log.gz 2023-11-26T07:24:38.071 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45626.log 2023-11-26T07:24:38.071 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50669.log.gz 2023-11-26T07:24:38.073 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54110.log 2023-11-26T07:24:38.073 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50747.log.gz 2023-11-26T07:24:38.074 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45626.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.45626.log.gz 2023-11-26T07:24:38.075 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44719.log 2023-11-26T07:24:38.076 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43575.log 2023-11-26T07:24:38.076 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54110.log.gz 2023-11-26T07:24:38.077 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52773.log 2023-11-26T07:24:38.078 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44719.log.gz 2023-11-26T07:24:38.079 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43575.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43575.log.gz 2023-11-26T07:24:38.079 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44058.log 2023-11-26T07:24:38.080 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46583.log 2023-11-26T07:24:38.080 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52773.log.gz 2023-11-26T07:24:38.082 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44058.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44058.log.gz 2023-11-26T07:24:38.082 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-26T07:24:38.084 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49286.log 2023-11-26T07:24:38.084 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46583.log.gz 2023-11-26T07:24:38.085 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50066.log 2023-11-26T07:24:38.086 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.49286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49286.log.gz 2023-11-26T07:24:38.086 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56039.log 2023-11-26T07:24:38.088 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49884.log 2023-11-26T07:24:38.088 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50066.log.gz 2023-11-26T07:24:38.089 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51438.log 2023-11-26T07:24:38.089 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56039.log.gz 2023-11-26T07:24:38.090 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49884.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49884.log.gz 2023-11-26T07:24:38.091 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53204.log 2023-11-26T07:24:38.092 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43439.log 2023-11-26T07:24:38.093 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51438.log.gz 2023-11-26T07:24:38.094 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53204.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53204.log.gz 2023-11-26T07:24:38.095 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48471.log 2023-11-26T07:24:38.095 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43439.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43439.log.gz 2023-11-26T07:24:38.096 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51552.log 2023-11-26T07:24:38.097 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48471.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48471.log.gz 2023-11-26T07:24:38.098 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2023-11-26T07:24:38.099 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51552.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51552.log.gz 2023-11-26T07:24:38.099 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56829.log 2023-11-26T07:24:38.101 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47957.log 2023-11-26T07:24:38.101 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2023-11-26T07:24:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56829.log: gzip -5 87.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.56829.log.gz -- 2023-11-26T07:24:38.102 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.45208.log 2023-11-26T07:24:38.104 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58132.log 2023-11-26T07:24:38.104 INFO:teuthology.orchestra.run.smithi031.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47957.log.gz 2023-11-26T07:24:38.105 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45842.log 2023-11-26T07:24:38.105 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45208.log.gz 2023-11-26T07:24:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52461.log 2023-11-26T07:24:38.107 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58132.log.gz 2023-11-26T07:24:38.108 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45842.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.45842.log.gz --verbose 2023-11-26T07:24:38.108 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph-client.admin.58340.log 2023-11-26T07:24:38.109 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52461.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52461.log.gz 2023-11-26T07:24:38.109 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55857.log 2023-11-26T07:24:38.111 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58340.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58340.log.gz 2023-11-26T07:24:38.111 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47426.log 2023-11-26T07:24:38.112 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55161.log 2023-11-26T07:24:38.113 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55857.log.gz 2023-11-26T07:24:38.114 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47426.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47426.log.gz 2023-11-26T07:24:38.115 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55161.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55161.log.gz 2023-11-26T07:24:38.116 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45494.log 2023-11-26T07:24:38.117 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46162.log 2023-11-26T07:24:38.119 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53394.log 2023-11-26T07:24:38.119 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45494.log.gz 2023-11-26T07:24:38.120 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46162.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46162.log.gz 2023-11-26T07:24:38.122 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53394.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53394.log.gz 2023-11-26T07:24:38.122 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46693.log 2023-11-26T07:24:38.124 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53500.log 2023-11-26T07:24:38.125 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58881.log 2023-11-26T07:24:38.125 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46693.log.gz 2023-11-26T07:24:38.126 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53500.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53500.log.gz 2023-11-26T07:24:38.127 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56292.log 2023-11-26T07:24:38.129 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51723.log 2023-11-26T07:24:38.129 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58881.log.gz 2023-11-26T07:24:38.130 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56292.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56292.log.gz 2023-11-26T07:24:38.131 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51723.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51723.log.gz 2023-11-26T07:24:38.132 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57462.log 2023-11-26T07:24:38.133 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53368.log 2023-11-26T07:24:38.134 INFO:teuthology.orchestra.run.smithi031.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-26T07:24:38.134 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57462.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57462.log.gz 2023-11-26T07:24:38.135 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50643.log 2023-11-26T07:24:38.136 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46783.log 2023-11-26T07:24:38.136 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53368.log.gz 2023-11-26T07:24:38.137 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50643.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50643.log.gz 2023-11-26T07:24:38.138 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54240.log 2023-11-26T07:24:38.139 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46783.log: gzip 89.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.46783.log.gz -- 2023-11-26T07:24:38.139 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.47643.log 2023-11-26T07:24:38.141 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53022.log 2023-11-26T07:24:38.141 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54240.log.gz 2023-11-26T07:24:38.142 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43601.log 2023-11-26T07:24:38.142 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47643.log.gz 2023-11-26T07:24:38.143 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53022.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53022.log.gz 2023-11-26T07:24:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52182.log 2023-11-26T07:24:38.144 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49673.log 2023-11-26T07:24:38.145 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43601.log.gz 2023-11-26T07:24:38.146 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57892.log 2023-11-26T07:24:38.147 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52182.log.gz 2023-11-26T07:24:38.147 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49673.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49673.log.gz 2023-11-26T07:24:38.148 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48419.log 2023-11-26T07:24:38.149 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48666.log 2023-11-26T07:24:38.149 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57892.log.gz 2023-11-26T07:24:38.150 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48419.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48419.log.gz 2023-11-26T07:24:38.150 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46505.log 2023-11-26T07:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56188.log 2023-11-26T07:24:38.151 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48666.log.gz 2023-11-26T07:24:38.152 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46505.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46505.log.gz 2023-11-26T07:24:38.153 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55458.log 2023-11-26T07:24:38.154 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49328.log 2023-11-26T07:24:38.154 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56188.log.gz 2023-11-26T07:24:38.155 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55458.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55458.log.gz 2023-11-26T07:24:38.155 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48731.log 2023-11-26T07:24:38.155 INFO:teuthology.orchestra.run.smithi097.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-26T07:24:38.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56725.log 2023-11-26T07:24:38.157 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49328.log.gz 2023-11-26T07:24:38.157 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48731.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48731.log.gz 2023-11-26T07:24:38.158 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58210.log 2023-11-26T07:24:38.159 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43467.log 2023-11-26T07:24:38.159 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56725.log.gz 2023-11-26T07:24:38.160 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58210.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58210.log.gz 2023-11-26T07:24:38.160 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52409.log 2023-11-26T07:24:38.161 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49354.log 2023-11-26T07:24:38.162 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43467.log.gz 2023-11-26T07:24:38.162 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52409.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52409.log.gz 2023-11-26T07:24:38.163 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-26T07:24:38.164 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46136.log 2023-11-26T07:24:38.164 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49354.log.gz 2023-11-26T07:24:38.165 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48192.log 2023-11-26T07:24:38.167 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47110.log 2023-11-26T07:24:38.167 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46136.log.gz 2023-11-26T07:24:38.168 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48192.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48192.log.gz 2023-11-26T07:24:38.169 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57280.log 2023-11-26T07:24:38.170 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47110.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47110.log.gz 2023-11-26T07:24:38.170 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45156.log 2023-11-26T07:24:38.172 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50617.log 2023-11-26T07:24:38.172 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57280.log.gz 2023-11-26T07:24:38.172 INFO:teuthology.orchestra.run.smithi097.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-26T07:24:38.173 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45156.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45156.log.gz 2023-11-26T07:24:38.173 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52338.log 2023-11-26T07:24:38.175 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50617.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50617.log.gz 2023-11-26T07:24:38.175 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44055.log 2023-11-26T07:24:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52338.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52338.log.gz 2023-11-26T07:24:38.176 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52260.log 2023-11-26T07:24:38.178 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54819.log 2023-11-26T07:24:38.178 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44055.log.gz 2023-11-26T07:24:38.179 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45234.log 2023-11-26T07:24:38.179 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52260.log.gz 2023-11-26T07:24:38.181 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54819.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54819.log.gz 2023-11-26T07:24:38.181 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55617.log 2023-11-26T07:24:38.183 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46809.log 2023-11-26T07:24:38.183 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45234.log.gz 2023-11-26T07:24:38.184 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55617.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55961.log 2023-11-26T07:24:38.184 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55617.log.gz 2023-11-26T07:24:38.185 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48088.log 2023-11-26T07:24:38.186 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46809.log.gz 2023-11-26T07:24:38.187 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57432.log 2023-11-26T07:24:38.187 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55961.log.gz 2023-11-26T07:24:38.188 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52539.log 2023-11-26T07:24:38.189 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48088.log.gz 2023-11-26T07:24:38.190 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57432.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57432.log.gz -5 2023-11-26T07:24:38.190 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.0.47574.log 2023-11-26T07:24:38.191 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57202.log 2023-11-26T07:24:38.191 INFO:teuthology.orchestra.run.smithi031.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52539.log.gz 2023-11-26T07:24:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.47574.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47574.log.gz 2023-11-26T07:24:38.192 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54266.log 2023-11-26T07:24:38.194 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48601.log 2023-11-26T07:24:38.194 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57202.log.gz 2023-11-26T07:24:38.196 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56318.log 2023-11-26T07:24:38.196 INFO:teuthology.orchestra.run.smithi031.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54266.log.gz 2023-11-26T07:24:38.197 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58027.log 2023-11-26T07:24:38.197 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48601.log.gz 2023-11-26T07:24:38.199 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51667.log 2023-11-26T07:24:38.199 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56318.log.gz 2023-11-26T07:24:38.200 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58027.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58027.log.gz 2023-11-26T07:24:38.201 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50773.log 2023-11-26T07:24:38.201 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51667.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51667.log.gz 2023-11-26T07:24:38.202 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47320.log 2023-11-26T07:24:38.203 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58184.log 2023-11-26T07:24:38.204 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50773.log.gz 2023-11-26T07:24:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47320.log: gzip -5 87.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47320.log.gz -- 2023-11-26T07:24:38.205 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.54526.log 2023-11-26T07:24:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55135.log 2023-11-26T07:24:38.207 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58184.log.gz 2023-11-26T07:24:38.208 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54526.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54526.log.gz -5 2023-11-26T07:24:38.208 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55778.log 2023-11-26T07:24:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44745.log 2023-11-26T07:24:38.210 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55135.log.gz 2023-11-26T07:24:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55778.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55778.log.gz 2023-11-26T07:24:38.211 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46757.log 2023-11-26T07:24:38.212 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44745.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44745.log.gz 2023-11-26T07:24:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57518.log 2023-11-26T07:24:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58803.log 2023-11-26T07:24:38.214 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46757.log.gz 2023-11-26T07:24:38.215 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57518.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57518.log.gz 2023-11-26T07:24:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58803.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58803.log.gz 2023-11-26T07:24:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58262.log 2023-11-26T07:24:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55428.log 2023-11-26T07:24:38.220 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49790.log 2023-11-26T07:24:38.221 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58262.log.gz 2023-11-26T07:24:38.222 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56881.log 2023-11-26T07:24:38.222 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55428.log.gz 2023-11-26T07:24:38.223 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49790.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49790.log.gz 2023-11-26T07:24:38.223 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43387.log 2023-11-26T07:24:38.224 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56881.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56881.log.gz 2023-11-26T07:24:38.225 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47242.log 2023-11-26T07:24:38.226 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43387.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43387.log.gz 2023-11-26T07:24:38.227 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43705.log 2023-11-26T07:24:38.228 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47242.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33108.log 2023-11-26T07:24:38.228 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47242.log.gz 2023-11-26T07:24:38.230 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53745.log 2023-11-26T07:24:38.230 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43705.log.gz 2023-11-26T07:24:38.230 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.tmp-client.admin.33108.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33108.log.gz 2023-11-26T07:24:38.231 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46193.log 2023-11-26T07:24:38.233 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47162.log 2023-11-26T07:24:38.233 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53745.log.gz 2023-11-26T07:24:38.234 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44563.log 2023-11-26T07:24:38.234 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46193.log.gz 2023-11-26T07:24:38.235 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47162.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47162.log.gz 2023-11-26T07:24:38.236 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55352.log 2023-11-26T07:24:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44563.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44563.log.gz 2023-11-26T07:24:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46110.log 2023-11-26T07:24:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55352.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55352.log.gz 2023-11-26T07:24:38.239 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51126.log 2023-11-26T07:24:38.240 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46110.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.46110.log.gz -5 2023-11-26T07:24:38.240 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2023-11-26T07:24:38.241 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51126.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51126.log.gz 2023-11-26T07:24:38.242 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55187.log 2023-11-26T07:24:38.243 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50338.log 2023-11-26T07:24:38.244 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gz 2023-11-26T07:24:38.245 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53316.log 2023-11-26T07:24:38.246 INFO:teuthology.orchestra.run.smithi031.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55187.log.gz 2023-11-26T07:24:38.246 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50338.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50338.log.gz 2023-11-26T07:24:38.247 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53316.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53316.log.gz 2023-11-26T07:24:38.248 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56266.log 2023-11-26T07:24:38.249 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57975.log 2023-11-26T07:24:38.251 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55482.log 2023-11-26T07:24:38.251 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56266.log.gz 2023-11-26T07:24:38.253 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56240.log 2023-11-26T07:24:38.253 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57975.log.gz 2023-11-26T07:24:38.254 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50981.log 2023-11-26T07:24:38.254 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55482.log.gz 2023-11-26T07:24:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58288.log 2023-11-26T07:24:38.257 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56240.log.gz 2023-11-26T07:24:38.257 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47899.log 2023-11-26T07:24:38.258 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50981.log.gz 2023-11-26T07:24:38.258 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58288.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-26T07:24:38.258 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58288.log.gz 2023-11-26T07:24:38.260 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47136.log 2023-11-26T07:24:38.260 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47899.log.gz 2023-11-26T07:24:38.261 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-26T07:24:38.261 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52487.log 2023-11-26T07:24:38.263 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47136.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47136.log.gz 2023-11-26T07:24:38.263 INFO:teuthology.orchestra.run.smithi031.stderr: 86.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-26T07:24:38.263 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44296.log 2023-11-26T07:24:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52487.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52487.log.gz 2023-11-26T07:24:38.264 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55003.log 2023-11-26T07:24:38.266 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58959.log 2023-11-26T07:24:38.266 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44296.log.gz 2023-11-26T07:24:38.268 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51178.log 2023-11-26T07:24:38.268 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55003.log.gz 2023-11-26T07:24:38.269 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50442.log 2023-11-26T07:24:38.269 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58959.log.gz 2023-11-26T07:24:38.270 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51178.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.51178.log.gz 2023-11-26T07:24:38.270 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46479.log 2023-11-26T07:24:38.272 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49729.log 2023-11-26T07:24:38.272 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50442.log.gz 2023-11-26T07:24:38.273 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46479.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.46479.log.gz -5 2023-11-26T07:24:38.274 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56645.log 2023-11-26T07:24:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49729.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49729.log.gz 2023-11-26T07:24:38.275 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47669.log 2023-11-26T07:24:38.276 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57918.log 2023-11-26T07:24:38.277 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56645.log.gz 2023-11-26T07:24:38.278 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59037.log 2023-11-26T07:24:38.278 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47669.log.gz 2023-11-26T07:24:38.279 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50189.log 2023-11-26T07:24:38.279 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57918.log.gz 2023-11-26T07:24:38.281 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43210.log 2023-11-26T07:24:38.281 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59037.log.gz 2023-11-26T07:24:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50189.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50189.log.gz 2023-11-26T07:24:38.282 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44262.log 2023-11-26T07:24:38.284 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58158.log 2023-11-26T07:24:38.284 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43210.log.gz 2023-11-26T07:24:38.285 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44262.log: gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.44262.log.gz /var/log/ceph/ceph-client.admin.58725.log 2023-11-26T07:24:38.285 INFO:teuthology.orchestra.run.smithi031.stderr: 2023-11-26T07:24:38.286 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58158.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58158.log.gz 2023-11-26T07:24:38.287 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54136.log 2023-11-26T07:24:38.288 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57780.log 2023-11-26T07:24:38.289 INFO:teuthology.orchestra.run.smithi031.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58725.log.gz 2023-11-26T07:24:38.290 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44641.log 2023-11-26T07:24:38.290 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54136.log.gz 2023-11-26T07:24:38.291 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55029.log 2023-11-26T07:24:38.292 INFO:teuthology.orchestra.run.smithi031.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57780.log.gz 2023-11-26T07:24:38.293 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54675.log 2023-11-26T07:24:38.293 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44641.log.gz 2023-11-26T07:24:38.294 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49086.log 2023-11-26T07:24:38.294 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55029.log.gz 2023-11-26T07:24:38.296 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43845.log 2023-11-26T07:24:38.296 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54675.log.gz 2023-11-26T07:24:38.297 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48864.log 2023-11-26T07:24:38.297 INFO:teuthology.orchestra.run.smithi031.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49086.log.gz 2023-11-26T07:24:38.299 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57723.log 2023-11-26T07:24:38.299 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43845.log.gz 2023-11-26T07:24:38.300 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48864.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48864.log.gz -5 2023-11-26T07:24:38.300 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51092.log 2023-11-26T07:24:38.302 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57723.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57723.log.gzgzip 2023-11-26T07:24:38.302 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49164.log 2023-11-26T07:24:38.302 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51092.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51092.log.gz 2023-11-26T07:24:38.303 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46635.log 2023-11-26T07:24:38.303 INFO:teuthology.orchestra.run.smithi097.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-26T07:24:38.305 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44260.log 2023-11-26T07:24:38.305 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49164.log.gz 2023-11-26T07:24:38.306 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46635.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46635.log.gz 2023-11-26T07:24:38.307 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44260.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44260.log.gz 2023-11-26T07:24:38.308 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58595.log 2023-11-26T07:24:38.309 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43243.log 2023-11-26T07:24:38.311 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47294.log 89.2% 2023-11-26T07:24:38.311 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58595.log.gz 2023-11-26T07:24:38.312 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54756.log 2023-11-26T07:24:38.313 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43243.log.gz 2023-11-26T07:24:38.314 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47294.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47294.log.gz 2023-11-26T07:24:38.315 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54756.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54756.log.gz 2023-11-26T07:24:38.316 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49759.log 2023-11-26T07:24:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47048.log 2023-11-26T07:24:38.319 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49759.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49759.log.gz 2023-11-26T07:24:38.319 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51284.log 2023-11-26T07:24:38.320 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52852.log 2023-11-26T07:24:38.321 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47048.log.gz 2023-11-26T07:24:38.322 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51484.log 2023-11-26T07:24:38.322 INFO:teuthology.orchestra.run.smithi031.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51284.log.gz 2023-11-26T07:24:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52852.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52852.log.gz 2023-11-26T07:24:38.323 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50096.log 2023-11-26T07:24:38.324 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.51484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51484.log.gz 2023-11-26T07:24:38.325 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50219.log 2023-11-26T07:24:38.326 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54493.log 2023-11-26T07:24:38.326 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50096.log.gz 2023-11-26T07:24:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50219.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50219.log.gz 2023-11-26T07:24:38.328 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51388.log 2023-11-26T07:24:38.329 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54493.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54493.log.gz 2023-11-26T07:24:38.330 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43495.log 2023-11-26T07:24:38.332 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50546.log 2023-11-26T07:24:38.332 INFO:teuthology.orchestra.run.smithi031.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51388.log.gz 2023-11-26T07:24:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43495.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43495.log.gz 2023-11-26T07:24:38.333 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49434.log 2023-11-26T07:24:38.334 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50546.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50546.log.gz 2023-11-26T07:24:38.335 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56907.log 2023-11-26T07:24:38.336 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45730.log 2023-11-26T07:24:38.337 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49434.log.gz 2023-11-26T07:24:38.338 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56777.log 2023-11-26T07:24:38.339 INFO:teuthology.orchestra.run.smithi031.stderr: 91.0%/var/log/ceph/ceph-client.admin.45730.log: -- replaced with /var/log/ceph/ceph-client.admin.56907.log.gz 2023-11-26T07:24:38.339 INFO:teuthology.orchestra.run.smithi031.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45730.log.gz 2023-11-26T07:24:38.339 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49034.log 2023-11-26T07:24:38.341 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45182.log 2023-11-26T07:24:38.341 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56777.log.gz 2023-11-26T07:24:38.342 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38077.log 2023-11-26T07:24:38.342 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49034.log.gz 2023-11-26T07:24:38.344 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45182.log: gzip -5 --verbose 88.5% -- -- replaced with /var/log/ceph/ceph-client.admin.45182.log.gz /var/log/ceph/ceph-client.admin.52234.log 2023-11-26T07:24:38.344 INFO:teuthology.orchestra.run.smithi031.stderr: 2023-11-26T07:24:38.345 INFO:teuthology.orchestra.run.smithi097.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-26T07:24:38.345 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45364.log 2023-11-26T07:24:38.345 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38077.log.gz 2023-11-26T07:24:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52234.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52234.log.gz 2023-11-26T07:24:38.347 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.admin.57356.log 2023-11-26T07:24:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52800.log 2023-11-26T07:24:38.348 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45364.log.gz 2023-11-26T07:24:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50721.log 2023-11-26T07:24:38.350 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57356.log.gz 2023-11-26T07:24:38.351 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52800.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52800.log.gz 2023-11-26T07:24:38.352 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48783.log 2023-11-26T07:24:38.352 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50721.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50721.log.gz 2023-11-26T07:24:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48810.log 2023-11-26T07:24:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45925.log 2023-11-26T07:24:38.355 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48783.log.gz 2023-11-26T07:24:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48810.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48810.log.gz 2023-11-26T07:24:38.357 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45925.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45925.log.gz 2023-11-26T07:24:38.358 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57751.log 2023-11-26T07:24:38.359 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49543.log 2023-11-26T07:24:38.361 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53801.log 2023-11-26T07:24:38.361 INFO:teuthology.orchestra.run.smithi031.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57751.log.gz 2023-11-26T07:24:38.362 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49543.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49543.log.gz 2023-11-26T07:24:38.362 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46003.log 2023-11-26T07:24:38.364 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53801.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.53801.log.gz --verbose 2023-11-26T07:24:38.364 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph-client.admin.44251.log 2023-11-26T07:24:38.365 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51641.log 2023-11-26T07:24:38.366 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46003.log.gz 2023-11-26T07:24:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44251.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44251.log.gz 2023-11-26T07:24:38.367 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50494.log 2023-11-26T07:24:38.368 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43308.log 2023-11-26T07:24:38.368 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51641.log.gz 2023-11-26T07:24:38.370 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51036.log 2023-11-26T07:24:38.370 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50494.log.gz 2023-11-26T07:24:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43308.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43308.log.gz 2023-11-26T07:24:38.371 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45520.log 2023-11-26T07:24:38.373 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51036.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.51036.log.gz /var/log/ceph/ceph-client.admin.53422.log 2023-11-26T07:24:38.373 INFO:teuthology.orchestra.run.smithi031.stderr: 2023-11-26T07:24:38.374 INFO:teuthology.orchestra.run.smithi031.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-26T07:24:38.374 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45951.log 2023-11-26T07:24:38.374 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45520.log.gz 2023-11-26T07:24:38.375 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53422.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53422.log.gz 2023-11-26T07:24:38.376 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56383.log 2023-11-26T07:24:38.377 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53995.log 2023-11-26T07:24:38.377 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45951.log.gz 2023-11-26T07:24:38.378 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-26T07:24:38.379 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56383.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56383.log.gz 2023-11-26T07:24:38.379 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48523.log 2023-11-26T07:24:38.379 INFO:teuthology.orchestra.run.smithi097.stderr: 2023-11-26T07:24:38.379 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m0.561s 2023-11-26T07:24:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m2.466s 2023-11-26T07:24:38.380 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.288s 2023-11-26T07:24:38.381 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53995.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53995.log.gz 2023-11-26T07:24:38.381 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49854.log 2023-11-26T07:24:38.382 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48549.log 2023-11-26T07:24:38.382 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48523.log.gz 2023-11-26T07:24:38.383 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43858.log 2023-11-26T07:24:38.384 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49854.log.gz 2023-11-26T07:24:38.385 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54188.log 2023-11-26T07:24:38.385 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48549.log.gz 2023-11-26T07:24:38.386 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43858.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43858.log.gz 2023-11-26T07:24:38.386 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44071.log 2023-11-26T07:24:38.388 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56671.log 2023-11-26T07:24:38.388 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54188.log.gz 2023-11-26T07:24:38.389 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44071.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44071.log.gz 2023-11-26T07:24:38.389 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58907.log 2023-11-26T07:24:38.390 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56671.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56671.log.gz 2023-11-26T07:24:38.391 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53100.log 2023-11-26T07:24:38.392 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58907.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58907.log.gz 2023-11-26T07:24:38.393 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-26T07:24:38.394 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53100.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53100.log.gz 2023-11-26T07:24:38.394 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54925.log 2023-11-26T07:24:38.396 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49944.log 2023-11-26T07:24:38.397 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54845.log 2023-11-26T07:24:38.397 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54925.log.gz 2023-11-26T07:24:38.398 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49944.log: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-26T07:24:38.399 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49944.log.gz 2023-11-26T07:24:38.399 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-26T07:24:38.400 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49190.log 2023-11-26T07:24:38.400 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54845.log.gz 2023-11-26T07:24:38.401 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53448.log 2023-11-26T07:24:38.403 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50286.log 2023-11-26T07:24:38.403 INFO:teuthology.orchestra.run.smithi031.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49190.log.gz 2023-11-26T07:24:38.404 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53448.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53448.log.gz 2023-11-26T07:24:38.405 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50286.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50286.log.gz 2023-11-26T07:24:38.406 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53689.log 2023-11-26T07:24:38.407 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45468.log 2023-11-26T07:24:38.409 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43153.log 2023-11-26T07:24:38.409 INFO:teuthology.orchestra.run.smithi031.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.53689.log.gz 2023-11-26T07:24:38.410 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45468.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45468.log.gz 2023-11-26T07:24:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43153.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43153.log.gz 2023-11-26T07:24:38.412 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-26T07:24:38.414 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57136.log 2023-11-26T07:24:38.415 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44909.log 2023-11-26T07:24:38.417 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57136.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57136.log.gz 2023-11-26T07:24:38.418 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44909.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44909.log.gz 2023-11-26T07:24:38.419 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52208.log 2023-11-26T07:24:38.420 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43653.log 2023-11-26T07:24:38.422 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52208.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52208.log.gz 2023-11-26T07:24:38.423 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43653.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43653.log.gz 2023-11-26T07:24:38.424 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54084.log 2023-11-26T07:24:38.425 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45572.log 2023-11-26T07:24:38.427 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54084.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54084.log.gz 2023-11-26T07:24:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45572.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45572.log.gz 2023-11-26T07:24:38.428 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53254.log 2023-11-26T07:24:38.430 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56013.log 2023-11-26T07:24:38.431 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53254.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53254.log.gz 2023-11-26T07:24:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56013.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56013.log.gz 2023-11-26T07:24:38.433 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45598.log 2023-11-26T07:24:38.435 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55752.log 2023-11-26T07:24:38.436 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45598.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45598.log.gz 2023-11-26T07:24:38.438 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55752.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55752.log.gz 2023-11-26T07:24:38.438 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44271.log 2023-11-26T07:24:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45816.log 2023-11-26T07:24:38.442 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44271.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44271.log.gz 2023-11-26T07:24:38.443 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45816.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45816.log.gz 2023-11-26T07:24:38.444 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49138.log 2023-11-26T07:24:38.445 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49914.log 2023-11-26T07:24:38.447 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49138.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49138.log.gz 2023-11-26T07:24:38.448 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49914.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49914.log.gz 2023-11-26T07:24:38.449 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49240.log 2023-11-26T07:24:38.450 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50127.log 2023-11-26T07:24:38.452 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49240.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49240.log.gz 2023-11-26T07:24:38.453 INFO:teuthology.orchestra.run.smithi031.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-26T07:24:38.453 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50127.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50127.log.gz 2023-11-26T07:24:38.454 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44511.log 2023-11-26T07:24:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57228.log 2023-11-26T07:24:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43549.log 2023-11-26T07:24:38.457 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44511.log.gz 2023-11-26T07:24:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48836.log 2023-11-26T07:24:38.458 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57228.log.gz 2023-11-26T07:24:38.460 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56136.log 2023-11-26T07:24:38.460 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43549.log.gz 2023-11-26T07:24:38.461 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47821.log 2023-11-26T07:24:38.461 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48836.log.gz 2023-11-26T07:24:38.463 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56136.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.56136.log.gz -5 2023-11-26T07:24:38.463 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44693.log 2023-11-26T07:24:38.464 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47821.log: gzip -5 --verbose 88.2% -- -- replaced with /var/log/ceph/ceph-client.admin.47821.log.gz /var/log/ceph/ceph-client.admin.45286.log 2023-11-26T07:24:38.464 INFO:teuthology.orchestra.run.smithi031.stderr: 2023-11-26T07:24:38.466 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44693.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44693.log.gz 2023-11-26T07:24:38.466 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55883.log 2023-11-26T07:24:38.467 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45286.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45286.log.gz 2023-11-26T07:24:38.467 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53152.log 2023-11-26T07:24:38.469 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57814.log 2023-11-26T07:24:38.469 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55883.log.gz 2023-11-26T07:24:38.470 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53152.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53152.log.gz 2023-11-26T07:24:38.470 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45704.log 2023-11-26T07:24:38.472 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50249.log 2023-11-26T07:24:38.472 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57814.log.gz 2023-11-26T07:24:38.473 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45704.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45704.log.gzgzip 2023-11-26T07:24:38.474 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55909.log 2023-11-26T07:24:38.475 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51152.log 2023-11-26T07:24:38.475 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50249.log.gz 2023-11-26T07:24:38.476 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55909.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46994.log 2023-11-26T07:24:38.477 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55909.log.gz 2023-11-26T07:24:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51152.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51152.log.gz 2023-11-26T07:24:38.478 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43523.log 2023-11-26T07:24:38.479 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46994.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46994.log.gz 2023-11-26T07:24:38.480 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44589.log 2023-11-26T07:24:38.481 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52908.log 2023-11-26T07:24:38.481 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43523.log.gz 2023-11-26T07:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48894.log 2023-11-26T07:24:38.483 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44589.log.gz 2023-11-26T07:24:38.484 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43413.log 2023-11-26T07:24:38.485 INFO:teuthology.orchestra.run.smithi031.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52908.log.gz 2023-11-26T07:24:38.486 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52942.log 2023-11-26T07:24:38.486 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48894.log.gz 2023-11-26T07:24:38.487 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43413.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.43846.log 2023-11-26T07:24:38.488 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43413.log.gz 2023-11-26T07:24:38.489 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54702.log 2023-11-26T07:24:38.489 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52942.log.gz 2023-11-26T07:24:38.490 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45790.log 2023-11-26T07:24:38.490 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43846.log.gz 2023-11-26T07:24:38.491 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54702.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54702.log.gz 2023-11-26T07:24:38.492 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51775.log 2023-11-26T07:24:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45790.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45790.log.gz 2023-11-26T07:24:38.493 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52968.log 2023-11-26T07:24:38.495 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56555.log 2023-11-26T07:24:38.495 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51775.log.gz 2023-11-26T07:24:38.496 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45338.log 2023-11-26T07:24:38.497 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52968.log.gz 2023-11-26T07:24:38.498 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57306.log 2023-11-26T07:24:38.498 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56555.log.gz 2023-11-26T07:24:38.500 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48061.log 2023-11-26T07:24:38.500 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45338.log.gz 2023-11-26T07:24:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57306.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57306.log.gz 2023-11-26T07:24:38.501 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47374.log 2023-11-26T07:24:38.502 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48061.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48061.log.gz 2023-11-26T07:24:38.503 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48166.log 2023-11-26T07:24:38.504 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47374.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47374.log.gz 2023-11-26T07:24:38.504 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43855.log 2023-11-26T07:24:38.505 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48166.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48166.log.gz 2023-11-26T07:24:38.506 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52565.log 2023-11-26T07:24:38.507 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51862.log 2023-11-26T07:24:38.508 INFO:teuthology.orchestra.run.smithi031.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43855.log.gz 2023-11-26T07:24:38.509 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52624.log 2023-11-26T07:24:38.509 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52565.log.gz 2023-11-26T07:24:38.510 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51862.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51862.log.gz 2023-11-26T07:24:38.511 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50929.log 2023-11-26T07:24:38.512 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52624.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52624.log.gz 2023-11-26T07:24:38.512 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43809.log 2023-11-26T07:24:38.513 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50929.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50929.log.gz 2023-11-26T07:24:38.514 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58673.log 2023-11-26T07:24:38.515 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58569.log 2023-11-26T07:24:38.516 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43809.log.gz 2023-11-26T07:24:38.517 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46271.log 87.4% 2023-11-26T07:24:38.517 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58673.log.gz 2023-11-26T07:24:38.518 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55831.log 2023-11-26T07:24:38.518 INFO:teuthology.orchestra.run.smithi031.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58569.log.gz 2023-11-26T07:24:38.519 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46271.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46271.log.gz 2023-11-26T07:24:38.520 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58985.log 2023-11-26T07:24:38.521 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55831.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55831.log.gz 2023-11-26T07:24:38.521 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58437.log 2023-11-26T07:24:38.523 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58985.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58985.log.gz 2023-11-26T07:24:38.523 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43679.log 2023-11-26T07:24:38.524 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49460.log 2023-11-26T07:24:38.524 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58437.log.gz 2023-11-26T07:24:38.526 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53585.log 2023-11-26T07:24:38.526 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43679.log.gz 2023-11-26T07:24:38.528 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49460.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49460.log.gz 2023-11-26T07:24:38.528 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43783.log 2023-11-26T07:24:38.529 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56583.log 2023-11-26T07:24:38.529 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53585.log.gz 2023-11-26T07:24:38.531 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46029.log 2023-11-26T07:24:38.531 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43783.log.gz 2023-11-26T07:24:38.532 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52286.log 2023-11-26T07:24:38.532 INFO:teuthology.orchestra.run.smithi031.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56583.log.gz 2023-11-26T07:24:38.533 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51749.log 2023-11-26T07:24:38.534 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46029.log.gz 2023-11-26T07:24:38.535 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52286.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.55404.log 2023-11-26T07:24:38.535 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52286.log.gz 2023-11-26T07:24:38.536 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53178.log 2023-11-26T07:24:38.537 INFO:teuthology.orchestra.run.smithi031.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51749.log.gz 2023-11-26T07:24:38.538 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55404.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.55404.log.gz -5 2023-11-26T07:24:38.538 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53637.log 2023-11-26T07:24:38.539 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47268.log 2023-11-26T07:24:38.540 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53178.log.gz 2023-11-26T07:24:38.541 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53637.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.51973.log 2023-11-26T07:24:38.542 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53637.log.gz 2023-11-26T07:24:38.542 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47268.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47268.log.gz 2023-11-26T07:24:38.543 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43334.log 2023-11-26T07:24:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51973.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51973.log.gz 2023-11-26T07:24:38.544 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52078.log 2023-11-26T07:24:38.546 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55300.log 2023-11-26T07:24:38.546 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43334.log.gz 2023-11-26T07:24:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48575.log 2023-11-26T07:24:38.547 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52078.log.gz 2023-11-26T07:24:38.549 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45894.log 2023-11-26T07:24:38.549 INFO:teuthology.orchestra.run.smithi031.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55300.log.gz 2023-11-26T07:24:38.550 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2023-11-26T07:24:38.550 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48575.log.gz 2023-11-26T07:24:38.552 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48980.log 2023-11-26T07:24:38.552 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45894.log.gz 2023-11-26T07:24:38.553 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58933.log 2023-11-26T07:24:38.553 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2023-11-26T07:24:38.554 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48980.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.48980.log.gz -5 2023-11-26T07:24:38.554 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53611.log 2023-11-26T07:24:38.556 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58053.log 2023-11-26T07:24:38.556 INFO:teuthology.orchestra.run.smithi031.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58933.log.gz 2023-11-26T07:24:38.557 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53611.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53611.log.gz 2023-11-26T07:24:38.558 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47616.log 2023-11-26T07:24:38.559 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47216.log 2023-11-26T07:24:38.559 INFO:teuthology.orchestra.run.smithi031.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58053.log.gz 2023-11-26T07:24:38.561 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57866.log 2023-11-26T07:24:38.561 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47616.log.gz 2023-11-26T07:24:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47216.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47216.log.gz 2023-11-26T07:24:38.562 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56214.log 2023-11-26T07:24:38.564 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44068.log 2023-11-26T07:24:38.564 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57866.log.gz 2023-11-26T07:24:38.565 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55565.log 2023-11-26T07:24:38.565 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56214.log.gz 2023-11-26T07:24:38.567 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44068.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44068.log.gz -5 2023-11-26T07:24:38.567 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58491.log 2023-11-26T07:24:38.568 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55506.log 2023-11-26T07:24:38.568 INFO:teuthology.orchestra.run.smithi031.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55565.log.gz 2023-11-26T07:24:38.570 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45312.log 2023-11-26T07:24:38.570 INFO:teuthology.orchestra.run.smithi031.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58491.log.gz 2023-11-26T07:24:38.571 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52312.log 2023-11-26T07:24:38.571 INFO:teuthology.orchestra.run.smithi031.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55506.log.gz 2023-11-26T07:24:38.573 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54871.log 2023-11-26T07:24:38.573 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45312.log.gz 2023-11-26T07:24:38.574 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50799.log 2023-11-26T07:24:38.574 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52312.log.gz 2023-11-26T07:24:38.575 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43863.log 2023-11-26T07:24:38.576 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54871.log.gz 2023-11-26T07:24:38.577 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46557.log 2023-11-26T07:24:38.577 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50799.log.gz 2023-11-26T07:24:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50006.log 2023-11-26T07:24:38.579 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43863.log.gz 2023-11-26T07:24:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46245.log 2023-11-26T07:24:38.580 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46557.log.gz 2023-11-26T07:24:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50006.log: gzip -5 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50006.log.gz 2023-11-26T07:24:38.581 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51999.log 2023-11-26T07:24:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55935.log 2023-11-26T07:24:38.583 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46245.log.gz 2023-11-26T07:24:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51999.log: gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51999.log.gz 2023-11-26T07:24:38.585 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57382.log 2023-11-26T07:24:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55935.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55935.log.gz 2023-11-26T07:24:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52721.log 2023-11-26T07:24:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56351.log 2023-11-26T07:24:38.588 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57382.log.gz 2023-11-26T07:24:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45134.log 2023-11-26T07:24:38.589 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52721.log.gz 2023-11-26T07:24:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56351.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56351.log.gz 2023-11-26T07:24:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45416.log 2023-11-26T07:24:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.2.45134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45134.log.gz 2023-11-26T07:24:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59063.log 2023-11-26T07:24:38.594 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44667.log 2023-11-26T07:24:38.594 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45416.log.gz 2023-11-26T07:24:38.595 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59063.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.44059.log 2023-11-26T07:24:38.595 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59063.log.gz 2023-11-26T07:24:38.597 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48244.log 2023-11-26T07:24:38.597 INFO:teuthology.orchestra.run.smithi031.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44667.log.gz 2023-11-26T07:24:38.598 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56803.log 2023-11-26T07:24:38.598 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44059.log.gz 2023-11-26T07:24:38.599 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55674.log 2023-11-26T07:24:38.600 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48244.log.gz 2023-11-26T07:24:38.601 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49974.log 2023-11-26T07:24:38.601 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56803.log.gz 2023-11-26T07:24:38.602 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55674.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55674.log.gz 2023-11-26T07:24:38.603 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45868.log 2023-11-26T07:24:38.604 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58855.log 2023-11-26T07:24:38.604 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49974.log.gz 2023-11-26T07:24:38.606 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56751.log 2023-11-26T07:24:38.606 INFO:teuthology.orchestra.run.smithi031.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45868.log.gz 2023-11-26T07:24:38.607 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58855.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58855.log.gz 2023-11-26T07:24:38.607 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47348.log 2023-11-26T07:24:38.608 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56751.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56751.log.gz 2023-11-26T07:24:38.608 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57045.log 2023-11-26T07:24:38.610 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47705.log 2023-11-26T07:24:38.610 INFO:teuthology.orchestra.run.smithi031.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47348.log.gz 2023-11-26T07:24:38.612 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44615.log 2023-11-26T07:24:38.612 INFO:teuthology.orchestra.run.smithi031.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57045.log.gz 2023-11-26T07:24:38.613 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43861.log 2023-11-26T07:24:38.613 INFO:teuthology.orchestra.run.smithi031.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47705.log.gz 2023-11-26T07:24:38.615 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47478.log 2023-11-26T07:24:38.615 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44615.log.gz 2023-11-26T07:24:38.617 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52994.log 2023-11-26T07:24:38.617 INFO:teuthology.orchestra.run.smithi031.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43861.log.gz 2023-11-26T07:24:38.618 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58829.log 2023-11-26T07:24:38.619 INFO:teuthology.orchestra.run.smithi031.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.47478.log.gz 2023-11-26T07:24:38.619 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52994.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52994.log.gzgzip 2023-11-26T07:24:38.620 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48928.log 2023-11-26T07:24:38.621 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43731.log 2023-11-26T07:24:38.621 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58829.log.gz 2023-11-26T07:24:38.622 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58001.log 2023-11-26T07:24:38.624 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43731.log: /var/log/ceph/ceph-client.admin.48928.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43731.log.gz 2023-11-26T07:24:38.624 INFO:teuthology.orchestra.run.smithi031.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48928.log.gz 2023-11-26T07:24:38.625 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58647.log 2023-11-26T07:24:38.626 INFO:teuthology.orchestra.run.smithi031.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58001.log.gz 2023-11-26T07:24:38.627 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56501.log 2023-11-26T07:24:38.629 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44297.log 2023-11-26T07:24:38.629 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58647.log.gz 2023-11-26T07:24:38.630 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44797.log 2023-11-26T07:24:38.630 INFO:teuthology.orchestra.run.smithi031.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56501.log.gz 2023-11-26T07:24:38.631 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44297.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44297.log.gz 2023-11-26T07:24:38.632 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51810.log 2023-11-26T07:24:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44797.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44797.log.gz 2023-11-26T07:24:38.633 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55237.log 2023-11-26T07:24:38.634 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51810.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.51810.log.gz -5 2023-11-26T07:24:38.635 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52156.log 2023-11-26T07:24:38.636 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43757.log 2023-11-26T07:24:38.636 INFO:teuthology.orchestra.run.smithi031.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55237.log.gz 2023-11-26T07:24:38.637 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52156.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52156.log.gz 2023-11-26T07:24:38.639 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43757.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43757.log.gz 2023-11-26T07:24:38.677 INFO:teuthology.orchestra.run.smithi031.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-26T07:24:38.681 INFO:teuthology.orchestra.run.smithi031.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-26T07:24:38.688 INFO:teuthology.orchestra.run.smithi031.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-26T07:24:38.728 INFO:teuthology.orchestra.run.smithi031.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-26T07:24:38.827 INFO:teuthology.orchestra.run.smithi031.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-26T07:24:38.829 INFO:teuthology.orchestra.run.smithi031.stderr: 2023-11-26T07:24:38.829 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m1.049s 2023-11-26T07:24:38.829 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m4.269s 2023-11-26T07:24:38.829 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m1.430s 2023-11-26T07:24:38.830 INFO:tasks.ceph:Archiving logs... 2023-11-26T07:24:38.830 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348/remote/smithi031/log 2023-11-26T07:24:38.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-26T07:24:39.509 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-11-26T07:24:39.510 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-11-26T07:24:39.648 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348/remote/smithi097/log 2023-11-26T07:24:39.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-26T07:24:39.938 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-26T07:24:39.947 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-26T07:24:39.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-26T07:24:39.951 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-26T07:24:40.040 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-26T07:24:40.049 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-26T07:24:40.050 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:24:40.054 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-26T07:24:40.066 INFO:teuthology.orchestra.run.smithi031.stdout: remote refid st t when poll reach delay offset jitter 2023-11-26T07:24:40.067 INFO:teuthology.orchestra.run.smithi031.stdout:============================================================================== 2023-11-26T07:24:40.067 INFO:teuthology.orchestra.run.smithi031.stdout:*hv01.front.sepi 44.4.53.2 2 u 4 64 377 0.092 -3.432 1.091 2023-11-26T07:24:40.067 INFO:teuthology.orchestra.run.smithi031.stdout:+hv02.front.sepi 63.231.80.2 3 u 7 64 377 0.077 -1.068 1.084 2023-11-26T07:24:40.067 INFO:teuthology.orchestra.run.smithi031.stdout:+hv03.front.sepi 74.6.168.72 3 u 2 64 377 0.079 -2.129 1.343 2023-11-26T07:24:40.067 INFO:teuthology.orchestra.run.smithi031.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2023-11-26T07:24:40.070 INFO:teuthology.orchestra.run.smithi097.stdout: remote refid st t when poll reach delay offset jitter 2023-11-26T07:24:40.070 INFO:teuthology.orchestra.run.smithi097.stdout:============================================================================== 2023-11-26T07:24:40.070 INFO:teuthology.orchestra.run.smithi097.stdout:*hv01.front.sepi 44.4.53.2 2 u 11 64 377 0.090 -0.960 1.684 2023-11-26T07:24:40.070 INFO:teuthology.orchestra.run.smithi097.stdout:+hv02.front.sepi 63.231.80.2 3 u 7 64 377 0.086 +0.430 1.147 2023-11-26T07:24:40.071 INFO:teuthology.orchestra.run.smithi097.stdout:+hv03.front.sepi 74.6.168.72 3 u 7 64 377 0.087 -0.339 1.172 2023-11-26T07:24:40.071 INFO:teuthology.orchestra.run.smithi097.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2023-11-26T07:24:40.072 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-26T07:24:40.080 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-26T07:24:40.081 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-26T07:24:40.089 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-26T07:24:40.100 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-26T07:24:40.108 INFO:teuthology.task.internal:Duration was 1197.868521 seconds 2023-11-26T07:24:40.108 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-26T07:24:40.117 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-26T07:24:40.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-26T07:24:40.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-26T07:24:40.258 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-26T07:24:40.258 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2023-11-26T07:24:40.258 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-26T07:24:40.281 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2023-11-26T07:24:40.281 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-26T07:24:40.304 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-26T07:24:40.304 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-26T07:24:40.328 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-26T07:24:40.386 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-26T07:24:40.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-26T07:24:40.412 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-26T07:24:40.496 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-26T07:24:40.506 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-26T07:24:40.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-26T07:24:40.528 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-26T07:24:40.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-26T07:24:40.558 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:24:40.572 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:24:40.587 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2023-11-26T07:24:40.609 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2023-11-26T07:24:40.629 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:24:40.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:24:40.656 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-26T07:24:40.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-26T07:24:40.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-26T07:24:40.688 INFO:teuthology.task.internal:Transferring archived files... 2023-11-26T07:24:40.689 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348/remote/smithi031 2023-11-26T07:24:40.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-26T07:24:40.722 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348/remote/smithi097 2023-11-26T07:24:40.722 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-26T07:24:40.746 INFO:teuthology.task.internal:Removing archive directory... 2023-11-26T07:24:40.746 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-26T07:24:40.764 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-26T07:24:40.800 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-26T07:24:40.810 INFO:teuthology.task.internal:Not uploading archives. 2023-11-26T07:24:40.810 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-26T07:24:40.820 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-26T07:24:40.820 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-26T07:24:40.824 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-26T07:24:40.829 INFO:teuthology.orchestra.run.smithi031.stdout: 655464 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 26 07:24 /home/ubuntu/cephtest 2023-11-26T07:24:40.851 INFO:teuthology.orchestra.run.smithi097.stdout: 658041 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 26 07:24 /home/ubuntu/cephtest 2023-11-26T07:24:40.853 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-26T07:24:40.863 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-26T07:24:40.882 INFO:teuthology.nuke:Checking targets against current locks 2023-11-26T07:24:40.909 DEBUG:teuthology.nuke:shortname: smithi031 2023-11-26T07:24:40.909 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-26T07:24:40.934 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:36.447848', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFdUem5b4uJ1RP0qDFIJWqlOYtyO8Uys9f1w0JQufAK1nRJT1UvZ7YTajtMvazOYiXkxfaefbDWQOCeDXdf4mw='} 2023-11-26T07:24:40.935 DEBUG:teuthology.nuke:shortname: smithi097 2023-11-26T07:24:40.935 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-26T07:24:40.957 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-26_06:51:12-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7467348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-26 06:52:36.446314', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRZs30f+Ls5Q5Etnx4X6xuZ6NMQ3Xg3Xl2Q7odrw+rGhijjK+x3/VA3x/9HKNd3t3oMpcyH0ELdOzKCFkCQ6XI='} 2023-11-26T07:24:40.986 INFO:teuthology.orchestra.console.smithi031:Power off 2023-11-26T07:24:40.987 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-26T07:24:41.010 INFO:teuthology.orchestra.console.smithi097:Power off 2023-11-26T07:24:41.011 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-26T07:24:41.083 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2023-11-26T07:24:41.084 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:24:41.093 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2023-11-26T07:24:41.093 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:24:41.107 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2023-11-26T07:24:41.119 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-11-26T07:24:45.108 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:24:45.120 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:24:45.219 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2023-11-26T07:24:45.230 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-11-26T07:24:49.222 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:24:49.230 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:24:49.333 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2023-11-26T07:24:49.340 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-11-26T07:24:49.434 INFO:teuthology.orchestra.console.smithi097:Power off completed 2023-11-26T07:24:53.344 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-26T07:24:53.454 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-11-26T07:24:53.555 INFO:teuthology.orchestra.console.smithi031:Power off completed 2023-11-26T07:24:53.660 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 client.1-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} duration: 1197.86852145195 failure_reason: 'Test failure: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=fe001eef6bde4faba3c9651212b11a88 status: fail success: false 2023-11-26T07:24:53.660 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-26T07:24:53.784 INFO:teuthology.run:FAIL